Data Protection

Wharton’s user data space (/home and /data) are built on redundant, network attached storage, making it a reliable, safe place for you to store your data and code.

Snapshots

Our networked file systems include automated daily and weekly ‘snapshots’ of all user data space, providing our users with multiple user-accessible copies of all of their data. We snapshot:

  • daily and always have the last seven (7) daily snapshots available
  • weekly and always have the last five (5) weekly snapshots available

Reserve Snapshots

In addition to user-accessible snapshots we also perform daily syncing to a secondary filesystem, which is also snapshotted, but on a longer-retained, less-frequent schedule, including:

  • weekly and always have the last five (5) weekly snapshots available
  • monthly and always have the last seven (6) monthly snapshots available

This is for disaster recovery (if our primary network storage has problems that we cannot resolve), and for assisting you with recovering something older than our on-HPCC snapshots contain.

Recovering Files

Depending on what you need to recover, there are two basic paths to file recovery in our environment.

Self Service

Most file recovery can be accomplished by you, the user, allowing for the fastest and most selective recovery.

Our snapshot system creates .snapshot directories in each directory in user-writeable storage areas, like your personal home or project directories. Because of the ‘.’ (dot), these files are hidden when you do an ‘ls‘. Trust us: they are there. Or look for yourself.

To explore and recover files, log on to the HPCC with your ssh client, and ‘cd‘ into any .snapshot directory:

$ cd .snapshot
$  ls -1
daily.2021-10-02_0010
daily.2021-10-03_0010
daily.2021-10-04_0010
daily.2021-10-05_0010
daily.2021-10-06_0010
daily.2021-10-07_0010
daily.2021-10-08_0010
weekly.2021-08-29_0015
weekly.2021-09-05_0015
weekly.2021-09-12_0015
weekly.2021-09-19_0015
weekly.2021-09-26_0015
weekly.2021-10-03_0015
$ cd 2021-10-02_0010
$ ls *.sh
script1.sh    script2.sh   script3.sh
$ cp script2.sh $HOME

That’s it! You can also be able to use an SFTP client or the MobaXterm SSH Browser to do recovery … remember the ‘.’ (dot), which will make the .snapshot directory invisible. Trust that it’s there!

Imbox_content
NOTE: .snapshot directories are not available from a SMB (Windows File System) share (mapped network drive).

 

Imbox_content
NOTE: Permissions in .snapshot directories are identical to those in your ‘normal’ directories. Only those with proper permissions can browse and restore your files.

Assisted Recovery

If the Self Service method (above) isn’t adequate—generally if the files have been out of the user space for more than 6 weeks—please contact research-computing@wharton.upenn.edu with as much detail as you can provide. Path and name of files, and when the files were last in your user space are the most important details.

This can take up to 2 business days to complete. We thank you for your patience!

Other Methods

We recommend that you also use a Repository Service (Version Control) or Dropbox syncing to manage your code. Both of these methods provide the ability to recover multiple file versions, along with other valuable features.