Wharton’s user data space (/home and /data) are built on redundant, network attached storage, making it a reliable, safe place for you to store your data and code.
Snapshots
Our networked file systems include automated daily and weekly ‘snapshots’ of all user data space, providing our users with multiple user-accessible copies of all of their data. We snapshot:
- daily and always have the last seven (7) daily snapshots available
- weekly and always have the last five (5) weekly snapshots available
Reserve Snapshots
In addition to user-accessible snapshots we also perform daily syncing to a secondary filesystem, which is also snapshotted, but on a longer-retained, less-frequent schedule, including:
- weekly and always have the last five (5) weekly snapshots available
- monthly and always have the last seven (6) monthly snapshots available
This is for disaster recovery (if our primary network storage has problems that we cannot resolve), and for assisting you with recovering something older than our on-HPCC snapshots contain.
Recovering Files
Depending on what you need to recover, there are two basic paths to file recovery in our environment.
Self Service
Most file recovery can be accomplished by you, the user, allowing for the fastest and most selective recovery.
Our snapshot system creates .snapshot directories in each directory in user-writeable storage areas, like your personal home or project directories. Because of the ‘.’ (dot), these files are hidden when you do an ‘ls‘. Trust us: they are there. Or look for yourself.
To explore and recover files, log on to the HPCC with your ssh client, and ‘cd‘ into any .snapshot directory:
$ cd .snapshot $ ls -1 daily.2021-10-02_0010 daily.2021-10-03_0010 daily.2021-10-04_0010 daily.2021-10-05_0010 daily.2021-10-06_0010 daily.2021-10-07_0010 daily.2021-10-08_0010 weekly.2021-08-29_0015 weekly.2021-09-05_0015 weekly.2021-09-12_0015 weekly.2021-09-19_0015 weekly.2021-09-26_0015 weekly.2021-10-03_0015 $ cd 2021-10-02_0010 $ ls *.sh script1.sh script2.sh script3.sh $ cp script2.sh $HOME
That’s it! You can also be able to use an SFTP client or the MobaXterm SSH Browser to do recovery … remember the ‘.’ (dot), which will make the .snapshot directory invisible. Trust that it’s there!
 NOTE: .snapshot directories are not available from a SMB (Windows File System) share (mapped network drive). |
NOTE: Permissions in .snapshot directories are identical to those in your ‘normal’ directories. Only those with proper permissions can browse and restore your files. |
Assisted Recovery
If the Self Service method (above) isn’t adequate—generally if the files have been out of the user space for more than 6 weeks—please contact research-computing@wharton.upenn.edu with as much detail as you can provide. Path and name of files, and when the files were last in your user space are the most important details.
This can take up to 2 business days to complete. We thank you for your patience!
Other Methods
We recommend that you also use a Repository Service (Version Control) or Dropbox syncing to manage your code. Both of these methods provide the ability to recover multiple file versions, along with other valuable features.