You may have seen my previous post on Parallel IPython with Univa Grid Engine (SGE). It involved a fair bit of step-by-step configuration and large chunks of boilerplate code. Well, that has now been simplified. It is now much easier to run Python functions across a hundred CPU cores on…Read More
HPCC Multi-processor Parallel Environments
The Wharton High-Performance Computing Cluster (HPCC) system is a powerful environment for running research code – code that may require a long run-time, a lot of memory, or numerous iterations. By default, research code on HPCC will run as a job with access to a single CPU core. However, by…Read More
Minimal Linux with Packer and Vagrant
As an IT professional, it’s important to take the time to sharpen your tools. And as a systems programmer and project leader in research computing, one of my most valuable tools is a lean, mean operating system install. Prior to the deployment of Wharton’s High Performance Computing Cluster (HPCC), the…Read More
Running LDA modeling on Wharton’s HPCC
I have been assisting a faculty member within the Marketing Department with an ongoing research project involving U.S. senatorial elections over the past 36 years. For each election a set of media articles were gathered along with some meta data (title, publication, etc.) making up a corpus. Each corpus contains, on average,…Read More
Parallel IPython with Univa Grid Engine (SGE)
I frequently sing the praises of Python as the go-to language for research programming. Python is extremely fast, in terms of human development time. There are typically more CPU cores available versus research assistants with specific programming skills. On that note, Python is also extremely fast, in terms of compute…Read More
Running Research Code in the Cloud
Part of the recent Wharton High-Performance Computing Cluster (HPCC) upgrade was augmentation with Amazon EC2 cloud resources. Wharton Research Computing recently piloted the deployment of a cloud-bursting solution with Univa UniCloud. After much help from friendly initial users, we have worked out most of the kinks. Now, we have a…Read More
How to capture MTurk workers’ IDs into your Qualtrics survey
When posting a Qualtrics survey link on MTurk, here’s how to capture a worker ID as embedded data: First, create a HIT (Human Intelligence Task) normally, selecting the Survey Link template type. Continue setting up your hit normally until you arrive at the Design Layout tab, then click the Source button and replace…Read More
HPCC: Taking WC from Local to Cloud Computing
These are exciting times here at Wharton Research Computing! Last semester, we announced the planned upgrade from the now-retired Wharton Grid to the new Wharton High Performance Computing Cluster (HPCC). Almost 700 git commits and 14,000 lines of code later, we are in full swing – with two new chassis loaded…Read More
Qualtrics Loop & Merge Tips
The Qualtrics Loop & Merge utility is a powerful tool and offers great flexibility when dealing with dynamic content in a survey. I want to show off some use cases ranging from very simple to advanced, which you can adapt for use in research. Simple looping Let’s say you want to implement a…Read More
Towards Research IT Support as a Service
Our team’s overarching strategic goal is to create a set of centralized and scaleable research-related IT services in order to attract and retain top quality researchers. Sounds simple, right? However at Wharton (and perhaps Penn), it turns out that teaching is much more efficiently and centrally served than research is (at least…Read More