The short.q job queue on the Wharton High Performance Computing Cluster (HPCC) is now available for general use. What this means is that all users may now have up to 256 more jobs simultaneously running! This is a way around the current 64-core limit. To use this new feature, one may…Read More
Seamless R and C++ integration with Rcpp
Are you an R programmer? Do you like high-performance computing? Do you want to accelerate your core R functions? Do you sound like a pirate when you talk about your code? Then, this book may be for you: Seamless R and C++ integration with Rcpp Rcpp is an R add-on…Read More
Improved Support for Parallel IPython on Wharton HPCC
You may have seen my previous post on Parallel IPython with Univa Grid Engine (SGE). It involved a fair bit of step-by-step configuration and large chunks of boilerplate code. Well, that has now been simplified. It is now much easier to run Python functions across a hundred CPU cores on…Read More
HPCC Multi-processor Parallel Environments
The Wharton High-Performance Computing Cluster (HPCC) system is a powerful environment for running research code – code that may require a long run-time, a lot of memory, or numerous iterations. By default, research code on HPCC will run as a job with access to a single CPU core. However, by…Read More
Minimal Linux with Packer and Vagrant
As an IT professional, it’s important to take the time to sharpen your tools. And as a systems programmer and project leader in research computing, one of my most valuable tools is a lean, mean operating system install. Prior to the deployment of Wharton’s High Performance Computing Cluster (HPCC), the…Read More
Running LDA modeling on Wharton’s HPCC
I have been assisting a faculty member within the Marketing Department with an ongoing research project involving U.S. senatorial elections over the past 36 years. For each election a set of media articles were gathered along with some meta data (title, publication, etc.) making up a corpus. Each corpus contains, on average,…Read More
Parallel IPython with Univa Grid Engine (SGE)
I frequently sing the praises of Python as the go-to language for research programming. Python is extremely fast, in terms of human development time. There are typically more CPU cores available versus research assistants with specific programming skills. On that note, Python is also extremely fast, in terms of compute…Read More
Running Research Code in the Cloud
Part of the recent Wharton High-Performance Computing Cluster (HPCC) upgrade was augmentation with Amazon EC2 cloud resources. Wharton Research Computing recently piloted the deployment of a cloud-bursting solution with Univa UniCloud. After much help from friendly initial users, we have worked out most of the kinks. Now, we have a…Read More
How to capture MTurk workers’ IDs into your Qualtrics survey
When posting a Qualtrics survey link on MTurk, here’s how to capture a worker ID as embedded data: First, create a HIT (Human Intelligence Task) normally, selecting the Survey Link template type. Continue setting up your hit normally until you arrive at the Design Layout tab, then click the Source button and replace…Read More
HPCC: Taking WC from Local to Cloud Computing
These are exciting times here at Wharton Research Computing! Last semester, we announced the planned upgrade from the now-retired Wharton Grid to the new Wharton High Performance Computing Cluster (HPCC). Almost 700 git commits and 14,000 lines of code later, we are in full swing – with two new chassis loaded…Read More