Gurobi

What’s Gurobi?

Gurobi’s website says they are: the state-of-the-art solver for linear programming (LP), quadratic and quadratically constrained programming (QP and QCP), and mixed-integer programming (MILP, MIQP, and MIQCP)

Usage in the Wharton HPCC

The Problem

By default, we set up the cluster so that all jobs run on one core only, and Gurobi doesn’t ‘honor’ all of the normal ways of telling software about that (primarily OMP_NUM_THREADS environment variable). By default, for some solvers Gurobi will try to use all cores on the box, so will start 16 threads … which will all run on one core, performing MUCH MUCH worse than it could be.

The Solution

Fortunately there’s a simple solution. For Gurobi you need to specify ‘Threads=1’ (in gurobi_cli), OR create a custom gurobi.env (in the directory from which you are running the job) with ‘Threads 1’. You can get that with:

That file must be in the directory where you run the job from.

So even better would be using OMP_NUM_THREADS environment variable, set automatically in every job, to set ‘Threads=’ in your job script, something like:

OMP_NUM_THREADS is set by default to 1, so that will ‘do the right thing’. You can use more than one with the above job script by adding
the ‘-pe openmp #’ option to your job submission command:

That job will be allowed to use multi-threading (8 cores), because you ‘requested’ them in your qsub command.

Max Threads (cores)

INTERACTIVE MODE (qrsh/qlogin): You will not be able to use more than 2 threads in interactive mode. This is due to the ‘batch’ nature of our queuing system, and our notion of ‘fair usage’.

BATCH MODE (qsub): You can use up to 16 in a script (if a system happens to have 16 available, which is rare), or 2 in interactive, for testing. Our boxes all currently have 16 cores … I recommend not generally using more than 8, as it’s unusual in our environment to have that many cores available on a single host. That said, sometimes usage is low, and you would have no problem with more! Feel free to experiment!

MATLAB Ingegration

To use Gurobi optimization in your MATLAB code, add the following line in your MATLAB code, above any Gurobi calls:

After that, Gurobi functions will be available:

Python Use

Gurobi is easily used in Python: you just need to add the correct Gurobi directory to your PYTHONPATH environment variable. Depending on your preferred Python version, that’s done like so:

Python 2.7

Python 3.6

That will only work on a compute node, so if working interactively you must first qlogin, then you would copy/paste that at the command line, or place the line in your qsub .sh job submission script, before (above) your python call, as in this Python 3.6 virtual environment example:

Obviously replace somevirtualenvdir with the virtualenv dir you already set up.

R Use

Gurobi is easily used in R. Install to your personal R libraries with:

NOTE: the version is occasionally upgraded, so the above path could change, as well. Let us know if you notice that this document is out of date.

If you get an error about permissions, see Our R Page for details on Installing R Packages (set up your environment).

Further Reading

For further details on Gurobi see Gurobi’s documentation. For an excellent Python 3 tutorial, with example code, etc, see http://www.gurobi.com/resources/seminars-and-videos/python-III-webinar