Gurobi

What’s Gurobi?

Gurobi’s website says they are: the state-of-the-art solver for linear programming (LP), quadratic and quadratically constrained programming (QP and QCP), and mixed-integer programming (MILP, MIQP, and MIQCP)

Usage in the Wharton HPC3

The Problem

By default, we set up the cluster so that all jobs run on one core only, and Gurobi doesn’t ‘honor’ all of the normal ways of telling software about that (primarily OMP_NUM_THREADS environment variable). By default, for some solvers Gurobi will try to use all cores on the box, so will start 16 threads … which will all run on one core, performing MUCH MUCH worse than it could be.

The Solution

Fortunately there’s a simple solution. For Gurobi you need to specify ‘Threads=1’ (in gurobi_cli), OR create a custom gurobi.env (in the directory from which you are running the job) with ‘Threads 1’. You can get that with:

qrsh cp /usr/local/gurobi-11.0.2/linux64/bin/gurobi.env .

That file must be in the directory where you run the job from.

So even better would be using OMP_NUM_THREADS environment variable, set automatically in every job, to set ‘Threads=’ in your job script, something like:

#$ -N gurobi
#$ -j y
gurobi_cl Threads=$OMP_NUM_THREADS /usr/local/gurobi-11.0.2/linux64/examples/data/coins.lp

OMP_NUM_THREADS is set by default to 1, so that will ‘do the right thing’. You can use more than one with the above job script by adding
the ‘-pe openmp #’ option to your job submission command:

qsub -pe openmp 8 jobscript.sh

That job will be allowed to use multi-threading (8 cores), because you ‘requested’ them in your qsub command.

Max Threads (cores)

INTERACTIVE MODE (qrsh/qlogin): You will not be able to use more than 2 threads in interactive mode. This is due to the ‘batch’ nature of our queuing system, and our notion of ‘fair usage’.

BATCH MODE (qsub): You can use up to 16 in a script (if a system happens to have 16 available, which is rare), or 2 in interactive, for testing. Our boxes all currently have 16 cores … I recommend not generally using more than 8, as it’s unusual in our environment to have that many cores available on a single host. That said, sometimes usage is low, and you would have no problem with more! Feel free to experiment!

Activating Gurobi for Python and R

To activate Gurobi library paths in your HPC3 account for Python and R, on a compute server load the gurobi module:

module load gurobi/11.0.2

You can add the above command to the MODULE SELECTION section of your ~/.bashrc file, to permanently enable Gurobi, or add to a job script above your call to R / Rscript commands.

NOTE: the Gurobi version is occasionally upgraded, so the above path could change, as well.

MATLAB Ingegration

To use Gurobi optimization in your MATLAB code, add the following line in your MATLAB code, above any Gurobi calls:

addpath /usr/local/gurobi-11.0.2/linux64/matlab
params.threads = 1

After that, Gurobi functions will be available:

help gurobi

Python Use

Gurobi is easily used in Python: you just need to load the gurobi module (Activating Gurobi for Python and R above), and import gurobipy.

That will only work on a compute node, so if working interactively you must first qlogin, then you would copy/paste that at the command line, or place the line in your qsub .sh job submission script, before (above) your python call, as in this Python 3.12.3 virtual environment example:

#!/bin/bash
#$ -N grbpy
#$ -j y
module load python/3.12.3
module load gurobi/11.0.2
source ~/somevirtualenvdir/bin/activate

python my-script.py

Obviously replace somevirtualenvdir with the virtualenv dir you already set up.

R Use

Gurobi is easily used in R. On a compute server (qlogin session), after Activating Gurobi for Python and R above, install to your personal R libraries with:

R CMD INSTALL /usr/local/gurobi-11.0.2/linux64/R/gurobi_11.0-2_R_4.4.0.tar.gz

NOTE: the version is occasionally upgraded, so the above path could change, as well. Let us know if you notice that this document is out of date.

If you get an error about permissions, see Our R Page for details on Installing R Packages (set up your environment).