What’s CPLEX?
The CPLEX website says that: IBM ILOG CPLEX Optimizer provides flexible, high-performance mathematical programming solvers for linear programming, mixed integer programming, quadratic programming, and quadratically constrained programming problems. These include a distributed parallel algorithm for mixed integer programming to leverage multiple computers to solve difficult problems
Usage in the Wharton HPC3
The Problem
By default, we set up the cluster so that all jobs run on one core only, and so we’ve also set up CPLEX to run on one core only. So you’re not (necessarily) taking advantage of all that CPLEX can do in our environment, because CPLEX can use multiple threads for some solvers. If you try to use more than one thread with CPLEX without performing the steps below, you will experience poorer performance than without using multiple threads. So read on for configuring for multiple thread use!
The Solution
Fortunately there’s a simple solution. As mentioned, by default you will be restricted to one thread with CPLEX. To use more than one and actually help instead of hurt yor job, in your job submission script, before your cplex line, or at the command line in an interactive qlogin session before starting cplex, do:
unset ILOG_CPLEX_PARAMETER_FILE
Then in your CPLEX script or in the CPLEX shell, do:
set threads #
where # is the number of threads. Then use the option -pe openmp # to your qsub command, make it match the CPLEX number, and you should be good to go, same as Gurobi, below:
qsub -pe openmp 8 jobscript.sh
That job will be allowed to use multi-threading (8 cores), because you ‘requested’ them in your qsub command.
Max Threads (cores)
INTERACTIVE MODE (qrsh/qlogin): You will not be able to use more than 2 threads in interactive mode. This is due to the ‘batch’ nature of our queuing system, and our notion of ‘fair usage’.
BATCH MODE (qsub): You can use up to 16 in a script (if a system happens to have 16 available, which is rare), or 2 in interactive, for testing. Our boxes all currently have 16 cores … I recommend not generally using more than 8, as it’s unusual in our environment to have that many cores available on a single host. That said, sometimes usage is low, and you would have no problem with more! Feel free to experiment!