pymc3 does not use all cores specified by OMP_NUM_THREADS

  openmp, pymc3, python

I have a fitting procedure written in pymc3 which utilizes dot products of matrices which are functions of random variables. On the original machine on which I wrote the program, it would run and by default use all available cores, as expected.

However, I recently started using a brand new machine, including a clean installation of python and pymc3. Unfortunately, this means many aspects of the system differ from the previous machine, so it is difficult to pin down what may be causing differences. However, I am befuddled by the fact that the new machine does not use all cores available or specified (CPU usage is capped at 100%, when it should freely float up to 800%).

My ~/.theanorc has openmp=True and OMP_NUM_THREADS=8. My .bashrc sets OMP, MKL, and GOTO threads to 8 just for good measure. Running check_blas.py in theano/misc/ shows 800% CPU usage. What could be causing the pymc3 application to fail to use all cores in a machine-dependent way?

Thank you.

Source: Python Questions

LEAVE A COMMENT