High cpu usage by kernel while using inv and solve from linalg

Hi guys,

So i run into a big trouble while using linalg.solver or linalg.inv. All my 8 cpus are running at 100% where most part is due to the kernel.

First the program:

import numpy as np
import time

np.random.seed(0)
n = 56
A = np.asarray(np.random(n,n))
b = np.eye(A.shape[0])
while True:
               c = np.linalg.solve(A, b)
               time.sleep(1e-2)

Here is what shows htop:

I am running under ubuntu 16.04.1. Kernel version: 4.4.0-38-generic Python version: 2.7.12 Numpy version: 1.11.0

Testing it on ubuntu 14.04 with older version of python (2.7.6) and numpy (1.8) seems to work fine.

Any help would be appreciated 😃

[njsmith]

To expand a bit:

• the difference between you and your colleague is that your colleague doesn’t have openblas installed
• openblas is generally much faster than the alternatives for large matrices; it’s faster on a single thread, and then on top of that it can use multiple threads (but you can disable the multiple threads if you want)
• for small matrices it has pointless overhead, sigh, and forcing it to use 1 thread instead will reduce this overhead.
• this may or may not matter, depending on whether your actual code really does involve running a tight loop of nothing but small matrix linear algebra operations. Small matrix operations are fast to start with, so if they go 2x slower while your large matrix operations go 10x faster, then this might be a win overall. Or not. It really depends on your machine and exactly what your code is doing.
• …there have occasionally been bugs where openblas on small-matrix operations has been more like 100x too slow than like 2x too slow. If you encounter that then you should let us know so we can take it up with the openblas folks. (Specifically: if you can find a reproducible example where setting OPENBLAS_NUM_THREADS=1 makes your code much much much faster then we can probably get them to fix that.)

Hope that helps. Don’t think there’s anything else here for numpy to do right now, so closing, but feel free to re-open if I’m wrong.

[juliantaylor]

you are using openblas, it is expected that that will use all cpu cores and is super wasteful on small matrices. You should be able to control it with the OPENBLAS_NUM_THREADS environment variable.