question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

lmplot() uses all server CPUs - any method to limit cores?

See original GitHub issue

When executing sns.lmplot() on a DataFrame with 500k rows and with an x-axis aggregation function (np.mean()) seaborn uses all available CPU cores on a 4-CPU server machine (with XXX cores available). We run python in Docker containers without any CPU limits imposed by the orchestrating platform (Openshift). This used to work, because data munging and plotting algos used to be single-threaded, and statistical modeling algos always give us the ability to limit the number of threads (to optimize performance at around 8-32 cores, depending on the algo). We need such a setting in seaborn as well (probably needs to be passed to the underying statistical functions).

The exact code to replicate (on a multi-core machine and using a DataFrame filled with two columns and 500k rows of random numbers):

import seaborn as sns
sns.lmplot(x='IntVar1", y='FloatVar1', data=df, x_estimator=np.mean, x_bins=10)

Python 3.7. Package versions:

  • seaborn: 0.10.0
  • matplotlib: 3.1.2
  • numpy: 1.18.1
  • pandas: 1.0.1

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
mirekphdcommented, Feb 7, 2020

I have found a hack (see #347) to speed things up from minutes to seconds by switching off lmplot’s computationally intensive bootstrapped confidence intervals (ci=None), but this is just masking the core issue of lmplot using all CPU threads without any method to limit such heavy usage.

0reactions
mwaskomcommented, Aug 5, 2021

No idea what adfuller is, @GF-Huang; pretty sure you’re asking in the wrong channel.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Limiting number of CPU cores used by Python on Unix server
I am running a Python script to process a large amount of data on a shared Unix computing cluster, with up to 64...
Read more >
How to limit number of CPU's in Red Hat Enterprise Linux on a ...
What is the procedure to disable/enable processors (on the fly) on systems with ... How to restrict server to "N" number of CPU's...
Read more >
seaborn.lmplot — seaborn 0.12.1 documentation - PyData |
It is intended as a convenient interface to fit regression models across conditional subsets of a dataset. When thinking about how to assign...
Read more >
CPU Settings for an Application Pool <cpu> | Microsoft Learn
Hold down the Windows key, press the letter X, and then click Control Panel. Click Administrative Tools, and then double-click Internet ...
Read more >
Limit number of cores used per instance - Licensing
I have a server which is using total of 12 cores running an instance ... That way we know SQL is not using...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found