question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Improvement] Deterministic value_counts

See original GitHub issue

Code Sample, a copy-pastable example if possible

import pandas as pd

df = pd.DataFrame(["a", "b", "c"], columns=["test"])

print(df["test"].value_counts())

Problem description

Using value_counts in a testsuite can be a problem, when the resulting values have the same count as they permutade on each call, e.g.:

$ python pandas_value_counts.py
a    1
b    1
c    1
Name: test, dtype: int64

$ python pandas_value_counts.py
c    1
a    1
b    1
Name: test, dtype: int64

Expected Output

Some stable/deterministic output or optionally additionally sorting of the keys, if they have the same counts

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.0.final.0 python-bits: 32 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel byteorder: little LC_ALL: None LANG: de_DE.UTF-8 LOCALE: None.None

pandas: 0.19.2 nose: 1.3.7 pip: 9.0.1 setuptools: 27.2.0 Cython: 0.25.2 numpy: 1.11.3 scipy: 0.18.1 statsmodels: 0.6.1 xarray: None IPython: 5.1.0 sphinx: 1.5.1 patsy: 0.4.1 dateutil: 2.6.0 pytz: 2016.10 blosc: None bottleneck: 1.2.0 tables: 3.2.2 numexpr: 2.6.1 matplotlib: 2.0.0 openpyxl: 2.4.1 xlrd: 1.0.0 xlwt: 1.2.0 xlsxwriter: 0.9.6 lxml: 3.7.2 bs4: 4.5.3 html5lib: None httplib2: None apiclient: None sqlalchemy: 1.1.5 pymysql: None psycopg2: None jinja2: 2.9.4 boto: 2.45.0 pandas_datareader: None

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:2
  • Comments:5 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
tomspurcommented, Apr 3, 2017

Yes, I would need this for testing and would like to have the highest count of the data. With just getting the first item of the value_counts, this is some random value, that has the maximum count and it would be great to have always the same value.

2reactions
jrebackcommented, Mar 29, 2017

see discussion in these related issues.

xref #12679 xref #11227 xref #14860

This is not guaranteed in any way, nor is performant to do so. Further why should this be anything but an arbitrary ordering? This is a mapping of value -> count.

Read more comments on GitHub >

github_iconTop Results From Across the Web

pandas value_counts: sort by value, then alphabetically?
I recommend to use this solution to make sure it runs deterministically across different versions. You can do this (if you don't really...
Read more >
How to Use Random Seeds Effectively | by Jai Bansal
This post is about an aspect of the predictive model-building process that doesn't typically get much attention: random seeds.
Read more >
What's new in 1.2.0 (December 26, 2020)
When using Series.value_counts() , the count of missing values is no longer necessarily last in the list of duplicate counts.
Read more >
API documentation for the value_counts pandas function ...
API documentation for the value_counts pandas function, detailing how to call the function with specific parameters. Displayed in a webpage.
Read more >
Offline Evaluation of Multi-Armed Bandit Algorithms in ...
movieId.value_counts())\ .loc[pd. ... a bandit would view each event as its own time step and make a large number of small improvements.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found