question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

pct_change calculation should have an additional optional parameter for calculation with negative numbers

See original GitHub issue

Problem description

I’m proposing addition of an additional optional parameter for pct_change calculation to help with negative numbers calculations. The new optional calculation should be calculated as s.diff()/s.shift().abs().

Example code of current and proposed behavior

s = pd.Series([-1,-2,-1, 1])
res = pd.concat((s, s.pct_change(), s.diff()/s.shift().abs()),1)
res.columns = ['series','current pct_change','proposed alternative']
res

	series	current pct_change	proposed alternative
0	 -1		NaN               NaN
1 	 -2	        1.0              -1.0
2	 -1	       -0.5               0.5
3    	  1            -2.0	          2.0

Intuition and Expected Behavior

In finance, when going from -1 to -2 one is going from less debt to more debt , hence one would expect negative -100% change not positive +100% change. Say you had -1$ debt, and now you have -2$ debt, your debt increased by -100%. In absolute terms, the value changed by 100%, but the direction is negative not positive. Similarly, when going reducing debt from -2 to -1, pandas produces -50% instead of +50%.

A more complex example is going from -1 (negative) to +1 (positive), one would think of that as a positive change, that is a person went from negative balance to positive balance. However, pandas produces negative -200% instead of +200%.

The issue is that there is no universal approach for calculating pct_change when there are negative numbers in the series. For that reason, pandas should provide an optional parameter for dealing with negative numbers.

To give you more details of the issue, why pandas pct_change calculation produces nonsensical values when negative numbers are in the series, you can take a look at this article:

(https://www.excelcampus.com/functions/percentage-change-formula-negative-numbers/)

Output of pd.show_versions()

pandas: 0.23.4

Original post updated to incorporate below comments.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Reactions:6
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

18reactions
WillAydcommented, Dec 9, 2018

The issue is that there is no universal approach for calculating pct_change when there are negative numbers in the series. For that reason, pandas should provide an optional parameter for dealing with negative numbers.

Agreed on the first point but not on the second. I don’t see a keyword providing much value here over a user using abs as desired to get the calculation they want. Closing as a result

9reactions
gpozzetticommented, Oct 8, 2019

probably a bit late to the game 😃 But something like : import numpy as np df[column].pct_change()*np.sign(df[column].shift(periods=1))

should do the job

Read more comments on GitHub >

github_iconTop Results From Across the Web

Handling negative values in pct_changes - Stack Overflow
I would like to do percentage change on the netprofit grouped by ticker and year. Ticker, Year, Net Income. AAPL, 2005, 151. AAPL,...
Read more >
Pandas DataFrame pct_change() Method - Studytonight
This article explains the Python pandas DataFrame.pct_change() method which calculates the percentage change between the current and a prior element.
Read more >
pandas.DataFrame.pct_change — pandas 1.5.2 documentation
Computes the percentage change from the immediately previous row by default. This is useful in comparing the percentage of change in a time...
Read more >
Percent Change Method Explanation in Pandas for Python
Complete explanation of the Pandas module's . pct_change () (for calculating percent change) method. Includes explanations of all parameters, ...
Read more >
Python | Pandas Series.pct_change() - GeeksforGeeks
Pandas pct_change() method is applied on series with numeric data to calculate Percentage change after n number of elements. By default ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found