Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

# Performance of compute residuals

See original GitHub issue

@ybagdasa wrote

@CamDavidsonPilon I’m trying to use compute_residuals on a dataframe with 5M observations and after tens of minutes it is unclear whether it will ever finish computing. I suspect the dataframe is probably too large to do the computation as is in a reasonable amount of time. I’d like to avoid significantly scaling down as events constitute a small fraction of the observations and I need the statistics. Is there an existing solution for this?

@ybagdasa, to confirm, you were computing the `schoenfeld` residuals?

### Issue Analytics

• State:
• Created 3 years ago

1reaction
ybagdasacommented, Jul 8, 2020

@CamDavidsonPilon Yes that’s fitting.

I ended up doing a little workaround where I divied up the data into 50 k samples and then ran those in parallel and combined the coefficients and covariance matrix afterwards using some normal approximation assumptions. Took about 6 hours to run. Not the most ideal, but it worked.

1reaction
ybagdasacommented, Jun 26, 2020

like 6 covariates to fit

#### Top Results From Across the Web

How to Find Residuals in Regression Analysis | Built In
Calculating the residual provides a valuable clue into how well your model fits the data set. To calculate residuals we need to find...
Residual Calculator | Analyse Linear Regression
The residual calculator helps you to calculate the residuals of a linear regression analysis.
Interpreting Residual Plots to Improve Your Regression
When you run a regression, Stats iQ automatically calculates and plots residuals to help you understand and improve your regression model.
Residual analysis - statistics - Encyclopedia Britannica
The analysis of residuals plays an important role in validating the regression ... These residuals, computed from the available data, are treated as...
AP Stats Unit 2 Notes: Residuals Study Guide - Fiveable
Calculating Residuals. In order to calculate a residual for a given data point, we need the LSRL for that data set and the...

#### Troubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free