question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Standardizing the documentation for `alpha` in `Ridge` and `Lasso`

See original GitHub issue

Describe the issue linked to the documentation

I’m helping to teach a machine learning course using sklearn. The documentation for alpha in Ridge is as follows:

Regularization strength; must be a positive float. Regularization improves the conditioning of the problem and reduces the variance of the estimates. Larger values specify stronger regularization. Alpha corresponds to 1 / (2C) in other linear models such as LogisticRegression or sklearn.svm.LinearSVC. If an array is passed, penalties are assumed to be specific to the targets. Hence they must correspond in number.

The documentation for alpha in Lasso is as follows

Constant that multiplies the L1 term. Defaults to 1.0. alpha = 0 is equivalent to an ordinary least square, solved by the LinearRegression object. For numerical reasons, using alpha = 0 with the Lasso object is not advised. Given this, you should use the LinearRegression object.

Between the different documentation for alpha in these models, plus the C being the inverse regularization in the closely related LogisticRegression there has understandably been some confusion for myself and some students.

Suggest a potential alternative/fix

The two models are closely related, and I think standardizing the description of alpha to be the same in Ridge and Lasso (making appropriate L2/L1 changes respectively) would go a long way to preventing some confusion students have been encountering. The documentation for alpha in Lasso seems to be most straightforward to me so I would propose using that description in both Lasso and Ridge.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8 (7 by maintainers)

github_iconTop GitHub Comments

0reactions
Rubix982commented, Nov 7, 2020

@Dpananos please take a look at the PR. Thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

A Complete Tutorial on Ridge and Lasso Regression in ...
Here is a complete tutorial on the regularization techniques of ridge and lasso regression to prevent overfitting in prediction in python.
Read more >
Regularization Tutorial: Ridge, Lasso and Elastic Net
Learn about regularization and how it solves the bias-variance trade-off problem in linear regression. Follow our step-by-step tutorial and dive into Ridge, ...
Read more >
Ridge and Lasso Regression: L1 and L2 Regularization
Complete Guide Using Scikit-Learn · The default value of regularization parameter in Lasso regression (given by α) is 1. · With this, out...
Read more >
Linear, Lasso, and Ridge Regression with R
The first step to build a lasso model is to find the optimal lambda value using the code below. For lasso regression, the...
Read more >
Lasso centering and standarization with R - Cross Validated
So the answer to your first question is, based on the glmnet documentation: If you use default values for all parameters, standardizing the ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found