Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Same observation being generatad
See original GitHub issueHi,
I tried to run the code below to optimize a XGBoost classifier, but get stuck with same observation being tested all time. I expected some new observation being generated… or am I wrong?
Console output (after initial points generated). Notice that all iterations generates the same observation:
('XGB', {'num_round': 20.0, 'subsample': 0.25, 'eta': 0.01, 'colsample_bytree': 0.25, 'max_depth': 2.0})
Iteration: 1 | Last sampled value: -0.680226 | with parameters: {'num_round': 20.0, 'subsample': 0.25, 'eta': 0.01, 'colsample_bytree': 0.25, 'max_depth': 2.0}
| Current maximum: -0.245901 | with parameters: {'num_round': 28.712248896201515, 'subsample': 0.88492808306639748, 'eta': 0.78136949498158781, 'colsample_bytree': 0.99625386365127699, 'max_depth': 5.3806033554623252}
| Time taken: 0 minutes and 10.953415 seconds
('XGB', {'num_round': 20.0, 'subsample': 0.25, 'eta': 0.01, 'colsample_bytree': 0.25, 'max_depth': 2.0})
Iteration: 2 | Last sampled value: -0.680226 | with parameters: {'num_round': 20.0, 'subsample': 0.25, 'eta': 0.01, 'colsample_bytree': 0.25, 'max_depth': 2.0}
| Current maximum: -0.245901 | with parameters: {'num_round': 28.712248896201515, 'subsample': 0.88492808306639748, 'eta': 0.78136949498158781, 'colsample_bytree': 0.99625386365127699, 'max_depth': 5.3806033554623252}
| Time taken: 0 minutes and 10.790525 seconds
('XGB', {'num_round': 20.0, 'subsample': 0.25, 'eta': 0.01, 'colsample_bytree': 0.25, 'max_depth': 2.0})
Iteration: 3 | Last sampled value: -0.680226 | with parameters: {'num_round': 20.0, 'subsample': 0.25, 'eta': 0.01, 'colsample_bytree': 0.25, 'max_depth': 2.0}
| Current maximum: -0.245901 | with parameters: {'num_round': 28.712248896201515, 'subsample': 0.88492808306639748, 'eta': 0.78136949498158781, 'colsample_bytree': 0.99625386365127699, 'max_depth': 5.3806033554623252}
| Time taken: 0 minutes and 10.6884 seconds
Full code for the program (uses the xgboost library)
import xgboost as xgb
from bayes_opt import BayesianOptimization
from sklearn.datasets import make_classification
X, y = make_classification(n_samples=2500, n_features=45, n_informative=12, n_redundant=7, n_classes=2, random_state=42)
def xgbcv(max_depth, eta, colsample_bytree, subsample, num_round):
print("XGB", locals())
dtrain = xgb.DMatrix(X, label=y)
params = {
'booster': 'gbtree',
'objective': 'multi:softprob',
'silent': 1,
'max_depth': int(round(max_depth)),
'eta': eta,
'colsample_bytree': colsample_bytree,
'subsample': subsample,
'num_class': 2,
'eval_metric': 'mlogloss',
'seed': 42
}
r = xgb.cv(params, dtrain, int(round(num_round)), nfold=4, metrics={'mlogloss'}, seed=45, show_stdv=False)
return -r['test-mlogloss-mean'].mean()
xgbBO = BayesianOptimization(xgbcv, {
'max_depth': (2, 6),
'eta': (0.01, 0.8),
'colsample_bytree': (0.25, 1.0),
'subsample': (0.25, 1.0),
'num_round': (20, 30),
}, verbose=True)
xgbBO.maximize(init_points=32, n_iter=6)
Thanks in advice!
Issue Analytics
- State:
- Created 8 years ago
- Comments:10 (5 by maintainers)
Top Results From Across the Web
Hidden Markov Models
The forward algorithm computes the observation probability by summing over the prob- abilities of all possible hidden state paths that could generate the ......
Read more >Hidden Markov Model. Elaborated with examples
Hidden Markov models are probabilistic frameworks where the observed data are modeled as a series of outputs generated by one of several (hidden) ......
Read more >Introduction to Hidden Markov Models
To define Markov model, the following probabilities have to be specified: transition probabilities ... What HMM is more likely to generate this observation....
Read more >Exploring Hidden Markov Models - Nipun Batra
In an HMM, an observation is generated from a hidden component, which is modeled as a Markov chain. The observation at time t...
Read more >Hidden Markov Models - an overview | ScienceDirect Topics
In HMM, each observation is generated by some states and observations are independent of each other.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I’m also seeing this “edge obsession” with on e of my params (alpha, seed_thresh, and mask_thresh).
The random / given initialization points give a good sample of the space, but once I get to the maximization portion of the code, the algorithm always chooses alpha=0 or alpha=1 and seed_thersh/mask_thresh =.4 or .9.
I’m using UCB with kappas of 10, 5, and 1. My scores are only positive and are fairly well behaved.
Code looks like this:
I am running into a similar issue: while during the Initialization phase the parameter space is sampled nicely, in the Optimization phase, for most parameters, only extreme values are tried. I’m optimizing an R2 score which can be negative. I tried optimizing 10 + R2 instead, because I read that that may a be a problem. While alleviated, the obsession for edges of the parameters space is still present. Why does the presence of negative values matter for one, and any suggestion on how to fix this?