Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] Different behavior in 0.1.8 and 0.2.1

See original GitHub issue


  • Kashgari 1.1.8 / 0.2.1

Issue Description

Different behavior in 0.1.8 and 0.2.1 In Kashgari 0.1.8 BLSTModel converge in training process and I see val_acc: 0.98 and train_acc: 0.9594 In Kashgari 0.2.1 BLSTModel is overfitting and I see val_acc ~0.5 and train_acc ~0.96 There is no difference in my code, only different versions of library.



from sklearn.model_selection import train_test_split
import pandas as pd
import nltk
from kashgari.tasks.classification import BLSTMModel

# get and process data

df1 = pd.read_csv('')
df1.fillna(' ', inplace = True)'punkt')

# split on train/test
X_train, X_test, y_train, y_test = train_test_split(df1.full_text[:3570].values, df1.textrubric[:3570].values, test_size=0.2, random_state=42)
X_train = [nltk.word_tokenize(sentence) for sentence in X_train]
X_test  = [nltk.word_tokenize(sentence) for sentence in X_test]
y_train = y_train.tolist()
y_test  = y_test.tolist()

# train model
model = BLSTMModel(), y_train, x_validate=X_test, y_validate=y_test, epochs = 10)

code in colab:


Last epoch:

Kashgari 0.1.8

Epoch 10/10 55/55 [==============================] - 90s 2s/step - loss: 0.1378 - acc: 0.9615 - val_loss: 0.0921 - val_acc: 0.9769

Kashgari 0.2.1

Epoch 10/10 44/44 [==============================] - 76s 2s/step - loss: 0.0990 - acc: 0.9751 - val_loss: 2.3739 - val_acc: 0.5323

Other Comment

In 0.2.1 all models now in different file and lr hyperparameter is given explicitly (1e-3) In 0.1.8 lr hyperparameter was omitted, I suppose that it used keras default, which is the same (1e-3)

Also in 0.1.8 you had (dense size = +1 classes on classifier) and ommited it in 0.2.1. I don’t see how this could affect training process.

I couldn’t find more differences between versions, could you help with this - why models began to overfit in new version of library?

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:17 (5 by maintainers)

github_iconTop GitHub Comments

BrikerMancommented, Mar 21, 2019

I have reproduced the problem. 0.2.1 does overfit in my dataset too. I have compared 0.1.8 with 0.2.1, we have changed BLSTMModel’s activation function from sigmoid to softmax. Please try this with 0.2.1.

hyper_parameters = {
    'activation_layer': {
        'activation': 'sigmoid'
model1 = BLSTMModel(hyper_parameters=hyper_parameters)

model2 = BLSTMModel()
BrikerMancommented, Jun 20, 2019

Really sorry to hear that, I have tested 0.2.4 and tf.keras version on two datasets, all works just fine. Maybe it is a bug in 0.1.8.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Cannot pip install torchvision==0.8.0+cu110 · Issue #2912
Steps to reproduce the behavior: Copy command from pytorch website. pip install torch==1.7.0+cu110 torchvision==0.8.0+cu110 torchaudio===0.7.
Read more >
cregg 0.4.0. Fixed a bug related to variances for clustered designs (e.g., multiple responses per respondent) that was erroneously handling clusters. The bug...
Read more >
x = 0:0.1:10... What's going on, really? - MATLAB Answers
This is in my opinion expected behavior, as 0.1 can't be represented accurately with floating point numbers. What's bugging me is the following:....
Read more >
Bug listing with status RESOLVED with resolution FIXED as at ...
Bug :2 - "How do I attach an ebuild. ... Bug:271 - "New version of media-gfx/pixieplus-kde/ (Version 0.2.1)" status:RESOLVED resolution:FIXED severity:normal ...
Read more >
Changelog — CompilerGym 0.2.5 documentation
CompilerGym v0.2.5 adds a new LLVM dataset, two new observation spaces, and includes numerous updates and bug fixes. Summary of Changes.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found