question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

`test_math` is flaky

See original GitHub issue

This is part of https://github.com/chainer/chainer/issues/6903 effort.

  • Failure example: https://ci.preferred.jp/chainer.py37/2259/
  • Failure test: chainerx_tests/unit_tests/routines_tests/test_math.py
  • Failure test case: test_Tan_param_17_{contiguous=None, in_dtypes=('float16',), out_dtype='float16', input=2, shape=(2, 3)}[native:0]
  • Tolerance:
E           Not equal to tolerance rtol=0.001, atol=0.001
E           
E           Mismatch: 100%
E           Max absolute difference: 0.00203618
E           Max relative difference: 0.00200944

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:6 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
imoscommented, Apr 15, 2019

It seems the test is still flaky in the master branch.

  • Failure example: https://ci.preferred.jp/chainer.py37/2762/
  • Failure test: chainerx_tests/unit_tests/routines_tests/test_math.py
  • Failure test case: test_LogSumExp_param_1_{keepdims=False}_param_1_{shape=(2,), axis=-1}_param_0_{in_dtypes=('float16',), out_dtype='float16'}[cuda:0]
  • Tolerance:
E       Not equal to tolerance rtol=0.001, atol=0.001
E       
E       Mismatch: 100%
E       Max absolute difference: 0.00116
E       Max relative difference: 0.011566
0reactions
asi1024commented, Apr 22, 2019
Read more comments on GitHub >

github_iconTop Results From Across the Web

What is a flaky test? Definition from WhatIs.com. - TechTarget
A flaky test is an analysis of web application code that fails to produce the same result each time the same analysis is...
Read more >
[go] runtime: convert flaky semaphore linearity test into benchmark
D src/internal/testmath/ttest.go. M src/runtime/sema.go. M src/runtime/sema_test.go 6 files changed, 79 insertions(+), 354 deletions(-)
Read more >
Flaky Tests: Getting Rid Of A Living Nightmare In Testing
A flaky test is one that fails to produce the same result each time the same analysis is run. The build will fail...
Read more >
test: locklinear.go failures with "lockmany: too slow" #32986
What I understand this issue to be about is a flaky test and ... internal/testmath: add two-sample Welch's t-test for performance tests
Read more >
How to Fix Flaky Tests - Semaphore CI
Flaky tests hinder development, slow down progress, hide design problems, and can cost a lot of money in the long run.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found