`test_math` is flaky
See original GitHub issueThis is part of https://github.com/chainer/chainer/issues/6903 effort.
- Failure example: https://ci.preferred.jp/chainer.py37/2259/
- Failure test: chainerx_tests/unit_tests/routines_tests/test_math.py
- Failure test case:
test_Tan_param_17_{contiguous=None, in_dtypes=('float16',), out_dtype='float16', input=2, shape=(2, 3)}[native:0]
- Tolerance:
E Not equal to tolerance rtol=0.001, atol=0.001
E
E Mismatch: 100%
E Max absolute difference: 0.00203618
E Max relative difference: 0.00200944
Issue Analytics
- State:
- Created 4 years ago
- Comments:6 (6 by maintainers)
Top Results From Across the Web
What is a flaky test? Definition from WhatIs.com. - TechTarget
A flaky test is an analysis of web application code that fails to produce the same result each time the same analysis is...
Read more >[go] runtime: convert flaky semaphore linearity test into benchmark
D src/internal/testmath/ttest.go. M src/runtime/sema.go. M src/runtime/sema_test.go 6 files changed, 79 insertions(+), 354 deletions(-)
Read more >Flaky Tests: Getting Rid Of A Living Nightmare In Testing
A flaky test is one that fails to produce the same result each time the same analysis is run. The build will fail...
Read more >test: locklinear.go failures with "lockmany: too slow" #32986
What I understand this issue to be about is a flaky test and ... internal/testmath: add two-sample Welch's t-test for performance tests
Read more >How to Fix Flaky Tests - Semaphore CI
Flaky tests hinder development, slow down progress, hide design problems, and can cost a lot of money in the long run.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
It seems the test is still flaky in the master branch.
test_LogSumExp_param_1_{keepdims=False}_param_1_{shape=(2,), axis=-1}_param_0_{in_dtypes=('float16',), out_dtype='float16'}[cuda:0]
test_Abs_param_24_{shape=(), contiguous='C', out_dtype='float32', input='random', in_dtypes=('float32',)}[native:0]