[BUG] ActionSerializerSuite has 3 failing unit tests in the master branch
See original GitHub issueBug
Describe the problem
Following unit tests fail:
- Metadata (with all defaults)
- Metadata - json serialization/deserialization
- Metadata with empty createdTime- json serialization/deserialization
Steps to reproduce
git checkout master
build/sbt "testOnly org.apache.spark.sql.delta.ActionSerializerSuite"
Metadata (with all defaults) - json serialization/deserialization *** FAILED *** (623 milliseconds)
[info] "{"metaData":{"id":"[10daabbc-d843-4fe4-abe5-0ccb83ba681e]","format":{"provide..." did not equal "{"metaData":{"id":"[testId]","format":{"provide..." (ActionSerializerSuite.scala:344)
[info] Analysis:
[info] "{"metaData":{"id":"[10daabbc-d843-4fe4-abe5-0ccb83ba681e]","format":{"provide..." -> "{"metaData":{"id":"[testId]","format":{"provide..."
============================================================
[info] Tests: succeeded 25, failed 3, canceled 0, ignored 0, pending 0
[info] *** 3 TESTS FAILED ***
[error] Failed tests:
[error] org.apache.spark.sql.delta.ActionSerializerSuite
[error] (core / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 24 s, completed Jul 21, 2022 9:56:50 AM
➜ delta git:(master)$ build/sbt "testOnly org.apache.spark.sql.delta.ActionSerializerSuite"
Observed results
Expected results
Further details
Environment information
- Delta Lake version:
- Spark version:
- Scala version:
Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?
- Yes. I can contribute a fix for this bug independently.
- Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
- No. I cannot contribute a bug fix at this time.
Issue Analytics
- State:
- Created a year ago
- Comments:9 (5 by maintainers)
Top Results From Across the Web
[BUG] ActionSerializerSuite has 3 failing unit tests in ... - GitHub
[BUG] ActionSerializerSuite has 3 failing unit tests in the master branch. Add new and updated issues to Needs Review #1325 #1325. Sign in...
Read more >Where to push a failing test?
The repository's master branch is not it's master branch. ... rule that there needs to be failing tests, that remind of outstanding bugs, ......
Read more >Why don't Unit Test failures show up in short term branches?
I've confirmed that my Go report is being ingested properly because all of the Tests measures show up in my Main branch and...
Read more >How to Fix Flaky Tests - Semaphore CI
Randomly failing tests are the hardest to debug. Here's a framework you can use to fix them and keep your test suite healthy....
Read more >Debugging Failing Tests and Test Pipelines - GitLab
Fix tests failing in master before other development work: Failing tests on master ... Failure due to bug: If a test failure is...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
The
id
for metadata is build takingUtils.isTesting
into account. Check this: https://github.com/delta-io/delta/blob/2871e550eabd9e4956aca1b02baeb20635a66931/core/src/main/scala/org/apache/spark/sql/delta/actions/actions.scala#L439The
isTesting
is basically this:System.getenv("SPARK_TESTING") != null || System.getProperty(IS_TESTING.key) != null
With no extra environmental variables, the 3 tests are failing exactly as you described. If I execute
export SPARK_TESTING=true
and then I run the tests, they all pass.@ganeshchand should we close this?
@scottsand-db could you review all of Metadata creation and make sure they are created in
test
body rather than outside?