question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] ActionSerializerSuite has 3 failing unit tests in the master branch

See original GitHub issue

Bug

Describe the problem

Following unit tests fail:

  • Metadata (with all defaults)
  • Metadata - json serialization/deserialization
  • Metadata with empty createdTime- json serialization/deserialization

Steps to reproduce

git checkout master
build/sbt "testOnly org.apache.spark.sql.delta.ActionSerializerSuite"
 Metadata (with all defaults) - json serialization/deserialization *** FAILED *** (623 milliseconds)
[info]   "{"metaData":{"id":"[10daabbc-d843-4fe4-abe5-0ccb83ba681e]","format":{"provide..." did not equal "{"metaData":{"id":"[testId]","format":{"provide..." (ActionSerializerSuite.scala:344)
[info]   Analysis:
[info]   "{"metaData":{"id":"[10daabbc-d843-4fe4-abe5-0ccb83ba681e]","format":{"provide..." -> "{"metaData":{"id":"[testId]","format":{"provide..."

============================================================
[info] Tests: succeeded 25, failed 3, canceled 0, ignored 0, pending 0
[info] *** 3 TESTS FAILED ***
[error] Failed tests:
[error]         org.apache.spark.sql.delta.ActionSerializerSuite
[error] (core / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 24 s, completed Jul 21, 2022 9:56:50 AM
➜  delta git:(master)$ build/sbt "testOnly org.apache.spark.sql.delta.ActionSerializerSuite"

Observed results

Expected results

Further details

Environment information

  • Delta Lake version:
  • Spark version:
  • Scala version:

Willingness to contribute

The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

  • Yes. I can contribute a fix for this bug independently.
  • Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
  • No. I cannot contribute a bug fix at this time.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:9 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
bogdanmrusucommented, Jul 26, 2022

The id for metadata is build taking Utils.isTesting into account. Check this: https://github.com/delta-io/delta/blob/2871e550eabd9e4956aca1b02baeb20635a66931/core/src/main/scala/org/apache/spark/sql/delta/actions/actions.scala#L439

The isTesting is basically this: System.getenv("SPARK_TESTING") != null || System.getProperty(IS_TESTING.key) != null

With no extra environmental variables, the 3 tests are failing exactly as you described. If I execute export SPARK_TESTING=true and then I run the tests, they all pass.

@ganeshchand should we close this?

0reactions
zsxwingcommented, Aug 4, 2022

@scottsand-db could you review all of Metadata creation and make sure they are created in test body rather than outside?

Read more comments on GitHub >

github_iconTop Results From Across the Web

[BUG] ActionSerializerSuite has 3 failing unit tests in ... - GitHub
[BUG] ActionSerializerSuite has 3 failing unit tests in the master branch. Add new and updated issues to Needs Review #1325 #1325. Sign in...
Read more >
Where to push a failing test?
The repository's master branch is not it's master branch. ... rule that there needs to be failing tests, that remind of outstanding bugs, ......
Read more >
Why don't Unit Test failures show up in short term branches?
I've confirmed that my Go report is being ingested properly because all of the Tests measures show up in my Main branch and...
Read more >
How to Fix Flaky Tests - Semaphore CI
Randomly failing tests are the hardest to debug. Here's a framework you can use to fix them and keep your test suite healthy....
Read more >
Debugging Failing Tests and Test Pipelines - GitLab
Fix tests failing in master before other development work: Failing tests on master ... Failure due to bug: If a test failure is...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found