question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

The race condition that TableDataManager is removed during segment download because of incorrect segment count.

See original GitHub issue

Current code does the followings:

  1. tableDataManager uses segmentDataManagerMap to track the segments. tableDataManager is created and initialized when adding the first segment to the server, and shutdown if the last segment is removed.
  2. The tmp directory for temporarily storing new downloaded segments will be removed and recreated as part of tableDataManager initialization.
  3. When adding a new segment to server, the server will download the segment first and update the segmentDataManagerMap after.

This can cause two issues:

  1. If the last segment is removed from segmentDataManagerMap during new segment download time and before updating segmentDataManagerMap, the tableDataManager will be shutdown, and the new segment will become unreachable. We will have partial query results with SERVER_SEGMENT_MISSING_ERROR.
  2. If tableDataManager is initialized during the segment download in scenario 1, the downloaded tar file will be removed. The server will take it as download failure, and the segment will stay in ERROR state until fixed manually by restarting the server. We have observed this issue recently after #7961 was merged.

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
Jackie-Jiangcommented, Mar 29, 2022

After some offline discussion, one way to solve this issue is to add a DELETE message, and only remove the tableDataManager when the table is deleted

0reactions
npawarcommented, May 3, 2022

has this been resolved? can the issue be closed?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Tracking down a race condition in the Apache Trafficserver ...
In some cases, the only existing documentation is flat out wrong. ... are writing in the other's segment, we avoid race conditions on...
Read more >
How can I delete segments from a failed download · Issue #1275
I am using Shaka player 2.2.8. I tried to download a video and got an error about halfway through (due to a bad...
Read more >
Setting a Segment to Adjust Automatically
You can set your device to automatically adjust the target race time of a segment based on your performance during the segment.
Read more >
Incubator Pinot Versions - Open Source Agenda
View the latest Incubator Pinot versions.
Read more >
How we found and fixed a rare race condition in our session ...
Reports from users. On March 2, 2021, we received a report via our support team from a user who, while using GitHub.com logged...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found