Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Delta LogStore Refactor - GCSLogStore

See original GitHub issue

Refactor GCSLogStore to Java for the new delta-storage artifact.

This also includes moving the tests from GCSLogStoreSuite to class PublicGCSLogStoreSuite inside of LogStoreSuite. See the contributing section in the project issue below.

Also, as the referenced contributing section states, leave the existing Scala file and Scala test suite alone for now.

For the project overview, requirements, and how to contribute, please see the https://github.com/delta-io/delta/issues/951.

Issue Analytics

State:
Created 2 years ago
Comments:8 (5 by maintainers)

Top GitHub Comments

1reaction

ksreecommented, Mar 23, 2022

@scottsand-db Sorry for the delay. I will raise a WIP branch soon, to get the runInNewThread implementation reviewed.

1reaction

scottsand-dbcommented, Mar 16, 2022

@ksree for now, just create a java implementation of ThreadUtils.runInNewThread as a private static method inside of GCSLogStore. if we ever need to expose this method to other logstores, we can expose it as a public util then.

Top Results From Across the Web

Delta Lake 1.0.0 ScalaDoc - io.delta.storage.GCSLogStore

The LogStore implementation for GCS, which uses gcs-connector to provide the necessary atomic and durability guarantees:.

LogStore - The Internals of Delta Lake

LogStore is an abstraction of transaction log stores (to read and write physical log files and checkpoints). LogStore is created using LogStoreProvider based...

Error when running Pytest with Delta Lake Tables

It looks like that you're using incompatible version of the Delta lake library. 0.7.0 was for Spark 3.0, but you're using another version ......

Overview - Project Nessie: Transactional Catalog for Data ...

Nessie is able to interact with Delta Lake by implementing a custom version of Delta's LogStore interface. This ensures that all filesystem changes...

delta 0.8.0 with minio S3 bucket write exception and question ...

logStore.class","org.apache.spark.sql.delta.storage.S3SingleDriverLogStore")\ ... .config("spark.jars.packages", "com.amazonaws:aws-java-sdk:1.11.234") \.