Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Investigate best method for zipping/unzipping instance backups

See original GitHub issue

What could be surprising, only instances unzipped by zip4j (net.lingala) are working properly. However, when backup zip are more heavier > 10 GB , noticed exception when unzipping instance zipped by ZipFile(file).apply { addFolder() } (when using zip4j for both zipping and unzipping):

        at net.lingala.zip4j.core.ZipFile.extractFile(ZipFile.java:500)
        at com.cognifide.gradle.aem.common.file.FileOperations.zipUnpackDir(FileOperations.kt:144)
        at com.cognifide.gradle.aem.common.instance.LocalInstanceManager$createFromBackup$1$1$1.invoke(LocalInstanceManager.kt:131)
        at com.cognifide.gradle.aem.common.instance.LocalInstanceManager$createFromBackup$1$1$1.invoke(LocalInstanceManager.kt:15)
        at com.cognifide.gradle.aem.common.build.ProgressIndicator.increment(ProgressIndicator.kt:82)
        at com.cognifide.gradle.aem.common.instance.LocalInstanceManager$createFromBackup$1$1.invoke(LocalInstanceManager.kt:130)
        at com.cognifide.gradle.aem.common.instance.LocalInstanceManager$createFromBackup$1$1.invoke(LocalInstanceManager.kt:15)
        at com.cognifide.gradle.aem.common.utils.UtilsKt.onEachApply(Utils.kt:23)
        at com.cognifide.gradle.aem.common.instance.LocalInstanceManager$createFromBackup$1.invoke(LocalInstanceManager.kt:129)
        at com.cognifide.gradle.aem.common.instance.LocalInstanceManager$createFromBackup$1.invoke(LocalInstanceManager.kt:15)
        at com.cognifide.gradle.aem.common.build.ProgressIndicator$launchAsync$1$blockJob$1.invokeSuspend(ProgressIndicator.kt:45)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:238)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742)
Caused by: net.lingala.zip4j.exception.ZipException: net.lingala.zip4j.exception.ZipException: invalid local header signature for file: author/crx-quickstart/repository/segmentstore/data00018b.tar
        at net.lingala.zip4j.unzip.UnzipEngine.unzipFile(UnzipEngine.java:98)
        at net.lingala.zip4j.unzip.Unzip.initExtractFile(Unzip.java:160)
        ... 20 more
Caused by: net.lingala.zip4j.exception.ZipException: invalid local header signature for file: author/crx-quickstart/repository/segmentstore/data00018b.tar
        at net.lingala.zip4j.core.HeaderReader.readLocalFileHeader(HeaderReader.java:858)
        at net.lingala.zip4j.unzip.UnzipEngine.checkLocalHeader(UnzipEngine.java:333)
        at net.lingala.zip4j.unzip.UnzipEngine.getInputStream(UnzipEngine.java:115)
        at net.lingala.zip4j.unzip.UnzipEngine.unzipFile(UnzipEngine.java:78)
        ... 21 more

when changing zipping method to e.g ziputil (zeroturnaround) or Gradle ZIP task then unzipping by zip4j works properly, but we have 2 different libraries here… which looks generally bad and confusing but is… only one working combination / most stable.

TODO find approach to use same zipping library / approach in Gradle for both creating backups and restoring from backups with including:

working instances after unzipping (all bundles active); known that instances are not working when unpacked by other lib than zip4j
supporting large backups > 10 GB; known that zip4j is failing in that case when it is used for zipping

I am proposing to start investigating stack trace above and try to still use ZipFile(file).apply { addFolder() } (and manipulate addFolder args or ZipParameters) , because it allows to interactively update progress bar just after completing zipping one instance.