Don't transform the object passed to extraDirectories.paths.path.from immediately

Environment:

• Jib version: 2.6.0
• *Build tool:*Gradle 6.7.1
• OS: macOS

Description of the issue:

I’d like to configure extraDirectories with the output from another task, however during the gradle configuration the task didn’t run yet, the files dod not exist yet

jib {
    extraDirectories {
        paths {
            path {
                from = {
                    unpackLibsodium.outputs
                            .files
                            .asFileTree
                            .matching { include "**/*.so" }
                            .singleFile
                            .parentFile
                }
// ...
}

tasks.jibBuildTar.dependsOn unpackLibsodium
tasks.jibDockerBuild.dependsOn unpackLibsodium
tasks.jib.dependsOn unpackLibsodium

This leads to the following error

* What went wrong:
A problem occurred evaluating project ':password-encryption-cli'.
> Expected task 'unpackLibsodium' output files to contain exactly one file, however, it contains no files.

And it is likely caused by the setFrom method that immediately pass the object to project.file, thus immediately executing the closure, however as the task has been executed it fails because there’s no output yet.

https://github.com/GoogleContainerTools/jib/blob/8d34a28a156894867c6fdd8416a9f03cdf04d863/jib-gradle-plugin/src/main/java/com/google/cloud/tools/jib/gradle/ExtraDirectoryParameters.java#L57

The only option is to use a string constant.

Expected behavior:

It would be nice if jib called the closure at execution time. In other word moving the project.files(from).toPath() outside the setter, at the place it is really needed. Maybe in the getFrom() method ?

[chanseokoh]

Just in case if you are blocked, at least I think there’s an easy workaround: set extraDirectories.paths to a temporary directory (e.g., extraDirectories.paths = ['build/my-libs-dir']), set up a Copy task to copy .so files into my-libs-dir, and have Jib depend on the Copy task. A bit of unnecessary copying here, but I think it will work. However, I’m not versed in Gradle at all, and I’m not sure if there’s a better workaround or even this makes sense.

And we still want to go deeper into this issue to figure out what can be done generally, so it’ll be helpful for us to have a reproducible case.

[bric3]

@loosebazooka Sorry, I missed this message, I have reworked the unpacking task for another reason.

Before when it didn’t work, the unpacking task was

task unpackLibsodium(dependsOn: downloadLibsodiumDeb) {
    inputs.property('libFilename', 'libsodium.so.23.3.0')
    inputs.property('debFile', downloadLibsodiumDeb.dest)
    outputs.file("${buildDir}/libsodium/linux/libsodium.so").withPropertyName('linuxLibraryFile')
    //...
}

This approach don’t work well as the task outputs (unpackLibsodium.outputs) are not present if the task didn’t run, also referring to unpackLibsodium.linuxLibraryFile don’t work (https://github.com/gradle/gradle/issues/15284).

After my refactoring the unpack task looks like this

task unpackLibsodium(dependsOn: downloadLibsodiumDeb) {
    description "Extract Libsodium Debian package file into $buildDir/libsodium directory"
    ext {
        debFile = downloadLibsodiumDeb.dest
        libFilename = 'libsodium.so.23.3.0'
        outDir = "${buildDir}/libsodium/linux"
        outFilename = 'libsodium.so'
        unpackedLibraryFile = "${outDir}/${outFilename}"
    }

    inputs.file(debFile)
    outputs.dir(outDir)
    outputs.file(unpackedLibraryFile)
    // ...
}

With this setup, the outDir property is known before the unpack task is run, and as it is declared in the ext block of the task, it can be referred to as unpackLibsodium.outDir in the jib, which does seem to work. Your suggestion to work around the issue is in fact very similar.

Still I think having the possibility to pass a closure there would be useful.