Fix Java Client code generation issues

Problem

Try to generate code on the java client:

cd client/java
./gradlew generateCode

results in the following error:

Starting a Gradle Daemon, 1 incompatible Daemon could not be reused, use --status for details

> Task :generateCode FAILED
[main] INFO io.openlineage.client.Generator - Generating code for schemas:
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/OpenLineage.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/OutputStatisticsOutputDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/LifecycleStateChangeDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/SourceCodeLocationJobFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/OwnershipJobFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/SourceCodeJobFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/DataQualityMetricsInputDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/ColumnLineageDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/DatasetVersionDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/ErrorMessageRunFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/OwnershipDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/ExternalQueryRunFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/DocumentationJobFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/SchemaDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/DataQualityAssertionsDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/ParentRunFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/NominalTimeRunFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/StorageDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/SQLJobFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/SymlinksDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/DatasourceDatasetFacet.json
file:/Users/howardyoo/git/OpenLineage/client/java/../../spec/facets/DocumentationDatasetFacet.json
[main] WARN io.openlineage.client.Generator - This version of the spec is not published yet: https://openlineage.io/spec/1-15-0/OpenLineage.json
>> absolutePointer: #/$defs/RunEvent
:/$defs/RunEvent::RunEvent
>> absolutePointer: #/$defs/Run
:/$defs/Run::Run
>> absolutePointer: facets/ErrorMessageRunFacet.json
Exception in thread "main" java.lang.RuntimeException: Error while visiting ObjectType{properties: [Field{name: eventType, type: EnumType{values=(START,RUNNING,COMPLETE,ABORT,FAIL,OTHER)}}, Field{name: eventTime, type: PrimitiveType{name: string}}, Field{name: run, type: RefType{pointer: #/$defs/Run}}, Field{name: job, type: RefType{pointer: #/$defs/Job}}, Field{name: inputs, type: ArrayType{items: RefType{pointer: #/$defs/InputDataset}}}, Field{name: outputs, type: ArrayType{items: RefType{pointer: #/$defs/OutputDataset}}}, Field{name: producer, type: PrimitiveType{name: string}}, Field{name: schemaURL, type: PrimitiveType{name: string}}]}
        at io.openlineage.client.SchemaParser$TypeVisitor.visit(SchemaParser.java:115)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:221)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:209)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:60)
        at io.openlineage.client.SchemaParser$RefType.accept(SchemaParser.java:143)
        at io.openlineage.client.TypeResolver.<init>(TypeResolver.java:229)
        at io.openlineage.client.Generator.generate(Generator.java:127)
        at io.openlineage.client.Generator.main(Generator.java:72)
Caused by: java.lang.RuntimeException: Error while visiting RefType{pointer: #/$defs/Run}
        at io.openlineage.client.SchemaParser$TypeVisitor.visit(SchemaParser.java:115)
        at io.openlineage.client.TypeResolver$1.resolveFields(TypeResolver.java:103)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:76)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:60)
        at io.openlineage.client.SchemaParser$ObjectType.accept(SchemaParser.java:365)
        at io.openlineage.client.SchemaParser$TypeVisitor.visit(SchemaParser.java:113)
        ... 7 more
Caused by: java.lang.RuntimeException: Error while visiting ObjectType{properties: [Field{name: runId, type: PrimitiveType{name: string}}, Field{name: facets, type: AllOfType{children: [RefType{pointer: facets/ErrorMessageRunFacet.json}, RefType{pointer: facets/ExternalQueryRunFacet.json}, RefType{pointer: facets/NominalTimeRunFacet.json}, RefType{pointer: facets/ParentRunFacet.json}]}}]}
        at io.openlineage.client.SchemaParser$TypeVisitor.visit(SchemaParser.java:115)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:221)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:209)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:60)
        at io.openlineage.client.SchemaParser$RefType.accept(SchemaParser.java:143)
        at io.openlineage.client.SchemaParser$TypeVisitor.visit(SchemaParser.java:113)
        ... 12 more
Caused by: java.lang.RuntimeException: Error while visiting AllOfType{children: [RefType{pointer: facets/ErrorMessageRunFacet.json}, RefType{pointer: facets/ExternalQueryRunFacet.json}, RefType{pointer: facets/NominalTimeRunFacet.json}, RefType{pointer: facets/ParentRunFacet.json}]}
        at io.openlineage.client.SchemaParser$TypeVisitor.visit(SchemaParser.java:115)
        at io.openlineage.client.TypeResolver$1.resolveFields(TypeResolver.java:103)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:76)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:60)
        at io.openlineage.client.SchemaParser$ObjectType.accept(SchemaParser.java:365)
        at io.openlineage.client.SchemaParser$TypeVisitor.visit(SchemaParser.java:113)
        ... 17 more
Caused by: java.lang.RuntimeException: Error while visiting RefType{pointer: facets/ErrorMessageRunFacet.json}
        at io.openlineage.client.SchemaParser$TypeVisitor.visit(SchemaParser.java:115)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:151)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:60)
        at io.openlineage.client.SchemaParser$AllOfType.accept(SchemaParser.java:224)
        at io.openlineage.client.SchemaParser$TypeVisitor.visit(SchemaParser.java:113)
        ... 22 more
Caused by: java.lang.StringIndexOutOfBoundsException: Range [0, -1) out of bounds for length 32
        at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:55)
        at java.base/jdk.internal.util.Preconditions$1.apply(Preconditions.java:52)
        at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:213)
        at java.base/jdk.internal.util.Preconditions$4.apply(Preconditions.java:210)
        at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:98)
        at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckFromToIndex(Preconditions.java:112)
        at java.base/jdk.internal.util.Preconditions.checkFromToIndex(Preconditions.java:349)
        at java.base/java.lang.String.checkBoundsBeginEnd(String.java:4589)
        at java.base/java.lang.String.substring(String.java:2703)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:185)
        at io.openlineage.client.TypeResolver$1.visit(TypeResolver.java:60)
        at io.openlineage.client.SchemaParser$RefType.accept(SchemaParser.java:143)
        at io.openlineage.client.SchemaParser$TypeVisitor.visit(SchemaParser.java:113)
        ... 26 more

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':generateCode'.
> Process 'command '/opt/homebrew/Cellar/openjdk/18.0.2.1_1/libexec/openjdk.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1

This happens when the issue #1189 is resolved and spec file now contains references that exist outside of the current json file. The logic inside https://github.com/OpenLineage/OpenLineage/blob/28eb919c7c847f559efc2d251b90e9070712d11a/client/java/generator/src/main/java/io/openlineage/client/TypeResolver.java#L182 will try to check for the location of the # character which may not exist in the given json file - which may not exist. Also, the type resolver does not seem to understand how to access the types defined elsewhere in other files.

Solution

• Add additional logic to make sure the types from other files can be properly accessed / used
• Make sure no other issues happen with the new spec changes mentioned in #1189

[pawel-big-lebowski]

The bottom allOf line makes sure that in case of any new anyOf added later on, it should extend BaseFacet. So it prevents from adding anyOf which is not BaseFacet.

[collado-mike]

Ah, I missed that other BaseFacet reference! Nice work.