[General] generator fails with oas 3 that includes relative local file ref that includes remote file ref

Description

Generator fails when it reads openapi.yaml that includes relative local file ref that includes remote file ref (i.e. root -> local ref -> remote ref).

I use “jaxrs-jersey” language for this sample, but I debugged and found that the problem is not limtied in jaxrs-jersey or java. All languages are affected, because the parser has the root problem.

openapi-generator version

current master(e6058824510f0d9548ad0af7a233e6e75e55bf91 )

OpenAPI declaration file content or url

https://gist.github.com/fujigon/e2e1594a011022245e66fa6c21954ea9

Command line used for generation

# git clone, etc
mvn package
java -jar modules/openapi-generator-cli/target/openapi-generator-cli.jar generate -i openapi.yaml -g jaxrs-jersey

Steps to reproduce

When I execute the generator with the gisted openapi.yaml file, I get

$ java -jar modules/openapi-generator-cli/target/openapi-generator-cli.jar generate -i openapi.yaml -g jaxrs-jersey
java.nio.file.InvalidPathException: Illegal char <:> at index 7: ./https://raw.githubusercontent.com/swagger-api/swagger-parser/v2.0.2/modules/swagger-parser-v3/src/test/resources/relative/globals.yaml
        at sun.nio.fs.WindowsPathParser.normalize(Unknown Source)
        at sun.nio.fs.WindowsPathParser.parse(Unknown Source)
        at sun.nio.fs.WindowsPathParser.parse(Unknown Source)
        at sun.nio.fs.WindowsPath.parse(Unknown Source)
        at sun.nio.fs.WindowsFileSystem.getPath(Unknown Source)
        at sun.nio.fs.AbstractPath.resolve(Unknown Source)
        at io.swagger.v3.parser.util.RefUtils.readExternalRef(RefUtils.java:148)
        at io.swagger.v3.parser.ResolverCache.loadRef(ResolverCache.java:116)
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:47)
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefSchema(ExternalRefProcessor.java:675)
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processProperties(ExternalRefProcessor.java:180)
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:142)
        at io.swagger.v3.parser.processors.SchemaProcessor.processReferenceSchema(SchemaProcessor.java:201)
        at io.swagger.v3.parser.processors.SchemaProcessor.processSchema(SchemaProcessor.java:36)
        at io.swagger.v3.parser.processors.ResponseProcessor.processResponse(ResponseProcessor.java:48)
        at io.swagger.v3.parser.processors.OperationProcessor.processOperation(OperationProcessor.java:63)
        at io.swagger.v3.parser.processors.PathsProcessor.processPaths(PathsProcessor.java:83)
        at io.swagger.v3.parser.OpenAPIResolver.resolve(OpenAPIResolver.java:49)
        at io.swagger.v3.parser.OpenAPIV3Parser.readLocation(OpenAPIV3Parser.java:53)
        at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:19)
        at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:524)
        at org.openapitools.codegen.cmd.Generate.run(Generate.java:341)
        at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:59)
[main] WARN  io.swagger.v3.parser.OpenAPIV3Parser - Exception while reading:
java.lang.RuntimeException: Unable to load RELATIVE ref: ./https://raw.githubusercontent.com/swagger-api/swagger-parser/v2.0.2/modules/swagger-parser-v3/src/test/resources/relative/globals.yaml
        at io.swagger.v3.parser.util.RefUtils.readExternalRef(RefUtils.java:159) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.ResolverCache.loadRef(ResolverCache.java:116) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:47) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefSchema(ExternalRefProcessor.java:675) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processProperties(ExternalRefProcessor.java:180) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:142) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.SchemaProcessor.processReferenceSchema(SchemaProcessor.java:201) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.SchemaProcessor.processSchema(SchemaProcessor.java:36) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.ResponseProcessor.processResponse(ResponseProcessor.java:48) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.OperationProcessor.processOperation(OperationProcessor.java:63) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.PathsProcessor.processPaths(PathsProcessor.java:83) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.OpenAPIResolver.resolve(OpenAPIResolver.java:49) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.OpenAPIV3Parser.readLocation(OpenAPIV3Parser.java:53) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:19) [openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:524) [openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at org.openapitools.codegen.cmd.Generate.run(Generate.java:341) [openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:59) [openapi-generator-cli.jar:3.2.2-SNAPSHOT]
Caused by: java.nio.file.InvalidPathException: Illegal char <:> at index 7: ./https://raw.githubusercontent.com/swagger-api/swagger-parser/v2.0.2/modules/swagger-parser-v3/src/test/resources/relative/globals.yaml
        at sun.nio.fs.WindowsPathParser.normalize(Unknown Source) ~[na:1.8.0_161]
        at sun.nio.fs.WindowsPathParser.parse(Unknown Source) ~[na:1.8.0_161]
        at sun.nio.fs.WindowsPathParser.parse(Unknown Source) ~[na:1.8.0_161]
        at sun.nio.fs.WindowsPath.parse(Unknown Source) ~[na:1.8.0_161]
        at sun.nio.fs.WindowsFileSystem.getPath(Unknown Source) ~[na:1.8.0_161]
        at sun.nio.fs.AbstractPath.resolve(Unknown Source) ~[na:1.8.0_161]
        at io.swagger.v3.parser.util.RefUtils.readExternalRef(RefUtils.java:148) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        ... 16 common frames omitted
[main] WARN  o.o.codegen.utils.ModelUtils - Failed to get the schema name: ./components/schemas/SchemaIncludingExternalRef.yaml#/components/schemas/SchemaIncludingExternalRef
[main] WARN  o.o.codegen.utils.ModelUtils - Failed to get the schema name: ./components/schemas/SchemaIncludingExternalRef.yaml#/components/schemas/SchemaIncludingExternalRef
Exception in thread "main" org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).
 | Error count: 1, Warning count: 1
Errors:
        -Unable to load RELATIVE ref: ./https://raw.githubusercontent.com/swagger-api/swagger-parser/v2.0.2/modules/swagger-parser-v3/src/test/resources/relative/globals.yaml
Warnings:
        -Unused model: SchemaIncludingExternalRef

        at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:541)
        at org.openapitools.codegen.cmd.Generate.run(Generate.java:341)
        at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:59)

Related issues/PRs

N/A

Suggest a fix/enhancement

The parser has problem with handling remote ref while resolving the ref. The ref is converted to ./https://raw.githubusercontent.com/swagger-api/swagger-parser/v2.0.2/modules/swagger-parser-v3/src/test/resources/relative/globals.yaml (./ is attached to the uri) while parsing.

https://github.com/fujigon/swagger-parser/commits/feature/relative-ref-with-http-ref-bugfix branch will fix this problem.

[jmini]

For the time beeing, I have merged https://github.com/OpenAPITools/swagger-parser/pull/15, meaning that the version 2.0.3-SNAPSHOT of our fork of swagger-parser will contain the change.

    <dependency>
      <groupId>org.openapitools.swagger.parser</groupId>
      <artifactId>swagger-parser</artifactId>
      <version>2.0.3-SNAPSHOT</version>
    </dependency>

Artifact is published in the sonatype snapshots repository:

https://oss.sonatype.org/content/repositories/snapshots/

---

To test it in OpenAPI-Generator, the swagger-parser-version value in the main pom:

https://github.com/OpenAPITools/openapi-generator/blob/3d06900e68b98f1b0b79d746c3d056f5746b9762/pom.xml#L1136

can be changed to 2.0.3-SNAPSHOT (for the moment we are already using the org.openapitools.swagger.parser value as groupId for the swagger-parser dependency)

But this can not be commited to our master or 3.3.x branches, because we do not want to use SNAPSHOT versions in our builds.

---

More background:

The idea (see https://github.com/OpenAPITools/openapi-generator/issues/68) is to release a 2.0.3-OpenAPITools.org-1 if swagger-parser release 2.0.3 does not comes in time for our OpenAPI-Generator 3.3.0 release.

[fujigon]

Thanks for quick reply.

• I created an issue in parser project. https://github.com/swagger-api/swagger-parser/issues/805
• I sent a PR to parser project, too https://github.com/swagger-api/swagger-parser/pull/806

Updating Swagger-Parser is something we can consider for 3.3.0 (due date by the end of this month).

Great. thanks for letting me know the schedule.