Implement a shaded in-proc Pravega cluster

Problem Description The work of #573 provides an in-process Pravega cluster (LocalPravegaEmulator) for use in client-side integration tests. Dependency conflicts are likely to happen in such an environment, for example when writing Flink connector tests that use Flink’s in-process cluster.

A solution to the conflict is to build a shaded JAR of the Pravega emulator that isolates most of Pravega’s dependencies. Ideally the library would leave only io.pravega.* unshaded.

Suggested Fix I suggest the overall dependency tree be (as seen from the app perspective, with shaded deps elided):

flink-job
├─ flink-streaming-java (provided)
├─ pravega-connector (compile)
│  └─ pravega-client (compile)
└─ pravega-standalone (intTest)
   └─ pravega-client (compile)

To achieve the above, most of the shading work that is presently done in the connector should move to the client library.

[EronWright]

Some exposition on the rationale for my suggestion of moving shading (of Netty, etc) to the client library.

When we apply shading to a library, our goal is to prevent a conflict with the application that uses the library by hiding its dependencies. I’ll use Netty as an example of a dependency that we wish to hide.

Hiding Netty has two aspects: 1. relocating the Netty classes to io.pravega.shaded.netty. 2. removing Netty as a listed dependency in the library’s POM. If we did (1) but not (2), shading would be ineffective because the application would still see Netty as a transitive dependency.

Now consider the pravega connector. Today it relocates all the dependencies and removes the client library from its effective POM. If the client were still listed as a dependency of the connector, the app would again see Netty and other client dependencies (which we want to avoid).

The ideal design would allow the connector to list the client as a dependency in the effective POM. Likewise pravega-standalone should list the client as a dependency. To achieve this, the client itself should take responsibility for shading its dependencies. No further shading would then be needed in the connector. Meanwhile the standalone library would need additional shading of the server dependencies.

[andreipaduroiu]

I closed it for inactivity for more than 1 year and no assignees that are actively working on the project.