File Dependencies relative path under pipeline file properties

Describe the issue My project contains folder structure as follows:

/servicelib/controllers/
/servicelib/models/
/pipelines/main/
/pipelines/experiment/

I have notebooks and related pipelines under /pipelines/main/ and some python libraries used by the notebooks under /servicelib/. This is why I added a relative path to the parent folder as ../../servicelib/ under File Dependency as part of the notebook properties in the pipeline. This runs successfully when running the pipeline in-place locally, but when running it using the kubflow runtime the following error occurs:

Traceback (most recent call last):
  File "/src/conda/envs/notebook/lib/python3.7/site-packages/tornado/web.py", line 1704, in _execute
    result = await result
  File "/src/conda/envs/notebook/lib/python3.7/site-packages/elyra/pipeline/handlers.py", line 89, in post
    response = await PipelineProcessorManager.instance().process(pipeline)
  File "/src/conda/envs/notebook/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 70, in process
    res = await asyncio.get_event_loop().run_in_executor(None, processor.process, pipeline)
  File "/src/conda/envs/notebook/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/src/conda/envs/notebook/lib/python3.7/site-packages/elyra/pipeline/processor_kfp.py", line 69, in process
    format(pipeline_name, pipeline_path), str(ex)) from ex
RuntimeError: ('Error compiling pipeline reftest-0527064628 at /tmp/tmpavdhj1ol/reftest-0527064628.tar.gz', "Node 'reftest' referenced dependencies that were not found: {'../../servicelib/controllers/*.py', '../../servicelib/services/*.py', '../../servicelib/models/*.py'}")

Screenshot of the properties window

Expected behavior Expected to be able to run the pipeline successfully with File Dependencies copied from relative parent path

Pipeline runtime environment If the issue is related to pipeline execution, identify the environment where the pipeline is executed

• Kubeflow Pipelines

[pacospace]

To summarize:

• the next 2.3 release includes support for symlinks
• development best practices will be documented (Document development best practices for notebook | script based nodes #1703)
• validation service will check for file references that are outside the pipeline [project’s] scope (Validation service initial changes #1664) to prevent the reported issue

Please let me know if I missed anything.

Thanks @ptitzler, what is ETA for 2.3 release stable?

[ptitzler]

To summarize:

• the next 2.3 release includes support for symlinks
• development best practices will be documented (#1703)
• validation service will check for file references that are outside the pipeline [project’s] scope (#1664) to prevent the reported issue

Please let me know if I missed anything.