File Dependencies relative path under pipeline file properties
See original GitHub issueDescribe the issue My project contains folder structure as follows:
/servicelib/controllers/
/servicelib/models/
/pipelines/main/
/pipelines/experiment/
I have notebooks and related pipelines under /pipelines/main/
and some python libraries used by the notebooks under /servicelib/
. This is why I added a relative path to the parent folder as ../../servicelib/
under File Dependency as part of the notebook properties in the pipeline. This runs successfully when running the pipeline in-place locally, but when running it using the kubflow runtime the following error occurs:
Traceback (most recent call last):
File "/src/conda/envs/notebook/lib/python3.7/site-packages/tornado/web.py", line 1704, in _execute
result = await result
File "/src/conda/envs/notebook/lib/python3.7/site-packages/elyra/pipeline/handlers.py", line 89, in post
response = await PipelineProcessorManager.instance().process(pipeline)
File "/src/conda/envs/notebook/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 70, in process
res = await asyncio.get_event_loop().run_in_executor(None, processor.process, pipeline)
File "/src/conda/envs/notebook/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/src/conda/envs/notebook/lib/python3.7/site-packages/elyra/pipeline/processor_kfp.py", line 69, in process
format(pipeline_name, pipeline_path), str(ex)) from ex
RuntimeError: ('Error compiling pipeline reftest-0527064628 at /tmp/tmpavdhj1ol/reftest-0527064628.tar.gz', "Node 'reftest' referenced dependencies that were not found: {'../../servicelib/controllers/*.py', '../../servicelib/services/*.py', '../../servicelib/models/*.py'}")
Screenshot of the properties window
Expected behavior Expected to be able to run the pipeline successfully with File Dependencies copied from relative parent path
Pipeline runtime environment If the issue is related to pipeline execution, identify the environment where the pipeline is executed
- Kubeflow Pipelines
Issue Analytics
- State:
- Created 2 years ago
- Comments:11 (9 by maintainers)
Thanks @ptitzler, what is ETA for 2.3 release stable?
To summarize:
Please let me know if I missed anything.