Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

file is not defined

See original GitHub issue

pipeline.yaml

tasks:
  - source: json_test.py
    product: 
      nb: output.ipynb
      data: empty.csv

json_test.py

import json
from pathlib import Path

# + tags=['parameters']
upstream = None
product = None
# -

json_path = Path(__file__).parent / 'simple.json'
data = json.load(json_path.open())

simple.json

{
    "A": [1,2,3],
    "B": [4,5,6]
}

Error

(edgar-kU9hEoDJ-py3.8) hopkir@ANC-WX740501173:~/edgar$ ploomber build -e test.yaml 
/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/spec/dagspec.py:336: UserWarning: The following placeholders are declared in the environment but unused in the spec: {'version', 'code', 'artifacts', 'sap_archived_thru_date', 'data'}
  warnings.warn('The following placeholders are declared in the '
Executing: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:01<00:00,  3.02cell/s]
Building task 'json_test': 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.37s/it]
Traceback (most recent call last):
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/cli/io.py", line 20, in wrapper
    fn(**kwargs)
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/cli/build.py", line 51, in main
    report = dag.build(force=args.force, debug=args.debug)
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/dag/dag.py", line 482, in build
    report = callable_()
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/dag/dag.py", line 581, in _build
    raise build_exception
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/dag/dag.py", line 513, in _build
    task_reports = self._executor(dag=self,
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/executors/serial.py", line 138, in __call__
    raise DAGBuildError(str(exceptions_all))
ploomber.exceptions.DAGBuildError: 
======================================================================================================= DAG build failed =======================================================================================================
-------------------------------------------------------------- NotebookRunner: json_test -> MetaProduct({'data': File('empty.csv'), 'nb': File('output.ipynb')}) ---------------------------------------------------------------
----------------------------------------------------------------------------------------------- /home/hopkir/edgar/json_test.py ------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/tasks/notebook.py", line 520, in run
    pm.execute_notebook(str(tmp), str(path_to_out_ipynb),
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/papermill/execute.py", line 122, in execute_notebook
    raise_for_execution_errors(nb, output_path)
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/papermill/execute.py", line 234, in raise_for_execution_errors
    raise error
papermill.exceptions.PapermillExecutionError: 
---------------------------------------------------------------------------
Exception encountered at "In [4]":
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/tmp/ipykernel_32573/1452827539.py in <module>
----> 1 json_path = Path(__file__).parent / 'simple.json'
      2 data = json.load(json_path.open())

NameError: name '__file__' is not defined


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/tasks/abc.py", line 562, in _build
    res = self._run()
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/tasks/abc.py", line 669, in _run
    self.run()
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/tasks/notebook.py", line 523, in run
    raise TaskBuildError('An error occurred when calling'
ploomber.exceptions.TaskBuildError: An error occurred when calling papermil.execute_notebook, partially executed notebook with traceback available at /home/hopkir/edgar/output.ipynb

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/executors/serial.py", line 186, in catch_exceptions
    fn()
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/executors/serial.py", line 159, in __call__
    return self.fn(**self.kwargs)
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/executors/serial.py", line 166, in catch_warnings
    result = fn()
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/executors/serial.py", line 159, in __call__
    return self.fn(**self.kwargs)
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/executors/serial.py", line 235, in build_in_subprocess
    report, meta = task._build(**build_kwargs)
  File "/home/hopkir/.cache/pypoetry/virtualenvs/edgar-kU9hEoDJ-py3.8/lib/python3.8/site-packages/ploomber/tasks/abc.py", line 581, in _build
    raise TaskBuildError(msg) from e
ploomber.exceptions.TaskBuildError: Error building task "json_test"
======================================================================================================= Summary (1 task) =======================================================================================================
NotebookRunner: json_test -> MetaProduct({'data': File('empty.csv'), 'nb': File('output.ipynb')})
======================================================================================================= DAG build failed =======================================================================================================