BUG: Error when mode set to offline - MissingFieldException: Field "sys/id" was not found.
See original GitHub issueDescribe the bug
In PyTorch Lightning Versions >= 1.6.0, the neptune logger does not work as expected while the logging mode is set to offline
.
Reproduction
https://colab.research.google.com/drive/1k1u25c-RxPiyhsVaOVq9difRtpmdaXVM?usp=sharing
Expected behavior
Logging the data offline as it used to in PyTorch Lightning Version<1.6.0.
Traceback
MissingFieldException Traceback (most recent call last) <ipython-input-8-ab2bf21b5ac7> in <module>() ----> 1 neptune_logger.experiment[“parameters/env/torch_version”] = “1.10.0+cu102”
11 frames /usr/local/lib/python3.7/dist-packages/neptune/new/handler.py in _get_attribute(self) 99 attr = self._container.get_attribute(self._path) 100 if attr is None: –> 101 raise MissingFieldException(self._path) 102 return attr 103
MissingFieldException:
----MissingFieldException-------------------------------------------------------
Field “sys/id” was not found.
There are two possible reasons: - There is a typo in a path. Double-check your code for typos. - You are fetching a field that other process created, but local representation is not synchronized. If you are sending metadata from multiple processes at the same time, synchronize the local representation before fetching values: run.sync()
Environment
The output of pip list
(relevant libraries):
absl-py 1.0.0
aiohttp 3.8.1
aiosignal 1.2.0
arrow 1.2.2
async-timeout 4.0.2
asynctest 0.13.0
attrs 21.4.0
boto3 1.22.9
botocore 1.25.9
bravado 11.0.3
bravado-core 5.17.0
cached-property 1.5.2
cachetools 4.2.4
certifi 2021.10.8
chardet 3.0.4
charset-normalizer 2.0.12
click 7.1.2
datascience 0.10.6
folium 0.2.1
fqdn 1.5.1
frozenlist 1.3.0
fsspec 2022.3.0
future 0.18.2
gitdb 4.0.9
GitPython 3.1.27
google-auth 1.35.0
google-auth-oauthlib 0.4.6
grpcio 1.44.0
idna 2.10
importlib-metadata 4.11.3
importlib-resources 5.7.1
isoduration 20.11.0
Jinja2 2.11.3
jmespath 1.0.0
jsonpointer 2.3
jsonref 0.2
jsonschema 4.3.3
Markdown 3.3.6
MarkupSafe 2.0.1
monotonic 1.6
msgpack 1.0.3
multidict 6.0.2
neptune-client 0.16.1
numpy 1.21.6
oauthlib 3.2.0
pandas 1.3.5
Pillow 7.1.2
pip 21.1.3
pip-tools 6.2.0
protobuf 3.17.3
psutil 5.4.8
pyasn1 0.4.8
pyasn1-modules 0.2.8
pyDeprecate 0.3.2
pyparsing 3.0.8
pyrsistent 0.18.1
pytest 3.6.4
python-apt 0.0.0
python-dateutil 2.8.2
python-utils 3.1.0
pytorch-lightning 1.6.0
pytz 2022.1
PyYAML 6.0
requests 2.23.0
requests-oauthlib 1.3.1
rfc3339-validator 0.1.4
rfc3987 1.3.8
rpy2 3.4.5
rsa 4.8
s3transfer 0.5.2
setuptools 57.4.0
simplejson 3.17.6
six 1.15.0
smmap 5.0.0
swagger-spec-validator 2.7.4
tensorboard 2.8.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorflow 2.8.0
torch 1.11.0+cu113
torchmetrics 0.8.2
torchsummary 1.5.1
torchvision 0.12.0+cu113
tqdm 4.64.0
typing-extensions 4.2.0
uri-template 1.2.0
webcolors 1.11.1
websocket-client 1.3.2
Werkzeug 1.0.1
wheel 0.37.1
yarl 1.7.2
zipp 3.8.0
- CUDA:
- GPU:
- Tesla K80
- available: True
- version: 11.3
- GPU:
- Packages:
- numpy: 1.21.6
- pyTorch_debug: False
- pyTorch_version: 1.11.0+cu113
- pytorch-lightning: 1.6.0
- tqdm: 4.64.0
- neptune-client: 0.16.1
- System:
- OS: Linux - Ubuntu 18.04.5 LTS
- architecture: 64bit
- processor: x86_64
- python: 3.7.13
- version: # 1 SMP Sun Apr 24 10:03:06 PDT 2022
- PyTorch Lightning Version: 1.6.0
- PyTorch Version: 1.11.0+cu113
- Python version: 3.7.13
Additional context
The diff of pytorch_lightning/loggers/neptune.py
between PyTorch Lightning Versions 1.5.9 and 1.6.0:
Issue Analytics
- State:
- Created a year ago
- Comments:10 (6 by maintainers)
Top GitHub Comments
Thanks for the fix!
Hi @DanTremonti
Thank you for reaching out once again!
We have it in our backlog, and I will let you know once it’s done.
Sorry for the delay, we are doing our best to keep all integrations up-to-date.