wandb fails on CUDA 8 and python3

• Weights and Biases version: 0.8.9 (according to pip)
• Python version: 3.7.1
• Operating System: linux

Description

During any action with wandb, it fails on import (SyntaxError). This happens both in any script or any CLI action. We are running CUDA 8.0, and I guess pynvml that is being imported doesn’t support python3.

Here’s CUDA version:

> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Tue_Jan_10_13:22:03_CST_2017
Cuda compilation tools, release 8.0, V8.0.61

Traceback on wandb import:

> python3 main.py
Traceback (most recent call last):
  File "main.py", line 8, in <module>
    import wandb
  File "/home/username/.local/lib/python3.7/site-packages/wandb/__init__.py", line 42, in <module>
    from wandb.apis import InternalApi, PublicApi, CommError
  File "/home/username/.local/lib/python3.7/site-packages/wandb/apis/__init__.py", line 95, in <module>
    from .public import Api as PublicApi
  File "/home/username/.local/lib/python3.7/site-packages/wandb/apis/public.py", line 21, in <module>
    from wandb.summary import HTTPSummary, download_h5
  File "/home/username/.local/lib/python3.7/site-packages/wandb/summary.py", line 15, in <module>
    from wandb.meta import Meta
  File "/home/username/.local/lib/python3.7/site-packages/wandb/meta.py", line 6, in <module>
    import pynvml
  File "/cm/local/apps/cuda/libs/current/pynvml/pynvml.py", line 1671
    print c_count.value
                ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(c_count.value)?

Traceback on CLI usage:

>  wandb
Traceback (most recent call last):
  File "/home/username/.local/bin/wandb", line 7, in <module>
    from wandb.cli import cli
  File "/home/username/.local/lib/python3.7/site-packages/wandb/__init__.py", line 42, in <module>
    from wandb.apis import InternalApi, PublicApi, CommError
  File "/home/username/.local/lib/python3.7/site-packages/wandb/apis/__init__.py", line 95, in <module>
    from .public import Api as PublicApi
  File "/home/username/.local/lib/python3.7/site-packages/wandb/apis/public.py", line 21, in <module>
    from wandb.summary import HTTPSummary, download_h5
  File "/home/username/.local/lib/python3.7/site-packages/wandb/summary.py", line 15, in <module>
    from wandb.meta import Meta
  File "/home/username/.local/lib/python3.7/site-packages/wandb/meta.py", line 6, in <module>
    import pynvml
  File "/cm/local/apps/cuda/libs/current/pynvml/pynvml.py", line 1671
    print c_count.value
                ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(c_count.value)?

Potential fix

I skimmed the code and the repo, it seems pynvml (Python bindings for NVIDIA Management Library) is used only in meta.py and stats.py to log used GPU stats and GPU count. A possible fix would be to wrap imports with try-catch + fail flag, or maybe an environment flag to skip that part entirely.

I can make a pull request over this weekend, but let me know if you have something else in mind.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.97. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[vanpelt]

Hey @Qnlp this is really strange. Can you print what experiment.config is after you’ve initialized the WandbLogger? A quick thing to try is our prerelease library. You can install with pip install wandb -U --pre.