[Bug] Aim UI not opening on Sagemaker Notebook

🐛 Bug

First of all, really thanks for building this! 😃 I tried to use Aim on a sagemaker notebook but the full page UI doesn’t seem to open up fine. I initialized Aim in one of my already existing project dirs. And when I use aim up in one of the Jupyterlab terminals this is what I get:

aim up
Running Aim UI on repo `<Repo#-8308802422050181745 path=/home/ec2-user/SageMaker/test_root/testing/cola-sentence-acceptability/.aim read_only=None>`
Open http://127.0.0.1:43800
Press Ctrl+C to exit

Usually sagemaker notebooks provide a proxy to access such localhost ports by going to https://<notebook_name>.notebook.us-east-1.sagemaker.aws/proxy/43800/. When I open this URL clearly, there are few network requests made in order to attempt loading the UI, but it doesn’t load up and a blank page is shown. When I kill the aim up command, and reload the above link Sagemaker clearly shows a 500 : Internal Server Error which means there’s no process running at this port. So this atleast verifies that, once aim up is run the url i mentioned above tries to load the correct UI.

I’ve tried setting up mlflow as well as tensorboard similarly on my sagemaker notebook, and can confirm they work perfectly (i.e, their full page UIs can load seamlessly using the same proxy url that I mentioned above). Please note the objective here is to not load the aim ui using the jupyter notebook method which Aim already provides (https://aimstack.readthedocs.io/en/latest/guides/integrations/basic_aim_jupyter_notebook.html) but to actually load the UI full screen to analyze results better (since i’ve a bunch of experiments so would be much more convenient/useful to load the entire UI to analyze all of them easily, especially given the full page UI for tensorboard and mlflow load seamlessly).

The only failed request that I see in Network tab is - a GET request to: https://console.aws.amazon.com/sagemaker/home?region=us-east-1&notebookState=L3N0YXRpYy1maWxlcy9tYW5pZmVzdC5qc29u which fails due to CORS error. I’m not sure from the URL what this request is actually trying to load, but is there any request for loading Aim UI that could fail due to this CORS error? If so, can this be fixed please? Also, after I got this suspicious error I verified by spawning tensorboard and mlflow, and there’s no CORS error when I actually try to load their full page urls through a similar proxy url as to what I mentioned above.

The best alternative that I’m using is mlflow, and I’d really like to move to Aim. My primary use-case is for seamless experiment tracking which clearly from the documentation looks like Aim provides better functionality for than mlflow.

Reference: See this for reference on how to open tensorboard on sagemaker notebook - https://levelup.gitconnected.com/how-to-use-tensorboard-in-an-amazon-sagemaker-notebook-instance-a41ce2fd973f . There are some answers on StackOverflow too regarding this proxy url of sagemaker notebook.

To reproduce

See details above

Expected behavior

Full page UI should open up correctly.

Environment

• Aim Version (e.g., 3.0.1) - Aim v3.5.3
• Python version - Python 3.7.12
• pip version - pip 21.3.1
• OS (e.g., Linux) - Linux on Sagemaker notebook
• Any other relevant information - See blog linked above on how to spawn tensorboard full page ui on sagemaker notebook’s jupyter lab based terminal

Additional context

NA

[ashutoshsaboo]

Yes it’s blocking me completely @SGevorg , i’m not able to use aim anyhow at all currently. Would really love to get aim ui up and running on sagemaker notebook. @rubenaprikyan , sure will join the Slack community. Would be really great if either one of you could help get this fixed. 😃

[ashutoshsaboo]

Thanks @rubenaprikyan @SGevorg . It’s opening on Sagemaker notebook so far atleast. But can we please wait for the proper fix PR to be merged https://github.com/jupyterhub/jupyter-server-proxy/pull/328#issue-1145874348 ? So once that’s shipped, Aim’s docs can be updated post that as well before we close this issue? As clearly using the continually maintained jupyter-server-proxy would be a more long term solution than a fork (which is the current case).