Include user provided session in dvc.api

Sagemaker uses session objects to interact with other AWS resources. This includes S3 buckets, which in case of Sagemaker’s Jupyter Instances use IAM roles to know which buckets it can or cannot access, and it doesn’t allow the creation of user profiles to pass to dvc.

Allowing user to directly provide a session (SM uses boto3.session.Session, instead of aiboto3.session.AiSession) would provide a direct way to use DVC from AWS’ Sagemaker.

I am promoting the use of the tool inside my team, so it would be really helpful if this could work in our main work environment which are sagemaker instances.

Thanks a lot!

[ElisaSuarez]

Thank you so so much! I feel kinda dumb now. I made various tests for pushing from SM, pulling into local, and vice-versa and worked every time.

I only have one last comment on this: I have been mostly following the DVC Getting started tutorials and in this one says to git add .dcv/.config, and this file will be different for each user… When I made the changes and pushed from sagemaker, it also changed the .config file, so the profile was overwritten from the name I gave when configuring the aws cli with SSO back to default, which in turns force me to always have to run dvc modify remote myremote --profile myprofile when pulling in my local machine after a git pull made from other machine. Is there a recommended way to deal with this? Specially when more than one person may be working with the data and the source code? (each person modifying the profile’s name to reflect their own aws cli setup?

[pmrowla]

Closing this issue since the boto3 session problem is resolved by installing the correct remote dependencies with dvc[s3]