[Feature][runtime env] refactor URI Protocol in runtime env.

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

Make different protocols become different plugins. In simple terms, I want to implement a SmartURIHandle, somewhat similar to io_path:

class BaseURIHandler:
    def __init__(self, uri):
        ...
    async def async_download(self, file_name):
        ...
    
    async def get_hash_from_uri(self):
        ...
    
    async def get_hash_from_file(self, local_path):    
        ...

class SmartURIHandler:

    @classmethod
    def register_handler(cls, handler: BaseURIHandler):
        ....
    
    async def async_download(self, urI):
        ...
    
    async def get_hash_from_uri(self):
        ...
    
    async def get_hash_from_file(self, local_path):    
        ...

# download file
async def download_package(uri, file_name):
    return await SmartURIHandler(uri).download(file_name)

This simplifies the implementation of runtime_env.packing.py, and in the future, we have the opportunity to let users define their own protocols outside of ray.

Use case

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[Catch-Bull]

@Catch-Bull the idea looks good, but I’m not sure about the specifics of this proposal. I think we could do something simpler like define an abstract URIHandler and then register the supported ones like this:

uri_handlers: Dict[str, URIHandler] = {"http": HTTPHandler}

ray.runtime_env.register_uri_handler("custom-uri", CustomURIHandler)

no problem and I will open a PR, we can discuss details in PR’s comment

[Catch-Bull]

This looks like a good idea to me. What does register_handler do?

make SmartURIHandler have the ability to handle a specific URI, for example:

class MyURIHandler:
    PROTOCOL = "my_uri"

# this will raise exception with message "unregister protocol: my_uri "
SmartURIHandler("my_uri://test.zip")

SmartURIHandler.register_handler(MyURIHandler)

# It will be OK
SmartURIHandler("my_uri://test.zip")