Custom Policy Network - Off-Policy Algorithm - Advanced Customization

📚 Documentation

As stated in the documentation, for using a Custom Policy Network for off-policy algorithms #425 is linked which shows an example of how to do this. At the end register_policy("CustomTD3Policy", CustomTD3Policy) is used to use the Custom Policy e.g. for TD3/DDPG. Since it looks like, that in the commit https://github.com/DLR-RM/stable-baselines3/commit/254bb10c42e8f892e43af9da25aefc7c604c317c the register_policy-function was deleted/replaced, how can this be handled with the current version of SB3?

Checklist

• I have checked that there is no similar issue in the repo
• I have read the documentation

[araffin]

Hello, if you want the same feature as the register policy, the actual syntax is:

from stable_baselines3 import TD3
from stable_baselines3.td3.policies import TD3Policy

TD3.policy_aliases["CustomTD3Policy"] = TD3Policy

TD3("CustomTD3Policy", "MountainCarContinuous-v0").learn(5_000)

I’ll update the issue

[qgallouedec]

Thank you for reporting. We will update the documentation.

Use the following syntax:

from stable_baselines3 import TD3
from stable_baselines3.td3.policies import TD3Policy


class CustomTD3Policy(TD3Policy):
    ...


TD3(CustomTD3Policy, "MountainCarContinuous-v0").learn(5_000)