How is multi-agent handled ?

Hello there, I was wondering how you handle multi-agent learning. Let’s take as an example the supported kaggle hungry-geese environment. There is a function in the environment class :

rule_based_action()

If it exists, does it mean that the trained policy represents 1 player and the three others are rule based with this function ?

[ikki407]

And if you want to train against different agents in self-play, you can set for specific opponents of self-play by re-writing generation.py (worker.py). (ref. gfootball example) Multi-agent learning and self play against specific opponents are not supported yet as a default function in HandyRL.

Multi-agent learning must consider the handling of GPU… Further consideration required.

[ikki407]

Hi, thank you for your consideration. You are right about self-play.

rule_based_action() is used in RuleBasedAgent class in evaluation.py. This agent is only used for evaluation of trained models (e.g. in the evaluation during python main.py --train, python main.py --eval.) However, RuleBasedAgent is not set as default agent in the current setting. If you want to evaluate the win-rate against rule-based agent during training, you need to change this line. And change this line if you want to change the opponent in model evaluation (python main.py --eval).