[autoscaler][azure] example-full.yaml fails to launch due to faulty head node permissions configuration

In the following, could it be that this should be a deep update rather than a normal one?

https://github.com/ray-project/ray/blob/6babd1928c6175803500c1db4ca23b621d89a83f/python/ray/autoscaler/_private/commands.py#L586

E.g. when utilizing the Azure autoscaler, we have a “azure_arm_parameters” key that is being set in

https://github.com/ray-project/ray/blob/6babd1928c6175803500c1db4ca23b621d89a83f/python/ray/autoscaler/_private/azure/config.py#L106

Next to that it is also being set in the YAML definition which will then overwrite the key

As an example, when running my object changes from:

{
  "azure_arm_parameters": {
    "adminUsername": "ubuntu",
    "publicKey": "<MASKED>"
  }
}

towards

{
  "azure_arm_parameters": {
    "vmSize": "Standard_D2s_v3",
    "imagePublisher": "microsoft-dsvm",
    "imageOffer": "ubuntu-1804",
    "imageSku": "1804-gen2",
    "imageVersion": "20.07.06"
  }
}

Removing the adminUsername and publicKey which are required in the Azure ARM templates

[ijrsvt]

The config issue has been resolved, but I am seeing something else popping up.

Great to hear!!!

Should I make a new issue for this one?

Yes, that would be great (please assign me to it as well 😃 )

[XavierGeerinck]

The config issue has been resolved, but I am seeing something else popping up. Should I make a new issue for this one?

Traceback (most recent call last):
  File "/home/xanrin/.pyenv/versions/3.8.6/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/xanrin/.pyenv/versions/3.8.6/lib/python3.8/site-packages/ray/autoscaler/_private/updater.py", line 129, in run
    self.provider.set_node_tags(
  File "/home/xanrin/.pyenv/versions/3.8.6/lib/python3.8/site-packages/ray/autoscaler/_private/azure/node_provider.py", line 29, in wrapper
    return f(self, *args, **kwargs)
  File "/home/xanrin/.pyenv/versions/3.8.6/lib/python3.8/site-packages/ray/autoscaler/_private/azure/node_provider.py", line 226, in set_node_tags
    self.compute_client.virtual_machines.update(
AttributeError: 'VirtualMachinesOperations' object has no attribute 'update'
  Failed to setup head node.