accelerate test fails with deepspeed and fp16 enabled in config

Hi there,

First, thanks for the great work.

I wanted to give accelerate a spin and followed the docs to setup a configuration file with both deepspeed and fp16 enabled. Here’s the resulting yaml:

compute_environment: LOCAL_MACHINE
deepspeed_config:
  gradient_accumulation_steps: 2
  offload_optimizer_device: cpu
  zero_stage: 3
distributed_type: DEEPSPEED
fp16: true
machine_rank: 0
main_process_ip: null
main_process_port: null
main_training_function: main
num_machines: 1
num_processes: 2

I then tried to test the setup using: accelerate test --config_file ./my_config.yaml

This then throws an error saying: AttributeError: 'DeepSpeedPlugin' object has no attribute 'fp16' which seems to be stemming from accelerate/state.py line 232: use_fp16 = self.deepspeed_plugin.fp16 if self.distributed_type == DistributedType.DEEPSPEED else self.use_fp16

Let me know if you need any more information 😃

[nsmdgr]

Sure. I have version 0.5.5 installed today via pip install deepspeed.

[chris-opendata]

This is fixed in the latest release.

Thank you for your update.