Understanding Multidevice Strategy

I have been trying to figure out how to max out both of my gpu’s that are in my system.

Tue Oct 13 15:15:00 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.80.02    Driver Version: 450.80.02    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  TITAN RTX           Off  | 00000000:01:00.0 Off |                  N/A |
| 41%   41C    P8    15W / 280W |    292MiB / 24220MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 1080    Off  | 00000000:02:00.0 Off |                  N/A |
| 21%   50C    P8     6W / 180W |      2MiB /  8119MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      2061      G   /usr/lib/xorg/Xorg                191MiB |
|    0   N/A  N/A      2745      G   ...mviewer/tv_bin/TeamViewer       13MiB |
|    0   N/A  N/A      2949      G   /usr/bin/gnome-shell               83MiB |
+-----------------------------------------------------------------------------+

GPU 0 has the most memory,

I’m trying to understand the -multidevice_strategy, how many layers are their… its not very clear to me what would be the best for 2 gpus, one with more memory than the other… or at least a starting off point…

I have just tried the value of 20 and this was the result…

[IridiumMaster]

@IridiumMaster The -multidevice_strategy parameter tells the code where to slice / cut the model, and in your case 2 GPUs means you want to have the model cut into 2 pieces (one for each GPU). So, for two GPUs you should only specify one value for -multidevice_strategy.

Thanks kindly, that worked very well for me.

[RobertGoacher]

@ProGamerGov Thank you so much for your reply; I really appreciate it.

I think I understand this now…but please correct me if I’m wrong.

So you need to use the multiple GPU strategy for high-resolution style transfers because individual GPUs don’t normally have enough memory to do the inference? If you have a GPU with lots of memory (for example a NVIDIA A100 GPU with 40GB of memory) you might be able to complete a render at a high resolution on that GPU without needing to use the multiple GPU strategy? But if you do need to use multiple GPUs you can split the processing (and therefore the memory usage) over multiple GPUs but there will be a decrease in speed and an increase in memory usage from using that strategy?