bug: kvm multi host plan fails

Problem

kcli cannot create VMs, via a plan, spread across multiple physical hosts

Scenario

I have 3 kvm/libvirt nodes with the following plan:

---
parameters:
  image: ubuntu2204
  disk_size: 20

dev-cp-1:
  client: lab-01
  profile: large
  image: {{ image }}
  reservehost: false
  disks:
  - size: {{ disk_size }}
  nets: [ br10 ]
dev-cp-2:
  client: lab-02
  profile: large
  image: {{ image }}
  reservehost: false
  disks:
  - size: {{ disk_size }}
  nets: [ br10 ]
dev-cp-3:
  client: lab-03
  profile: large
  image: {{ image }}
  reservehost: false
  disks:
  - size: {{ disk_size }}
  nets: [ br10 ]

Observations

1. When I run kcli create plan -f ~/.kcli/plans/k3s-dev1.yml k3s-dev the first VM is created on the correct kvm node however kcli reports

lunix@lumbergh]  (main) -> kcli create plan -f ~/.kcli/plans/k3s-dev1.yml k3s-dev
Deploying Vms...
Hypervisor not compatible with nesting. Skipping
Waiting for vm dev-cp-1 to finish customisation
VM dev-cp-1 not found

If I look on lab-01 the vm does exist.

Reading https://github.com/karmab/kcli/blob/master/kvirt/config.py#L2427 and I think perhaps the wait_finish method is connecting to my default ‘client’(kvm host) and not the appropriate one specified in the plan.

1. If I change the plan so that each node has client: lab-01 and run kcli -C lab-01 create plan -f ~/.kcli/plans/k3s-dev1.yml k3s-dev then the plan completes.

[karmab]

to use kube workflows with multihosts, you will need to leverage the vmrules functionality that allows you to overwrite attributes of vms at deployment time, matching them by their names (either through globbing or regex).

For instance, you would use the following in your parameter file

vmrules:
- dev-cp-1:
    client: lab-01
- dev-cp-2:
    client: lab-02
- dev-cp-3:
    client: lab-03

Actually, you can use this paramfile with your plan and remove the client lines you currently have and you will have the same behaviour

[aussielunix]

fd49454 should fix it (and also allow the “builtin” kube to actually work as expected accross multiple clients)

Thanks @karmab. I was able to run kcli create plan -f ~/.kcli/plans/k3s-dev1.yml k3s-dev and it completed just fine now.

lunix@lumbergh]  (main) -> kcli -C all get plans
+---------+--------+----------+
|   Plan  |  Host  |   Vms    |
+---------+--------+----------+
| k3s-dev | lab-01 | dev-cp-1 |
| k3s-dev | lab-02 | dev-cp-2 |
| k3s-dev | lab-03 | dev-cp-3 |
+---------+--------+----------+

I haven’t tested using the built in kube creation because I am not sure how to do multi-host with that.