`"use ceph-volume to create {{ osd_objectstore }} osds"` is very slow, because it calls `ceph volume lvm list` once per OSD on a host

Bug Report

What happened: Running ceph-ansible on a Luminous / ceph-volume lvm / bluestore cluster takes a very long time, most of which is in the "use ceph-volume to create {{ osd_objectstore }} osds" task. You can see below that it takes at least 2 hours per host - and this is not creating any OSDs, just going through the already-deployed ones on each host.

ceph-osd : use ceph-volume to create bluestore osds ------------------ 11228.03s
ceph-osd : use ceph-volume to create bluestore osds ------------------ 8466.27s
ceph-osd : use ceph-volume to create bluestore osds ------------------ 8390.08s

Looking at it, this is because the ceph-volume plugin runs ceph volume lvm list individually for each OSD, and that takes a while (between about 15 and 120s). It should, instead, call ceph volume lvm list once, and process the output itself.

Environment:

• OS (e.g. from /etc/os-release): Ubuntu 16.04 and 18.04 (happens on both)
• Docker version if applicable (e.g. docker version): n/a
• Ansible version (e.g. ansible-playbook --version): 2.6
• ceph-ansible version (e.g. git head or tag or stable branch): 3.2.3
• Ceph version (e.g. ceph -v): Luminous

[dsavineau]

@mcv21 Would you be able to test the ceph-volume fix provided by [1] ? The associated patch for Luminous is [2]. This will be part of the next 12.2.13 release.

[1] https://tracker.ceph.com/issues/37490 [2] https://github.com/ceph/ceph/pull/30094

[mcv21]

We do have a faster ceph-volume at last. Next time I run ceph-ansible against that cluster I’ll see what the timing’s like…