ceph-rgw: endpoints incorrectly set when https protocol used

Bug Report

What happened: Zone endpoints are incorrectly set in zone_endpoint_pairs list to use IP addresses instead of FQDN when HTTPS protocol is set:

rgw_multisite_proto: https

What you expected to happen: endpoint value of zone_endpoint_pairs list should contain URL with FQDN instead of IP address when https protocol is used.

How to reproduce it (minimal and precise):

Adding couple of debug tasks demonstrates it clearly that task create a list of zones and all their endpoints substitutes proper endpoint value from rgw_instances_all to endpoint value with IP address in zone_endpoint_pairs list:

diff --git a/roles/ceph-rgw/tasks/multisite/create_realm_zonegroup_zone_lists.yml b/roles/ceph-rgw/tasks/multisite/create_realm_zonegroup_zone_lists.yml
index 03e37b30..d94fbefa 100644
--- a/roles/ceph-rgw/tasks/multisite/create_realm_zonegroup_zone_lists.yml
+++ b/roles/ceph-rgw/tasks/multisite/create_realm_zonegroup_zone_lists.yml
@@ -5,6 +5,10 @@
   loop: "{{ rgw_instances_all }}"
   run_once: true
 
+- debug: var=realms
+
+- debug: var=rgw_instances_all
+
 - name: create list secondary_realms
   set_fact:
     secondary_realms: "{{ secondary_realms | default([]) | union([{ 'realm': item.rgw_realm, 'endpoint': item.endpoint, 'system_access_key': item.system_access_key, 'system_secret_key': item.system_secret_key, 'is_master': hostvars[item.host]['rgw_zonemaster'] }]) }}"
@@ -26,14 +30,22 @@
   loop: "{{ rgw_instances_all }}"
   run_once: true
 
+- debug: var=rgw_instances_all
+
 - name: create a list of dicts with each rgw endpoint and it's zone
   set_fact:
     zone_endpoint_pairs: "{{ zone_endpoint_pairs | default([]) | union([{ 'endpoint': hostvars[item.host]['rgw_multisite_proto'] + '://' + item.radosgw_address + ':' + item.radosgw_frontend_port | string, 'rgw_zone': item.rgw_zone, 'rgw_realm': item.rgw_realm, 'rgw_zonegroup': item.rgw_zonegroup, 'rgw_zonemaster': hostvars[item.host]['rgw_zonemaster']}]) }}"
   loop: "{{ rgw_instances_all }}"
   run_once: true
 
+- debug: var=zone_endpoint_pairs
+
 - name: create a list of zones and all their endpoints
   set_fact:
     zone_endpoints_list: "{{ zone_endpoints_list | default([]) | union([{'zone': item.rgw_zone, 'zonegroup': item.rgw_zonegroup, 'realm': item.rgw_realm, 'is_master': item.rgw_zonemaster, 'endpoints': ','.join(zone_endpoint_pairs | selectattr('rgw_zone','==',item.rgw_zone) | selectattr('rgw_realm','==',item.rgw_realm) | selectattr('rgw_zonegroup', '==', item.rgw_zonegroup) | map(attribute='endpoint'))}]) }}"
   loop: "{{ zone_endpoint_pairs }}"
   run_once: true
+
+- debug: var=zone_endpoints_list
+
+- fail:

Produces following output:

TASK [ceph-rgw : create list zones] *********************************************************************************************
Thursday 19 March 2020  09:43:24 +0100 (0:00:00.297)       0:05:15.845 ******** 
ok: [ceph2a] => (item={'instance_name': 'rgw0', 'radosgw_address': '172.16.xx.135', 'radosgw_frontend_port': 443, 'rgw_realm': 'ceph', 'rgw_zonegroup': 'aaa', 'rgw_zone': 'xxx', 'system_access_key': '6kWkikvapSnHyE22P7nO', 'system_secret_key': 'MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt', 'rgw_zone_user': 'zone_user', 'rgw_zone_user_display_name': 'Zone User', 'endpoint': 'https://ceph1.example.com:443', 'host': 'ceph2a'})
ok: [ceph2a] => (item={'instance_name': 'rgw0', 'radosgw_address': '172.16.xx.135', 'radosgw_frontend_port': 443, 'rgw_realm': 'ceph', 'rgw_zonegroup': 'aaa', 'rgw_zone': 'xxx', 'system_access_key': '6kWkikvapSnHyE22P7nO', 'system_secret_key': 'MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt', 'rgw_zone_user': 'zone_user', 'rgw_zone_user_display_name': 'Zone User', 'endpoint': 'https://ceph1.example.com:443', 'host': 'ceph2b'})
ok: [ceph2a] => (item={'instance_name': 'rgw0', 'radosgw_address': '172.16.xx.135', 'radosgw_frontend_port': 443, 'rgw_realm': 'ceph', 'rgw_zonegroup': 'aaa', 'rgw_zone': 'xxx', 'system_access_key': '6kWkikvapSnHyE22P7nO', 'system_secret_key': 'MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt', 'rgw_zone_user': 'zone_user', 'rgw_zone_user_display_name': 'Zone User', 'endpoint': 'https://ceph1.example.com:443', 'host': 'ceph2c'})

TASK [ceph-rgw : debug] *********************************************************************************************************
Thursday 19 March 2020  09:43:24 +0100 (0:00:00.371)       0:05:16.216 ******** 
ok: [ceph2a] => 
  rgw_instances_all:
  - endpoint: https://ceph1.example.com:443
    host: ceph2a
    instance_name: rgw0
    radosgw_address: 172.16.xx.135
    radosgw_frontend_port: 443
    rgw_realm: ceph
    rgw_zone: xxx
    rgw_zone_user: zone_user
    rgw_zone_user_display_name: Zone User
    rgw_zonegroup: aaa
    system_access_key: 6kWkikvapSnHyE22P7nO
    system_secret_key: MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt
  - endpoint: https://ceph1.example.com:443
    host: ceph2b
    instance_name: rgw0
    radosgw_address: 172.16.xx.135
    radosgw_frontend_port: 443
    rgw_realm: ceph
    rgw_zone: xxx
    rgw_zone_user: zone_user
    rgw_zone_user_display_name: Zone User
    rgw_zonegroup: aaa
    system_access_key: 6kWkikvapSnHyE22P7nO
    system_secret_key: MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt
  - endpoint: https://ceph1.example.com:443
    host: ceph2c
    instance_name: rgw0
    radosgw_address: 172.16.xx.135
    radosgw_frontend_port: 443
    rgw_realm: ceph
    rgw_zone: xxx
    rgw_zone_user: zone_user
    rgw_zone_user_display_name: Zone User
    rgw_zonegroup: aaa
    system_access_key: 6kWkikvapSnHyE22P7nO
    system_secret_key: MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt
[snip]

TASK [ceph-rgw : create a list of dicts with each rgw endpoint and it's zone] ***************************************************
Thursday 19 March 2020  09:43:24 +0100 (0:00:00.320)       0:05:16.537 ******** 
ok: [ceph2a] => (item={'instance_name': 'rgw0', 'radosgw_address': '172.16.xx.135', 'radosgw_frontend_port': 443, 'rgw_realm': 'ceph', 'rgw_zonegroup': 'aaa', 'rgw_zone': 'xxx', 'system_access_key': '6kWkikvapSnHyE22P7nO', 'system_secret_key': 'MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt', 'rgw_zone_user': 'zone_user', 'rgw_zone_user_display_name': 'Zone User', 'endpoint': 'https://ceph1.example.com:443', 'host': 'ceph2a'})
ok: [ceph2a] => (item={'instance_name': 'rgw0', 'radosgw_address': '172.16.xx.135', 'radosgw_frontend_port': 443, 'rgw_realm': 'ceph', 'rgw_zonegroup': 'aaa', 'rgw_zone': 'xxx', 'system_access_key': '6kWkikvapSnHyE22P7nO', 'system_secret_key': 'MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt', 'rgw_zone_user': 'zone_user', 'rgw_zone_user_display_name': 'Zone User', 'endpoint': 'https://ceph1.example.com:443', 'host': 'ceph2b'})
ok: [ceph2a] => (item={'instance_name': 'rgw0', 'radosgw_address': '172.16.xx.135', 'radosgw_frontend_port': 443, 'rgw_realm': 'ceph', 'rgw_zonegroup': 'aaa', 'rgw_zone': 'xxx', 'system_access_key': '6kWkikvapSnHyE22P7nO', 'system_secret_key': 'MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt', 'rgw_zone_user': 'zone_user', 'rgw_zone_user_display_name': 'Zone User', 'endpoint': 'https://ceph1.example.com:443', 'host': 'ceph2c'})

TASK [ceph-rgw : debug] *********************************************************************************************************
Thursday 19 March 2020  09:43:25 +0100 (0:00:00.472)       0:05:17.009 ******** 
ok: [ceph2a] => 
  zone_endpoint_pairs:
  - endpoint: https://172.16.xx.135:443
    rgw_realm: ceph
    rgw_zone: xxx
    rgw_zonegroup: aaa
    rgw_zonemaster: false
ok: [ceph2b] => 
  zone_endpoint_pairs:
  - endpoint: https://172.16.xx.135:443
    rgw_realm: ceph
    rgw_zone: xxx
    rgw_zonegroup: aaa
    rgw_zonemaster: false
ok: [ceph2c] => 
  zone_endpoint_pairs:
  - endpoint: https://172.16.xx.135:443
    rgw_realm: ceph
    rgw_zone: xxx
    rgw_zonegroup: aaa
    rgw_zonemaster: false

TASK [ceph-rgw : create a list of zones and all their endpoints] ****************************************************************
Thursday 19 March 2020  09:43:25 +0100 (0:00:00.316)       0:05:17.326 ******** 
ok: [ceph2a] => (item={'endpoint': 'https://172.16.xx.135:443', 'rgw_zone': 'xxx', 'rgw_realm': 'ceph', 'rgw_zonegroup': 'aaa', 'rgw_zonemaster': False})

TASK [ceph-rgw : debug] *********************************************************************************************************
Thursday 19 March 2020  09:43:25 +0100 (0:00:00.176)       0:05:17.502 ******** 
ok: [ceph2a] => 
  zone_endpoints_list:
  - endpoints: https://172.16.xx.135:443
    is_master: false
    realm: ceph
    zone: xxx
    zonegroup: aaa
ok: [ceph2b] => 
  zone_endpoints_list:
  - endpoints: https://172.16.xx.135:443
    is_master: false
    realm: ceph
    zone: xxx
    zonegroup: aaa
ok: [ceph2c] => 
  zone_endpoints_list:
  - endpoints: https://172.16.xx.135:443
    is_master: false
    realm: ceph
    zone: xxx
    zonegroup: aaa

In consequent tasks, connections to these endpoints fail, since HTTPS connection to IP address fails.

Share your group_vars files, inventory:

[xxx]
ceph2a
ceph2b
ceph2c
 
[mons:children]
xxx
 
[mgrs:children]
xxx
 
[rgws:children]
xxx

group_vars/all.yml:

radosgw_address: "{{ ansible_default_ipv4.address }}"
radosgw_num_instances: 1
radosgw_frontend_ssl_certificate: "/etc/ceph/ceph1.example.com.pem"
radosgw_frontend_port: 443

#############
# MULTISITE #
#############
rgw_multisite: True
rgw_multisite_proto: "https"

rgw_zonegroup: "aaa"
rgw_zone_user: "zone_user"
rgw_zone_user_display_name: "Zone User"
rgw_realm: "ceph"
system_access_key: "{{ lookup('env','CEPH_SYSTEM_ACCESS_KEY') }}"
system_secret_key: "{{ lookup('env','CEPH_SYSTEM_SECRET_KEY') }}"

group_vars/xxx.yml:

rgw_zone: "aaa"
rgw_zonemaster: False
rgw_zonesecondary: True
rgw_zonegroupmaster: False

rgw_pull_proto: "{{ rgw_multisite_proto }}"
rgw_pull_port: 443
rgw_pullhost: "ceph1.example.com"

cluster_network: "172.16.yy.0/24"

Environment:

• OpenBSD on Ansible controller, CentOS 7.7 on CEPH nodes.
• ansible-playbook 2.9.5
• ceph-ansible version 4-stable
• ceph version 14.2.8 (2d095e947a02261ce61424021bb43bd3022d35cb) nautilus (stable)

[alimaredia]

@dsavineau @p37ruh4: What if instead of radosgw_address in that we set a variable for each instance called rgw_multisite_host that defaults to the conditions @p37ruh4 laid out?

This way that value could be changed instance by instance.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.