VM disk monitoring not working as expected

Description After enabling the workaround of #4791 something is working but it looks like the sent monitoring data from the remote hosts is not processed, despite the Successfully monitored VM: $VM_ID log message

To Reproduce install opennebula and a remote KVM host instantiate a VM in the default ssh SYSTEM datatore check the VM disk size

Expected behavior The VM disks size should be updated

Details

• Affected Component: [Monitring]
• Hypervisor: [KVM]
• Version: [development]

Additional context As far as I understand there are two ways to feed VM disk stats to OpenNebula:

1. on the frontend node - TM_MAD/monitor when called by for SYSTEM datastore which is not tagged for “at the host” monitoring (Cant twist my mind to name it “tagged for local monitoring”…)
2. On the hosts - TM_MAD/monitor_ds called by the remote monitoring probe

the monitoring message should be

VM = [ ID = $VM_ID, MONITOR = "<base64>
DISK_SIZE=[ID=$DISK_ID,SIZE=$DISK_SIZE]
SNAPSHOT_SIZE = [ID=$SNAP_ID, DISK_ID=$DISK_ID, SIZE=$SNAP_SIZE]
</base64>" ]

The message is logged as successfully monitored by monitors:

Fri May 22 10:41:02 2020 [Z0][HMM][I]: Successfully monitored VM: 3

But in the VMs the disks sizes are 0:

[root@one512 ssh]# onevm show 3
VIRTUAL MACHINE 3 INFORMATION                                                   
ID                  : 3                   
NAME                : Ttylinux - KVM-3    
USER                : oneadmin            
GROUP               : oneadmin            
STATE               : ACTIVE              
LCM_STATE           : RUNNING             
LOCK                : None                
RESCHED             : No                  
HOST                : vs04                
CLUSTER ID          : 0                   
CLUSTER             : default             
START TIME          : 05/20 16:15:55      
END TIME            : -                   
DEPLOY ID           : 6d8f347e-fbce-4d26-9df5-d455d6a2c11b

VIRTUAL MACHINE MONITORING                                                      
CPU                 : 1.01                
MEMORY              : 106.1M              
NETTX               : 0K                  
NETRX               : 0K                  
DISKRDBYTES         : 13855320            
DISKRDIOPS          : 974                 
DISKWRBYTES         : 596992              
DISKWRIOPS          : 303                 
ID                  : 3                   
TIMESTAMP           : 1590133292          

PERMISSIONS                                                                     
OWNER               : um-                 
GROUP               : ---                 
OTHER               : ---                 

VM DISKS                                                                        
 ID DATASTORE  TARGET IMAGE                               SIZE      TYPE SAVE
  0 default    vda    Ttylinux - KVM                      -/200M    file   NO
  2 system     sda    1024M                               -/1024M   swap   NO
  1 -          hda    CONTEXT                             -/-       -       -

patched the monitor_ds script to save the base64 encoded string to a temp file

    echo -e "$vm_monitor" | base64 -w 0 - | tee /tmp/monitor_ssh${vm}.b64

And there is the following data provided:

[root@vs04 tmp]# cat /tmp/monitor_ssh3.b64 | base64 -d 

DISK_SIZE = [ ID=0, SIZE=27]
DISK_SIZE = [ ID=1, SIZE=1]
DISK_SIZE = [ ID=2, SIZE=1]

Progress Status

• Branch created
• Code committed to development branch
• Testing - QA
• Documentation
• Release notes - resolved issues, compatibility, known issues
• Code committed to upstream release/hotfix branches
• Documentation committed to upstream release/hotfix branches

[paczerny]

@feldsam Documentation for the new monitoring: http://docs.opennebula.io/5.11/deployment/open_cloud_host_setup/monitoring.html#extending-the-monitor-system

@atodorov-storpool We will address your issues with nc and MONITOR_ADDRESS soon

[atodorov-storpool]

just fo reference, issue with vm disk monitoring was there until now on EL7 platforms. Fixed in this #5855

Still using our own implementation so missed/not affected/ by #5855