[BUG] WALA Provisioning Report Failure Hits WireServer Character Limits Causing WireServer HTTP 500 Return

Issue When WALA encounters a provisioning error, it reports not ready (reports failure) to WireServer through a health report XML. It includes the entire exception traceback in the health report XML’s Description field, which hits the WireServer 4096 char limit for the Description field. This causes WireServer to return HTTP 500.

Additional context code and code -> code -> code -> code -> code -> code -> code -> code -> code

The description field in the health report XML is hitting the 4096 char limit for this field, which is causing WireServer to return HTTP 500. The description field should be accordingly trimmed.

Notes and Suggestions

• The description XML subfield should be trimmed under a limit.
• Take note that the health report XML may be further mutated (and thus lengthened) when it is encoded in utf-8, so take this into account when trimming the length of the health report description.
• There may also be further permutations on the health report XML by the Host components along the way (such as escaping characters), so leaving a buffer is a good idea.

Distro and WALinuxAgent details (please complete the following information):

• Distro and Version: all distros and versions
• WALinuxAgent version: all versions

Logs

2020-09-17T23:42:26.434679Z VERBOSE Daemon Daemon HTTP connection [POST] [/machine?comp=health] [<?xml version="1.0" encoding="utf-8"?><Health xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><GoalStateIncarnation>17</GoalStateIncarnation><Container><ContainerId>4bcabc02-2806-4363-897c-fb73e5e05213</ContainerId><RoleInstanceList><Role><InstanceId>2a7b412d-2d44-46eb-a75d-99ed08ec8a65._pepldy00206</InstanceId><Health><State>NotReady</State><Details><SubStatus>ProvisioningFailed</SubStatus><Description>[ProtocolError] [CopyOvfEnv] Error mounting dvd: [OSUtilError] Failed to get dvd device from /dev
Inner error: The following devices were found, but none matched the pattern [(sr[0-9]|hd[c-z]|cdrom[0-9]|cd[0-9])]: ['.blkid.tab', '.blkid.tab.old', 'sdg1', 'guard_fsmon', 'guard_ktap5', 'guard_ktap4', 'guard_ktap3', 'guard_ktap2', 'guard_ktap1', 'guard_ktap', 'fsmon_106401', 'ktap5_106401', 'ktap4_106401', 'ktap3_106401', 'ktap2_106401', 'ktap1_106401', 'ktap_106401', 'vcsa6', 'vcs6', 'vcsa5', 'vcs5', 'vcsa4', 'vcs4', 'vcsa3', 'vcs3', 'vcsa2', 'vcs2', 'vcsa10', 'vcs10', 'xconsole', 'vgsybase', 'dm-251', 'dm-250', 'dm-249', 'dm-248', 'dm-247', 'dm-246', 'dm-245', 'dm-244', 'dm-243', 'dm-242', 'dm-241', 'dm-240', 'dm-239', 'dm-238', 'dm-237', 'dm-236', 'dm-235', 'dm-234', 'dm-233', 'dm-232', 'dm-231', 'dm-230', 'dm-229', 'dm-228', 'dm-227', 'dm-226', 'dm-225', 'dm-224', 'dm-223', 'dm-222', 'dm-221', 'dm-220', 'dm-219', 'dm-218', 'dm-217', 'dm-216', 'dm-215', 'dm-214', 'dm-213', 'dm-212', 'dm-211', 'vgsybdbAP', 'dm-210', 'dm-209', 'dm-208', 'dm-207', 'dm-206', 'dm-205', 'dm-204', 'dm-203', 'dm-202', 'dm-201', 'dm-200', 'dm-199', 'dm-198', 'dm-197', 'dm-196', 'dm-195', 'dm-194', 'dm-193', 'dm-192', 'dm-191', 'dm-190', 'dm-189', 'dm-188', 'dm-187', 'dm-186', 'dm-185', 'dm-184', 'dm-183', 'dm-182', 'dm-181', 'dm-180', 'dm-179', 'dm-178', 'dm-177', 'dm-176', 'dm-175', 'dm-174', 'dm-173', 'dm-172', 'dm-171', 'dm-170', 'dm-169', 'dm-168', 'dm-167', 'dm-166', 'dm-165', 'dm-164', 'dm-163', 'dm-162', 'dm-161', 'dm-160', 'dm-159', 'dm-158', 'dm-157', 'dm-156', 'dm-155', 'dm-154', 'dm-153', 'dm-152', 'dm-151', 'dm-150', 'dm-149', 'dm-148', 'dm-147', 'dm-146', 'dm-145', 'dm-144', 'dm-143', 'dm-142', 'dm-141', 'dm-140', 'dm-139', 'dm-138', 'dm-137', 'dm-136', 'dm-135', 'dm-134', 'dm-133', 'dm-132', 'dm-131', 'dm-130', 'dm-129', 'dm-128', 'dm-127', 'dm-126', 'dm-125', 'dm-124', 'dm-123', 'dm-122', 'dm-121', 'dm-120', 'dm-119', 'dm-118', 'dm-117', 'dm-116', 'dm-115', 'dm-114', 'dm-113', 'dm-112', 'dm-111', 'dm-110', 'dm-109', 'dm-108', 'dm-107', 'dm-106', 'dm-105', 'dm-104', 'dm-103', 'dm-102', 'dm-101', 'dm-100', 'dm-99', 'dm-98', 'dm-97', 'dm-96', 'dm-95', 'dm-94', 'dm-93', 'dm-92', 'dm-91', 'dm-90', 'dm-89', 'dm-88', 'dm-87', 'dm-86', 'dm-85', 'dm-84', 'dm-83', 'dm-82', 'dm-81', 'dm-80', 'dm-79', 'dm-78', 'dm-77', 'dm-76', 'dm-75', 'dm-74', 'dm-73', 'dm-72', 'dm-71', 'dm-70', 'dm-69', 'dm-68', 'dm-67', 'dm-66', 'dm-65', 'dm-64', 'dm-63', 'dm-62', 'dm-61', 'dm-60', 'dm-59', 'dm-58', 'dm-57', 'dm-56', 'dm-55', 'dm-54', 'dm-53', 'dm-52', 'dm-51', 'dm-50', 'dm-49', 'dm-48', 'dm-47', 'dm-46', 'dm-45', 'dm-44', 'dm-43', 'dm-42', 'dm-41', 'dm-40', 'dm-39', 'dm-38', 'dm-37', 'dm-36', 'dm-35', 'dm-34', 'dm-33', 'dm-32', 'dm-31', 'dm-30', 'dm-29', 'dm-28', 'dm-27', 'dm-26', 'dm-25', 'dm-24', 'dm-23', 'dm-22', 'dm-21', 'dm-20', 'dm-19', 'dm-18', 'dm-17', 'dm-16', 'dm-15', 'dm-14', 'dm-13', 'dm-12', 'dm-11', 'dm-10', 'dm-9', 'dm-8', 'kvm', 'dm-7', 'dm-6', 'dm-5', 'dm-4', 'dm-3', 'root', 'ptp0', 'vmbus', 'snd', 'vhost-net', 'vhci', 'uinput', 'ppp', 'net', 'btrfs-control', 'fuse', 'hugepages', 'initctl', 'mqueue', 'dm-2', 'dm-1', 'vg00', 'dm-0', 'disk', 'sdg', 'sg6', 'sdf', 'sde', 'sg5', 'sdd', 'sg4', 'sdc', 'sg3', 'sdb', 'sg2', 'sg1', 'sda5', 'sda2', 'sda1', 'sda', 'bsg', 'sg0', 'block', 'fd0', 'fb0', 'rtc', 'vcsa7', 'vcs7', 'char', 'vfio', 'uhid', 'mapper', 'loop-control', 'log', 'stderr', 'stdout', 'stdin', 'fd', 'core', 'pts', 'shm', 'autofs', 'memory_bandwidth', 'network_throughput', 'network_latency', 'cpu_dma_latency', 'mcelog', 'rtc0', 'psaux', 'input', 'lightnvm', 'hwrng', 'nvram', 'hpet', 'ttyS31', 'ttyS30', 'ttyS29', 'ttyS28', 'ttyS27', 'ttyS26', 'ttyS25', 'ttyS24', 'ttyS23', 'ttyS22', 'ttyS21', 'ttyS20', 'ttyS19', 'ttyS18', 'ttyS17', 'ttyS16', 'ttyS15', 'ttyS14', 'ttyS13', 'ttyS12', 'ttyS11', 'ttyS10', 'ttyS9', 'ttyS8', 'ttyS7', 'ttyS6', 'ttyS5', 'ttyS4', 'ttyS3', 'ttyS2', 'ttyS1', 'ttyS0', 'ptmx', 'snapshot', 'tty63', 'tty62', 'tty61', 'tty60', 'tty59', 'tty58', 'tty57', 'tty56', 'tty55', 'tty54', 'tty53', 'tty52', 'tty51', 'tty50', 'tty49', 'tty48', 'tty47', 'tty46', 'tty45', 'tty44', 'tty43', 'tty42', 'tty41', 'tty40', 'tty39', 'tty38', 'tty37', 'tty36', 'tty35', 'tty34', 'tty33', 'tty32', 'tty31', 'tty30', 'tty29', 'tty28', 'tty27', 'tty26', 'tty25', 'tty24', 'tty23', 'tty22', 'tty21', 'tty20', 'tty19', 'tty18', 'tty17', 'tty16', 'tty15', 'tty14', 'tty13', 'tty12', 'tty11', 'tty10', 'tty9', 'tty8', 'tty7', 'tty6', 'tty5', 'tty4', 'tty3', 'tty2', 'tty1', 'vcsa1', 'vcs1', 'vcsa', 'vcs', 'tty0', 'console', 'tty', 'kmsg', 'urandom', 'random', 'full', 'zero', 'port', 'null', 'kmem', 'mem', 'vga_arbiter']
</Description></Details></Health></Role></RoleInstanceList></Container></Health>] [{'Content-Type': 'text/xml;charset=utf-8', 'x-ms-version': '2012-11-30', 'x-ms-agent-name': 'WALinuxAgent', 'Connection': 'close', 'User-Agent': 'WALinuxAgent/2.2.47'}]
2020-09-17T23:42:26.618119Z VERBOSE Daemon Daemon [HTTP Response] Status Code 500
2020-09-17T23:42:26.620405Z VERBOSE Daemon Daemon [HTTP Retry] Attempt 13 of 30 will delay 5655 seconds: [HTTP Retry] POST http://168.63.129.16/machine?comp=health -- Status Code 500

[narrieta]

@trstringer - could you take a look at this one? thanks!

[trstringer]

This issue is fixed in #2020.