Encryption failing on Ubuntu 16.04
See original GitHub issueI’m building Azure Disk Encryption into an ARM template and have configured encryption of both OS disk and attached data disks within the template, to happen after VM provisioning and software installation using the newer Script VM Extension for Linux 2.0 used in Quickstart templates (https://github.com/Azure/azure-quickstart-templates/issues/2340).
The template deployment succeeds and reports successful, but the encryption operation is failing to encrypt the osDisk and the data disks are not encrypted (I guess the process fails on the osDisk and doesn’t get to the data disks).
Here’s a snippet of the extension log at /var/log/azure/Microsoft.Azure.Security.AzureDiskEncryptionForLinux/0.1.0.999283/extension.log
(I can provide the full one if needed)
2017/02/03 00:16:54 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: [StatusReport (0)] op: EnableEncryptionOSVolume
2017/02/03 00:16:54 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: [StatusReport (0)] status: error
2017/02/03 00:16:54 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: [StatusReport (0)] code: 19
2017/02/03 00:16:54 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: [StatusReport (0)] msg: Failed to encrypt OS volume with error: Attempt #1 to unmount /oldroot failed with error: Command umount /oldroot failed with return code 32
2017/02/03 00:16:54 stdout:
2017/02/03 00:16:54
2017/02/03 00:16:54 stderr:
2017/02/03 00:16:54 umount: /oldroot: target is busy
2017/02/03 00:16:54 (In some cases useful info about processes that
2017/02/03 00:16:54 use the device is found by lsof(8) or fuser(1).)
2017/02/03 00:16:54 , stack trace: Traceback (most recent call last):
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/oscrypto/ubuntu_1604/Ubuntu1604EncryptionStateMachine.py", line 166, in start_encryption
2017/02/03 00:16:54 self.enter_unmount_oldroot()
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 222, in trigger
2017/02/03 00:16:54 return self.machine.process(f)
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 526, in process
2017/02/03 00:16:54 return trigger()
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 247, in _trigger
2017/02/03 00:16:54 if t.execute(event):
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 148, in execute
2017/02/03 00:16:54 self._change_state(event_data)
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 159, in _change_state
2017/02/03 00:16:54 event_data.machine.get_state(self.dest).enter(event_data)
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 48, in enter
2017/02/03 00:16:54 event_data.machine.callback(oe, event_data)
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 518, in callback
2017/02/03 00:16:54 func(*event_data.args, **event_data.kwargs)
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/oscrypto/ubuntu_1604/Ubuntu1604EncryptionStateMachine.py", line 114, in on_enter_state
2017/02/03 00:16:54 super(Ubuntu1604EncryptionStateMachine, self).on_enter_state()
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/oscrypto/OSEncryptionStateMachine.py", line 65, in on_enter_state
2017/02/03 00:16:54 self.state_objs[self.state].enter()
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/oscrypto/ubuntu_1604/encryptstates/UnmountOldrootState.py", line 134, in enter
2017/02/03 00:16:54 self.command_executor.Execute('umount /oldroot', True)
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/CommandExecutor.py", line 70, in Execute
2017/02/03 00:16:54 raise Exception(msg)
2017/02/03 00:16:54 Exception: Command umount /oldroot failed with return code 32
2017/02/03 00:16:54 stdout:
2017/02/03 00:16:54
2017/02/03 00:16:54 stderr:
2017/02/03 00:16:54 umount: /oldroot: target is busy
2017/02/03 00:16:54 (In some cases useful info about processes that
2017/02/03 00:16:54 use the device is found by lsof(8) or fuser(1).)
2017/02/03 00:16:54
2017/02/03 00:16:54 , stack trace: Traceback (most recent call last):
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/handle.py", line 1522, in daemon_encrypt
2017/02/03 00:16:54 os_encryption.start_encryption()
2017/02/03 00:16:54 File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/oscrypto/ubuntu_1604/Ubuntu1604EncryptionStateMachine.py", line 184, in start_encryption
2017/02/03 00:16:54 raise Exception(message)
2017/02/03 00:16:54 Exception: Attempt #1 to unmount /oldroot failed with error: Command umount /oldroot failed with return code 32
2017/02/03 00:16:54 stdout:
2017/02/03 00:16:54
2017/02/03 00:16:54 stderr:
2017/02/03 00:16:54 umount: /oldroot: target is busy
2017/02/03 00:16:54 (In some cases useful info about processes that
2017/02/03 00:16:54 use the device is found by lsof(8) or fuser(1).)
Checking the encryption status through the Azure PowerShell SDK correlates with the problem in the log:
Get-AzureRmVmDiskEncryptionStatus -ResourceGroupName "encrypted-cluster" -VMName "data-0"
Get-AzureRmVmDiskEncryptionStatus : Long running operation failed with status 'Failed'.
ErrorCode: VMExtensionProvisioningError
ErrorMessage: VM has reported a failure when processing extension 'AzureDiskEncryptionForLinux'. Error message: "Failed to encrypt OS volume with error: Attempt #1 to unmount /oldroot failed with
error: Command umount /oldroot failed with return code 32
stdout:
stderr:
umount: /oldroot: target is busy
(In some cases useful info about processes that
use the device is found by lsof(8) or fuser(1).)
, stack trace: Traceback (most recent call last):
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/oscrypto/ubuntu_1604/Ubuntu1604EncryptionStateMachine.py", line 166, in start_encryption
self.enter_unmount_oldroot()
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 222, in trigger
return self.machine.process(f)
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 526, in process
return trigger()
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 247, in _trigger
if t.execute(event):
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 148, in execute
self._change_state(event_data)
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 159, in _change_state
event_data.machine.get_state(self.dest).enter(event_data)
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 48, in enter
event_data.machine.callback(oe, event_data)
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/transitions/transitions/core.py", line 518, in callback
func(*event_data.args, **event_data.kwargs)
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/oscrypto/ubuntu_1604/Ubuntu1604EncryptionStateMachine.py", line 114, in on_enter_state
super(Ubuntu1604EncryptionStateMachine, self).on_enter_state()
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/oscrypto/OSEncryptionStateMachine.py", line 65, in on_enter_state
self.state_objs[self.state].enter()
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/oscrypto/ubuntu_1604/encryptstates/UnmountOldrootState.py", line 134, in enter
self.command_executor.Execute('umount /oldroot', True)
File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999283/main/CommandExecutor.py", line 70, in Execute
raise Exception(msg)
Exception: Command umount /oldroot failed with return code 32
My understanding is that it is possible to encrypt the disks on a running VM based on the examples in the quickstart templates. The data disks are RAID0ed as part of the script that installs the software.
Should the encryption happen before the VM Script extension runs and software is deployed, or if it can happen after this, is this a bug in the encryption process?
Issue Analytics
- State:
- Created 7 years ago
- Reactions:1
- Comments:21
Top GitHub Comments
Sorry, then. I did it because, I could not wait for response in next few months (
On Fri, Jun 30, 2017 at 4:05 PM, Philipp Singer notifications@github.com wrote:
I installed DM-Crypt, 2 hours and problem is solved
On Fri, Jun 30, 2017 at 11:15 AM, Philipp Singer notifications@github.com wrote: