When this problem occurs, the Azure portal shows that the VM is running. However, you cannot access the VM through a secure shell (SSH) connection.
Note We highly recommend that you make a backup of the virtual hard disk (VHD) from the inaccessible VM before you follow the recovery steps. You can stop the VM and back up the VHD by using Microsoft Storage Explorer.
For troubleshooting information, see the serial log output in the boot diagnostics folder in the following portal path:
Virtual Machines > [VMNAME] > All settings > Boot diagnostics
The log content may resemble the examples in the More Information section.
Recover the VM
To recover the VM, follow these steps:
- Attach the OS disk to a recovery VM. To do this, go to the following Azure article, and follow the steps from the beginning up to the “Fix issues on original virtual hard disk” section:
- After you mount the OS disk as a data disk on the new VM, you must correct the fstab file. To do this, follow these steps.
Note We recommend that you back up the fstab file before you make changes.
- Open fstab in your favorite text editor. The following example command for Nano assumes that the disk is mounted per the steps in the Azure troubleshooting article from step 1.
$ nano /mnt/troubleshootingdisk/etc/fstabImportant Make sure that you are working on the fstab file that is located on the mounted disk and not the fstab file of the rescue VM itself.
- Review the listed file systems. Each line in fstab represents a file system that is mounted when the VM starts. For more information about the syntax of the fstab file, run the man fstab command. To troubleshoot a failure to start, review each line to make sure that it is correct in both structure and content.
- Fields on each line are separated by tabs or spaces. Blank lines are ignored. Lines that have a number sign (#) as the first character are comments. Commented lines can remain in the fstab but will not be processed. It is a best practice to comment fstab lines that you are unsure about instead of removing the lines.
- For the VM to recover and start, the OS file system partitions should be the only required partitions. The VM may experience application errors about additional commented partitions. However, the VM should start without the additional partitions. You can later uncomment any commented lines.
- It is a best practice to mount data disks on VMs in Azure by using the universally unique identifier (UUID) of the file system partition. For example, run the following command:
/dev/sdc1: LABEL="cloudimg-rootfs" UUID="b0348738-4ecc-4837-9793-49ce236d2692" TYPE="ext4" PARTUUID="000ae840-01"To determine the UUID of the file system, use the blkid command. (Use man blkid for more information.)
- Be aware that the disk that you want to recover is now mounted on a new VM. Although the UUIDs should be consistent, the device partition IDs (for example, "/dev/sda1") are different on this VM. The file system partitions of the original, failing VM that are located on a non-OS VHD are not available to the recovery VM by using the steps in the Azure troubleshooting article.
- The nofail option helps to make sure that the VM starts even if the file system is corrupted or the file system does not exist at startup. It is a best practice to use the nofail option in fstab to enable startup to continue after errors occur in partitions that are not required for the VM to start.
- Revisit any of the fstab lines that were changed or commented out during recovery.
- Make sure that you are using UUID and nofail appropriately.
- Test any fstab changes before you restart the VM. To do this, use the following command:
$ sudo mount -a
- Create an additional copy of the corrected fstab file for use in future recovery scenarios.
Examples of log messages in the boot diagnostics folder
Example 2: An unattached device is missing on CentOS
Example 3: A VM cannot start because of an fstab misconfiguration or because the disk is no longer attached
Example 4: A serial log entry shows an incorrect UUID
Article ID: 3206699 - Last Review: Mar 8, 2017 - Revision: 21