You cannot connect to Azure Linux VM by using Secure Shell (SSH). When you check the Boot Diagnostics on Azure portal, you see the following log messages:
Checking all file systems.[/sbin/fsck.ext4 (1) — /] fsck.ext4 -a /dev/sda1/dev/sda1 contains a file system with errors, check forced./dev/sda1: Inodes that were part of a corrupted orphan linked list found./dev/sda1: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY
EXT4-fs (sda1): INFO: recovery required on readonly filesystemEXT4-fs (sda1): write access will be enabled during recoveryEXT4-fs warning (device sda1): ext4_clear_journal_err:4531: Filesystem error recorded from previous mount: IO failureEXT4-fs warning (device sda1): ext4_clear_journal_err:4532: Making fs in need of filesystem check.
[ 14.252404] EXT4-fs (sda1): Couldn’t remount RDWR because of unprocessed orphan inode list. Please unmount/remount insteadAn error occurred while mounting /.
This one in particular is the result of a clean fsck. In this specific case there is also additional data disks attached to the VM (/dev/sdc1 and /dev/sde1) Checking all file systems.
Checking all file systems. [/sbin/fsck.ext4 (1) — /] fsck.ext4 -a /dev/sda1/dev/sda1: clean, 65405/1905008 files, 732749/7608064 blocks[/sbin/fsck.ext4 (1) — /tmp] fsck.ext4 -a /dev/sdc1[/sbin/fsck.ext4 (2) — /backup] fsck.ext4 -a /dev/sde1/dev/sdc1: clean, 12/1048576 files, 109842/4192957 blocks/dev/sde1 : clean, 51/67043328 files, 4259482/268173037 blocks
This issue may occur if the file system was not shut down cleanly.
Note We highly recommend that you make a backup of the virtual hard disk (VHD) from the inaccessible VM before you follow the recovery steps. You can stop the VM and back up the VHD by using Microsoft Storage Explorer.
To recover the VM, follow these steps:
- Attach the OS disk to a recovery VM. To do this, go to the following Azure article, and follow the steps from the beginning up to the “Fix issues on original virtual hard disk” section:
- After you mount the OS disk as a data disk on the new VM, locate the drive name of the OS disk you mounted, and then check relevant log file. Note that each Linux is slightly different.
grep SCSI /var/log/kern.log
CentOS, SUSE, Oracle, Red Hat
grep SCSI /var/log/messages
- Identify the partition that we need to run the disk check.
sudo -ifdisk -ldf -h
Sample outputs from both commands:
Disk /dev/sdc: 32.2 GB, 32212254720 bytes<255 heads, 63 sectors/track, 3916 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x000c23d3Device Boot Start End Blocks Id System/dev/sdc1 * 1 3789 30432256 83 Linux/dev/sdc2 3789 3917 1024000 82 Linux swap / Solarisdf -hFilesystem Size Used Avail Use% Mounted on/dev/sda1 29G 2.2G 25G 9% /tmpfs 776M 0 776M 0% /dev/shm/dev/sdb1 69G 180M 66G 1% /mnt/resource
In the output, we can determine that sda1 and sdb1 are mounted as part of the local OS, sdc1 is mounted. So, in this case, we need to run fsck against /dev/sdc1.
Run fsck on the partition which cannot be mounted.
fsck -yM /dev/sdc1
The following is a sample of the result after you run the command:
fsck from util-linux-ng 2.17.2e2fsck 1.41.12 (17-May-2010)/dev/sdc1: clean, 57029/1905008 files, 672768/7608064 blocks
Unmount and detach original virtual hard disk, and create VM from original OS disk.
Article ID: 3213321 - Last Review: 11 Jun 2017 - Revision: 21