CMS - NFS Not Mounted local disk full
CMS Disk getting full reason was daily backups where being stored loally as NFS mount did not automatically mount after a reboot.
[root@lab-cms /]# df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 9.8G 9.8G 0 100% /
tmpfs 32G 0 32G 0% /dev/shm
/dev/sda1 546M 38M 480M 8% /boot
/dev/sda3 9.8G 1.8G 7.5G 19% /cms
/dev/sda7 32G 6.1G 24G 21% /export/home
/dev/sda10 12G 759M 11G 7% /opt
/dev/sda6 87G 13G 70G 15% /storage
/dev/sda9 16G 45M 15G 1% /tmp
/dev/sda8 26G 297M 24G 2% /var
See output here, there is no nfs drive mounted.
I have helped to remount the drive like this:
[root@lab-cms /]# mount -a
[root@lab-cms /]# df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 9.8G 9.8G 0 100% /
tmpfs 32G 0 32G 0% /dev/shm
/dev/sda1 546M 38M 480M 8% /boot
/dev/sda3 9.8G 1.8G 7.5G 19% /cms
/dev/sda7 32G 6.1G 24G 21% /export/home
/dev/sda10 12G 759M 11G 7% /opt
/dev/sda6 87G 13G 70G 15% /storage
/dev/sda9 16G 45M 15G 1% /tmp
/dev/sda8 26G 8.6G 16G 36% /var
10.10.10.122:/CMS-Share
50G 35G 16G 70% /nfsbu
As the issue is marked as SBI, I have helped to move the backup files which is around 8GB from local partition to nfs location. And the whole procedure is not service impacting.
Now the partition issue is solved
[root@lab-cms var]# df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 9.8G 1.5G 7.8G 16% /
tmpfs 32G 0 32G 0% /dev/shm
/dev/sda1 546M 38M 480M 8% /boot
/dev/sda3 9.8G 1.8G 7.5G 19% /cms
/dev/sda7 32G 6.1G 24G 21% /export/home
/dev/sda10 12G 759M 11G 7% /opt
/dev/sda6 87G 13G 70G 15% /storage
/dev/sda9 16G 45M 15G 1% /tmp
/dev/sda8 26G 297M 24G 2% /var
10.10.10.122:/CMS-Share
50G 43G 7.2G 86% /nfsbu