Versions used are: Avamar 19.2 vCenter 6.7
Going through my emailed reports from Avamar, I will see failed backups that need to be investigated. To do so I can go to the Avamar GUI and click on the red button next to “All Failures”
Once opened you may need to select the “Activity Monitor” tab at the top left. Here you will see a list of the previous nights backup and failures.
In this case we will want to view the logs for the GK-App server. To do this we will double click the highlighted blue line to view the log. The great thing about these logs are that the error is at the top allows you to click and jump to the event in the log.
Here we can quickly see that before the error occurred Avamar found “0 snapshots, and 2 snapshot files”
This show signs of an orphaned snapshot file, but we need to validate this. To do this we move over to vCenter and open “Snapshots” on the GK-App server. Notice how the “Revert to Latest Snaps” is grayed out which means the server believes it has snapshots to go back to.
Now we go into “Edit Settings…” to validate the mounted disk are not snapshots.
We expand the hard disks to view the “Disk File” portion. We are looking for leading zeros at the end of the name indicating that the hard disk is actually a snapshot disk. As we see here there is no zeros at the end before the .vmdk extension.
Now we know that the VM is has not snapshots and is not riding on a mounted snapshot image we now need to go into the storage to view the files associated with this VM server. We can see from the above “Disk File” name that the VM is riding on XIO_4 storage so we will start there. We will navigate to the storage and find the server in the list.
When we view the files for this server we can now see the vmdk files with the zeros at the end of its name. These are the typical look of a disk with a snapshot. We also see the vmdk files that are truly mounted to the VM server without the zeros
Now we could just go ahead and delete the zero vmdk files here, but if you have other storage to move the server to, I would do a vMotion of the server and the storage to isolate the files needing to be deleted even further. This will help to guaranty the uptime of the server as we don’t want to delete running files killing our server.
This time we will go to “Migrate..” to perform the vMotion.
We will select “Change both compute resource and storage”
Select a different host then it currently sits on.
Select a different storage.
Compatibility Checks are good.
Schedule vMotion with high priority.
Click “Finish” and watch in “Recent Tasks” for the results.
Now we go back to XIO_4 and see what was left behind. Here we can see that the only files left in the folder are the orphaned snapshot files.
We can either select the files and click delete and then remove the folder or just delete the folder.
or
Lastly go back and validate your server is still okay and your done! Your Avamar backup should run fine at next backup schedule.
