Replacing a Failed Drive on a Gen8 HP Microserver Running FreeNAS

I recently had one of my freeNAS9.3 report the following issues:

  • CRITICAL: Device: /dev/ada2, 8 Currently unreadable (pending) sectors
  • CRITICAL: Device: /dev/ada2, 8 Offline uncorrectable sectors

unreadable sectors

After a bit of reading I decided it best I replace the drive, as I don’t want to take chances since I’m only running RAID-Z1 (for the space), instead of the preferred and safer RAID-Z2.

Then I hit a problem: What physical drive is ada2?  The Gen8 HP Microserver G2020T doesn’t have drive lights to indicate which ones are active.

What I did was took a bit of a pun that the drives are numbered left to right, and it proved correct.

What you really need to do before doing anything, is take a screenshot of the different drive serial #’s. Don’t rely on the drive numbers!!! The reason I say that is because once you pull out a drive, the drive numbers get remapped! When I pulled out ada2, what was previously ada3 became the new ada2! This can get confusing and cause you to pull the wrong drive and screw your data – so concentrate on the serial numbers.

I don’t need to repeat the full instructions, but will link you to them here: http://doc.freenas.org/9.3/freenas_storage.html#replacing-a-failed-drive

Here’s my screenshot of my drives. Note the serial of ada2.

Pre drive replace

I now offlined ada2 and shutdown FreeNAS in order to pull the drive out, as the drives in the Gen8 HP Microserver are apparently not hot-swappable (that’s something they really should address!). I booted up to make sure that the correct expected drive SERIAL NUMBER disappeared. Notice how what was ada3 is now ada2.

Post drive pull

Happy that I have pulled the right drive, I now proceed to shut down again, and then insert the new 4TB Seagate NAS drive. After booting up again:

New drive inserted

OK so far so good. I now highlighted the new ada2 as per the screenshot, and then clicked on “Replace”. I then had to confirm that I wanted to replace the original ada2, but I didn’t get a screenshot of that. It then went about reslivering, which is the process of recreating the data on the redundant drive.

resilvering

For me it got up to about 5% after 10 mins, so figured it would take somewhere between 3 and 4 hours, so kicked it off before I went to bed. The interesting thing is that the old drive’s volume ID (8482932750830730262) is still listed in the array during the resilvering process. It’s as if you can cancel the resilvering and go back to the original drive if you so wished (if you had a failed resilver perhaps?) but I didn’t test this theory. Once resilvering is complete, this old drive/volume reference goes away.

Hope this helps. Happy and safe NASing!

 

Update: 20 August 2015.  There was an a file in my /tmp directory called “.smartalert” which seemed to contain the source of the alert. I deleted that file and rebooted, and the alarms cleared.

 

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.