Monthly Archives: August 2013

HP Microserver Gen8 FreeNAS Tips and Tricks

Well I’ve been spending some time with the new HP Microserver Gen8 (G2020T), and wanted to post a few little tips and tricks for using it with FreeNAS

  1. Make the internal USB slot external
  2. Use MicroSD for swap space
  3. Bind CIFS to certain interfaces

Make the Internal USB slot External

With my older HP N40L NAS I was used to accessing the bootable internal USB by opening the front door. With the new NAS, you can’t get to the internal bootable USB without taking the case cover off. I had a spare USB extension cable so I put it to work as per the following photos:

I can now sit it on top of my NAS which gives me the following benefits:

  1. I can get to it easy. The rear USB2 are practically blocked by the two NIC ports I’m using, so I don’t have to reach around the back to get to them.
  2. It keeps the two front USB ports clear, so I don’t have a stick jutting out, and I can keep them free for if I need to plug in a mouse and keyboard.

Just so you know, you can boot from the external USB ports – the USB2 ones at least. I’m pretty sure I read somewhere that you can only boot from USB2 though – the USB3 ports are made available via OS drivers post-boot. It was quite easy to bend back the bit of metal and feed the plug through.  BTW – you can see my Intel 2-port server NIC there that I use to team two interfaces into my ESXi box for iSCSI disks straight off the NAS.

Use the MicroSD for the FreeBSD/FreeNAS OS Swap Space

This is my favorite tip. See the MicroSD card slot on the photo above? I thought about what use that could be for a dedicated NAS box, where I boot off the USB drive and use the HDD array for storage – surely it’s useless? Well I found a use for it that I’m quite happy with.

What I didn’t know is that by default, not all of your HDD space is used for data storage. In the default settings of FreeNAS (8 and now 9.1) is that it reserves 2GB per-disk for OS swap space.

Where it’s set is SETTINGS -> ADVANCED… “Swap size on each drive in GiB, affects new disks only. Setting this to 0 disables swap creation completely (STRONGLY DISCOURAGED).”

Here is a discussion on the matter.  If a disk fails and you’re using the swap space on it, that could leave your server inoperable. I don’t like that sound of that. What you can do is use the MicroSD for OS swap. This will then leave your HDD’s purely for ZFS data storage.

There are new MicroSD cards from Sandisk  (the creators of the format) that can transfer up to 80MB/s that should be adequate for emergency swap space. Mind you I say “emergency” swap. I am using my box as a NAS with 8GB and soon to be 16GB RAM – I don’t expect to be using swap at all, but like to have it for any emergency needs, which is preferable to the server crashing.  If you are smashing the swap space, then you’d need more RAM, and if you’re still smashing it, then perhaps get a dedicated SSD that can sit in the thin optical drive bay (there are third-party solutions for this).

To set this up this is what you do:-

Set swap size on each drive to “0” before you create any ZFS volumes! Otherwise you’ll have to blow those away and start again.

Put the MicroSD card in. I’m not sure if it’s hot swap, but since it’s a one-off task I do it cold. You can can either check the “dmesg” or via the GUI check that it shows up as a drive. On FreeNAS 9.1 it shows up on my server as “da1”.

On the CLI you type:

swapon /dev/da1

Check that it’s using it:

freenas# swapctl -l
Device:       1024-blocks     Used:
/dev/da1         992000         0

That’s a tiny 1GB MicroSD I pulled from a mobile phone as I didn’t have any others spare. I’ll put the order in for a 16GB Sandisk Extreme soon enough. I think it’s good form to be at least as big as the size of your RAM.
You also need to make sure it enables the swap on boot, so in the FreeNAS web GUI in “System” then under the pre-init scripts section, add a new pre-init script of:

swapon /dev/da1

If your ZFS volumes already have swap on them you might see something like this:

 freenas# swapctl -l
 Device:       1024-blocks     Used:
 /dev/ada0p1.eli   2097152         0
 /dev/ada1p1.eli   2097152         0
 /dev/ada2p1.eli   2097152         0

…because /etc/fstab will have these entries:

 freenas# cat /etc/fstab
 /dev/ufs/FreeNASs1a / ufs ro 1 1
 /dev/ufs/FreeNASs3 /cfg ufs rw,noauto 2 2
 /dev/ufs/FreeNASs4 /data ufs rw,noatime 2 2
 /dev/ada0p1.eli none                    swap            sw              0       0
 /dev/ada1p1.eli none                    swap            sw              0       0
 /dev/ada2p1.eli none                    swap            sw              0       0

You’ll need to build your volumes from scratch to get rid of that swap, unless you know how to hack this and extend the ZFS volume(s) to fill that extra space, which sounds problematic for live data. For me I copied the data off and rebuilt the volumes and datasets, just to be safe.

Bind CIFS to Only Certain Interfaces

With the extra NIC that I didn’t have before, I decided to use the extra port for “Out of Band Management”. This is handy for security so that only computers on the OOBM network can access the web GUI (if you pin the web GUI to a certain IP address). It’s good practice to have all your network devices managed this way where possible. I like to use the default VLAN for this, with general data on VLAN10 and voice traffic on VLAN20, and if I have a dedicated “SAN” (with jumbo frames usually), then I use VLAN30 for this.
Of course I could have set this up previously with VLAN interfaces which is fine from a security perspective, but I do like the idea of OOBM being on a totally separate physical interface for a pure OOBM setup.

You can either bind Samba to individual interfaces, or a range of interfaces by specifying a network. The latter is my preference as it requires less thought. The commands I  use are:

interfaces = 10.10.10.0/24
bind interfaces only = yes

Where you apply this is under “Services –> CIFS –> Auxiliary parameters

That subnet listed is what I have on my data VLAN10. It’s good for me as my PC has NICs on both Data and OOBM, and means I don’t accidentally mount Samba shares across the OOBM network. BTW that “interfaces” statement doesn’t have to match the subnet mask of your interface. You can just have written “10.0.0.0/8” which depending on your setup might match one or more interfaces. Otherwise you can specify interface names like “bge0”, or “em0” – i.e. the OS-given device name – not the name you gave it with the FreeNAS web GUI.

FreeNAS USB3.0 on the New HP Gen8 G2020T Microserver

Well the long-running saga of not having any working USB3.0 on my microserver NAS continues. In my previous post I said “It works perfectly”. Well when I dug a bit further I found that to not be the case. I assumed quite incorrectly that since the 4TB Seagate Expansion Desktop spun up, that it was working at USB3.0 speeds. Bad assumption.

To recap, I needed an external disk drive to my NAS so I can backup files from the NAS using ZFS snapshots that I can browse through with Windows VSS.

Here’s what happens when I plug into Renesas D720201-powered USB3 card:

 umass1: <vendor 0x0bc2 product 0x3312, class 0/0, rev 3.00/3.19, addr 1> on usbus0
 (probe0:umass-sim1:1:0:0): INQUIRY. CDB: 12 00 00 00 24 00
 (probe0:umass-sim1:1:0:0): CAM status: CCB request completed with an error
 (probe0:umass-sim1:1:0:0): Retrying command
 (probe0:umass-sim1:1:0:0): INQUIRY. CDB: 12 00 00 00 24 00
 (probe0:umass-sim1:1:0:0): CAM status: CCB request completed with an error
 (probe0:umass-sim1:1:0:0): Retrying command
 (probe0:umass-sim1:1:0:0): INQUIRY. CDB: 12 00 00 00 24 00
 (probe0:umass-sim1:1:0:0): CAM status: CCB request completed with an error
 (probe0:umass-sim1:1:0:0): Retrying command
 (probe0:umass-sim1:1:0:0): INQUIRY. CDB: 12 00 00 00 24 00
 (probe0:umass-sim1:1:0:0): CAM status: CCB request completed with an error
 (probe0:umass-sim1:1:0:0): Retrying command
 (probe0:umass-sim1:1:0:0): INQUIRY. CDB: 12 00 00 00 24 00
 (probe0:umass-sim1:1:0:0): CAM status: CCB request completed with an error
 (probe0:umass-sim1:1:0:0): Error 5, Retries exhausted
 ugen0.2: <vendor 0x0bc2> at usbus0 (disconnected)
 umass1: at uhub0, port 1, addr 1 (disconnected)

I also tried another card that has a newer Renesas D720202 chip – same deal. Here’s what happens plugging into USB2

 ugen2.2: <Seagate> at usbus2
 umass1: <Seagate Expansion Desk, class 0/0, rev 2.10/3.19, addr 2> on usbus2
 da1 at umass-sim1 bus 1 scbus7 target 0 lun 0
 da1: <Seagate Expansion Desk 0319> Fixed Direct Access SCSI-6 device
 da1: 40.000MB/s transfers
 da1: 3815447MB (976754645 4096 byte sectors: 255H 63S/T 60800C)
 da1: quirks=0x2<NO_6_BYTE>

The disk shows up fine, but of course is slow being USB2.

Not happy with that, I bit the bullet and bought the new HP G2020T Gen8 Microserver.  It was a big outlay but when an issue gets under my skin like this – I go all out! I was hoping that somehow it was the Renesas line of chipsets at fault not being properly supported in FreeBSD/FreeNAS. Surely the Intel native chipset will work?!

As an aside, when I was swapping disks over from my old NAS I dropped a disk and it was a write off. I ended up buying four new 4TB Seagate NAS drives as I’m a big fan of Seagate’s reliability.

Unfortunately, even with the newly-released FreeNAS 9.1, I still can’t get USB3.0 working with my Seagate external drive, even on the new Gen8 Microserver! I believe it’s an issue with the FreeBSD XHCI USB3.0 driver. I’ll have to run it at USB2.0 speeds for the time being and keep an eye on the errata for new versions of FreeNAS.

Update: in my new Gen8 Microserver with FreeNAS, I have a MicroSSD card and it shows up as “da1”, and the external USB2 drive shows up as “da2”. In the old NAS, the external USB2 drive shows up as “da1”. Be mindful of this if upgrading NAS and copying your settings across.