Overview
I back up about 400 Gigabytes of photo RAW files and Lightroom (LR) catalogs to the cloud using CrashPlan. I used to have these files on a single hard drive inside my computer – dangerous!
I realised that I needed my photo files on a RAID array so that I don’t lose everything in case of a single disk failure. Instead of putting a RAID array inside my PC, or directly attaching a RAID array to it using USB or Thunderbolt (called a “DAS” for “directly attached”), I realised I didn’t need to spend money when I already have a perfectly good RAID box already – my NAS running FreeNAS!
I had a rude shock though when it came to backing up with CrashPlan running on my PC, and having my work files on a mapped network drive. CrashPlan refused to touch the files on the mapped network drive! I then to take the plunge and move the CrashPlan engine to my NAS, and do back ups from there. Brilliant!
About Running CrashPlan “Headless”
CrashPlan has two basic parts – the CrashPlan application, and the CrashPlan engine. The engine runs continuously and backs up even when the client isn’t running. The client just checks the engine status, and is used to configure it. The Client app is designed to connect to an engine on the local machine and not on a remote machine. Luckily it uses TCP ports, so we can hack the configuration in order to get it to connect to a remote (headless) machine.
Install the CrashPlan Plugin Jail
Setup you jail configuration, if you haven’t already. Mine is as follows:
Install the CrashPlan plugin jail by going to “Plugins > Available” and then highlighting “CrashPlan” and then clicking the “Install” button.
After it has installed, map the files you want to back up into the jail under “View Jails > Storage”. You’ll find detailed instructions on this on the FreeNAS documentation homepage.
My jail storage is as follows:
“volume1” is my raid array volume, and “lacie” is an external 12TB USB3.0 drive volume. I initially decided to use CrashPlan to back up all my files (software, multimedia and music) to my external drive, but I found that too slow. Now I just have a backup set to back up my “RAW files” and “Calalogs”, which are contained within my “/software/photography”, as I didn’t want to create another dataset just for those two.
It makes sense to only map your source files as read-only as I have done here – no need to give CrashPlan more permissions than it needs to do its job, and safeguards the files in case something goes drastically wrong. Mapped like this – you can only trash your backups, and not the source.
Update the Plugin Jail
I usually run the following for any new jail to get it up to date:
pkg clean # clean out old cache
pkg update # gets the latest list of files
pkg upgrade # updates the jail software
I also like to install bash with “pkg install bash” and then log out and back into the jail under bash:
sarlacc# jls
JID IP Address Hostname Path
1 - crashplan_1 /mnt/volume1/jails/crashplan_1
2 - dnsmasq /mnt/volume1/jails/dnsmasq
3 - plexmediaserver_1 /mnt/volume1/jails/plexmediaserver_1
4 - sabnzbd_1 /mnt/volume1/jails/sabnzbd_1
sarlacc# jexec 1 bash
[root@crashplan_1 /]#
Configure SSH in the Plugin Jail
You’ll need SSH in order to connect your PC to the CrashPlan engine running on the NAS. This is quite straightforward:
Edit “/etc/ssh/sshd_config" and uncomment/edit as follows:
PermitRootLogin yes
PasswordAuthentication yes
AllowTcpForwarding yes
For more security you can create another user such as “adduser crashplan” or “adduser backupuser” etc, but I don’t bother – I just use the root user and set a strong root password (in the jail) with “passwd root” command.
Next get sshd going:
sysrc sshd_enable=YES # allows sshd to be started as a service
service sshd keygen # generate sshd keys
service sshd start # start the sshd service
service sshd status # check sshd service status - should return the process ID
Update and Start CrashPlan Engine in Plugin Jail
The current problem we face is that the plugin is only version 3.6.3_1, and that’s way behind the exiting GUI version of 4.7, and there are compatibility issues. No problem – just manually update the jail:
su - # if not already root
cd /usr/pbi/crashplan-amd64/share/crashplan
wget --no-check-certificate https://download.code42.com/installs/linux/install/CrashPlan/CrashPlan_4.7.0_Linux.tgz
tar -xf CrashPlan_4.7.0_Linux.tgz
cd crashplan-install
cpio -idv < CrashPlan_4.7.0.cpi
service crashplan stop
cd ..
rm -r lib*
cp -r crashplan-install/lib* .
sysrc crashplan_enable=YES
The above assumes that 4.7 is the latest version, and that the crashplan TARGETDIR is “/usr/pbi/crashplan-amd64/share/crashplan“. Check the install vars here:
root@crashplan_1:/usr/pbi/crashplan-amd64/share/crashplan # cat install.vars
TARGETDIR=/usr/pbi/crashplan-amd64/share/crashplan
BINSDIR=/usr/pbi/crashplan-amd64/share/crashplan/bin
JAVACOMMON=/usr/pbi/crashplan-amd64/linux-sun-jre1.7.0/bin/java
APP_BASENAME=CrashPlan
DIR_BASENAME=crashplan
JRE_X64_DOWNLOAD_URL=http://download.code42.com/installs/proserver/jre/jre-7u45-linux-x64.tgz
JRE_I586_DOWNLOAD_URL=http://download.code42.com/installs/proserver/jre/jre-7u45-linux-i586.tgz
you may have to change “JAVACOMMON=/usr/pbi/crashplan-amd64/share/crashplan/jre/bin/java” to: “JAVACOMMON=/usr/pbi/crashplan-amd64/bin/java” if you get an error message in /var/log/crashplan/engine_error.log complaining about “libjli.so”
How I found the correct java:
[root@crashplan_1 /usr/pbi/crashplan-amd64/share/crashplan]# find / -name "java"
/usr/pbi/crashplan-amd64/linux-sun-jre1.7.0/bin/java
/usr/pbi/crashplan-amd64/share/java
/usr/pbi/crashplan-amd64/share/crashplan/jre/bin/java
/usr/pbi/crashplan-amd64/bin/java
/usr/pbi/crashplan-amd64/linux-sun-jre1.7.0/bin/java -version Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
/usr/pbi/crashplan-amd64/share/java -version directory
/usr/pbi/crashplan-amd64/share/crashplan/jre/bin/java -version - breaks with libjli.so issue
/usr/pbi/crashplan-amd64/bin/java -version (build 1.7.0_51-b13)
Go to Plugins > CrashPlan, in the left hand side tree menu in order to accept the Java licence agreement. This trips a lot of people up.
Now start Crashplan
service crashplan start
You can check that CrashPlan is running with the following:
root@crashplan_1:/mnt/lacie # sockstat -4 | grep java
root java 4859 88 tcp4 10.69.10.24:10303 103.8.239.9:443
root java 4859 105 tcp4 127.0.0.1:4243 *:*
root java 4859 108 tcp4 127.0.0.1:4243 127.0.0.1:12317
root java 4859 119 tcp4 10.69.10.24:56178 216.17.8.11:443
Line 2 is a connection to 103.8.239.9, which is Code42 Australia, where I am backing up some files to.
Line 3 is listening on the local server for new connections.
Line 4 is a an SSH port map from my Windows PC where I run the GUI. We’ll get to that.
Line 5 is a connection to 216.17.8.11 , which is Code42 (makers of CrashPlan) in the USA. Possibly a license server.
Don’t be alarmed when you see “crashplan is not running”, when issuing a “service crashplan status”. If Java is listening on the 4243 port then it should be fine. 🙂
Configure SSH in Windows
I use a program called SecureCRT to easily setup the portmap, connecting to my jail IP of 10.69.10.24, and CrashPlan port of 4243, using local Windows port of 4200:
It’s handy to create a save a session for this, and then create a desktop shortcut to the session, so you can just double-click the icon and start it. I like to start it minimised. The target for the shortcut for me is “C:\Program Files\VanDyke Software\SecureCRT\SecureCRT.exe" /S "CrashPlan“
NB: Check that 4243 is actually your CrashPlan engine port, with the “sockstat -4 | grep java” command above.
Connecting Windows CrashPlan Client to FreeNAS CrashPlan Engine
After setting up the portmap, we need to finish off by connecting the Windows Client to the FreeNAS server. To do this we need to update the following file: “C:\ProgramData\CrashPlan\.ui_info” changing port and API key.
e.g. from:
4255,736d39b8-5de2-4aa7-9ef9-67bc5d7177e5,127.0.0.1
to
4200,e047e36f-beef-42a4-babe-3b8eff05cafe,127.0.0.1
The format is <local port>,<api key>,<IP address>. Where is the API key you might ask? Answer – from the server’s .ui_info file. Run this on the FreeNAS box to check:
cat /var/lib/crashplan/.ui_info
You can double check the server’s port config there.
Once the Windows “.ui_info file” is saved, you should now be able to start the CrashPlan application on your PC and connect to the server.
The .ui_info file reverts back to the previous settings on every reboot of Windows. It is therefore important to create a .bat file to update this on reboot.
Windows Batch File
- Copy your newly configured “.ui_info” file to a new file called “freenas.ui_info” in the same directory.
- Create a file called “cpcfg.bat” (short for “crashplan configuration”) in the same directory, with the following contents:
copy C:\ProgramData\CrashPlan\FreeNAS.ui_info C:\ProgramData\CrashPlan\.ui_info
- Create a shortcut to that same cpcfg.bat file in the same folder.
- Once the shortcut has been created, right-click the file and select Cut.
- Press the WindowsKey+R to get to the “Run” dialog box.
- Type “shell:startup” in the Run dialog box and hit “OK”.
- Paste your “cpcfg.bat” shortcut into that folder.
- Right-click on the shortcut and go to “Properties > Shortcut (tab) > Advanced, and click “Run as Administrator”, and then OK, Apply, OK, to save.
Now everytime you reboot, that file will have the correct info. If that doesn’t work then you’ll have to just manually run the .bat file.
Troubleshooting and Tips
I did the following when I was troubleshooting, just following tips on forums, as you do. I’m not sure if they made my setup work or not, but if you have trouble, then it doesn’t hurt to try the following on the server, within the CrashPlan plugin jail:
ln -s /usr/local/bin/bash /bin/bash
/usr/bin/cpuset -l 0 /usr/local/share/crashplan/bin/CrashPlanEngine restart
In the GUI you can doubleclick on the CrashPlan “House” and logo on the top right and bring up the GUI CLI. Type “connection.info” and you should see something like this:
connection.info
Address=127.0.0.1
Port=4242
UI Port=4243
HTTP Port=4244
Address=127.0.0.1 and UI Port=4243 is correct if you’re mapping local port 4200 to server port 4243
CPU
Big tip here is to set the CPU usage to 100% (for user present and idle) in the FreeNAS GUI. This is because FreeNAS does CPU management for jails, and 100% within the jail means about 60% overall. The more CPU you throw at it, the better.
It does help to have a very grunty box when creating local backups at speed. I found that my speeds went up, the more I ramped CPU up to 100%, so it’s definitely CPU-bound. I get about 325Mbps (bits not bytes) to my external Lacie 12TB box over USB3.0 (5Gbps throughput). That would definitely go up with more CPU clock cycles.
Compression
There’s no need to compress your files within the jail, if you’ve already turned compression on at the dataset level. You’re just wasting your time and CPU otherwise. It is good to compress when going over the Internet though, so save your network bandwidth.
Credits
This draws heavily from these two links:
Using CrashPlan On A Headless Computer
FreeNAS Forums: CrashPlan 4.5 Setup
