Category Archives: Media

Setting up a Crashplan FreeNAS Plugin Jail

Overview

I back up about 400 Gigabytes of photo RAW files and Lightroom (LR) catalogs to the cloud using CrashPlan. I used to have these files on a single hard drive inside my computer – dangerous!

I realised that I needed my photo files on a RAID array so that I don’t lose everything in case of a single disk failure. Instead of putting a RAID array inside my PC, or directly attaching a RAID array to it using USB or Thunderbolt (called a “DAS” for “directly attached”), I realised I didn’t need to spend money when I already have a perfectly good RAID box already – my NAS running FreeNAS!

I had a rude shock though when it came to backing up with CrashPlan running on my PC, and having my work files on a mapped network drive. CrashPlan refused to touch the files on the mapped network drive! I then to take the plunge and move the CrashPlan engine to my NAS, and do back ups from there. Brilliant!

About Running CrashPlan “Headless”

CrashPlan has two basic parts – the CrashPlan application, and the CrashPlan engine. The engine runs continuously and backs up even when the client isn’t running. The client just checks the engine status, and is used to configure it. The Client app is designed to connect to an engine on the local machine and not on a remote machine. Luckily it uses TCP ports, so we can hack the configuration in order to get it to connect to a remote (headless) machine.

Install the CrashPlan Plugin Jail

Setup you jail configuration, if you haven’t already. Mine is as follows:

jail configuration

Install the CrashPlan plugin jail by going to “Plugins > Available” and then highlighting “CrashPlan” and then clicking the “Install” button.

After it has installed, map the files you want to back up into the jail under “View Jails > Storage”. You’ll find detailed instructions on this on the FreeNAS documentation homepage.

My jail storage is as follows:

jail storage

“volume1” is my raid array volume, and “lacie” is an external 12TB USB3.0 drive volume. I initially decided to use CrashPlan to back up all my files (software, multimedia and music) to my external drive, but I found that too slow. Now I just have a backup set to back up my “RAW files” and “Calalogs”, which are contained within my “/software/photography”, as I didn’t want to create another dataset just for those two.

It makes sense to only map your source files as read-only as I have done here – no need to give CrashPlan more permissions than it needs to do its job, and safeguards the files in case something goes drastically wrong. Mapped like this – you can only trash your backups, and not the source.

Update the Plugin Jail

I usually run the following for any new jail to get it up to date:

pkg clean     # clean out old cache
pkg update    # gets the latest list of files
pkg upgrade   # updates the jail software

I also like to install bash with “pkg install bash” and then log out and back into the jail under bash:

sarlacc# jls
 JID IP Address Hostname Path
 1 - crashplan_1 /mnt/volume1/jails/crashplan_1
 2 - dnsmasq /mnt/volume1/jails/dnsmasq
 3 - plexmediaserver_1 /mnt/volume1/jails/plexmediaserver_1
 4 - sabnzbd_1 /mnt/volume1/jails/sabnzbd_1
sarlacc# jexec 1 bash
[root@crashplan_1 /]#

Configure SSH in the Plugin Jail

You’ll need SSH in order to connect your PC to the CrashPlan engine running on the NAS. This is quite straightforward:

Edit “/etc/ssh/sshd_config" and uncomment/edit as follows:

PermitRootLogin yes
PasswordAuthentication yes
AllowTcpForwarding yes

For more security you can create another user such as “adduser crashplan” or “adduser backupuser” etc, but I don’t bother – I just use the root user and set a strong root password (in the jail) with “passwd root” command.

Next get sshd going:

sysrc sshd_enable=YES  # allows sshd to be started as a service
service sshd keygen    # generate sshd keys
service sshd start     # start the sshd service
service sshd status    # check sshd service status - should return the process ID

Update and Start CrashPlan Engine in Plugin Jail

The current problem we face is that the plugin is only version 3.6.3_1, and that’s way behind the exiting GUI version of 4.7, and there are compatibility issues. No problem – just manually update the jail:

su -                    # if not already root
cd /usr/pbi/crashplan-amd64/share/crashplan
wget --no-check-certificate https://download.code42.com/installs/linux/install/CrashPlan/CrashPlan_4.7.0_Linux.tgz
tar -xf CrashPlan_4.7.0_Linux.tgz
cd crashplan-install
cpio -idv < CrashPlan_4.7.0.cpi
service crashplan stop
cd ..
rm -r lib*
cp -r crashplan-install/lib* .
sysrc crashplan_enable=YES

The above assumes that 4.7 is the latest version, and that the crashplan TARGETDIR is “/usr/pbi/crashplan-amd64/share/crashplan“. Check the install vars here:

root@crashplan_1:/usr/pbi/crashplan-amd64/share/crashplan # cat install.vars 
TARGETDIR=/usr/pbi/crashplan-amd64/share/crashplan
BINSDIR=/usr/pbi/crashplan-amd64/share/crashplan/bin
JAVACOMMON=/usr/pbi/crashplan-amd64/linux-sun-jre1.7.0/bin/java
APP_BASENAME=CrashPlan
DIR_BASENAME=crashplan
JRE_X64_DOWNLOAD_URL=http://download.code42.com/installs/proserver/jre/jre-7u45-linux-x64.tgz
JRE_I586_DOWNLOAD_URL=http://download.code42.com/installs/proserver/jre/jre-7u45-linux-i586.tgz

you may have to change “JAVACOMMON=/usr/pbi/crashplan-amd64/share/crashplan/jre/bin/java” to: “JAVACOMMON=/usr/pbi/crashplan-amd64/bin/java” if you get an error message in /var/log/crashplan/engine_error.log complaining about “libjli.so”

How I found the correct java:

[root@crashplan_1 /usr/pbi/crashplan-amd64/share/crashplan]# find / -name "java"
/usr/pbi/crashplan-amd64/linux-sun-jre1.7.0/bin/java
/usr/pbi/crashplan-amd64/share/java
/usr/pbi/crashplan-amd64/share/crashplan/jre/bin/java
/usr/pbi/crashplan-amd64/bin/java

/usr/pbi/crashplan-amd64/linux-sun-jre1.7.0/bin/java -version   Java(TM) SE Runtime Environment (build 1.7.0_51-b13) 
/usr/pbi/crashplan-amd64/share/java -version  directory
/usr/pbi/crashplan-amd64/share/crashplan/jre/bin/java -version   - breaks with libjli.so issue
/usr/pbi/crashplan-amd64/bin/java -version   (build 1.7.0_51-b13)

Go to Plugins > CrashPlan, in the left hand side tree menu in order to accept the Java licence agreement. This trips a lot of people up.

Now start Crashplan

 service crashplan start

You can check that CrashPlan is running with the following:

root@crashplan_1:/mnt/lacie # sockstat -4 | grep java
root java 4859 88 tcp4 10.69.10.24:10303 103.8.239.9:443
root java 4859 105 tcp4 127.0.0.1:4243 *:*
root java 4859 108 tcp4 127.0.0.1:4243 127.0.0.1:12317
root java 4859 119 tcp4 10.69.10.24:56178 216.17.8.11:443

Line 2 is a connection to 103.8.239.9, which is Code42 Australia, where I am backing up some files to.
Line 3 is listening on the local server for new connections.
Line 4 is a an SSH port map from my Windows PC where I run the GUI. We’ll get to that.
Line 5 is a connection to  216.17.8.11 , which is Code42 (makers of CrashPlan) in the USA. Possibly a license server.

Don’t be alarmed when you see “crashplan is not running”, when issuing a “service crashplan status”. If Java is listening on the 4243 port then it should be fine. 🙂

Configure SSH in Windows

I use a program called SecureCRT to easily setup the portmap, connecting to my jail IP of 10.69.10.24, and CrashPlan port of 4243, using local Windows port of 4200:

Crashplan port forward secure CRT with crashplan settings

It’s handy to create a save a session for this, and then create a desktop shortcut to the session, so you can just double-click the icon and start it. I like to start it minimised. The target for the shortcut for me is “C:\Program Files\VanDyke Software\SecureCRT\SecureCRT.exe" /S "CrashPlan

NB: Check that 4243 is actually your CrashPlan engine port, with the “sockstat -4 | grep java” command above.

Connecting Windows CrashPlan Client to FreeNAS CrashPlan Engine

After setting up the portmap, we need to finish off by connecting the Windows Client to the FreeNAS server. To do this we need to update the following file:  “C:\ProgramData\CrashPlan\.ui_info” changing port and API key.

e.g. from:

4255,736d39b8-5de2-4aa7-9ef9-67bc5d7177e5,127.0.0.1

to

4200,e047e36f-beef-42a4-babe-3b8eff05cafe,127.0.0.1

The format is <local port>,<api key>,<IP address>. Where is the API key you might ask? Answer – from the server’s .ui_info file. Run this on the FreeNAS box to check:

cat /var/lib/crashplan/.ui_info

You can double check the server’s port config there.

Once the Windows “.ui_info file” is saved, you should now be able to start the CrashPlan application on your PC and connect to the server.

The .ui_info file reverts back to the previous settings on every reboot of Windows. It is therefore important to create a .bat file to update this on reboot.

Windows Batch File

  1. Copy your newly configured “.ui_info” file to a new file called “freenas.ui_info” in the same directory.
  2. Create a file called “cpcfg.bat” (short for “crashplan configuration”) in the same directory, with the following contents:
    copy C:\ProgramData\CrashPlan\FreeNAS.ui_info C:\ProgramData\CrashPlan\.ui_info
  3. Create a shortcut to that same cpcfg.bat file in the same folder.
  4. Once the shortcut has been created, right-click the file and select Cut.
  5. Press the WindowsKey+R to get to the “Run” dialog box.
  6. Type “shell:startup” in the Run dialog box and hit “OK”.
  7. Paste your “cpcfg.bat” shortcut into that folder.
  8. Right-click on the shortcut and go to “Properties > Shortcut (tab) > Advanced, and click “Run as Administrator”, and then OK, Apply, OK, to save.

Now everytime you reboot, that file will have the correct info. If that doesn’t work then you’ll have to just manually run the .bat file.

Troubleshooting and Tips

I did the following when I was troubleshooting, just following tips on forums, as you do. I’m not sure if they made my setup work or not, but if you have trouble, then it doesn’t hurt to try the following on the server, within the CrashPlan plugin jail:

ln -s /usr/local/bin/bash /bin/bash
/usr/bin/cpuset -l 0 /usr/local/share/crashplan/bin/CrashPlanEngine restart

In the GUI you can doubleclick on the CrashPlan “House” and logo on the top right and bring up the GUI CLI. Type “connection.info” and you should see something like this:

connection.info 

Address=127.0.0.1
Port=4242
UI Port=4243
HTTP Port=4244

Address=127.0.0.1 and UI Port=4243 is correct if you’re mapping local port 4200 to server port 4243

CPU

Big tip here is to set the CPU usage to 100% (for user present and idle) in the FreeNAS GUI. This is because FreeNAS does CPU management for jails, and 100% within the jail means about 60% overall. The more CPU you throw at it, the better.

It does help to have a very grunty box when creating local backups at speed. I found that my speeds went up, the more I ramped CPU up to 100%, so it’s definitely CPU-bound. I get about 325Mbps  (bits not bytes) to my external Lacie 12TB box over USB3.0 (5Gbps throughput). That would definitely go up with more CPU clock cycles.

Compression

There’s no need to compress your files within the jail, if you’ve already turned compression on at the dataset level. You’re just wasting your time and CPU otherwise. It is good to compress when going over the Internet though, so save your network bandwidth.

Credits

This draws heavily from these two links:
Using CrashPlan On A Headless Computer
FreeNAS Forums: CrashPlan 4.5 Setup

 

Getting Tvheadend Picons to Work in Plex

Picons are handy if you are using the Plex Tvheadend channel and you want the TV station icons to show up.

I was able to pull down all the Australian TV icons online from Beyonwiz (this is for my Ubuntu Linux Tvheadend server):

apt-get install git
cd /usr/src
git clone https://bitbucket.org/beyonwiz/picons-australia.git

 

Then you can set the Tvheadend (TVH) server “Configuration -> General” tab to prefer picons over channel name, and set the path to file:///usr/src/picons-australia/picon

The other thing you need to do is under “Configuration -> Access Entries”, add a new entry with the following:

Enabled: tick
Username: *
Password: *
Network prefix:  The IP address of your Plex server or Kodi player, or even local subnet if you want. e.g. "10.1.0.45/32" or "10.1.0.0/24" (or "127.0.0.1/32" if Plex and TVH are on the same box).
Streaming: tick

That should be enough access to get the icons working.

When I get time I’ll see if I can feed Kodi the picons in a similar way, as I prefer this server-side method of delivering TV channel icons, rather than client-side. For now I just point Kodi to a local directory with PNG images named the same as the channel names, which seems to work fine.

The only issue with my current picon set-up for the Plex Tvheadend Channel, and it is a minor one, is that the picons get truncated on my iPhone as they aren’t square format. They do look look fine on the PC though. I’m tossing up whether it’s worth my time to create square icons for the 22 stations I make use of in Melbourne Australia.

Other than that, I’m pretty stoked with being able to get the icons/picons to display!

Installing Tvheadend on Ubuntu

Update April 2016: You don’t have to build your own Ubuntu packages any more, as they are maintained here.

I’ve chosen to install Tvheadend (TVH) on a vanilla Ubuntu Server 14.04.2 installation. Incidentally I have Ubuntu (64-bit version) setup on a ESXi 6.0 host. Here’s how you can do it to:

Install Ubuntu Server 14.04.2

I recommend installing these at install time:

ssh server
samba server

Otherwise don’t install them straight up, but later once in the CLI you can do this:

apt-get install ssh
apt-get install samba

If the timezone is somehow messed up you can:

dpkg-reconfigure tzdata

In Ubuntu, you don’t log on as root or set the root password, but rather sudo -i, which will get you root privs.

Finalise the install:

sudo -i
rm -rf /var/lib/apt/lists/*
apt-get update
apt-get upgrade
reboot

Install Required Libraries

apt-get install build-essential git pkg-config libssl-dev bzip2 wget
apt-get install libavahi-client-dev zlib1g-dev libavcodec-dev
apt-get install libavutil-dev libavformat-dev libswscale-dev
apt-get install libcurl4-gnutls-dev liburiparser-dev
apt-get install debhelper

Install Tvheadend

Go to your building area:

cd /usr/src/

Get a snapshot of the latest TVH:

git clone https://github.com/tvheadend/tvheadend.git

This will install the latest development code branch. At the moment this is 4.1. If you want to install the stable 4.0 branch then try this:

git clone --branch release/4.0 https://github.com/tvheadend/tvheadend.git

Now change into the “tvheadend” directory and list the build optons:

cd tvheadend/
./configure --help

Build it with hdhomerun support, and some other goodies required for transcoding:

AUTOBUILD_CONFIGURE_EXTRA=" --enable-hdhomerun_client --enable-avahi --enable-hdhomerun_static --enable-libffmpeg_static" ./Autobuild.sh -t precise-amd64

This should create a tvheadend deb package a level up, which you can now install using the distro’s install tool “dpkg”:

cd ..
# dpkg -i tvheadend_<your freshly created package>.deb
# e.g.
dpkg -i tvheadend_4.0.7-11~g398e4fe~precise_amd64.deb

Now run it:

service tvheadend start

You should be able to browse to your TVH server on port 9981. eg. http://10.69.10.42:9981/

login is tvhadmin/tvhadmin.   You can update that once you’re in. Enjoy.

Keeping TVH Up to Date

Go into the /usr/src/tvheadend directory and run git pull .  After that, do a build just as you would before, and install the new deb package which will update TVH.  Simples! You should still have the previous deb package if anything mucks up, and you can apt-get remove tvheadend the current version, and then re-install the older version.

You can keep the base system up to date with:

apt-get upgrade
apt-get autoremove

 

Tvheadend vs MythTV for Kodi – TVH the Clear Winner

As you can see from my previous blogs, I’ve been playing around with MythTV of late, as a backend TV server for Kodi.  Even though I was successful in setting this up, I’ve hit some frustrating limitations and have so decided to pull the pin on that experiment, and go back to Tvheadend (TVH for short). My major annoyances are/were:

  • I couldn’t find a way to split DVB-T TV and DVB-T radio stations, the way TVH does in Kodi. With MythTV, they all appear as TV stations.
  • The ability to set channel groups seems to be lacking in MythTV.

I was looking further and further into these issues and there was some talk in some forum somewhere about being able to do these things with SQL commands in MySQL, but I figured I didn’t want to waste any more time – I’d invested far too much already! In TVH it *just works*.  Other gripes with MythTV include:

  • Since Myth’s both a front-end and back-end, there are often parts of config that relate to the front-end that I’ll never touch, so it’s a bit confused when you’re only using the backend. This is especially apparent with some iPhone apps I bought, where some screens of the app are for the backend server, and some for the frontend. The benefit of TVH is that it is a pure server – there is none of this confusion of backend vs frontend!
  • It’s pretty fiddly just to get the MythTV server running. You also have to jump through a lot of hoops to get the mythweb webserver going as well. Say goodbye to a weekend!

A fairer comparison is MythBuntu vs TVH-Ubuntu, rather than MythTV-FreeBSD.  I’ve tried MythBuntu, and set-up is a bit easier than all that work I did on FreeBSD, but it’s still kludgy to my mind, and still suffers from my major gripes with it.

I have found it to be actually quite straightforward to install TVH from a vanilla Ubuntu install, and to keep it up to date with Git pulls. I’ll add a blog post soon to show how this is done. My ideal setup would be for TVH to stabilise and add native HDHomeRun support for FreeBSD (the way MythTV manages to do!), and then have the FreeBSD people update the Ports collection with this stable code. Hopefully TVH will get there in the next year or two. Then I could run TVH on my FreeNAS/FreeBSD box, and shut down my separate Linux server.

Installing Mythweb into Your FreeNAS MythTV Jail

In my last blog I showed how I installed a MythTV backend PVR server into a FreeNAS Jail.

A must-have for usability is the Mythweb HTTP-based configuration tool. There’s a lot of hoops to jump through, but you should hopefully make it through unscathed!

Install Mythweb and Dependencies

pkg install mythplugin-mythweb

For me this says that it will install the following:

New packages to be INSTALLED:
 mythplugin-mythweb: 0.27_1
 php56-session: 5.6.7
 php56: 5.6.7
 apache24: 2.4.12
 php56-posix: 5.6.7
 php56-json: 5.6.7
 php56-mysql: 5.6.7

Since it is installing PHP 5.6, you’ll need the “56” version of mod_php as well:

pkg install mod_php56

There are important messages from the installs (shown here) which we’ll address below:

Message for apache24-2.4.12:
To run apache www server from startup, add apache24_enable="yes"
in your /etc/rc.conf. Extra options can be found in startup script.

Your hostname must be resolvable using at least 1 mechanism in
/etc/nsswitch.conf typically DNS or /etc/hosts or apache might
have issues starting depending on the modules you are using.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
- apache24 default build changed from static MPM to modular MPM
- more modules are now enabled per default in the port
- icons and error pages moved from WWWDIR to DATADIR
 If build with modular MPM and no MPM is activated in 
 httpd.conf, then mpm_prefork will be activated as default
 MPM in etc/apache24/modules.d to keep compatibility with 
 existing php/perl/python modules!
Please compare the existing httpd.conf with httpd.conf.sample
and merge missing modules/instructions into httpd.conf!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Message for mythplugin-mythweb-0.27_1:
mythweb has been installed into:
 /usr/local/www/mythweb
You need to configure mythweb now according to the instructions in
 /usr/local/www/mythweb/INSTALL
For lighttpd you should have a look at the MythTV Wiki
 http://www.mythtv.org/wiki/MythWeb_on_Lighttpd

Message for mod_php56-5.6.7:
***************************************************************
Make sure index.php is part of your DirectoryIndex.
You should add the following to your Apache configuration file:
<FilesMatch ".php$">
 SetHandler application/x-httpd-php
</FilesMatch>
<FilesMatch ".phps$">
 SetHandler application/x-httpd-php-source
</FilesMatch>
***************************************************************

Server-Wide Config

Update “/etc/rc.conf" with the following:

apache24_enable="YES"

Update “/etc/hosts" to something similar to the below – changing the parts in red to suite your IP, hostname and domain:

127.0.0.1 localhost localhost.localdomain mythserver mythserver.gavowen.local
10.69.10.22 mythserver mythserver.gavowen.local

Mythweb Config

You can choose to put your mythweb into the root of the web server, but I prefer to put it off to the side it its own “mythweb” directory.

Copy the mythweb files to where the web server can use them:

mkdir /usr/local/www/apache24/data/mythweb
cp -R /usr/local/www/mythweb/* /usr/local/www/apache24/data/mythweb

Apache 2.4 runs as user and group “www” on FreeBSD/FreeNAS, so update the permissions accordingly:

chgrp -R www /usr/local/www/apache24/data
chmod -R g+rw /usr/local/www/apache24/data

TV Icons

I copied mine into my jail, then across to MythWeb’s TV icons directory:

cp /mnt/storage/channelicons/* /usr/local/www/apache24/data/mythweb/data/tv_icons

Mythweb Apache Config File

The Mythweb Apache config file (“/usr/local/www/apache24/data/mythweb/mythweb.conf.apache“) is a bit out of date for Apache 2.4 and PHP 5.6, so pull down the latest version directly to the directory you need it in.

cd /usr/local/etc/apache24/extra/
wget --no-check-certificate https://raw.githubusercontent.com/MythTV/mythweb/master/mythweb.conf.apache
cp /usr/local/etc/apache24/extra/mythweb.conf.apache /usr/local/etc/apache24/extra/mythweb.conf

Edit the “mythweb.conf" file and change the following:

<Directory “/var/www/html/data”>
to
<Directory “/usr/local/www/apache24/data/mythweb/data”>

and
<Directory “/var/www/html”>
to
<Directory “/usr/local/www/apache24/data/mythweb”>

My NAS has plenty of RAM, so I increase the PHP “php_value memory_limit” to 256M. Save the file and quit.

Make sure these lines are uncommented:

 setenv db_server "localhost" 
 setenv db_name "mythconverg" 
 setenv db_login "mythtv" 
 setenv db_password "mythtv"

Apache Config

Backup Apache’s original config file before editing “httpd.conf":

cp /usr/local/etc/apache24/httpd.conf /usr/local/etc/apache24/httpd.conf.original

Uncomment all these mods. The first two are usually on by default:

mod_env
mod_headers
mod_rewrite
mod_deflate
mod_auth_digest

Uncomment ServerName and change it to match what you’ve done previously in “/etc/hosts" e.g.:

ServerName mythserver.gavowen.local:80

In the “Supplemental Configuration” of the file, I place the following two lines:

# Mythweb configuration
Include etc/apache24/extra/mythweb.conf

Further down you’ll find the DirectoryIndex, where I add “index.php”

<IfModule dir_module>
 DirectoryIndex index.php index.html
</IfModule>

Directly below that part I add the following:

<FilesMatch ".php$">
 SetHandler application/x-httpd-php
</FilesMatch>
<FilesMatch ".phps$">
 SetHandler application/x-httpd-php-source
</FilesMatch>

Restart Apache:

service apache24 start

Now open a browser. Enjoy.

NB: Subsequent restarts of apache can be done with “apachectl graceful"

Troubleshooting

If the page doesn’t load try this:

create user 'mythtv'@'127.0.0.%' identified by 'mythtv'; 
create user 'mythtv'@'127.0.1.%' identified by 'mythtv'; 
set password for 'mythtv'@'127.0.0.%' = password('mythtv'); 
set password for 'mythtv'@'127.0.1.%' = password('mythtv'); 
connect mythconverg; 
grant all privileges on *.* to 'mythtv'@'127.0.0.%' with grant option; 
grant all privileges on *.* to 'mythtv'@'127.0.1.%' with grant option; 
flush privileges; 
exit;

There are other possible issues addressed at this link.

Time Issue

You might get a warning such as:

User Notice at /usr/local/share/mythtv/bindings/php/MythBackend.php, line 132:
 !!NoTrans: Failed to set php timezone to AEST Response from backend was Array ( [0] => AEST [1] => 36000 [2] => 2015-04-25T10:14:06Z ) !!
 Warning: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in /usr/local/www/apache24/data/mythweb/includes/errors.php on line 161

If that’s the case I find installing NTP a good solution:

pkg install ntp

Update “/etc/rc.conf" with the following:

ntpd_enable="YES"

Then start it with “service ntpd start". Issue “date" to see that it’s accurate. Then:

cp /usr/local/etc/php.ini-production /usr/local/etc/php.ini

search for “date.timezone” and change it to suit. For me it’s date.timezone = "Australia/Melbourne"

 

MythTV Server in a FreeNAS Jail

MythTV is is one of the most well-known open source LiveTV/PVR servers, and worth trying out and comparing to Tvheadend for serving up live TV to Kodi. I have previously given some background info on both of them. Here’s how you can install the MythTV backend in a FreeNAS jail.

Creating your Jail

First make sure your jail environment is set-up correctly:

Jail setup

I’ve carved out a range of IPs from my local LAN /24 subnet of 10.69.10.0/24

Next, go to the Jails tab and “Add Jail”. Click on “Advanced Mode” if not already.

I like to call my MythTV Jail “mythserver” – I feel this is preferable to calling it say “mythtv” because if the name is too close to the program name, it can get confusing later on in system log files.

I let it auto-select the next available free IP address, and then manually add the “IPv4 default gateway” – the router’s LAN IP which for me is 10.69.10.1.

I leave “autostart” and “VIMAGE” selected.

Once you hit “Ok” you can edit the jail you just created, and you should see that it’s added a MAC address, and for FreeNAS 9.3 it’s also added “allow.raw_sockets=true” in the “Sysctls” section.  If this is your first jail, it’ll take a while as it downloads the jail template from FreeNAS, before it finalises setting up the jail.

Jail Storage

You’ll want to map some extra storage into your jail for recordings. I give it full permissions:

mythstorage

I limit it to 40 Gigs in the Options:

mythoptions

Then I mount it into the jail:

mythmountstorage

Jail Login

SSH to your FreeNAS and list your jails:

jls

e.g.:

root@sarlacc# jls
 JID IP Address Hostname Path
 1 - dhcp_dns /mnt/volume1/jails/dhcp_dns
 2 - mythserver /mnt/volume1/jails/mythserver

My MythTV jail called “mythserver” is jail ID #2 at the moment. Log into the jail:

jexec 2 /bin/csh

Jail Update

pkg update
pkg upgrade

The “pkg update” is redundant as the upgrade does an update first, but I still like to do it regardless. Just say yes to all the upgrade prompts. You can run the same commands again after the upgrade, just to confirm the software is now current.

Bash Shell

If you prefer Bash like I do, install it tweak it.

pkg install bash
vi ~/.bashrc

and add some aliases…

# some useful aliases
alias h='fc -l'
alias j=jobs
alias m=$PAGER
alias ll='ls -laFo'
alias l='ls -l'
alias g='egrep -i'

Log out and back in with bash:

exit
jexec 2 /usr/local/bin/bash

That’s if jail ‘2’ is your myth jail. You may only need to type jexec 2. Try it and see.

Helper Software

Install “GNU Make” for compiling LAME:

pkg install gmake

Install the FreeBSD Ports tree and compile the LAME encoder:

cd /usr/ports
portsnap fetch extract
cd /usr/ports/audio/lame
make install clean
cd /

NB: Even though the directories are there, if you don’t do a fresh “extract” then the system will complain, so you have to run this the first time after creating the jail. If you are doing further installs some time in the future (say next week), all you need to run before compilation is portsnap fetch update

Install the graphical packages. You’ll need these for configuring MythTV by the setup GUI which runs on the X-Windows system. Note the upper case X in libXv:

pkg install libXv
pkg install qt4-webkit
pkg install xauth
pkg install xorg-fonts

Install MythTV

pkg install mythtv

You have time to go grab a coffee now, as it’ll take a while. You’ll see a message at the end like:

**********
MythTV has now been installed, but it still needs to be configured.
1. To create the database, use the following command:
mysql -uroot -p < /usr/local/share/mythtv/database/mc.sql
2. Next, run mythtv-setup.
See http://www.mythtv.org/docs/ for more information.
**********

You’ll get to do (1.) and (2.) later. For now, update /etc/rc.conf with the following:

sshd_enable="YES" #change the existing "NO" entry to "YES"
mysql_enable="YES"
mythbackend_enable="YES"

I found that the hostname was in that file twice, so I deleted the duplicate.

Tip: You can quickly update system variables with the following:

sysrc sshd_enable=YES
sysrc mysql_enable="YES"
sysrc mythbackend_enable="YES"

Install MyQSL

Of course we don’t have MySQL installed yet, so we need to grab that too, and start it up:

pkg install mysql56-server
service mysql-server start

Populate mysql with the MythTV database:

mysql -uroot -p < /usr/local/share/mythtv/database/mc.sql

Just hit the “enter” key at the password prompt, as a password isn’t yet set. Out of interest, this does the following:

==========
CREATE DATABASE IF NOT EXISTS mythconverg;
GRANT ALL ON mythconverg.* TO mythtv@localhost IDENTIFIED BY "mythtv";
FLUSH PRIVILEGES;
GRANT CREATE TEMPORARY TABLES ON mythconverg.* TO mythtv@localhost IDENTIFIED BY "mythtv";
FLUSH PRIVILEGES;
ALTER DATABASE mythconverg DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
==========

Login into mysql (hit enter at the password prompt) and check that “mythconverg” is in there.

mysql -p
show databases;
exit

MySQL Timezone Update

Update timezone info with:

mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p mysql

I found I got the “Data too long for column ‘Abbreviation’ Error” so I did the following:

mysql_tzinfo_to_sql /usr/share/zoneinfo > /tmp/timezonefix
vi /tmp/timezonefix

I searched through the file and found the stanza (parts between two semicolons) that contain “use tzsetup” and edited out the following lines:

;
INSERT INTO time_zone (Use_leap_seconds) VALUES ('N');
SET @time_zone_id= LAST_INSERT_ID();
INSERT INTO time_zone_name (Name, Time_zone_id) VALUES ('Factory', @time_zone_id);
INSERT INTO time_zone_transition_type (Time_zone_id, Transition_type_id, Offset, Is_DST, Abbreviation) VALUES
 (@time_zone_id, 0, 0, 0, 'Local time zone must be set--use tzsetup')
;

I then ran:

cat /tmp/timezonefix | mysql -u root -p mysql

This now completed without error.

Set MySQL Password

mysqladmin -u root password <yourpassword>
e.g
mysqladmin -u root password letmein

Test with:

mysql -p

You should have to put in that password now. Type “exit” to quit out.

Setup X11 Forwarding

Edit /etc/ssh/sshd_config with the following updates, replacing the IP address below with your IP:

ListenAddress 10.69.10.22
PermitRootLogin yes
PasswordAuthentication yes
X11Forwarding yes
X11UseLocalhost no

Now start sshd:

service sshd start

This generates keys and starts the SSH server. I like to test the server with a  service sshd restart , to make sure it starts cleanly now that the keys are present.

Set a root password, and test the password along with external connectivity to the mythtv-setup script. If all’s successful, it’ll time out and say that it cannot connect to the X server. That’s the desired outcome, as you want to connect to an X server on another machine. Remember to update the IP address below to that of your jail.

[root@mythserver /]# passwd 
Changing local password for root
New Password:
Retype New Password:
[root@mythserver /]# exit
[root@mythserver /]# ssh -Y root@10.69.10.22 /usr/local/bin/mythtv-setup
Password:
mythtv-setup: cannot connect to X server

Setup Remote X Server

Download Xming from here:
Download PuTTY from here:

1) Install Xming server and start it
2) Install PuTTY on your local machine
3) Configure the default profile in PuTTY with the IP and SSH port of your MythTV Server
4) Go to the Connection panel then select ssh and enable compression, then the x11 tab and check “Enable x11 Forwarding” box, then enter localhost:0 in the “X Display Location” and check “MIT-Magic-Cookie-”
5) Go to “Session”, type “MythTV Server” in the Saved Sessions, and hit the Save button.

Configuring MythTV

With the Xming server started, open the PuTTY session you saved and login (with the user you created earlier) to the myth server, then type….

mythtv-setup

This will open an Xwindow on your local screen and load the myth setup screen. You might have to run this a second or third time, as it might quit after a country and language update, and also after a database schema update. You’ll get there eventually.

I recommend you follow the MythBuntu setup guide here, skipping right down to the “MythTV Backend Setup” section, and taking it from there.

Start the Backend

The backend service now needs to be started if not already:

service mythbackend start

Also make sure it’s filled with your config settings:

mythfilldatabase

Congratulations – you’ve now configured MythTV backend on FreeNAS in a jail.

Troubleshooting

If you have any issues, check where the logserver stores its files and check the logs.

[root@mythserver /var/log/mythtv]# ll
total 12988
drwxr-xr-x 2 root wheel - 5 Apr 21 03:00 ./
drwxr-xr-x 3 root wheel - 22 Apr 21 03:01 ../
-rw-r--r-- 1 root wheel - 219163 Apr 21 03:00 mythbackend.20150420163447.83755.log
-rw-r--r-- 1 root wheel - 12917053 Apr 21 19:40 mythbackend.20150420170022.85871.log
-rw-r--r-- 1 root wheel - 4998 Apr 21 03:00 mythlogserver.20150420163448.83757.log
[root@mythserver /var/log/mythtv]#

tail the latest backend log file:

[root@mythserver /var/log/mythtv]# tail -5 mythbackend.20150420170022.85871.log
2015-04-21 19:41:31.854481 E [85871/101782] HttpServer107 servicehost.cpp:426 (ProcessRequest) - No Security Pin assigned. Run mythtv-setup to set one.
2015-04-21 19:41:31.862285 I [85871/101739] ProcessRequest mainserver.cpp:1420 (HandleAnnounce) - MainServer::ANN Monitor
2015-04-21 19:41:31.862295 I [85871/101739] ProcessRequest mainserver.cpp:1422 (HandleAnnounce) - adding: kodi as a client (events: 0)
2015-04-21 19:41:31.875402 E [85871/101782] HttpServer114 servicehost.cpp:143 (Invoke) - MethodInfo::Invoke - An Exception Occurred: No Security Pin assigned. Run mythtv-setup to set one.
2015-04-21 19:41:31.875436 E [85871/101782] HttpServer114 servicehost.cpp:426 (ProcessRequest) - No Security Pin assigned. Run mythtv-setup to set one.
[root@mythserver /var/log/mythtv]#

ooops – didn’t add a security pin – have to set that up!

Other tips:

  • Go to another box and “telnet <your myth IP> 3306” to make sure your MySQL database is working and allowing connections.
  • Got to http:<your MythTV IP>:6544  to make sure you have web connectivity
  • All else fails – RTFM: MythTV Wiki, and the User Manual.

Other Tasks

It’s advisable you setup MythWeb for managing your TV recordings.

A Breakdown of Kodi LiveTV PVR Backends for FreeNAS

I use the awesome open source Kodi media centre for all my movies and TV shows, so it’s only natural that I use it as a live TV frontend. Since I already have an equally awesome FreeNAS NAS, then it makes sense to see if I can utilise its “Jails” functionality to run the live TV / PVR backend, instead of utilising a separate box. It makes sense – the NAS has a grunty CPU; teamed 1Gig Ethernet links; 16Gig of RAM; loads of storage space for recording TV, and is always on – so it’s the perfect candidate, so long as I can get the backend software to run!

This page currently lists four backends for Linux:  TVheadendMythTVVDR and DVBLink.  Out of those four, only the first three are possible under FreeBSD, as DVBLink is commercial software with only binary code downloads for Linux (among others), but not FreeBSD. The others have downloadable source code which can be compiled on FreeBSD. Discussing them further:

Tvheadend

Tvheadend (aka “TVH”) is basically the unofficial TV backend for Kodi. One distribution – OpenELEC – even includes the server in it’s distribution (although SoC systems like RaspberryPi miss out). I was actually using the inbuilt TVH server in OpenELEC before going to a RaspberryPi2 front end, which cut down noise in my lounge room by not having to run a PC with fan near my TV.

On their currently out-of-date website they say “With the integration of PVR functionality into XBMC [kodi], this has now become possibly the most popular TVH client. Indeed much of the recent development of HTSP has been focused on improving integration with XBMC [kodi].

Although the TVH devs don’t much update the website, the actual code is coming along in leaps and bounds with a lot of active development over at GitHub. From what I can gather here, this code was at version 3.4 when one developer by the name of Adam Sutton did a lot of work up till that point, but with a young family he found it hard to continue to commit to the project. That branch is currently known as the “stable” branch. Thankfully another team of devs lead by Andreas Öman came to the fore, and the project was fully opened up as an open source project, and the current code being updated to 3.9.x and is deemed the “unstable” branch. Still this is the branch that you want to be using as it has the transcoding, timeshifting (pause/rewind live TV) and other goodies, and it’s what you’ll get from Github.

Hopefully the 3.9 branch stabilises into 4.0 soon, and FreeBSD ports update their code, and the iPhone TvhClient gets updated as well so it supports all the 3.9 features like transcoding. This does seem to be the backend for Kodi users to watch in the future.

The problem main problem with Tvheadend on FreeBSD is seemingly no support for the SiliconDust HDHomeRun. According to this page, the way TVH supports HDHomeRun is with a Linux “dvbhdhomerun” driver, which only works on Linux as far as I’m aware. This really is a deal-breaker for me, as the HDHomeRun is what I use, and FreeBSD/FreeNAS is my preferred server platform. I really hope they can find some other way to support HDHomeRun on the *BSD’s.

MythTV

This one has been around for years, and has the most tedious set-up process, although is arguably the most flexible. On a generic server or ESXi, you can install the MythBuntu distribution, which makes install a lot easier, but on FreeBSD in a headless server, there’s a lot more heavy lifting.

I recommend setting up a MythTV jail, even if you have Tvheadend installed, and see which one works best for you.  The good thing about the FreeBSD jails in FreeNAS is that you can set them up without stepping on each other’s toes, and it’s a fun tech exercise. The only thing with MythTV is that it engages all the HDHomeRun tuners at all times even when not watching TV, so you’ll need to shut that jail if you’ve got a Tvheadend jail running and are trying that out. Tvheadend is a lot friendlier in this regard – it only uses a HDHomeRun tuner when it needs one, so most of the time it’s either using none or one tuner, and only using more than one tuner if you’re watching live TV on one mux, and recording on another mux (or recording two shows simultaneously on two different muxes).

VDR

Well I can’t say I’ve actually used this for Kodi. I’ve stayed away mostly because of what I perceive as limited support for the HDHhomeRun network TV tuner. A web search for “VDR HDHomeRun” shows up http://www.fepg.org/hdhomerun/. This isn’t very confidence-inspiring because if you look at the history, it shows it’s at version 0.0.1 last updated five years ago in 2010! Yes I think I’ll give this one a miss for now.

Summary

If it wasn’t for wanting to use the HDHomeRun I’d probably use Tvheadend for its ease of use, but since I do, my only choice is to use MythTV as a TV/PVR backend for Kodi. I’ll put together an install guide on my blog for MythTV, and also add some info into my wiki.

OpenELEC + HDHomeRun + Tvheadend

Update March 2015 – Some of this info is out of date. Will update the info, and put a good guide on the wiki.

 

Intro

I’ve recently set up the OpenELEC (XBMC/Kodi based) entertainment centre with HDHomeRun as the TV tuner, and fed into the system with the Tvheadend backend and front end.

This is a quick guide to setting it up, and hopefully stop you from depriving yourself of sleep (like I did) as you bang away getting it sorted.

Props to: Code Bytes for getting me started!

Prerequisites

  • You’ve got XBMC/Kodi set up, and you have a skin that supports live TV.
  • You have your HDHomeRun plugged into your home network and with an antenna feed into it.

OK let’s do this.

Hardware

Don’t assume that the HDHomeRun signal is fine! These boxes need a strong signal to drive them – something stronger usually that what your TV needs to get a decent picture.

I had to take the antenna cable into an amplified splitter. Without it, I found that (in Australia) my channel 10 was a bit iffy, and ABC was poo, and community channel 31 couldn’t even be found.

The amp I got is a GME  Kingray 162.  It cost me about $50 from Dick Smith Electronics. It is an outdoor unit and supposed to be mounted on the antenna as close to the antenna as possible. I’m in a rented unit so don’t want to get on the roof and muck around with the antenna so I have it inside where the cable comes in. The lower down it is – the more noise you amplify. Still it did the trick for me, where it amplified the signal enough for the HDHomeRun to run clean and with digital – that’s all that matters. 🙂

Some tips

  • Always use the F-type screw in cables where possible and avoid the PAL/Bellings Lee adapters.
  • Experiment with cables. I found that for some reason, the cable that came with the HDHomeRun worked better that the new Dick Smith cable I bought.
  • Run the Windows software to gauge differences before and after amplification, and with different cables. Use the very best cable from amp to HDHomeRun.
  • Do a scan with the Windows software (or Mac software) and see that all is sweet before you start blaming XBMC/Kodi for your woes!

Add-Ons

Install these plugins. They are both in the OpenELEC standard repository.
HDHomeRun
TVHeadEnd

Config

HDHomeRun

Here’s the tricky bit, and where I banged my head for ages! The first thing you want to do is configure HDHomeRun to be setup as a working source of TV, so it looks to the Linux OS like it’s attached to the box and so the Tvheadend can use it!

Now I was screwing around with this late at night so I don’t know if some steps are necessary but when you test your setup you’ll know I guess.

Start by running the configuration for the “hdhomerun” add-on
System -> Add-ons -> Enabled  -> Program  -> hdhomerun

General
 -- Pre-wait time [sec]    "2"  [default]
    -- Post wait time [sec]   "1"  [default]
    -- Enable userhdhomerun logging   "on" [changed]
    -- Enable libhdhomerun logging   "on" [changed]
    -- Enable suspend/resume the driver "off" [default]
Tuner Settings
    -- Enable modifying settings    "yes"  [changed?]

Now hit the “Refresh tuners”… and your tuner should show up. Set it to DVB-T, use full names to whatever you like (I choose “off”), and set number of tuners to 2, if it has 2. Don’t disable….that would not be sensible.

It’s very important to enable the logging above, and I’ll explain why. You see, HDHomeRun has a userspace component and a kernel component. The “userhdhomerun” userspace program scans the network to find the devices on the network, and then the “libhdhomerun” sets up kernel devices in /dev

OpenELEC:~ # ls /dev/dvb/
 adapter0  adapter1
 OpenELEC:~ #

If things aren’t sweet, you won’t see those devices. Now these two programs have to run and setup the devices before Tvheadend runs, or you won’t see the devices to choose from the Tvheadend web interface, that I’ll get to later.

Fix for missing adapters

The HDHomeRun add-on wiki page it says, “This driver is started from tvheadend and vdr addon”. Well Tvheadend may do this once you’ve got HDHomeRun configured already, but I found in order to get the two HDHomeRun tuners to show up on the Tvheadend web interface in the initial step, then I had two options:

  • Install and enable “vdr” add-on, which I didn’t need; or
  • enable logging for “userhdhomerun” and “libhdhomerun”

I chose the latter. It seems that enabling logging starts those processes up before Tvheadend starts, which is what you want. If the adapters weren’t there and I ran “userhdhomerun” on the CLI, then the adapters would show up in /dev/dvb/ but Tvheadend wouldn’t use them, as they weren’t there when it scanned for /dev/dvb/ devices! I lost a lot of sleep because of this, and was fully relieved when enabling logging solved my issues! (but I still had a reboot issue… see below).

Playing with config files

\[kodi IP address without brackets\]Userdataaddon_datadriver.dvb.hdhomerun

In there, there should be the following files – all created by the setup tool

  • adapters.txt
  • dvbhdhomerun.conf
  • dvbhdhomerun.sample
  • settings.xml

I didn’t need to create any of those. If something goes wrong and you, for whatever reason, need to hack on this manually then grab the dvbhdhomerun.sample from /storage/.xbmc/addons/driver.dvb.hdhomerun/config and copy it into that network share. Then copy that file to the same network share and rename it “dvbhdhomerun.conf” which is the config file it’ll use. My conf file has some comments and the meat of it says:

[1110AE0D-0]
 tuner_type=DVB-T
 use_full_name=true
 [1110AE0D-1]
 tuner_type=DVB-T
 use_full_name=true
[libhdhomerun]
 enable=true
 logfile=/var/log/dvbhdhomerun_libhdhomerun.log

….. where 1110AE0D is the “Device ID” you’ll find printed on the device, and which can also be found by running “userhdhomerun” on the CLI.

My “adapters.txt” says
“Tue Nov 25 13:21:22 2014 Name of device: 1110AE0D-0 Tue Nov 25 13:21:22 2014 Name of device: 1110AE0D-1”

…which was created automatically for me.

TVheadend

Go to webpage of TVHeadend: http://[kodi IP address without brackets]:9981
Go to Configuration -> DVB Inputs -> TV Adapters

Select the first adapter.
Click “Add DVB Network by Location”
Choose your location
Click “Enabled”
Click “Save”

TVheadend now starts scanning for channels. You can monitor its progress in the pane to the right. When “Muxes awaiting initial scan” is zero, it’s done.
Click “Map DVB Services to Channels”.
The mapping will also take some minutes to complete depending on how many channels are present.

In the web config in Configuration -> TV Adapters: After all channels have been configured, disable “Autodetect Muxes” and “Idle Scanning“. Having these enabled completely destroyed the stream from TVHeadEnd, making the image look garbled and stuttering. It looked like a low bandwidth connection or bad signal.

Missing Channels?

I had an issue where the muxes for “Australia -> au_Melbourne” didn’t have the newly added “C31” community channel, and the changed SBS settings. Doh!

What I had to do was a full channel scan by deleting all the existing Multiplexes, then choosing the “Add DVB Network by Location” and choosing the “–Generic–” -> “auto_Australia”. This does a fairly exhaustive scan off all the muxes devoted to DVB in Australia – 100 at the moment! Grab a coffee and do something else for a good half an hour as it takes a while!

Then when done, over on the Multiplexes tab, select all the muxes with no IDs and delete them. I was left with:

  • Seven Network
  • SBS Melbourne
  • Nine Network Australia
  • Network TEN
  • ABC Melbourne
  • C31

…yay they are all there!

I ended up with 35 “services”. One was bogus in that it had no details at all. So we’re left with 34 services. A few of those were duplicates, so we’re left with 31 *actual* services. 5 of those are radio (ABC Jazz, Double J, SBS Radio 1, SBS Radio 2, and SBS Radio 3), and 26 free to air TV channels.  You don’t have to worry about removing those 3 duplicate TV channels – the front end seems to filter out the dupes.

You should now be able to go to the front end and filter out the garbage shopping channels, and rearrange your channel order.

Things Messed on Reboot?

I found that every time I rebooted, I had a message about there being no hardware found, and couldn’t play live TV. The fist was to go back into the HDHomeRun add-on settings and set the “Pre wait time [sec] from “2” to “7”, which gave HDHomeRun more time to initialise. I found “5” didn’t work, and “7” did. I didn’t try “6” but even if it did work, I prefer to give it that extra second to be on the safe side.

I also found I could untick the logging options on HDHomeRun which I’d setup earlier.

TVheadend Tweaks

Channel Cleanup

Go to the “Channel / EPG” and delete all the home shopping channels. I also deleted ABC3 (kids shows), and SBS3 (duplicate of another SBS channel).

Time shifting

Go to Recording / Timeshift, and enable this, and up the max size to something a lot bigger. I’ve gone with “10240” (i.e. 10 gigs) for now. Make sure you have a SSD drive for maximum smoothness. This will allow you to pause and rewind live TV.

Channel Icons

I found some great TV logos / Channel Icons for Australian stations here:
You need to make sure that the image file names are exactly the same name as channel names – including case. For ABC2 / ABC4 – that contains a “/” in the name so for that one you have to manually pressing  “C” on a keyboard attached to your OpenELEC box and then select “Choose Thumbnail”.

Future Versions

I have been reading the forums and see that OpenELEC and Kodi 14 are now in beta, with a final release probably available by early 2015. This has a much updated version of Tvheadend that includes direct support for HDHomeRun devices, and also promises a lot of improvements for TV/PVR functionality, so much of what I’ve written above will change.

Still, there’s a lot of code changes under the hood with all the XBMC to Kodi name changes, so I’d suggest backing up your complete system before making the jump! I personally will wait for a couple of point release versions of openELEC before making the jump to OpenELEC 5.x (based on Kodi 14.x).