Replacing a Failed Drive on a Gen8 HP Microserver Running FreeNAS

I recently had one of my freeNAS9.3 report the following issues:

  • CRITICAL: Device: /dev/ada2, 8 Currently unreadable (pending) sectors
  • CRITICAL: Device: /dev/ada2, 8 Offline uncorrectable sectors

unreadable sectors

After a bit of reading I decided it best I replace the drive, as I don’t want to take chances since I’m only running RAID-Z1 (for the space), instead of the preferred and safer RAID-Z2.

Then I hit a problem: What physical drive is ada2?  The Gen8 HP Microserver G2020T doesn’t have drive lights to indicate which ones are active.

What I did was took a bit of a pun that the drives are numbered left to right, and it proved correct.

What you really need to do before doing anything, is take a screenshot of the different drive serial #’s. Don’t rely on the drive numbers!!! The reason I say that is because once you pull out a drive, the drive numbers get remapped! When I pulled out ada2, what was previously ada3 became the new ada2! This can get confusing and cause you to pull the wrong drive and screw your data – so concentrate on the serial numbers.

I don’t need to repeat the full instructions, but will link you to them here: http://doc.freenas.org/9.3/freenas_storage.html#replacing-a-failed-drive

Here’s my screenshot of my drives. Note the serial of ada2.

Pre drive replace

I now offlined ada2 and shutdown FreeNAS in order to pull the drive out, as the drives in the Gen8 HP Microserver are apparently not hot-swappable (that’s something they really should address!). I booted up to make sure that the correct expected drive SERIAL NUMBER disappeared. Notice how what was ada3 is now ada2.

Post drive pull

Happy that I have pulled the right drive, I now proceed to shut down again, and then insert the new 4TB Seagate NAS drive. After booting up again:

New drive inserted

OK so far so good. I now highlighted the new ada2 as per the screenshot, and then clicked on “Replace”. I then had to confirm that I wanted to replace the original ada2, but I didn’t get a screenshot of that. It then went about reslivering, which is the process of recreating the data on the redundant drive.

resilvering

For me it got up to about 5% after 10 mins, so figured it would take somewhere between 3 and 4 hours, so kicked it off before I went to bed. The interesting thing is that the old drive’s volume ID (8482932750830730262) is still listed in the array during the resilvering process. It’s as if you can cancel the resilvering and go back to the original drive if you so wished (if you had a failed resilver perhaps?) but I didn’t test this theory. Once resilvering is complete, this old drive/volume reference goes away.

Hope this helps. Happy and safe NASing!

 

Update: 20 August 2015.  There was an a file in my /tmp directory called “.smartalert” which seemed to contain the source of the alert. I deleted that file and rebooted, and the alarms cleared.

 

Getting Tvheadend Picons to Work in Plex

Picons are handy if you are using the Plex Tvheadend channel and you want the TV station icons to show up.

I was able to pull down all the Australian TV icons online from Beyonwiz (this is for my Ubuntu Linux Tvheadend server):

apt-get install git
cd /usr/src
git clone https://bitbucket.org/beyonwiz/picons-australia.git

 

Then you can set the Tvheadend (TVH) server “Configuration -> General” tab to prefer picons over channel name, and set the path to file:///usr/src/picons-australia/picon

The other thing you need to do is under “Configuration -> Access Entries”, add a new entry with the following:

Enabled: tick
Username: *
Password: *
Network prefix:  The IP address of your Plex server or Kodi player, or even local subnet if you want. e.g. "10.1.0.45/32" or "10.1.0.0/24" (or "127.0.0.1/32" if Plex and TVH are on the same box).
Streaming: tick

That should be enough access to get the icons working.

When I get time I’ll see if I can feed Kodi the picons in a similar way, as I prefer this server-side method of delivering TV channel icons, rather than client-side. For now I just point Kodi to a local directory with PNG images named the same as the channel names, which seems to work fine.

The only issue with my current picon set-up for the Plex Tvheadend Channel, and it is a minor one, is that the picons get truncated on my iPhone as they aren’t square format. They do look look fine on the PC though. I’m tossing up whether it’s worth my time to create square icons for the 22 stations I make use of in Melbourne Australia.

Other than that, I’m pretty stoked with being able to get the icons/picons to display!

Installing Tvheadend on Ubuntu

Update April 2016: You don’t have to build your own Ubuntu packages any more, as they are maintained here.

I’ve chosen to install Tvheadend (TVH) on a vanilla Ubuntu Server 14.04.2 installation. Incidentally I have Ubuntu (64-bit version) setup on a ESXi 6.0 host. Here’s how you can do it to:

Install Ubuntu Server 14.04.2

I recommend installing these at install time:

ssh server
samba server

Otherwise don’t install them straight up, but later once in the CLI you can do this:

apt-get install ssh
apt-get install samba

If the timezone is somehow messed up you can:

dpkg-reconfigure tzdata

In Ubuntu, you don’t log on as root or set the root password, but rather sudo -i, which will get you root privs.

Finalise the install:

sudo -i
rm -rf /var/lib/apt/lists/*
apt-get update
apt-get upgrade
reboot

Install Required Libraries

apt-get install build-essential git pkg-config libssl-dev bzip2 wget
apt-get install libavahi-client-dev zlib1g-dev libavcodec-dev
apt-get install libavutil-dev libavformat-dev libswscale-dev
apt-get install libcurl4-gnutls-dev liburiparser-dev
apt-get install debhelper

Install Tvheadend

Go to your building area:

cd /usr/src/

Get a snapshot of the latest TVH:

git clone https://github.com/tvheadend/tvheadend.git

This will install the latest development code branch. At the moment this is 4.1. If you want to install the stable 4.0 branch then try this:

git clone --branch release/4.0 https://github.com/tvheadend/tvheadend.git

Now change into the “tvheadend” directory and list the build optons:

cd tvheadend/
./configure --help

Build it with hdhomerun support, and some other goodies required for transcoding:

AUTOBUILD_CONFIGURE_EXTRA=" --enable-hdhomerun_client --enable-avahi --enable-hdhomerun_static --enable-libffmpeg_static" ./Autobuild.sh -t precise-amd64

This should create a tvheadend deb package a level up, which you can now install using the distro’s install tool “dpkg”:

cd ..
# dpkg -i tvheadend_<your freshly created package>.deb
# e.g.
dpkg -i tvheadend_4.0.7-11~g398e4fe~precise_amd64.deb

Now run it:

service tvheadend start

You should be able to browse to your TVH server on port 9981. eg. http://10.69.10.42:9981/

login is tvhadmin/tvhadmin.   You can update that once you’re in. Enjoy.

Keeping TVH Up to Date

Go into the /usr/src/tvheadend directory and run git pull .  After that, do a build just as you would before, and install the new deb package which will update TVH.  Simples! You should still have the previous deb package if anything mucks up, and you can apt-get remove tvheadend the current version, and then re-install the older version.

You can keep the base system up to date with:

apt-get upgrade
apt-get autoremove

 

Tvheadend vs MythTV for Kodi – TVH the Clear Winner

As you can see from my previous blogs, I’ve been playing around with MythTV of late, as a backend TV server for Kodi.  Even though I was successful in setting this up, I’ve hit some frustrating limitations and have so decided to pull the pin on that experiment, and go back to Tvheadend (TVH for short). My major annoyances are/were:

  • I couldn’t find a way to split DVB-T TV and DVB-T radio stations, the way TVH does in Kodi. With MythTV, they all appear as TV stations.
  • The ability to set channel groups seems to be lacking in MythTV.

I was looking further and further into these issues and there was some talk in some forum somewhere about being able to do these things with SQL commands in MySQL, but I figured I didn’t want to waste any more time – I’d invested far too much already! In TVH it *just works*.  Other gripes with MythTV include:

  • Since Myth’s both a front-end and back-end, there are often parts of config that relate to the front-end that I’ll never touch, so it’s a bit confused when you’re only using the backend. This is especially apparent with some iPhone apps I bought, where some screens of the app are for the backend server, and some for the frontend. The benefit of TVH is that it is a pure server – there is none of this confusion of backend vs frontend!
  • It’s pretty fiddly just to get the MythTV server running. You also have to jump through a lot of hoops to get the mythweb webserver going as well. Say goodbye to a weekend!

A fairer comparison is MythBuntu vs TVH-Ubuntu, rather than MythTV-FreeBSD.  I’ve tried MythBuntu, and set-up is a bit easier than all that work I did on FreeBSD, but it’s still kludgy to my mind, and still suffers from my major gripes with it.

I have found it to be actually quite straightforward to install TVH from a vanilla Ubuntu install, and to keep it up to date with Git pulls. I’ll add a blog post soon to show how this is done. My ideal setup would be for TVH to stabilise and add native HDHomeRun support for FreeBSD (the way MythTV manages to do!), and then have the FreeBSD people update the Ports collection with this stable code. Hopefully TVH will get there in the next year or two. Then I could run TVH on my FreeNAS/FreeBSD box, and shut down my separate Linux server.

Installing Mythweb into Your FreeNAS MythTV Jail

In my last blog I showed how I installed a MythTV backend PVR server into a FreeNAS Jail.

A must-have for usability is the Mythweb HTTP-based configuration tool. There’s a lot of hoops to jump through, but you should hopefully make it through unscathed!

Install Mythweb and Dependencies

pkg install mythplugin-mythweb

For me this says that it will install the following:

New packages to be INSTALLED:
 mythplugin-mythweb: 0.27_1
 php56-session: 5.6.7
 php56: 5.6.7
 apache24: 2.4.12
 php56-posix: 5.6.7
 php56-json: 5.6.7
 php56-mysql: 5.6.7

Since it is installing PHP 5.6, you’ll need the “56” version of mod_php as well:

pkg install mod_php56

There are important messages from the installs (shown here) which we’ll address below:

Message for apache24-2.4.12:
To run apache www server from startup, add apache24_enable="yes"
in your /etc/rc.conf. Extra options can be found in startup script.

Your hostname must be resolvable using at least 1 mechanism in
/etc/nsswitch.conf typically DNS or /etc/hosts or apache might
have issues starting depending on the modules you are using.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
- apache24 default build changed from static MPM to modular MPM
- more modules are now enabled per default in the port
- icons and error pages moved from WWWDIR to DATADIR
 If build with modular MPM and no MPM is activated in 
 httpd.conf, then mpm_prefork will be activated as default
 MPM in etc/apache24/modules.d to keep compatibility with 
 existing php/perl/python modules!
Please compare the existing httpd.conf with httpd.conf.sample
and merge missing modules/instructions into httpd.conf!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Message for mythplugin-mythweb-0.27_1:
mythweb has been installed into:
 /usr/local/www/mythweb
You need to configure mythweb now according to the instructions in
 /usr/local/www/mythweb/INSTALL
For lighttpd you should have a look at the MythTV Wiki
 http://www.mythtv.org/wiki/MythWeb_on_Lighttpd

Message for mod_php56-5.6.7:
***************************************************************
Make sure index.php is part of your DirectoryIndex.
You should add the following to your Apache configuration file:
<FilesMatch ".php$">
 SetHandler application/x-httpd-php
</FilesMatch>
<FilesMatch ".phps$">
 SetHandler application/x-httpd-php-source
</FilesMatch>
***************************************************************

Server-Wide Config

Update “/etc/rc.conf" with the following:

apache24_enable="YES"

Update “/etc/hosts" to something similar to the below – changing the parts in red to suite your IP, hostname and domain:

127.0.0.1 localhost localhost.localdomain mythserver mythserver.gavowen.local
10.69.10.22 mythserver mythserver.gavowen.local

Mythweb Config

You can choose to put your mythweb into the root of the web server, but I prefer to put it off to the side it its own “mythweb” directory.

Copy the mythweb files to where the web server can use them:

mkdir /usr/local/www/apache24/data/mythweb
cp -R /usr/local/www/mythweb/* /usr/local/www/apache24/data/mythweb

Apache 2.4 runs as user and group “www” on FreeBSD/FreeNAS, so update the permissions accordingly:

chgrp -R www /usr/local/www/apache24/data
chmod -R g+rw /usr/local/www/apache24/data

TV Icons

I copied mine into my jail, then across to MythWeb’s TV icons directory:

cp /mnt/storage/channelicons/* /usr/local/www/apache24/data/mythweb/data/tv_icons

Mythweb Apache Config File

The Mythweb Apache config file (“/usr/local/www/apache24/data/mythweb/mythweb.conf.apache“) is a bit out of date for Apache 2.4 and PHP 5.6, so pull down the latest version directly to the directory you need it in.

cd /usr/local/etc/apache24/extra/
wget --no-check-certificate https://raw.githubusercontent.com/MythTV/mythweb/master/mythweb.conf.apache
cp /usr/local/etc/apache24/extra/mythweb.conf.apache /usr/local/etc/apache24/extra/mythweb.conf

Edit the “mythweb.conf" file and change the following:

<Directory “/var/www/html/data”>
to
<Directory “/usr/local/www/apache24/data/mythweb/data”>

and
<Directory “/var/www/html”>
to
<Directory “/usr/local/www/apache24/data/mythweb”>

My NAS has plenty of RAM, so I increase the PHP “php_value memory_limit” to 256M. Save the file and quit.

Make sure these lines are uncommented:

 setenv db_server "localhost" 
 setenv db_name "mythconverg" 
 setenv db_login "mythtv" 
 setenv db_password "mythtv"

Apache Config

Backup Apache’s original config file before editing “httpd.conf":

cp /usr/local/etc/apache24/httpd.conf /usr/local/etc/apache24/httpd.conf.original

Uncomment all these mods. The first two are usually on by default:

mod_env
mod_headers
mod_rewrite
mod_deflate
mod_auth_digest

Uncomment ServerName and change it to match what you’ve done previously in “/etc/hosts" e.g.:

ServerName mythserver.gavowen.local:80

In the “Supplemental Configuration” of the file, I place the following two lines:

# Mythweb configuration
Include etc/apache24/extra/mythweb.conf

Further down you’ll find the DirectoryIndex, where I add “index.php”

<IfModule dir_module>
 DirectoryIndex index.php index.html
</IfModule>

Directly below that part I add the following:

<FilesMatch ".php$">
 SetHandler application/x-httpd-php
</FilesMatch>
<FilesMatch ".phps$">
 SetHandler application/x-httpd-php-source
</FilesMatch>

Restart Apache:

service apache24 start

Now open a browser. Enjoy.

NB: Subsequent restarts of apache can be done with “apachectl graceful"

Troubleshooting

If the page doesn’t load try this:

create user 'mythtv'@'127.0.0.%' identified by 'mythtv'; 
create user 'mythtv'@'127.0.1.%' identified by 'mythtv'; 
set password for 'mythtv'@'127.0.0.%' = password('mythtv'); 
set password for 'mythtv'@'127.0.1.%' = password('mythtv'); 
connect mythconverg; 
grant all privileges on *.* to 'mythtv'@'127.0.0.%' with grant option; 
grant all privileges on *.* to 'mythtv'@'127.0.1.%' with grant option; 
flush privileges; 
exit;

There are other possible issues addressed at this link.

Time Issue

You might get a warning such as:

User Notice at /usr/local/share/mythtv/bindings/php/MythBackend.php, line 132:
 !!NoTrans: Failed to set php timezone to AEST Response from backend was Array ( [0] => AEST [1] => 36000 [2] => 2015-04-25T10:14:06Z ) !!
 Warning: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected the timezone 'UTC' for now, but please set date.timezone to select your timezone. in /usr/local/www/apache24/data/mythweb/includes/errors.php on line 161

If that’s the case I find installing NTP a good solution:

pkg install ntp

Update “/etc/rc.conf" with the following:

ntpd_enable="YES"

Then start it with “service ntpd start". Issue “date" to see that it’s accurate. Then:

cp /usr/local/etc/php.ini-production /usr/local/etc/php.ini

search for “date.timezone” and change it to suit. For me it’s date.timezone = "Australia/Melbourne"

 

MythTV Server in a FreeNAS Jail

MythTV is is one of the most well-known open source LiveTV/PVR servers, and worth trying out and comparing to Tvheadend for serving up live TV to Kodi. I have previously given some background info on both of them. Here’s how you can install the MythTV backend in a FreeNAS jail.

Creating your Jail

First make sure your jail environment is set-up correctly:

Jail setup

I’ve carved out a range of IPs from my local LAN /24 subnet of 10.69.10.0/24

Next, go to the Jails tab and “Add Jail”. Click on “Advanced Mode” if not already.

I like to call my MythTV Jail “mythserver” – I feel this is preferable to calling it say “mythtv” because if the name is too close to the program name, it can get confusing later on in system log files.

I let it auto-select the next available free IP address, and then manually add the “IPv4 default gateway” – the router’s LAN IP which for me is 10.69.10.1.

I leave “autostart” and “VIMAGE” selected.

Once you hit “Ok” you can edit the jail you just created, and you should see that it’s added a MAC address, and for FreeNAS 9.3 it’s also added “allow.raw_sockets=true” in the “Sysctls” section.  If this is your first jail, it’ll take a while as it downloads the jail template from FreeNAS, before it finalises setting up the jail.

Jail Storage

You’ll want to map some extra storage into your jail for recordings. I give it full permissions:

mythstorage

I limit it to 40 Gigs in the Options:

mythoptions

Then I mount it into the jail:

mythmountstorage

Jail Login

SSH to your FreeNAS and list your jails:

jls

e.g.:

root@sarlacc# jls
 JID IP Address Hostname Path
 1 - dhcp_dns /mnt/volume1/jails/dhcp_dns
 2 - mythserver /mnt/volume1/jails/mythserver

My MythTV jail called “mythserver” is jail ID #2 at the moment. Log into the jail:

jexec 2 /bin/csh

Jail Update

pkg update
pkg upgrade

The “pkg update” is redundant as the upgrade does an update first, but I still like to do it regardless. Just say yes to all the upgrade prompts. You can run the same commands again after the upgrade, just to confirm the software is now current.

Bash Shell

If you prefer Bash like I do, install it tweak it.

pkg install bash
vi ~/.bashrc

and add some aliases…

# some useful aliases
alias h='fc -l'
alias j=jobs
alias m=$PAGER
alias ll='ls -laFo'
alias l='ls -l'
alias g='egrep -i'

Log out and back in with bash:

exit
jexec 2 /usr/local/bin/bash

That’s if jail ‘2’ is your myth jail. You may only need to type jexec 2. Try it and see.

Helper Software

Install “GNU Make” for compiling LAME:

pkg install gmake

Install the FreeBSD Ports tree and compile the LAME encoder:

cd /usr/ports
portsnap fetch extract
cd /usr/ports/audio/lame
make install clean
cd /

NB: Even though the directories are there, if you don’t do a fresh “extract” then the system will complain, so you have to run this the first time after creating the jail. If you are doing further installs some time in the future (say next week), all you need to run before compilation is portsnap fetch update

Install the graphical packages. You’ll need these for configuring MythTV by the setup GUI which runs on the X-Windows system. Note the upper case X in libXv:

pkg install libXv
pkg install qt4-webkit
pkg install xauth
pkg install xorg-fonts

Install MythTV

pkg install mythtv

You have time to go grab a coffee now, as it’ll take a while. You’ll see a message at the end like:

**********
MythTV has now been installed, but it still needs to be configured.
1. To create the database, use the following command:
mysql -uroot -p < /usr/local/share/mythtv/database/mc.sql
2. Next, run mythtv-setup.
See http://www.mythtv.org/docs/ for more information.
**********

You’ll get to do (1.) and (2.) later. For now, update /etc/rc.conf with the following:

sshd_enable="YES" #change the existing "NO" entry to "YES"
mysql_enable="YES"
mythbackend_enable="YES"

I found that the hostname was in that file twice, so I deleted the duplicate.

Tip: You can quickly update system variables with the following:

sysrc sshd_enable=YES
sysrc mysql_enable="YES"
sysrc mythbackend_enable="YES"

Install MyQSL

Of course we don’t have MySQL installed yet, so we need to grab that too, and start it up:

pkg install mysql56-server
service mysql-server start

Populate mysql with the MythTV database:

mysql -uroot -p < /usr/local/share/mythtv/database/mc.sql

Just hit the “enter” key at the password prompt, as a password isn’t yet set. Out of interest, this does the following:

==========
CREATE DATABASE IF NOT EXISTS mythconverg;
GRANT ALL ON mythconverg.* TO mythtv@localhost IDENTIFIED BY "mythtv";
FLUSH PRIVILEGES;
GRANT CREATE TEMPORARY TABLES ON mythconverg.* TO mythtv@localhost IDENTIFIED BY "mythtv";
FLUSH PRIVILEGES;
ALTER DATABASE mythconverg DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
==========

Login into mysql (hit enter at the password prompt) and check that “mythconverg” is in there.

mysql -p
show databases;
exit

MySQL Timezone Update

Update timezone info with:

mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root -p mysql

I found I got the “Data too long for column ‘Abbreviation’ Error” so I did the following:

mysql_tzinfo_to_sql /usr/share/zoneinfo > /tmp/timezonefix
vi /tmp/timezonefix

I searched through the file and found the stanza (parts between two semicolons) that contain “use tzsetup” and edited out the following lines:

;
INSERT INTO time_zone (Use_leap_seconds) VALUES ('N');
SET @time_zone_id= LAST_INSERT_ID();
INSERT INTO time_zone_name (Name, Time_zone_id) VALUES ('Factory', @time_zone_id);
INSERT INTO time_zone_transition_type (Time_zone_id, Transition_type_id, Offset, Is_DST, Abbreviation) VALUES
 (@time_zone_id, 0, 0, 0, 'Local time zone must be set--use tzsetup')
;

I then ran:

cat /tmp/timezonefix | mysql -u root -p mysql

This now completed without error.

Set MySQL Password

mysqladmin -u root password <yourpassword>
e.g
mysqladmin -u root password letmein

Test with:

mysql -p

You should have to put in that password now. Type “exit” to quit out.

Setup X11 Forwarding

Edit /etc/ssh/sshd_config with the following updates, replacing the IP address below with your IP:

ListenAddress 10.69.10.22
PermitRootLogin yes
PasswordAuthentication yes
X11Forwarding yes
X11UseLocalhost no

Now start sshd:

service sshd start

This generates keys and starts the SSH server. I like to test the server with a  service sshd restart , to make sure it starts cleanly now that the keys are present.

Set a root password, and test the password along with external connectivity to the mythtv-setup script. If all’s successful, it’ll time out and say that it cannot connect to the X server. That’s the desired outcome, as you want to connect to an X server on another machine. Remember to update the IP address below to that of your jail.

[root@mythserver /]# passwd 
Changing local password for root
New Password:
Retype New Password:
[root@mythserver /]# exit
[root@mythserver /]# ssh -Y root@10.69.10.22 /usr/local/bin/mythtv-setup
Password:
mythtv-setup: cannot connect to X server

Setup Remote X Server

Download Xming from here:
Download PuTTY from here:

1) Install Xming server and start it
2) Install PuTTY on your local machine
3) Configure the default profile in PuTTY with the IP and SSH port of your MythTV Server
4) Go to the Connection panel then select ssh and enable compression, then the x11 tab and check “Enable x11 Forwarding” box, then enter localhost:0 in the “X Display Location” and check “MIT-Magic-Cookie-”
5) Go to “Session”, type “MythTV Server” in the Saved Sessions, and hit the Save button.

Configuring MythTV

With the Xming server started, open the PuTTY session you saved and login (with the user you created earlier) to the myth server, then type….

mythtv-setup

This will open an Xwindow on your local screen and load the myth setup screen. You might have to run this a second or third time, as it might quit after a country and language update, and also after a database schema update. You’ll get there eventually.

I recommend you follow the MythBuntu setup guide here, skipping right down to the “MythTV Backend Setup” section, and taking it from there.

Start the Backend

The backend service now needs to be started if not already:

service mythbackend start

Also make sure it’s filled with your config settings:

mythfilldatabase

Congratulations – you’ve now configured MythTV backend on FreeNAS in a jail.

Troubleshooting

If you have any issues, check where the logserver stores its files and check the logs.

[root@mythserver /var/log/mythtv]# ll
total 12988
drwxr-xr-x 2 root wheel - 5 Apr 21 03:00 ./
drwxr-xr-x 3 root wheel - 22 Apr 21 03:01 ../
-rw-r--r-- 1 root wheel - 219163 Apr 21 03:00 mythbackend.20150420163447.83755.log
-rw-r--r-- 1 root wheel - 12917053 Apr 21 19:40 mythbackend.20150420170022.85871.log
-rw-r--r-- 1 root wheel - 4998 Apr 21 03:00 mythlogserver.20150420163448.83757.log
[root@mythserver /var/log/mythtv]#

tail the latest backend log file:

[root@mythserver /var/log/mythtv]# tail -5 mythbackend.20150420170022.85871.log
2015-04-21 19:41:31.854481 E [85871/101782] HttpServer107 servicehost.cpp:426 (ProcessRequest) - No Security Pin assigned. Run mythtv-setup to set one.
2015-04-21 19:41:31.862285 I [85871/101739] ProcessRequest mainserver.cpp:1420 (HandleAnnounce) - MainServer::ANN Monitor
2015-04-21 19:41:31.862295 I [85871/101739] ProcessRequest mainserver.cpp:1422 (HandleAnnounce) - adding: kodi as a client (events: 0)
2015-04-21 19:41:31.875402 E [85871/101782] HttpServer114 servicehost.cpp:143 (Invoke) - MethodInfo::Invoke - An Exception Occurred: No Security Pin assigned. Run mythtv-setup to set one.
2015-04-21 19:41:31.875436 E [85871/101782] HttpServer114 servicehost.cpp:426 (ProcessRequest) - No Security Pin assigned. Run mythtv-setup to set one.
[root@mythserver /var/log/mythtv]#

ooops – didn’t add a security pin – have to set that up!

Other tips:

  • Go to another box and “telnet <your myth IP> 3306” to make sure your MySQL database is working and allowing connections.
  • Got to http:<your MythTV IP>:6544  to make sure you have web connectivity
  • All else fails – RTFM: MythTV Wiki, and the User Manual.

Other Tasks

It’s advisable you setup MythWeb for managing your TV recordings.

A Breakdown of Kodi LiveTV PVR Backends for FreeNAS

I use the awesome open source Kodi media centre for all my movies and TV shows, so it’s only natural that I use it as a live TV frontend. Since I already have an equally awesome FreeNAS NAS, then it makes sense to see if I can utilise its “Jails” functionality to run the live TV / PVR backend, instead of utilising a separate box. It makes sense – the NAS has a grunty CPU; teamed 1Gig Ethernet links; 16Gig of RAM; loads of storage space for recording TV, and is always on – so it’s the perfect candidate, so long as I can get the backend software to run!

This page currently lists four backends for Linux:  TVheadendMythTVVDR and DVBLink.  Out of those four, only the first three are possible under FreeBSD, as DVBLink is commercial software with only binary code downloads for Linux (among others), but not FreeBSD. The others have downloadable source code which can be compiled on FreeBSD. Discussing them further:

Tvheadend

Tvheadend (aka “TVH”) is basically the unofficial TV backend for Kodi. One distribution – OpenELEC – even includes the server in it’s distribution (although SoC systems like RaspberryPi miss out). I was actually using the inbuilt TVH server in OpenELEC before going to a RaspberryPi2 front end, which cut down noise in my lounge room by not having to run a PC with fan near my TV.

On their currently out-of-date website they say “With the integration of PVR functionality into XBMC [kodi], this has now become possibly the most popular TVH client. Indeed much of the recent development of HTSP has been focused on improving integration with XBMC [kodi].

Although the TVH devs don’t much update the website, the actual code is coming along in leaps and bounds with a lot of active development over at GitHub. From what I can gather here, this code was at version 3.4 when one developer by the name of Adam Sutton did a lot of work up till that point, but with a young family he found it hard to continue to commit to the project. That branch is currently known as the “stable” branch. Thankfully another team of devs lead by Andreas Öman came to the fore, and the project was fully opened up as an open source project, and the current code being updated to 3.9.x and is deemed the “unstable” branch. Still this is the branch that you want to be using as it has the transcoding, timeshifting (pause/rewind live TV) and other goodies, and it’s what you’ll get from Github.

Hopefully the 3.9 branch stabilises into 4.0 soon, and FreeBSD ports update their code, and the iPhone TvhClient gets updated as well so it supports all the 3.9 features like transcoding. This does seem to be the backend for Kodi users to watch in the future.

The problem main problem with Tvheadend on FreeBSD is seemingly no support for the SiliconDust HDHomeRun. According to this page, the way TVH supports HDHomeRun is with a Linux “dvbhdhomerun” driver, which only works on Linux as far as I’m aware. This really is a deal-breaker for me, as the HDHomeRun is what I use, and FreeBSD/FreeNAS is my preferred server platform. I really hope they can find some other way to support HDHomeRun on the *BSD’s.

MythTV

This one has been around for years, and has the most tedious set-up process, although is arguably the most flexible. On a generic server or ESXi, you can install the MythBuntu distribution, which makes install a lot easier, but on FreeBSD in a headless server, there’s a lot more heavy lifting.

I recommend setting up a MythTV jail, even if you have Tvheadend installed, and see which one works best for you.  The good thing about the FreeBSD jails in FreeNAS is that you can set them up without stepping on each other’s toes, and it’s a fun tech exercise. The only thing with MythTV is that it engages all the HDHomeRun tuners at all times even when not watching TV, so you’ll need to shut that jail if you’ve got a Tvheadend jail running and are trying that out. Tvheadend is a lot friendlier in this regard – it only uses a HDHomeRun tuner when it needs one, so most of the time it’s either using none or one tuner, and only using more than one tuner if you’re watching live TV on one mux, and recording on another mux (or recording two shows simultaneously on two different muxes).

VDR

Well I can’t say I’ve actually used this for Kodi. I’ve stayed away mostly because of what I perceive as limited support for the HDHhomeRun network TV tuner. A web search for “VDR HDHomeRun” shows up http://www.fepg.org/hdhomerun/. This isn’t very confidence-inspiring because if you look at the history, it shows it’s at version 0.0.1 last updated five years ago in 2010! Yes I think I’ll give this one a miss for now.

Summary

If it wasn’t for wanting to use the HDHomeRun I’d probably use Tvheadend for its ease of use, but since I do, my only choice is to use MythTV as a TV/PVR backend for Kodi. I’ll put together an install guide on my blog for MythTV, and also add some info into my wiki.

OpenELEC + HDHomeRun + Tvheadend

Update March 2015 – Some of this info is out of date. Will update the info, and put a good guide on the wiki.

 

Intro

I’ve recently set up the OpenELEC (XBMC/Kodi based) entertainment centre with HDHomeRun as the TV tuner, and fed into the system with the Tvheadend backend and front end.

This is a quick guide to setting it up, and hopefully stop you from depriving yourself of sleep (like I did) as you bang away getting it sorted.

Props to: Code Bytes for getting me started!

Prerequisites

  • You’ve got XBMC/Kodi set up, and you have a skin that supports live TV.
  • You have your HDHomeRun plugged into your home network and with an antenna feed into it.

OK let’s do this.

Hardware

Don’t assume that the HDHomeRun signal is fine! These boxes need a strong signal to drive them – something stronger usually that what your TV needs to get a decent picture.

I had to take the antenna cable into an amplified splitter. Without it, I found that (in Australia) my channel 10 was a bit iffy, and ABC was poo, and community channel 31 couldn’t even be found.

The amp I got is a GME  Kingray 162.  It cost me about $50 from Dick Smith Electronics. It is an outdoor unit and supposed to be mounted on the antenna as close to the antenna as possible. I’m in a rented unit so don’t want to get on the roof and muck around with the antenna so I have it inside where the cable comes in. The lower down it is – the more noise you amplify. Still it did the trick for me, where it amplified the signal enough for the HDHomeRun to run clean and with digital – that’s all that matters. 🙂

Some tips

  • Always use the F-type screw in cables where possible and avoid the PAL/Bellings Lee adapters.
  • Experiment with cables. I found that for some reason, the cable that came with the HDHomeRun worked better that the new Dick Smith cable I bought.
  • Run the Windows software to gauge differences before and after amplification, and with different cables. Use the very best cable from amp to HDHomeRun.
  • Do a scan with the Windows software (or Mac software) and see that all is sweet before you start blaming XBMC/Kodi for your woes!

Add-Ons

Install these plugins. They are both in the OpenELEC standard repository.
HDHomeRun
TVHeadEnd

Config

HDHomeRun

Here’s the tricky bit, and where I banged my head for ages! The first thing you want to do is configure HDHomeRun to be setup as a working source of TV, so it looks to the Linux OS like it’s attached to the box and so the Tvheadend can use it!

Now I was screwing around with this late at night so I don’t know if some steps are necessary but when you test your setup you’ll know I guess.

Start by running the configuration for the “hdhomerun” add-on
System -> Add-ons -> Enabled  -> Program  -> hdhomerun

General
 -- Pre-wait time [sec]    "2"  [default]
    -- Post wait time [sec]   "1"  [default]
    -- Enable userhdhomerun logging   "on" [changed]
    -- Enable libhdhomerun logging   "on" [changed]
    -- Enable suspend/resume the driver "off" [default]
Tuner Settings
    -- Enable modifying settings    "yes"  [changed?]

Now hit the “Refresh tuners”… and your tuner should show up. Set it to DVB-T, use full names to whatever you like (I choose “off”), and set number of tuners to 2, if it has 2. Don’t disable….that would not be sensible.

It’s very important to enable the logging above, and I’ll explain why. You see, HDHomeRun has a userspace component and a kernel component. The “userhdhomerun” userspace program scans the network to find the devices on the network, and then the “libhdhomerun” sets up kernel devices in /dev

OpenELEC:~ # ls /dev/dvb/
 adapter0  adapter1
 OpenELEC:~ #

If things aren’t sweet, you won’t see those devices. Now these two programs have to run and setup the devices before Tvheadend runs, or you won’t see the devices to choose from the Tvheadend web interface, that I’ll get to later.

Fix for missing adapters

The HDHomeRun add-on wiki page it says, “This driver is started from tvheadend and vdr addon”. Well Tvheadend may do this once you’ve got HDHomeRun configured already, but I found in order to get the two HDHomeRun tuners to show up on the Tvheadend web interface in the initial step, then I had two options:

  • Install and enable “vdr” add-on, which I didn’t need; or
  • enable logging for “userhdhomerun” and “libhdhomerun”

I chose the latter. It seems that enabling logging starts those processes up before Tvheadend starts, which is what you want. If the adapters weren’t there and I ran “userhdhomerun” on the CLI, then the adapters would show up in /dev/dvb/ but Tvheadend wouldn’t use them, as they weren’t there when it scanned for /dev/dvb/ devices! I lost a lot of sleep because of this, and was fully relieved when enabling logging solved my issues! (but I still had a reboot issue… see below).

Playing with config files

\[kodi IP address without brackets\]Userdataaddon_datadriver.dvb.hdhomerun

In there, there should be the following files – all created by the setup tool

  • adapters.txt
  • dvbhdhomerun.conf
  • dvbhdhomerun.sample
  • settings.xml

I didn’t need to create any of those. If something goes wrong and you, for whatever reason, need to hack on this manually then grab the dvbhdhomerun.sample from /storage/.xbmc/addons/driver.dvb.hdhomerun/config and copy it into that network share. Then copy that file to the same network share and rename it “dvbhdhomerun.conf” which is the config file it’ll use. My conf file has some comments and the meat of it says:

[1110AE0D-0]
 tuner_type=DVB-T
 use_full_name=true
 [1110AE0D-1]
 tuner_type=DVB-T
 use_full_name=true
[libhdhomerun]
 enable=true
 logfile=/var/log/dvbhdhomerun_libhdhomerun.log

….. where 1110AE0D is the “Device ID” you’ll find printed on the device, and which can also be found by running “userhdhomerun” on the CLI.

My “adapters.txt” says
“Tue Nov 25 13:21:22 2014 Name of device: 1110AE0D-0 Tue Nov 25 13:21:22 2014 Name of device: 1110AE0D-1”

…which was created automatically for me.

TVheadend

Go to webpage of TVHeadend: http://[kodi IP address without brackets]:9981
Go to Configuration -> DVB Inputs -> TV Adapters

Select the first adapter.
Click “Add DVB Network by Location”
Choose your location
Click “Enabled”
Click “Save”

TVheadend now starts scanning for channels. You can monitor its progress in the pane to the right. When “Muxes awaiting initial scan” is zero, it’s done.
Click “Map DVB Services to Channels”.
The mapping will also take some minutes to complete depending on how many channels are present.

In the web config in Configuration -> TV Adapters: After all channels have been configured, disable “Autodetect Muxes” and “Idle Scanning“. Having these enabled completely destroyed the stream from TVHeadEnd, making the image look garbled and stuttering. It looked like a low bandwidth connection or bad signal.

Missing Channels?

I had an issue where the muxes for “Australia -> au_Melbourne” didn’t have the newly added “C31” community channel, and the changed SBS settings. Doh!

What I had to do was a full channel scan by deleting all the existing Multiplexes, then choosing the “Add DVB Network by Location” and choosing the “–Generic–” -> “auto_Australia”. This does a fairly exhaustive scan off all the muxes devoted to DVB in Australia – 100 at the moment! Grab a coffee and do something else for a good half an hour as it takes a while!

Then when done, over on the Multiplexes tab, select all the muxes with no IDs and delete them. I was left with:

  • Seven Network
  • SBS Melbourne
  • Nine Network Australia
  • Network TEN
  • ABC Melbourne
  • C31

…yay they are all there!

I ended up with 35 “services”. One was bogus in that it had no details at all. So we’re left with 34 services. A few of those were duplicates, so we’re left with 31 *actual* services. 5 of those are radio (ABC Jazz, Double J, SBS Radio 1, SBS Radio 2, and SBS Radio 3), and 26 free to air TV channels.  You don’t have to worry about removing those 3 duplicate TV channels – the front end seems to filter out the dupes.

You should now be able to go to the front end and filter out the garbage shopping channels, and rearrange your channel order.

Things Messed on Reboot?

I found that every time I rebooted, I had a message about there being no hardware found, and couldn’t play live TV. The fist was to go back into the HDHomeRun add-on settings and set the “Pre wait time [sec] from “2” to “7”, which gave HDHomeRun more time to initialise. I found “5” didn’t work, and “7” did. I didn’t try “6” but even if it did work, I prefer to give it that extra second to be on the safe side.

I also found I could untick the logging options on HDHomeRun which I’d setup earlier.

TVheadend Tweaks

Channel Cleanup

Go to the “Channel / EPG” and delete all the home shopping channels. I also deleted ABC3 (kids shows), and SBS3 (duplicate of another SBS channel).

Time shifting

Go to Recording / Timeshift, and enable this, and up the max size to something a lot bigger. I’ve gone with “10240” (i.e. 10 gigs) for now. Make sure you have a SSD drive for maximum smoothness. This will allow you to pause and rewind live TV.

Channel Icons

I found some great TV logos / Channel Icons for Australian stations here:
You need to make sure that the image file names are exactly the same name as channel names – including case. For ABC2 / ABC4 – that contains a “/” in the name so for that one you have to manually pressing  “C” on a keyboard attached to your OpenELEC box and then select “Choose Thumbnail”.

Future Versions

I have been reading the forums and see that OpenELEC and Kodi 14 are now in beta, with a final release probably available by early 2015. This has a much updated version of Tvheadend that includes direct support for HDHomeRun devices, and also promises a lot of improvements for TV/PVR functionality, so much of what I’ve written above will change.

Still, there’s a lot of code changes under the hood with all the XBMC to Kodi name changes, so I’d suggest backing up your complete system before making the jump! I personally will wait for a couple of point release versions of openELEC before making the jump to OpenELEC 5.x (based on Kodi 14.x).

To Google Apps and Back

I’m just in the process of migrating my Gmail back to the free service from the paid Google Apps service. Why did I setup Google Apps to begin with?
Answer:

  • To see what features it has
  • To see how easy it is to setup
  • To see what advantages it has over standard Gmail
  • To indulge my curiosity

I have my own domain for my email “gavowen.com” for sites that don’t allow “@gmail.com” addresses (mostly e-commerce sites – strange I know!) When I upgraded my iPhone to a 5S, I saw that I could no-longer get the push email using the Exchange client in the iPhone. This had been grandfathered and would no longer be supported for the free Gmail service (even though Apple iCloud does push – the competition has one-upped Google on this now).

I wanted to keep this functionality, and I also believed (at the time) that I needed to setup Google Apps in order to send from a @gavowen.com address. After I setup Google Apps I realised this was not the case (and really was my primary reason for setting it up!)

 

DNS Changes

This is what I had to do to setup Google Apps with my gavowen.com zone:

CNAMES

calendar ghs.googlehosted.com 14400
docs ghs.googlehosted.com 14400
sites ghs.googlehosted.com 14400
mail ghs.googlehosted.com 14400

Although not strictly necessary, they make for nice logins for email which would now be customised as “mail.gavowen.com”. Likewise for the other features of docs, calendar and sites.

MX

1 @ aspmx.l.google.com 14400
5 @ alt1.aspmx.l.google.com 14400
5 @ alt2.aspmx.l.google.com 14400
10 @ aspmx2.googlemail.com 14400
10 @ aspmx3.googlemail.com 14400

This gives the primary and backup mail exchanges for Google Apps

TXT

_domainkey o=~ 14400
@ google-site-verification=FX4zlpoBR3ZTOPH6GLnNoPx3skkPuuvDT0kKD21QJqg 14400
@ v=spf1 include:_spf.google.com ~all 14400

The above is for email security

SRV

_xmpp-server _tcp 5 0 5269 xmpp-server.l.google.com 14400
_xmpp-server _tcp 20 0 5269 alt1.xmpp-server.l.google.com 14400
_xmpp-server _tcp 20 0 5269 alt2.xmpp-server.l.google.com 14400
_xmpp-server _tcp 20 0 5269 alt3.xmpp-server.l.google.com 14400
_xmpp-server _tcp 20 0 5269 alt4.xmpp-server.l.google.com 14400

These are to aid Google Chat that uses XMPP protocol.

 

Chat Client

Speaking of chat client, Pigeon didn’t work for me until I set “Require encryption” and set the Connect server to the following:

talk.google.com     on port 5222

 

Migrating Email

I used the very convoluted Google method for migrating all my Gmail email to Google Apps.  This doesn’t do contacts and calendars – you have to do those manually yourself!

If you are struggling with the csv file format for the migration the see here:
userlist.csv
<gmail address>#<old password>, <google apps email address>

It’s just one line per account. Don’t inclue the ‘<‘ and ‘>’

In future I think I’ll pay money and use this service: https://www.migrationapp.com/ which I used that to migrate back to Gmail and it was very hassle free, and did calendaring and contacts as well!

 

Why leave Google Apps?

Well besides the money factor, I ran into a snag – I wanted to set up youtube but couldn’t. After many hours of frustration I logged a tech support ticket.  Here’s what I got back:

Thank you for your message. I understand that you are trying to enable the Youtube service on your domain gavowen.com.I have investigated this for you and have found that the Youtube for Business service is currently not available on your Google Apps for Business account. The reason that it is not available is because your default country was selected as Angola when you initially set up your account. The Youtube for Business service has not yet been released in Angola and as a result is not available to enable on your Admin console.

There is currently no way in which you can change your default country in your Admin console. To do this you would need to delete your Google Apps for Business account and set it up again using a different Default country. If you do follow this method you will also need to migrate the data you require away from your Apps account as once you delete this your data will be completely removed from the system.

Yes – when I set up my Google Apps I chose the Angola and Not Australia. Hmmm don’t know what happened there! Must have been a PEBKAC issue! Still it’s a paid they couldn’t change it – so had to back out.I don’t think I’ll re-register Google Apps. I’ve had a poke around. Besides 30gigs instead of 15 and push email, I didn’t see any other benefits. I’ll stick with the free Gmail. I use the great “Mailbox” iphone app now to manage my mobile email anyways.

Still it was fun.