Android RSS Video Feed

From MythTV Official Wiki
Revision as of 03:07, 27 December 2010 by Allenmelon (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Fail whale.jpg FAIL: When using languages that have native MySQL modules, don't ignore all that hard work and use the command line `mysql` client instead.

List-add.png Todo: Script needs to be updated to use Perl bindings for database credentials.

The new android devices are a great smartphone. The Motorola Droid has a nice display and 3G makes streaming quite possible. This HOWTO describes the steps necessary to get your server to stream video to your Android.


Assumptions

In designing this solution, you may be presented with various options for solving this problem yourself. As such, these directions may not apply directly to your situation. Thus, your mileage may vary for this HOWTO, but the general plan should work.

  • Server: The guide will assume Ubuntu 9.10, but I don't see why any others wouldn't work. You just may need to tweak the packages you install and the configurations for some of the commands.
  • MythTV: This guide will assume version 0.22 running on the back-end.


Introduction

There are three main tasks that need to be solved in getting your recorded shows to stream on your android:

  1. Get an RSS application that will play streaming video for your Android device.
  2. Make an RSS feed available to your Android device via the Internet.
  3. Create a job that will make your recordings into mp4s that can be streamed on your mp4 device.

RSS Android Application

I chose using RSS to present the recordings because I figured it would be simple to find an application that already exists that could do it. After some time trying out different applications on the market, Blue RSS. Your can find it by searching for Blue RSS on the market. It has a simple interface and will use Android's default media player for your video feeds.

Once you have your RSS feed set up, you'll have a url like: http://www.mydomain.com/feeds/recordings.xml. Menu > Add Feed will allow you to input your feed and your set up! Simple, eh? The rest is a bit more tricky (but not much).

Creating an RSS Feed

Here is the code for creating your RSS feed.


Script.png android_rss.pl

#!/usr/bin/perl
#
# Create Date: 2010-02-25
# Last Updated: 2010-02-25
# Author: Joshua Marsh
#
# Copyright (c) 2010, Joshua Marsh <joshua@themarshians.com>
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
#    * Redistributions of source code must retain the above copyright
#      notice, this list of conditions and the following disclaimer.
#    * Redistributions in binary form must reproduce the above
#      copyright notice, this list of conditions and the following
#      disclaimer in the documentation and/or other materials
#      provided with the distribution.
#    * Neither the name of Joshua Marsh nor the names of its
#      contributors may be used to endorse or promote products
#      derived from this software without specific prior written
#      permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

use strict;
use feature qw(say);

####### Global Variables #######
my $mythdir = '/data/mythtv';
my $feedsdir = "$mythdir/feeds";
my $outdir = "$feedsdir/recordings/";

my $mysql = 'mysql';
my $user = 'myth';
my $pass = 'myth';
my $db = 'mythconverg';
my $opts = '-Ns';
my $mysqlcmd = "$mysql $opts -u$user -p$pass -D$db -e ";

my $url = "http://www.mydomain.com";

# Get a list of the recorded programs.
my @programs = get_program_list ();

# Get a list of the files in the output directory.
my @encoded = get_files_in_output_dir ();

# Delete files in the video directory that are no longer in the
# recorded programs list.
clear_old_encodings (\@programs, \@encoded);

# Print out the header information for the RSS Feeds.
print_header ();

# Loop through each of the recorded programs.
print_dir_to_xml (\@programs, 'recordings', 'mp4');

# TODO Add video, mp3s, etc here.

# Print out the footer information for the RSS Feeds.
print_footer ();

####### Helper Functions #######
sub get_program_list
{
    my $query = 'SELECT CONCAT_WS(\'_\', chanid, DATE_FORMAT(starttime, \'%Y%m%d%H%i%s\')) FROM recorded WHERE recgroup <> \'LiveTV\';';
    my @list = `$mysqlcmd "$query"`;

    chomp (@list);

    return @list;
}

sub get_files_in_output_dir
{
    my @list = `ls $outdir | grep -v "xml" | sed -e 's/\.mp4//g'`;

    chomp (@list);

    return @list;
}

sub clear_old_encodings
{
    my $programs = shift;
    my $encoded = shift;

    for my $encodee (@$encoded) 
    {
	my $found = 0;
	for (@$programs) {
	    if ($_ eq $encodee) {
		$found = 1;
		last;
	    }
	}
	
	if (!$found) {
	    print STDERR "WARN: '$encodee' is no longer in the database. " .
		"Deleting it.\n";
	    `rm -f $outdir/$encodee.mp4`;
	}
    }
}

sub print_header
{
    say '<?xml version="1.0" encoding="UTF-8"?>';
    say '<rss version="2.0">';
    say '<channel>';
    say '<title>My MythTV Smartphone Recordings</title>';
    say '<description>My RSS feed of available video feeds.</description>';
    say '<link>'. $url  .'</link>';
    say '<language>en-us</language>';
}

sub print_footer
{
    say '</channel>';
    say '</rss>';
}

sub get_program_info
{
    my $progID = shift;

    my $query = "SELECT title, subtitle, description, TIMESTAMPDIFF(MINUTE, starttime, endtime) AS duration, category, starttime FROM recorded WHERE CONCAT_WS('_', chanid, DATE_FORMAT(starttime, '%Y%m%d%H%i%s')) = '$progID';";

    my $data = `$mysqlcmd "$query"`;

    chomp $data;
    my @vals = split /\t/, $data;

    my %info = (
	title       => $vals[0],
	subtitle    => $vals[1],
	description => $vals[2],
	duration    => $vals[3],
	category    => $vals[4],
	date        => $vals[5],
	uid         => $progID
    );
    
    return \%info;    
}

sub print_program_info
{
    my $pi = shift;
    my $subdir = shift;
    my $extension = shift;

    say "<item>";
    say "<title>$pi->{'title'} - $pi->{'subtitle'}</title>";
    say "<link>$url/$subdir/$pi->{'uid'}.$extension</link>";
    say "<guid>\"$url/$subdir/$pi->{'uid'}.$extension\"</guid>";
    say "<description>$pi->{description}</description>";
    say "<enclosure url=\"$url/$subdir/$pi->{'uid'}.$extension\" length=\"$pi->{duration} min\" type=\"video/mp4\"/>";
    say "<category>$pi->{'category'}</category>";
    say "<pubDate>$pi->{'date'}</pubDate>";
    say "</item>";
}

sub print_dir_to_xml
{
    my $files = shift;
    my $subdir = shift;
    my $extension = shift;

    foreach (@$files)
    {
	chomp;
	
	# Verify the file exists
	if (!-e "$feedsdir/$subdir/$_.$extension")
	{
	    print STDERR "ERROR: '$feedsdir/$subdir/$_.$extension' does not exits. Skipping.\n";
	    next;
	}
	
	# Get the information about the file from the DB for the RSS Feeds.
	my $program_info = get_program_info ($_);
	
	# Print out the information in XML format.
	print_program_info ($program_info, $subdir, $extension);
    }
}

The global variables are used to describe where the mpg's are stored and the mp4s will go. It also contains the mysql credentials and the url to your web site. You should change these to fit your needs. You may also want to modify the contents of the print_header sub-routine. It has no real bearing on the feed, but will make your Blue RSS display useful information.

The effect of this scripts is fairly simple. It uses the myth database to find files that you've set to record. It then checks the place where your mp4's are stored. If there are mp4's that don't have a record anymore, they are cleaned up (I presumed I wouldn't want files that I've watched and deleted. If you don't feel the same way, comment out the clear_old_encodings line.). Then it prints out the RSS XML feed to stdout.

By doing this, you can create a cron job to redirect the RSS XML feed to the location of your web directory. My cron job looks like this:

0 */4 * * * /var/www/icub3d/application/scripts/android_rss.pl >/data/mythtv/feeds/recordings.xml

With the RSS feed in place, we just need to make sure apache is serving up the feed appropriately. I set up the script so that you could eventually add different feeds (perhaps to your mp3's or videos). Each of the types can have their own sub-directory. The default is recordings. So, a sample feed might look like:


Script.png recordings.xml

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
<channel>
<title>My MythTV Smartphone Recordings</title>
<description>My RSS feed of available video feeds.</description>
<link>https://www.mydomain.com/</link>
<language>en-us</language>
<item>
<title>Oprah Winfrey - </title>
<link>http://www.mydomain.com/recordings/1021_20100226165300.mp4</link>
<guid>"http://www.mydomain.com/recordings/1021_20100226165300.mp4"</guid>

<description>Oprah discusses topics with guests, before a studio audience.</description>
<enclosure url="http://www.mydomain.com/recordings/1021_20100226165300.mp4" length="7 min" type="video/mp4"/>
<category>Talk</category>
<pubDate>2010-02-26 16:53:00</pubDate>
</item>
</channel>
</rss>

What this means is, you'll want to make sure the place where your mp4's are stored are available by in the /recordings/ directory of your web server. For my setup, my recordings were in /data/mythtv/feeds/recordings/, so I simply made a symbolic link to the root of my web server directory.

I was interested in really locking down my files and access to my RSS feed, so I created a PHP page that would authenticate the user via GET values and forced SSL to see the feed. This meant that the RSS feed wouldn't be available unless the right user name, password, and type are selected. I then added the -Indexes option to the recordings directory in the apache config. You are welcome to do something similar, but it's out of the scope of this document.

Set up a User Job

At this point, the only thing we need to do now is make sure our record files get transcoded to a streamable mp4 that the Android can read. Here is how I did it:


Script.png mp4.sh

#!/bin/bash

if [ ! $# == 1 ]; then
  echo "Usage: $0 FileID"
  exit
fi

# The Unique FileID from the database (chan_id + '_' + startdate)
FID="$1";

MYTHDIR="/data/mythtv"
OUTDIR="$MYTHDIR/feeds/recordings"

INFILE="${1}.mpg"
OUTFILE="${1}.mp4"

INPATH=`find $MYTHDIR -name "$INFILE" 2>/dev/null | head -n1`;

# Figure out what dimensions we should encode it in via the aspect.
VINFO=`mplayer -vo null -ao null -frames 10 -identify "${MYTHDIR}/${INFILE}" 2>/dev/null`
ASPECT=`echo $VINFO | egrep -oe "Movie-Aspect is [0-9:.]+" | egrep -o "[0-9:.]+"`
if [ "$ASPECT" == "1.78:1" ]
then
  WIDTH=432
  HEIGHT=240
else
  WIDTH=480
  HEIGHT=360
fi

VBITRATE=248k
SIZE="${WIDTH}x${HEIGHT}"
ABITRATE=128k
ACODEC="-acodec libfaac"

# Create the MP4
ffmpeg -i "${INPATH}" -s ${SIZE} -b ${VBITRATE} -f mp4 "${OUTDIR}/${OUTFILE}" 
MP4Box -tmp "${MYTHDIR}/tmp/" -hint "${OUTDIR}/${OUTFILE}"

The configurations for this file are similar to the perl script. Make sure the directory variables fit your system and that they are writable by the mythtv user.

This shell script has a few interesting parts. First, it tries to figure out if the video is in 4:4 or 16:9. It uses that to figure out the right dimensions for the video. In my mind, the less we have to transcode and transmit over the Internet the better. The bitrate is also set kind of low. This was intentional though. If you think you can get more bandwidth out of your home system, you can up the rate.

The second thing it does is transcode the file into an mp4 with the new size and bitrate. ffmpeg does the job well. You may have to install special audo or video codecs for it. Ubuntu installs most of these from the Medibuntu repos. libfaac for example, was a good codec. I never ended up using it though as ffmpeg seemed to do a good enough job on it's own. On Unbuntu 10.04 I had to change the line to ffmpeg -i "${INPATH}" -s ${SIZE} -b ${VBITRATE} -f mp4-ac 2 -ar 24000 -ab 65535 "${OUTDIR}/${OUTFILE}" to get it to work with my DVB recordings.

If you did nothing else, you could play the mp4 on your Android device by copying it over, but you won't be able to stream. The problem is that ffmpeg doesn't put the hints for skipping around in the video up front. MP4Box will do that for us though. Ubuntu user can get MP4Box by installing the gpac package.

You should now make this one of your user jobs. It should be run like this:

/path/to/script/mp4.sh %CHANID%_%STARTTIME%

Make sure that your job is enabled. I set mine to run by default on all recordings. You should also make sure your job is set as being able to run. My myth back-end had it shut off, even though the UI had it configured and set to auto-run. It took some time for me to figure out why it wouldn't run automatically.

Notes

There are still some tweaks I need to make to my system. One problem I've noticed is that ffmpeg will occassionally (about 1 in 5) offset the audio by a half a second. It's nearly unnoticable at low video bit rates, but at the higher ones it gets pretty annoying.

Also, your video sizes may change when you are transcoding. These settings work well on the Motorola Droid, but I'm not an A/V expert. If you have better settings, I'd be glad to hear it.