Difference between revisions of "Mythfs.pl"

From MythTV Official Wiki
Jump to: navigation, search
(mythtvfs-perl)
Line 7: Line 7:
 
|S26=yes}}
 
|S26=yes}}
  
=mythtvfs-perl=
+
=mythfs-perl=
  
This is a FUSE filesystem for MythTV (www.mythtv.org) which mounts the recordings on a Mythtv backend as files and directories on the same or different machine. It is distinguished from [[Mythtvfs]] and [[Mythfs.py]] in that it works via the Myth 0.25 REST API. Hence the recordings do not need to be on a shared
+
This is a FUSE filesystem for MythTV (www.mythtv.org). It uses the
 +
Myth 0.25 API to mount the TV recordings known to a MythTV master back
 +
end onto a virtual filesystem on the client machine for convenient
 +
playback with mplayer or other video tools. Because this uses the
 +
MythTV network protocol, the recordings do not need to be on a shared
 
NFS-mounted disk, nor does the Myth database need to be accessible
 
NFS-mounted disk, nor does the Myth database need to be accessible
from the client. It also uses threading, providing good performance when accessing many files.
+
from the client.
  
 
==Installation==
 
==Installation==
Line 18: Line 22:
 
distribution:
 
distribution:
  
  $ '''git clone git://github.com/lstein/mythfs-perl.git'''
+
  $ <b> git clone git://github.com/lstein/mythfs-perl.git</b>
  $ '''cd mythfs-perl'''
+
  $ <b>cd mythfs-perl</b>
  $ '''./Build.PL'''
+
  $ <b>./Build.PL</b>
  $ '''./Build test'''
+
  $ <b>./Build test</b>
  $ '''sudo ./Build install'''
+
  $ <b>sudo ./Build install</b>
  
 
If you get messages about missing dependencies, run:
 
If you get messages about missing dependencies, run:
  
  $ '''./Build installdeps'''
+
  $ <b>./Build installdeps</b>
  
and then "sudo ./Build install".
+
and then "sudo ./Build install". See Fuse Notes if you get
 +
Fuse-related errors when attempting to install dependencies.
 +
 
 +
For best performance, Perl must have been compiled with support for
 +
IThreads. In addition, you will need at Fuse version 0.15 to run
 +
correctly under Perl version >= 5.14.  Threading will be automatically
 +
disabled if not available. See Fuse Notes for additional information.
  
 
==Usage==
 
==Usage==
Line 35: Line 45:
 
a local filesystem named "/tmp/mythfs" use this command:
 
a local filesystem named "/tmp/mythfs" use this command:
  
  $ '''mythfs.pl MyHost /tmp/mythfs'''
+
  $ <b>mkdir /tmp/mythfs</b>
 +
$ <b>mythfs.pl MyHost /tmp/mythfs</b>
  
 
The script will fork into the background and should be stopped with
 
The script will fork into the background and should be stopped with
 
fusermount. The mounted /tmp/mythfs directory will contain a series of
 
fusermount. The mounted /tmp/mythfs directory will contain a series of
 
human-readable recordings organized by title (directory) and subtitle
 
human-readable recordings organized by title (directory) and subtitle
(file). In the case that there is no subtitle, then the recording file
+
(file).  
will be placed into the top level of the directory.
+
  
 
To unmount:
 
To unmount:
  
  $ '''fusermount -u /tmp/mythfs'''
+
  $ <b>fusermount -u /tmp/mythfs</b>
  
 
Note do NOT try to kill the mythfs.pl process. This will only cause a
 
Note do NOT try to kill the mythfs.pl process. This will only cause a
 
hung filesystem that needs to be unmounted with fusermount.
 
hung filesystem that needs to be unmounted with fusermount.
  
There are a number of options that you can pass to mythfs.pl, the most
+
There are a number of options that you can pass to mythfs.pl,
useful of which is "-o allow_other". If this is present, then others
+
including the ability to customize the filesystem layout and set the
on the system (including root) can see the mounted filesystem. Call
+
interval that the backend is checked for new and deleted
with the -h option for more help.
+
recordings. Call mythfs.pl with the -h option for the complete help
 +
text.
  
==Understanding the Directory Layout==
+
==Local Recordings==
 +
 
 +
The default behavior of this filesystem is to use the Myth API to
 +
stream recordings across the network when you attempt to read from
 +
them. This is done in an efficient way that fetches just the portion
 +
of the file you wish to read. However, if the underlying recording
 +
files are directly accessible (either in a regular director or via an
 +
NFS mount), you can get better performance by passing mythfs.pl the
 +
--mountpt option with the path to the directory in which the
 +
recordings can be found. The filesystem will then be set up as a set
 +
of symbolic links that point from a human readable file name to the
 +
recording file.
 +
 
 +
The main advantage of creating symbolic links is that NFSv4 can be
 +
significantly faster than the backend streaming protocol -- about 1.6X
 +
in my informal tests. The main limitation is that this mode does not
 +
understand storage groups, so all recordings need to be located in a
 +
single storage group in a locally-accessible directory. However if a
 +
recording file is not found in local directory, then mythfs.pl will
 +
fall back to the streaming protocol, so the recording will never
 +
become inaccessible.
 +
 
 +
==The Default Directory Layout==
  
 
Recordings that are part of a series usually have a title (the series
 
Recordings that are part of a series usually have a title (the series
Line 61: Line 94:
 
using a two-tier directory structure in which the top-level directory
 
using a two-tier directory structure in which the top-level directory
 
is the series name, and the contents are a series of recorded
 
is the series name, and the contents are a series of recorded
episodes.
+
episodes. The corresponding pattern (as described in the next section)
 +
is "%T/%S".
  
 
For recordings that do not have a subtitle, typically one-off movie
 
For recordings that do not have a subtitle, typically one-off movie
Line 74: Line 108:
 
Here is an example directory listing:
 
Here is an example directory listing:
  
<pre>
 
 
  % <b>ls -lR  /tmp/mythfs</b>
 
  % <b>ls -lR  /tmp/mythfs</b>
 
  total 35
 
  total 35
Line 94: Line 127:
 
  total 4
 
  total 4
 
  -r--r--r-- 1 lstein lstein 3512038152 Apr 24 00:00 Flirting With Disaster 2013-04-24-00:00.mpg
 
  -r--r--r-- 1 lstein lstein 3512038152 Apr 24 00:00 Flirting With Disaster 2013-04-24-00:00.mpg
</pre>
 
  
==Troubleshooting==
+
The size of directories corresponds to the number of recordings (not
 +
counting subdirectories) contained within it.
  
 +
==Customizing the Directory Listing==
  
Your Perl must have been compiled with IThreads in order for this
+
You may customize the directory listing by providing a pattern for
script to work. To check if this is the case. you may run:
+
naming each recording using the -p option. For example:
  
  ''' $ perl -V | grep useithreads'''
+
  $ mythfs.pl -p '%C/%T:%S (%od-%ob-%oY)' mythbackend ~/Myth
    useithreads=define, usemultiplicity=define
+
  
 +
This will create filenames that look like this:
 +
 +
Sitcom/The Simpsons:The Food Wife (13-Nov-2011).mpg
 +
 +
Patterns contain a combination of constant strings plus substitution
 +
patterns consisting of the "%" sign plus 1 to three characters. A
 +
slash will be interpreted as a directory level: multiple levels are
 +
allowed.
 +
 +
Commonly-used substitution patterns are:
 +
 +
    %T  = Title (show name)
 +
    %S  = Subtitle (episode name)
 +
    %C  = Category
 +
    %cn  = Channel: channel number
 +
    %cN  = Channel: channel name
 +
    %y  = Recording start time:  year, 2 digits
 +
    %Y  = Recording start time:  year, 4 digits
 +
    %m  = Recording start time:  month, leading zero
 +
    %b  = Recording start time:  abbreviated month name
 +
    %B  = Recording start time:  full month name
 +
    %d  = Recording start time:  day of month, leading zero
 +
    %h  = Recording start time:  12-hour hour, with leading zero
 +
    %H  = Recording start time:  24-hour hour, with leading zero
 +
    %i  = Recording start time:  minutes
 +
    %s  = Recording start time:  seconds
 +
    %a  = Recording start time:  am/pm
 +
    %A  = Recording start time:  AM/PM
 +
 +
A full list of patterns can be obtained by running "mythfs.pl -p
 +
help".
 +
 +
Patterns are largely compatible with the excellent mythlink.pl
 +
(http://www.mythtv.org/wiki/Mythlink.pl) script, but there are a small
 +
number of enhancements, such as the ability to generate the month
 +
name. Also, the patterns that generate the month name without a
 +
leading zero are not supported.
 +
 +
You may wish to use a delimiter to separate fields of the recording
 +
name, for example "%T:%S" to generate "Title:Subtitle". Occasionally a
 +
recording field is empty, leading to names like "The Wild
 +
Ones:.mpg". To avoid this, pass the --trim option with the delimiter
 +
you use, and dangling/extra delimiters will be trimmed:
 +
 +
$ mythfs.pl -p '%T:%S' --trim=':' backend /tmp/myth
 +
 +
==Fuse Notes==
 +
 +
For best performance, you will need to run this filesystem using a
 +
version of Perl that supports IThreads. Otherwise it will fall back to
 +
non-threaded mode, which will introduce occasional delays during
 +
directory listings and have notably slower performance when reading
 +
from more than one file simultaneously.
 +
 +
If you are running Perl 5.14 or higher, you *MUST* use at least 0.15
 +
of the Perl Fuse module. At the time this was written, the version of
 +
Fuse 0.15 on CPAN was failing its regression tests on many
 +
platforms. I have found that the easiest way to get a fully
 +
operational Fuse module is to clone and compile a patched version of
 +
the source, following this recipe:
 +
 +
$ <b>git clone git://github.com/isync/perl-fuse.git</b>
 +
$ <b>cd perl-fuse</b>
 +
$ <b>perl Makefile.PL</b>
 +
$ <b>make test</b>  (optional)
 +
$ <b>sudo make install</b>
 +
 +
 +
==Troubleshooting==
  
 
This script has not yet undergone diligent testing. Try running with
 
This script has not yet undergone diligent testing. Try running with
Line 113: Line 215:
  
 
Lincoln Stein <lincoln.stein@gmail.com>
 
Lincoln Stein <lincoln.stein@gmail.com>
3 May 2013
+
8 May 2013

Revision as of 17:13, 8 May 2013

Author Lincoln Stein
Description This script maps TV recordings on a local or remote MythTV backend to a human-readable directory structure on the local machine.
Supports Version25.png  Version26.png 


mythfs-perl

This is a FUSE filesystem for MythTV (www.mythtv.org). It uses the Myth 0.25 API to mount the TV recordings known to a MythTV master back end onto a virtual filesystem on the client machine for convenient playback with mplayer or other video tools. Because this uses the MythTV network protocol, the recordings do not need to be on a shared NFS-mounted disk, nor does the Myth database need to be accessible from the client.

Installation

Run the following commands from within the top-level directory of this distribution:

$  git clone git://github.com/lstein/mythfs-perl.git
$ cd mythfs-perl
$ ./Build.PL
$ ./Build test
$ sudo ./Build install

If you get messages about missing dependencies, run:

$ ./Build installdeps

and then "sudo ./Build install". See Fuse Notes if you get Fuse-related errors when attempting to install dependencies.

For best performance, Perl must have been compiled with support for IThreads. In addition, you will need at Fuse version 0.15 to run correctly under Perl version >= 5.14. Threading will be automatically disabled if not available. See Fuse Notes for additional information.

Usage

To mount the recordings contained on the master backend "MyHost" onto a local filesystem named "/tmp/mythfs" use this command:

$ mkdir /tmp/mythfs
$ mythfs.pl MyHost /tmp/mythfs

The script will fork into the background and should be stopped with fusermount. The mounted /tmp/mythfs directory will contain a series of human-readable recordings organized by title (directory) and subtitle (file).

To unmount:

$ fusermount -u /tmp/mythfs

Note do NOT try to kill the mythfs.pl process. This will only cause a hung filesystem that needs to be unmounted with fusermount.

There are a number of options that you can pass to mythfs.pl, including the ability to customize the filesystem layout and set the interval that the backend is checked for new and deleted recordings. Call mythfs.pl with the -h option for the complete help text.

Local Recordings

The default behavior of this filesystem is to use the Myth API to stream recordings across the network when you attempt to read from them. This is done in an efficient way that fetches just the portion of the file you wish to read. However, if the underlying recording files are directly accessible (either in a regular director or via an NFS mount), you can get better performance by passing mythfs.pl the --mountpt option with the path to the directory in which the recordings can be found. The filesystem will then be set up as a set of symbolic links that point from a human readable file name to the recording file.

The main advantage of creating symbolic links is that NFSv4 can be significantly faster than the backend streaming protocol -- about 1.6X in my informal tests. The main limitation is that this mode does not understand storage groups, so all recordings need to be located in a single storage group in a locally-accessible directory. However if a recording file is not found in local directory, then mythfs.pl will fall back to the streaming protocol, so the recording will never become inaccessible.

The Default Directory Layout

Recordings that are part of a series usually have a title (the series name) and subtitle (the episode name). Such recordings are displayed using a two-tier directory structure in which the top-level directory is the series name, and the contents are a series of recorded episodes. The corresponding pattern (as described in the next section) is "%T/%S".

For recordings that do not have a subtitle, typically one-off movie showings, the recording is placed at the top level.

In all cases, the time the recorded was started is attached to the filename, along with an extension indicating the recording type (.mpg or .nuv). The file create and modification times correspond to the recording start time. For directories, the times are set to the most recent recording contained within the directgory.

Here is an example directory listing:

% ls -lR  /tmp/mythfs
total 35
-r--r--r-- 1 lstein lstein 12298756208 Dec 30 00:00 A Funny Thing Happened on the Way to the Forum 2012-12-30-00:00.mpg
-r--r--r-- 1 lstein lstein 14172577964 Dec 25 16:00 A Heartland Christmas 2012-12-25-16:00.mpg
dr-xr-xr-x 1 lstein lstein           5 Mar 11 03:00 Alfred Hitchcock Presents
dr-xr-xr-x 1 lstein lstein           8 May  2 00:00 American Dad
...
/home/lstein/Myth/Alfred Hitchcock Presents:
total 3
-r--r--r-- 1 lstein lstein 647625408 Dec 25 15:30 Back for Christmas 2012-12-25-15:30.mpg
-r--r--r-- 1 lstein lstein 647090360 Dec  7 00:00 Dead Weight 2012-12-07-00:00.mpg
-r--r--r-- 1 lstein lstein 660841056 Mar 11 03:00 Rose Garden 2013-03-11-03:00.mpg
-r--r--r-- 1 lstein lstein 647524452 Dec 25 00:00 Santa Claus and the 10th Ave. Kid 2012-12-25-00:00.mpg
-r--r--r-- 1 lstein lstein 649819932 Dec 27 00:00 The Contest of Aaron Gold 2012-12-27-00:00.mpg
/home/lstein/Myth/American Dad:
total 4
-r--r--r-- 1 lstein lstein 3512038152 Apr 24 00:00 Flirting With Disaster 2013-04-24-00:00.mpg

The size of directories corresponds to the number of recordings (not counting subdirectories) contained within it.

Customizing the Directory Listing

You may customize the directory listing by providing a pattern for naming each recording using the -p option. For example:

$ mythfs.pl -p '%C/%T:%S (%od-%ob-%oY)' mythbackend ~/Myth

This will create filenames that look like this:

Sitcom/The Simpsons:The Food Wife (13-Nov-2011).mpg

Patterns contain a combination of constant strings plus substitution patterns consisting of the "%" sign plus 1 to three characters. A slash will be interpreted as a directory level: multiple levels are allowed.

Commonly-used substitution patterns are:

   %T   = Title (show name)
   %S   = Subtitle (episode name)
   %C   = Category
   %cn  = Channel: channel number
   %cN  = Channel: channel name
   %y   = Recording start time:  year, 2 digits
   %Y   = Recording start time:  year, 4 digits
   %m   = Recording start time:  month, leading zero
   %b   = Recording start time:  abbreviated month name
   %B   = Recording start time:  full month name
   %d   = Recording start time:  day of month, leading zero
   %h   = Recording start time:  12-hour hour, with leading zero
   %H   = Recording start time:  24-hour hour, with leading zero
   %i   = Recording start time:  minutes
   %s   = Recording start time:  seconds
   %a   = Recording start time:  am/pm
   %A   = Recording start time:  AM/PM

A full list of patterns can be obtained by running "mythfs.pl -p help".

Patterns are largely compatible with the excellent mythlink.pl (http://www.mythtv.org/wiki/Mythlink.pl) script, but there are a small number of enhancements, such as the ability to generate the month name. Also, the patterns that generate the month name without a leading zero are not supported.

You may wish to use a delimiter to separate fields of the recording name, for example "%T:%S" to generate "Title:Subtitle". Occasionally a recording field is empty, leading to names like "The Wild Ones:.mpg". To avoid this, pass the --trim option with the delimiter you use, and dangling/extra delimiters will be trimmed:

$ mythfs.pl -p '%T:%S' --trim=':' backend /tmp/myth

Fuse Notes

For best performance, you will need to run this filesystem using a version of Perl that supports IThreads. Otherwise it will fall back to non-threaded mode, which will introduce occasional delays during directory listings and have notably slower performance when reading from more than one file simultaneously.

If you are running Perl 5.14 or higher, you *MUST* use at least 0.15 of the Perl Fuse module. At the time this was written, the version of Fuse 0.15 on CPAN was failing its regression tests on many platforms. I have found that the easiest way to get a fully operational Fuse module is to clone and compile a patched version of the source, following this recipe:

$ git clone git://github.com/isync/perl-fuse.git
$ cd perl-fuse
$ perl Makefile.PL
$ make test   (optional)
$ sudo make install


Troubleshooting

This script has not yet undergone diligent testing. Try running with the -debug flag to see where the problems are occurring and report issues to https://github.com/lstein/mythfs-perl.

Author

Lincoln Stein <lincoln.stein@gmail.com> 8 May 2013