Difference between revisions of "Jamu"

From MythTV Official Wiki
Jump to: navigation, search
(Jamu limitations)
Line 9: Line 9:
 
#[[#Mass Updates]] of graphics and text meta data:
 
#[[#Mass Updates]] of graphics and text meta data:
 
#*Renaming existing TV series graphics to the new format (series Season - #) and update the video file's data base record. e.g. "Fringe Season 1.jpg"
 
#*Renaming existing TV series graphics to the new format (series Season - #) and update the video file's data base record. e.g. "Fringe Season 1.jpg"
#*Renaming existing Movie graphics to the reference number of the source of the meta data, generally themovie.com TMDB number or an imdb.com IMDB number.
+
#*Renaming existing Movie graphics to the reference number of the source of the meta data IMDB number.
 
#*Fetch the graphics directories from the MythTV data base (Posters, Fan art and  Banners)
 
#*Fetch the graphics directories from the MythTV data base (Posters, Fan art and  Banners)
 
#*Download missing Posters/Cover Image, Banners and Fan art for movies and TV Series
 
#*Download missing Posters/Cover Image, Banners and Fan art for movies and TV Series
Line 25: Line 25:
 
#*User definable fields that control what triggers a meta data update. This can be valuable if you have bandwidth concerns or do not care that information like director, userrating ... etc are updated.
 
#*User definable fields that control what triggers a meta data update. This can be valuable if you have bandwidth concerns or do not care that information like director, userrating ... etc are updated.
 
#Full Storage Group support for Videos and Graphics
 
#Full Storage Group support for Videos and Graphics
#[[#Interactive Session]] supporting the selection of movie TMDB, IMDB or TV Series id numbers
+
#[[#Interactive Session]] supporting the selection of movie IMDB or TV Series id numbers
#*All non-interactive use of Jamu is based on the myth database having a TMDB or IMDB number for movies while TV episodes must have file names that contain series name and season/episode numbers.  
+
#*All non-interactive use of Jamu is based on the myth database having a IMDB number for movies while TV episodes must have file names that contain series name and season/episode numbers.  
 
#*Web links are included with the displayed choices
 
#*Web links are included with the displayed choices
#*You can enter the TMDB, IMDB or TV Series id numbers directly
+
#*You can enter the IMDB or TV Series id numbers directly
#Have Jamu guess at the correct TV series or Movie with only exact matches considered a successful guess. A successful guess will assign the appropriate TMDB or TVDB number to that video files metadata. This option (-MG) works best when the video file name is very close to those used by themoviedb.com or thetvdb.com.
+
#Have Jamu guess at the correct TV series or Movie with only exact matches considered a successful guess. A successful guess will assign the appropriate IMDB number to that video files metadata. This option (-MG) works best when the video file name is very close to those used by themoviedb.com or thetvdb.com.
 
#Download missing graphics for [[#Scheduled and Recorded]] programs
 
#Download missing graphics for [[#Scheduled and Recorded]] programs
#'''Find your missing TMDB, IMDB or TV Series id numbers'''. This invokes an interactive session that allows you to focus only on video files that are lacking the essential id numbers.
+
#'''Find your missing IMDB or TV Series id numbers'''. This invokes an interactive session that allows you to focus only on video files that are lacking the essential id numbers.
 
#'''Seamless integration''' with Mythvideo download feature. Jamu conforms to the same update processes as Mythvideo downloads does for a single video file. This includes the same graphics file naming conventions for movies and TV episodes.
 
#'''Seamless integration''' with Mythvideo download feature. Jamu conforms to the same update processes as Mythvideo downloads does for a single video file. This includes the same graphics file naming conventions for movies and TV episodes.
 
#A [[#Janitor]] option is provided which cleans up graphics that are no longer used by MythUI.
 
#A [[#Janitor]] option is provided which cleans up graphics that are no longer used by MythUI.
Line 63: Line 63:
  
 
Click on the follow link to find the jamu script archive:
 
Click on the follow link to find the jamu script archive:
[http://svn.mythtv.org/trac/ticket/6440 #6440] make sure to download the latest version (v0.3.9)
+
[http://svn.mythtv.org/trac/ticket/6440 #6440] make sure to download the latest version (v0.4.0)
 +
 
 +
You must have the python IMDbPy library v3.8 or higher installed. Find this package in your distribution's repository or from [http://imdbpy.sourceforge.net/?page=download]
  
 
=== Installing jamu.py ===
 
=== Installing jamu.py ===
Line 116: Line 118:
 
::'''''"For those who are about to die we salute you"''''' - Gladiator
 
::'''''"For those who are about to die we salute you"''''' - Gladiator
 
==== Interactive Session ====
 
==== Interactive Session ====
Jamu depends on video file data base records having either TMDB or IMDB numbers for movies and TV episodes having series names, season and episode numbers. An interactive session is required to get that information input into the MythTV data base. Invoke an interactive session to update the MythTV database through the following command line:
+
Jamu depends on video file data base records having either IMDB numbers for movies and TV episodes having series names, season and episode numbers. An interactive session is required to get that information input into the MythTV data base. Invoke an interactive session to update the MythTV database through the following command line:
 
  /usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -MIV
 
  /usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -MIV
 
The option 'V' adds verbose messages to the console. This helps you understand what is happening.<BR>After adding information for a couple TV shows or Movies abort the interactive session by hitting ctrl-C a few times. Your updates will be retained and the data base will be OK.
 
The option 'V' adds verbose messages to the console. This helps you understand what is happening.<BR>After adding information for a couple TV shows or Movies abort the interactive session by hitting ctrl-C a few times. Your updates will be retained and the data base will be OK.
Line 236: Line 238:
 
  Number of metadata downloads.........(  578)
 
  Number of metadata downloads.........(  578)
 
  Number of 2nd source metadata found .(  79)
 
  Number of 2nd source metadata found .(  79)
Number of movies still using IMDB#s..(  50)
 
 
  Number of symbolic links created.....(    0)
 
  Number of symbolic links created.....(    0)
 
  Number of Myth database updates......(    5)
 
  Number of Myth database updates......(    5)
Line 248: Line 249:
 
  Jason And The Argonauts
 
  Jason And The Argonauts
 
   
 
   
------------Movies using IMDB#-------------
 
Soldier has IMDB# (0088146)
 
A Soldiers Story has IMDB# (0088146)
 
....
 
Time And Tide has IMDB# (0251433)
 
Young Lions has IMDB# (0052415)
 
 
 
 
  ---------------Under sized Poster----------
 
  ---------------Under sized Poster----------
 
  This Island Earth
 
  This Island Earth
Line 367: Line 361:
 
#There is no chaining of TV series episodes or video files. A movie that is split into multiple video files are treated as separate video files. You could use a video utility like ffmpeg or mkvmerge (append feature) to combine multiple video files.
 
#There is no chaining of TV series episodes or video files. A movie that is split into multiple video files are treated as separate video files. You could use a video utility like ffmpeg or mkvmerge (append feature) to combine multiple video files.
 
#.ts video files are not included as a supported video file extension. This can be changed by add the "ts" extension to a user configuration file variable '''video_file_exts'''. See "jamu-example.conf". Again file chaining is not supported.
 
#.ts video files are not included as a supported video file extension. This can be changed by add the "ts" extension to a user configuration file variable '''video_file_exts'''. See "jamu-example.conf". Again file chaining is not supported.
#If you use a customized naming convention for your graphics then DO NOT use Jamu. Jamu conforms to the naming convention used by Mythvideo downloads ("tmdb#.ext" or "imdb#.ext" for movies and "series Season #.ext" for TV series). Your graphic files will be renamed to conform to these standards. This is the only way Jamu can work seamlessly with Mythvideo downloads.
+
#If you use a customized naming convention for your graphics then DO NOT use Jamu. Jamu conforms to the naming convention used by Mythvideo downloads ("imdb#.ext" for movies and "series Season #.ext" for TV series). Your graphic files will be renamed to conform to these standards. This is the only way Jamu can work seamlessly with Mythvideo downloads.
 
#Jamu does not download or deal with screen shot graphics. There has been discussion that screen shot capabilities is going to be added to MythTV.
 
#Jamu does not download or deal with screen shot graphics. There has been discussion that screen shot capabilities is going to be added to MythTV.
 
#When there are Storage groups for graphics Jamu will download new graphics to the FIRST graphic storage group per graphic type (cover art, banners and/or Fan art). Jamu does not support any download balancing of graphics.
 
#When there are Storage groups for graphics Jamu will download new graphics to the FIRST graphic storage group per graphic type (cover art, banners and/or Fan art). Jamu does not support any download balancing of graphics.
Line 375: Line 369:
 
*The script tmdb_api author dbr/Ben was copied and extensively modified. tmdb.py can be found at [http://github.com/dbr/themoviedb/blob/22e7fa6c8286f954c28862a6983e60950a8b2d38/tmdb.py]<BR>
 
*The script tmdb_api author dbr/Ben was copied and extensively modified. tmdb.py can be found at [http://github.com/dbr/themoviedb/blob/22e7fa6c8286f954c28862a6983e60950a8b2d38/tmdb.py]<BR>
 
*The routines to select video files was copied and modified from tvnamer.py mentioned above.
 
*The routines to select video files was copied and modified from tvnamer.py mentioned above.
*The interactive console interface for selecting TMDB or IMDB numbers and TV Series was copied and modified from tvnamer
+
*The interactive console interface for selecting IMDB numbers and TV Series was copied and modified from tvnamer
 
*The routine "_save_video_metadata_to_mythdb" has been copied and modified from "find_meta.py" author Pekka Jääskeläinen.<BR>
 
*The routine "_save_video_metadata_to_mythdb" has been copied and modified from "find_meta.py" author Pekka Jääskeläinen.<BR>
 
*The routine "_addCastGenre" was copied and modified from "tvdb-bulk-update.py" by author David Shilvock <davels@telus.net>. Found at http://bitbucket.org/davels/tvdb-bulk-update/<BR>
 
*The routine "_addCastGenre" was copied and modified from "tvdb-bulk-update.py" by author David Shilvock <davels@telus.net>. Found at http://bitbucket.org/davels/tvdb-bulk-update/<BR>
 
*Additional scripts are called if specified as secondary data sources e.g. find_poster.py author Pekka Jääskeläinen (gmail: pekka.jaaskelainen)
 
*Additional scripts are called if specified as secondary data sources e.g. find_poster.py author Pekka Jääskeläinen (gmail: pekka.jaaskelainen)
 
*The Graphite theme by Robert McNamara found at http://www.gossamer-threads.com/lists/mythtv/theming/386359?page=last
 
*The Graphite theme by Robert McNamara found at http://www.gossamer-threads.com/lists/mythtv/theming/386359?page=last

Revision as of 21:34, 22 August 2009

What is Jamu?

Jamu - Just.Another.Metadata.Utility
This python script performs a variety of utility functions on mythvideo metadata and its associated video files and directories. The focus is to maximize your experience with the new MythUI only available in trunk 0.22. The primary sources of graphics and meta data for movies is themoviedb.orgwiki and for TV Series thetvdb.comwiki

Jamu's Features

  1. #Mass Updates of graphics and text meta data:
    • Renaming existing TV series graphics to the new format (series Season - #) and update the video file's data base record. e.g. "Fringe Season 1.jpg"
    • Renaming existing Movie graphics to the reference number of the source of the meta data IMDB number.
    • Fetch the graphics directories from the MythTV data base (Posters, Fan art and Banners)
    • Download missing Posters/Cover Image, Banners and Fan art for movies and TV Series
    • Populate the Mythvideo file's data base fields for:
      • TV series season and episode numbers where applicable
      • Movie graphics (Posters and Fanart) file names
      • TV series/season graphics (Posters, Banners, Fan art) file names
      • Movie and TV episode text information (meta data)
      • Add Cast members and Genres and associate with a video file
    • Create symbolic links to posters so that the MythUI shows the posters icons in place of the generic directory icon
  2. #Regular Maintenance (usually as a cron job)
    • Update MythTV video file data base records filling in missing information when new information becomes available
    • Download season specific posters and banners for TV series when they become available
    • User definable threshold that identifies low quality Poster/Cover Images. This is a trigger for the maintenance process to look for a better graphic from the primary and secondary graphics sources. The default setting is a width of 200 pixels.
    • User definable fields that control what triggers a meta data update. This can be valuable if you have bandwidth concerns or do not care that information like director, userrating ... etc are updated.
  3. Full Storage Group support for Videos and Graphics
  4. #Interactive Session supporting the selection of movie IMDB or TV Series id numbers
    • All non-interactive use of Jamu is based on the myth database having a IMDB number for movies while TV episodes must have file names that contain series name and season/episode numbers.
    • Web links are included with the displayed choices
    • You can enter the IMDB or TV Series id numbers directly
  5. Have Jamu guess at the correct TV series or Movie with only exact matches considered a successful guess. A successful guess will assign the appropriate IMDB number to that video files metadata. This option (-MG) works best when the video file name is very close to those used by themoviedb.com or thetvdb.com.
  6. Download missing graphics for #Scheduled and Recorded programs
  7. Find your missing IMDB or TV Series id numbers. This invokes an interactive session that allows you to focus only on video files that are lacking the essential id numbers.
  8. Seamless integration with Mythvideo download feature. Jamu conforms to the same update processes as Mythvideo downloads does for a single video file. This includes the same graphics file naming conventions for movies and TV episodes.
  9. A #Janitor option is provided which cleans up graphics that are no longer used by MythUI.
  10. #Moving video files and dynamically update the video file's data base record
    • Single files or whole directories. Great for moving TV episodes into season specific folders.
    • Video file wild card selection. e.g Only move a single season of episodes "Fringe S01E*"
    • Optionally rename TV episodes if the video file name contains season and episode numbers
    • Multiply "target and destination" pairs can be specified on the command line
  11. #Simulation (Dry Run) so that you can see what would be changed. Messages are issued for all changes to your the Myth data base, file downloads, file moves and files created. No actual changes are made to your environment. This is an essential feature when getting started.
  12. Support user specified secondary sources for graphics and/or text metadata
  13. User definable regex patterns to add support for TV series episode file names that Jamu does not natively include. See section "regex" in "jamu-example.conf".
  14. #User Configuration File which exposes almost all the script's variables for extensive customization
  15. #Statistics Reports for mass updates, maintenance, file moves and Janitor processing
  16. Jamu can be used for many non-MythTV specific purposes related to TV Series. To see examples type:
./Jamu.py --examples

Language Support

This option is only supported for TV Series data and then only from the prime source thetvdb.comwiki.

Jamu data requests default to English unless the -l option is used. If you specify a language (e.g. -l es), jamu will first look for data in that language but if none is available jamu will return the English equivalent assuming any exists.

Currently thetvdb.com wiki supports the following languages: (da, fi, nl, de, it, es, fr, pl, hu, el, tr, ru, he, ja, pt, zh, cs, sl, hr, ko, en, sv, no)

Currently themoviedb.orgwiki does not support meta data selection by a specific language. Multi-language support is a feature mentioned as a future enhancement.

Getting Started

Downloading jamu.py script

As of change set [21275] (14AUG09) Jamu v0.3.9 or higher is required

Currently jamu.py does not ship with MythTV and only works with trunk (v0.22) revision 21275 or higher.

Click on the follow link to find the jamu script archive: #6440 make sure to download the latest version (v0.4.0)

You must have the python IMDbPy library v3.8 or higher installed. Find this package in your distribution's repository or from [1]

Installing jamu.py

Jamu can only be run on a MythTV backend. If you have multiple MythTV backends you will need to install and run Jamu on each of them.

The MythVideo scripts are typically installed in /usr/share/mythtv/mythvideo/scripts/ or /usr/local/share/mythtv/mythvideo/scripts/. Installing jamu in the same directory as ttvdb.py will insure that all required python modules are available.

With sudo privileges un-archive the jamu archive files into your mythvideo scripts directory as described above. You can then delete the jamu archive file as it is no longer required.

Configuring your graphics directories

Set up the directories for the graphic artwork. Utilities/Setup|Setup|Media Settings|Videos Settings|General Settings on page 1/8, enter the directories where the graphic files will be stored. It is essential to use separate directories for each type of graphic file (Posters, Banners, Fan art and Screen shots).

User Configuration File

A user configuration file is necessary to get the most out of what Jamu has to offer. This is where you can specify secondary sources to get data and deal with issues like movies that look like TV episodes to Jamu (e.g. the movie "U-571").

Almost every variable in the Jamu script can be set through the user configuration file but remember

"With great power comes great responsibility" - Stan Lee

There is limited validation of the variables so you can cause Jamu to abort or not work as intended.

If you have examined the "jamu-example.conf" you may be overwhelmed by the number of variables. The reality is almost all variables should be left to their defaults.
This documentation will guide you through the creation of your own configuration file and the variables that should be customised before using Jamu.

After changing configuration variables check that the variables are set as you expect run Jamu with the -f option and all the settings will be displayed:

./jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -Mf | less

Use the "less" command as there are more than a page worth of variables. The variables are sorted alphabetically. Hitting 'q' key will exit "less".

Creating your own configuration file

Copy and rename "jamu-example.conf" to a directory of your choosing (e.g. ~/.jamu/jamu-maintenance.conf). Then edit the new file and make changes as instructed below. The example configuration file contains documentation for all sections and variables. To have jamu use your configuration file use the -C option, For example:

./jamu -C "/home/user/.jamu/jamu-maintenance.conf" ... 

If you include Jamu in a mythbackend user job always specify a full qualified path for the configuration file, do not use the "~" short cut.

Configuration options that should be customized

The configuration file contains ample documentation on what each section is for and what each variable does. That documentation will not be repeated here. This guide will help most users get running ASAP.
To activate a variable remove the leading '#' (pound) character from the front of the variable.

  1. Find the local_language variable and set the two letter language of choice. As long as you use the configuration file you will not need to set your preferred language on the command line.
  2. Find the video_file_exts variable. If one of your video file extensions is not already included in the defaults then add it to the default string.
  3. Find the series_name_override section. Activate all variables (TV Series and Movies) that are applicable to your video collection. Add entries that you suspect should be included. Especially movie titles that may be interpreted as a TV series with season and episode numbers (e.g. "U-571"), TV series which have had remakes (e.g. The Beast, Life on Mars ... etc) or movies remakes (e.g. Planet of the Apes)
  4. Find the ep_name_massage section. If you have the TV Series '24' episodes then activate the variable.
  5. Find the movies-secondary-sources and tv-secondary-sources add/activate any secondary sources for graphics and data

That was not so bad was it? Remember to check that the variables are set as you intended with:

./jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -Mf | less

Variables you should revisit after you are in maintenance mode

  1. Find the tmdb_genre_filter variable. It is advised NOT to change the default settings until after you have done your mass update. Read the configuration file documentation to understand why there is a filter or use "tmdb.pl -D imdb#" from the command line to see what is returned as genres.
  2. Find the metadata_exclude_as_update_trigger variable. You may find that unimportant bits of missing meta data are causing frequent downloads of meta data even though it is unlikely that new data is available.
    Activate this variable and add keys to the exclusion list will reduce unnecessary meta data downloads. Adding plot would not serve you well but director or user rating may help. View the statistics reports from your maintenance log to determine if you need to change the defaults for this variable. If you do a maintenance run while using the -V (verbose) option you will be able to see which field is triggering the most meta data download.
  3. Find the min_poster_size variable. Only change from the default minimum poster width of 200 pixels if it is too large or you feel that too many poster downloads are being triggered due to the majority of your posters being less than the 200 pixel threshold. If you activate this variable and change the number to '0' zero Jamu will not attempt to get a higher resolution poster during maintenance processing.

Mass Updates

Using Jamu to perform a mass update of the MythTV data base and download of graphics for your video collection is the fastest way to enjoying the "Bling" that MythUI can display. Follow these steps to set-up and execute a mass update.

Steps to performing a Mass Update

Set up your own configuration file.

See #User Configuration File

Back up your MythTV data base

See data base backups. Jamu writes to the following data base tables 'videometadata', 'videometadatacast', 'videometadatagenre', 'videogenre' and 'videocast'. Jamu has been tested on a collection of over 1800 video files but anything can happen, so if you skip backing up the your data base then:

"For those who are about to die we salute you" - Gladiator

Interactive Session

Jamu depends on video file data base records having either IMDB numbers for movies and TV episodes having series names, season and episode numbers. An interactive session is required to get that information input into the MythTV data base. Invoke an interactive session to update the MythTV database through the following command line:

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -MIV

The option 'V' adds verbose messages to the console. This helps you understand what is happening.
After adding information for a couple TV shows or Movies abort the interactive session by hitting ctrl-C a few times. Your updates will be retained and the data base will be OK.

Simulation (Dry Run)

Perform a simulation run to check if everything reacts as expected. A simulation goes through all normal processes EXCEPT it does not download graphics, meta data or alter your data base. You will want to pipe the messages to a log file and examine the log at your leasure.

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -sMV > "~/tmp/jamu.log"

During processing, you can tail the log file in a separate terminal session or open it in a text editor, to see that all is well.
The length of this simulation run is depended on the size of your video collection, the speed of your Internet connection, how much meta data needs to be updated and the responsiveness of the primary source web sites. You can safely abort the simulation at any time with a few ctrl-C in a row.
Examine the log and if everything looks OK then you are ready to complete your interactive session.

Restart Interactive Session

Complete your interactive session with the following command line:

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -MIV

When you have added all the required information meta data and graphics downloads will start just for the video files you updated.
Start up the mythfrontend and check out what the MythUI looks like for the updated videos. Assume you like what you see you are ready for a mass update.

Running the Mass Update

Start a Mass Update with the following command:

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -MV > "~/tmp/jamu.log"

If the session is going on for too long you can abort the script (a few ctrl-C in a row) without harming your data base. You will lose the statistics report but any data base updates and downloaded graphics will be retained. You can restart the script with the command line above and it will restart with the next video that has missing information.
Mass updates can take a while and are best done in off prime hours (prime hours are 4-8PM EST). Check the log file every 15 minutes.

Regular Maintenance

Using Jamu for regular maintenance is as easy as setting up a cron job. Here is a suggested cron command string:

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -M > "/home/user/tmp/jamu_maintenance.log"

Modify the cron command string for language and location of the configuration and log files. The assumption is that you have already successfully gone through the steps for a mass update and are just keeping your environment up to date.
Do not schedule maintenance more than once a day and at that only during off hours. The data does not get changed that frequently and you do not want to over tax these FREE data sources. Currently both primary sources are going through teething pains from a rapidly growing user community.

Scheduled and Recorded

With the release of the Graphite theme (see: [2]), Posters and Fan art are now supported in the Watch Recordings screen. With the (-MW) option Jamu can download the graphics for Scheduled and Recorded programs which will enable their displayed on the Watch Recordings screen. Graphics for both TV shows and Movies will be downloaded. It is suggested to use a nightly cron job to keep your graphics up to date. Only missing graphics will be downloaded. Here is an example cron command string:

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -MW > "~/tmp/jamu_watched.log"

See #Scheduled & Recorded Statistics for the generated report example.
Over time a number of unused graphic files may be left from old recordings. Use Jamu's #Janitor feature to remove the unused graphics.

This stupid thing does not work!

The usual suspects:

  1. One or more of the source sites are down (thetvdb.com, themoviedb.org and any of your secondary sites)
  2. You have not followed the installation or mass update set up instructions. The best check of a proper installation is that you can use Mythvideo "Manage Video|Download Metadata" to successfully download both movies and TV episodes graphics and meta data.
  3. Your TV episode file names are not formated so that Jamu can extract series name, season and episode numbers. You can use the utility tvnamer to rename your video files.
  4. You are not running your machine with locale of 'utf-8'. Your configuration should look something like:
    > locale
    LANG=en_CA.UTF-8
    LC_CTYPE="en_CA.UTF-8"
    LC_NUMERIC="en_CA.UTF-8"
    LC_TIME="en_CA.UTF-8"
    LC_COLLATE="en_CA.UTF-8"
    LC_MONETARY="en_CA.UTF-8"
    LC_MESSAGES="en_CA.UTF-8"
    LC_PAPER="en_CA.UTF-8"
    LC_NAME="en_CA.UTF-8"
    LC_ADDRESS="en_CA.UTF-8"
    LC_TELEPHONE="en_CA.UTF-8"
    LC_MEASUREMENT="en_CA.UTF-8"
    LC_IDENTIFICATION="en_CA.UTF-8"
    LC_ALL=
  5. If you are running Jamu as a cron job and you see a Unicode error message, but the same command works when run from a command line, most likely your cron environment is not running as UTF8. This can happen even if root and users run with a locale of UTF8. This generally happens when directories or video files contain Unicode characters. Please refer to your distribution for instructions on setting crons locale environment variables. You could also run a script that sets the locale environment variables as part of the cron command line. For example:
    /home/user/bin/set_to_utf8.sh && /usr/local/bin/jamu ...
  6. One user pointed out that the utility 'PuTTY' (a Telnet and SSH client) was defaulting to 'latin-1' and therefore converting the TV series title so that TV series searches never found a result.
  7. If you are running Jamu as a cron job and you see a "! Warning - Creating an instance caused an error for one of: MythTV, MythVideo or MythDB" message, but the same command works when run from a command line, the problem may be that you have two different "mysql.txt" files. One for root and one for your user account. Check to see if they both use the correct settings.

What are these error messages?

If you have installed Python 2.6 (e.g. Ubuntu 9.04 Jaunty Jackalope) you will see these warning messages. These messages are not generated by Jamu and hopefully will be eliminated sometime in the future (see: [3]). They do not cause any issues with Jamu:

/var/lib/python-support/python2.6/MySQLdb/__init__.py:34: DeprecationWarning: the sets module is deprecated from sets import ImmutableSet

Indication that IMDB.com is having issues responding (Try again later)

imdb._exceptions.IMDbDataAccessError: {'url': 'http://akas.imdb.com/title/tt0074860/combined', 'error
type': 'http_error_default', 'errcode': 502, 'headers': <httplib.HTTPMessage instance at 0x24ab5a8>,
'proxy': , 'errmsg': 'Bad Gateway'}
imdb._exceptions.IMDbDataAccessError: {'url': 'http://akas.imdb.com/title/tt0028231/combined', 'error
type': 'http_error_default', 'errcode': 503, 'headers': <httplib.HTTPMessage instance at 0x1a8ca70>,
'proxy': , 'errmsg': 'Service Temporarily Unavailable'}

find_poster.py - indications that IMDB.com is having issues responding (Try again later)

imdb._exceptions.IMDbDataAccessError: {'url': 'http://akas.imdb.com/title/tt0159784/plotsummary',
'error type': 'http_error_default', 'errcode': 404, 'headers': <httplib.HTTPMessage instance at
0xf9d908>, 'proxy': , 'errmsg': 'Not Found'}

Oops I picked the wrong movie/TV series. What do I do now?

This is easy to fix. Through Mythvideo select your movie/TV episode and "Manage Video|Reset Metadata" then "Manage Video|Download Metadata".

Moving video files

Jamu can be used to reorganise your video collection without loosing meta data or associated graphics. After working with the MythUI and the graphics I found that moving away from a flat directory structure for TV Series had advantages. This option was written just for that purpose. Always add the target (file/directory) and destination (directory) in pairs. Multiple target and destination pairs can be specified on a single command line. Below are command strings that can be used for most move tasks.

Move a single video file

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -M "/PathToVideo/fringe.1x02.avi" "/media/video/Fringe/S01"

Move a directory

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -M "/PathToVideo/fringe" "/media/video/Fringe"

Move a season's worth of TV episodes into a season directory (wild cards)

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -M "/media/video/Fringe/fringe.01x*" "/media/video/Fringe/S01"

Move a movie or TV episode video files and rename to a standard format

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -MF "/PathToVideo/fringe" "/media/video/Fringe/S01"

All video files such as "fringe.s1x02.avi" would be moved and renamed as "Fringe - S01E02 - The Same Old Story.avi"

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -MF "/PathToVideo/" "/media/video/The Duchess"

A movie video file such as "the duchess.mkv"" would be moved and renamed to "The Duchess (2008).mkv""

Janitor

As your collection changes over time some of the graphics that had been downloaded can get orphaned. A Mythvideo rescan only removes Myth data base records that no longer have video files. The Jamu Janitor (-J) option removes the orphaned Posters, Fan art and Banners that are not associated with any video file or with a Scheduled or Recorded program. Use this command line to have Jamu clean up orphaned graphics:

/usr/local/share/mythtv/mythvideo/scripts/jamu.py -l en -C "/home/user/.jamu/jamu-maintenance.conf" -MJ

Statistics Reports

At the end of mass updates, maintenance, file moves, Scheduled/Recorded and Janitor processing a statistics report is out put to the console.

Interactive Session Statistics

------------------Statistics---------------
Number of video files processed .....(    2)
Number of Fanart graphics downloaded (    0)
Number of Poster graphics downloaded (    2)
Number of Banner graphics downloaded (    0)
Number of 2nd source graphics downld (    1)
Number of metadata downloads.........(    2)
Number of 2nd source metadata found .(    1)
Number of symbolic links created.....(    0)
Number of Myth database updates......(    2)
Number of undersized posters ........(    0)

--------------Updated Video Files----------
Village Of The Damned
Fringe - S01E15 - Inner Child

Maintenance and Mass update Statistics

------------------Statistics----------------
Number of video files processed .....( 1811)
Number of Fanart graphics downloaded (    2)
Number of Poster graphics downloaded (    5)
Number of Banner graphics downloaded (    1)
Number of 2nd source graphics downld (    5)
Number of metadata downloads.........(  578)
Number of 2nd source metadata found .(   79)
Number of symbolic links created.....(    0)
Number of Myth database updates......(    5)
Number of undersized posters ........(  134)

--------------Updated Video Files----------
Kings - S01E04 - Insurrection
Lost - S05E11 - Whatever Happened, Happened
24 - S07E17 - Day 7: 12:00 AM - 1:00 AM
The No. 1 Ladies' Detective Agency - S01E02 - Poison
Jason And The Argonauts

---------------Under sized Poster----------
This Island Earth
Three Days of the Condor
...
The Hoax
Titus

Move Statistics

--------------Move Statistics---------------
Number of subdirectories ............(    0)
Number of files moved ...............(    1)
Number of symbolic links recreated...(    0)
Number of renamed TV episode files.. (    0)
Number of Myth database updates .... (    0)
--------------------------------------------

Mythtv video database maintenance start: 2009-04-08 13:22

Mythtv video database maintenance ends at  : 2009-04-08 13:22

------------------Statistics---------------
Number of video files processed .....(    1)
Number of Fanart graphics downloaded (    0)
Number of Poster graphics downloaded (    0)
Number of Banner graphics downloaded (    0)
Number of 2nd source graphics downld (    0)
Number of metadata downloads.........(    1)
Number of 2nd source metadata found .(    0)
Number of symbolic links created.....(    0)
Number of Myth database updates......(    1)
Number of undersized posters ........(    0)

--------------Updated Video Files----------
Fringe - S01E15 - Inner Child

Janitor Statistics

--------------Janitor Statistics---------------
    total fanarts ......................(  738)
    total banners ......................(  133)
    total posters ......................(  884)
  deleted fanarts ......................(    2)
  deleted banners ......................(    2)
  deleted posters ......................(    2)
new total fanarts ......................(  737)
new total banners ......................(  131)
new total posters ......................(  883)

----------------Deleted fanart files---------------
/media/virtual/MythTV_Storage/MythFanArt/The Mentalist.jpg
/media/virtual/MythTV_Storage/MythFanArt/The Mentalist Season 1.jpg

----------------Deleted banner files---------------
/media/virtual/MythTV_Storage/MythBanners/The Mentalist.jpg
/media/virtual/MythTV_Storage/MythBanners/The Mentalist Season 1.jpg

----------------Deleted poster files---------------
/media/virtual/MythTV_Storage/MythVideo/The Mentalist.jpg
/media/virtual/MythTV_Storage/MythVideo/The Mentalist Season 1.jpg

Scheduled & Recorded Statistics

-----Scheduled & Recorded Statistics-------
Number of Scheduled & Recorded ......(   18)
Number of Fanart graphics found .....(   10)
Number of Poster graphics found .....(   10)
Number of Banner graphics found .....(    7)
Number of Fanart graphics downloaded (    2)
Number of Poster graphics downloaded (    2)
Number of Banner graphics downloaded (    1)

-------------Scheduled & Recorded----------
Austin City Limits
American Masters
Kings
...
The Duchess (2008)
True Blood
Hung

Interactive Session Examples

Movies

Now processing video file (Village Of The Damned)(0)(0)

Entry exists in MythDB but category is 0 and year is 1895 (default values).
Updating (Village Of The Damned).
1 -> The Village (2004) # http://www.imdb.com/title/tt0368447
2 -> The Village Barbershop # http://www.imdb.com/title/tt0859644
Direct search of IMDB # http://www.imdb.com/find?s=all&q=Village+Of+The+Damned&x=0&y=0
Enter choice (Enter key equals number one (1)) or input the IMDB directly, ? for help):
0114852

Updated Mythdb for video file(Village Of The Damned)

TV Series

Now processing video file (Fringe - S01E15 - Inner Child)(1)(15)

Entry does not exist in MythDB.  Adding (Fringe - S01E15 - Inner Child).
1 -> Fringe # http://thetvdb.com/index.php?tab=series&id=82066
Direct search of thetvdb # http://thetvdb.com/?tab=advancedsearch
Enter choice (Enter key equals number one (1)) or input the Series id directly, ? for help):
1
Updated Mythdb for video file(Fringe - S01E15 - Inner Child)

Cool, but I wish I could add my DVD collection

Mythvideo is not intended to be a DVD collection manager. Even if you manually entered your DVD collection meta data the next video file scan will remove all that you entered.

One way to get around this is to created a directory called DVDs and created zero length video files who's file name are the name of the DVDs (e.g. "/media/video/DVDs/A to F/A Beautiful Mind.mkv"). Now Jamu can perform all data and graphics downloads as if this was a regular video file.

For a large collection manually creating zero length files with the proper DVD name can be too difficult, but some of you have your collections catalogued. Some of the catalogue applications can output your collection as a delimited file. A simple script can use the delimiter file to create the required zero length video files.

Jamu limitations

  1. The "find_meta.py" script supports ripped DVD's with multiple files making up a single video. Jamu does not deal with DVD rips. This may become a additional feature at some time in the future.
  2. There is no chaining of TV series episodes or video files. A movie that is split into multiple video files are treated as separate video files. You could use a video utility like ffmpeg or mkvmerge (append feature) to combine multiple video files.
  3. .ts video files are not included as a supported video file extension. This can be changed by add the "ts" extension to a user configuration file variable video_file_exts. See "jamu-example.conf". Again file chaining is not supported.
  4. If you use a customized naming convention for your graphics then DO NOT use Jamu. Jamu conforms to the naming convention used by Mythvideo downloads ("imdb#.ext" for movies and "series Season #.ext" for TV series). Your graphic files will be renamed to conform to these standards. This is the only way Jamu can work seamlessly with Mythvideo downloads.
  5. Jamu does not download or deal with screen shot graphics. There has been discussion that screen shot capabilities is going to be added to MythTV.
  6. When there are Storage groups for graphics Jamu will download new graphics to the FIRST graphic storage group per graphic type (cover art, banners and/or Fan art). Jamu does not support any download balancing of graphics.

Credits

  • The jamu script could not exist without the tvdb_api module written by dbr/Ben. Time and again dbr/Ben has added functionality to accommodate the needs of MythTV. dbr/Ben has another useful script called tvnamer which renames video files to a standard format, tvnamer and tvdb_api can be found at:
    http://wiki.github.com/dbr/tvdb_api/tvnamer
    http://wiki.github.com/dbr/tvdb_api
    You can install tvnamer which will also install a version of tvdb_api without fear of being out of sync with the version of tvdb_api that Jamu uses as they are packaged separately.
  • The script tmdb_api author dbr/Ben was copied and extensively modified. tmdb.py can be found at [4]
  • The routines to select video files was copied and modified from tvnamer.py mentioned above.
  • The interactive console interface for selecting IMDB numbers and TV Series was copied and modified from tvnamer
  • The routine "_save_video_metadata_to_mythdb" has been copied and modified from "find_meta.py" author Pekka Jääskeläinen.
  • The routine "_addCastGenre" was copied and modified from "tvdb-bulk-update.py" by author David Shilvock <davels@telus.net>. Found at http://bitbucket.org/davels/tvdb-bulk-update/
  • Additional scripts are called if specified as secondary data sources e.g. find_poster.py author Pekka Jääskeläinen (gmail: pekka.jaaskelainen)
  • The Graphite theme by Robert McNamara found at http://www.gossamer-threads.com/lists/mythtv/theming/386359?page=last