MythArchive

From MythTV Official Wiki
Revision as of 01:03, 22 February 2009 by Wormholio (talk | contribs) (removing initial section header shows article summary before TOC)

Jump to: navigation, search

MythArchive is a new plugin for MythTV that lets you create DVDs from your recorded shows, MythVideo files and any video files available on your system. It can also archive recordings in a format that contains not only the file but also all the associated metadata like title, description and cut list information which will mean you can create backups of myth recordings which can later be restored or it will also allow you to move recordings between myth systems without losing any of the metadata. It is a complete rewrite of the old MythBurn bash scripts, now using python, and the mythfrontend UI plugin.

Contents

MythTV Setup

MythArchive is only guaranteed to work on combined frontend/backends. If you have MythArchive installed on a frontend-only machine it will not archive the video files stored on the backend, unless the recordings/video files folder is shared using NFS or CIFS (Samba) and even then, it is not guaranteed to work.

Main Features of the DVD creation part of MythArchive

  • Creates DVDs complete with a themed menu structure
  • New menu themes can easily be added
  • By using different themes it is possible to create auto play DVDs that have no menus etc.
  • Can use files from MythVideo and any file accessible from your file system in addition to recordings made with MythTV.
  • Can use mythtranscode to cut out commercials from MPEG-2 recordings.
  • If a file isn't an MPEG-2 file or the file doesn't have a DVD compliant resolution it will be re-encoded using ffmpeg so it is DVD compliant.
  • Can use any video file that ffmpeg (at the time it was last synced into mythtv's source code) can decode and re-encode to MPEG-2.
  • When a re-encode of the video and audio is neccessary the re-encoding parameters can be chosen using easily selectable encoding profiles.
  • New re-encoding profiles can be added by simply editing an xml file.
  • Can re-encode the audio stream to ac3 for better compatibility in NTSC countries.
  • Tries to select the best audio track based on what is set as the preferred language in Myth
  • Can support two audio tracks per title. (not tested fully)
  • Can use single layer or dual layer DVD-Rs
  • Can use rewritable DVD+/- RW
  • Can force an erase of a rewritable disc
  • Can create an ISO image that can be copied/moved to another filesystem location to facilitate playback or burning on another machine.

Main features of the raw archive format

  • Can create backups of both MythTV recordings and MythVideo files.
  • Saves not only the file but also all metadata belonging to the file like its title, description, cut list and markup map etc and any preview or coverart files.
  • Creates an easily parseable xml file containing all the metadata.
  • Can archive to any directory on a machine's file system (if allowed by permissions) or can also create an ISO image and optionally burn that image to a DVD.

MythArchive Installation

Dependencies

Package.png
The MythArchive plug-in itself does not require anything to compile and install, but the script it uses has the following dependencies that need to be installed before it will work properly:
  • Python - v2.3.5 or later
  • mysqldb module for python (dev-python/mysql-python on Gentoo; MySQL-python on Fedora; python-mysql on openSUSE; python-mysqldb on Ubuntu/Debian)
  • imaging (PIL) module for python (dev-python/imaging on Gentoo; python-imaging on Fedora 8 (PIL on earlier Fedora?); python-imaging on Ubuntu/Debian)
  • mjpegtools - v 1.6.2 or later
  • dvdauthor - v 0.6.11
  • ffmpeg - v 0.4.9
  • dvd+rw-tools - v5.21.4.10.8
  • cdrtools - v2.01
  • transcode (optional; only needed for tcrequant, but tcrequant is required for rescaling MPEGs whose content exceeds your DVD's capacity during encoding)

The versions above are what are currently being used by me on Gentoo.

Compile and Install MythArchive

From MythTV 0.20 MythArchive is included with the mythplugins package.

Run the top level configure script for mythplugins with --enable-mytharchive added to the list of parameters to make sure MythArchive will be compiled. Then compile and install the plugins in the usual way.

Additional options are --enable-create-dvd which adds DVD creation support and --enable-create-archive which adds native archive support. Both options are enable by default.

How to use MythArchive

When mythfrontend is run there should now be two extra menu items. In the Optical Disks menu, there is an Archive Files item that runs the MythArchive plugin. In the Utilities / Setup | Setup | Media Settings menu is the item Archive Files Settings to change MythArchive's settings.


Important.png Important The first thing you should do is check the settings. Make sure the temp work directory setting is set correctly. There should be a lot of space at this location for the script to create all its work files. Change the video format to what is required (PAL, NTSC) and change the location of the system's DVD drive. Most of the other setting shouldn't need changing unless some of the tools are not in $PATH in which case you will have to supply the full path so the script knows where to find them.

If using Mythtv 0.20 or earlier, first select files for archiving from the Find Files To Archive menu. You can choose MythTV recordings, MythVideo files or any media files accessible from the system's filesystem. As of Mythtv 0.21, files are selected in the second screen in the Create DVD/Native Archive menus.

To create a DVD

  • Choose Export Video Files | Create DVD
  • Choose what size DVD is required and whether files will be copied from the file system or if an ISO image for later burning or actually burn the ISO image to a blank DVD. Press 'Next'.
  • Choose what files to burn to DVD. If the file is a MythTV recording and the file has a cut list MythArchive can edit out sections of the recording according to the cutlist. If editing the files' metadata is necessary (title, subtile, description etc) press 'Menu' to show the popup menu and choose 'Edit Details'. Press 'Next'.
  • Choose the menu theme to use on the produced DVD. Press 'Next'.
  • The final page of the wizard displays the shows chosen to burn to disc and the order in which they will appear on the DVD. It also shows whether a cutlist will be used. An item can be moved up and down the list by clicking SELECT on file in the list and using the up/down keys to move the file to its new position. Pressing SELECT again will fix the file in its new position. On this page the re-encoding profile can also be chosen. Each file can have a different encoding profile if so desired. If the file is already a DVD compliant file the default is to not re-encode the file but this can be overridden with other profiles (possibly to reduce the file size to make more room for files on the final disc). The Thumb Image Selector from can also be run from this page - See below. Press 'Finish'
  • The script to create the DVD will now be run and the log viewer will now be shown.

Keys :-

  • C - toggle use cut list

To create a native archive backup

  • Choose Export Video Files | Create Native Archive
  • Choose what size DVD you are going to use and whether you want to just create the file system or create an ISO image for later burning or actually burn the ISO image to a blank DVD. Press 'Next'.
  • Choose what files you want to backup. Only MythTV recordings or MythVideo files are supported.
  • The final page of the wizard shows you the files you have chosen. Press 'Finish'
  • The script to create the backup files will now be run and the log viewer will now be shown.

To import a native archive backup

  • Choose Import Video Files
  • Use the file selector to find the xml file containing the metadata for the file you want to import. Press 'Next'
  • dvd's might be found under /media/cdrecorder
  • On the next page if the file you want to import is a MythTV recording you can choose what TV channel should be associated with the recording - this is mostly usefull if the exporting system and importing system have different channel line ups. MythArchive tries hard to find a reasonable match but there may be instances where it can't find one so you may have to choose a channel yourself. Press 'Finish'
  • The script to import the archive will now be run and the log viewer will now be shown.

Using the log viewer

Whenever a script is run to create a DVD or native archive or when the import native archive script is run the log viewer will automatically be shown. By default the viewer will automatically update every 5 seconds. You can turn this off or change the update frequency by changing the options at the top of the viewer. You can if you wish exit the viewer and continue to use Myth. If you do this and return to MythArchive and the script is still runing the log viewer will be shown again automatically. If you want to stop the script you can press 'Cancel' and the script will be asked to stop. It may take several minutes for the script to respond, depending on what it is doing at the time. The update button will force the log viewer to update.

Using the Thumb Image Selector

Only available on the svn trunk version

The thumb image selector allows you to choose what images to use on the DVD main and chapter menu's. You access it from the last page of the Create DVD wizard. First select which file you want to find the thumbs for in the list then press I/INFO to show the Thumb Image Selector screen. The screen has three main areas a large Preview Image, an info area, and an image grid that shows the selected thumbs.

You use the MENU key to change what has the focus.

When the Preview Image has focus:- The LEFT/RIGHT keys jump forward/backward using the current 'Seek Amount'. Use the UP/DOWN keys to change the seek amount. Pressing SELECT chooses the current preview image for the selected thumb image. Pressing the number keys (0-8) will jump to that thumb image.


Keys for all screens

  • Up/Down/Left/Right - move focus/change selection
  • Select - activate push button/toggle selection
  • Exit - return to myth main menu

FAQ

MythArchive doesn't work. It gets to the log viewer and just sits there. What's wrong?

The most common reason for this is python is missing some required modules. The script requires imaging (PIL) and mysqldb both of which are not part of a standard python install so have to be installed separately.

This is also a symptom of an incorrect temporary files setting. Note that the user running mythfrontend needs write access in this directory, regardless of the user running mythbackend.

Once you have addressed the above dependencies and permissions, you may need to clear out your temp DVD directory to get the log viewer to start over, making files with the correct support.

Finally, the problem might be that your frontend hasn't permission to write to one of the two directories that you've configured in MythArchive's setup. The easiest way to figure out what's wrong in a case like this is to start mythfrontend with it's stdout and stderr pointed to a useful logfile, if you don't already do this.

The path to mytharchivehelper is hard coded to mythburn.py. Double check if it is found in $PATH

MythArchive doesn't work. Selecting files does not change the used/available space, and attempting to build/burn the disk just sits there. What's wrong?

It's possible the user you're running the front end as (as opposed to the mythtv user) doesn't have write permissions on the MythArchive temporary directory. Either make that directory tree writable by the user running the front end, or change the MythArchive temp directory to one that user can write to,

After a crash/reboot MythArchive seems to have got stuck and always shows the log viewer when started

This can happen if something should prevent the script from completing like a crash, a reboot or if you kill the script before it finishes. When this happens the script leaves behind a lock file which prevents the script from restarting.

In order to fix this you need to locate the lock file called 'mythburn.lck' and delete it. You will be able to locate it in %WORK_DIR%/logs/mythburn.lck where %WORK_DIR% is the directory you told MythArchive to use to store its temporary work files ('Myth Archive Temp Directory' setting on the first page of MythArchive settings).

NOTE: The newer version of MythArchive in svn trunk which will be 0.21 is a lot smarter and will automatically remove the lock file if it detects the script is no longer running.

The log says "OSError: [Errno 13] Permission denied: '/dev/dvd/'

The user MythArchive is being run as does not have access rights to write to the dvd. Check what group is needed to access it with (for example) : 'ls -l /dev/dvd'. If /dev/dvd is a symlink follow this symlink to see what group is needed. For example, with Gentoo it would be necessary to execute:

 usermod -a -G cdrom mythtv

MythArchive fails to burn the DVD with the error "IOError: [Errno 5] Input/output error"

This error and fix applies to latest svn from trunk only

The exact error looks something like this

 Traceback (most recent call last):
 File "/usr/share/mythtv/mytharchive/scripts/mythburn.py", line 4681, in main
   processJob(job)
 File "/usr/share/mythtv/mytharchive/scripts/mythburn.py", line 4483, in processJob
   BurnDVDISO()
 File "/usr/share/mythtv/mytharchive/scripts/mythburn.py", line 2204, in BurnDVDISO
   r = ioctl(f,CDROM.CDROMEJECT, 0)
 IOError: [Errno 5] Input/output error


This is caused by the media monitor mounting and locking the DVD. The fix until a better way can be found is to tell the media monitor not to monitor the drive you use to burn DVD's. You simply add it to the list of devices to ignore on the fourth page of general settings. Just unchecking the option to monitor devices doesn't seem to work.

Why does MythArchive take a lot longer than the old MythBurn to create a DVD?

The old MythBurn scripts would add any mpeg to a DVD even if it didn't have a DVD compatible resolution. MythArchive on the other hand tries to make sure the DVDs it creates are as close to the DVD standard as possible and will re-encode any files that don't have a DVD compliant resolution.


How can I speed up DVD creation?

If the source of recordings is a hardware encoding tuner card MPEG-2 files and the card allows the capture resolution to be set(PVR-x50 for example), a lot of re-encoding time can be eliminated by setting the capture resolution to a DVD compliant resolution.

For PAL countries they are 720x576, 704x576, 352x576 and 352x288

For NTSC countries they are 720x480, 704x480, 352x480 and 352x240

My created DVDs play OK but the audio is out of sync with the video?

This can usually be fixed by telling the script to always run

mythtranscode --mpeg2 

to fix any errors in a file. You do this by setting the 'Always run MythTranscode' option on the MythArchive settings page.

One or more of the files on my DVD seem to have the audio missing

This is often caused by lame players not supporting DVD's with a mix of AC3 and MP2 audio tracks. The internal player is known to have this problem. The best fix is to enable the 'Always Encode to AC3' option so that all the tracks are in AC3 format. If you have already created DVDs with this problem you can usually still play them OK if you tell the player to switch audio tracks.

My recordings have two audio tracks but the created DVDs only have one. How do I add the second track?

MythArchive can add one or two audio tracks to DVDs it creates. It will always choose tracks that best match you prefered languages you have set on the localization settings page. If both languages are set to the same then it will never add the second audio track even if one is available. So basically to get two tracks on the DVD you need to choose two different preferred languages.


So what if my broadcaster sends two tracks both flagged as English and I want to add both tracks to the DVD?

Choose 'English' and 'Undefined' for the preferred languages. If MythArchive can't find an exact match for any track it will automatically choose the best track it can find preferring ac3 over mp2 if a choice is available.

What if I want the first audio track to be in my native language and the second track in 'English'?

Set the first language preference to your native language and the second to 'English'. If your broadcaster sets the language codes correctly then MythArchive will always select the correct tracks. Even if your broadcaster doesn't always flag the audio tracks with the correct language the track selection algorithm used will usually do the right thing, it will select the first and second tracks available in the stream which should be the primary (native language) and secondary (English) tracks. The only time it may fail is if there are 3 or more tracks available and the broadcaster hasn't added the correct language codes. In that case it will use the first two best tracks available.


Looking at the log output I see that mplex exited and reported:

**ERROR: [mplex] Too many frame drops -exiting

This can be due to the input recording exceeding the maximum dvd datarate, see http://forum.doom9.org/archive/index.php/t-100234.html, in the cases where recording is being performed without transcoding.

If you had selected 'record without transcoding' during mytharchive setup try switching to the HQ profile. This should transcode to the dvd max rate and avoid the problem.

MythArchive reports that files are not available locally

MythTV usually streams video if you have a separate frontend and backend. MythArchive cannot use streamed video: it needs to be able to access the file locally. This can be accomplished by exporting the directory containing your recordings via NFS and then adding the mountpoint to the frontend's Storage Group.

Mythtv 0.20.2 don't use Storage Groups so you get these error in the MythArchive log if your recording path are not NFS exported.

This error and fix applies to Mythtv 0.20.2 only. Thanks to Bob http://notepad.bobkmertz.com/2007/09/mytharchive-on-seperate-front-end.html

The exact error looks something like this

Traceback (most recent call last):
 File "/usr/share/mythtv/mytharchive/scripts/mythburn.py", line 3600, in ?
   processJob(job)
 File "/usr/share/mythtv/mytharchive/scripts/mythburn.py", line 3358, 
in processJob
   files=copyRemote(files,getTempPath())
 File "/usr/share/mythtv/mytharchive/scripts/mythburn.py", line 3261, 
in copyRemote
   tmpfile = string.split(os.path.join(recordingpath, 
node.attributes["filename"].value), '/')
 File "/usr/lib/python2.4/posixpath.py", line 62, in join
   elif path ==  or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'

Exporting the directory containing your recordings via NFS and then add the mountpoint in Mythtv database. In Mythtv 0.20.2 you must insert the setting 'RecordFilePrefix' into the database for the frontend profile.

mysql -h 192.168.1.3 -u mythtv -p mythconverg
(Replace the IP with your database server - usually same as your backend)
insert into settings (value, data, hostname) VALUES ('RecordFilePrefix', '/storage/recordings', 'mythfrontend.local');
(You'll need to change 'mythfrontend.local' to whatever the hostname is of your frontend and change to the correct path recordings path)

How do I add my own encoding parameters to the encoding profiles?

Place copies of the profiles found in INSTALLDIR/share/mythtv/mytharchive/encoder_profiles/ into MythArchive's config directory (~/.mythtv/MythArchive). MythArchive will then use these profiles in preference to the shared ones. You can either edit the original profiles or add your own new ones. Just make sure you choose the correct profile in the UI when you want to re-encode a file using your custom profile settings.

My DVD player has problems playing some DVDs created by MythArchive - Playback is very jumpy.

Some DVD players seem to not detect whether a video is interlaced or not properly and relies on the header in the file to be correct. Sometimes when a DVD is re-encoded using ffmpeg the header gets created with the wrong interlaced flags.

The fix is to add the following line to the re-encoding profiles in ffmpeg_dvd_ntsc.xml for NTSC or ffmpeg_dvd_pal.xml for PAL:

   <parameter name="-flags" value="+ilme+ildct"/>

See above on how to use custom encoding profiles.

After upgrading to Mythtv 0.21, the "select files" screen in MythArchive has old videos. How do I remove them?

With the implementation of storage groups, files to be archived are now stored in the database with a full path.

Your database have some old (pre 0.21) archive items in the archiveitems table that don't have the full path. Run this query in mysql to remove all the old items and start with a clean table.

 TRUNCATE archiveitems;

After upgrading to Mythtv 0.21, I don't see a way to add files to my archive.

The second screen of the Create DVD and Create Native Archive menus have buttons for Add Videos, Add Recordings, and Add Files. If these buttons are not visible, try using the MythCenter theme in Setup | Appearance menu. Older and unsupported themes won't have support for these new buttons.

There should also be a Remove menu item to allow one to remove an entry from the list. And perhaps a Clear menu item to clear the entire list. The "clear" option would also take care of the problem immediately above. --Wormholio 00:19, 8 December 2008 (UTC)

Links



Mytharchive.jpg