Difference between revisions of "User:Mrsdonovan"

Revision as of 03:00, 5 April 2008

I've been using MythTV since 0.18 and most of my contributions have been to the wiki in regards to modules I've worked with, like Firewire, Mythgame and playing recordings directly from Mythweb.

The setup below is used daily by the kids, parents and grandparents alike and has a high WAF factor. The external VHS and DVD players rarely never get used anymore.

My current statistics are:
Number of shows: 2236
Number of episodes: 13248
First recording: Friday May 6th, 2005
Last recording: Friday April 4th, 2008
Total Running Time: 2 years 10 months 29 days 22 hrs 21 mins
Total Recorded: 1 year 4 months 23 days 45 mins
Percent of time spent recording: 47%

Setup

A - MythTV Master backend/ HD Frontend / Asterisk server

• 3Ghz HT Intel with 2GB of dual channel RAM
• 1GB ethernet jack connected to a 10/100/1000Mbps Dlink 8 port router
• HD capable Geforce 6200 video card.
• RAID 5 with four SATA harddrives, roughly about 900GB of space for recordings and 560GB for videos/music/pictures/ROMs.
• ASUS P4P800-E-Deluxe
• Haupauge PVR-150
• Haupauge PVR-250 with remote
• Digium Wildcard TDM400P with three S100-M FXS expansion cards
• Asterisk @ Home
• Two Motorola DCT2524 cable tuners controlled by two IR blasters connected to a serial card. (Custom built lirc_serial kernel module, ledxmit_serial)
• DCT-6200 tuner connected via an Agere firewire card for HD
• Linux Fedora Core 7
• 19" LCD

B - MythTV HD Frontend / Asterisk

• ASUS Pundit P1-AH2 (black)
• AMD X2 3800+ Dual Core
• 1GB of RAM
• Onboard NVIDIA Geforce 6150
• Linux FC7
• Digium Wildcard TDM400P with three FXS expansion cards
• Asterisk @ Home
• MythGame with MAME emulation and a wireless Logitech Rumblepad 2
• Samsung LT-P266W 26" HDtv flat-panel LCD

C - MythTV HD Slave Backend / Frontend

• 
• 
• 
• 
• Akai LCT2701TD.jpg
• 

• AMD Sempron 3400+ with Thermaltake Blue Orb CPU Cooler
• ASRock motherboard with onboard Geforce 6100
• 1GB of RAM
• Via Firewire card connected to a DCT6214 (Yes, it is a PVR, I know..)
• Roughly 600GB in a Raid 1/Mirrored setup and 525GB LVM for video backup (rsync'd each night with master server)
• MythGame with MAME emulation and a wireless Logitech Rumblepad 2
• Fedora Core 7
• Akai LCT2701TD - 27" LCD with DVD player (not used)

D - MythTV Frontend

• 
• Sony universal RM-V302.jpg
• 

• 2.4Ghz Celeron (Not used for HD)
• 80GB HD
• 768MB of RAM
• Asus Pundit (The SIS version)
• Linux FC7
• Sony Trinitron 32" Flat Screen CRT
• Goes into shutdown after 3 hours using sleepd (svideo doesn't recover correctly on resume!?)
• Generic Sony Pre-programmed remote with ira-3 serial receiver.

Notes

• The link between the MythTV frontends B, D and the master and slave backends A, C is over an 5.8Ghz 802.11a wireless bridge using dual-band bridges from Trendnet (TEW-510apb) and 9dbi gain antennas. Amazingly, the 720p High Def shows work smoothly over the link although the 1080p still needs more bandwidth. The link is affected by thick ice on the roof and really heavy rain.
• All the machines have access to a shared video/pictures/music/ROM 500+GB LVM partition on the main server (A) and over a terabyte of space for recordings.
• We currently use link2voip.com as our DID and outbound provider for Asterisk with great ping times of 10 to 15ms.
• We use most of the default plugins including MythGame, pictures, videos, weather and MythPhone CallerID with XYAC.

Tips from the HD Front Lines

I have been using MythTV with HD now for a couple years and wanted to share what has worked and what has not. These comments could save you a lot of time and frustration:

Firewire cards - get the best firewire cards that works with your tuner(s). I have used NEC, VIA (onboard and PCI) cards and now finally, Agere cards. Agere cards, the lastest chip revisions anyway, are by far the best with the Motorola DCT-62xx series boxes. Ironically, "test-mpeg2" does not work making any priming script useless, but "broadcast" mode works pretty much flawlessly and fast! You do still have to prime the firewire card, but I do it once at startup with the "firewire_tester -n 0 -B" script.

VIA firewire chipsets will work, as they have for over a year with my system in P2P mode, but it sucks. VIA chips require a priming script to run each time a channel change occurs and have two negative effects - first, because the priming is done at each channel change, channel surfing during Live TV times-out 2/3rds of the time. Recordings seem to work %100 of the time. Secondly, the priming script + VIA chipset + DCT-6200 crashed the DCT-6200 tuner when tuning to SD channels (seriously! Mine would hard lock and have to be unplugged). I have no clue why because the same combination but with a DCT-6214 worked fine. The tuner hard lock also doesn't happen on HD channels.

Conclusion - With DCT-62xx tuners, get Agere-5t FW323 v61 firewire chips, dump the priming script and use Broadcast at 200mbps.

Video cards - I have had NO luck getting a color OSD or fixing the stuttering problem with anything but onboard NVIDIA Geforce 6150s which work out of the box. I have tried FX-5200 and 6200 cards but both have black & white crappy looking OSDs and cause stuttering with an overlay on HD. The onboard 6150s, for some unknown reason, have did have color OSD and do not cause stuttering. (something changed in v0.21)

Conclusion - Get onboard NVIDIA Geforce 6150s and with a large enough processor, HD will work flawlessly with the OSD in color and no stuttering.

Processor - Get the largest and fastest one for HD because it makes a difference. Everything is magnified by two with HD content, which means it takes more processor power to do playback, commercial detection and transcoding. A faster processor also helps with database queries. (As an aside, I also suggest looking into keeping the database in memory)

Conclusion - Faster = better.

Remote Control - From a usability stand point, an excellent remote is important. I have settled on the Microsoft MCE remotes because they have lots of features that make it all come together - First, they can be trained with your existing TV remote to shut off the TV and control the volume of the TV. The volume control in MythTV frankly sucks because it only works when a recording is actually playing and doesn't go high enough (a fault of the hardware actually). Being able to turn off the TV and control it's volume directly mean you can chuck your TV's remote. If you are playing DVDs on your MythTV system, you can now have one remote to control everything and that rocks. Better, if you wreck the remote, they are inexpensive ($35 locally) and widely available. The receiver has wide reception, a quick response rate and the layout on the remote is excellent with important buttons in easy to reach places. The downside is that because it is back lit, it uses batteries quickly (every four months) but hopefully rechargeable ones in your case.

Conclusion - Get the Microsoft MCE remote and train it to control your TV as well as your MythTV box. One remote to control them all.

Hard drive Size - On the main backend (A) I started out with an LVM which tied a bunch of disparately sized disks together, but I wanted more redundancy and speed. I tried RAID 5 but ... [UPDATE] I have switched back to RAID 5 and it seems to work fine - best use the latest kernel to get all the latest updates. (Others have reported success with RAID 5) Having said that, the slave backend (C) [UPDATE] now has RAID 1 redundancy and records HD to two SATA II drives which seems plenty fast enough.

Hard drive size really only translates into one thing - longer time to watch something before it expires. For just recordings, our system has a combined space of about a terabyte which means our recordings last about a month. Our system records, on average, about eighteen one hour shows a day on weekdays and upwards of thirty or more on weekends with a mix of everything from sports to movies to the daily kid's shows. Lots of it never gets watched. Of those recordings we want to keep longer term, the best solution has been to transcode and move them into our video section that exists on a separate harddrive (An LVM of all the free space left over from the RAID actually). That harddrive is then RSYNC'd each night from the main server (A) to extra space on the slave backend (C).

Conclusion - Larger = Longer lasting. RAID is your friend.

Keeping HD In Sync When Transcoding

One of the benefits of MythTV is that once the shows are recorded, they can be manipulated in many different ways, for example, the commercials stripped out and the rest dumped onto a DVD or transcoded with an open source codec like Xvid. Transcoding HD recordings from either over-the-air or firewire captures presents significant difficulties, mostly related to keeping the audio in sync with the video.

I have found, through experience, that an MPEG-2 TS file fresh from being recorded over firewire from a DCT-62xx tuner, and edited with something like Avidemux2, will result in an audio sync that will get worse as the recording plays. By the end of the recording, what started as a 250ms sync, is now out by a couple of seconds.

There are two problems related to keeping the audio and video in sync, the first are errors in the stream, which over time create small differences. Unlike VLC which constantly adjusts the sync based on time codes, editors like Avidemux do not. The other problem is commercials, which not only interrupt the MPEG-2 stream, but do other nasty things like change framerates and drop to lower resolutions.

Looking over hundreds of forums, the consensus has been to pre-process the stream with ProjectX, an open source Java utility, which, for the longest time, worked. I would edit out the commercials with ProjectX's slow interface, demux the final cut, load it into Avidemux, load the filters and transcode. I like Avidemux because it is open source, has a slick interface and rips to a wide array of formats with lots of filters, like cropping, deinterlacing, framerate and resizing. Moreover, it's greatest feature is that it runs on both Windows and Linux which means the many hour transcodes can occur, as a background process, on our 24/7 Linux boxes. Unfortunately, Avidemux happens to be terrible at decoding raw MPEG-2 streams and isn't the most stable of programs.

However, ProjectX has recently not been able to deocode the MPEG-2 streams nor can repair utilities like mpeg2repair fix them. I was stuck.

The great part is that another MythTV user created a script to rip HD, that, for some reason, keeps the audio in sync. Unfortunately, the script only works on raw MPEG-2 streams which means it doesn't run on the transcoded commercial free recordings that MythTV can create. Besides doing the commercial detection automatically, MythTV has a fast interface for fixing the cut points and it is suppose to be able to do a straight lossless MPEG-2 to MPEG-2 copy. Unfortunately, because of a long outstanding bug with mythtranscode, it doesn't work.

If mythtranscodes's lossless copy did work, the whole process could be automated with a few clicks of the remote:

1. Manually adjust any cut points (if necessary) from what MythTV has detected
2. Tell MythTV to do a lossless MPEG-2 to MPEG-2 transcode which removes the commercials
3. Tell MythTV to run the user HD script to make an Xvid archival copy, dumping the result in a certain spot to be renamed and moved into an appropriate directory later.

Without step 2, you have to run the HD script on the raw recording with the commercials included, load the result into Avidemux and manually find and cut the commercials. There is still an audio video sync problem, but I have found that whatever mencoder does, the audio delay is a straight 250ms delay, consistent from recording to recording and constant across the entire recording. The other good news is that if you don't have to crop anything, the Xvid to Xvid copy is fast, taking less then a minute for a two hour long show.

The short of all this is - if you want to transcode an HD recording that you cannot edit with ProjectX use the following process:

1. Process the raw HD MPEG-2 TS recording with the HD script
   • I had to modify the script to accept larger files and create a smaller end result (640x360). I can post the modified script here if anyone is interested.
   • This transcode takes a long time, upwards of six to eight hours on a Pentium 3Ghz machine.
2. Load the resulting file into Avidemux, leave the audio and video as "copy", cut the commercials and set the audio delay to 250ms
3. Save it and hit "yes" to smart copy. Sixty seconds later you have a perfectly transcoded and in sync result.

Other Random Tips

• FreePBX includes PHPMyadmin which is useful for optimizing the database(s) which saves space and makes them run faster. PHPMyadmin also comes in handy for editing the database directly, like getting rid of commercial detection jobs if mythcommflag hangs, or reordering the tuners.
• Concerning Asterisk, I tried ADSI, the ability to program phones with custom scripts, but some phones just didn't work because they are "locked" by their manufacturer. The comedian mail app for ADSI works, but it's clunky and slow and it was much simpler just to autologin and play the messages.
• Monitoring of the Myth systems work great using Gkrellm and I highly recommend it. Works on Linux and Windows and looks great with a large variety of skins.

Personal Setup Notes

Notes for my own setup:

Distinctive Ringing

Setting up distinctive ringing for ring groups.
1. Via A@H or FreePBX, create two ZAP extensions for each ZAP channel,

a. The lower number one, i.e. ext 201 uses a "dial" command "ZAP/1r2", where 1 is the ZAP daughter card in slot 1 and "r2" stands for ring type 2 (or r3 or r4). This extension will ring the phone using the distinctive single/long ring.
b. The second ZAP extension, i.e 211, uses just "ZAP/1" as the dial command and will ring the same ZAP phone with a regular US ring tone.

2. Then set it up such that internal callers go to ring group 200 which rings all the distinctive ZAP extensions, i.e. x201, etc. When external callers call, their separate ring group calls x211, etc. with regular ring commands.

The great thing is that the modifications are done through A@H and don't need custom modification of any script.

• To avoid entering a password every time for voice mail, add the voice mail's extension "sXXX" into extensions.conf -> app-messagecenter instead of "default", i.e. s402

Caller ID

Code for sending caller ID from Asterisk to MythTV Frontends. Requires XYAC. Use the startup script below to start up the XYAC daemon at login.

exten => s,1,System(/bin/echo -n -e "@CALL${ARG1} ~${ARG2}" | nc -w 1 127.0.0.1 10629 &)

Note:

• The "&" at the end runs the "nc" daemon in the background where it can fail silently and not cause delays in answering the calls. Without the "&", the far side can intermittently time-out just as the phones start ringing.

Idle Three Hour Sleepd Daemon

• Below is a method of using the sleepd daemon to shut off a MythTV frontend after three hours and to pop up a 30 second warning that the user can cancel.

# First in the [user]/.kde/Autostart/ directory add the following bash script that
# starts up the sleepd daemon, the caller ID dameon, etc.
# As the MythTV user autologs in, the script uses a nice kdialog status pop-up so 
# the user can see what is happening or if it is getting stuck anywhere:
#!/bin/sh

# Open the kdialog progress bar
dcopRef1=`kdialog --progressbar "Starting up MythTV..." 5`
dcop $dcopRef1 setProgress 1

dcop $dcopRef1 setLabel "Now starting Mythfrontend"
dcop $dcopRef1 setProgress 2

# Run the MythFront End and append it's log
/usr/bin/mythfrontend >> /home/mythtv/mythfrontend.log  &

dcop $dcopRef1 setLabel "Now running the CallerID Listener"
dcop $dcopRef1 setProgress 3

/usr/local/bin/xyac -c yellow -f -adobe-utopia-bold-r-normal--*-400-*-*-*-*-* -s 1 -d 15 & 

dcop $dcopRef1 setProgress 4
dcop $dcopRef1 setLabel "Starting Sleep Daemon to shutdown in three hours."
# Start up the sleep daemon
# -U waits for 10800 seconds
# -i watches IRQ 4, the remote to reset the counter
# -a Also watches the mouse and keyboard
# -s then runs the standby.sh script
/usr/sbin/sleepd -U 10800 -i 4 -a -s /home/mythtv/standby.sh

dcop $dcopRef1 setProgress 5
dcop $dcopRef1 setLabel "Finishing Up - Checking for Alien Infestation ;]"
sleep 3
dcop $dcopRef1 close

Premptable Shutdown Script

• Here is the standby.sh script called by the sleepd daemon for shutdown:

Note: shutdown requires root access which means using sudo

#!/bin/sh

# Not sure why, but it was necessary to run this before the showCancelButton call
# would work reliably.
sudo kbuildsycoca

# Play the sounds - gives an audio as well as visual indicator of shutdown
# I threw a bunch of sounds together with Audacity.
mplayer cut_power.mp3 &

# Now call the main kdialog box
# The "30" on the end is the number of sections the progressbar has before being %100
dcopRef=`kdialog --title "Shutting Down" --progressbar "Shutting down computer in 30 seconds. Press Cancel at Any time" 30`
# Then show the cancel button
dcop $dcopRef showCancelButton true

# Now every second, loop through, updating the progress bar, checking to see if
# cancel was pressed.  If no cancel was pressed, it enters the if statement &
# prints a useless notification and immediately shuts down.
# This loop doesn't cleanly exit if no cancel button is pressed, but who cares
# cause the computer is shutting down anyway...
until test "true" == `dcop $dcopRef wasCancelled`; do
    sleep 1
    inc=$((`dcop $dcopRef progress` + 1))
    dcop $dcopRef setProgress $inc;
    if [ $inc -eq 30 ] 
    then
	# Close the dialog
	dcop $dcopRef close
	kdialog --title "Shutting Computer Down Now!" --passivepopup "Nooooooo...." 5 &
	sudo /sbin/shutdown -h now
    fi
done

# The cancel button was pressed, clean-up time
dcop $dcopRef close
# There is a possibility the user pressed the cancel button before mplayer 
# had finished, but this will also kill any movie playing in the background too! 
# Oops.
killall mplayer

The Plus and Minus of MythTV

• Note this was a comment added to an article comparing TIVO 3 and MythTV.

I've been a MythTV user for a couple years now and there are a number of things the article missed or glossed over ("+" means a feature and "-" a problem).

(+) Plugins - MythTV has lots of plugins that let you do everything from surf the web, view local weather information, play old style ROM arcade games, watch recordings over the LAN on a windows machine, view a slide show of pictures, to watching XviD encoded videos, all of which we use. The video collection has been of great utility with kids. The usefulness of the plugins was a big reason I chose MythTV and have stayed with it. (Actually TIVO was never considered because of the subscription costs)

(-) Setup and Maintenance - It should not be underestimated how much time it can take getting MythTV to work and then maintaining it. As a MythTV user you will have to know the Linux command line, how to download and install drivers and fix things when they inevitably don't work. If you haven't installed a PCI card before, and don't like googling stuff to figure out why it doesn't work, then MythTV is not for you. MythTV will probably not save money over a commercial system either because I've found that the money I would have spent on a subscription went into hardware upgrades.

(+) The flip side to the cost and time commitment is that MythTV has immense flexibility, for example, in the size and redundancy of storage capability, the type of peripherals and the split arrangement between backend, slave backend and frontends. Take our ever growing video collection (most ripped from DVDs, fyi), where any new videos are backed up onto a separate machine each night (rsync'd). Try doing that with a TIVO.

(+) The machines are also full blown general purpose computers that can do other things. For example, once you have sunk the money into the MythTV hardware, you can use the same machines for VoIP (via Asterisk) which really saves us money, about $100CND a month. The pay back period for the extra hardware to do VoIP was less then a year, and gets cheaper as time goes on. We use dedicated Digium TDM-400 cards that cost ~$500CND about a year ago. Again, our long distance costs are pennies per minute, each phone is a separate extension, we can make multiple outgoing or incoming calls, late-night callers are prompted by a menu first before the phones ring, we can control our own caller-ID, etc.

(-) DRM can still be a problem with MythTV, for example, some of our HD channels, which we get from the tuner over a firewire cable, have the "5C" flag set. The end result is that those channels and some HD broadcasts are not transmitted over the firewire and MythTV records nothing. For HD tuners using firewire and 5C flagged channels, there is no current work around (as of Feb. 21st, 2007). But I'm sure that will change.

Outstanding Issues

A long suffering problem is the MythBackend restarting with errors of the type:

2007-12-12 08:40:47.268 RingBuf(??ythTV/recordings/1084_20071211183200.mpg): Could not open ??ythTV/recordings/1084_20071211183200.mpg.

A few of those and the process eventually dies and gets reset by the watchdog process. Clearing the schedule data and re-adding it fixes the problem for a few months, but eventually the process starts crashing every couple of days. It is not a huge issue cause it interrupts a few shows which get deleted and rerecorded. Hopefully this has been solved in 0.21