User:Mrsdonovan

From MythTV Official Wiki
Revision as of 21:54, 25 December 2010 by Mrsdonovan (talk | contribs) (Statistics: Oops, wrong size for video parition space.)

Jump to: navigation, search

I've been using MythTV since 0.18 and most of my contributions have been to the wiki in regards to modules I've worked with, like Firewire, Mythgame and playing recordings directly from Mythweb.

The setup below is used daily by the kids, parents and grandparents alike and has a high WAF factor. The external DVD players rarely get used anymore.


Statistics

My current statistics are:

Number of shows: 3064
Number of episodes: 38393
First recording: Friday May 6th, 2005
Last recording: Saturday December 25th, 2010
Total Running Time: 5 years 7 months 20 days 4 hrs 22 mins
Total Recorded: 3 years 11 months 17 days 21 hrs 2 mins


System A Recording Space: 2,289,109 MB (RAID 5)
System B Recording Space: 2,623,048 MB (LVM)
Total Recording Space Available: 4,912,157 MB (5 TB)


System A Video space: 1,022,540 MB
System B Video space: 1,465,076 MB
Total Space for Videos Available: 2,487,616 MB (2.5 TB, All RAID 5)


Setup

System A

This system is used at my house and consists of the following:

A.a - MythTV Slave backend/ HD Frontend / Asterisk server

  • 3Ghz HT Intel with 2GB of dual channel RAM
  • 1GB ethernet jack connected to a 10/100/1000Mbps Dlink 8 port router
  • HD capable Geforce 6200 video card.
  • RAID 5 with four SATA harddrives, roughly about 900GB of space for recordings and 1TB for videos/music/pictures/ROMs.
  • ASUS P4P800-E-Deluxe
  • Haupauge PVR-150
  • Haupauge PVR-250 with remote
  • Digium Wildcard TDM400P with three S100-M FXS expansion cards
  • Freepbx with Asterisk 1.6
  • Two Motorola DCT2524 cable tuners controlled by two IR blasters connected to a serial card. (Custom built lirc_serial kernel module, ledxmit_serial)
  • DCT-6214 (Yes, it is a PVR, I know..) tuner connected via an Agere firewire card for HD
  • MythBuntu 9.10 (32bit)
  • 22" LCD
  • Belkin 1500VA UPS with Bulldog software


A.b - MythTV HD Master Backend / Frontend

  • Intel Core Dual E8400 3.0Ghz
  • Motherboard with onboard Geforce 7100 (no XVMC!) Nvidia 7900GT
  • 1GB of RAM
  • Via Firewire card connected to a DCT6200
  • Roughly 1.2TB in a Raid 5 setup and 1TB LVM for video backup (rsync'd each night with master server)
  • MythGame with SDLMame emulation and a wireless Logitech Rumblepad 2
  • MythBuntu 9.10
  • Toshiba Regza 42RV530U - 42" LCD driven at max resolution of 1900x1080 via an DMI to HDMI cable. We use the TV remote with an USB MCE receiver.
  • Belkin 1200VA UPS with Bulldog software


A.c - MythTV Frontend (System not currently in use)

  • 2.4Ghz Celeron (Not for HD)
  • 80GB HD
  • 768MB of RAM
  • Asus Pundit (The SIS version)
  • MythBuntu 9.10
  • MCE Remote with programmable buttons to control the TV.


A.d - MythTV SD Frontend

  • 2.4Ghz Celeron (Not for HD)
  • 80GB HD
  • 768MB of RAM
  • Asus Pundit (The SIS version)
  • MythBuntu 9.10
  • Sony Trinitron 32" Flat Screen CRT
  • Goes into hibernation after 3 hours using sleepd (svideo doesn't recover correctly on resume!?)
  • MCE Remote with programmable buttons to control the TV.
  • This system connects through a hard-to-find Dlink DAP-1555 wireless bridge.


A.e - MythTV HD Frontend (Virtual)

Screen shot of a five monitor setup, one running virtual MythBuntu
  • A virtual 2.4Ghz VMware Guest (The host machine is a six core 3.2Ghz AMD with 8GB RAM running stock Windows 7 off two 80GB SSD raided)
  • virtual 6GB HD
  • virtual 1GB Ram
  • MythBuntu 9.10
  • 1900x1080 Monitor
  • Both SD and HD play fine under VMware Workstation (which uses proper Directx 9c drivers, unlike the free VMware Server)
  • Here is a picture of the setup


System B

This system lives at my brother's house although I maintain it and it is a separate system.

B.a - MythTV HD Frontend

  • INTEL 2.8Ghz P4
  • 1.5GB of RAM
  • NVIDIA Geforce 8500
  • Samsung LN46B540P 46" LCD
  • Mythbuntu 9.10 (32bit)
  • MythGame with SDLMame emulation and a wireless Logitech Rumblepad 2
  • Goes into hibernation after 3 hours using sleepd


B.b - MythTV HD Backend / Asterisk

  • Quad Core 2.5Ghz Intel running on Mythbuntu 10.4 LTS (overkill, but wow, responsive)
  • 700GB of RAID 5 for videos/pictures/music/ROMs
  • 2.5TB LVM for recordings.
  • 500GB for backups of other computers.
  • Haupauge PVR-150 connects to a cheap ass Motorola tuner and is controlled via an IR blaster controlled by the USB MCE remote receiver.
  • Digium Wildcard TDM400P with three FXS expansion cards
  • Freepbx with Asterisk 1.6
  • Samsung LT-P266W 26" HDtv flat-panel LCD
  • Belkin 900VA UPS with Belkin Bulldog software


Notes

  • The link between the MythTV frontends B, D and the master and slave backends A, C is over a 5.8Ghz 802.11a wireless bridge using dual-band bridges from Trendnet (TEW-510apb) and one 21dbi and one 9dbi gain antenna. Average throughput for 1080p HD is about 13mbps and during high motion scenes, tops out at 19mbps. With the high gain antennas, the TEW-510apbs can operate at their maximum throughput of 21Mbps and HD is smooth. On rare occasions, the link is affected by thick ice on the roof and really heavy rain. (System B is now separate)
  • Given three of the systems (A.a, A.b and B.b) are on 24/7, they each have a UPS rated at 900VA or better, which lasts an average of twenty minutes before cleanly shutting down (important for the RAIDed drives).
  • We currently use link2voip.com as our DID and outbound provider for Asterisk with great ping times of 10 to 15ms.
  • Most of our machines use gigabyte Ethernet and both systems use Linksys WRT54GL variants running the open source firmware, Tomato or the mini DRT variant. There have been a lot of gateways in this house but the open source Tomato is by far the best. It handles QoS properly for VoIP and has all the bells and whistles every other gateway lacks. We also use OpenDNS for all the DNS queries.
  • We use most of the default plugins including MythGame, pictures, videos, weather and XYAC for CallerID.
  • Both Asterisk boxes are connected two each other over an IAX2 link that works well through NAT and our mutual ISP.
  • On System A, we also run two TrendNet (TV-IP252 and TV-IP422W) cameras for security that record any motion detected to A.a

Tips from the HD Front Lines

I have been using MythTV with HD now for a couple years and wanted to share what has worked and what has not. These comments could save you a lot of time and frustration:

Firewire cards - get the best firewire cards that works with your tuner(s). I have used NEC, VIA (onboard and PCI) cards and now finally, Agere cards. Agere cards, the lastest chip revisions anyway, are by far the best with the Motorola DCT-62xx series boxes. Ironically, "test-mpeg2" does not work making any priming script useless, but "broadcast" mode works pretty much flawlessly and fast! You do still have to prime the firewire card, but I do it once at startup with the "firewire_tester -n 0 -B" script.

VIA firewire chipsets will work, as they have for over a year with my system in P2P mode, but it sucks. VIA chips require a priming script to run each time a channel change occurs and have two negative effects - first, because the priming is done at each channel change, channel surfing during Live TV times-out 2/3rds of the time. Recordings seem to work %100 of the time. Secondly, the priming script + VIA chipset + DCT-6200 crashed the DCT-6200 tuner when tuning to SD channels (seriously! Mine would hard lock and have to be unplugged). I have no clue why because the same combination but with a DCT-6214 worked fine. The tuner hard lock also doesn't happen on HD channels.

Information.png Conclusion - With DCT-62xx tuners, get Agere-5t FW323 v61 firewire chips, dump the priming script and use Broadcast at 200mbps.

Video cards - I have had NO luck getting a color OSD or fixing the stuttering problem with anything but onboard NVIDIA Geforce 6150s which work out of the box. I have tried FX-5200 and 6200 cards but both have black & white crappy looking OSDs and cause stuttering with an overlay on HD. The onboard 6150s, for some unknown reason, have did have color OSD and do not cause stuttering. (something changed in v0.21)

Information.png Conclusion - Get onboard NVIDIA "Purevideo" with XVMC acceleration and with a large enough processor, HD will work flawlessly with the OSD.

Processor - Get the largest and fastest one for HD because it makes a difference. Everything is magnified by two with HD content, which means it takes more processor power to do playback, commercial detection and transcoding. A faster processor also helps with database queries. (As an aside, I also suggest looking into keeping the database in memory)

Information.png Conclusion - Faster = better.

Remote Control - From a usability stand point, an excellent remote is important. I have settled on the Microsoft MCE remotes because they have lots of features that make it all come together - First, they can be trained with your existing TV remote to shut off the TV and control the volume of the TV. The volume control in MythTV frankly sucks because it only works when a recording is actually playing and doesn't go high enough (a fault of the hardware actually). Being able to turn off the TV and control it's volume directly mean you can chuck your TV's remote. If you are playing DVDs on your MythTV system, you can now have one remote to control everything and that rocks. Better, if you wreck the remote, they are inexpensive ($35 locally) and widely available. The receiver has wide reception, a quick response rate and the layout on the remote is excellent with important buttons in easy to reach places. The downside is that because it is back lit, it uses batteries quickly (every four months) but hopefully rechargeable ones in your case.

Information.png Conclusion - Get the Microsoft MCE remote and train it to control your TV as well as your MythTV box. One remote to control them all.

Hard drive Size - On the main backend (A) I started out with an LVM which tied a bunch of disparately sized disks together, but I wanted more redundancy and speed. I tried RAID 5 but ... [UPDATE] I have switched back to RAID 5 and it seems to work fine - best use the latest kernel to get all the latest updates. (Others have reported success with RAID 5) Having said that, the slave backend (C) [UPDATE] now has RAID 1 redundancy and records HD to two SATA II drives which seems plenty fast enough.

Hard drive size really only translates into one thing - longer time to watch something before it expires. For just recordings, our system has a combined space of about a terabyte which means our recordings last about a month. Our system records, on average, about eighteen one hour shows a day on weekdays and upwards of thirty or more on weekends with a mix of everything from sports to movies to the daily kid's shows. Lots of it never gets watched. Of those recordings we want to keep longer term, the best solution has been to transcode and move them into our video section that exists on a separate harddrive (An LVM of all the free space left over from the RAID actually). That harddrive is then RSYNC'd each night from the main server (A) to extra space on the slave backend (C).

Information.png Conclusion - Larger = Longer lasting. RAID is your friend.

Keeping HD In Sync When Transcoding

One of the benefits of MythTV is that once the shows are recorded, they can be manipulated in many different ways, for example, the commercials stripped out and the rest dumped onto a DVD or transcoded with an open source codec like Xvid. Transcoding HD recordings from either over-the-air or firewire captures presents significant difficulties, mostly related to keeping the audio in sync with the video.

I have found, through experience, that an MPEG-2 TS file fresh from being recorded over firewire from a DCT-62xx tuner, and edited with something like Avidemux2, will result in an audio sync that will get worse as the recording plays. By the end of the recording, what started as a 250ms sync, is now out by a couple of seconds.

There are two problems related to keeping the audio and video in sync, the first are errors in the stream, which over time create small differences. Unlike VLC which constantly adjusts the sync based on time codes, editors like Avidemux do not. The other problem is commercials, which not only interrupt the MPEG-2 stream, but do other nasty things like change framerates and drop to lower resolutions.

Looking over hundreds of forums, the consensus has been to pre-process the stream with ProjectX, an open source Java utility, which, for the longest time, worked. I would edit out the commercials with ProjectX's slow interface, demux the final cut, load it into Avidemux, load the filters and transcode. I like Avidemux because it is open source, has a slick interface and rips to a wide array of formats with lots of filters, like cropping, deinterlacing, framerate and resizing. Moreover, it's greatest feature is that it runs on both Windows and Linux which means the many hour transcodes can occur, as a background process, on our 24/7 Linux boxes. Unfortunately, Avidemux happens to be terrible at decoding raw MPEG-2 streams and isn't the most stable of programs.

However, ProjectX has recently not been able to deocode the MPEG-2 streams nor can repair utilities like mpeg2repair fix them. I was stuck.

The great part is that another MythTV user created a script to rip HD, that, for some reason, keeps the audio in sync. Unfortunately, the script only works on raw MPEG-2 streams which means it doesn't run on the transcoded commercial free recordings that MythTV can create. Besides doing the commercial detection automatically, MythTV has a fast interface for fixing the cut points and it is suppose to be able to do a straight lossless MPEG-2 to MPEG-2 copy. Unfortunately, because of a long outstanding bug with mythtranscode, it doesn't work.

If mythtranscodes's lossless copy did work, the whole process could be automated with a few clicks of the remote:

  1. Manually adjust any cut points (if necessary) from what MythTV has detected
  2. Tell MythTV to do a lossless MPEG-2 to MPEG-2 transcode which removes the commercials
  3. Tell MythTV to run the user HD script to make an Xvid archival copy, dumping the result in a certain spot to be renamed and moved into an appropriate directory later.

Without step 2, you have to run the HD script on the raw recording with the commercials included, load the result into Avidemux and manually find and cut the commercials. There is still an audio video sync problem, but I have found that whatever mencoder does, the audio delay is a straight 250ms delay, consistent from recording to recording and constant across the entire recording. The other good news is that if you don't have to crop anything, the Xvid to Xvid copy is fast, taking less then a minute for a two hour long show.

Information.png Conclusion - if you want to transcode an HD recording that you cannot edit with ProjectX use the following process:

  1. Process the raw HD MPEG-2 TS recording with the HD script
    • I had to modify the script to accept larger files and create a smaller end result (640x360). I can post the modified script here if anyone is interested.
    • This transcode takes a long time, upwards of six to eight hours on a Pentium 3Ghz machine.
  2. Load the resulting file into Avidemux, leave the audio and video as "copy", cut the commercials and set the audio delay to 250ms
  3. Save it and hit "yes" to smart copy. Sixty seconds later you have a perfectly transcoded and in sync result.

Other Random Tips

  • FreePBX includes PHPMyadmin which is useful for optimizing the database(s) which saves space and makes them run faster. PHPMyadmin also comes in handy for editing the database directly, like getting rid of commercial detection jobs if mythcommflag hangs, or reordering the tuners.
  • Concerning Asterisk, I tried ADSI, the ability to program phones with custom scripts, but some phones just didn't work because they are "locked" by their manufacturer. The comedian mail app for ADSI works, but it's clunky and slow and it was much simpler just to autologin and play the messages.
  • Monitoring of the Myth systems work great using Gkrellm and I highly recommend it. Works on Linux and Windows and looks great with a large variety of skins.

Personal Setup Notes

Notes for my own setup:

Distinctive Ringing

Setting up distinctive ringing for ring groups.
1. Via A@H or FreePBX, create two ZAP extensions for each ZAP channel,

a. The lower number one, i.e. ext 201 uses a "dial" command "ZAP/1r2", where 1 is the ZAP daughter card in slot 1 and "r2" stands for ring type 2 (or r3 or r4). This extension will ring the phone using the distinctive single/long ring.
b. The second ZAP extension, i.e 211, uses just "ZAP/1" as the dial command and will ring the same ZAP phone with a regular US ring tone.

2. Then set it up such that internal callers go to ring group 200 which rings all the distinctive ZAP extensions, i.e. x201, etc. When external callers call, their separate ring group calls x211, etc. with regular ring commands.

The great thing is that the modifications are done through A@H and don't need custom modification of any script.

  • To avoid entering a password every time for voice mail, add the voice mail's extension "sXXX" into extensions.conf -> app-messagecenter instead of "default", i.e. s402

Caller ID

Code for sending caller ID from Asterisk to MythTV Frontends. Requires XYAC. Use the startup script below to start up the XYAC daemon at login.

exten => s,1,System(/bin/echo -n -e "@CALL${ARG1} ~${ARG2}" | nc -w 1 127.0.0.1 10629 &)

Note:

  • The "&" at the end runs the "nc" daemon in the background where it can fail silently and not cause delays in answering the calls. Without the "&", the far side can intermittently time-out just as the phones start ringing.

Idle Three Hour Sleepd Daemon

  • Below is a method of using the sleepd daemon to shut off a MythTV frontend after three hours and to pop up a 30 second warning that the user can cancel.
# This is called script is called from xfce's autostart manager under "Settings" -> "Sessions and Startup"
#!/bin/sh

# Start up the sleep daemon
# -U waits for 10800 seconds
# -i watches IRQ 4, the remote to reset the counter
# -a Also watches the mouse and keyboard
# -s then runs the standby.sh script
/usr/sbin/sleepd -U 10800 -i 4 -a -s /home/mythtv/standby.sh

Premptable Shutdown Script

  • Here is the standby.sh script called by the sleepd daemon for shutdown:

Note: shutdown requires root access which means using sudo

#!/bin/sh

# Play the sounds
#cd /home/mythtv
#mplayer -softvol -softvol-max 20 cut_power.mp3 &

(
# First wait 30 seconds
echo "10" ; sleep 2
echo "# Oh, it's nothing, whew!  That was close! On with the shutdown" ; sleep 2
echo "20" ; sleep 2
echo "# Ok, on with the shutdown - Resetting the bubble blaster!!" ; sleep 2
echo "30" ; sleep 2
echo "# Bubble blaster reset, now we discharge the Nancy tubes!!" ; sleep 2
echo "40" ; sleep 2
echo "# Tubes discharge, don't forget to put out the teddy sparker!!" ; sleep 2
echo "50" ; sleep 2
echo "# Sparker stuffed! Now we can cancel the impending dog runs!!" ; sleep 2
echo "75" ; sleep 2
echo "# Dog runs cancelled - Now finally ready to stop all run-away princesses" ; sleep 3
echo "# Shutting dddoooowwww [NO CARRIER]" ; sleep 2
echo "100" ; sleep 2
# First kill the sleep daemon 
sleepctl off
# Hibernate
sudo pm-hibernate
# Upon resume, restart the sleep daemon
sleepctl on
) |
# Zenity is the program used for message boxes by Gnome/xfce
zenity --progress \
--title="Shutting Down MythTV FrontEnd Four" \
--text="Wait, there is something wrong!!!" \
--percentage=0 --width=400  --auto-close

The Plus and Minus of MythTV

  • Note this was a comment added to an article comparing TIVO 3 and MythTV.

I've been a MythTV user for a couple years now and there are a number of things the article missed or glossed over ("+" means a feature and "-" a problem).

(+) Plugins - MythTV has lots of plugins that let you do everything from surf the web, view local weather information, play old style ROM arcade games, watch recordings over the LAN on a windows machine, view a slide show of pictures, to watching XviD encoded videos, all of which we use. The video collection has been of great utility with kids. The usefulness of the plugins was a big reason I chose MythTV and have stayed with it. (Actually TIVO was never considered because of the subscription costs)

(-) Setup and Maintenance - It should not be underestimated how much time it can take getting MythTV to work and then maintaining it. As a MythTV user you will have to know the Linux command line, how to download and install drivers and fix things when they inevitably don't work. If you haven't installed a PCI card before, and don't like googling stuff to figure out why it doesn't work, then MythTV is not for you. MythTV will probably not save money over a commercial system either because I've found that the money I would have spent on a subscription went into hardware upgrades.

(+) The flip side to the cost and time commitment is that MythTV has immense flexibility, for example, in the size and redundancy of storage capability, the type of peripherals and the split arrangement between backend, slave backend and frontends. Take our ever growing video collection (most ripped from DVDs, fyi), where any new videos are backed up onto a separate machine each night (rsync'd). Try doing that with a TIVO.

(+) The machines are also full blown general purpose computers that can do other things. For example, once you have sunk the money into the MythTV hardware, you can use the same machines for VoIP (via Asterisk) which really saves us money, about $100CND a month. The pay back period for the extra hardware to do VoIP was less then a year, and gets cheaper as time goes on. We use dedicated Digium TDM-400 cards that cost ~$500CND about a year ago. Again, our long distance costs are pennies per minute, each phone is a separate extension, we can make multiple outgoing or incoming calls, late-night callers are prompted by a menu first before the phones ring, we can control our own caller-ID, etc.

(-) DRM can still be a problem with MythTV, for example, some of our HD channels, which we get from the tuner over a firewire cable, have the "5C" flag set. The end result is that those channels and some HD broadcasts are not transmitted over the firewire and MythTV records nothing. For HD tuners using firewire and 5C flagged channels, there is no current work around (as of Feb. 21st, 2007). But I'm sure that will change.

Outstanding Issues

None