Difference between revisions of "User:Mrsdonovan"

From MythTV Official Wiki
Jump to: navigation, search
(A.b - MythTV HD Master Backend / Frontend: New motherboard, CPU and Memory with an SSD)
(Notes: Updated network info)
(30 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
I've been using MythTV since 0.18 and here are my paltry [http://www.mythtv.org/wiki/Special:Contributions/Mrsdonovan my contributions].
 
I've been using MythTV since 0.18 and here are my paltry [http://www.mythtv.org/wiki/Special:Contributions/Mrsdonovan my contributions].
  
The setup below is used daily by the grandparents and has a high [http://en.wikipedia.org/wiki/Woman_acceptance_factor WAF factor].  The external DVD players rarely get used anymore.
+
The setup below is used <del>daily</del> weekly by the grandparents and has a low [http://en.wikipedia.org/wiki/Woman_acceptance_factor WAF factor].  Sadly, the grandparents have reverted to the PVR that comes in the cable tuner using the MythTV as backup.  Once a component dies, I will be reducing the system to a single master backend with an HD capable front-end that will sleep most of the time.  Having two computer systems on 24/7 costs money for electricity, replacement HDs and UPS batteries.  I just don't watch enough TV to care anymore - it won't be a better PVR that kills MythTV, just cloud based options like NetFlix.
 
----
 
----
 
==Statistics==   
 
==Statistics==   
 
My current statistics are:<br>
 
My current statistics are:<br>
 
<blockquote>
 
<blockquote>
Number of shows: 4773<br>
+
Number of shows: 4861<br>
Number of episodes: 61615<br>
+
Number of episodes: 67238<br>
 
First recording: Friday May 6th, 2005<br>
 
First recording: Friday May 6th, 2005<br>
Last recording: Monday October 7th, 2013<br>
+
Last recording: Friday September 12th, 2014<br>
Total Running Time: 8 years 5 months 2 days 5 hrs 54 mins<br>
+
Total Running Time: 9 years 4 months 7 days 17 hrs 34 mins<br>
Total Recorded: 6 years 10 months 18 days 18 hrs 11 mins<br>
+
Total Recorded: 7 years 10 months 1 day 7 hrs 44 mins<br>
 
<hr align="center" size="5" width="30%">
 
<hr align="center" size="5" width="30%">
 
System A Recording Space: 2,289,109 MB (RAID 5)<br>
 
System A Recording Space: 2,289,109 MB (RAID 5)<br>
Line 28: Line 28:
 
This system is used at my house and consists of the following:
 
This system is used at my house and consists of the following:
  
==== '''A.a - MythTV Slave backend/ SD Frontend / Asterisk server''' ====
+
==== '''A.a - MythTV Slave backend / SD Frontend / Asterisk server / Bitcoin Slave ''' ====
 
<gallery>
 
<gallery>
 
Image:Quantum_case.jpg
 
Image:Quantum_case.jpg
 
Image:Dct2500_bigview_2.jpg
 
Image:Dct2500_bigview_2.jpg
 
Image:Dct2500_bigview_2.jpg
 
Image:Dct2500_bigview_2.jpg
Image:DCT6200.jpg
 
 
Image:Digium_DGM_TDM_card_side.jpg
 
Image:Digium_DGM_TDM_card_side.jpg
 
Image:PVR150.jpg
 
Image:PVR150.jpg
Line 39: Line 38:
 
Image:serial_card.gif
 
Image:serial_card.gif
 
Image:agere_firewire_card.jpg
 
Image:agere_firewire_card.jpg
 +
Image:Single-SC-ASIC-Butterfly-Labs.jpg
 
Image:Belkin_1500VA.jpg
 
Image:Belkin_1500VA.jpg
 
</gallery>
 
</gallery>
Line 47: Line 47:
 
* RAID 5 with four SATA harddrives, roughly about 900GB of space for recordings and 1TB for videos/music/pictures/ROMs.
 
* RAID 5 with four SATA harddrives, roughly about 900GB of space for recordings and 1TB for videos/music/pictures/ROMs.
 
* ASUS P4P800-E-Deluxe
 
* ASUS P4P800-E-Deluxe
* Haupauge PVR-150
+
* <del>Haupauge PVR-150</del> - Died
* Haupauge PVR-250 with remote
+
* Haupauge PVR-250 <del>with remote </del>
* Digium Wildcard TDM400P with two S100-M FXS expansion cards
+
* Digium Wildcard TDM400P with <s>two</s> one S100-M FXS expansion cards
 
* [http://www.freepbx.org/ Freepbx] with Asterisk 1.6
 
* [http://www.freepbx.org/ Freepbx] with Asterisk 1.6
* Two Motorola DCT2524 cable tuners controlled by two IR blasters connected to a serial card.  (Custom built lirc_serial kernel module, ledxmit_serial)
+
* <s>Two</s> One Motorola DCT2524 cable tuner controlled by IR blasters connected to a serial card.  (Custom built lirc_serial kernel module, ledxmit_serial)
* DCT-6214 (Yes, it is a PVR, I know..) tuner connected via an Agere firewire card for HD
+
* <del> DCT-6214 (Yes, it is a PVR, I know..) tuner connected via an Agere firewire card for HD </del> - Moved it to the A.b MythFrontEndHD system
 
* [http://www.mythbuntu.org/ MythBuntu 10.4] (32bit)
 
* [http://www.mythbuntu.org/ MythBuntu 10.4] (32bit)
 
* 22" LCD
 
* 22" LCD
* Belkin 1500VA UPS with Bulldog software
+
* Butterfly Labs "Single" 25GH - now junk a year later, sigh
 +
* Belkin 1500VA UPS <s>with Bulldog software</s> (includes replacing the batteries on a roughly three year cycle.)
 
<br>
 
<br>
  
==== '''A.b - MythTV HD Master Backend / Frontend''' ====
+
==== '''A.b - MythTV HD Master Backend / Frontend / Virtualbox / Simulation Slave ''' ====
 
<gallery>
 
<gallery>
 
Image:X-QPACK_apevia.jpg
 
Image:X-QPACK_apevia.jpg
Line 71: Line 72:
 
* Asus Sabertooth v1.0 AM3+ with Zotac GT430 (using VDPAU)
 
* Asus Sabertooth v1.0 AM3+ with Zotac GT430 (using VDPAU)
 
** With VDPAU, HD 1080p works flawlessly at 1920x1080.
 
** With VDPAU, HD 1080p works flawlessly at 1920x1080.
* [[Hauppauge_HD-PVR]] and uses 6200ch to change channel through onboard firewire connected to DCT-6200
+
* <s>Two</s> One [[Hauppauge_HD-PVR]] running the latest firmware (v1.7.1.30058) and uses 6200ch to change channel through onboard firewire connected to <s>two daisy chained</s> DCT-6200s (<s>as an example, using "6200ch -n 1 -p 0" or "6200ch -n 2 -p 0"</s> - proved unstable)
 
* 3TB Raid 5 for recordings and video backup rsync'd each night with master server.
 
* 3TB Raid 5 for recordings and video backup rsync'd each night with master server.
 
* <del>MythGame with SDLMame emulation and a wireless Logitech Rumblepad 2</del> - never used.
 
* <del>MythGame with SDLMame emulation and a wireless Logitech Rumblepad 2</del> - never used.
 
* [http://www.ubuntu.com/ Ubuntu 12.04 LTS] running from an 80GB Intel SSD
 
* [http://www.ubuntu.com/ Ubuntu 12.04 LTS] running from an 80GB Intel SSD
 
* Toshiba Regza 42RV530U - 42" LCD driven at max resolution of 1900x1080 via an DMI cable.  We use the TV's remote with a USB MCE receiver (using Kernel drivers - no LIRC!).
 
* Toshiba Regza 42RV530U - 42" LCD driven at max resolution of 1900x1080 via an DMI cable.  We use the TV's remote with a USB MCE receiver (using Kernel drivers - no LIRC!).
* <s>Belkin 1200VA UPS with Bulldog software</s> - Batteries died and not worth replacing.
+
* Belkin 1200VA UPS <s>with Bulldog software - Batteries died and not worth replacing, Bulldog doesn't work on 10.4</s> - Replaced batteries for about $100.
 
<br>
 
<br>
  
Line 94: Line 95:
 
</strike>
 
</strike>
  
==== '''A.d - MythTV SD Frontend''' ====
+
==== <del> '''A.d - MythTV SD Frontend''' </del> (System not currently in use) ====
 
<gallery>
 
<gallery>
 
Image:Asus_Pundit.jpg
 
Image:Asus_Pundit.jpg
Line 100: Line 101:
 
Image:MCE-Remote-2-alt.jpg  
 
Image:MCE-Remote-2-alt.jpg  
 
</gallery>
 
</gallery>
 +
<strike>
 
* 2.4Ghz Celeron (Not for HD)
 
* 2.4Ghz Celeron (Not for HD)
 
* 80GB HD
 
* 80GB HD
Line 106: Line 108:
 
* [http://www.mythbuntu.org/ MythBuntu 10.4]
 
* [http://www.mythbuntu.org/ MythBuntu 10.4]
 
* Sony Trinitron 32" Flat Screen CRT
 
* Sony Trinitron 32" Flat Screen CRT
* Goes into hibernation after 3 hours using sleepd <s>(svideo doesn't recover correctly on resume!?)</s>
+
* Goes into hibernation after 3 hours using sleepd (svideo doesn't recover correctly on resume!?)
 
* MCE Remote with programmable buttons to control the TV.
 
* MCE Remote with programmable buttons to control the TV.
 
* This system connects through a hard-to-find Dlink DAP-1555 wireless bridge.
 
* This system connects through a hard-to-find Dlink DAP-1555 wireless bridge.
 
<br>
 
<br>
 +
</strike>
  
 
==== '''A.e - MythTV HD Frontend (Virtual)''' ====
 
==== '''A.e - MythTV HD Frontend (Virtual)''' ====
 
[[Image:screenshot_of_MythTV_running_in_virtual_machine.jpg|none|thumb|650px|Screen shot of a six monitor setup, one running virtual MythBuntu]]
 
[[Image:screenshot_of_MythTV_running_in_virtual_machine.jpg|none|thumb|650px|Screen shot of a six monitor setup, one running virtual MythBuntu]]
* A virtual dual core VMware Workstation guest running on a host with an [http://products.amd.com/en-us/DesktopCPUDetail.aspx?id=809 eight core 4Ghz AMD FX-8350] on a [http://www.asus.com/Motherboards/SABERTOOTH_990FX_R20/ ASUS Sabertooth 990FX R2] with 32GB RAM (DDR3 1333Ghz) running stock Windows 7 off one 80GB SSD and the VMware guest off a Samsung 840 240GB SSD.
+
* A virtual dual core VMware Workstation guest running on a host with an [http://products.amd.com/en-us/DesktopCPUDetail.aspx?id=809 eight core 4Ghz AMD FX-8350] on a [http://www.asus.com/Motherboards/SABERTOOTH_990FX_R20/ ASUS Sabertooth 990FX R2] with 32GB RAM (DDR3 1333Ghz) running stock Windows 7 off a <s>80GB</s> 256GB SSD and the VMware guest off a Samsung 840 <s>240GB</s> 500GB SSD.
 +
* Belkin 1500VA UPS <s>with Bulldog software</s> with Windows Software for shutdown (includes replacing the batteries on a roughly three year cycle.)
 
* virtual 9GB HD
 
* virtual 9GB HD
 
* virtual 500MB Ram
 
* virtual 500MB Ram
Line 119: Line 123:
 
* 2560x1600 30" DELL Monitor
 
* 2560x1600 30" DELL Monitor
 
* Both SD and HD 720p/1080i work great under VMware Player 6.0 <s>Workstation! (still) stutters but it at least %90 watchable now.</s>  
 
* Both SD and HD 720p/1080i work great under VMware Player 6.0 <s>Workstation! (still) stutters but it at least %90 watchable now.</s>  
* [http://www.desktopped.com/2010/08/four-monitor-portrait-array-that-emulates-a-52-display/ Here is a picture of the setup] - Which is out of date now, there are now five portrait monitors, four on the Ergotron Triple Display, one on a separate mount and a horizontal 30" DELL on it's regular stand to the side.
+
* Here is a picture of the setup, there are five 24" 1080x1920 portrait monitors, four on the Ergotron Triple Display, one on a separate mount and a horizontal 30" DELL 2560x1600 on it's regular stand to the side.
 +
[[Image:six_monitor_setup_mrsdonovan_five_portrait.jpg|none|thumb|650px|Screen shot of a six monitor setup]]
 
<br>
 
<br>
  
Line 125: Line 130:
  
 
=== System B ===
 
=== System B ===
This system lives at my brother's house although I maintain it and it is a separate system.
+
This system lives at my brother's house although I maintain it.
  
==== '''B.a - MythTV HD Frontend''' ====
+
==== <del> '''B.a - MythTV HD Frontend''' </del>  (System Died) ====
 
<gallery>
 
<gallery>
 
Image:Silverstone_HTPC_Case_LC13BC.jpg
 
Image:Silverstone_HTPC_Case_LC13BC.jpg
Line 144: Line 149:
 
</strike>
 
</strike>
 
<br>
 
<br>
Computer died, probably going to be a kid's computer and will use Boxee for access RAID on backend server( B.B)
+
Computer died they use a Boxee to access the video RAID on backend server(B.B)
  
==== '''B.b - MythTV HD Backend / Asterisk ''' ====
+
==== '''B.b - <del>MythTV HD Backend</del> / Asterisk ''' ====
 
<gallery>
 
<gallery>
Image:Samsung_LT-P266W.jpg
 
Image:MCE-Remote-2-alt.jpg
 
Image:PVR150.jpg
 
Image:Rumblepad_2_wireless.jpg
 
 
Image:Digium_DGM_TDM_card_side.jpg
 
Image:Digium_DGM_TDM_card_side.jpg
 
Image:Belkin_F6C900.jpg
 
Image:Belkin_F6C900.jpg
 
</gallery>
 
</gallery>
 +
 
* Quad Core 2.5Ghz Intel running on Mythbuntu 10.4 LTS (overkill, but wow, responsive)
 
* Quad Core 2.5Ghz Intel running on Mythbuntu 10.4 LTS (overkill, but wow, responsive)
 
* 1.3TB of RAID 5 for videos/pictures/music/ROMs
 
* 1.3TB of RAID 5 for videos/pictures/music/ROMs
* 2.5TB LVM for recordings.
+
* <strike> 2.5TB LVM for recordings.</strike>
 
* 500GB for backups of other computers.
 
* 500GB for backups of other computers.
 
* <del>Haupauge PVR-150 connects to a cheap ass Motorola tuner and is controlled via an IR blaster controlled by the USB MCE remote receiver.</del> Family uses Netflix now and has no cable provider.
 
* <del>Haupauge PVR-150 connects to a cheap ass Motorola tuner and is controlled via an IR blaster controlled by the USB MCE remote receiver.</del> Family uses Netflix now and has no cable provider.
 
* Digium Wildcard TDM400P with <del>three</del> <s>two</s> one (damn lightning) FXS expansion cards
 
* Digium Wildcard TDM400P with <del>three</del> <s>two</s> one (damn lightning) FXS expansion cards
 
* [http://www.freepbx.org/ Freepbx] with Asterisk 1.6
 
* [http://www.freepbx.org/ Freepbx] with Asterisk 1.6
* Samsung LT-P266W 26" HDtv flat-panel LCD
+
* <strike>Samsung LT-P266W 26" HDtv flat-panel LCD</strike> Headless
 
* Belkin 900VA UPS with Belkin Bulldog software
 
* Belkin 900VA UPS with Belkin Bulldog software
 +
 +
This system is only used as an Asterisk box now.
  
 
=== Notes ===
 
=== Notes ===
  
* <del>The link between the MythTV frontends [[User:Mrsdonovan#B_-_MythTV_HD_Frontend_.2F_Asterisk |B]], [[User:Mrsdonovan#D_-_MythTV_Frontend |D]] and the master and slave backends [[User:Mrsdonovan#A_-_MythTV_Master_backend.2F_HD_Frontend_.2F_Asterisk_server |A]], [[User:Mrsdonovan#C_-_MythTV_HD_Slave_Backend_.2F_Frontend |C]] is over a 5.8Ghz 802.11a wireless bridge using dual-band bridges from Trendnet (TEW-510apb) and one 21dbi and one 9dbi gain antenna.  Average throughput for 1080p HD is about 13mbps and during high motion scenes, tops out at 19mbps.  With the high gain antennas, the TEW-510apbs can operate at their maximum throughput of 21Mbps and HD is smooth. On rare occasions, the link is affected by thick ice on the roof and really heavy rain.</del>  (System B is now separate)
 
 
* Given three of the systems (A.a, A.b and B.b) are on 24/7, they each have a UPS rated at 900VA or better, which lasts an average of twenty minutes before cleanly shutting down (important for the RAIDed drives).
 
* Given three of the systems (A.a, A.b and B.b) are on 24/7, they each have a UPS rated at 900VA or better, which lasts an average of twenty minutes before cleanly shutting down (important for the RAIDed drives).
 
* We currently use <del>link2voip.com</del> The Spout (.ca) as our DID and outbound provider for Asterisk.
 
* We currently use <del>link2voip.com</del> The Spout (.ca) as our DID and outbound provider for Asterisk.
* Most of our machines use gigabyte Ethernet and both systems use Linksys WRT54GL variants running the open source firmware, [http://www.dd-wrt.com/ DD-WRT] <del>[http://www.polarcloud.com/tomato Tomato]</del> (Tomato isn't updated enough and has some intermittent bugs).  There have been a lot of gateways in this house but the open source DD-WRT is by far the best.  It handles QoS properly for VoIP and has all the bells and whistles every other gateway lacks.  We also use OpenDNS for all the DNS queries.  
+
* Most of our machines use gigabyte Ethernet and the System A (A.a and A.b) route through a <s>$40 [http://www.amazon.com/BUFFALO-AirStation-HighPower-Wireless-WHR-300HP/dp/B0096239EC Buffalo WHR-300HP]</s> (proved buggy) $80 [http://www.asus.com/ca-en/Networking/RTN16/ Asus RT-N16] running the open source firmware, [http://www.dd-wrt.com/ DD-WRT] <del>[http://www.polarcloud.com/tomato Tomato]</del> (Tomato isn't updated enough and has some intermittent bugs).  There have been a lot of gateways in this house but the open source DD-WRT is by far the best.  It handles QoS properly for VoIP and has all the bells and whistles every other gateway lacks and a consistent interface across multiple devices.  We also use OpenDNS for all the DNS queries.  
 
* We use most of the default plugins including <del>MythGame</del> (Who am I kidding, kids game on iPhone/Pads these days), pictures (rarely), videos, <del>weather</del> and [[User:Mrsdonovan#Caller_ID |XYAC]] for CallerID.
 
* We use most of the default plugins including <del>MythGame</del> (Who am I kidding, kids game on iPhone/Pads these days), pictures (rarely), videos, <del>weather</del> and [[User:Mrsdonovan#Caller_ID |XYAC]] for CallerID.
 
* Both Asterisk boxes are connected to each other over an IAX2 link that works well through NAT and our mutual ISPs.
 
* Both Asterisk boxes are connected to each other over an IAX2 link that works well through NAT and our mutual ISPs.
* On System A, we also run two TrendNet (TV-IP851WIC and TV-IP422W) cameras for security that record any motion detected to A.a
+
* <s>On System A, we also run two TrendNet (TV-IP851WIC and TV-IP422W) cameras for security that record any motion detected to A.a</s> - Giving up on remote cameras, they can't connect to new 802.11n routers, break and stop working much too frequently.
 +
* <del>The link between the MythTV frontends [[User:Mrsdonovan#B_-_MythTV_HD_Frontend_.2F_Asterisk |B]], [[User:Mrsdonovan#D_-_MythTV_Frontend |D]] and the master and slave backends [[User:Mrsdonovan#A_-_MythTV_Master_backend.2F_HD_Frontend_.2F_Asterisk_server |A]], [[User:Mrsdonovan#C_-_MythTV_HD_Slave_Backend_.2F_Frontend |C]] is over a 5.8Ghz 802.11a wireless bridge using dual-band bridges from Trendnet (TEW-510apb) and one 21dbi and one 9dbi gain antenna.  Average throughput for 1080p HD is about 13mbps and during high motion scenes, tops out at 19mbps.  With the high gain antennas, the TEW-510apbs can operate at their maximum throughput of 21Mbps and HD is smooth. On rare occasions, the link is affected by thick ice on the roof and really heavy rain.</del>  (System B is now separate)
  
 
== Tips from the HD Front Lines ==
 
== Tips from the HD Front Lines ==
I have been using MythTV with HD now for a couple years and wanted to share what has worked and what has not.  These comments could save you a lot of time and frustration:
+
[Updated April, 2014] I have been using MythTV with HD now for <s>a couple</s> many years and wanted to share what has worked and what has not.  These comments could save you a lot of time and frustration:
  
'''Firewire cards''' - get the best firewire cards that [[Firewire_Cablebox_Compatability |works with your tuner(s)]].  I have used NEC, VIA (onboard and PCI) cards and now finally, Agere cards.  Agere cards, the lastest chip revisions anyway, are by far the best with the Motorola DCT-62xx series boxes.  Ironically, "test-mpeg2" does not work making any priming script useless, but "broadcast" mode works pretty much flawlessly and fast!  You do still have to prime the firewire card, but I do it once at startup with the "firewire_tester -n 0 -B" script.
+
'''Firewire cards''' - <s>get the best firewire cards that [[Firewire_Cablebox_Compatability |works with your tuner(s)]].  I have used NEC, VIA (onboard and PCI) cards and now finally, Agere cards.  Agere cards, the lastest chip revisions anyway, are by far the best with the Motorola DCT-62xx series boxes.  Ironically, "test-mpeg2" does not work making any priming script useless, but "broadcast" mode works pretty much flawlessly and fast!  You do still have to prime the firewire card, but I do it once at startup with the "firewire_tester -n 0 -B" script.
  
VIA firewire chipsets will work, as they have for over a year with my system in P2P mode, but it sucks.  VIA chips require a priming script to run each time a channel change occurs and have two negative effects - first, because the priming is done at each channel change, channel surfing during Live TV times-out 2/3rds of the time.  Recordings seem to work %100 of the time.  Secondly, the priming script + VIA chipset + DCT-6200 crashed the DCT-6200 tuner when tuning to SD channels (seriously! Mine would hard lock and have to be unplugged).  I have no clue why because the same combination but with a DCT-6214 worked fine.  The tuner hard lock also doesn't happen on HD channels.
+
VIA firewire chipsets will work, as they have for over a year with my system in P2P mode, but it sucks.  VIA chips require a priming script to run each time a channel change occurs and have two negative effects - first, because the priming is done at each channel change, channel surfing during Live TV times-out 2/3rds of the time.  Recordings seem to work %100 of the time.  Secondly, the priming script + VIA chipset + DCT-6200 crashed the DCT-6200 tuner when tuning to SD channels (seriously! Mine would hard lock and have to be unplugged).  I have no clue why because the same combination but with a DCT-6214 worked fine.  The tuner hard lock also doesn't happen on HD channels.</s>  - Just use an onboard firewire for changing channels and get a:
 +
 
 +
'''Hauppage HD PVR''' - Gamers use this box to record their gaming without loading their computer down and you can find older versions for half price ($80CND) or better in second hand markets.  HD PVR boxes take in the analog signal (blue, red and green cables) and convert it to a USB which is recognized by the HDPVR driver in Linux.  Works great with MythTV and both our HD tuners now use an HD-PVR.  It is also immune to the "5C" encoding that Cable TV providers randomly flip on.
 
<blockquote>
 
<blockquote>
[[Image:Information.png|20px]] ''Conclusion'' - With DCT-62xx tuners, get Agere-5t FW323 v61 firewire chips, dump the priming script and use Broadcast at 200mbps.
+
[[Image:Information.png|20px]] ''Conclusion'' - With DCT-62xx tuners, use firewire to change channels with the 6200ch script and an Hauppauge HDPVR to capture the HD video.
 
</blockquote>
 
</blockquote>
  
'''Video cards''' - As XvMC support is going to be discontinued in MythTV 0.25 and replaced with VPDAU which supports a much large set of codecs.  NVIDIA has perhaps the best support and look for a card that uses "PureVideo HD" and perhaps the cheapest is the GeForce 210 which works great.
+
'''Video cards''' - As XvMC support is going to be discontinued in MythTV 0.25 and replaced with VPDAU which supports a much large set of codecs.  NVIDIA has perhaps the best support and look for a card that uses "PureVideo HD".
 
<blockquote>
 
<blockquote>
[[Image:Information.png|20px]] ''Conclusion'' - Get a Geforce 210 based video card and HD will work flawlessly with the OSD using VDPAU.
+
[[Image:Information.png|20px]] ''Conclusion'' - Get a Geforce 430 based video card and HD 1080p will work flawlessly with the OSD using VDPAU.
 
</blockquote>
 
</blockquote>
  
'''Processor''' - Get the largest and fastest one for HD because it makes a difference.  Everything is magnified by two with HD content, which means it takes more processor power to do playback, commercial detection and transcoding.  A faster processor also helps with database queries.  (As an aside, I also suggest looking into [[User:Yeffetn#Making_sure_mythconverg_.28mysql_db.29_is_in_memory_all_the_time | keeping the database in memory]])
+
'''Processor''' - Get the largest and fastest one for HD because it makes a difference.  Everything is magnified by two with HD content, which means it takes more processor power to do playback, commercial detection and transcoding.  A faster SSD <s>processor</s> also helps with database queries.  (As an aside, I also suggest looking into [[User:Yeffetn#Making_sure_mythconverg_.28mysql_db.29_is_in_memory_all_the_time | keeping the database in memory]])
 
<blockquote>
 
<blockquote>
 
[[Image:Information.png|20px]] ''Conclusion'' - Faster = better.
 
[[Image:Information.png|20px]] ''Conclusion'' - Faster = better.
Line 199: Line 205:
 
[[Image:Information.png|20px]] ''Conclusion'' - Get the Microsoft MCE remote and train it to control your TV as well as your MythTV box.  One remote to control them all.
 
[[Image:Information.png|20px]] ''Conclusion'' - Get the Microsoft MCE remote and train it to control your TV as well as your MythTV box.  One remote to control them all.
 
</blockquote>
 
</blockquote>
'''Hard drive Size''' - On the main backend ([[User:Mrsdonovan#A_-_MythTV_Master_backend.2F_HD_Frontend_.2F_Asterisk_server |A]]) I started out with an LVM which tied a bunch of disparately sized disks together, but I wanted more redundancy and speed.  I tried RAID 5 but ... [UPDATE] I have switched back to RAID 5 and it seems to work fine - best use the latest kernel to get all the latest updates. ([[User_talk:Mrsdonovan |Others have reported success with RAID 5]])  Having said that, the slave backend ([[User:Mrsdonovan#C_-_MythTV_HD_Slave_Backend_.2F_Frontend |C]]) [UPDATE] now has RAID 1 redundancy and records HD to two SATA II drives which seems plenty fast enough.
+
'''Hard drive Size''' - On the main backend ([[User:Mrsdonovan#A_-_MythTV_Master_backend.2F_HD_Frontend_.2F_Asterisk_server |A]]) I started out with an LVM which tied a bunch of disparately sized disks together, but I wanted more redundancy and speed.  I have switched to RAID 5 using the mdadm software and it seems to work fine.  
  
Hard drive size really only translates into one thing - longer time to watch something before it expires.  For just recordings, our system has a combined space of about a terabyte which means our recordings last about a month.  Our system records, on average, about eighteen one hour shows a day on weekdays and upwards of thirty or more on weekends with a mix of everything from sports to movies to the daily kid's shows.  Lots of it never gets watched. Of those recordings we want to keep longer term, the best solution has been to transcode and move them into our video section that exists on a separate harddrive (An LVM of all the free space left over from the RAID actually).  That harddrive is then RSYNC'd each night from the main server ([[User:Mrsdonovan#A_-_MythTV_Master_backend.2F_HD_Frontend_.2F_Asterisk_server |A]]) to extra space on the slave backend ([[User:Mrsdonovan#C_-_MythTV_HD_Slave_Backend_.2F_Frontend |C]]).
+
Hard drive size really only translates into one thing - longer time to watch something before it expires.  For recordings, our system has a combined space of about a terabyte which means our recordings last about a month.  Our system records, on average, about eighteen one hour shows a day on weekdays and upwards of thirty or more on weekends with a mix of everything from sports to movies to the daily kid's shows.  Lots of it never gets watched. Of those recordings we want to keep longer term, the best solution has been to transcode and move them into our video section that exists on a separate harddrive (An LVM of all the free space left over from the RAID actually).  That harddrive is then RSYNC'd each night from the main server ([[User:Mrsdonovan#A_-_MythTV_Master_backend.2F_HD_Frontend_.2F_Asterisk_server |A]]) to extra space on the slave backend ([[User:Mrsdonovan#C_-_MythTV_HD_Slave_Backend_.2F_Frontend |C]]).
 
<blockquote>
 
<blockquote>
 
[[Image:Information.png|20px]] ''Conclusion'' - Larger = Longer lasting.  RAID is your friend.
 
[[Image:Information.png|20px]] ''Conclusion'' - Larger = Longer lasting.  RAID is your friend.
 
</blockquote>
 
</blockquote>
  
==== Keeping HD In Sync When Transcoding ====
 
One of the benefits of MythTV is that once the shows are recorded, they can be manipulated in many different ways, for example, the commercials stripped out and the rest dumped onto a DVD or transcoded with an open source codec like Xvid.  Transcoding HD recordings from either over-the-air or firewire captures presents significant difficulties, mostly related to keeping the audio in sync with the video.
 
  
I have found, through experience, that an MPEG-2 TS file fresh from being recorded over firewire from a DCT-62xx tuner, and edited with something like [http://avidemux.org/ Avidemux2], will result in an audio sync that will get worse as the recording playsBy the end of the recording, what started as a 250ms sync, is now out by a couple of seconds.
+
==== Other Random Tips ====
 +
* FreePBX includes PHPMyadmin which is useful for optimizing the database(s) which saves space and makes them run faster.  PHPMyadmin also comes in handy for editing the database directly, like getting rid of commercial detection jobs if mythcommflag hangs, or reordering the tuners.
 +
* Concerning Asterisk, I tried ADSI, the ability to program phones with custom scripts, but some phones just didn't work because they are "locked" by their manufacturer.  The comedian mail app for ADSI works, but it's clunky and slow and it was much simpler just to autologin and play the messages.
 +
* Monitoring of the Myth systems work great using [http://en.wikipedia.org/wiki/GKrellM Gkrellm] and I highly recommend it. Works on Linux and Windows and looks great with a large variety of skins. [Update November 2013 - This is still true ten years later which is rare especially since the windows client hasn't been updated in over five years but it works on Windows 7, the last Windows OS I plan to buy.]
 +
* Get ECC RAM - Your MythTV backend is on all the time, and it is exactly these conditions that the ECC ram (*unbuffered*) was created forAMD desktop processors and related motherboards support ECC out of the box unlike Intel which require server grade (expensive) CPUs.
 +
* If you run mdadm software RAID, a Uninterruptable Power Supply is good at keeping the array "clean" for those unexpected power outages.  Use a "True Sine Wave" UPS if you can afford it and your computer power supplies will appreciate it.
 +
 
 +
== Personal Setup Notes ==
 +
Notes for my own setup:
  
There are two problems related to keeping the audio and video in sync, the first are errors in the stream, which over time create small differences. Unlike VLC which constantly adjusts the sync based on time codes, editors like Avidemux do not. The other problem is commercials, which not only interrupt the MPEG-2 stream, but do other nasty things like change framerates and drop to lower resolutions.
+
==== Hauppauge HD PVR Audio Sync ====
 +
The 1445 model of the HD PVR has an audio sync problem (with 0x15 firmware version and up, [http://code.mythtv.org/trac/ticket/10007 bug documented here]) and the fix is to add:
 +
<pre>export FORCE_DTS_TIMESTAMPS=1</pre>
 +
to /etc/mythtv/session-settings ([http://ubuntuforums.org/archive/index.php/t-2054422.html thanks!]) on each Myth Frontend.
 +
 
 +
Updating the HD-PVR firmware likely works too if you have a newer kernel (3.3 or better)
 +
 
 +
==== VNC server Xstartup ====
 +
 
 +
Getting a headless vnc server to run correctly took a bit of work and most of it came down to the /home/<user>/.vnc/xstartup which in my case was:
 +
 
 +
<pre>
 +
#!/bin/sh
  
Looking over hundreds of forums, the consensus has been to pre-process the stream with ProjectX, an open source Java utility, which, for the longest time, worked.  I would edit out the commercials with ProjectX's slow interface, demux the final cut, load it into Avidemux, load the filters and transcode.  I like [http://avidemux.org/ Avidemux] because it is open source, has a slick interface and rips to a wide array of formats with lots of filters, like cropping, deinterlacing, framerate and resizing.  Moreover, it's greatest feature is that it runs on both Windows and Linux which means the many hour transcodes can occur, as a background process, on our 24/7 Linux boxes. Unfortunately, Avidemux happens to be terrible at decoding raw MPEG-2 streams and isn't the most stable of programs.
+
# I don't think this line is used given it is for Windows Remote desktop software, XRBD which is not installed.
 +
xrbd $HOME/.Xresources
 +
xsetroot -solid grey
  
However, ProjectX has recently not been able to deocode the MPEG-2 streams nor can repair utilities like mpeg2repair fix them.  I was stuck.
+
# Added this for some reason, now forgotten
 +
export XKL_XMODMAP_DISABLE=1
  
The great part is that [[User:Steveadeff#HR.HDTV_script |another MythTV user created a script to rip HD]], that, for some reason, keeps the audio in sync.  Unfortunately, the script only works on raw MPEG-2 streams which means it doesn't run on the transcoded commercial free recordings that MythTV can create.  Besides doing the commercial detection automatically, MythTV has a fast interface for fixing the cut points and it is suppose to be able to do a straight lossless MPEG-2 to MPEG-2 copy.  Unfortunately, because [http://svn.mythtv.org/trac/ticket/2077 of a long outstanding bug with mythtranscode], it doesn't work. 
+
# Absolutely necessary for the dbus message bus to work when apps like update-manager,
 +
# etc. require elevated privileges (Took me three hours to figure why pressing "install" on the Ubuntu software Center did nothing!)
 +
unset SESSION_MANAGER
 +
unset DBUS_SESSION_BUS_ADDRESS
  
If mythtranscodes's lossless copy did work, the whole process could be automated with a few clicks of the remote:
+
# Kicks off the gnome 2d desktop which includes Unity
 +
# - need some packages installed for this to work, gnome-core??
 +
gnome-session --session=ubuntu-2d &
 +
</pre>
  
#Manually adjust any cut points (if necessary) from what MythTV has detected
+
To control the size, start the server with "vncserver -geometry 2500x1500" which looks good on a 2560x1600 monitor.  Need "vncserver" (vnc4?) installed - avoiding Xtightvnc for now as consistently lost connect after a while.
#Tell MythTV to do a lossless MPEG-2 to MPEG-2 transcode which removes the commercials
 
#Tell MythTV to run the user HD script to make an Xvid archival copy, dumping the result in a certain spot to be renamed and moved into an appropriate directory later.
 
  
Without step 2, you have to run the HD script on the raw recording with the commercials included, load the result into Avidemux and manually find and cut the commercials.  There is still an audio video sync problem, but I have found that whatever mencoder does, the audio delay is a straight 250ms delay, consistent from recording to recording and constant across the entire recording.  The other good news is that if you don't have to crop anything, the Xvid to Xvid copy is fast, taking less then a minute for a two hour long show.
+
==== Kernel devinput instead of LIRC ====
 +
With Ubuntu 12.04 64 bit, the drivers for the remotes are now in the kernel and LIRC, at least in my case, wasn't needed.
  
 +
Here is how I moved from LIRC to devinput:
 
<blockquote>
 
<blockquote>
[[Image:Information.png|20px]] ''Conclusion'' - if you want to transcode an HD recording that you cannot edit with ProjectX use the following process:
+
1. apt-get install [http://manpages.ubuntu.com/manpages/natty/man1/ir-keytable.1.html ir-keytable], then run ir-keytable which returns a description of the receiver capabilities, i.e:
 +
<pre>
 +
Found /sys/class/rc/rc0 (/dev/input/event4) with:
 +
  Driver mcesub, table rc-rc6-mce
 +
  Supported protocols: NEC RC-5 RC-6 JVC SONY LIRC other
 +
  Enabled protocols: NEC
 +
  Repeat delay = 500ms, repeat_period = 125ms
 +
</pre>
 +
2. using 'ir-keytable -t' (replaces the old "irw") I found that my remote was being recognized by the MCE USB receiver, because for each keypress, it produced something like (pressing the Enter key for example):
 +
<pre><some long time stamp>: event MSC: scancode = 45bc44</pre>
 +
3. A grep of all the remotes for that scancode in /lib/udev/rc_keymaps returned no results which meant my remote had not been recorded.
 +
 
 +
4. I created a new keymap file under /etc/rc_keymaps with the following necessary format:
 +
<pre>
 +
==> NEC <==
 +
# Comments go here
 +
0x45bc44 KEY_ENTER
 +
...
 +
</pre>
 +
 
 +
5. The location of the file isn't important as you reference it directly, but the format of it is, i.e. having the NEC or RC6, etc.
 +
 
 +
6. Then configured it in rc.local with:
 +
<pre>ir-keytable -c --period=500 -p NEC -w /etc/rc_keymaps/<name of your remote></pre>
 +
where
 +
<pre>
 +
"-c" clears the previous keytable,
 +
"--period" (or "-P") sets the repeat period to slow down repeats
 +
"-p" enables just the NEC protocol (or RC-6, SONY, etc) and
 +
"-w" adds the keys from the keytable file.
 +
</pre>
 +
7. Pressing the key on the remote then gets passed to MythTV directly as if the remote was like a keyboard, sweet!  No .lirc/mythtv configuration or LIRCD daemon to crash, nor worrying about which /dev/input/eventxxx the receiver is plugged into.
 +
* To stop the remote repeating too quickly, I ran the "Keyboard" app under Unity to change the "Delay" from short to long and the "Speed" to slow.
 +
 
 +
 
 +
8. All this was figured out after a couple hours of googling and trial and error.  Lots of people still use the kernel drivers to send the key inputs to an LIRC daemon, but I couldn't get that to work and the kernel drivers weren't hard to setup.
 
</blockquote>
 
</blockquote>
#Process the raw HD MPEG-2 TS recording with the [[User:Steveadeff#HR.HDTV_script |HD script]]
 
#*I had to modify the script to accept larger files and create a smaller end result (640x360).  I can post the modified script here if anyone is interested.
 
#*This transcode takes a long time, upwards of six to eight hours on a Pentium 3Ghz machine.
 
#Load the resulting file into Avidemux, leave the audio and video as "copy", cut the commercials and set the audio delay to 250ms
 
#Save it and hit "yes" to smart copy.  Sixty seconds later you have a perfectly transcoded and in sync result.
 
  
==== Other Random Tips ====
 
* FreePBX includes PHPMyadmin which is useful for optimizing the database(s) which saves space and makes them run faster.  PHPMyadmin also comes in handy for editing the database directly, like getting rid of commercial detection jobs if mythcommflag hangs, or reordering the tuners.
 
* Concerning Asterisk, I tried ADSI, the ability to program phones with custom scripts, but some phones just didn't work because they are "locked" by their manufacturer.  The comedian mail app for ADSI works, but it's clunky and slow and it was much simpler just to autologin and play the messages.
 
* Monitoring of the Myth systems work great using [http://en.wikipedia.org/wiki/GKrellM Gkrellm] and I highly recommend it. Works on Linux and Windows and looks great with a large variety of skins.
 
 
== Personal Setup Notes ==
 
Notes for my own setup:
 
 
==== Distinctive Ringing ====
 
==== Distinctive Ringing ====
 
<blockquote>
 
<blockquote>
Line 256: Line 312:
  
 
</blockquote>
 
</blockquote>
*To avoid entering a password every time for voice mail, add the voice mail's extension "sXXX" into extensions.conf -> app-messagecenter instead of "default", i.e. s402
+
*Voicemail is emailed to us now and deleted.  <del>To avoid entering a password every time for voice mail, add the voice mail's extension "sXXX" into extensions.conf -> app-messagecenter instead of "default", i.e. s402</del>
 +
 
 
==== Caller ID ====
 
==== Caller ID ====
Code for sending caller ID from Asterisk to MythTV Frontends.  Requires [http://www.voip-info.org/wiki/view/Asterisk+tips+MythTV+integration XYAC].  Use the [[User:Mrsdonovan#Idle_Three_Hour_Sleepd_Daemon |startup script below]] to start up the XYAC daemon at login.   
+
The code for sending caller ID from Asterisk to MythTV Frontends.  Requires [http://www.voip-info.org/wiki/view/Asterisk+tips+MythTV+integration XYAC].  Use the [[User:Mrsdonovan#Idle_Three_Hour_Sleepd_Daemon |startup script below]] to start up the XYAC daemon at login.   
 
<br>
 
<br>
 
<blockquote>
 
<blockquote>
Line 267: Line 324:
 
Note:
 
Note:
 
* The "&" at the end runs the "nc" daemon in the background where it can fail silently and not cause delays in answering the calls.  Without the "&", the far side can intermittently time-out just as the phones start ringing.
 
* The "&" at the end runs the "nc" daemon in the background where it can fail silently and not cause delays in answering the calls.  Without the "&", the far side can intermittently time-out just as the phones start ringing.
 +
* This code lives in /etc/asterisk/extensions_custom.conf file
 +
* The Makefile for XYAC needs to have two changes to compile under Ubuntu 12.04:
 +
<pre>
 +
INCLUDES=-I/usr/include
 +
</pre>
 +
and
 +
<pre>
 +
xyac: xyac.o xosd.o
 +
  $(CC) -o xyac $(LIBS) xyac.o xosd.o $(LFLAGS)
 +
</pre>
 +
(the gcc compiler now uses the --as-needed flag instead of --no-as-needed which means it has to have the library after the compiling module, otherwise the linker doesn't think any of the functions in the library are needed)
 +
* The line to startup XYAC and increase the font size is:
 +
<pre> xyac -c yellow --align=center -f -*-*-*-*-*-*-72-*-*-*-*-*-*-* --shadow=4 --delay=45 </pre>
 +
The line after the "-f" is the font specifier in [https://wiki.archlinux.org/index.php/X_Logical_Font_Description XLFD format] and depends on what fonts are installed, but the generic one shown uses the first system font with a size of 72, which on a 1080p screen looks big.  A good way to find what fonts you have installed is to use "xfontsel" and a common system font is "Bitstream", which you would specify like this <pre>-bitstream-*-*-*-*-*-72-*-*-*-*-*-*-*</pre>
  
 
==== Idle Three Hour Sleepd Daemon ====
 
==== Idle Three Hour Sleepd Daemon ====
Line 393: Line 464:
 
* <del>Not directly related to MythTV but, every couple of months, mdadm kicks all the HDs out of the RAID 5 array on System B.  Thankfully, a reboot brings the array right back up, but very strange.</del>  This was a hardware issue, which after the MOBO upgrade, is fixed.
 
* <del>Not directly related to MythTV but, every couple of months, mdadm kicks all the HDs out of the RAID 5 array on System B.  Thankfully, a reboot brings the array right back up, but very strange.</del>  This was a hardware issue, which after the MOBO upgrade, is fixed.
 
* <del>Every once in a while, the front end will crash, and it seems that a reset of all the MythTV processes is required.  I have created a script, which when a "power" button is pressed on the remote, resets the local master backend and frontend and through a SSH tunnel it opens on the fly, resets the remote MythTV backend process (that took a bit of debugging to get a remote user to reset a different user's process) </del> - The latest 0.25 version of MythTV now has a frontend restarter built in and my custom script was created multiple backend processes which completely messed things up.  Stability is still an issue over weeks and weeks.
 
* <del>Every once in a while, the front end will crash, and it seems that a reset of all the MythTV processes is required.  I have created a script, which when a "power" button is pressed on the remote, resets the local master backend and frontend and through a SSH tunnel it opens on the fly, resets the remote MythTV backend process (that took a bit of debugging to get a remote user to reset a different user's process) </del> - The latest 0.25 version of MythTV now has a frontend restarter built in and my custom script was created multiple backend processes which completely messed things up.  Stability is still an issue over weeks and weeks.
* As of January, 2013, the HDPVR driver does not have the ID code, 4903, for my HDPVR unit which means /dev/video does not show up and fixing the problem means editing the source and recompiling.  Because recompiling a driver is a bitch, I used a hex editor and changed the driver directly (Warning: This is the WRONG way to fix this problem. Don't ever do it. Ever. However, it did work for me in 10.04. Use a hex editor to edit hdpvr.ko, search for text 4901 and replace with 4903, search for hex 0149 and replace with 0349"  There is a great thread on this problem [http://ubuntuforums.org/showthread.php?t=1590146 here].)
+
* <del>As of January, 2013, the HDPVR driver does not have the ID code, 4903, for my HDPVR unit which means /dev/video does not show up and fixing the problem means editing the source and recompiling.  Because recompiling a driver is a bitch, I used a hex editor and changed the driver directly (Warning: This is the WRONG way to fix this problem. Don't ever do it. Ever. However, it did work for me in 10.04. Use a hex editor to edit hdpvr.ko, search for text 4901 and replace with 4903, search for hex 0149 and replace with 0349"  There is a great thread on this problem [http://ubuntuforums.org/showthread.php?t=1590146 here].)</del> Ubuntu 12.04 doesn't have this problem.
 
[[Category:MythPeople]][[Category:User_Systems]]
 
[[Category:MythPeople]][[Category:User_Systems]]

Revision as of 23:47, 12 September 2014

I've been using MythTV since 0.18 and here are my paltry my contributions.

The setup below is used daily weekly by the grandparents and has a low WAF factor. Sadly, the grandparents have reverted to the PVR that comes in the cable tuner using the MythTV as backup. Once a component dies, I will be reducing the system to a single master backend with an HD capable front-end that will sleep most of the time. Having two computer systems on 24/7 costs money for electricity, replacement HDs and UPS batteries. I just don't watch enough TV to care anymore - it won't be a better PVR that kills MythTV, just cloud based options like NetFlix.


Statistics

My current statistics are:

Number of shows: 4861
Number of episodes: 67238
First recording: Friday May 6th, 2005
Last recording: Friday September 12th, 2014
Total Running Time: 9 years 4 months 7 days 17 hrs 34 mins
Total Recorded: 7 years 10 months 1 day 7 hrs 44 mins


System A Recording Space: 2,289,109 MB (RAID 5)
System B Recording Space: 2,623,048 MB (LVM)
Total Recording Space Available: 4,912,157 MB (5 TB)


System A Video space: 1,022,540 MB
System B Video space: 1,465,076 MB
Total Space for Videos Available: 2,487,616 MB (2.5 TB, All RAID 5)


Setup

System A

This system is used at my house and consists of the following:

A.a - MythTV Slave backend / SD Frontend / Asterisk server / Bitcoin Slave

  • 3Ghz HT Intel with 2GB of dual channel RAM
  • 1GB ethernet jack connected to a 10/100/1000Mbps Tredndnet 8 port router
  • HD capable Geforce 6200 video card.
    • This system can play 720p but not 1080p as it stutters horribly. The motherboard has an AGP port though and everything will have to upgraded to play 1080p.
  • RAID 5 with four SATA harddrives, roughly about 900GB of space for recordings and 1TB for videos/music/pictures/ROMs.
  • ASUS P4P800-E-Deluxe
  • Haupauge PVR-150 - Died
  • Haupauge PVR-250 with remote
  • Digium Wildcard TDM400P with two one S100-M FXS expansion cards
  • Freepbx with Asterisk 1.6
  • Two One Motorola DCT2524 cable tuner controlled by IR blasters connected to a serial card. (Custom built lirc_serial kernel module, ledxmit_serial)
  • DCT-6214 (Yes, it is a PVR, I know..) tuner connected via an Agere firewire card for HD - Moved it to the A.b MythFrontEndHD system
  • MythBuntu 10.4 (32bit)
  • 22" LCD
  • Butterfly Labs "Single" 25GH - now junk a year later, sigh
  • Belkin 1500VA UPS with Bulldog software (includes replacing the batteries on a roughly three year cycle.)


A.b - MythTV HD Master Backend / Frontend / Virtualbox / Simulation Slave

  • AMD Phenom X6 1090T with 16GB (64 bit)
  • Asus Sabertooth v1.0 AM3+ with Zotac GT430 (using VDPAU)
    • With VDPAU, HD 1080p works flawlessly at 1920x1080.
  • Two One Hauppauge_HD-PVR running the latest firmware (v1.7.1.30058) and uses 6200ch to change channel through onboard firewire connected to two daisy chained DCT-6200s (as an example, using "6200ch -n 1 -p 0" or "6200ch -n 2 -p 0" - proved unstable)
  • 3TB Raid 5 for recordings and video backup rsync'd each night with master server.
  • MythGame with SDLMame emulation and a wireless Logitech Rumblepad 2 - never used.
  • Ubuntu 12.04 LTS running from an 80GB Intel SSD
  • Toshiba Regza 42RV530U - 42" LCD driven at max resolution of 1900x1080 via an DMI cable. We use the TV's remote with a USB MCE receiver (using Kernel drivers - no LIRC!).
  • Belkin 1200VA UPS with Bulldog software - Batteries died and not worth replacing, Bulldog doesn't work on 10.4 - Replaced batteries for about $100.


A.c - MythTV Frontend (System not currently in use)

  • 2.4Ghz Celeron (Not for HD)
  • 80GB HD
  • 768MB of RAM
  • Asus Pundit (The SIS version)
  • MythBuntu 10.4
  • MCE Remote with programmable buttons to control the TV.


A.d - MythTV SD Frontend (System not currently in use)

  • 2.4Ghz Celeron (Not for HD)
  • 80GB HD
  • 768MB of RAM
  • Asus Pundit (The SIS version)
  • MythBuntu 10.4
  • Sony Trinitron 32" Flat Screen CRT
  • Goes into hibernation after 3 hours using sleepd (svideo doesn't recover correctly on resume!?)
  • MCE Remote with programmable buttons to control the TV.
  • This system connects through a hard-to-find Dlink DAP-1555 wireless bridge.


A.e - MythTV HD Frontend (Virtual)

Screen shot of a six monitor setup, one running virtual MythBuntu
  • A virtual dual core VMware Workstation guest running on a host with an eight core 4Ghz AMD FX-8350 on a ASUS Sabertooth 990FX R2 with 32GB RAM (DDR3 1333Ghz) running stock Windows 7 off a 80GB 256GB SSD and the VMware guest off a Samsung 840 240GB 500GB SSD.
  • Belkin 1500VA UPS with Bulldog software with Windows Software for shutdown (includes replacing the batteries on a roughly three year cycle.)
  • virtual 9GB HD
  • virtual 500MB Ram
  • Ubuntu Desktop 12.04
  • 2560x1600 30" DELL Monitor
  • Both SD and HD 720p/1080i work great under VMware Player 6.0 Workstation! (still) stutters but it at least %90 watchable now.
  • Here is a picture of the setup, there are five 24" 1080x1920 portrait monitors, four on the Ergotron Triple Display, one on a separate mount and a horizontal 30" DELL 2560x1600 on it's regular stand to the side.
Screen shot of a six monitor setup



System B

This system lives at my brother's house although I maintain it.

B.a - MythTV HD Frontend (System Died)

  • INTEL 2.8Ghz P4
  • 1.5GB of RAM
  • NVIDIA Geforce 8500
  • Samsung LN46B540P 46" LCD
  • Mythbuntu 9.10 (32bit)
  • MythGame with SDLMame emulation and a wireless Logitech Rumblepad 2
  • Goes into hibernation after 3 hours using sleepd


Computer died they use a Boxee to access the video RAID on backend server(B.B)

B.b - MythTV HD Backend / Asterisk

  • Quad Core 2.5Ghz Intel running on Mythbuntu 10.4 LTS (overkill, but wow, responsive)
  • 1.3TB of RAID 5 for videos/pictures/music/ROMs
  • 2.5TB LVM for recordings.
  • 500GB for backups of other computers.
  • Haupauge PVR-150 connects to a cheap ass Motorola tuner and is controlled via an IR blaster controlled by the USB MCE remote receiver. Family uses Netflix now and has no cable provider.
  • Digium Wildcard TDM400P with three two one (damn lightning) FXS expansion cards
  • Freepbx with Asterisk 1.6
  • Samsung LT-P266W 26" HDtv flat-panel LCD Headless
  • Belkin 900VA UPS with Belkin Bulldog software

This system is only used as an Asterisk box now.

Notes

  • Given three of the systems (A.a, A.b and B.b) are on 24/7, they each have a UPS rated at 900VA or better, which lasts an average of twenty minutes before cleanly shutting down (important for the RAIDed drives).
  • We currently use link2voip.com The Spout (.ca) as our DID and outbound provider for Asterisk.
  • Most of our machines use gigabyte Ethernet and the System A (A.a and A.b) route through a $40 Buffalo WHR-300HP (proved buggy) $80 Asus RT-N16 running the open source firmware, DD-WRT Tomato (Tomato isn't updated enough and has some intermittent bugs). There have been a lot of gateways in this house but the open source DD-WRT is by far the best. It handles QoS properly for VoIP and has all the bells and whistles every other gateway lacks and a consistent interface across multiple devices. We also use OpenDNS for all the DNS queries.
  • We use most of the default plugins including MythGame (Who am I kidding, kids game on iPhone/Pads these days), pictures (rarely), videos, weather and XYAC for CallerID.
  • Both Asterisk boxes are connected to each other over an IAX2 link that works well through NAT and our mutual ISPs.
  • On System A, we also run two TrendNet (TV-IP851WIC and TV-IP422W) cameras for security that record any motion detected to A.a - Giving up on remote cameras, they can't connect to new 802.11n routers, break and stop working much too frequently.
  • The link between the MythTV frontends B, D and the master and slave backends A, C is over a 5.8Ghz 802.11a wireless bridge using dual-band bridges from Trendnet (TEW-510apb) and one 21dbi and one 9dbi gain antenna. Average throughput for 1080p HD is about 13mbps and during high motion scenes, tops out at 19mbps. With the high gain antennas, the TEW-510apbs can operate at their maximum throughput of 21Mbps and HD is smooth. On rare occasions, the link is affected by thick ice on the roof and really heavy rain. (System B is now separate)

Tips from the HD Front Lines

[Updated April, 2014] I have been using MythTV with HD now for a couple many years and wanted to share what has worked and what has not. These comments could save you a lot of time and frustration:

Firewire cards - get the best firewire cards that works with your tuner(s). I have used NEC, VIA (onboard and PCI) cards and now finally, Agere cards. Agere cards, the lastest chip revisions anyway, are by far the best with the Motorola DCT-62xx series boxes. Ironically, "test-mpeg2" does not work making any priming script useless, but "broadcast" mode works pretty much flawlessly and fast! You do still have to prime the firewire card, but I do it once at startup with the "firewire_tester -n 0 -B" script.

VIA firewire chipsets will work, as they have for over a year with my system in P2P mode, but it sucks. VIA chips require a priming script to run each time a channel change occurs and have two negative effects - first, because the priming is done at each channel change, channel surfing during Live TV times-out 2/3rds of the time. Recordings seem to work %100 of the time. Secondly, the priming script + VIA chipset + DCT-6200 crashed the DCT-6200 tuner when tuning to SD channels (seriously! Mine would hard lock and have to be unplugged). I have no clue why because the same combination but with a DCT-6214 worked fine. The tuner hard lock also doesn't happen on HD channels. - Just use an onboard firewire for changing channels and get a:

Hauppage HD PVR - Gamers use this box to record their gaming without loading their computer down and you can find older versions for half price ($80CND) or better in second hand markets. HD PVR boxes take in the analog signal (blue, red and green cables) and convert it to a USB which is recognized by the HDPVR driver in Linux. Works great with MythTV and both our HD tuners now use an HD-PVR. It is also immune to the "5C" encoding that Cable TV providers randomly flip on.

Information.png Conclusion - With DCT-62xx tuners, use firewire to change channels with the 6200ch script and an Hauppauge HDPVR to capture the HD video.

Video cards - As XvMC support is going to be discontinued in MythTV 0.25 and replaced with VPDAU which supports a much large set of codecs. NVIDIA has perhaps the best support and look for a card that uses "PureVideo HD".

Information.png Conclusion - Get a Geforce 430 based video card and HD 1080p will work flawlessly with the OSD using VDPAU.

Processor - Get the largest and fastest one for HD because it makes a difference. Everything is magnified by two with HD content, which means it takes more processor power to do playback, commercial detection and transcoding. A faster SSD processor also helps with database queries. (As an aside, I also suggest looking into keeping the database in memory)

Information.png Conclusion - Faster = better.

Remote Control - From a usability stand point, an excellent remote is important. I have settled on the Microsoft MCE remotes because they have lots of features that make it all come together - First, they can be trained with your existing TV remote to shut off the TV and control the volume of the TV. The volume control in MythTV frankly sucks because it only works when a recording is actually playing and doesn't go high enough (a fault of the hardware actually). Being able to turn off the TV and control it's volume directly mean you can chuck your TV's remote. If you are playing DVDs on your MythTV system, you can now have one remote to control everything and that rocks. Better, if you wreck the remote, they are inexpensive ($35 locally) and widely available. The receiver has wide reception, a quick response rate and the layout on the remote is excellent with important buttons in easy to reach places. The downside is that because it is back lit, it uses batteries quickly (every four months) but hopefully rechargeable ones in your case.

Information.png Conclusion - Get the Microsoft MCE remote and train it to control your TV as well as your MythTV box. One remote to control them all.

Hard drive Size - On the main backend (A) I started out with an LVM which tied a bunch of disparately sized disks together, but I wanted more redundancy and speed. I have switched to RAID 5 using the mdadm software and it seems to work fine.

Hard drive size really only translates into one thing - longer time to watch something before it expires. For recordings, our system has a combined space of about a terabyte which means our recordings last about a month. Our system records, on average, about eighteen one hour shows a day on weekdays and upwards of thirty or more on weekends with a mix of everything from sports to movies to the daily kid's shows. Lots of it never gets watched. Of those recordings we want to keep longer term, the best solution has been to transcode and move them into our video section that exists on a separate harddrive (An LVM of all the free space left over from the RAID actually). That harddrive is then RSYNC'd each night from the main server (A) to extra space on the slave backend (C).

Information.png Conclusion - Larger = Longer lasting. RAID is your friend.


Other Random Tips

  • FreePBX includes PHPMyadmin which is useful for optimizing the database(s) which saves space and makes them run faster. PHPMyadmin also comes in handy for editing the database directly, like getting rid of commercial detection jobs if mythcommflag hangs, or reordering the tuners.
  • Concerning Asterisk, I tried ADSI, the ability to program phones with custom scripts, but some phones just didn't work because they are "locked" by their manufacturer. The comedian mail app for ADSI works, but it's clunky and slow and it was much simpler just to autologin and play the messages.
  • Monitoring of the Myth systems work great using Gkrellm and I highly recommend it. Works on Linux and Windows and looks great with a large variety of skins. [Update November 2013 - This is still true ten years later which is rare especially since the windows client hasn't been updated in over five years but it works on Windows 7, the last Windows OS I plan to buy.]
  • Get ECC RAM - Your MythTV backend is on all the time, and it is exactly these conditions that the ECC ram (*unbuffered*) was created for. AMD desktop processors and related motherboards support ECC out of the box unlike Intel which require server grade (expensive) CPUs.
  • If you run mdadm software RAID, a Uninterruptable Power Supply is good at keeping the array "clean" for those unexpected power outages. Use a "True Sine Wave" UPS if you can afford it and your computer power supplies will appreciate it.

Personal Setup Notes

Notes for my own setup:

Hauppauge HD PVR Audio Sync

The 1445 model of the HD PVR has an audio sync problem (with 0x15 firmware version and up, bug documented here) and the fix is to add:

export FORCE_DTS_TIMESTAMPS=1

to /etc/mythtv/session-settings (thanks!) on each Myth Frontend.

Updating the HD-PVR firmware likely works too if you have a newer kernel (3.3 or better)

VNC server Xstartup

Getting a headless vnc server to run correctly took a bit of work and most of it came down to the /home/<user>/.vnc/xstartup which in my case was:

#!/bin/sh

# I don't think this line is used given it is for Windows Remote desktop software, XRBD which is not installed.
xrbd $HOME/.Xresources
xsetroot -solid grey

# Added this for some reason, now forgotten
export XKL_XMODMAP_DISABLE=1

# Absolutely necessary for the dbus message bus to work when apps like update-manager,
# etc. require elevated privileges (Took me three hours to figure why pressing "install" on the Ubuntu software Center did nothing!)
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS

# Kicks off the gnome 2d desktop which includes Unity 
# - need some packages installed for this to work, gnome-core??
gnome-session --session=ubuntu-2d &

To control the size, start the server with "vncserver -geometry 2500x1500" which looks good on a 2560x1600 monitor. Need "vncserver" (vnc4?) installed - avoiding Xtightvnc for now as consistently lost connect after a while.

Kernel devinput instead of LIRC

With Ubuntu 12.04 64 bit, the drivers for the remotes are now in the kernel and LIRC, at least in my case, wasn't needed.

Here is how I moved from LIRC to devinput:

1. apt-get install ir-keytable, then run ir-keytable which returns a description of the receiver capabilities, i.e:

Found /sys/class/rc/rc0 (/dev/input/event4) with:
  Driver mcesub, table rc-rc6-mce
  Supported protocols: NEC RC-5 RC-6 JVC SONY LIRC other
  Enabled protocols: NEC
  Repeat delay = 500ms, repeat_period = 125ms

2. using 'ir-keytable -t' (replaces the old "irw") I found that my remote was being recognized by the MCE USB receiver, because for each keypress, it produced something like (pressing the Enter key for example):

<some long time stamp>: event MSC: scancode = 45bc44

3. A grep of all the remotes for that scancode in /lib/udev/rc_keymaps returned no results which meant my remote had not been recorded.

4. I created a new keymap file under /etc/rc_keymaps with the following necessary format:

==> NEC <==
# Comments go here
0x45bc44 KEY_ENTER
...

5. The location of the file isn't important as you reference it directly, but the format of it is, i.e. having the NEC or RC6, etc.

6. Then configured it in rc.local with:

ir-keytable -c --period=500 -p NEC -w /etc/rc_keymaps/<name of your remote>

where

"-c" clears the previous keytable, 
"--period" (or "-P") sets the repeat period to slow down repeats
"-p" enables just the NEC protocol (or RC-6, SONY, etc) and 
"-w" adds the keys from the keytable file.

7. Pressing the key on the remote then gets passed to MythTV directly as if the remote was like a keyboard, sweet! No .lirc/mythtv configuration or LIRCD daemon to crash, nor worrying about which /dev/input/eventxxx the receiver is plugged into.

  • To stop the remote repeating too quickly, I ran the "Keyboard" app under Unity to change the "Delay" from short to long and the "Speed" to slow.


8. All this was figured out after a couple hours of googling and trial and error. Lots of people still use the kernel drivers to send the key inputs to an LIRC daemon, but I couldn't get that to work and the kernel drivers weren't hard to setup.

Distinctive Ringing

Setting up distinctive ringing for ring groups.
1. Via A@H or FreePBX, create two ZAP extensions for each ZAP channel,

a. The lower number one, i.e. ext 201 uses a "dial" command "ZAP/1r2", where 1 is the ZAP daughter card in slot 1 and "r2" stands for ring type 2 (or r3 or r4). This extension will ring the phone using the distinctive single/long ring.
b. The second ZAP extension, i.e 211, uses just "ZAP/1" as the dial command and will ring the same ZAP phone with a regular US ring tone.

2. Then set it up such that internal callers go to ring group 200 which rings all the distinctive ZAP extensions, i.e. x201, etc. When external callers call, their separate ring group calls x211, etc. with regular ring commands.

The great thing is that the modifications are done through A@H and don't need custom modification of any script.

  • Voicemail is emailed to us now and deleted. To avoid entering a password every time for voice mail, add the voice mail's extension "sXXX" into extensions.conf -> app-messagecenter instead of "default", i.e. s402

Caller ID

The code for sending caller ID from Asterisk to MythTV Frontends. Requires XYAC. Use the startup script below to start up the XYAC daemon at login.

exten => s,1,System(/bin/echo -n -e "@CALL${ARG1} ~${ARG2}" | nc -w 1 127.0.0.1 10629 &)

Note:

  • The "&" at the end runs the "nc" daemon in the background where it can fail silently and not cause delays in answering the calls. Without the "&", the far side can intermittently time-out just as the phones start ringing.
  • This code lives in /etc/asterisk/extensions_custom.conf file
  • The Makefile for XYAC needs to have two changes to compile under Ubuntu 12.04:
INCLUDES=-I/usr/include

and

xyac: xyac.o xosd.o
   $(CC) -o xyac $(LIBS) xyac.o xosd.o $(LFLAGS)

(the gcc compiler now uses the --as-needed flag instead of --no-as-needed which means it has to have the library after the compiling module, otherwise the linker doesn't think any of the functions in the library are needed)

  • The line to startup XYAC and increase the font size is:
 xyac -c yellow --align=center -f -*-*-*-*-*-*-72-*-*-*-*-*-*-* --shadow=4 --delay=45 
The line after the "-f" is the font specifier in XLFD format and depends on what fonts are installed, but the generic one shown uses the first system font with a size of 72, which on a 1080p screen looks big. A good way to find what fonts you have installed is to use "xfontsel" and a common system font is "Bitstream", which you would specify like this
-bitstream-*-*-*-*-*-72-*-*-*-*-*-*-*

Idle Three Hour Sleepd Daemon

  • Below is a method of using the sleepd daemon to shut off a MythTV frontend after three hours and to pop up a 30 second warning that the user can cancel.
# This is called script is called from xfce's autostart manager under "Settings" -> "Sessions and Startup"
#!/bin/sh

# Start up the sleep daemon
# -U waits for 10800 seconds
# -i watches IRQ 4, the remote to reset the counter
# -a Also watches the mouse and keyboard
# -s then runs the standby.sh script
/usr/sbin/sleepd -U 10800 -i 4 -a -s /home/mythtv/standby.sh

Premptable Shutdown Script

  • Here is the standby.sh script called by the sleepd daemon for shutdown:

Note: shutdown requires root access which means using sudo

#!/bin/sh

# Play the sounds
#cd /home/mythtv
#mplayer -softvol -softvol-max 20 cut_power.mp3 &

(
# First wait 30 seconds
echo "10" ; sleep 2
echo "# Oh, it's nothing, whew!  That was close! On with the shutdown" ; sleep 2
echo "20" ; sleep 2
echo "# Ok, on with the shutdown - Resetting the bubble blaster!!" ; sleep 2
echo "30" ; sleep 2
echo "# Bubble blaster reset, now we discharge the Nancy tubes!!" ; sleep 2
echo "40" ; sleep 2
echo "# Tubes discharge, don't forget to put out the teddy sparker!!" ; sleep 2
echo "50" ; sleep 2
echo "# Sparker stuffed! Now we can cancel the impending dog runs!!" ; sleep 2
echo "75" ; sleep 2
echo "# Dog runs cancelled - Now finally ready to stop all run-away princesses" ; sleep 3
echo "# Shutting dddoooowwww [NO CARRIER]" ; sleep 2
echo "100" ; sleep 2
# First kill the sleep daemon 
sleepctl off
# Hibernate
sudo pm-hibernate
# Upon resume, restart the sleep daemon
sleepctl on
) |
# Zenity is the program used for message boxes by Gnome/xfce
zenity --progress \
--title="Shutting Down MythTV FrontEnd Four" \
--text="Wait, there is something wrong!!!" \
--percentage=0 --width=400  --auto-close

Backend Slave and Master MythTV Backend Process Reset

The WAF had been lower of late mostly because our master/save system seems to hang once in a while. I added a method whereby the master, slave backends and the frontend are reset at the push of a remote button. A local script is run when the button is pushed that does the following (call it "restart_entire_myth.sh"):

#!/bin/bash
echo "---> First Kill the Mythfront end..."
killall mythfrontend.real
echo "---> Next, restart the local mythtv server... "
sudo restart mythtv-backend
echo "---> Now wait a few seconds and then kill the remote MythTV backend server ...."
sleep 2
echo "---> Restarting remote process...."
ssh <username>@<hostname> -t "sudo restart mythtv-backend"

I'm not sure the "-t" is necessary now, as I was using it for debugging purposes.

To get that SSH command working, you need to have a properly working SSH link between the computers that doesn't require a password, but uses a public key authentication mechanism. There are lots of resources on the net to set this up and one useful one is here.

You also need to allow a regular MythTV user to restart the processes on both the master and slave backends. With root privileges on each, run visudo and add at the end of the file

<username> <hostname>= NOPASSWD: /sbin/restart mythtv-backend

Where <username> is the user your are either login into via SSH or under which you are running the MythTV frontend. You can test the NOPASSWD setting by exiting root privileges (#exit) and then using "sudo -k" to remove the root access. Then try restarting the mythtv-backend process ($restart mythtv-backend) If the process reset correctly without root access, then it will work for the local or SSH user.

Once you can then run the "restart_entire_myth.sh" script mentioned above to see if actually restarts all the appropriate processes. Once the restart script works, then it is necessary to tie it into the power button on the remote. The first step is to modify the .lircrc file that is used by MythTV to make sure it passes the command to run to "irexec"

begin
 remote = yourremotename
 prog = irexec
 button = Power
 config = restart_entire_myth.sh
 repeat = 0
 delay = 0
end

The directory where "restart_entire_myth.sh" lives is in the path used by the OS to check for files (can't remember how to set this up), but worst case, use the direct path (i.e. /home/my_custom_scripts/restart_entire_myth.sh). The next tricky bit is to get irexec to take input from the right screen. I also had issues with irexec not working unless it was run in a TTY capable terminal (no idea why). To get around these issues, I ran irexec via a script that run when the MythTV user logged in:

#
if [ ! "$(pidof irexec)" ]
then
	sleep 10
        echo "Irexec is not started. Starting Irexec..."
        xfce4-terminal -H -x "irexec"
else
        echo "Irexec is already started."
fi

The "-H" is probably not necessary and just holds the window open, but the "-x" runs the "irexec" command. You can also run this script and test to see if the button press from the remote is getting caught and running the restart script, the output of which appears in the terminal window. I use xfce desktop obviously, but I'm sure the same can be done for the Gnome and KDE terminal programs.

The Plus and Minus of MythTV

  • Note this was a comment added to an article comparing TIVO 3 and MythTV.

I've been a MythTV user for a couple years now and there are a number of things the article missed or glossed over ("+" means a feature and "-" a problem).

(+) Plugins - MythTV has lots of plugins that let you do everything from surf the web, view local weather information, play old style ROM arcade games, watch recordings over the LAN on a windows machine, view a slide show of pictures, to watching XviD encoded videos, all of which we use. The video collection has been of great utility with kids. The usefulness of the plugins was a big reason I chose MythTV and have stayed with it. (Actually TIVO was never considered because of the subscription costs)

(-) Setup and Maintenance - It should not be underestimated how much time it can take getting MythTV to work and then maintaining it. As a MythTV user you will have to know the Linux command line, how to download and install drivers and fix things when they inevitably don't work. If you haven't installed a PCIe card before, and don't like googling stuff to figure out why it doesn't work, then MythTV is not for you. MythTV will probably not save money over a commercial system either because I've found that the money I would have spent on a subscription went into hardware upgrades.

(+) The flip side to the cost and time commitment is that MythTV has immense flexibility, for example, in the size and redundancy of storage capability, the type of peripherals and the split arrangement between backend, slave backend and frontends. Take our ever growing video collection (most ripped from DVDs, fyi), where any new videos are backed up onto a separate machine each night (rsync'd). Try doing that with a TIVO.

(+) The machines are also full blown general purpose computers that can do other things. For example, once you have sunk the money into the MythTV hardware, you can use the same machines for VoIP (via Asterisk) which really saves us money, about $100CND a month. The pay back period for the extra hardware to do VoIP was less then a year, and gets cheaper as time goes on. We use dedicated Digium TDM-400 cards that cost ~$500CND about a year ago. Again, our long distance costs are pennies per minute, each phone is a separate extension, we can make multiple outgoing or incoming calls, late-night callers are prompted by a menu first before the phones ring, we can control our own caller-ID, etc.

(+) DRM You can now buy the Hauppauge HD-PVR which takes the component analog outputs and converts it to a digital USB H.264 MPEG4 stream that works great with MythTV. "5C" encrypted channels can be recorded.For HD tuners using firewire and 5C flagged channels, there is no current work around (as of Feb. 21st, 2007).

Outstanding Issues

  • Not directly related to MythTV but, every couple of months, mdadm kicks all the HDs out of the RAID 5 array on System B. Thankfully, a reboot brings the array right back up, but very strange. This was a hardware issue, which after the MOBO upgrade, is fixed.
  • Every once in a while, the front end will crash, and it seems that a reset of all the MythTV processes is required. I have created a script, which when a "power" button is pressed on the remote, resets the local master backend and frontend and through a SSH tunnel it opens on the fly, resets the remote MythTV backend process (that took a bit of debugging to get a remote user to reset a different user's process) - The latest 0.25 version of MythTV now has a frontend restarter built in and my custom script was created multiple backend processes which completely messed things up. Stability is still an issue over weeks and weeks.
  • As of January, 2013, the HDPVR driver does not have the ID code, 4903, for my HDPVR unit which means /dev/video does not show up and fixing the problem means editing the source and recompiling. Because recompiling a driver is a bitch, I used a hex editor and changed the driver directly (Warning: This is the WRONG way to fix this problem. Don't ever do it. Ever. However, it did work for me in 10.04. Use a hex editor to edit hdpvr.ko, search for text 4901 and replace with 4903, search for hex 0149 and replace with 0349" There is a great thread on this problem here.) Ubuntu 12.04 doesn't have this problem.