Difference between revisions of "User Manual:Periodic Maintenance"

From MythTV Official Wiki
Jump to: navigation, search
m (Optimize the Database)
m (Link corrected | Pathname added)
(25 intermediate revisions by 15 users not shown)
Line 1: Line 1:
{{Navigate|User Manual:Regression Testing|User Manual:Index|User_Manual:Technical_Details}}
+
{{User Manual TOC}}
 +
 
 +
This page is up-to-date to MythTV version 0.20, the current release is {{CurrentRelease}}
 +
 
  
 
== Maintaining your MythTV ==
 
== Maintaining your MythTV ==
  
Your MythTV system can continue to run for weeks, months or even years. Of course, this is unlikely for several reasons.  First of all, you can't keep your hands off! Yes, you're a techie and you like to fiddle. Second, hardware can and will fail eventually. Third, power-outages, tornados, alien takeovers and other unfriendly environmental disasters happen.  But, if you are interested in keeping your MythTV alive for as long as possible, here's a few suggestions.
+
Your MythTV system can continue to run for weeks, months or even years. Of course, hardware fails, techies fiddle, and power-outages, tornadoes, and alien takeovers happen.  But, if you are interested in keeping your MythTV alive for as long as possible, here's a few suggestions.
  
 
== Backup ==
 
== Backup ==
  
Your system is only as good as your backup. What? No Backup? You can't be serious...  Well, now is the time to start backing it up.  Why? Although you had fun building it and are confident you can rebuild quickly so you don't miss your favorite shows, there are two parts of MythTV that aren't easily replaceable: the database, and your media content (TV recordings, videos, photos, music, etc.)
+
Your system is only as good as your backup. Backup NOW!!!!! You may be confident that you can rebuild quickly, but two parts of MythTV aren't easily replaceable: the database, and your media content (TV recordings, videos, photos, music, etc.)
  
 
=== The database ===
 
=== The database ===
  
The database stores a whole lot of stuff you will only miss when it's gone.  For example, it keeps a history of every show you've recorded, which is very nice when you don't want to re-record the same show over again. It keeps all the settings of your backends and frontends, it keeps metadata of all your videos (titles, director, parental levels, etc), it keeps all the commercial flaggings, and more.
+
You'll miss the database only when it's gone.  The database keeps a history of every show you've recorded, which is very nice when you don't want to re-record the same show over again. It keeps all the settings of your backends and frontends, it keeps metadata of all your videos (titles, director, parental levels, etc.), it keeps all the commercial flagging data, bookmarks, and more.
 
 
You can take a backup of your database with the mysqldump command
 
 
 
$ mysqldump -u mythtv -pmythtv mythconverg -c > mythtv_backup.sql
 
 
 
To restore: (assuming that you've dropped the database)
 
  
$ mysql -u root
+
You can create a backup of your database with the backup script. Before doing so, you must configure the backup script by specifying the directory to use for backups:
  mysql>create database mythconverg;
 
  mysql>exit
 
  $ mysql -u mythtv -pmythtv mythconverg < mythtv_backup.sql
 
  
Probably the easiest way to backup the database is to create a cron job that will dump the data out to a file. Here's a sample script that I put in /etc/cron.daily/
+
$ echo "DBBackupDirectory=/home/mythtv" > ~/.mythtv/backuprc
  
#!/bin/sh
+
Then, from that point on, you may create backups by simply running:
#Dumps the mythconverg database - daily backup
 
#Keeps the last 7 days
 
DAY=($(/bin/date +%u))
 
DUMPFILE="mythbe_$DAY.sql.gz"
 
/usr/bin/mysqldump -u mythtv -pmythtv mythconverg -c \
 
      > /mnt/foo/sqldump/$DUMPFILE
 
exit 0
 
  
This will keep a rotating backup of the last 7 days. In this example, I've used a link (/mnt/foo/sqldump/) to a location on a remote server for storing the dump, for added protection.
+
$ mythconverg_backup.pl
  
As dumpfiles of the MythTV database tend to be pretty big, it is probably a good idea to compress these files (especially if you transfer the backup to a remote server). To do this using bzip2, just modify lines 5 & 6 as follows:
+
By default, the backup script will compress and rotate backups, keeping the last 5 backups.  More information and other usage scenarios are shown on the [[Database_Backup_and_Restore|Database Backup and Restore]] page.
  
DUMPFILE="mythdb_$DAY.sql.bz2"
+
If you ever need to [[Database_Backup_and_Restore#Database_Restore|restore the backup]] see the [[Database_Backup_and_Restore|Database Backup and Restore]] page.
/usr/bin/mysqldump -u mythtv -pmythtv mythconverg -c |\
 
      /usr/bin/bzip2 -cq9 > /mnt/foo/sqldump/$DUMPFILE
 
  
 
=== Backing up to a remote server ===
 
=== Backing up to a remote server ===
Line 48: Line 33:
 
<pre>
 
<pre>
 
#!/bin/bash
 
#!/bin/bash
# In this example, the source machine is "startrek", and the remote backup server is "stargate".
+
# In this example, the source machine is "startrek", and the remote backup
 +
# server is "stargate".
 +
LOCAL_HOSTNAME=startrek
 +
BACKUP_SERVER=stargate
 +
# The backup directory
 +
BACKUP_DIR=/home/mythtv/backups
  
 
# Requires ssh tools to make connection to remote host.
 
# Requires ssh tools to make connection to remote host.
 
# (If you use Debian/Ubuntu, apt-get install keychain).
 
# (If you use Debian/Ubuntu, apt-get install keychain).
source /root/.keychain/startrek-sh
+
source /root/.keychain/${LOCAL_HOSTNAME}-sh
  
# Store knowledge of all currently installed packages. Restore with dpkg --set-selections
+
# Store knowledge of all currently installed packages. Restore with dpkg
# (This is good for Debian/Ubuntu users, along with any other distro based on apt/dpkg pkg mgt).
+
# --set-selections (This is good for Debian/Ubuntu users, along with any other
dpkg --get-selections > /startrek_pkgs.txt
+
# distro based on apt/dpkg pkg mgt).
 +
dpkg --get-selections > /${LOCAL_HOSTNAME}_pkgs.txt
  
# Backup up all databases. To restore, shutdown the database, move the ib_* files somewhere else,
+
# Backup up the mythconverg database.
# startup the database, and the mysql -u root < /mysql_backup.txt.
+
mythconverg_backup.pl
# (P.S. This is a good place to put in the example listed above!)
 
mysqldump -A -u root > /mysql_backup.sql
 
  
# Push all the files to the backup disk using rsync. This preserves file permissions,
+
# Push all the files to the backup disk using rsync. This preserves file
# and also deletes old files on the receiving end not found on the sending end.
+
# permissions, and also deletes old files on the receiving end not found on the
 +
# sending end.
 
#
 
#
# This example pushes everything from the listed folders out to destination "stargate",
+
# This example pushes everything from the listed folders out to destination
# stored in a folder designated for this local machine "startrek". It backs up a wiki site,
+
# ${BACKUP_SERVER}, stored in a folder designated for this local machine,
# all the kernels built as "deb's", the /home partition, the /etc files, and all the  
+
# ${LOCAL_HOSTNAME}. It backs up a wiki site, all the kernels built as "deb's",
# /usr/local/src packages (like MythTV SVN!)
+
# the /home partition, the /etc files, and all the /usr/local/src packages
rsync -avq --delete --stats --progress
+
# (like MythTV source code!)
         /startrek_pkgs.txt /mysql_backup.sql /var/www/mediawiki/images
+
rsync -avq --delete --stats --progress \
         /var/www/mediawiki/LocalSettings.php /usr/src/*.deb /home* /usr/local/src
+
         /${LOCAL_HOSTNAME}_pkgs.txt ${BACKUP_DIR} /var/www/mediawiki/images \
        /etc**  
+
         /var/www/mediawiki/LocalSettings.php /usr/src/*.deb /home* \
         stargate:/mnt/backup/startrek
+
        /usr/local/src /etc** \
 +
         ${BACKUP_SERVER}:/mnt/backup/${LOCAL_HOSTNAME}
 
</pre>
 
</pre>
  
 
=== Optimize the Database ===
 
=== Optimize the Database ===
  
In the contrib directory there is a Perl script, [[optimize_mythdb.pl]] to run MYSQL utilities Repair and Optimize on each table in your MythTV database. It is recommended to run it daily from a cron job.
+
Myth interacts with its database all the time.  Without optimization, some of the queries will run slower and slower.  For example, the scheduler query evaluates all of the recording rules against every upcoming listing and against all previously recorded shows.  This is a complicated query that takes a noticeable amount of time to run on an optimized database.  It can easily take 5 times longer on a fragmented database.
 +
 
 +
In the contrib directory there is a Perl script, [[Optimize_mythdb.pl|optimize_mythdb.pl]] to run MYSQL utilities Repair and Optimize on each table in your MythTV database. It is recommended to run it regularly from a cron job. It uses the [[perl bindings]] to connect to the database and therefore should run with no further configuration. 
 +
 
 +
Note that each table in the database is locked while the repair and optimize is performed.  Therefore, the script should only be run when Myth is otherwise idle.  Monthly execution may be sufficient.  Daily may be overkill but ought to be harmless if the system would not otherwise be busy.  Be sure that you have an appropriate backup strategy in place, as well. [[Optimize_mythdb.pl|optimize_mythdb.pl]] can be found in
 +
 
 +
    /usr/share/doc/mythtv-backend/contrib/maintenance
 +
 
  
 
First make sure it is executable:
 
First make sure it is executable:
  
     chmod 755 /usr/share/doc/mythtv-0.20/contrib/optimize_mythdb.pl
+
     chmod 755 /usr/share/doc/mythtv-docs-0.21/contrib/optimize_mythdb.pl
  
Create a shell script to run daily:
+
Create a shell script to run the job:
  
 
  ###### optimize_mythdb.sh
 
  ###### optimize_mythdb.sh
 
  #!/bin/sh
 
  #!/bin/sh
 
   
 
   
  OPT_MYTHDB='/usr/share/doc/mythtv-0.20/contrib/optimize_mythdb.pl'
+
  OPT_MYTHDB='/usr/share/doc/mythtv-docs-0.21/contrib/optimize_mythdb.pl'
 
  LOG='/var/log/mythtv/optimize_mythdb.log'
 
  LOG='/var/log/mythtv/optimize_mythdb.log'
 
   
 
   
Line 97: Line 95:
 
  echo "Finished ${OPT_MYTHDB} on `date`" >> ${LOG}  
 
  echo "Finished ${OPT_MYTHDB} on `date`" >> ${LOG}  
  
run with your daily cron jobs
+
run with your (daily|weekly|monthly) cron jobs
  
 
=== The media ===
 
=== The media ===
Line 105: Line 103:
 
You can backup recordings with a command line program called [https://svn.forevermore.net/nuvexport/ nuvexport]. When run, you will be given an option to export a chosen recording to a location of your choice, giving a new .nuv file along with a .sql file containing information on the recording. This can also be used to import the recording into another mythtv system.
 
You can backup recordings with a command line program called [https://svn.forevermore.net/nuvexport/ nuvexport]. When run, you will be given an option to export a chosen recording to a location of your choice, giving a new .nuv file along with a .sql file containing information on the recording. This can also be used to import the recording into another mythtv system.
  
====[[Mythrename.pl | mythrename.pl]]====
+
====[[Mythlink.pl | mythlink.pl]]====
 +
 
 +
A utility written by Chris Peterson (of mythweb fame) creates a view of human-readable symlinks to your recordings.  This allows you to archive recordings to other media by copying from the symlink view (and dereferencing the link) so you can identify your archived videos by filename.
  
There is a utility written by Chris Peterson (of mythweb fame), which will create human readable filenames of your recordings.  This allows you to archive them away and know what they are by reading the filename.  It can optionally create symlinks instead of renaming the files.
+
For more info, see the output of <tt>mythlink.pl --help</tt>
  
For more info, see the output of <tt>mythrename.pl --help</tt>
+
If you can't locate the mythlink.pl script, try looking in <tt>/usr/share/doc/mythtv-0.23/contrib</tt> (insert actual mythtv version as appropriate).
  
If you can't locate the mythrename.pl script, try looking in <tt>/usr/share/doc/mythtv-0.20/contrib</tt> (insert actual mythtv version as appropriate).
+
====[[Mythfs.pl]]====
 +
 
 +
A utility written by Lincoln Stein mounts a virtual filesystem corresponding to your recordings. It uses the 0.25 API so that the backend storage directories do not need to be NFS mounted on the client, nor does the MySQL database need to be accessible to the user mounting the filesystem.
 +
 
 +
Once the filesystem is mounted, you may use the human-readable filenames to play or archive your recordings.
 +
 
 +
The script can be found on github. See [[Mythfs.pl]] for instructions.
  
 
== OS and software maintenance ==
 
== OS and software maintenance ==
  
I'll start by saying: If it ain't broke, don't fix it! This is more difficult to follow if your MythTV serves other purposes besides Myth. I might add that if you have non-technical people using Myth for their daily TV watching, they will appreciate you keeping your hands off. If you must mess with Myth, please be sure to backup before you do. Expect that fixing one thing will break another.
+
If it ain't broke, don't fix it! This is difficult to follow if your MythTV serves other purposes besides Myth. If non-techies are using Myth for their daily TV watching, they will appreciate you keeping your hands off! If you must mess with Myth, please backup first. Fixing one thing may break another.
 +
 
 +
== XFS Filesystem Defragmentation ==
 +
 
 +
If you use XFS to store your media it can get to a fragmentation level where your system will be slower to respond.  As the root user check your fragmentation level.
 +
 
 +
  xfs_db -c frag -r /xfs_disk1
 +
 
 +
Defragment your filesystem.
 +
 
 +
  xfs_fsr -v /xfs_disk1
 +
 
 +
[[Category:HOWTO]]

Revision as of 14:00, 25 January 2016


This page is up-to-date to MythTV version 0.20, the current release is 34.0


Maintaining your MythTV

Your MythTV system can continue to run for weeks, months or even years. Of course, hardware fails, techies fiddle, and power-outages, tornadoes, and alien takeovers happen. But, if you are interested in keeping your MythTV alive for as long as possible, here's a few suggestions.

Backup

Your system is only as good as your backup. Backup NOW!!!!! You may be confident that you can rebuild quickly, but two parts of MythTV aren't easily replaceable: the database, and your media content (TV recordings, videos, photos, music, etc.)

The database

You'll miss the database only when it's gone. The database keeps a history of every show you've recorded, which is very nice when you don't want to re-record the same show over again. It keeps all the settings of your backends and frontends, it keeps metadata of all your videos (titles, director, parental levels, etc.), it keeps all the commercial flagging data, bookmarks, and more.

You can create a backup of your database with the backup script. Before doing so, you must configure the backup script by specifying the directory to use for backups:

$ echo "DBBackupDirectory=/home/mythtv" > ~/.mythtv/backuprc

Then, from that point on, you may create backups by simply running:

$ mythconverg_backup.pl

By default, the backup script will compress and rotate backups, keeping the last 5 backups. More information and other usage scenarios are shown on the Database Backup and Restore page.

If you ever need to restore the backup see the Database Backup and Restore page.

Backing up to a remote server

Using rsync along with a trusted ssh connection is a way to move backup files from the current machine (your MythTV server) to an alternate machine you may have set up for backup. rsync supports many protocols, including rsh and ssh. This example demonstrates ssh.

#!/bin/bash
# In this example, the source machine is "startrek", and the remote backup
# server is "stargate".
LOCAL_HOSTNAME=startrek
BACKUP_SERVER=stargate
# The backup directory
BACKUP_DIR=/home/mythtv/backups

# Requires ssh tools to make connection to remote host.
# (If you use Debian/Ubuntu, apt-get install keychain).
source /root/.keychain/${LOCAL_HOSTNAME}-sh

# Store knowledge of all currently installed packages. Restore with dpkg
# --set-selections (This is good for Debian/Ubuntu users, along with any other
# distro based on apt/dpkg pkg mgt).
dpkg --get-selections > /${LOCAL_HOSTNAME}_pkgs.txt

# Backup up the mythconverg database.
mythconverg_backup.pl

# Push all the files to the backup disk using rsync. This preserves file
# permissions, and also deletes old files on the receiving end not found on the
# sending end.
#
# This example pushes everything from the listed folders out to destination
# ${BACKUP_SERVER}, stored in a folder designated for this local machine,
# ${LOCAL_HOSTNAME}. It backs up a wiki site, all the kernels built as "deb's",
# the /home partition, the /etc files, and all the /usr/local/src packages
# (like MythTV source code!)
rsync -avq --delete --stats --progress \
         /${LOCAL_HOSTNAME}_pkgs.txt ${BACKUP_DIR} /var/www/mediawiki/images \
         /var/www/mediawiki/LocalSettings.php /usr/src/*.deb /home* \
         /usr/local/src /etc** \
         ${BACKUP_SERVER}:/mnt/backup/${LOCAL_HOSTNAME}

Optimize the Database

Myth interacts with its database all the time. Without optimization, some of the queries will run slower and slower. For example, the scheduler query evaluates all of the recording rules against every upcoming listing and against all previously recorded shows. This is a complicated query that takes a noticeable amount of time to run on an optimized database. It can easily take 5 times longer on a fragmented database.

In the contrib directory there is a Perl script, optimize_mythdb.pl to run MYSQL utilities Repair and Optimize on each table in your MythTV database. It is recommended to run it regularly from a cron job. It uses the perl bindings to connect to the database and therefore should run with no further configuration.

Note that each table in the database is locked while the repair and optimize is performed. Therefore, the script should only be run when Myth is otherwise idle. Monthly execution may be sufficient. Daily may be overkill but ought to be harmless if the system would not otherwise be busy. Be sure that you have an appropriate backup strategy in place, as well. optimize_mythdb.pl can be found in

   /usr/share/doc/mythtv-backend/contrib/maintenance


First make sure it is executable:

   chmod 755 /usr/share/doc/mythtv-docs-0.21/contrib/optimize_mythdb.pl

Create a shell script to run the job:

###### optimize_mythdb.sh
#!/bin/sh

OPT_MYTHDB='/usr/share/doc/mythtv-docs-0.21/contrib/optimize_mythdb.pl'
LOG='/var/log/mythtv/optimize_mythdb.log'

echo "Started ${OPT_MYTHDB} on `date`" >> ${LOG}
${OPT_MYTHDB} >> ${LOG}
echo "Finished ${OPT_MYTHDB} on `date`" >> ${LOG} 

run with your (daily|weekly|monthly) cron jobs

The media

TODO: media backup

You can backup recordings with a command line program called nuvexport. When run, you will be given an option to export a chosen recording to a location of your choice, giving a new .nuv file along with a .sql file containing information on the recording. This can also be used to import the recording into another mythtv system.

mythlink.pl

A utility written by Chris Peterson (of mythweb fame) creates a view of human-readable symlinks to your recordings. This allows you to archive recordings to other media by copying from the symlink view (and dereferencing the link) so you can identify your archived videos by filename.

For more info, see the output of mythlink.pl --help

If you can't locate the mythlink.pl script, try looking in /usr/share/doc/mythtv-0.23/contrib (insert actual mythtv version as appropriate).

Mythfs.pl

A utility written by Lincoln Stein mounts a virtual filesystem corresponding to your recordings. It uses the 0.25 API so that the backend storage directories do not need to be NFS mounted on the client, nor does the MySQL database need to be accessible to the user mounting the filesystem.

Once the filesystem is mounted, you may use the human-readable filenames to play or archive your recordings.

The script can be found on github. See Mythfs.pl for instructions.

OS and software maintenance

If it ain't broke, don't fix it! This is difficult to follow if your MythTV serves other purposes besides Myth. If non-techies are using Myth for their daily TV watching, they will appreciate you keeping your hands off! If you must mess with Myth, please backup first. Fixing one thing may break another.

XFS Filesystem Defragmentation

If you use XFS to store your media it can get to a fragmentation level where your system will be slower to respond. As the root user check your fragmentation level.

 xfs_db -c frag -r /xfs_disk1

Defragment your filesystem.

  xfs_fsr -v /xfs_disk1