NexTView Import
This How-To describes a procedure to harvest EPG data for MythTV using the nexTView protocol. This method is only interesting for those who have access to the European continent. This data is not sent in the rest of the world.
Note: The scripts and methods used to import Nextview data into MythTV shown on this page are not officially supported or recommended. The preferred solution would be to create a fully XMLTV compliant grabber. This would be much easier to setup and changes within mythtv would not require scripts to be changed or re-written.
Contents
Required Software
nxtvepg - This software provides an interface to the nexTView protocol, both in console and as a GUI (X) application. 2.7.5 should be the lowest version that you should install.
Install 'nxtvepg'
sudo apt-get install nxtvepg
Install 'scantv'
sudo apt-get install scantv
Setup and Testing nxtvepg
Get the latest software as described above. Now start 'nxtvepg' to do some tests, try to get your EPG data.
nxtvepg
If this works you should now have a new window, there go to
Configure -> Provider scan -> start-scan
Configure -> Select Provider
you can exit nxtvepg. Now try to harvest data to be exported to XMLTV format. This is done with the following command:
nxtvepg -daemon -acqonce full -provider d92 -nodetach
This runs nxtvepg without GUI and acquires the provider d92 (Kabel1 in Germany) and leaves the console locked until the harvesting is done. Be patient - this may take a long time (in the range of 20 minutes to 50min). Now we will try to export the harvested data to a XML file.
nxtvepg -provider d92 -dump xml > /tmp/nxtvepgtest.xml
You should now have a file in /tmp of about 1 Megabyte size. Open it up in your favorite editor to check for correctness.
Scripts
At first copy the cnilist and the 2 Scripts(autochannel.sh & harvest) to a folder e.g. /home/yourname/epg give them the name they have. After that, come back here and follow the rest of the How to.
Ensure that you have an empty line at the end of every script
Channel list
Following list of channels should be extended to include those you might need:
# CNI-Code | Name (must fit to scantvdump and is Case Sensitive) | callsign # This list is based on SF 1, you may have to change to your needs. CNI04C1=SF 1=SF 1 CNI04C7=SF 2=SF 2 CNI04CC=SFi=SFi CNI0DC1=ARD=ARD CNI0DC2=ZDF=ZDF CNI0497=3PLUS=3+ CNI0DC7=3sat=3SAT CNI0D85=Arte=ARTE CNI0DC9=ARD/ZDF Kinderkanal=[[Ki Ka]] CNI0DCB=BR-3=BR 3 CNI0DFE=MDR-3 Landesweit=MDR3 CNI0DE0=SW 3 Baden-Wuerttemberg=SW 3 CNI1DE6=WDR-3 Landesweit=WDR3 CNI0D8D=DSF=DSF CNI0D91=Eurosport=EUSP CNI0D92=Kabel 1=KAB1 CNI0D8C=n-tv=NTV CNI0487=PRO 7=PRO7 CNI0DAB=RTL=RTL CNI0D8F=RTL 2=RTL2 CNI0DB9=SAT 1=SAT1 CNI0D8A=Super RTL=SRTL CNI0D8E=VOX=VOX CNI0AC1=ORF 1=ORF1 CNI0AC2=ORF 2=ORF2 CNI04C2=TSR 1=TSR1 CNI04C8=TSR 2=TSR2 CNI33F1=TF1=TF 1 CNI33F2=France 2=FR 2 CNI33F3=France 3=FR 3 CNI33F6=M6=M6 CNI33F8=RTL 9=RTL9 CNIF500=TV5=TV 5 CNI04C3=TSI 1=TSI1 CNI04C9=TSI 2=TSI2 CNI3901=RAI 1=RAI1 CNI3902=RAI 2=RAI2 CNIFA05=Canale 5=CNL5 CNIFA06=Italia 1=IT 1 CNIFA04=RETE 4=RET4 CNI9001=TRT-1=TRT1 CNI0DD4=Nord-3 (NDR/SFB/RB)=N3 CNI0DBA=NEUN LIVE=9LIV CNI0DCF=Hessischer Rundfunk=HR 3 CNI0D82=Radio Berlin-Brandenburg=RBB CNI0D88=VIVA 1=VIVA CNIFFEF=MTV=MTV CNI0D7A=N 24=N24 CNI0D78=Tele 5=Tele5 ???????=BBC Prime=BBC1 ???????=RTP1=RTP1 CNI0x0486=Star TV=STAR CNI0x01F2=CNN International=CNNI ???????=Canal+=CNL+ CNI0x0481=TeleZüri=TZ
Automatic import channel data
The following script scans for channels then combines the channels with the cnilist and adds these to MythTV.
To test it run the command: (for Hauppauge read further down)
scantv -o /home/yours/scantv_output
Hauppauge PVR owner
I testet the script with PVR-350 and PVR-500(2x150inside). I had to do this change in the script autochannel.sh:
(You still get an error-message, but it works anyway)
# Scan input for channel information and dump it into a file #scantv -a -f europe-west -n PAL-BGH -o $autochannel # comment out the line above for PVR and use this instead scantv -a -f europe-west -n PAL-BGH -c /dev/video0 -C /dev/vbi0 -o $autochannel
The script autochannel.sh will run a full scan and takes 30 - 60min, you will see the lines in the bash passing by, while scanning.
In the path you start it, it will create a file called scantvdump_(day-number).txt .
It is recommended using this only if you have not added channels manually, this is because it will mix up your already entered channels.
If you can live with it, go ahead.
First you need to make it executeable sudo chmod 777 autochannel.sh. Then enter your
password in this script (it is in the 4 line before bottom) for the MySQL-Database you can find it in /etc/mythtv/mysql.txt.
#!/bin/bash # Scan tv source for channel information and insert it into MythTV's database # Written by zagibu@gmx.ch # Distributed under the terms of the GNU General Public License v2 # Date: 21.05.2005 # Script update: 27.01.2008 added line for HPG PVR's, MySQl options changed to REPLACE, hint about password, removed CH for channel entry version=v0.1 command_list="scantv mysql head tail cut wc" autochannel=scantvdump_$(date +\%d).txt # Verify that each command we use exists for command in $command_list; do path=$(which $command | grep "no $command in") if [ ! -x $(which $command) -a "$path" ]; then echo -e "\n\nERROR: $command not found! Check your \ $command_list and/or your \$PATH" exit -1 fi done echo -e "\nScanner $version" echo "============" # Scan input for channel information and dump it into a file scantv -a -f europe-west -n PAL-BGH -o $autochannel # comment out the line above for PVR and use this instead #scantv -a -f europe-west -n PAL-BGH -c /dev/video0 -C /dev/vbi0 -o $autochannel # Count lines of scantv dump lines=$(cat $autochannel | wc -l | cut -f1 -d' ') # Subtract amount of header lines (7) lines=$(expr $lines - 7) # Read each channel entry of scantv dump for (( i=0; i<$lines; i=$i+3 )) do chanid=$(expr $(expr $i + 3) / 3) channum=$chanid freqid=$(tail -n$(expr $lines - $(expr $i + 1)) $autochannel | head -n1 | cut -f3 -d' ') name=$(tail -n$(expr $lines - $i) $autochannel | head -n1) name=$(echo $name | cut -f2 -d'[' | cut -f1 -d']') callsign=$(cat cnilist | grep "=$name=" | cut -f3 -d'=') xmltvid=$(cat cnilist | grep "=$name=" | cut -f1 -d'=') # Insert gathered channel info into mythtv's db # If you use the short option form (-p), you CANNOT have a space between the option and the password mysql -u mythtv -pyour_passw -D mythconverg -e"INSERT INTO channel (chanid, channum, freqid, sourceid, callsign, name, xmltvid, tvformat) VALUES ($chanid, '$channum', '$freqid', 1, '$callsign', '$name', '$xmltvid', 'PAL');" done
Check the import of channels and xmltvID
Instead of doing it one by one with mythtv-setup, we can read out the mySQL-Database. Please be very careful, it is YOUR database!
To do that, you need to know the password for your database. You already read it out before to run the script autochannel.sh or you look in
/etc/mythtv/ there you find the file mysql.txt inside is your password (on a mythbuntu).
Login to your mySQL type:
mysql -u mythtv -pyour_passw -D mythconverg
select channum, xmltvid, name, callsign, chanid, tvformat, freqid from channel where sourceid = "1";
Now you should get a pretty output like:
mySQL output of channels
+---------+-----------+-------------------------+---------------+--------+----------+--------+
- channum : xmltvid : name : callsign : chanid : tvformat : freqid :
+---------+-----------+-------------------------+---------------+--------+----------+--------+
- 1 : CNI04C1 : SF 1 : SF 1 : 1 : PAL : 175600 :
- 2 : CNI04C7 : SF 2 : SF 2 : 2 : PAL : 210910 :
- 3 : CNI0DC7 : 3sat : 3SAT : 3 : PAL : S33 :
- 4 : CNI0AC1 : ORF 1 : ORF1 : 4 : PAL : SE5 :
- 5 : CNI0497 : 3+ : 3Plus : 5 : PAL : 260310 :
- 6 : CNI0D8E : VOX : VOX : 6 : PAL : 30 :
- 7 : CNI0487 : PRO 7 : PRO7 : 7 : PAL : 27 :
...
- 36 : : Home Shopping Europe : Home SE : 36 : PAL : 495230 :
+---------+-----------+-------------------------+---------------+--------+----------+--------+
Do a clean logout of mySQL with: quit
Acquiring EPG-data and Sync Clock
The following script allows to automate the process we tested above of harvesting EPG-data. Make sure to change the provider code in the script, to your preferred provider - see nexTView Provider Table
First you need to make the script executeable sudo chmod 777 harvest
It must be tested (started/run) as myth-user.
Go to the to the directory you saved it and start it with:
./harvest
While it is acquiring data, you can stop it any time with Ctrl+c to see if it works instead of waiting 20-50min.
A Logfile helps us to keep track of the EPG-Data updates. Create an empty text file in /var/log
sudo touch /var/log/epg_data.log
Then change the owner of the file
sudo chown yourusername:mythtv /var/log/epg_data.log
SHELL=/bin/bash #!/bin/bash # Get time and epg data from teletext, then insert it into MythTV's database # Written by zagibu@gmx.ch # Distributed under the terms of the GNU General Public License v2 # Date: 21.05.2005 # Script update: 27.01.2008 added start-/finshtime to print # Script update: 07.04.2008 removed the '-1' at the last command # Script update: 13.08.2008 added: SHELL=/bin/bash at top, code for start/stop output new into logfile. version=v0.1 command_list="nxtvepg hwclock mythfilldatabase" epgdump=nxtvepgdump_$(date +\%d).txt # verify that each command we use exists for command in $command_list; do path=$(which $command | grep "no $command in") if [ ! -x $(which $command) -a "$path" ]; then echo -e "\n\nERROR: $command not found! Check your \ $command_list and/or your $PATH" exit -1 fi done echo "Adjusting time (using teletext as reference)..." # Set system time to current teletext time nxtvepg -clock set # Save system time to hardware clock hwclock --systohc # Remember to change this to your NexTView provide of choice echo "* * Acquiring NexTView EPG from provider Kabel 1 & SF 1... started `date '+%R %a %d %b'` * *" >> /var/log/epg_data.log echo "* * Acquiring NexTView EPG from provider Kabel 1 & SF 1... started `date '+%R %a %d %b'` * *" nxtvepg -acqonce full -daemon -nodetach -provider 0D92 # Dump current EPG data into a file (xmltv5 format must be set in .nxtvepgrc) echo "* Acquiring NexTView EPG from provider Kabel 1 & SF 1...finished `date '+%R %a %d %b'` *" >> /var/log/epg_data.log echo "* finished acquiring `date +%R` *" nxtvepg -dump xml -provider 0D92 > $epgdump echo "Loading [[Next View]] EPG data into MythTV..." # Insert EPG data into MythTV's database, do not add more channels mythfilldatabase --update --file 1 $epgdump echo "All processes completed. Enjoy your [[Myth Box]]!"
Cron Job to automate the process
How to automate the harvest
.
It depends on your use, if the machine is running all day long you may want it in cron
to be done at 3 o'clock in the morning.
If you shutdown your machine daily you may want it to run right in the boot-process.
cron and/or anacron
Anacron is meant for machines that don't run 24x7.
- cron - daemon to execute scheduled commands
- anacron - runs commands periodically
What is anacron:
Anacron (like `anac(h)ronistic') is a periodic command scheduler.
It executes commands at intervals specified in days. Unlike cron, it does not assume that the system is running continuously.
It can therefore be used to control the execution of daily, weekly and monthly jobs (or anything with a period of n days), on systems that don't run 24 hours a day. When installed and configured properly, Anacron will make sure that the commands are run at the specified intervals as closely as machine-uptime permits.
more details you can find in your command-console 'bash'
- man cron
- man 5 crontab
- man anacron
- man anacrontab
Required Software? None, cron is a standard and anacron too for missed jobs.
cron while booting and daily
Every user has a separate crontab stored in /var/spool/cron/crontabs/ or /var/cron/tabs
To edit the crontab simply type:
crontab -e
Then enter the information for reboot and every 24h at 3 o'clock in the morning.
The file is at first a /tmp but, when you save it, it will tell you:
no crontab for yourusername - using an empty one
crontab: installing new crontab
#----------------------------------------------------------------- # This script shall start a script at boot/reboot to get the # EPG-Data for the MediaCenter # Shell variable for cron SHELL=/bin/bash # PATH variable for cron PATH=/usr/local/bin:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin:/usr/bin/X11 #M S T M W Befehl #----------------------------------------------------------------- @reboot /home/username/epg/cronjob_harvest # run at 03:01am every night 1 3 * * * /home/username/epg/cronjob_harvest #-----------------------------------------------------------------
script called-up by the cronjob
At first copy this sript to a folder e.g. /home/yourname/epg give it the name it has. Change 3times the yourname in the script to your directory.
Then you need to make it executeable sudo chmod 777 cronjob_harvest
After that, you can test it by starting it.
/home/yourname/epg/cronjob_harvest
SHELL=/bin/bash #!/bin/bash # remove old files and start harvesting EGP-Data, then insert it into MythTV's database # Written by reto007@hotm....com # Distributed under the terms of the GNU General Public License v2 # Date: 21.08.2008 # changes: version=v0.1 command_list="crontab nxtvepg mythfilldatabase" # verify that each command we use exists for command in $command_list; do path=$(which $command | grep "no $command in") if [ ! -x $(which $command) -a "$path" ]; then echo -e "\n\nERROR: $command not found! Check your \ $command_list and/or your $PATH" exit -1 fi done # remove old files rm -v /home/yourname/epg/nxtve* >> /var/log/epg_data.log # go to directory cd /home/yourname/epg/ # start harvesting EPG-Data /home/yourname/epg/harvest
experiences - problems
27. Jan. 2008, by Tido, I tested it with Mythbuntu 7.10 PVR-500 and PVR-350 it runs fine. Aferwards I worked a lot on this page to make it better for the next. So please leave comment here.
18. Sep. 2010, by Tido, it still works on my old machine like a charme. No Website updating or anything. Just runs :-)
About
This Howto is based on several forum threads: