XMLvideolist

From MythTV Official Wiki
Jump to: navigation, search

Scope

After using MythTV for some time now, I have a rather large sum of recorded video files. I transcoded those and store them on a large RAID. I searched for a way to manage those from everywhere through a web browser. In particular I wanted to know wether I already have a video file for a particular movie when deciding on wether or not to record that movie. Further helpful would be to know if I had a movie previously recorded as SDTV that was now aried as HDTV. Thus I would need technical details about the movie video file on disk. I came up with the following solution:

  • A Cron job searches through my directory and all subdirs, queries all video files and adds the relevant info to an XML file.
  • This XML is then copied to a directory of the apache webserver, that is running for mythweb.
  • A small patch to Mythweb allows searching the XML catalog file from within mythweb.


Demo

I made a small demo here, so it is easier to understand what this is all about:


So how does it work?

Generating XML File

First step is a series of scripts, that search trough the video directory for video files. These are queried via Mediainfo for their technical properties. The query results are stored in a catalog.xml file. Unfortunately I don't know how to integrate these three scripts into one. Second, all video files are querried one after the other, so with a large video dir this can take several hours. Maybe someone can help me improve this?


This is the first script (start.sh), called via cron: (Cron jobs can easily set up e.g. with Gnome-shedule)

#!/bin/bash
#start.sh
#select subdirs
starttime=$(date +%F-%H-%M-%S)
library="System1" #I have different video librarys
catalog="/path/to/videocatalog.xml" #U need to customize
mv "$catalog" "$catalog-old"
touch "$catalog"
echo "creating database $catalog"
echo "processing $library"
find /path/to/video/basedir -type d  -exec /path/to/step2.sh "{}" "$library" "$catalog" \; 
stoptime=$(date +%F-%H:%M:%S)
echo "done with $library at $stoptime"
# U could insert other libraries with different video base dir here
echo "<videocatalog-"$starttime">" > "$catalog-new"
cat $catalog | sed 's/&/+/g' >> "$catalog-new"
echo "</videocatalog-"$starttime">" >> "$catalog-new"
mv "$catalog-new" "$catalog"
# upload $catalog to your webserver directory
exit 0

This is the second script (step2.sh), called from the first script.

#!/bin/bash 
#step2.sh
#$1=path, $2=library $3 catalog
if [ -d "${1}" ] ; then
  cd "${1}"
else
  echo "Error: bad argument. Expected a valid directory name for the first argument"
  echo "Bad directory name = ${1}"
  exit 1
fi
######
echo "##########################################"
echo "Starting: path: $1 library: $2 catalog: $3"
echo "##########################################"

find *.avi -exec /path/to/step3.sh "{}" "$1" "$2" "$3" \; 
find *.mkv -exec /path/to/step3.sh "{}" "$1" "$2" "$3" \; 
find *.mp4 -exec /path/to/step3.sh "{}" "$1" "$2" "$3" \; 
find *.m4v -exec /path/to/step3.sh "{}" "$1" "$2" "$3" \; 
# you may add other file types here
exit 0

The second script then calles the final script (step3.sh), wich does the query on a single video file. It uses Mediainfo, so that needs to be installed.

 
#!/bin/bash
#step3.sh
#variables: $1=file, $2=path, $3=library, $4=catalogfile
catalog="$4"
echo "querying: $1"
entry="<film>"
exit="</film>"
path='<path>'"$2"'</path>' 
library='<library>'"$3"'</library>' 

#Reading Infos from File

#Getting Number of Audio Tracks
acount="$(mediainfo --Inform="General;%AudioCount%" "${1}")" 

#Get General Info
general="$(mediainfo --Inform="General;<name>%FileName%</name>\r<filesize>%FileSize/String%</filesize>\r<duration>%Duration/String3%</duration>\r<gformat>%Format%</gformat>\r<gbitrate>%OverallBitRate/String%</gbitrate>\r<acount>%AudioCount%</acount>\r" "${1}")"

#Get Video Info
video="$(mediainfo --Inform="Video;<aspectr>%DisplayAspectRatio/String%</aspectr>\r<vformat>%Format%</vformat>\r<vbitrate>%BitRate/String%</vbitrate>\r<vwidth>%Width%</vwidth>\r<vheight>%Height%</vheight>\r<fps>%FrameRate%</fps>\r" "${1}")"

#Get Audio Info
aformat="$(mediainfo --Inform="Audio;<track>%Format%</track>\r" "${1}")" 
aformat2="$(mediainfo --Inform="Audio;%Format%" "${1}")" 
channels="$(mediainfo --Inform="Audio;<track>%Channel(s)%</track>\r" "${1}")" 
channels2="$(mediainfo --Inform="Audio;%Channel(s)%" "${1}")" 
abiteratem="$(mediainfo --Inform="Audio;<track>%BitRate_Mode/String%</track>\r" "${1}")"'\n'
abiteratem2="$(mediainfo --Inform="Audio;%BitRate_Mode/String%" "${1}")" 
abiterate="$(mediainfo --Inform="Audio;<track>%BitRate/String%</track>\r" "${1}")"'\n'
abiterate2="$(mediainfo --Inform="Audio;%BitRate/String%" "${1}")" 
sampling="$(mediainfo --Inform="Audio;<track>%SamplingRate/String%</track>\r" "${1}")"'\n'
sampling2="$(mediainfo --Inform="Audio;%SamplingRate/String%" "${1}")" 
language="$(mediainfo --Inform="Audio;<track>%Language/String%</track>\r" "${1}")"'\n' 
language2="$(mediainfo --Inform="Audio;%Language/String%" "${1}")" 
#asselmble line for unknown audio items
EMPTYTRACK='<track>unknown</track>'
for (( I=1; I <= "$acount"; I++ ))
do
unknown=${unknown}${EMPTYTRACK}'\n'
done
#apply unknown als value
if [[ -z "$abiteratem2" ]]; then
abiteratem="$unknown"
fi  
if [[ -z "$abiterate2" ]];then
abiterate="$unknown"
fi  
if [[ -z "$sampling2" ]]; then
sampling="$unknown"
fi 
if [[ -z "$language2" ]]; then
language="$unknown"
fi   
#build complete audio variable
audio='<aformat>''\n'"$aformat"'\n''</aformat>''\n''<channels>''\n'"$channels"'\n''</channels>''\n''<abitratem>''\n'"$abiteratem"'</abitratem>''\n''<abitrate>''\n'"$abiterate"'</abitrate>''\n''<sampling>''\n'"$sampling"'</sampling>''\n''<language>''\n'"$language"'</language>''\n'
total="$entry"'\n'"$general"'\n'"$library"'\n'"$path"'\n'"$video"'\n'"$audio""$exit"
echo -e "$total" >> "$catalog"
exit 0

Now you should have a nice XML file like the one linked above.


Displaying XLM File

Second step is now to find a nice way of displaying and searching the info stored in the XML file. You would't want to look at it in raw form now, would you?

I coded a html file to dispay it (see link at top):

  • optimized for 1920x180 (can be customized via CSS)
  • search function
  • display wikipedia info for the film (actors, story, etc)
  • can be searched from witin mythweb (via javascript function searchonload)
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//DE" "http://www.w3.org/TR/html4/strict.dtd">
<html><head><meta content="text/html; charset=UTF8" http-equiv="content-type">
<title>Video Catalog</title>
<style type="text/css">
table.sample {
	border: 6px inset #8B8378;
	-moz-border-radius: 2px;
}
table.sample td {
	border: 1px solid black;
	padding: 0.2em 2ex 0.2em 2ex;
	color: black;
}
table.sample tr.d0 td {
	background-color: #c3d0e2;
}
table.sample tr.d1 td {
	background-color: #FEFEF2;
}
#showList {
  background: #fff;
  float:left; 
  width: 34%;
  height:900px;
  margin-right: 66%;
  left: 0px;
  overflow: scroll;
  overflow-x: hidden;
}
#showWiki{
  background: #fff;
  float: center;
  position: fixed;
  top: 13%;
  height: 85%;
  width: 30%;
  margin-left: 34%;
  margin-right: 35%;
<<!--  overflow: auto;-->
}
#showFilm{
  background: #fff;
  position: fixed;
  float: right;
  top: 13%;
  height: 85%;
  width: 34%;
  margin-left: 65%;
  margin-right: 5px;
  overflow: auto;
}
#suchmaske{
  background: ##5f9ea0;
  float: right;
  position: fixed;
  top: 005px;
  height: 12%;
  width: 100%;
  left: 55%;
  right: 0px;
}

</style>

<script type="text/javascript">
//  Loads the XML File
if (window.XMLHttpRequest)
  {// code for IE7+, Firefox, Chrome, Opera, Safari
  xmlhttp=new XMLHttpRequest();
  }
else
  {// code for IE6, IE5
  xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");
  }
xmlhttp.open("GET","catalog.xml",false); //loading xml file
xmlhttp.send();
xmlDoc=xmlhttp.responseXML; 
x=xmlDoc.getElementsByTagName("film"); //separator from xml file


function getQueryVariable(variable) {
//Function to search video catalog for url substring like http://ur.website.com?x=queryvariable
var query = window.location.search.substring(1);
//  Replace Geman Umlauts
  var query = query.replace("%C3%9F", "ß");
  var query = query.replace("%C3%B6", "ö");
  var query = query.replace("%C3%96", "Ö;");
  var query = query.replace("%C3%BC", "ü");
  var query = query.replace("%C3%9C", "Ü");
  var query = query.replace("%C3%A4", "ä");
  var query = query.replace("%C3%84", "Ä");
//  Unfortunately search doenst work perfect yet: only the first to words of a string are searched for
  var query2 = query.replace("+", " ");
  var vars = query2.split("+");
  for (var i=0;i<vars.length;i++) {
   var pair = vars[i].split("=");
//var pair = pair.replace("+", " ");
    if (pair[0] == variable) {
      return pair[1];
    }
  } 
}


function searchOnload() { // search the index on page load
	// get the search term from the URL
	var searchterm = getQueryVariable("x");	
	results = new Array;
	if (searchterm.length < 3) {
		alert("Suchwort braucht mind. 3 Buchstaben.");
	} else {
		for (var i=0;i<x.length;i++) {
// see if the XML entry matches the search term,
// and (if so) store it in an array
			var name = (x[i].getElementsByTagName("name")[0].childNodes[0].nodeValue);
			//var name = allitems[i].lastChild.nodeValue;
			var exp = new RegExp(searchterm,"i");
			if ( name.match(exp) != null) {
				results.push(x[i]);
			}
		}
// send the results to another function that displays them to the user
	showResults(results, searchterm);
	}
}



function displayFilmInfo(i)
{
//this function reads the info from the xml file and stores it in the variable called txt to show them later
//General Information
Name=(x[i].getElementsByTagName("name")[0].childNodes[0].nodeValue);
var Name = Name.replace(/_/g, " ");
Library=(x[i].getElementsByTagName("library")[0].childNodes[0].nodeValue);
Path=(x[i].getElementsByTagName("path")[0].childNodes[0].nodeValue);
Gformat=(x[i].getElementsByTagName("gformat")[0].childNodes[0].nodeValue);
Size=(x[i].getElementsByTagName("filesize")[0].childNodes[0].nodeValue);
Duration=(x[i].getElementsByTagName("duration")[0].childNodes[0].nodeValue);
Gbitrate=(x[i].getElementsByTagName("gbitrate")[0].childNodes[0].nodeValue);
//Video Infos
Vformat=(x[i].getElementsByTagName("vformat")[0].childNodes[0].nodeValue);
Vbitrate=(x[i].getElementsByTagName("vbitrate")[0].childNodes[0].nodeValue);
Width=(x[i].getElementsByTagName("vwidth")[0].childNodes[0].nodeValue);
Height=(x[i].getElementsByTagName("vheight")[0].childNodes[0].nodeValue);
Aspect=(x[i].getElementsByTagName("aspectr")[0].childNodes[0].nodeValue);
Fps=(x[i].getElementsByTagName("fps")[0].childNodes[0].nodeValue);
//Audio Track1 Infos
audiocount=(x[i].getElementsByTagName("acount")[0].childNodes[0].nodeValue);
if (audiocount=="unknown")
{
Audio="";
}
else
{
//Loop the audio tracks
Audio="";
trackcount=audiocount;
for (var k=0;k<trackcount;k++)
{
Afk=(x[i].getElementsByTagName("aformat")[0].getElementsByTagName("track")[k].childNodes[0].nodeValue);
Abm=(x[i].getElementsByTagName("abitratem")[0].getElementsByTagName("track")[k].childNodes[0].nodeValue);
Abi=(x[i].getElementsByTagName("abitrate")[0].getElementsByTagName("track")[k].childNodes[0].nodeValue);
Asp=(x[i].getElementsByTagName("language")[0].getElementsByTagName("track")[k].childNodes[0].nodeValue);
Ach=(x[i].getElementsByTagName("channels")[0].getElementsByTagName("track")[k].childNodes[0].nodeValue);
Audio=Audio+"<tr><td rowspan=3 valign=middle>Track "+(k+1)+"</td><td>Format: "+Afk+", "+Ach+"Channels</td></tr><tr><td>Language: "+Asp+"</td></tr><tr><td>Bitrate: "+Abm+" "+Abi+"</td></tr>";
};
}

//modify name for wikipedia search and save modified name to varible wikiname
wikiname=(x[i].getElementsByTagName("name")[0].childNodes[0].nodeValue);
var wikiname = wikiname.replace(".avi", "");
var wikiname = wikiname.replace(".mkv", "");
var wikiname = wikiname.replace("1von2", "");
var wikiname = wikiname.replace("1von3", "");
var wikiname = wikiname.replace("2von3", "");
var wikiname = wikiname.replace("2von2", "");
var wikiname = wikiname.replace("3von3", "");
var wikiname = wikiname.replace("1v2", "");
var wikiname = wikiname.replace("2v2", "");
var wikiname = wikiname.replace("1v3", "");
var wikiname = wikiname.replace("2v3", "");
var wikiname = wikiname.replace("3v3", "");
var wikiname = wikiname.replace("CD1", "");
var wikiname = wikiname.replace("CD2", "");
var wikiname = wikiname.replace("CD3", "");
var wikiname = wikiname.replace("CD", "");
var wikiname = wikiname.replace("Ac3", "");
var wikiname = wikiname.replace("AC3", "");
var wikiname = wikiname.replace("Dolby", "");
var wikiname = wikiname.replace("Deutsch", "");
var wikiname = wikiname.replace("Deu", "");
var wikiname = wikiname.replace("German", "");
var wikiname = wikiname.replace("English", "");
var wikiname = wikiname.replace("Englisch", "");
var wikiname = wikiname.replace("Engl", "");
var wikiname = wikiname.replace("Spielfilm", "");
var wikiname = wikiname.replace("Kömmodie", "");
var wikiname = wikiname.replace("Frankreich", "");
var wikiname = wikiname.replace("Deutschland", "");
var wikiname = wikiname.replace("USA", "");
var wikiname = wikiname.replace(/_/g, "+");
var wikiname = wikiname.replace(/ /g, "+");
var wikiname = wikiname.replace(/-/g, "+");
var wikiname = wikiname.replace(/\(.+\)/g, "");
wiki="<iframe width=100% height=100% src=http://de.wikipedia.7val.com/de/wiki?search="+wikiname+"></iframe>";
var searchurl="http://de.wikipedia.org/wiki?search=";
var wikiurl =searchurl.concat(wikiname);
document.getElementById("showWiki").innerHTML=wiki;


// store all obtained info to variable txt
txt="<table width=95% class='sample'><tr class='d0'><td colspan=2><b>General</b></td></tr><tr><td>Name:</td><td><a href="+wikiurl+" target=_blank>"+Name+"</a></td></tr><tr><td width=15%>Library:</td><td>"+Library+"</td></tr><tr><td>Path:</td><td>"+Path+"</td></tr><tr><td>Filesize:</td><td>"+Size+"</td></tr><tr><td>Duration:</td><td>"+Duration+"</td></tr><tr><td>Filetype:</td><td>"+Gformat+"</td></tr><tr class='d0'><td colspan=2><b>Video</b></td></tr><tr><td>Resolution:</td><td>"+Width+"x"+Height+", "+Fps+" FPS"+"</td></tr><tr><td>Aspect:</td><td>"+Aspect+"</td></tr><tr><td>Videocodec:</td><td>"+Vformat+"</td></tr><tr><td>Videobitrate:</td><td>"+Vbitrate+"</td></tr><tr class='d0'><td colspan=2><b>Audio</b></td></tr><td>Audiotracks</td><td>Count: "+audiocount+"</td></tr>"+Audio ;
// store all obtained info to variable txt
document.getElementById("showFilm").innerHTML=txt; //show variale txt in css element showFilm

}

function searchIndex() { // search the index on button press
	// get the search term from a form field with id 'searchme'
	var searchterm = document.getElementById("searchme").value;
	results = new Array;
	if (searchterm.length < 3) {
		alert("Searchterm needs at least three characters.");
	} else {
		for (var i=0;i<x.length;i++) {
// see if the XML entry matches the search term,
// and (if so) store it in an array
			var name = (x[i].getElementsByTagName("name")[0].childNodes[0].nodeValue);
			var exp = new RegExp(searchterm,"i");
			if ( name.match(exp) != null) {
				results.push(x[i]);
			}
		}
// send the results to another function that displays them to the user
	showResults(results, searchterm);
	}
}

function showResults(results, searchterm) {
	// if there are any results, write them to a table 
	if (results.length > 0) {
		// sort first
		results.sort(function(a, b){
		 var nameA=a.getElementsByTagName("name")[0].childNodes[0].nodeValue, nameB=b.getElementsByTagName("name")[0].childNodes[0].nodeValue
		 if (nameA < nameB) //sort string ascending
		  return -1
		 if (nameA > nameB)
		  return 1
		 return 0 //default return value (no sorting)
		})
		//reorder films alphabetically
		for(var i=results.length-1;i>=0;i--)
		{
		  results[i]
		   .parentNode
		     .insertBefore(results[i],
		                  results[i].parentNode.getElementsByTagName("film")[0]);
		}
		//End sorting, now show results
		var searchresult="";
		for(var i=0; i<results.length; i++) {
		// document.write('<tr>');
		 var name = results[i].getElementsByTagName("name")[0].childNodes[0].nodeValue;
		 var library = results[i].getElementsByTagName("library")[0].childNodes[0].nodeValue;
		 //Audioinfo:
		 var audiocount=(results[i].getElementsByTagName("acount")[0].childNodes[0].nodeValue);
		 if (audiocount=="unknown")
		 {
		 var Language="<td>Language: unknown</td>";
		 }
		 else
		 {
		 //Loop the audiotracks
		 var Language="";
		 var trackcount=audiocount;
		 for (var k=0;k<trackcount;k++)
		 {
		 Asp=(results[i].getElementsByTagName("language")[0].getElementsByTagName("track")[k].childNodes[0].nodeValue);
		if (k<1) { //Erster Durchgang noch keinen Slash voranstellen
		Language=Language+Asp;
 		}
		else {
		Language=Language+" - "+Asp; 
		}
		};
		 }
		//in variable searchresult we'll store the html code for the first lineto display
		searchresult=searchresult+"<tr onclick='displayFilmInfo(" + i + ")'><td>"+name+"</td><td>"+library+"</td><td>"+Language+"</td></tr>"
		}
		//in variable suchoutput we'll store the complete html code
		var suchoutput='<br>You searched for <b><i>'+searchterm+'</i></b><br><br><table border="0" width=99%><tr><td><b>Filmname</b></td><td><b>library</b></td><td><b>Language</b></td>'+searchresult;
		document.getElementById("showList").innerHTML=suchoutput; //show varible txt defined above
				
	} else {
		// report if no results
		var nixfind='<br>You searched for <b><i>'+searchterm+'</i></b><br><br>No results for '+searchterm+'!';
		document.getElementById("showList").innerHTML=nixfind
		//document.close();
	}
}
</script>
</head>

<body onLoad="searchOnload()">
<div id='showFilm'>Klick a film name for more infos.</div>
<div id='showList' style="overflow:true;">
<div id='showWiki'>Here wikipedia info will be shown for a film.</div>
<script type="text/javascript">
document.write("<table border='0'>");
for (var i=0;i<x.length;i++) //loop to display the left list
  { 
  document.write("<tr onclick='displayFilmInfo(" + i + ")'>");
  document.write("<td>");
  document.write(x[i].getElementsByTagName("name")[0].childNodes[0].nodeValue);
  document.write("</td><td>");
}
document.write("</table>");
</script>
</div>
<div id='suchmaske'>
<form action=""><b>Search:  </b><input id="searchme" type="text" size="20">  <input value="Start search!" onclick="searchIndex(); return false;" type="submit"></form><input type=button onClick="location.href='catalog.html'" value='Entire List'> <input type=button onClick="location.href='catalog.html'" value='New Search'> 
</div>
</body>
</html>


Searching XML from within MythWeb

To search from within mythweb, you need to modifiy a single php file. For RHEL6 / CentOS6 / SientificLinux6 with the default mythweb theme, this is /var/www/html/mythweb/modules/tv/tmpl/default/detail.php

There at line 400 starts the default search options for IMDB, TV.com, etc This is a snip of lines 400-411 of the named file before modification.

<?php           if ($schedule->title) { ?>
                <a href="http://www.themoviedb.org/search/movies?search%5btext%5d=<?php echo urlencode($schedule->title) ?>"><?php echo t('Search $1', 'themoviedb') ?></a>
                <a href="http://www.imdb.com/search/title?title=<?php echo urlencode($schedule->title) ?>"><?php echo t('Search $1', 'IMDB') ?></a>
                <a href="http://www.thetvdb.com/?string=<?php echo urlencode($schedule->title) ?>&searchseriesid=&tab=listseries&function=Search"><?php echo t('Search $1', 'TheTVDB') ?></a>
                <a href="http://www.tv.com/search.php?type=11&stype=all&qs=<?php echo urlencode($schedule->title) ?>"><?php echo t('Search $1', 'TV.com') ?></a>
                <a href="http://www.google.com/search?q=<?php echo urlencode($schedule->title) ?>"><?php echo t('Search $1', 'Google') ?></a>
                <a href="<?php echo root_url ?>tv/search/<?php echo str_replace('%2F', '/', rawurlencode('^'.$schedule->title.'$')) ?>?field=title"><?php
                    if ($_GET['recordid'])
                        echo t('Find showings of this program');
                    else
                        echo t('Find other showings of this program');
                ?></a>

I modiefied the first search entry (I never search themoviedb). So line 401 became:

<a href="http://url/to/your/webserver/catalog.html?x=<?php echo urlencode($schedule->title) ?>"><?php echo t('Search $1', '<font color="red">video file catalog</font>') ?></a>

This modification is probably not the most elagant way. I however have not found a setting to customize these search entries. I suggest you backup detail.php before and after editing - updates from your linux distro might overwrite it.