[mythtv-users] Re: Bleb.org down??

David myth at dgreaves.com
Tue Apr 20 17:04:26 EDT 2004


Rob Willett wrote:

>David,
>
>I have looked at the Showshifter site and they have said that Digiguide won't
>play ball with them at all. They and PureTelly are the only people I know doing
>decent listings that are free(ish). I did look at trying to scrape PureTelly and
>then gave up.
>  
>
Yes, I've heard that.

>In light of this I have now written most of a very simple distribution service.
>I offer this up as a sacrifical lamb...
>  
>
OK - lets see :)
(Send me a private copy if you like )

>It works on a three tier approach. 
>
>Somebody, somewhere must create the master XML data files. This is the bit to
>do... <cough>. I've written software to handle the distribution only. It will
>work with a single server but clearly the more servers we have the wider the
>distribution and the less load we each have to bear. It will also work around
>the world by the user simply choosing a different Tier-2 server list when they
>start. 
>  
>
Have 2 servers (A and B) both of them:
* grab the data we want (tv_grab_uk_rt)
* split it into individual files (see post by Henk on tv_* utils - I 
guess we want tv_split - thanks Henk ;) )
IF we want to be clever then A and B exchange xml files (wget) and do a 
diff (tv_sort sounds like it's our friend here) - if they're unequal 
then another grab takes place and we can implement majority rules or 
repeat until no errors.

>The software works as follows. A master server (bleb.org?) is used to seed a set
>of Tier 2 servers. Seeding is simply copying the XML files off the master server
>to the Tier 2 servers. I have assumed the XML files are appropriate for mythtv.
>  
>
OK this is mirroring - good - wget has options to do this.

>All the software does at the moment is distribute. The Tier 2 servers then make
>all the XML files and the list of Tier 2 servers available to all and sundry.
>Simple and easy, no databases, no hard work. All a tier-2 provider need do is
>daily update their own files from either the master server or from another Tier
>2 provider.
>
Cron, wget --mirror

>They will need a web server and preferably a dedicated directory.
>  
>
They're cheap - the directories that is ;)

>They will not need mySQL, PHP though they will need Perl, ping and wget (so far).
>  
>
just wget

>All the software is written in Perl. It consists of a perl script and a file
>containing the list of Tier-2 servers and which directory the xml files are in.
>This software will randomly choose one of the Tier 2 servers (never the master)
>and try and download the XML software and the server list. If a server is down
>it will choose another server until it either succeeds or runds out of servers. 
>  
>
Only users need this - see my earlier mail.

>At the moment I have the core software written but am struggling a little with
>wget when it isn't pulling back web pages, just files. I want the power of rsync
>but on port 80. I'll crack that hopefully later today and will then send files out.
>  
>
don't spend too much time - perl should be able to pull back easily w/o 
wget .
See LWP - it's not hard :

  # Create a user agent object
  use LWP::UserAgent;
  $ua = LWP::UserAgent->new;
  # Create a request
  my $req = HTTP::Request->new(GET => 'http://www.hmmm.co.uk/xmltv/2004/04/21/sky_one.sky.com.xml');
  # Pass request to the user agent and get a response back
  my $xml = $ua->request($req);
and that's it ;O

>Rather than clogging up the list if there is enough interest I'll setup a
>mailing list for anybody interested in helping out. If you are send me an e-mail
>with to rob.mythtv.devlist.no.spam at robertwillett.com. Clearly remove the
>.no.spam  to get this work.
>  
>
It's already clogged - I think this is valid Myth stuff at the minute 
given Henk and Jeffs unexpected interest ;)



More information about the mythtv-users mailing list