[mythtv-users] Import and Export of Program Listing Data

Robert Current robert.current at gmail.com
Thu Sep 6 05:03:52 UTC 2007


Hi Dean from Down Under!  I hope you don't mind to much, I hijacked
your comment and re-titled it with the discussion that might be more
on topic now?

> Dean Harding dean.harding at dload.com.au

> I just thought I'd point out some stuff about what we do in Australia with
> regards to scheduling information.
>
> There are basically two common sources of XMLTV data in Australia. One is
> icetv.com.au who charge $13 a month (or less if you subscribe for 6 months
> or 12 months at a time).
>
> The other is http://minnie.tuhs.org/twiki/bin/view/TVGuide/ which is a
> community-run source of data. Basically, people collect the data manually
> and upload it. You need to sign up for an account, but it's free. The data
> is free of copyright because people are entering it manually (presumably...)

Presumably.  Hehehehe, ok.  I'll by that for a dime if you throw in
two nickels for the price.

But more importantly, it works, and hasn't been hit with legal action?
 I thought I read about some data source in Australia recently beating
some legal action, but I didn't pay much attention then, did that have
to do with this project?

> Before that second source existed, there used to be a bunch of scripts that
> you could get which scraped various websites to get the data (as I
> understand it, that's what you're proposing?). The problem with scrapers,
> though, was that they were always extremely fragile -- the websites would
> change the HTML they output every now and then (presumably once they
> realised lots of scrapers were running) and the scripts would break and
> someone would have to fix them. Now, it generally wasn't HARD to fix, but
> they'd break *all the time*.


Now, all those scripts your talking about, did that have to do with
this Sheppard project?  Or is that something else?

And, my final question is, how do you use this data in Myth?  Is there
a GUI front end somewhere?  Is it a "Direct Data" option that someone
hacked into the MythTV-Setup screens?  Do you import it from xmltv?
If so, using an automated script to pull it and import it weekly?


> Now, I haven't followed this thread all the way through, but I understand
> you're suggesting some sort of web scraper for schedules data? My suggestion
> would be rather than do that, set up something like the Aussie
> community-supported site. If lots of people want free guide data (and, in
> Australia, people like free EVERYTHING :p) you'll get plenty of people
> signing up.


Well, it seems, IMHO, people won't do that here because they are a bit
to chicken to get involved, fearing they will have their door kicked
in and equipment confiscated or some such paranoia.

I'm not sure the suggestion was scrapers…  But the question was what
free data sources can one find.  The data sources, scrapers, and
projects have been found, and instantly turned in to a red-hot debate
about legality.  So, the next question was how to distribute and
import the data.

Since the legal debate will all but completely scare anyone from
hosting such a central data source, the topic of file sharing by way
of UseNet or torrent or decentralized methods came up.

Obviously, decentralized methods like UseNet, torrents, etc.. have
drawbacks in efficiency.  But, IMHO, they are the ONLY way around this
issue unless someone has the balls, time, and money to open up a
centralized server in the US like you mentioned in Australia.  You
might find the money and time, but doubtful anyone has the balls to do
it.

But, once the other two things are clearer (import, export), and a
distribution system comes about, you might find a LOT of people in the
US willing to input "copyright free manually entered data."  ;-)


More information about the mythtv-users mailing list