[mythtv] Brainstorming a new idea for the program guide

[Robert Johnston]

On Mon, Sep 21, 2009 at 16:28, Rob Smith <kormoc at gmail.com> wrote:
> On Mon, Sep 21, 2009 at 2:59 PM, Robert Johnston <anaerin at gmail.com> wrote:
>> It could also be possible, if TVDB are willing...
>
> So, as you just said here, TVDB is not Myth, nor are we a part of
> TVDB. We have no control over their code, have no idea if their
> servers could handle it or not (and I'd venture to guess (yes guess)
> that they couldn't).
>
> All this is entirely meaningless on here until someone contacts TVDB
> and asks them permission to increase their (already overloaded)
> servers by a few thousand-fold.
>
> It's not even that your "Cache preload burst" would save that much
> over just a pipe-lined datadump, as the zipping wouldn't actually
> compress that well (these images are already compressed), so you're
> eating up a lot of cpu time, and then you can't really cache the
> results, as everyone's lineups are different. For example, I have
> FIOS, but I have all my sports and shopping channels out of my
> line-up, along with a majority of news channels. But not all of them.
> That's a (semi)unique setup for me, with a (semi)unique pile of
> images. It's a lot of work for very little gain.
>
> Also keep in mind how many users, let's do some math
>
> In the past month (Aug 21 to Sept 20th), we had 13,000 downloads of
> 0.21 from mythtv.org. Let's assume that 10,000 of them actually used
> the software and they have 5,000 shows in their guide data, at 5kb per
> show. (My guide listing has 5954 shows fyi, SELECT COUNT(DISTINCT
> TITLE) FROM program ).
>
> 10000 * 5000 * (5 kilobytes) = 238.418579 gigabytes
>
> and for the current 500kb art downloads
> 10000 * 5000 * (500 kilobytes) = 23.2830644 terabytes
>
> Just for the last month of downloads, from mythtv.org. Not counting
> the ones via package managers, shipped with the os, etc.
>
> That's a lot of data, and the potential is much much more. Sure,
> there's plenty of ways they might be able to change to make it work,
> but you'll need to talk with *them* to see if it could work and not
> argue with us about how trivial it is/could be/if they're even willing
> to allow us to do this.

There are some things to remember here.

1. Not every show will be eligible for images/data.

Here's the number I came up with: 2588 shows (
   SELECT COUNT( DISTINCT TITLE )
   FROM `program`
   WHERE category_type != 'movie'
   AND category != 'Paid Programming'
   AND category != ''
)
down from 3235 shows (
   SELECT COUNT(DISTINCT TITLE)
   FROM program
)
That's on 71 channels of programming.

2. That presumes that every single show listed will have artwork for
it. This will most likely not be the case.

3. This is only the first download. Subsequent fetches will have
little of the data needed, as it will be cached client-side. The only
issue will be when the new TV seasons launch, and new programmes start
appearing.

4. P2P Technologies (libtorrent, for example) could be integrated to
make Myth "Play nice", sharing the data around, between
frontends/backends and systems, and taking a lot of the load off of
TVDB (Which I'm sure they'd appreciate).

So things aren't near as dire as they look at first glance.
-- 
Robert "Anaerin" Johnston
Sent from Regina, Saskatchewan, Canada