[mythtv-users] Duplicate recordings because of bad SD data?

f-myth-users at media.mit.edu f-myth-users at media.mit.edu
Thu Jul 22 00:17:08 UTC 2010


    > Date: Wed, 21 Jul 2010 18:14:56 -0500
    > From: Robert Eden <rmeden at yahoo.com>

    > On 7/21/2010 5:18 PM, f-myth-users at media.mit.edu wrote:
    > >     > When I spoke to TMS about this, new Episodes-IDs are also generated for 
    > >     > significant program edits.  For example, "The Apprentice" would show one 
    > >     > show on NBC and then an "extended version" on CNBC.  Both would get 
    > >     > different Episode-IDs.  Sometimes the edits are not apparent to us, but 
    > >     > Tribune knows all. :)
    > >
    > > Well, the cases I've seen, I've typically compared the CC data 1:1.
    > >   
    > Let me rephrase... TMS *IS LEAD TO BELIEVE* that the show *COULD* have 
    > different content.  I don't know if that's what's really going on, just 
    > a guess.  If you can provide a specific example, I'll ask them.  It will 
    > be interesting to hear.

Ah!

Okay, here's one I just tripped over today, again on "How Do They Do It?",
which airs on the Science Channel.  The one that aired today is:

EP008159770162  How Do They Do It?      Produce Power from Waste; Customize Supercars; Make Super Steel Swords

That field to the right is its actual subtitle.  Its description is blank.

One that aired on July 24, 2009 didn't have any subtitle---just the main title.
Its programid was EP008159770127, but it had a nonblank description of:
"Waste to power; Brabus Supercar; Damascus Steel."

Both have seriesid EP00815977.  Clearly those are intended to be the same episode.

The XML I have for the showing today is:

<program id='EP008159770162'>
<title>How Do They Do It?</title>
<subtitle>Produce Power from Waste; Customize Supercars; Make Super Steel Swords</subtitle>
<showType>Series</showType>
<series>EP00815977</series>
<originalAirDate>2010-07-21</originalAirDate>
</program>

and I -ALSO- have this in the very same download:

<program id='EP008159770127'>
<title>How Do They Do It?</title>
<description>Waste to power; Brabus Supercar; Damascus Steel.</description>
<showType>Series</showType>
<series>EP00815977</series>
<syndicatedEpisodeNumber>602</syndicatedEpisodeNumber>
<originalAirDate>2009-07-23</originalAirDate>
</program>

Yes, these are both in my listings simultaneously.  The one with the
programid that I recorded last year was correctly marked a repeat and
not recorded; the one with the programid I hadn't seen before was just
recorded a couple hours ago.  Note also that one of them has a
syndicatedEpisodeNumber and one of them doesn't.

Since it might be useful for debugging, I have arranged to redundantly
record the one that Myth thought was a repeat before today (the "waste
to power" one, EP008159770127), because it's airing on July 28th.  We
could see if that one has identical captions to the one I recorded
last year; see below.

The XML for the showing last year is:

<program id='EP008159770127'>
<title>How Do They Do It?</title>
<description>Waste to power; Brabus Supercar; Damascus Steel.</description>
<showType>Series</showType>
<series>EP00815977</series>
<originalAirDate>2009-07-23</originalAirDate>
</program>

...which matches the one that Myth correctly concluded was a repeat in
my current listings, though this one is missing syndicatedEpisodeNumber.

So clearly -BOTH- descriptions exist, and I can currently see both
alternatives (subtitle but no description vs no subtitle but description)
in MythWeb's Search page.

The software running last year and this year is absolutely identical;
so is the hardware and the signal sources.  I have only one source,
standard-def digital cable, feeding STBs going to PVR-250s.

The CC data for the two programs has an overlap of 65%, which is
normally crappy when I compare two showings of something; a true
repeat should be up in the 90's modulo commercials and padding at the
beginning and end of the recording.  This overlap is crappy because
the CC data for the two showings has small differences throughout.
(If anyone would like to analyze them completely, I can send them;
each is about 20K.)  If this sort of retitling is what TMS considers
a significant difference, fine, but I've also seen cases where the
overlap is 95% and yet programid's differ.

[My percentages come from a filter that, essentially, strips all
linebreaks, then inserts a newline at each space, then does
"diff -iwBE -d --side-by-side --left-column | egrep -c '\($'"
(call this value "count") on them, and then takes the ratio of count
over the sum of "wc -w" on each of the inputs so as to normalize by
length.  [Yes, that -d is absolutely critical but is out of scope
here.]  Think of this as a poor man's dot-product of the words in the
files, with the order preserved.  It was a fast kluge that turned out
to be good enough.  If this is important I can go look at the code;
I'm quoting from memory and I wrote the code a at least a couple years
ago.]

Today:

>> Narrator: COMING UP ON
"HOW DO THEY DO IT?"...
HOW DO THEY TURN THE MESS FROM
THE WORLD'S BIGGEST
ST. PATRICK'S DAY PARADE INTO
MEGAWATTS OF ELECTRICITY TO
POWER NEW YORK?
HOW DO THEY MAKE THE WORLD'S
BEST CARS EVEN BETTER,
TRANSFORMING A LUXURY SEDAN INTO
A TIRE-BURNING RACER?
AND HOW DO THEY TURN ORDINARY
STEEL INTO SUPER STEEL TO CREATE
THE ULTIMATE BLADE FOR
21st-CENTURY SAMURAI?
WE SHOW YOU HOW ON
"HOW DO THEY DO IT?"
--Captions by VITA -&www.vitac.com
CAPTIONS PAID FOR BY DISCOVERY COMMUNICATIONS
[ BAGPIPES PLAY ]
[ CHEERS AND APPLAUSE ]
ST. PATRICK'S DAY, NEW YORK.
OVER 200,000 PEOPLE LINE
5th AVENUE FOR WHAT NEW YORKERS
CLAIM IS THE BIGGEST CELEBRATION
OF IRISH CULTURE ANYWHERE IN THE
WORLD, EVEN IRELAND.
AND LIKE ANY OTHER BIG CROWD,
THEY MAKE A BIG MESS.
BUT BY THIS TIME TOMORROW, THE
STREETS WILL BE CLEAN AND
50 TONS OF PARADE GARBAGE WILL
BE ELECTRICITY POWERING
5th AVENUE AND BEYOND.
SO, HOW DO THEY DO IT?
NEW YORK -- THE MOST HEAVILY
POPULATED CITY IN THE USA.
OVER 8 MILLION INHABITANTS
CONSUME THE ENERGY EQUIVALENT OF
280 MILLION BARRELS OF OIL PER
YEAR.
THANKFULLY, THEY ALSO HELP
GENERATE IT WITH THEIR TRASH --
OVER 13,000 TONS OF IT EVERY
DAY.
AND ON ST. PATRICK'S DAY,
THERE'S EVEN MORE.
[ PERCUSSION PLAYS ]
[ CHEERS ]
[ BAGPIPES PLAY ]
BUT WHEN HALF A MILLION PEOPLE
GET TOGETHER TO PARTY, PAINT THE
TOWN GREEN, AND ENJOY A TASTY
PINT...
[ CHEERS AND APPLAUSE ]
THE LAST THING ON THEIR MIND IS
FINDING THE NEAREST GARBAGE CAN.
THE PROBLEM OF PARADE GARBAGE IS
COLOSSAL, BUT FEAR NOT, BECAUSE
BRINGING UP THE REAR IS
NEW YORK CITY'S DEPARTMENT OF
SANITATION.
THIS IS THEIR TURF.
THE 7,900 SANITATION WORKERS AND
THEIR 6,000 VEHICLES ARE A
MECHANIZED ARMY AT WAR WITH
FILTH.
AND ST. PATRICK'S DAY IS THEIR
"D" DAY.

Last year:

>> Narrator: HOW COULD THEY TURN
THE MEGAWATTS TO POWER NEW YORK?
HOW DO THEY MAKE THE WORLD'S
BEST CARS EVEN BETTER,
TRANSFORMING A LUXURY SEDAN INTO
A TIRE BURNING RACER?
AND HOW DO THEY TURN ORDINARY
STEEL INTO SUPER STEEL TO CREATE
THE ULTIMATE BLADE FOR 21st
CENTURY SAMURAIS.
WE SHOW YOU HOW ON HOW DO THEY
DO IT.
CLOSED CAPTIONING PROVIDED BY
DISCOVERY COMMUNICATIONS

>> Narrator: ST. PATRICK'S DAY,
NEW YORK.
THEY ARE LINING THE AVENUE FOR
WHAT THEY CLAIM IS THE BIGGEST
COLLECTION OF IRISH CULTURE
ANYWHERE.
LIKE ANY OTHER BIG CROWD, THEY
MAKE A BIG MESS.
BY THIS TIME TOMORROW, THE
STREETS WILL BE CLEAN AND 50
TONS OF PARADE GARBAGE WILL BE
ELECTRICITY POWERING FIFTH
AVENUE AND BEYOND.
SO HOW DO THEY DO IT?
NEW YORK.
THE MOST HEAVILY POPULATED CITY
IN THE USA.
OVER EIGHT MILLION INHABITANTS
CONSUME THE ENERGY EQUIVALENT OF
280 MILLION BARRELS OF OIL PER
YEAR.
THANKFULLY, THEY ALSO HELP
GENERATE IT WITH THEIR TRASH.
OVER 13,000 TONS OF IT EVERY
DAY.
AND ON ST. PATRICK'S DAY, THERE
IS EVEN MORE.
BUT WHEN HALF A MILLION PEOPLE
GET TOGETHER TO PARTY, TO PAINT
THE TOWN GREEN AND ENJOY A TASTY
PINT, THE LAST THING ON THEIR
MIND IS FINDING THE NEAREST
GARBAGE CAN.
THE PROBLEM OF PARADE GARBAGE IS
COLOSSAL.
BUT FEAR NOT.
BECAUSE BRINGING UP THE REAR IS
NEW YORK CITY'S DEPARTMENT OF
SANITATION.
THIS IS THEIR TURF.
THE 7900 SANITATION WORKERS AND
THEIR 6,000 VEHICLES ARE A
MECHANIZED ARMY AT WAR WITH
FILTH.
AND ST. PATRICK'S DAY IS THEIR
D-DAY.
LONG AFTER THE PARTY'S MOVE TO
MANHATTAN'S 158 IRISH BARS,
THEY'RE STILL SWEEPING, STILL
LOADING AND STILL DRIVING.
THE BIG QUESTION IS WHAT TO DO
WITH ALMOST 55 TONS OF GARBAGE.
THE ANSWER THE S TO TAKE IT TEN
MILE DOWN THE ROAD AND FEED IT


More information about the mythtv-users mailing list