[mythtv] Ticket #2194: Patch to randomise mythfilldatabase times

[Chris Pinkham]

* On Fri Sep 15, 2006 at 05:34:40PM +1000, William Uther wrote:
> >  longEnough = ((period * oneday) - oneday/2);
> 
> Yes.  It does have that side-effect.  This is because the old logic  
> was "don't run more frequently than every 24 hours", NOT "run once  
> per calendar day".

I'm wondering if there should be another check put into place to
make sure that it doesn't run more than once in a calendar day.  With
the current patch (including your attached mods), it looks like that
is still possible, or am I misreading it?

> I was after an effect that was something like "grab once per day, at  
> a random time".  If I left the old logic, then when the system  
> grabbed late one day it would be forced to grab late the next day too  
> (else it would have grabbed more than once in 24 contiguous hours).   
> The only way it could get back to grabbing early would be to ratchet  
> back in 5 minute intervals, or skip a day entirely.

Yeah, I understand this and think it is a good improvement.  The
-600 on the longEnough calculation was supposed to help work around
that, but it basically would mean that most times we'd run at the same
time every day.

> I decided that I did't want to fill the database too frequently, so I  
> allow it to ratchet back in 12 hour increments rather than 5 minute  
> increments.  You could adjust that 6 hour increments if you want.   
> I'd suggest leaving it as 12 hours - it isn't going to ratchet back  
> easily because of the random delays.

We don't want to take the chance on doubling the amount of data
downloaded each day though, and if you set longEnough to 12 hours
then we could be downloading every 12-24 hours so we're somewhere
between 1-2 times per day average.  I think this is easily resolved
though, so it's not a big issue.  If period is 1 then we just need
to check to see if we've already downloaded on today's calendar day,
otherwise we don't 'want to run'.  Your mod to wantToRun affects all
jobs that the housekeeper runs, so any job we want to only run once
a day could have the potential of running twice a day with your
current changes.  This probably isn't a big deal with the other current
things the housekeeper does, but it negates the effect of setting
period = 1 and could have negative impacts for future additions to
the housekeeper.

The current code works fine in your case where you limit the hours a
task can run, but there are other builtin tasks where you can't edit
the min/max.  They are hardcoded at 0/24 and some users may have their
mythfilldatabase min/max set to 0/24 as well.

> There is another issue that I thought about after submitting the  
> patch... the patch currently assumes that your machine is running all  
> the time.  I haven't played with the auto shutdown-startup stuff in  
> myth.  It might not play nice with this patch.

As far as I can tell, this isn't handled by the existing code either.
There are various HOWTO's and Wiki entries telling people how to get
around this.

> Myth is making that check every 5 minutes.  So in your example you  
> have 24*12 = 288 chances to succeed, each of which, individually, has  
> probability 1/150.  The probability you don't run at all is (149/150) 
> ^288 = 0.15.  So you are right in that it is slightly biased.   
> Exactly how biased depends upon the range of time you give it to run.

OK, makes a bit more sense now and a quick test using a perl script
seems to validate.

> >  Can you check the logs to see when your mythfilldatabase runs were

> I checked this before posting the patch.  But here is the data from  
> my current logs.  I set my grabber to run between 2 and 4 pm daily.

Thanks.  This is the kind of patch that would have a bunch of people
screaming if it caused them to miss data but it takes a long time to
test before committing the patch, so seeing your logs helps. :)

> So here is the complete set of fixes - same as before but with a  
> uniform distribution and non-inclusive max hour.

Given my above comment about making sure we don't run twice on
the same day, does the following mod look like it still does what
you're trying to accomplish?

>               if (((now.toTime_t() - lastrun.toTime_t()) > longEnough) &&
                    (now.toString(QString("d")).toInt() !=
                     lastrun.toString(QString("d")).toInt()))
>               {
>                   int hour = now.toString(QString("h")).toInt();
> -                if ((hour >= minhour) && (hour <= maxhour))
> -                    runOK = true;
> +                if ((hour >= minhour) && (hour < maxhour))
> +                {

We only have a chance of running if it has been long enough and
we haven't run yet today.  I think that with this mod, we could use
the following:

    longEnough = (period - 1) * oneday;

The day comparison would keep us from running twice in the same day
and your random mod would run at a random time during the min/max
hour period.

If that makes sense I'll go ahead and get this into SVN.

--
Chris