[mythtv] [mythtv-commits] Ticket #2782: automatic simultaneous jobs scaling
Michael T. Dean
mtdean at thirdcontact.com
Sat Dec 9 03:17:55 UTC 2006
On 12/08/2006 02:29 AM, osma ahvenlampi wrote:
> On 12/8/06, Michael T. Dean <mtdean at thirdcontact.com> wrote:
>> The idle value /never/ changes on any of the hosts, thanks to a little
>> program I call AthWarm (most call SETI/BOINC). :)
> Ah, I didn't consider someone wanting to run their HTPC hot - I've
> been more concerned about running it quiet. How would you propose
> detecting between time spent in SETI vs time spent by jobs? The kernel
> won't distinguish between the niced processes..
Yeah, my 2 backends and 1 frontend are not in the media room.
Therefore, noise from running hot has no effect on my viewing
experience. I like my media room clean--one 67" HDTV, speakers, and
places to sit--no ugly A/V receivers, STB's, PC's, tangle o' cables, ...
>> No problem, though, I wouldn't mind disabling job scaling. (Now,
>> disabling BOINC, OTOH, will not happen :).
> I didn't make this configurable, because I would have liked to make it
> "work right" without configuring. That can be done though, but I'd
> rather figure out a better solution.
Not a problem for me--I keep my own patch tree. ;)
> At least one change would need to happen for this to become
> commit-worthy, I guess: right now jobqueue will not start *any*
> processes if there's no idle time, but it will start them if it can't
> read idle time at all. I'll change it so that it will start *one*
> process in either of those situations, but scale up from there only if
> there's still time available. Would that be satisfactory?
Yep. With that change, I wouldn't even bother disabling the scaling. I
run with a max of 1 job on all my hosts because of the I/O issues that
Daniel mentioned. So, if it were allowed to start one job even at 0%
idle (i.e. with 99% nice), it would be the same as I have currently.
Basically, I prefer running on the safe side. After all, I have > 480
hours of TV (including commercials) to catch up on, so I /never/ have to
watch something "as soon as possible" after it airs--even waiting until
the next day to give commflagging plenty of time to finish isn't a problem.*
Perhaps a more "general-purpose" approach that would work for others,
too, would be detecting the percentage of CPU time spent on idle plus
the CPU time spent on processes running at a higher nice level than the
job. IIRC, jobs are run at nice 17, 10 or 0 depending on the value of
JobQueueCPU, so any job running "nicer" than the nice setting for the
job queue should be treated as idle time (i.e. time that could be used
for the job queue). (This assumes that we'll still have a JobQueueCPU
setting after the patch--I haven't looked at the patch, yet.) I run
BOINC at nice 19--and expect most people running "hot" do so with
similarly-low-priority tasks. Any process running at higher priority
than the job queue jobs should probably be treated as more important,
anyway, so this seems like a good way to decide whether a new job is
I don't know if doing this would require parsing every /proc/*/stat to
retrieve the priority/nice value(s) (fields 18, 19, and/or 38) and the
per-process CPU usage. If so, it's probably not worth the effort, but
it's the thought that counts, right?
*In fact, I have my Myth box record all new series every season. Then,
as networks cancel some of the new series, I delete them without
watching them. If "the word on the street" says the show isn't worth my
time, I delete the series without watching it. If everyone's talking
about the show--and saying good things about it--I'll actually watch
it. So, generally, I don't watch start to watch shows until they're at
least a couple months old. Then, I'm far enough behind "real-time" that
I don't catch up until after the season finishes.
More information about the mythtv-dev