[mythtv-users] Auto-expire extra space setting and slow deletes

[Michael T. Dean]

On 04/02/2009 10:02 AM, Boleslaw Ciesielski wrote:
> Tom Dexter wrote:
>> I'm trying to get an idea of what extra space setting I should use
>> that, in theory, would allow a "full" system to continue without
>> problems. I guess part of that depends on the actual speed of slow
>> deletes.  I don't believe I've ever read exactly what that is, if
>> someone knows offhand.
> The slow delete speed is calculated to be at least 20% faster than 
> your estimated max recording rate. See MainServer::TruncateAndClose in 
> programs/mythbackend/mainserver.cpp for details.

Yeah, the slowest slow delete is about 2.75MiB/half second 
(5.5MiB/second).  This works out to about 1GiB/3min or 19.5GiB/hr.  
(And, yes, these are MiB/GiB, not MB/GB--they're binary 
mebibytes/gibibytes.)

If you really want to see what your backends are using, run them at:

mythbackend -v important,general,file

for a while (I'd recommend you do so until the first file is deleted--as 
the -file logging is very verbose) and you'll see lines, "Truncating 
<filename> by <amount> MB every <increment> milliseconds."  So, I'd 
recommend shutting down your backends when they're not busy, then 
starting one or all of them and deleting a file*** until you see the 
line specified in the logs, then you can shut down the backends**** and 
restart them with normal/default logging options.

***One file owned by each system whose rate you want to see, that is.

****Even if they're not done deleting that file, yet, as shutting down a 
backend that's busy deleting a file will delete the file completely 
using "fast deletes".  Note, though, that doing so may make shutting 
down the backend take a bit longer than usual (as it won't appear to 
have shut down until the file(s) are deleted).

> Also, I believe that nothing will get auto-expired as long as slow 
> deletes are still in progress. At least that's how it used to be, it 
> may have changed. I am not sure how this works across multiple backends. 

Right, but--as you were trying to imply--that won't hurt anything from a 
filesystem filling up perspective since Myth is still deleting the 
manually deleted shows faster than the recordings can progress and it 
has the benefit that it means Myth won't unnecessarily put shows in the 
queue to autoexpire when the shows it's already deleting will make 
enough room.

The one thing that /can/ cause an issue is the frequency at which 
autoexpire checks are performed.  Generally, Myth will check once every 
15 minutes to see whether autoexpiration needs to be performed, but it 
may check as often as once every 3 minutes (see your backend 
log--assuming default verbosity, or at least -v important--for 
"AutoExpire: CalcParams(): Max required Free Space" lines which give the 
frequency).  If Myth is only checking once every 15 minutes and some 
non-Myth process (i.e. such as MySQL with binary logging enabled) throws 
a lot of data on the filesystem during the time between checks, it's 
possible for the drive to fill up and the system/Myth/whatever to crash.

Note, also, that the "Extra Disk Space" setting is global--across all 
backends /and/ all filesystems of all directories defined in all Storage 
Groups.  See 
http://www.gossamer-threads.com/lists/mythtv/users/361991#361991 for more.

Mike