[mythtv] Ticket #2364: Increase sleep_time in truncate-delete patch to accomodate remote mounts

[Michael T. Dean]

On 09/27/06 15:17, MythTV wrote:

>#2364: Increase sleep_time in truncate-delete patch to accomodate remote mounts
>
> I am pleased to report that, with Mike Dean's patch, truncated deletes
> work again on my 0.20 (ATrpms) system ...
>  
>
Thanks for testing.

> ... the only outstanding issue is 3), having the cifsxxxx stub
> sticking around.
>  
>
Well, Chris responded with the short answer before I got around to 
sending this, but since I've already written it up...

You may already know this, but here's a little background, just in 
case.  The truncate-delete code is using a very oft used approach on 
*nix systems typically called "delete on last close" to ensure that even 
in the event of a crash, the deleted file will be deleted--even if we 
hadn't finished truncating it, yet (this is often used for temp files by 
other applications).  With this approach, the file is actually deleted 
before we begin truncating it, but because Myth had opened the file 
before deleting it, it's not actually removed from the filesystem until 
Myth closes the file (or until Myth is shut down or crashes).  If 
everything works as designed, the file is small (less than the truncate 
increment--often 4MiB) at the time it's closed, so deleting it takes 
very little I/O activity--even on filesystems with poor delete performance.

Some network filesystems, such as CIFS and NFS, because they're 
stateless, cannot support "delete on last close."  However, because the 
approach is so commonly used in *nix applications and because 
applications should see all filesystems on a *nix machine the same way, 
clients for these filesystems "fake" this behavior by renaming the file 
(to the .cifs* or .nfs* name you've seen) instead of deleting it if the 
file is open at the time of the delete request.  (Believe it or not, 
this is actually called a "silly rename.")  Because the new filename 
starts with ".", it's a "hidden" file, so it's not generally visible, 
but it's just a normal file to the CIFS/NFS client.  Note, also, that 
the server has no idea that the file is being silly renamed--it's just a 
rename to the server.

So, since the CIFS/NFS client (i.e. in this case the cifs vfs kernel 
module you're probably using) fakes the delete on last close behavior by 
renaming the file, it's the responsibility of the CIFS/NFS client to 
remove the file when it's closed.  TTBOMK, the client need not guarantee 
that the file is removed immediately upon close, but is responsible for 
deleting the file at some point after the close.

So, the only way that the non-removal of these could be caused by Myth 
is if Myth keeps an open filehandle.  If that were the case, then even 
on stateful filesystems the files wouldn't be removed until Myth is shut 
down.  This doesn't seem to be the case--files on my (local) filesystem 
are actually deleted upon close after truncation (I don't have any 
remote filesystems for my Myth recordings).  And, I looked at the code 
and didn't see any locations where the file was accidentally left open, 
so I'm pretty certain it's not Myth's fault.

Therefore, if there's an issue, it's in the CIFS/NFS client.  However, 
it's quite possible that given time those files will actually get 
cleaned up; or they may be cleaned up when the filesystem is unmounted 
or--worst case--when the server is recycled (i.e. the NAS gets 
rebooted?).  Do a search on "silly rename" (on Google or somewhere 
besides the MythTV mailing list archive--this is probably the first post 
on Myth's lists to use the term ;) and you'll see a lot of discussions 
about the idea and even about cleaning up silly renamed files.

> In conclusion, my sincerest thanks to Mike and to the other contributors
> in getting this issue resolved. 
>
Thanks for taking the time to get the backtrace.  Would have been pretty 
much impossible to figure out without it as the CIFS drivers act very 
differently from *nix filesystem drivers.

Mike