[mythtv-users] pdflush is killing me

[Tony Lill]

"Steven Adeff" <adeffs.mythtv at gmail.com> writes:

> On 2/5/07, Tony Lill <ajlill at ajlc.waterloo.on.ca> wrote:
>> THis worked for me (basicly by making pdflush run more often, but
>> with less work to do).
>>
>> # % of physical memory after which a process doing a write will be paused
>> # untill dirty pages have been written
>> # default 40
>> echo 50 > /proc/sys/vm/dirty_ratio
>>
>> # % of physical memory at which pdflush will be woken. Hopefull this will limit
>> # system pauses for flushes
>> # default 10
>> echo 5 > /proc/sys/vm/dirty_background_ratio
>
> having seen pdflush run while observing top (not knowing if it was
> causing any actual issues...), I've taken an interest in what you've
> written here. Would you mind elaborating on what this does for people
> like me that don't know what that means?

When you write data to a file, it doesn't go directly to the file, the
data is buffered up in memory, and the kernel then tried to find the
"best" time to write it out. Usually this means when the disk head
happens to be near where the data should be written. This reduces seek
time and since a lot of files are small and the data gets read again
soon after it is written, keeping them in memory is a big performance
boost.

This is great for a compiler, because when it creates small temporary
files, turns around and reads them, then deletes them. By caching
this data in memory, there's a good chance that they will never
actually be written to disk.

Then there's myth, which writes a huge (larger than physical memory)
file and maybe doesn't read it for hours.

Modulo other parameters, pdflush will get woken up when
/proc/sys/vm/dirty_background_ratio percent of memory is full of data
to be flushed to disk, and then it will clean it out. If you've got 1G
of memory, that's 100M. 

So if nothing much else is happening and myth is writing at 1M/s to a
new file, and when 100M has accumulated, pdflush wakes up and writes
that out at one shot. That can hog down your system enough to cause
playback to pause or recording to drop frames. By turning it down to
5%, that's only 50M at one shot, so no hiccup. If you have 2G, turn it
down even more.

The fun thing is, you can also cure this behaviour by doing something
like compiling a kernel!

The other value will let your program to use up more memory for write
buffers before a write system call will cause the program to be
suspended. 

--
Tony Lill,                         Tony.Lill at AJLC.Waterloo.ON.CA
President, A. J. Lill Consultants        fax/data (519) 650 3571
539 Grand Valley Dr., Cambridge, Ont. N3H 2S2     (519) 241 2461
--------------- http://www.ajlc.waterloo.on.ca/ ----------------

Understatement of the century:
"Hello everybody out there using minix - I'm doing a (free) operating
 system (just a hobby, won't be big and professional like gnu) for
 386(486) AT clones"

	- Linus Torvalds, August 1991