[mythtv-users] Software RAID and ionice

[David Rees]

On 5/24/07, Richard Freeman <r-mythtv at thefreemanclan.net> wrote:
> Gene Stapp wrote:
> > I got around this problem by creating a raid0 partition just for myth to
> > use (and any not very important video) - two 500GB drives do a great
> > job. That keeps the other raid arrays clear for other uses and for
> > permanent storage of important video. Raid 5 arrays suck at write
> > operations compared to raid 0s...
>
> Does the RAID performance matter at all as long as it can sustain the
> writes needed by mythbackend with reasonable overhead?  I'm thinking
> that if CFQ is actually being used then ionice should take care of the
> rest.

The issue Gene (and I previous) have alluded to is that RAID5 sucks
for small writes, especially with software raid. A small random write
(smaller than a chunk size) involves a read AND a write so that
updated parity information can be written out.

Myth becomes IOBOUND when it can't flush data to disk fast enough.
The SyncLoop periodically (once a second) flushes data to disk.
The default buffer size is 2MB.

Let's say your recordings take up 2 GB/hour. This is appx 600
KB/second. This means that if for some reason a write blocks for 4
seconds mythtv will become IOBOUND. This isn't that uncommon when a
system is under heavy IO load.

I would suggest trying the following things:
1. Try removing the sync calls.
2. Try increasing the buffer size to 5-10MB.
3. Combinations of the above.

> Now, I'm wondering if myth's fsync behavior is causing the problem.
> Maybe the kernel is prioritizing everything so that there is always room
> in the buffer for myth, but the use of fsync means that it isn't enough
> that there is room in the buffer - it becomes an issue what order the
> data is written since myth isn't happy leaving data in the buffer.

I'm not sure what happens when you try to write to a FD that is in the
process of being synced, but I suspect that the write will block. It
would be interesting to log how much data myth is writing at a time,
how long each write takes along with how long the fsync or fdatasync
takes.

> Also - I'm still leery about xfs until I can find some documentation
> that clearly indicates that it doesn't zero out entire files if they
> were open when the system dies.

http://oss.sgi.com/projects/xfs/faq.html#nulls

Like I said before, the only nulls you'll see in files is for data
that hasn't been synced to disk. With mythtv syncing you shouldn't see
more than a second's worth of nulls. Without it, it will vary on your
VM settings but I doubt you would see more than 30 seconds worth of
nulls.

-Dave