[mythtv-users] software raid 5 xfs

Brian Guilfoos mythtv at guilfoos.com
Sat Apr 14 23:08:11 UTC 2007


Charles Mason wrote:
> I should say the RAID array is connected through a high point SATA 8
> channel rocket raid controller. This was the cheapest card I could
> find that would let me use 5 hds in raid 5.

This is very similar to my setup.  I'll go ahead and document my storage
setup and the lessons I've learned, mistakes I've made, and things I'd
do different.

My fileserver is an Athlon Sempron 2400+ with 1.5 GB RAM and a OS hard
drive.  I've got a High Point 8-channel IDE card that also uses
proprietary drivers that do _some_ portion of the XOR on-card,
allegedly.  I've got 8 250GB disks, arranged such that each IDE
channel's master is in one RAID5 array, and each slave is in another.
Then, in software, I striped those two together for a total of 1.5TB.  I
put that drive in an LVM group, and split it into a few partitions (for
resizability).

I get pretty good performance - my backend mounts a share off of this
drive via NFS, and I can record 2 HD and one SD stream (plus one
commflag / transcode), and watch one stream with _minimal_ glitches (I
might lose 5 seconds per hour, tops).  Of course, there are some
variables that are (currently) outside of my control that can result in
99% wait state, and subsequent glitches in recordings.

1) The fileserver is also my web server and email server.  While those
functions don't cause much disk activity, they do cause occasional
transient processor loads.

2) I very VERY stupidly put my swap partition on the mega-raid array.
("It'll be fast!").  This caused significant additional disk load that
got progressively worse under higher system loads.  I upped my original
RAM from 512MB to 1.5GB, which solved the problem, but if I had it to do
over, I'd have put the swap on the system drive.  If I had a CDROM in
the machine, I'd boot a live CD and resize the / partition to make room
for a new swap.  If I get a bee in my bonnet, I may yank the machine,
crack it open, hook up a drive, and do just that.

3) My home directories (including Windows roaming profiles) are also on
this machine, and the Samba PDC is also running here.  This isn't too
bad, but it presents the possibility for heavy disk IO interfering with
MythTV operations.

4) The MySQL server is running on this machine, and I moved the db
storage to a directory on the RAID array.  I did it ostensibly for the
redundancy, but I probably could have (still could) get actual backup by
replicating the DB to another machine, eliminating additional disk activity.

5) CentOS + RAID + LVM + XFS = doubleplusungood.  I suspect it was the
stack problem mentioned elsewhere in this thread, but during recordings
+ frequent small writes (say, a torrent downloading) kernel panics were
extremely frequent.  I elected to go to ext3 + slow deletes, which works
great.  When XFS worked, it was brilliant.

6) My ext3 journal was too small (32MB) for such a large partition
(1.5TB).  I followed the instructions I found here on the list to grow
my journal to 384MB just yesterday, and it appears to have had a
positive effect.

There are a few other tweaks I may try to optimize disk performance,
like tweaking buffers and page writing times.  For my current usage, I'm
99% there.

What my future plan is (or, what I'd do if I could do it all over
again):  Build a FreeNAS box that will only provide MythTV storage.  Use
faster SATA drives.  Probably _won't_ RAID the disks together (just LVM
them together - recordings aren't nearly as important as, say, home
directories).  Since it's only a MythTV box, I'll probably try whichever
of XFS or JFS is most mature on FreeBSD.  I'll keep all the other
services on the current server, which will just serve home directories
(and maybe the music files, since those are relatively small compared to
recordings and MythVideo files, and I can tune the filesystem a little
bit to that effect).

While the number of streams I record simultaneously probably won't grow
much (I have two machines that process - commflag or transcode -
streams, but to reduce load only one of those is processing the job
queue between 4 PM and midnight), I want to add additional frontends, so
I need to be able to handle more streams.  I think this planned setup
should be able to handle it.


More information about the mythtv-users mailing list