[mythtv-users] Another take on large-capacity disk enclosures

[f-myth-users at media.mit.edu]

    > Date: Mon, 15 Jun 2009 16:45:45 -0600
    > From: Brian Wood <beww at beww.org>

    > I didn't count the SuperMicro card as "standard PCI", though it will work in 
    > such slots, at reduced performance.

Yes.  Fortunate, since otherwise my proposed design would be a
complete nonstarter, since the cheapest mobos I see on Newegg
(at least) that seem to support PCI-X are in the $3-500 range and
it'd be cheaper to just get a raft of the TR8M-B's and a desktop
and hook them together.  (Except for one $66 mobo from Tyan that
I don't trust and has iffy reviews.)

Btw, I cannot figure out if the Supermicro board could use a 66 Mhz
(but still 32-bit) PCI slot; its keying (according to the Newegg
image viewer) seems to allow all of 3.3V, 5V, and PCI-X (c.f.
http://www94.web.cern.ch/hsi/s-link/devices/s32pci64/slottypes.html
and their images), but that alone doesn't tell me about 66 MHz.
[I also don't know how common 66 Mhz 32-bit PCI slots are on typical
motherboards, so it may be a moot point.]

    > It's the difference between 64 bits clocked at 133Mhz. (PCI-X) and 32-bits 
    > clocked at 33Mhz. (PCI) That a very substantial difference IMHO, even if the 
    > PCI-X is clocked down to 100Mhz. due to using two cards.

Yes, but...  very naive scaling says that PCI-X is 1064 MB/sec and PCI
is 132 MB/sec.  Average speed off of a WD 1.5T that I happen to be
poking at right now is 72 MB/sec.  (Yes, some parts are no doubt
faster and some are slower, but this the average taken over reading
or writing the entire device with sdd; a full read or full write takes
5h47m.  If this was a filesystem, there's be some head motion, too.)

I know PCI can't possibly be getting 100% of its theoretical speed,
but even if it's getting only 54% of that, it's keeping up (on
average).  Since I'm not running a RAID and will either be doing
file I/O that involves seeking (so the PCI bus is the least of my
problem) or running a small number of streams at video rates
(approximately---and SD video at that), it seems like I'm not going to
get too near the bandwidth limit on the PCI bus.  Others who might
like to use a setup like this precisely as a video NAS may have
similar constraints, e.g., don't spend hundreds on fast access to
disks when you're just not hauling the data on/off of them that fast
to begin with.

[This would be an issue, maybe, if I was trying to clone the entire
pile of disks elsewhere at once and was in a hurry---then I'd
presumably be limited by either PCI bandwidth or GigE bandwidth.]

    > Gigabit ethernet is actually faster than a standard PCI buss, though there are 
    > factors that can negate that speed advantage.

I can see that, given that theoretical max on 32-bit PCI seems to be
132 MB/sec or 1056 Gb/sec; if we assume that there is -zero- overhead
in GigE, maybe it might be slightly faster or slightly slower.  If
we're running the GigE full-duplex, though, then it supports twice
that (1 Gb/sec in each direction) and that's clearly 2x faster than
a PCI bus could support.  In my application, that's almost never going
to happen (and I can tolerate less than top-of-the-line performance in
return for saving many hundreds on the master & slave machines).