[mythtv-users] some more RAID and HD speed questions

[kteague@speakeasy.net]

> -----Original Message-----
> From: Steve Adeff [mailto:adeffs at gmail.com]
> Sent: Wednesday, January 18, 2006 12:30 AM
> To: mythtv-users at mythtv.org
> Subject: Re: [mythtv-users] some more RAID and HD speed questions
> >
> > I don't see why you shoudln't be able to use nice against the cp command
> > (or whatever means you're using to transfer the data).  
> 
> which I have been doing, I just didn't think it would work for this. But I'll 
> give it a try and hopefully won't have any issues =)

While nice may be able to regulate how much CPU time is given to cp, cp isn't going to treat the drives any differently, so what you may end up with is something similar to burst transfers, which I don't think is what you're looking for.  I believe you're looking for something to regulate how much priority cp has over the hard disk.

 
> > As an alternative, 
> > there is QoS (Quality of Service).  It will prioritize network
> > packets/traffic based on packet type.  So if your myth streams are comming
> > in with type X, and your copy is using type Y, you can regulate type Y
> > traffic to be lower priority than type X.  QoS is mainly used in an
> > environment where multiple services (say, VoIP and data transfer from
> > servers) travel over the same medium.  The VoIP traffic can be given a
> > higher priority over the data transfer in order to keep the voice comm.
> > from chopping or the message from transferring late due to packet loss or
> > late packet delivery.
> 
> that sounds more like for network traffic?

Correct, but the data is being transferred over the network, using conventional tools (cp, rcp, scp, ftp, etc.), and is using standard TCP/IP protocols as a transport medium.  I think this is more of what you're looking for.  The program performing the transfer will only be able to transfer at a certain rate.  This should work fine, so long as you prioritize your data transfer packet types to have a low priority.  This will leave the data transfer process with minimal bandwidth and, as such, it will not thrash at the hard disk and interrupt other processes that need higher priority read/write access.  

There's also Traffic Shaper, but this affects the entire NIC's network performance, not just certain packet types.  With Traffic Shaper, you can configure a NIC to only pump out so much data.  

So, lets take your scenario as an example:
  1) you have 2 HD streams and 1 SD stream that needs to have the highest priority.
  2) you want to transfer data from mythbackend to transcodebox for transcoding, and you do this via scp.
  3) you want to transfer data from mythbackend to your desktop PC to view an MPEG on the box, and you do this with FTP.

A) If you used traffic shaper, both scp and FTP will be limited to X amount of bandwidth.

B) If you use QoS, scp can be configured to have low priority traffic, and FTP can be configured to have medium priority traffic.  I'm using the terms "low" and "medium" loosely.  If you do this right, you can have an scp process transferring a show to be transcoded to transcodebox, while 2 HD streams and 1 SD stream are being recorded, *and* FTP an MPEG to your desktop for viewing without interrupting the recording stream.  Even though QoS is totally ignorant of local processes (it's only aware of network I/O), you could potentially encounter the problem you're fearing if the data transfer is given  too high of a priority.

I honestly can't tell you what MythTV would do if it couldn't write to the hard drive due to some other process hogging the read/writes.  Is it able to buffer the writes?  If so, how big is this buffer?  The size of the buffer, along with some math, can tell you how much of a delay you can accomidate in such an event.

Going back to this thread (new replies have arrived), Chris and Jarod have replied with some great insight while I was drafting this.

- Ken