Difference between revisions of "Storage Groups Weighting"

From MythTV Official Wiki
Jump to: navigation, search
(Was way out of date. Now it just needs to be rewritten properly to contain all the information and without the direct copy/paste of mailing list responses.)
(Fix Link)
 
(3 intermediate revisions by 2 users not shown)
Line 1: Line 1:
I recently (early March 2007) asked on the mailing list if there was a way of playing with the weights given each of the storage groups so that local filesystems are preferred over remote file systems. Turns out that is already how it works!
+
#REDIRECT [[Setup Storage Directories#Storage Group Disk Scheduler]]
 
 
Here is the response I received from Chris Pinkham. I'm pasting it here so that other's might find it useful as well.
 
 
 
http://www.gossamer-threads.com/lists/mythtv/users/256666
 
 
 
{{Note box| The Storage Group Disk Scheduler has been updated to provide 3 different implementations, which can be selected with the mythtv-setup General setting "Storage Group Disk Scheduler".  The implementation described below is Combination.  Information on the new implementations is available at http://www.gossamer-threads.com/lists/mythtv/users/424443#424443 .}}
 
 
 
<pre>
 
* On Tue Mar 06, 2007 at 04:53:31PM +1030, Darryl Ross wrote:
 
> > Firstly, thanks heaps to the devs and testers who have got Storage
 
> > Groups working really well. It's simplified my setup hugely!
 
 
 
Glad it's working well for you.  :)
 
 
 
> > Just wondering if there is any capability (or plans) or specify a
 
> > preference to how the storage groups are utilised?
 
 
 
There is limited ability for this via some undocumented settings, but
 
there is no GUI to set them up.
 
 
 
> > At the moment I believe Myth will record to the drive with the most
 
> > amount of free space if there's only a single recording and will spread
 
> > simultaneous the recordings across all available drives to reduce head
 
> > movements.
 
 
 
It's a bit more complicated than that.  Myth uses weights to determine
 
what filesystem/directory to record to.  The following default values are
 
used:
 
 
 
  SGweightPerRecording  = 10
 
  SGweightPerPlayback  =  5
 
  SGweightPerCommFlag  =  5
 
  SGweightPerTranscode  =  5
 
  SGweightLocalStarting = -1.99 * SGweightPerRecording
 
  SGmaxRecOverlapMins  =  3
 
 
 
The drive with the lowest weight is used first.  In a tie, the drive with
 
the highest amount of free disk space is used first.
 
 
 
Each Storage Group Directory has its own weight, but Myth is smart enough
 
to know if several directories are on the same shared filesystem.  When
 
a weight is applied to a directory that is in use, it is applied to all
 
directories on that filesystem because we are trying to spread the load
 
out across filesystems, not directories on the same filesystem.
 
 
 
The starting weight of all drives is 0.  Local filesystems/directories
 
are then offset by SGweightLocalStarting, so by default they are then
 
at -19 because SGweightPerRecording defaults to 10.  This makes it so
 
that local drives are preferred over remote drives.  A local drive has
 
to have an effective weight of 20 before a remote drive will be used to
 
store a new recording.  If you do not have any playback going on, this
 
would mean that you'd have to have 2 recordings going to a local drive
 
before a remote drive would get used.  If you have 2 local drives and
 
1 remote drive, you'd have to have 4 recordings going on locally before
 
a remote drive would get used.
 
 
 
So, if you have less than 4 tuners and 2 local drives, then the default
 
should already give you what you are looking for.  If you have more than
 
4 tuners or want to guarantee that all recordings go to the local drives
 
unless they fill up, you can do so using the undocumented SGweightPerDir
 
setting.
 
 
 
In order to make Myth not use a filesystem/directory, we need to
 
artificially inflate the starting weight for that directory.  We can do
 
this by insertting a setting in the database.
 
 
 
The key for the setting is SGweightPerDir:HOSTNAME:DIRECTORY.  The
 
hostname is the hostname that sees the directory.  So if the directory
 
is actually on the fileserver which is server1 but is mounted via NFS on
 
server2 which is running mythbackend, we'd use server2 here.  The
 
directory is the local path on HOSTNAME, so you'd put /mnt/video or
 
whatever you use here for the remotely mounted directory.
 
 
 
  SGweightPerDir:server2:/mnt/video
 
 
 
The value that we put in this setting will be applied as an offset to
 
the initial weight for this directory.  You can play it safe by setting
 
this to something large like 99 or 100 or even higher.  So in the
 
example here, the actual setting key would be:
 
 
 
You need to run the following SQL to insert that into the settings table:
 
 
 
  INSERT settings
 
      (value, data, hostname)
 
  VALUES
 
      ("SGweightPerDir:server2:/mnt/video", 99, "server2");
 
 
 
So, now when the Storage Groups scheduling code runs, /mnt/video will
 
start out with a weight of 0.  Since it is a remote drive, the only
 
offsets that will be initially applied will be the one we specified in
 
the settings table with the SGweightPerDir setting.  Your two local
 
directories would each end up starting out at -19 and the remote
 
directory would start out at +99.  So unless you have a huge number of
 
recordings and playbacks going on on each of your local drives, the
 
remote directory will never be used unless the locals fill up.
 
 
 
If you run mythbackend with the "-v schedule,file" option, you can
 
see the weights as they are applied and the logs will show you why
 
Myth chose one directory over another when determining where to put
 
the next recording.
 
</pre>
 

Latest revision as of 15:45, 14 August 2015