[mythtv-users] storage directories

Michael T. Dean mtdean at thirdcontact.com
Tue Jul 13 17:00:34 UTC 2010


  On 07/12/2010 01:20 PM, Mike Perkins wrote:
> Michael T. Dean wrote:
>> Deleting a Storage Group from the master backend deletes the group 
>> *and* all
>> directory list overrides on other hosts (i.e. you can't have a 
>> directory list
>> override if the group doesn't exist).
>>
>> Therefore, you can go through each existing SG that you want to keep 
>> and write down the directory list for each SG.  Then, delete all the 
>> SGs from the
>> master.  Then, you can simply define the SG's on the master and add 
>> all the
>> directories to them, as appropriate.  With this approach, you'll know 
>> for
>> sure that you have things defined in the most-maintainable way--once 
>> on the
>> master backend, meaning there's only one place to update the 
>> directory lists
>> (that being on the master backend). Since you won't have any 
>> overrides adding
>> confusion, it will be easy to see where the information is coming from.
> Information, please. I'm slightly puzzled by how the above information 
> is used. If you're saying that you now only define SGs on the master 
> back end - and in my view that's the right place to do things like that

Yes.  That's how 99.9999999999999999999999999999999999% of users should 
do things.  IMHO, the remaining fractional percent only needs to do 
otherwise because they misconfigured their systems.

> - then how does a slave backend find it's storage?

A Storage Group is simply a mapping between a user-friendly name 
("Default", "Movies", "Archive," ...) and a list of directory paths.  
Note, also, that I did /not/ say anything about them containing 
information about the host that contains those directories.

Therefore, when you *define* a Storage Group (which can only be done on 
the master backend) and set up a list of directories, that Storage Group 
exists on *all* hosts.  Note, also, that I did not say that all the 
directories in the directory list must exist on all the hosts--only that 
the Storage Group exists on all hosts (the /mapping/ between a 
user-friendly name and a list of directory paths exists on all hosts).

When you use mythtv-setup on any host other than the master backend to 
edit Storage Groups, all you're doing is overriding the directory list 
for an already-defined Storage Group--you're saying, "The list I gave on 
the master backend is wrong."  (So, really, you should just fix the list 
on the master backend...)

> Two situations I can think of as examples, there may be other 
> permutations. Firstly, a slave with it's own storage.

This is exactly the setup I have.

> Does the MBE have to be able to access the SBE storage location using 
> samba/NFS/whatever in order to register the SG directory?

Nope.  I never run NFS or Samba on my network.  My master backend has 4 
file systems it's using for recordings and my remote backend has 3 file 
systems it's using for recordings.  The master backend has no access to 
the file systems on the remote backend and vice versa.

Regardless, I have *only* defined my Storage Groups on my master 
backend.  In my case, my Default Storage Group has a directory list:

/srv/mythtv/tv/a/recordings
/srv/mythtv/tv/b/recordings
/srv/mythtv/tv/c/recordings
/srv/mythtv/tv/d/recordings

where a, b, c, d are mount points and I've put my actual recordings into 
subdirectories so that the backend(s) can tell if the file system is 
mounted or not (otherwise, if I put a, b, c, d in the directory list, it 
would write to the directory whether or not the file system is 
mounted--meaning it would fill up the parent file system, which is my 
root file system).

Note, also, as mentioned before, my master backend has 4 file systems 
and my remote backend has only 3 file systems.  This isn't a problem.  
On my master backend, I have a different file system mounted at a, b, c, 
and d.  On my remote backend, I did not (yet--as I keep adding new 
storage) mount a file system at d.  Therefore, my remote backend has 
directories:

/srv/mythtv/tv/a/recordings
/srv/mythtv/tv/b/recordings
/srv/mythtv/tv/c/recordings
/srv/mythtv/tv/d

Since there's no file system mounted at d, there's no 
/srv/mythtv/tv/d/recordings (which is the one the Default storage group 
uses), so MythTV sees that the directory doesn't exist and doesn't use 
it (it simply ignores it) on my remote backend.

This means that I have no Storage Groups/Storage Group directory entries 
defined for any host other than my master backend.  (IMHO,) There's 
absolutely no reason to ever touch Storage Groups on any host other than 
the master backend.

If you prefer, you can do things slightly differently.  Let's say you 
don't want to "re-use" directory names on your different backends 
because you may find it confusing to have 2 different file systems using 
the same directory path.  In that case, the Storage Group definition on 
the master backend should contain the "union" of directory paths used by 
all backends.  So, it may have something like:

/srv/mythtv/tv/master/a/recordings
/srv/mythtv/tv/master/b/recordings
/srv/mythtv/tv/remote1/c/recordings
/srv/mythtv/tv/remote1/d/recordings

(where picking directory names is the hard part, but the above gives you 
the idea).  Again, a, b, c, d are all mount points and recordings is a 
directory existing within the file system mounted there.  Then, since 
there's no /srv/mythtv/tv/master/a/recordings directory on the remote 
backend, and no /srv/mythtv/tv/remote/c/recordings directory on the 
master backend, MythTV will know to ignore those directories on the 
hosts in question.

> Secondly, the reverse, a slave with no storage. Does it have to access 
> storage on the MBE (or somewhere else?) using samba/NFS/whatever or is 
> all the communication done using Myth's streaming protocol?

If you use NFS or Samba, you may have all your storage on the master 
backend or some NAS or whatever.  If so, just list all the directories 
in the Storage Group definition on the master backend and other hosts 
will use the same list of directories.  Since all recording is done to 
"local" file systems, the remote file systems must be mounted at the 
same absolute path on all hosts.

The only time this approach of never overriding Storage Group directory 
lists won't work is if you have your storage on different backends (as 
in the above example with (/srv/mythtv/tv/master/a/recordings and 
/srv/mythtv/tv/remote1/c/recordings--so each backend has its own hard 
drives)  and you NFS mount the TV directories on all hosts.  With this 
approach, your remote backend may write to 
/srv/mythtv/tv/master/a/recordings across the network using NFS while 
the master backend writes to /srv/mythtv/tv/remote1/c/recordings across 
the network using NFS--which is extremely inefficient compared to having 
each host write to its own local storage.  This can be fixed by proper 
choice of a Storage Group scheduler, but--IMHO--it's actually a 
misconfiguration.  There's no benefit to NFS-mounting TV directories on 
remote (backend or frontend) systems if you don't want the host to 
record to those directories.  So, to fix it (properly ;), just don't NFS 
mount your TV recordings directories on other hosts.  Again, though, if 
you feel you must have the TV directories NFS mounted on each MythTV 
system, you can, but you need to choose an appropriate Storage Groups 
scheduler.

So, basically, there's no need for any backend to have access to any 
file system other than the file systems to which that backend records 
TV.  In other words, all MythTV recordings access (recording, playback, 
etc.) is performed through the recording host's mythbackend.  The only 
exception is when you enable the setting:

Master backend override
If enabled, the master backend will stream and delete files if it finds 
them in the video directory. Useful if you are using a central storage 
location, like a NFS share, and your slave backend isn't running.

which is only meant to be used when you have a single file server 
containing all file systems and you want to be able to stream/playback 
recordings from remote backends even when the remote backends are shut 
down or when *all* of your storage exists on the master backend and 
remote backends record to it over the network--so without this setting, 
streaming from the recording backend would mean the recording backend 
reads a recording over NFS and then sends it across the network to the 
frontend.  Instead, enabling the override allows the master backend to 
read it from local storage and send it across the network to the 
frontend (so, only one trip across the network).

Moral of the story: Never, ever, ever, ever override a Storage Group 
directory list.  (In other words, *only* manage storage groups on the 
master backend.  Never even enter the Storage Directories section of 
mythtv-setup on any host other than the master backend host.)  This 
makes your MythTV configuration, your understanding of which directories 
each backend /might/ use, maintaining the list of directories in use, 
adding new directories, removing directories, and life in (general) much 
easier.  :)

Quick Rules:
1) Only set Storage Group directory lists on the master backend. (Never 
create Storage Group directory list overrides on the remote backends.)
2) Once properly configured (with no SG overrides on the remote 
backends), only manage Storage Groups using mythtv-setup on the master 
backend
3) Make the master backend's Storage Group directory list contain the 
union of all directories on all hosts--whether those directories exist 
on all hosts or not.
4) Directories in a Storage Group directory list need not exist on any 
host (those that don't exist on a particular host are simply ignored).
5) If you don't want a directory used on a particular host make sure 
that directory doesn't exist on that host.
6) Never put a mount point into a Storage Group directory list--always 
put some subdirectory under the mount point so MythTV will know to 
ignore the (non-existent) directory entry if the file system fails to 
mount properly.
7) If you think you need to override a SG directory list on a remote 
backend or remote frontend, you're making things more difficult than 
they need to be.

Mike



More information about the mythtv-users mailing list