[mythtv-users] Storage groups behaving unexpectedly with slave backend.

[Andre Newman]

On 13 Mar 2010, at 17:45, Michael T. Dean wrote:

> On 03/13/2010 08:09 AM, Andre Newman wrote:
>> I have a master backend with four tuners and a couple of slave backends with a tuner each and some local storage, all running 2:0.22.0-fixes23670-0ubuntu2 on Unbuntu 9.10, one is 64 bit the rest 32bit but that doesn't seem to affect my problem.
>> 
>> Yes I've searched this list and the website and the wiki and anything else that google can find for me and I can't see any clues.
>> 
>> 
>> I have three drives on the master backend each configured as a separate storage group in default, database and everything non mythtv is on a separate raid 1.
>> Slave BE 1 has two drives each as a separate storage group and used to have the three master backend drives mounted on the same paths as the master over nfs.
>> Slave BE 2 has 1 drive and used to have the three master backend drives mounted on the same paths as the master over nfs.
>> 
>> It seems since disabling the nfs mounts I can no longer play any recordings that were made by the slave be's onto those mounts! I can see the recordings on the master backend, the files are present and MythWeb can find them to let me download them! A frontend on either the master backend or on any other frontend reports that it can't find the file!
>> 
>> I realise I've messed the system around by removing the nfs mounts but I thought that MythTV searched for files in any storage directory, I've read many stories of people moving files between storage directories and myself I've moved files from remote backends to the master with impunity to make space on the slave disks and have recordings available when the slave is off.
>> 
>> So is there some sort of cache somewhere that's telling MythTV to look for those recordings only on the Slave BE via the nfs mount and not to look locally? If so how do I flush this cache? I've tried removing the now unused directories from the slave be's and restarting all the backends, master last. If it's relevant I have thumbs for these "missing" shows in MythWeb and in the frontend.
>> 
>> Before anyone says "just restore the nfs mounts" I can't right now, I'm running a 2.6.33 ppa kernel to try to resolve a sata driver problem that's been dropping disks and corrupting files on the master backend ever since the upgrade to 9.10 (so far so good) and that's completely broken nfs also I never got good results with recording over nfs (although that may have been the sata driver problem) so I'd rather not have to re-instate it.
>> 
>> Any ideas gratefully received.
>>   
> 
> To know for sure, you would need to list all the directory paths and the hosts for which they're configured (all the information you entered in Storage Directories in mythtv-setup).
> 
> However, my best guess is that you (like 99.999999999999999% of users) didn't understand how Storage Groups work and configured them in such a way as to cause issues when you change the layout.

I did spend some time trying to understand the "right way" to configure it but failed and went with what seemed to be the consensus method on the list, evidently wrong :-(
> 
> Every storage group directory defined on the master backend is inherited by *all* remote backends.  Therefore, you defined some directories on the master backend as part of the Default storage group (say, /srv/mythtv/recordings/mbe/1 through 3), then you NFS mounted those directories on the remote backends.  Then you defined some directories as part of the Default storage group for the remote backends (say, /srv/mythtv/recordings/sbe1/1 and 2 and /srv/mythtv/recordings/sbe2/1).  That meant that sbe1 had (and may have used for recording) the directory list:
> /srv/mythtv/recordings/mbe/1
> /srv/mythtv/recordings/mbe/2
> /srv/mythtv/recordings/mbe/3
> /srv/mythtv/recordings/sbe1/1
> /srv/mythtv/recordings/sbe1/2

That's about the size and shape of it, how did you guess ;-)


> 
> and that sbe2 had (and may have used for recording) the directory list:
> /srv/mythtv/recordings/mbe/1
> /srv/mythtv/recordings/mbe/2
> /srv/mythtv/recordings/mbe/3
> /srv/mythtv/recordings/sbe2/1
> 
> So, any recordings the remote backends made to the /srv/mythtv/recordings/mbe/* directories is now unavailable to the recording host when you unmount the NFS mounts.  Now, sbe1 only has available the directories /srv/mythtv/recordings/sbe1/1 and 2 and sbe2 only has available /srv/mythtv/recordings/sbe2/1 .
> 
> I.e. it's as if you moved the recording file without telling the backend you did so, so it's expecting the wrong host to have that file.

Spot on, which I _would_ expect to be a problem, except for the many comments about backends searching all local groups and my experience of being able to move recordings around, evidently with limitations that I'm now discovering!
I had mistakenly thought that setting "Master Backend Override" would make the mbe find all those files locally but it seems it would have done if only I hadn't messed up the storage group configuration. 

> 
> This would definitely have happened if you did not have local storage on the remote backends originally (and, so, all recordings were made to the file server through NFS).

They were at first but that didn't work out well so those recordings are all deleted, evidently some remain from an intermediate stage of workingness.


>  If you had local and NFS-mounted storage on the remote backends, it would be especially likely to happen if you have specified Balanced Free Space for the mythtv-setup setting (in General):

I've been using Balanced Disk I/O, I have had big problems with simultaneous HD recordings so have been trying to throw spindles at the problem.

> 
> Storage Group Disk Scheduler
> - Balanced Free Space
> - Balanced Disk I/O
> - Combination
> This setting controls how the Storage Group scheduling code will balance new recordings across directories.  'Balanced Free Space' is the recommended method for most users.
> 
> (see http://www.gossamer-threads.com/lists/mythtv/users/424443#424443 ) where the default for users who created their MythTV databases with post-0.21-fixes r21225 or higher is Balanced Free Space and the default for users who created their MythTV databases before trunk r21225 and upgraded to 0.22-fixes or higher is Combination.  Combination would be the one least likely to result in the problem mentioned above, but it could happen even with Combination or Balanced Disk I/O, depending on configuration.

Mine's an awful lot older than that!

> 
> To fix it, you'll have to change the hostname for all the recordings that were recorded by remote backends onto the master backend file systems to the master backend hostname.

Ok, I know how to find out how to do that, after backing up said database.

Thanks for taking the time on a 99.999999999999999% faq or is a fmq frequently misunderstood question?

So..... to make it all good going forward (assuming I get reliable nfs when my sata problem gets fixed) I should have:

All mbe storage group directories nfs mounted on all slaves with identical paths but not explicitly configured in the slave be because they will be inherited and therefore understood to be the same. Only sbe local storage configured locally.

I'd like to get to mbe and just one diskless sbe/rfe using wol and that just to make use of the two extra satellite feeds in the living room, one day....


Thanks again.

Andre