[mythtv-users] Disk Space Problem

[Tom Hayward]

On Tue, Oct 30, 2012 at 12:49 PM, Michael T. Dean
<mtdean at thirdcontact.com> wrote:
> On 10/27/2012 09:39 AM, Peter Bennett (cats22) wrote:
>> I also wonder how it is deciding which file system to use. The "Balanced
>> Free Space" scheduler is selected, and according to the documentation
>> that will allocate recordings on the disk with most free space. Since
>> all disks are 99% full, the one with the most free space will randomly
>> depend on the size of the last program expired and what was recorded
>> since it was expired. I have three files systems of very different sizes
>> and I expect it to make more use of the largest file system. I don't
>> think this algorithm will achieve that. It may even select a smaller
>> file system with nothing expirable on it, and then be unable to complete
>> the recording, instead of using another file system which may have many
>> expirable recordings available.
<snip>
>>   Or have I
>> misunderstood how the balanced disk scheduler works?
>>
>
> It's that users with nearly (or completely) full file systems should almost
> definitely be using either Balanced Disk I/O or Combination Storage Group
> Disk Scheduler, and not Balanced Free Space.
>
> http://www.mythtv.org/wiki/Storage_Groups#Storage_Group_Disk_Scheduler
>
> As you've seen, Balanced Free Space makes little sense when file systems are
> nearly full.
>
> Mike

I use the Combination disk scheduler, but I also find it makes
decisions "randomly depending on the size of the last program expired
and what was recorded since it was expired" when all other things are
equal.

For example:
- 2 filesystems are local
- Both filesystems are nearly full, #1 is 1 GB less than MythTV's free
space minimum, #2 is 5 GB less.
- No recordings or playback in progress
- Most of the existing recordings are 1 hour programs of similar
bitrate--almost all are 6 GB.
- Both disks have auto-expirable recordings, but the recordings on
disk #1 all happen to have 0 priority, and on disk #2 they all happen
to have 2 priority.

A recording starts. The Combination disk scheduler will choose
filesystem #2. By the time the recoding completes, filesystem #2 has
reached the minimum free space limit and MythTV will expire one of the
6 GB priority 2 recordings from filesystem #2. Now there is 5 GB free
on filesystem #2 again.

An hour later, another recording starts. The combination disk
scheduler chooses filesystem #2 because it has more free space.
Another 6 GB priority 2 recording is expired, despite filesystem #1
being full of lower priority recordings.

The Combination scheduler will continue choosing filesystem #2 until a
new recording is exactly between 4 and 5 GB (leaving between 0 and 1
GB available), or simultaneous recordings force the balanced IO part
of the algorithm possibly filling and expiring something from
filesystem #1.

So when multiple filesystems are nearly full and there are seldom
simultaneous recordings, the Combination disk scheduler tends to
always pick the same filesystem.

If I understand the IO disk scheduler correctly, it will always pick
the first filesystem if all filesystems are idle (no simultaneous
recordings or playback). This is no better. I think the IO disk
scheduler would be better if in the case of equal weights, it randomly
chose a filesystem.

Or even better: in the case of equal weights, the IO scheduler (or
possibly a new fourth scheduler) chose the filesystem with the next
eligible auto expire recording. This way it is choosing filesystems
based on what will expire next, not what expired last.

Another way to do this would be to incorporate the recordings with the
lowest priority into the free space calculation, so the scheduler
would favor filesystems with low priority auto-expirable recordings.

I hope this makes as much sense here as it does in my head :-) Now I
need to go look at what code would be required to add another disk
scheduler...

Tom