[mythtv] Alternative to LVM/Raid for multiple disks?

[Mike Benoit]

Interesting you brought this up at the time you did. I wrote a
"quick'n'dirty" script that does exactly this, for the exact reasons you
specified a couple days ago. (wanted to get ready for the CSI marathon
on Spike TV. ) I have 5 machines in my house, ranging from Linux to
Windows, desktops to laptops that all have disk space I would like to
utilize.

Its basically just a prototype bash script, it would probably do much
better in perl, or if someone who knew bash better than myself could
improve it quite a bit I'm sure. 

The basic idea is you specify your primary mount point (default MythTV
recording destination), and an unlimited amount of secondary mount
points. You then specify how much disk space you always want to keep
available on each specific mount point. (so you don't run your girl
friends machine out of disk space of course. ;)

The script can be run how ever often you want, and what it will do is
basically fill up each mount point to the "level" you specified. So if
you have /mnt/desktop_1 which has 25gb free, and you always want to keep
10gb free on that mount point, the first run of the script will move
15gb of recordings to it.

The script also takes care of deleting the remote mount point
recordings, by simply checking the secondary mount points for .nuv files
that don't exist in the primary mount point.

So far it works pretty well, but USE WITH CARE. You've been warned.

PS. It only works with recordings 24hrs or older, so it doesn't have to
worry about moving an "in progress" recording. However it doesn't take
in to account if the recording is being watched or not. 

On Tue, 2004-09-07 at 22:51 -0400, Art Morales wrote:
> Hi All,
> 
> I just finished discussing this idea with a fellow Myther...  Right
> now, there are two ways to add more disk space in multiple drives to a
> mythbox: LVM and RAID.  I don't like the idea of LVM because the MBTF
> is now decreased overall, and if I loose a drive, I loose the whole
> volume.  Raid takes care of this, but now I'm either loosing ~30% of
> the diskspace (Raid 5) or gain no redundancy (Raid 1 (or is it 0?)). 
> Also, with Raid, I'm limited for the most part to equal size drives,
> which is not what people have usually lying around, and makes adding
> drives later kind of a pain.
> 
> Anyways, what if instead of having just one entry for video storage on
> the backend, we allow multiple ones (/var/video1 /var/video2 etc.)? 
> We could then add some very simple logic to the backend:  if less than
> X gb on current disk, store on next disk, and so on.  I would assume
> it would be a pretty simple addition to the code, but unfortunately, 
> this programming is beyong me...  I don't foresee any significant
> changes to the schema either...
> 
> The more I think about this idea, the more I like it because what if
> instead of using local drives, we use network shares mounted on
> diffent machines?  I have multiple desktops and servers in the house
> and they all have extra diskspace.  If I shared this space and made it
> available to the backend, I could suddenly have a 400gb store without
> buying anything new... (basically acting as a distributed file
> system).  On a simple gigabit network, there should be no issues with
> bandwidth, even with HDTV content.
> 
> Once you have such a system in place, one could use cron jobs to move
> certain files from one machine to the other and then transcode/process
> them there on a faster CPU (since the backend is more likely to have a
> slower CPU than the frontends).
> 
> Any takers?  I'm willing to help design the system a bit better if
> needed, but my programming skills are not up to par (unless you are
> talking Perl, then I can help :)
> 
> Art
> _______________________________________________
> mythtv-dev mailing list
> mythtv-dev at mythtv.org
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev
-- 
Mike Benoit <ipso at snappymail.ca>
-------------- next part --------------
#- NO TRAILING SLASHES
primary_mount_point=/mnt/video

mount_points=( '/mnt/video2' '/mnt/video3' ) 
#- IN BYTES
free_space=( 15000000000 10000000000 )



get_free_space() {
	freespace=`df --block-size=1 $1 | tail -n1 | tr -s " " | cut -d" " -f4`;
}

get_file_size() {
	filesize=`du -b $1 | cut -f1`
}

mount_point_count=${#mount_points[@]}
echo "Total mount Points: $mount_point_count"

for mythtv_file in `find $primary_mount_point/*.nuv -type f -mtime +1 | sort -t "_" -k2` ; do

	if [ -f $mythtv_file ] ; then

		get_file_size $mythtv_file
		current_mythtv_file_size=$filesize

		echo "NUV File: $mythtv_file - Size: $current_mythtv_file_size"
	
		index=0
		freespace=0
		while [ "$index" -lt "$mount_point_count" ] ; do
			get_free_space ${mount_points[$index]}
			current_free_space=$freespace

			echo "Mount Point: ${mount_points[$index]} - Min. Free Space: ${free_space[$index]} Bytes - Current Free Space: $current_free_space Bytes"
		
			if [ "$current_free_space" -gt "${free_space[$index]}" ] && [ "$current_free_space" -gt "$current_mythtv_file_size" ]; then
				echo "Mount Point: ${mount_points[$index]} has enough free space for file, copying file"
				nice -n 15 cp $mythtv_file ${mount_points[$index]}
				cp_retval=$?
			
				if [ "$cp_retval" -eq "0" ] ; then
					echo "File copy completed successfully, symlinking"
					rm -f $mythtv_file
					ln -s ${mount_points[$index]}/`basename $mythtv_file` $mythtv_file
				fi;

				#don't copy the file to any other mount points.
				break;
			else
				echo "Mount Point: ${mount_points[$index]} is full"
			fi;
		
			let "index = $index + 1"
		done;
	else
		echo "File is most likely a symlink"
	fi;


done

index=0
while [ "$index" -lt "$mount_point_count" ] ; do

	for mount_point_file in `ls -1tr ${mount_points[$index]}/*.nuv` ; do

		if [ ! -e $primary_mount_point/`basename $mount_point_file` ] ; then
			echo "File: $mount_point_file does not exist in primary mountpoint, delete from remote mount point"
			rm -f $mount_point_file
		else
			echo "File: $mount_point_file DOES exist in primary mountpoint, leaving alone"
		fi;

	done

	let "index = $index + 1"
done

exit 0;