MythDVBcut

From MythTV Official Wiki
Jump to: navigation, search

This script is based on one I've been using for several years to process my recordings from dvb-t, and another, less often, for recordings from dvb-t2.

Mythcutprojectx Mythpycutter

This updated script works for me with 0.25-fixes and 0.26-fixes for both dvb-t and dvb-t2. It provides more info about what it has done and, although restricted to cutting at keyframes, usually cuts where I would like it to. It might well be useful for other formats, at least for removing pre-and-post run.

The user interface is largely unchanged; I use it from the keyboard.

The link between the editor-based cutlist and the cutpoints actually used differs slightly from both the earlier versions. I find that edits at the last/first keyframes that show a wanted picture usually give acceptable results. This may well depend on the editing processes at source.

--Johnp 12:08, 8 March 2013 (UTC)

Soon after this was posted, Michael Stucky reported that the call to mplex failed when the script was used on an HD recording from an HDHR device. He gave a workaround and comments that I reproduce here for convenience:

"Further tests indicate that a trial-and-error process is/maybe required for each HD recording (channel??) to get a clean run of mplex. The only way I have been able to get consistent results is to use mythffmpeg to remux into an mkv container.

mythffmpeg -fflags +genpts -i $DEMUXPREF.m2v -i $DEMUXAUDIO -vcodec copy -acodec copy "$OUTFILE"

Where $OUTFILE is the recording name with an .mkv extension. I move the resulting file to my Video storage group, the file is playable on my MythFEs and all my other media players."

Here's the original thread:

http://www.gossamer-threads.com/lists/mythtv/users/540360#540360 --Johnp 20:52, 10 March 2013 (UTC)

Two 'features' have become apparent since the script was first posted:

MythTV 0.26 filenames are based on UTC, while the Mythfrontend display uses local time. I'm still using mental arithmetic to convert, but users not in the UK may find this more inconvenient.

An unannounced change at my local transmitter makes mythffmpeg find multiple 'Video' streams in some DVB-T channels; this causes an error exit. In the 'grep' lines for stream count and video stream selection replace 'mpeg2video' or 'Video' by 'mpeg2video (Main)'.

--Johnp 07:51, 3 July 2013 (UTC)

I'm posting a new version that can create either 'DVD format' or matroska files after cutting by ProjectX. The choice is made by editing the MAKEMKV Boolean variable in the script. Thanks to Michael Stucky for his suggestions.

http://www.gossamer-threads.com/lists/mythtv/users/546679

--Johnp 10:59, 4 July 2013 (UTC)

I'm still using essentially the same process here, but with some changes for 0.28-pre. Current version attached as mythDVBcut_20150728.sh This is a direct paste of the script as I use it with v0.28-pre-2683-g7f68683 under SL7. Editing for local conditions will be required, as before. I have also removed the version from March 2013.

--Johnp (talk) 12:33, 28 July 2015 (UTC)

Builds of MythTV Master after mid-April 2015 (v0.28-pre-2827) may insert a 'mythical' bookmark at the scheduled programme start. Mythfrontend playback of programmes recorded with pre-roll and cut with MythDVBcut would then start at a point some time after the start-of-new-file. v0.28-pre-3055 introduced 'mythutil --clearbookmarks' that will allow restoration of the expected behaviour. It can be called here immediately after, and in the same way as, the existing 'mythutil --clearcutlist' section below. See Ticket #11713.

--Johnp (talk) 18:51, 25 August 2015 (UTC)

I have added an experimental script, mythTScut20180309.sh, originally intended for cutting h264-encoded recordings but potentially useful for other .ts files. Input and output files are in .ts format. It uses tools from mythffmpeg and does not use mkv tools or Project-X. At present it has options to read single video and audio streams either 'together' or 'independently' (1 or 2). I feel that 2 is likely to be better eventually, but at present it fails on raw recordings with 'interactive tv' streams not recognised by mythffmpeg. Both options usually work after a first pass with option 1 and no cutlist.

Processed recordings usually play without trouble in mythfrontend or via UPnP but the positioning of switchpoints is still slightly imprecise and there may be a/v sync transients after them.

I hope that it will be possible to find ffmpeg options to address these issues.

Updated mythTScut.ts to 20180414. Tweaks to cutpoint positioning. Added options for output as Recording, Video, or Test; reading input as 1 or 2 streams, and NC to ignore cutlist.



Script.png mythDVBcut_20150728.sh


#!/bin/bash

# Copyright (C) 2014 John Pilkington 
# Uses ideas from scripts posted by Tino Keitel and Kees Cook in the Mythtv lists.
# The suggestion to convert HD mpeg2 files to .mkv, and code to do it, came from Michael Stucky.

# **** I have not tried running this script on an .mkv file that it has created *****

# Usage: "$0" <recording>   ...or...
# ionice -c3 ./"$0" <recording> will reduce io priority and is recommended.
# <recording> is a file recorded by MythTV with a valid DB entry and seektable,
# e.g. 1234_20100405123400.ts or 1234_20100405123400.mpg
# in one of the RECDIRs defined below.
# The output file replaces the input file in the list of recordings, perhaps with a new suffix.
# The input file is renamed to <recording>.old

# This script is essentially a terminal-based replacement for the 'lossless' mpeg2 mythtranscode.
# It was developed from mythcutprojectx but will now cut some recording formats that defeat Project-X.
# All cuts are made at keyframes.  

# Project-X is used here to demux and apply a cutlist, and also to discard all but one video and one audio stream.
# If the audio is initially in .wav format it will be converted to .mp2; mp2 or ac3 will be unchanged.
# For non-mpeg2 video all streams are passed through unchanged, but with cuts applied at the video keyframes.  
  
# The script then clears the cutlist, updates some entries in the database, rebuilds the seek table and creates a new preview.

# If the script is edited to have MAKEMKV=false, the result should be acceptable as a recording within MythTV
# and perhaps as an input to MythArchive.  After MAKEMKV=true, IIUC, the file must be treated as a Video.

# The logfile includes the positions in the new file at which deletions have been made. 

# The script needs to be edited to define some local variables and folders.  
# It will not recognise Storage Groups.

# The Project-X java-based demuxer for mpeg2 video may be packaged for your distro, although perhaps not fully updated.
# A tarball of the most recent version of Project-X can be downloaded from the link near the bottom of this webpage:-

# http://project-x.cvs.sourceforge.net/viewvc/project-x/Project-X/

# I'm running it with java-1.7.0-openjdk under SL7; if you need to build it, install java*jdk-devel and run build.sh

# Some of the comments in this script have been left as reminders of things that might be useful.
 
####################

#. ~/.mythtv/mysql.txt # for DB access info
#PASSWD=`grep "^DBPassword" ~/.mythtv/mysql.txt | cut -d '=' -f 2-`
#PASSWD=mythtv

# mysql.txt is no longer referenced in 0.26+
# The values are now defined within ~/.mythtv/config.xml but it seems easier to
# get them into the script by editing.  They are:

# DBUserName, DBPassword, DBLocalHostName, DBName

# Define the variables used for DB access
#
DBUserName="mythtv"
DBPassword="mythtv" 
DBLocalHostName="localhost" 
DBName="mythconverg"

 
BN="$1"     # "Basename" of file as given in the command line but without special attributes.

MAKEMKV=false   # Do not convert to .mkv.  After running Project-X use mplex to create a DVD-profile file.
# Remultiplexing with mplex fails at higher bitrates as found in HD mpeg2, but mkv can cope.
# MAKEMKV=true     # after ProjectX create an .mkv file

# If TESTRUN is set to true, cutlists will be shown but the recording will be unchanged.

# TESTRUN=true    
TESTRUN=false   # the recording will be processed
if [ $TESTRUN != "false" ] ; then TESTRUN=true
fi

#CUTMODE="FRAMECOUNT"  # Not used here.
CUTMODE="BYTECOUNT"

USEPJX=true  # Can be set to false here if you wish; will be set to false if not mpeg2video (Main)
TIMEOUT=20   # Longest 'thinking time' in seconds allowed before adopting the automatically selected audio stream.

# Variables RECDIR1, TEMPDIR1, RECDIR2, TEMPDIR2, LOGDIR, PROJECTX, HOST1, HOST2, HOST3 need to be customised.
# The TEMPDIRs are used to hold the demuxed recording, so should have a few GiB available. 

# Use $HOSTNAME (as given by running 'hostname') to find out which machine is being used and assign working directories.
HOST1="PresV5000"  # laptop with single disk, FE/BE
HOST2="gateway12"  # twin-disk box, FE/BE
#HOST3="HP-kub"    #
HOST3="HP_Box"     # another single-disk FE/BE

if [ $HOSTNAME = ${HOST1} ] ; then

   RECDIR1=/home/john/Mythrecs
#   TEMPDIR1=/home/john/MythTmp
   TEMPDIR1=/mnt/VidsOnRoot/tmpPX     # Not a good location!

   RECDIR2=${RECDIR1}
   TEMPDIR2=${TEMPDIR1}

   LOGDIR=/home/john/Logs

   #PROJECTX=/path/to/ProjectX.jar (or to a link to it)
   PROJECTX=~/projectx_link

elif [ $HOSTNAME = ${HOST2} ] ; then

   # Section below for twin-disk setup using Project-X. 

   # RECDIR1 and TEMPDIR1 should if possible be on different drive spindles.  Likewise RECDIR2 and TEMPDIR2.
   # This will reduce the load on individual disks and disk-controllers in the IO-heavy processing. 

   RECDIR1=/mnt/f10store/myth/reca
   TEMPDIR1=/mnt/sam1/tempb

   RECDIR2=/mnt/sam1/recb
   TEMPDIR2=/mnt/f10store/myth/tempa

   LOGDIR=/home/John/Documents/PXcutlogs

   #PROJECTX=/path/to/ProjectX.jar (or to a link to it)
   PROJECTX=~/projectx_link
   
elif [ $HOSTNAME = ${HOST3} ] ; then

   RECDIR1=/home/john/SGs/RecsSG1
   TEMPDIR1=/home/john/MythTmp

#   RECDIR2=${RECDIR1}
   RECDIR2=/home/john/SGs/LivetvSG1
   TEMPDIR2=${TEMPDIR1}

   LOGDIR=/home/john/Logs

   #PROJECTX=/path/to/ProjectX.jar (or to a link to it)
   PROJECTX=~/projectx_link
else
   echo "Hostname $HOSTNAME not recognised."
   exit 1
fi 

if [ "$BN" = "-h" ] || [ "$BN" = "--help" ] ; then
echo "Usage: "$0" <recording>"
echo "<recording> is a file known to MythTV as a 'recording' with a valid DB entry and a seektable."
echo "e.g. 1234_20100405123400.ts or 1234_20100405123400.mpg "
echo "in one of the RECDIRs defined in the script."
echo "The input file will be renamed to <recording>.old and"
echo "MythTV will see the output file instead."
exit 0
fi

# exit if .old file exists

if  [ -f ${RECDIR1}/"$BN".old ] ; then 
    echo " ${RECDIR1}/"$BN".old exists: giving up." ; exit 1
fi

if  [ -f ${RECDIR2}/"$BN".old ] ; then 
    echo " ${RECDIR2}/"$BN".old exists: giving up." ; exit 1
fi
 
# Customize with paths to alternative recording and temp folders

cd ${RECDIR1}
RECDIR=${RECDIR1}
TEMP=${TEMPDIR1}
if  [ ! -f "$BN" ] ; then
  cd ${RECDIR2}
  RECDIR=${RECDIR2}
  TEMP=${TEMPDIR2} 
     if  [ ! -f "$BN" ] ; then 
       echo " "$BN" not found.  Giving up"
       cd ~
       exit 1
     fi
fi

# Is it an mpeg-2 recording?
echo
echo
#mythffmpeg -i "$BN" 2>&1 | grep -C 4 Video | tee "temp$$.txt"       # no longer adequate
#mythffmpeg -i "$BN" 2>&1 | grep -B 2 -A 4 Video | tee "temp$$.txt"  # no longer adequate
#mythffmpeg -i "$BN" 2>&1 | grep -B 2 -A 4 "mpeg2video (Main)" | tee "temp$$.txt" # a reminder

# mythffmpeg may fail to identify streams if recording started before, or continued after,
# the period of activity of a part-time channel. 
# ffmpeg usually works for me, but
# 'dd bs=1M skip=numblock1 count=numblock2 if=infile of=outfile' 
# gives a fallback.
# Specify mythffmpeg here because it should be generally available and usually works.

ffmpeg -i "$BN" 2>&1 | grep -B 2 -A 4 "mpeg2video (Main)" | tee "temp$$.txt"

mpeg=$(grep -c "mpeg2video (Main)" temp$$.txt) 
echo "mpeg2video (Main) stream count:  $mpeg"
if [ $mpeg != 1 ] ; then 
  echo "Not mpeg2video (Main), or no or multiple video streams"
  if [ $mpeg = 0 ] ; then
     USEPJX=false  # really ought to see if it's a radio channel.
  else
     exit 1
  fi
fi

if [ $# -lt 3  ]
then
   echo "Needs one or three arguments." 
   echo  
   cat temp$$.txt
   echo
   
#  Examples (BBC FOUR SD, March 2015) :

# $ mythffmpeg -version
# ffmpeg version 2.3.1 Copyright (c) 2000-2014 the FFmpeg developers
# built on Feb 19 2015 23:55:22 with gcc 4.8.2 (GCC) 20140120 (Red Hat 4.8.2-16)

#  Duration: 01:05:59.66, start: 29170.850711, bitrate: 3786 kb/s
#  Program 1 
#    Stream #0:0[0x191]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 704x576 [SAR 16:11 DAR 16:9], max. 15000 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
#    Stream #0:1[0x192](eng): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 254 kb/s
#    Stream #0:2[0x196](eng): Audio: mp3 ([3][0][0][0] / 0x0003), 0 channels, s16p (visual impaired)
#    Stream #0:3[0x195](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
#    Stream #0:4[0x1c2]: Unknown: none ([5][0][0][0] / 0x0005)

   # Thanks to Christopher Meredith for the basic parsing magic here. 
   VPID=`grep "mpeg2video (Main)"  temp$$.txt | head -n1 | cut -f 1,1 -d']' | sed 's+.*\[++g'`
   # It has to be tweaked for multiple audio streams.  This (with head -n1 ) selects the first listed by ffmpeg.
   # You may alternatively wish to select for language, format, etc.   May be channel, programme, user dependent.
   APID=`grep Audio  temp$$.txt | head -n1 | cut -f 1,1 -d']' | sed 's+.*\[++g'`

   echo -e "Choosing the first audio track listed by \" mythffmpeg -i \".  It may not be the one you want."
   echo -e "\nThe selected values would be "$VPID" and "$APID".  The track info for these is \n"

   grep "$VPID" temp$$.txt
   grep "$APID" temp$$.txt

   echo -e "\nTo accept these values press \"a\", or wait....\n"  
   echo  "If you want to select other values, or to quit and think about it, press another key within $TIMEOUT seconds."
   echo -e "If the format is not mpeg2video all streams will be passed unchanged.\n"
 
   read -t $TIMEOUT -n 1 RESP
   if  [ $? -gt 128 ] ; then    
       RESP="a"
   fi

   if [ "$RESP" != "a" ] ; then
       echo -e "Quitting: if you want to select the PIDs from the command line its expected form is   \n"
       echo " "$0" 1234_20070927190000.ts (or .mpg)  0xvvv 0xaaa " 
       echo -e "                    filename_in_DB           vPID  aPID \n" 
       cd ~
       exit 1
   fi

   echo -e "Going on: processing with suggested values $VPID  $APID \n"
   grep "$VPID" temp$$.txt
   grep "$APID" temp$$.txt
   echo
else
   VPID="$2"
   APID="$3"
fi
########################
# Now do the actual processing

# recordedid is now the prime search key in the DB.
recordedid=$(echo "select recordedid from recorded where basename=\"$BN\";" |
mysql -N -u${DBUserName} -p${DBPassword} -h${DBLocalHostName} $DBName )

# but some tables and utils still need chanid and starttime.
chanid=$(echo "select chanid from recorded where recordedid = '$recordedid' ;" |
mysql -N -u${DBUserName} -p${DBPassword} -h${DBLocalHostName} $DBName )

starttime=$(echo "select starttime from recorded where recordedid = '$recordedid' ;" |
mysql -N -u${DBUserName} -p${DBPassword} -h${DBLocalHostName} $DBName )

#exit
echo -e "\nLogfile listing:\n" > log$$
echo -e "Logfile is log$log$$ \n" | tee -a log$$

echo "chanid ${chanid}   starttime ${starttime}  recordedid ${recordedid} "  >> log$$
starttime=$(echo  ${starttime} | tr -d ': -')

echo "Reformatted starttime ${starttime} " >> log$$

command="mythutil --getcutlist --chanid  $chanid --starttime $starttime -q"
echo "Running: "${command}"" >> log$$
mythutilcutlist=$($(echo ${command}))
echo "${mythutilcutlist}" |tee -a  log$$

shortlist=$(echo ${mythutilcutlist} | sed 's/Cutlist://' )

echo -e "\nIf you want to reset this cutlist, after restoring the .old file 
and resetting the DB with mythsetsize, you could try: \n
mythutil --setcutlist ${shortlist} --chanid  $chanid --starttime $starttime \n
-but seektables from different tools may differ!!" | tee -a log$$

#exit 0

if [ "${mythutilcutlist}" = "Cutlist: " ] ; then
  echo "Cutlist was empty; inserting dummy EOF" >> log$$
  mythutilcutlist=" 9999999 "
fi

echo -e "\nCutframe list from editor: " >> log$$
echo "${mythutilcutlist}" | tr -d [:alpha:] | tr [:punct:] " " | tee  edlist$$ >> log$$

#cat edlist$$ | tee -a log$$
echo

#
#Reverse the sense of the cutlist 
#
echo -n > revedlist$$
for i in  $(cat edlist$$) ; 
do
   if  [ $i -eq 0 ] ;  then
      for j in  $(cat edlist$$) ; 
      do
         if  [ $j != 0 ] ; then echo -n "$j " >> revedlist$$
         fi
      done
   else
      echo -n " 0 " >> revedlist$$
      for j in  $(cat edlist$$) ; 
      do
          echo -n "$j " >> revedlist$$
      done
   fi
   break
done
echo >> revedlist$$
echo -e "\nPassframe (reversed cutframe) list from editor: " >> log$$
cat revedlist$$ | tee -a log$$
 
# For a byte-count cutlist, (PX CutMode=0)
# mark is the frame count in the seektable and is compared here with the frame count defined by the cutlist editor.
# mark type 9 is MARK_GOP_BYFRAME (see eg trac ticket #1088) with adjacent values typically separated by 12 or more,
# so that is presumably the cutpoint frame granularity in bytecount mode. 
# Subjectively the granularity seems smaller, but this may not apply to locally encoded recordings; 
# in dvb-t and similar systems the position of keyframes often appears to depend on pre-transmission edits of content. 
#
# Find the next keyframe (mark type 9) after a position slightly before the frame identified by the cutlist editor.
# The original version, which I have been using for years, effectively had lag=0.
# The 'recordedseek' table doesn't know about recordedid.

lag=4        #  In frames.  Best value might depend on recording source, MythTV version and seektable history.   
scope=2000   #  Sometimes h264 keyframes in the wild are much more widely spaced than expected.
             #  This might only have been true for 'rebuilt' seektables, but the large value should do no harm.
             
for frame in $(cat revedlist$$) 
do
    i=$((${frame} - ${lag})) 
    j=$((${i})) 
    k=$((${i} + ${scope}))
    echo  "select offset, mark from recordedseek
    where chanid=$chanid and starttime='$starttime' and type=9 
    and mark >= ${j} and mark < ${k}  order by offset limit 3 ;" |
    mysql -N -u${DBUserName} -p${DBPassword} -h${DBLocalHostName} ${DBName}          
done > tmp0$$

echo "Full results of DB read:"
cat tmp0$$
cat tmp0$$  | sed -n '1,${p;n;n;}' > tmp1$$  # select lines 1,4,7...
cat tmp0$$  | sed -n '2,${p;n;n;}' > tmp2$$  # 2,5,8...
cat tmp0$$  | sed -n '3,${p;n;n;}' > tmp3$$
rm tmp0$$
             
echo

# write the byte offset and frame number into one-line cutlists.

echo -e "\nActive keyframe passlist via DB.  First is a cut-in:" >> log$$
cut -f2 tmp1$$ | tr "\n" " "  | tee -a log$$ > keylist$$
echo >> keylist$$
echo >> log$$

echo -e "\n2nd keyframes, via DB.  For info only." >> log$$
cut -f2 tmp2$$ |  tr "\n" " "  >> log$$
echo >> log$$

echo -e "\n3rd keyframes, via DB.  For info only." >> log$$
cut -f2 tmp3$$ |  tr "\n" " "  >> log$$
echo >> log$$

echo -e "\nByte offsets of switchpoints in original file, via DB.  First is a cut-in:" >> log$$
cut -f1 tmp1$$ |  tr "\n" " "  >> log$$     
echo >> log$$

echo -e "\nByte offsets of 2nd keyframes, via DB.  Compare with PjX log." >> log$$
cut -f1 tmp2$$ |  tr "\n" " "  >> log$$ 
echo >> log$$

mv tmp1$$ tmp$$  # for use

rm  tmp2$$  tmp3$$ 

# Now apply a one-packet offset to the byte positions passed to Project-X.
# With this 188-byte adjustment in place, the byte-positions that PjX reports 
# having used are exactly equal to those held in the myth DB for 
# the keyframe that next follows the one selected:- i.e. it seems to report the 
# end-of-GOP position when that GOP has been processed. 

# Editing may be frame-accurate if edit-points are the first or last keyframes
# for which a wanted picture is displayed.

FILESIZE=$( du -bL "$BN" | cut -f 1 )

echo -e "\nCreating the cutlist for Project-X with 188 byte adjustment" >> log$$
cut -f1 tmp$$ > PXraw$$
for i in $( cat PXraw$$ ) ;
do 
 echo $(( $i + 188 ))
done > PXadj$$

rm tmp$$

echo "PXraw$$"
cat PXraw$$
echo "EOF"
echo "PXadj$$"
cat PXadj$$
echo "EOF"

# Add a limiting filesize value for use if EOF will be reached
J=$( cat PXraw$$ | wc -l )
echo "PXraw$$ has $J lines"
if [ $(( $J % 2 )) -eq 1 ] ; then
   echo "Adding EOF endmark"
   echo "$FILESIZE" >> PXraw$$
   echo "$FILESIZE" >> PXadj$$
fi

echo "###########  Start of recording " | tee -a log$$
hexdump -C -n 40 -s 0 ${1} | tee -a log$$

echo "###########  PXraw$$ ### " | tee -a log$$
for i in $( cat PXraw$$ ) ;
do 
  echo ${i} | tee -a log$$
  hexdump -C -n 40 -s  $i ${1} | tee -a log$$
done

echo "##########  PXadj$$ ###" | tee -a log$$
for i in $( cat PXadj$$ ) ;
do 
  echo ${i}  | tee -a log$$
  hexdump -C -n 40 -s  $i ${1}  | tee -a log$$
done


# Set up Project-X for bytecount mode
echo "CollectionPanel.CutMode=0" > projx$$
cat PXadj$$ >> projx$$

echo
echo >> log$$

cat projx$$ | tee -a log$$

# Don't apply the 188-byte offset when using pyscript
cat PXraw$$ | tr "\n" " " > bytelist$$
echo >> bytelist$$

# Prepare pyscript$$, the script that would run pycut.py
echo -e " #!/bin/bash\n mv  '$RECDIR/$BN' '$RECDIR/$BN.old' "  > pyscript$$
echo -en " ionice -c3 ~/pycut.py '$RECDIR/$BN.old' '$RECDIR/$BN' " >> pyscript$$
cat bytelist$$  >>  pyscript$$  

if ${USEPJX} ; then
   # Do apply the offset
   cat PXadj$$ | tr "\n" " " > bytelist$$
   echo >> bytelist$$
fi

rm PXraw$$ PXadj$$

# These calculations work only if no streams or frames have been dropped
# so should probably be 'if ! ${USEPJX} ; then'
if ! ${USEPJX} ; then
  echo -e "\nSwitchbyte positions in new file:" >> log$$
  J=0
  S=0                           # 0 or 1 for cut or pass lists
  for i in  $(cat bytelist$$) ;  
  do 
    if [ $S -eq 0 ] ; then
        J=$((J - i))           
        S=1
    else
        J=$((J + i))
        S=0
        echo -n "$J " >> log$$
     fi
  done 
  echo >> log$$

  echo -e "\nSwitchframe positions in new file:" >> log$$
  J=0
  S=0                           # 0 or 1 for cut or pass lists
  for i in  $(cat keylist$$) ;  
  do 
    if [ $S -eq 0 ] ; then
        J=$((J - i))           
        S=1
    else
        J=$((J + i))
        S=0
        echo -n "$J " >> log$$
     fi
  done 
  echo >> log$$
fi
echo
cat log$$

# create script to be run
echo -e "\nThis is pyscript$$, the script that will be run if TESTRUN is false 
and you are not trying to use PjX instead:\n"
cat pyscript$$
chmod +x pyscript$$
echo -e "\nTo run this use ./pyscript$$. \n"

if $TESTRUN ; then
   echo "Quitting because TESTRUN is ${TESTRUN}"
   rm -f cutlist$$
   rm -f temp$$   
   cd ~
   exit 0
fi
###########################

# Now do the actual cutting and concatenation

TEMPHEAD=$TEMP/tempcut${$}
OUTFILEHEAD=$( echo $BN  | tr -d [:alpha:] | tr -d [.] )

if ${USEPJX} ; then

   #################

   # For mpeg2 format only....
   # use ProjectX to de-multiplex selected streams with the created cutlist

   mv  "$BN" "$BN".old

   CMD="ionice -c3 java -jar "$PROJECTX" -name tempcut$$ -id ${VPID},${APID} \
   -out $TEMP -cut projx$$ "$BN".old"
   echo "running: "${CMD}""
   ${CMD}
   # 

   # if demuxed audio is in .wav format (maybe direct camera output) convert it to mp2
   if [ -f $TEMPHEAD.wav ] ; then
      twolame $TEMPHEAD.wav
   fi

   if [ -f $TEMPHEAD.mp2 ] ; then
       TEMPAUDIO=$TEMPHEAD.mp2
   else
       TEMPAUDIO=$TEMPHEAD.ac3
   fi

   if ! ${MAKEMKV} ; then

     # Now remux to MPEG2_PS.  
     # mplex -f 8, mplex -f 9 both use DVD profile. 
     # mplex -f 8 inserts blank DVD-nav sectors, -f 9 does not. 
     #
     # This seems to be a 'corner case' infrequent warning from mplex.  
     # I have never detected a problem during playback but it may make MythArchive fail. 
     # ++ WARN: [mplex] Stream e0: data will arrive too late sent(SCR)=7314 required(DTS)=7200
     # ++ WARN: [mplex] Video e0: buf= 101237 frame=000001 sector=00000050

     OUTFILE=$OUTFILEHEAD.mpg
     echo "Newfile name will be:  $OUTFILE "
     CMD="ionice -c3  mplex -o "$OUTFILE" -V -f 9 $TEMPHEAD.m2v $TEMPAUDIO" 
     echo "running: ${CMD}"
     ${CMD}
   else
     # Remux to .mkv
     # This does play as a recording via uPnP but won't seek or skip in Mythfrontend.

#     OUTFILE=$( echo $BN | sed 's/mpg/mkv/')
     OUTFILE=$OUTFILEHEAD.mkv
     echo "Newfile name will be:  $OUTFILE "
     CMD="ionice -c3 mythffmpeg -fflags +genpts -i $TEMPHEAD.m2v -i $TEMPAUDIO -vcodec copy -acodec copy $OUTFILE " 
     echo "running: ${CMD}"
     ${CMD}
   fi 
   
   # tell mythDB about new filename and set the 'transcoded' flag
   echo "update recorded set basename='${OUTFILE}', transcoded = 1 where recordedid = '$recordedid' ; " |
   mysql -N -u${DBUserName} -p${DBPassword} -h${DBLocalHostName} ${DBName}
       

   # Update the container type, needed for uPnP playback.  Empty will work too, if the transcoded flag is set.
   # This is a HACK pending 0.28 release, when mythutil should do it.  See Ticket #12388
   
   echo "update recordedfile set basename = '${OUTFILE}', container = 'MPEG2-PS' where recordedid = '$recordedid' ;" |
   mysql -N -u${DBUserName} -p${DBPassword} -h${DBLocalHostName} ${DBName}

   CMD="rm  $TEMPHEAD.m2v   $TEMPAUDIO "  # Large; usually best to remove these. 
   echo "running: "${CMD}""
   ${CMD}

   ################

else

   ################
   # Use pyscript, which isn't restricted to mpeg2; but PjX can repair some mpeg2 defects.
   echo "Running : " 
   cat pyscript$$
   echo
   echo
   ./pyscript$$
   echo
   ###############
   OUTFILE=$BN
fi

# Cutting completed.  Now clean up.
CMD="ionice -c3 mythutil  --clearcutlist  --chanid "$chanid" --starttime "$starttime" -q "
echo "running: "${CMD}""
${CMD}
echo -e "Cutlist has been cleared.\n" 

# Rebuild seek table; this now also resets filesize in DB 
# TODO? With MKV output, seektable is probably not useful and may cause problems. 
CMD="ionice -c3 mythcommflag -q --rebuild --file $OUTFILE "
echo "running: "${CMD}""
${CMD}
#echo -e "Seek table has been rebuilt.\n"

#echo  "The cutlist was applied in ** "$CUTMODE" ** mode."
echo -e "Output file is $OUTFILE. \n" 

# Get tech details of output file into the log.
echo -e "\nRunning:  mythffmpeg -i "$OUTFILE" 2>&1 | grep -C 4 Video" | tee -a log$$
echo
mythffmpeg -i "$OUTFILE" 2>&1 | grep -C 4 Video | tee -a log$$
echo

CMD="grep -A 2 Switch log$$ " # put non-ProjectX outfile switchpoints onto terminal
echo "running: "${CMD}""
${CMD}

# The PjX-listed switchpoints are usually 2 frames further apart than those calculated above.
# The editor often sees them about 1 GOP earlier.  Dropped frames and/or seektable problems?  
echo "outfile switchpoints listed by ProjectX"
grep 'cut-in' ${TEMPHEAD}_log.txt 
grep '.Video (m2v):' ${TEMPHEAD}_log.txt  \
   |  awk '{print "Overall :                       " $3, $4,"    " $5}'
cat log$$ >> ${TEMPHEAD}_log.txt

echo -e "\nWhile the .old file still exists you can examine it with\n
hexdump -C -n 40 -s {byteoffset} ${RECDIR}/$BN.old\n
32-bit hexdump may misinterpret start offsets > 2 GiB.\n "

grep "we have" ${TEMPHEAD}_log.txt  # show PX 'errorcount' nearer to end of screen output.
# Values up to a few tens are usually ok, but rare backward jumps in timestamps, 
# which may truncate PX output, give much larger numbers.
# 
echo

CMD="mv ${TEMPHEAD}_log.txt  ${LOGDIR}/${OUTFILE}.PXcut$$.txt"
echo "running: "${CMD}""
${CMD}

rm bytelist$$
rm edlist$$
rm keylist$$
rm revedlist$$
rm pyscript$$
rm projx$$
rm temp$$.txt
rm log$$
rm -f $BN.png   # old preview, which may not exist
cd ~

# mythpreviewgen isn't essential here so put it where failure won't cause other problems.
# Creates a blank frame from .mkv file.

CMD="ionice -c3 mythpreviewgen --chanid "$chanid" --starttime "$starttime" -q "
echo "running: "${CMD}""
${CMD}
echo -e "Preview created.\n"

exit 0


Script.png mythDVBcut_20130704.sh

#!/bin/bash

# Copyright (C) 2013 John Pilkington 
# Uses ideas from scripts posted by Tino Keitel and Kees Cook in the Mythtv lists.
# The suggestion to convert HD files to .mkv, and code to do it, came from Michael Stucky.

# **** I have not tried running this script on an .mkv file that it has created *****

# Usage: "$0" <recording>   ...or...
# ionice -c3 ./"$0" <recording> will reduce io priority and is recommended.
# <recording> is a file recorded by MythTV with a valid DB entry and seektable.
# e.g. 1234_20100405123400.mpg in one of the defined RECDIRs
# The output file replaces the input file which is renamed to <recording>.old

# This script is essentially a terminal-based replacement for the 'lossless' mpeg2 mythtranscode.
# It was developed from mythcutprojectx but will now cut some recording formats that defeat Project-X.
# All cuts are made at keyframes.  

# Project-X is used here to apply a cutlist, and also to remove all but one video and one audio stream.
# If the audio is initially in .wav format it will be converted to .mp2; mp2 or ac3 will be unchanged.
# For non-mpeg2video all streams are passed through unchanged, but with cuts applied at the video keyframes.  
  
# It then clears the cutlist, updates the filesize in the database, rebuilds the seek table and creates a new preview.

# If the script is edited to have MAKEMKV=false, the result should be acceptable as a recording within MythTV
# and perhaps as an input to MythArchive.  After MAKEMKV=true the file must be treated as a Video.

# The logfile includes the positions in the new file at which deletions have been made. 

# The script needs to be edited to define some local variables and folders.  
# It will not recognise Storage Groups.

# A tarball of the most recent version of Project-X can be downloaded from the link at the bottom of this page:

# http://project-x.cvs.sourceforge.net/viewvc/project-x/Project-X/

# Some of the comments in this script have been left as reminders of things that might be useful.
 
####################

#. ~/.mythtv/mysql.txt # for DB access info
#PASSWD=`grep "^DBPassword" ~/.mythtv/mysql.txt | cut -d '=' -f 2-`
#PASSWD=mythtv

# mysql.txt is no longer referenced in 0.26+
# The values are now defined within ~/.mythtv/config.xml but it seems easier to
# get them into the script by editing.  They are:

# DBUserName, DBPassword, DBLocalHostName, DBName

# Define the variables used for DB access
#
DBUserName="mythtv"
DBPassword="mythtv" 
DBLocalHostName="localhost" 
DBName="mythconverg"

 
BN="$1"     # "Basename" of file as given in the command line but without special attributes.

MAKEMKV=false   # after ProjectX use mplex to create a DVD profile file
# mplex fails with higher bitrates, but mkv can cope.
# MAKEMKV=true     # after ProjectX create an .mkv file

# If TESTRUN is set to true, no changes will be made to the input file.

# TESTRUN=true    # cutlists will be shown but the recording will be unchanged 
TESTRUN=false   # the recording will be processed
if [ $TESTRUN != "false" ] ; then TESTRUN=true
fi
#CUTMODE="FRAMECOUNT"  # Not used here.
CUTMODE="BYTECOUNT"

USEPJX=true  # Will be set to false if not mpeg2video (Main)
TIMEOUT=20   # Longest 'thinking time' in seconds allowed before adopting the automatically selected audio stream.

# Variables RECDIR1, TEMPDIR1, RECDIR2, TEMPDIR2, LOGDIR, PROJECTX, HOST1, HOST2 need to be customised.
# The TEMPDIRs are used to hold the demuxed recording, so should have a few GiB available. 

HOST1="PresV5000"  # laptop with single disk, SL6, MythTV 0.26-fixes FE/BE
HOST2="gateway12"  # twin-disk box, Fedora 17, MythTV 0.26-fixes FE/BE

if [ $HOSTNAME = ${HOST1} ] ; then

   RECDIR1=/home/john/Mythrecs
   TEMPDIR1=/home/john/MythTmp

   RECDIR2=${RECDIR1}
   TEMPDIR2=${TEMPDIR1}

   LOGDIR=/home/john/Logs

   #PROJECTX=/path/to/ProjectX.jar (or to a link to it)
   PROJECTX=~/projectx_link

elif [ $HOSTNAME = ${HOST2} ] ; then

   # Section below for twin-disk setup using Project-X. 

   # RECDIR1 and TEMPDIR1 should if possible be on different drive spindles.  Likewise RECDIR2 and TEMPDIR2.

   RECDIR1=/mnt/f10store/myth/reca
   TEMPDIR1=/mnt/sam1/tempb

   RECDIR2=/mnt/sam1/recb
   TEMPDIR2=/mnt/f10store/myth/tempa

   LOGDIR=/home/John/Documents/PXcutlogs

   #PROJECTX=/path/to/ProjectX.jar (or to a link to it)
   PROJECTX=~/projectx_link

else
   echo "Hostname $HOSTNAME not recognised."
   exit 1
fi 

if [ "$BN" = "-h" ] || [ "$BN" = "--help" ] ; then
echo "Usage: "$0" <recording>"
echo "<recording> is a file recorded by MythTV with a valid DB entry and seektable."
echo "e.g. 1234_20100405123400.mpg in one of the defined RECDIRs"
echo "The output file replaces the input file which is renamed to <recording>.old"
exit 0
fi

# exit if .old file exists

if  [ -f ${RECDIR1}/"$BN".old ] ; then 
    echo " ${RECDIR1}/"$BN".old exists: giving up." ; exit 1
fi

if  [ -f ${RECDIR2}/"$BN".old ] ; then 
    echo " ${RECDIR2}/"$BN".old exists: giving up." ; exit 1
fi
 
# Customize with paths to alternative recording and temp folders

cd ${RECDIR1}
RECDIR=${RECDIR1}
TEMP=${TEMPDIR1}
if  [ ! -f "$BN" ] ; then
  cd ${RECDIR2}
  RECDIR=${RECDIR2}
  TEMP=${TEMPDIR2} 
     if  [ ! -f "$BN" ] ; then 
       echo " "$BN" not found.  Giving up"
       cd ~
       exit 1
     fi
fi

# Is it an mpeg-2 recording?
echo
echo
#mythffmpeg -i "$BN" 2>&1 | grep -C 4 Video | tee "temp$$.txt"
#mythffmpeg -i "$BN" 2>&1 | grep -B 2 -A 4 Video | tee "temp$$.txt"
mythffmpeg -i "$BN" 2>&1 | grep -B 2 -A 4 "mpeg2video (Main)" | tee "temp$$.txt"

mpeg=$(grep -c "mpeg2video (Main)" temp$$.txt) 
echo "mpeg2video (Main) stream count:  $mpeg"
if [ $mpeg != 1 ] ; then 
  echo "Not mpeg2video (Main), or no or multiple video streams"
  if [ $mpeg = 0 ] ; then
     USEPJX=false  # really ought to see if it's a radio channel.
  else
     exit 1
  fi
fi

if [ $# -lt 3  ]
then
   echo "Needs one or three arguments." 
   echo  
   cat temp$$.txt
   echo

#    Examples
#    Stream #0.0[0xe0]: Video: mpeg2video, yuv420p, 720x576 [PAR 64:45 DAR 16:9], 15000 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
#    Stream #0.1[0xc0](deu): Audio: mp2, 48000 Hz, 2 channels, s16, 256 kb/s

   # Thanks to Christopher Meredith for the basic parsing magic here. 
   VPID=`grep "mpeg2video (Main)"  temp$$.txt | head -n1 | cut -f 1,1 -d']' | sed 's+.*\[++g'`
   # It has to be tweaked for multiple audio streams.  This (with head -n1 ) selects the first listed by ffmpeg.
   # You may alternatively wish to select for language, format, etc.   May be channel, programme, user dependent.
   APID=`grep Audio  temp$$.txt | head -n1 | cut -f 1,1 -d']' | sed 's+.*\[++g'`

   echo -e "Choosing the first audio track listed by \" mythffmpeg -i \".  It may not be the one you want."
   echo -e "\nThe selected values would be "$VPID" and "$APID".  The track info for these is \n"

   grep "$VPID" temp$$.txt
   grep "$APID" temp$$.txt

   echo -e "\nTo accept these values press \"a\", or wait....\n"  
   echo  "If you want to select other values, or quit to think about it, press another key within $TIMEOUT seconds."
   echo -e "If the format is not mpeg2video all streams will be passed unchanged.\n"
 
   read -t $TIMEOUT -n 1 RESP
   if  [ $? -gt 128 ] ; then    
       RESP="a"
   fi

   if [ "$RESP" != "a" ] ; then
       echo -e "Quitting: if you want to select the PIDs from the command line its expected form is   \n"
       echo " "$0" 1234_20070927190000.mpg  0xvvv 0xaaa " 
       echo -e "                    filename_in_DB           vPID  aPID \n" 
       cd ~
       exit 1
   fi

   echo -e "Going on: processing with suggested values $VPID  $APID \n"
   grep "$VPID" temp$$.txt
   grep "$APID" temp$$.txt
   echo
else
   VPID="$2"
   APID="$3"
fi

# Now do the actual processing
# chanid and starttime identify the recording in the DB
chanid=$(echo "select chanid from recorded where basename=\"$BN\";" |
mysql -N -u${DBUserName} -p${DBPassword} -h${DBLocalHostName} $DBName )

starttime=$(echo "select starttime from recorded where basename=\"$BN\";" |
mysql -N -u${DBUserName} -p${DBPassword} -h${DBLocalHostName} $DBName )

#exit
echo -e "\nLogfile listing:\n" > log$$
echo -e "Logfile is log$log$$ \n" | tee -a log$$

echo "chanid ${chanid}   starttime ${starttime} "  >> log$$
starttime=$(echo  ${starttime} | tr -d ': -')

echo "Reformatted starttime ${starttime} " >> log$$

command="mythutil --getcutlist --chanid  $chanid --starttime $starttime -q"
echo "Running: "${command}"" >> log$$
mythutilcutlist=$($(echo ${command}))
echo "${mythutilcutlist}" |tee -a  log$$

shortlist=$(echo ${mythutilcutlist} | sed 's/Cutlist://' )

echo -e "\nIf you want to reset this cutlist, after restoring the .old file 
and resetting the DB with mythsetsize, you could try: \n
mythutil --setcutlist ${shortlist} --chanid  $chanid --starttime $starttime \n
-but seektables from different tools may differ!!" | tee -a log$$

#exit 0

if [ "${mythutilcutlist}" = "Cutlist: " ] ; then
  echo "Cutlist was empty; inserting dummy EOF" >> log$$
  mythutilcutlist=" 9999999 "
fi

echo -e "\nCutframe list from editor: " >> log$$
echo "${mythutilcutlist}" | tr -d [:alpha:] | tr [:punct:] " " | tee  edlist$$ >> log$$

#cat edlist$$ | tee -a log$$
echo

#
#Reverse the sense of the cutlist 
#
echo -n > revedlist$$
for i in  $(cat edlist$$) ; 
do
   if  [ $i -eq 0 ] ;  then
      for j in  $(cat edlist$$) ; 
      do
         if  [ $j != 0 ] ; then echo -n "$j " >> revedlist$$
         fi
      done
   else
      echo -n " 0 " >> revedlist$$
      for j in  $(cat edlist$$) ; 
      do
          echo -n "$j " >> revedlist$$
      done
   fi
   break
done
echo >> revedlist$$
echo -e "\nPassframe (reversed cutframe) list from editor: " >> log$$
cat revedlist$$ | tee -a log$$
 
# For a byte-count cutlist, (PX CutMode=0)
# mark is the frame count in the seektable and is compared here with the editpoint
# mark type 9 is MARK_GOP_BYFRAME (see eg trac ticket #1088) with adjacent values typically separated by 12 or more,
# so that is presumably the cutpoint frame granularity in bytecount mode. 
# Subjectively the granularity seems smaller, but this may not apply to locally encoded recordings; 
# in dvb-t and similar systems the spacing of keyframes apparently depends on pre-transmission edits of content. 
#
# Find the next keyframe (mark type 9) after a position slightly before the cutmark found by the editor.
# The original version, which I have been using for years, effectively had lag=0.

lag=4        #  In frames.  Best value might depend on recording source, MythTV version and seektable history.   
scope=2000   #  Sometimes h264 keyframes in the wild are much more widely spaced than expected.

for frame in $(cat revedlist$$) 
do
    i=$((${frame} - ${lag})) 
    j=$((${i})) 
    k=$((${i} + ${scope}))
    echo  "select offset, mark from recordedseek
    where chanid=$chanid and starttime='$starttime' and type=9 
    and mark >= ${j} and mark < ${k}  order by offset limit 3 ;" |
    mysql -N -u${DBUserName} -p${DBPassword} -h${DBLocalHostName} ${DBName}          
done > tmp0$$

echo "Full results of DB read:"
cat tmp0$$
cat tmp0$$  | sed -n '1,${p;n;n;}' > tmp1$$  # select lines 1,4,7...
cat tmp0$$  | sed -n '2,${p;n;n;}' > tmp2$$
cat tmp0$$  | sed -n '3,${p;n;n;}' > tmp3$$
rm tmp0$$
             
echo

# write the byte offset and frame number into one-line cutlists.

echo -e "\nActive keyframe passlist via DB.  First is a cut-in:" >> log$$
cut -f2 tmp1$$ | tr "\n" " "  | tee -a log$$ > keylist$$
echo >> keylist$$
echo >> log$$

echo -e "\n2nd keyframes, via DB.  For info only." >> log$$
cut -f2 tmp2$$ |  tr "\n" " "  >> log$$
echo >> log$$

echo -e "\n3rd keyframes, via DB.  For info only." >> log$$
cut -f2 tmp3$$ |  tr "\n" " "  >> log$$
echo >> log$$

echo -e "\nByte offsets of switchpoints in original file, via DB.  First is a cut-in:" >> log$$
cut -f1 tmp1$$ |  tr "\n" " "  >> log$$     
echo >> log$$

echo -e "\nByte offsets of 2nd keyframes, via DB.  Compare with PjX log." >> log$$
cut -f1 tmp2$$ |  tr "\n" " "  >> log$$ 
echo >> log$$

mv tmp1$$ tmp$$  # for use

rm  tmp2$$  tmp3$$ 

# With this one-packet-length adjustment in place, the byte-positions that PjX reports 
# having used are exactly equal to those held in the myth DB for 
# the keyframe that next follows the one selected:- i.e. it seems to report the 
# end-of-GOP position when that GOP has been processed. 

# Editing may be frame-accurate if edit-points are the first or last keyframes
# for which a wanted picture is displayed.

FILESIZE=$( du -bL "$BN" | cut -f 1 )

echo -e "\nCreating the cutlist for Project-X with 188 byte adjustment" >> log$$
cut -f1 tmp$$ > PXraw$$
for i in $( cat PXraw$$ ) ;
do 
 echo $(( $i + 188 ))
done > PXadj$$

rm tmp$$

echo "PXraw$$"
cat PXraw$$
echo "EOF"
echo "PXadj$$"
cat PXadj$$
echo "EOF"

# Add a limiting filesize value for use if EOF will be reached
J=$( cat PXraw$$ | wc -l )
echo "PXraw$$ has $J lines"
if [ $(( $J % 2 )) -eq 1 ] ; then
   echo "Adding EOF endmark"
   echo "$FILESIZE" >> PXraw$$
   echo "$FILESIZE" >> PXadj$$
fi

echo "###########  Start of recording " | tee -a log$$
hexdump -C -n 40 -s 0 ${1} | tee -a log$$

echo "###########  PXraw$$ ### " | tee -a log$$
for i in $( cat PXraw$$ ) ;
do 
  echo ${i} | tee -a log$$
  hexdump -C -n 40 -s  $i ${1} | tee -a log$$
done

echo "##########  PXadj$$ ###" | tee -a log$$
for i in $( cat PXadj$$ ) ;
do 
  echo ${i}  | tee -a log$$
  hexdump -C -n 40 -s  $i ${1}  | tee -a log$$
done

echo "CollectionPanel.CutMode=0" > projx$$
cat PXadj$$ >> projx$$

echo
echo >> log$$

cat projx$$ | tee -a log$$

cat PXadj$$ | tr "\n" " " > bytelist$$
echo >> bytelist$$

# Prepare pyscript$$, the script that would run pycut.py
echo -e " #!/bin/bash\n mv  '$RECDIR/$BN' '$RECDIR/$BN.old' "  > pyscript$$
echo -en " ionice -c3 ~/pycut.py '$RECDIR/$BN.old' '$RECDIR/$BN' " >> pyscript$$
cat bytelist$$  >>  pyscript$$  

rm PXraw$$ PXadj$$ 

echo -e "\nSwitchbyte positions in new file:" >> log$$
J=0
S=0                           # 0 or 1 for cut or pass lists
for i in  $(cat bytelist$$) ;  
do 
  if [ $S -eq 0 ] ; then
      J=$((J - i))           
      S=1
  else
      J=$((J + i))
      S=0
      echo -n "$J " >> log$$
   fi
done 
echo >> log$$

echo -e "\nSwitchframe positions in new file:" >> log$$
J=0
S=0                           # 0 or 1 for cut or pass lists
for i in  $(cat keylist$$) ;  
do 
  if [ $S -eq 0 ] ; then
      J=$((J - i))           
      S=1
  else
      J=$((J + i))
      S=0
      echo -n "$J " >> log$$
   fi
done 
echo >> log$$
echo
cat log$$

# create script to be run
echo -e "\nThis is pyscript$$, the script that will be run if TESTRUN is false 
and you are not trying to use PjX instead:\n"
cat pyscript$$
chmod +x pyscript$$
echo -e "\nTo run this use ./pyscript$$. \n"

if $TESTRUN ; then
   echo "Quitting because TESTRUN is ${TESTRUN}"
   rm -f cutlist$$
   rm -f temp$$   
   cd ~
   exit 0
fi

# Now do the actual cutting and concatenation

TEMPHEAD=$TEMP/tempcut${$}
OUTFILE="$BN"

if ${USEPJX} ; then

   #################

   # For mpeg2 format only....
   # use ProjectX to de-multiplex selected streams with the created cutlist

   mv  "$BN" "$BN".old

   CMD="ionice -c3 java -jar "$PROJECTX" -name tempcut$$ -id ${VPID},${APID} \
   -out $TEMP -cut projx$$ "$BN".old"
   echo "running: "${CMD}""
   ${CMD}
   # 

   # if demuxed audio is in .wav format (maybe direct camera output) convert it to mp2
   if [ -f $TEMPHEAD.wav ] ; then
      twolame $TEMPHEAD.wav
   fi

   if [ -f $TEMPHEAD.mp2 ] ; then
       TEMPAUDIO=$TEMPHEAD.mp2
   else
       TEMPAUDIO=$TEMPHEAD.ac3
   fi

   if ! ${MAKEMKV} ; then

     # Now remux.  mplex -f 8, mplex -f 9 both use DVD profile. 
     # mplex -f 8 inserts blank DVD-nav sectors, -f 9 does not.  
     # Neither plays as a 'Recording' via uPnP on my Panasonic TV.

     CMD="ionice -c3  mplex -o "$OUTFILE" -V -f 9 $TEMPHEAD.m2v $TEMPAUDIO" 
     echo "running: ${CMD}"
     ${CMD}
   else
     # Remux to .mkv
     # This does play as a recording via uPnP but won't seek or skip in Mythfrontend.

     OUTFILE=$( echo $BN | sed 's/mpg/mkv/')
     echo "Newfile name will be:  $OUTFILE "
     CMD="ionice -c3 mythffmpeg -fflags +genpts -i $TEMPHEAD.m2v -i $TEMPAUDIO -vcodec copy -acodec copy $OUTFILE " 
     echo "running: ${CMD}"
     ${CMD}

     # tell mythDB about new filename
     echo "update recorded set basename='${OUTFILE}' where chanid=$chanid and starttime='$starttime';" |
     mysql -N -u${DBUserName} -p${DBPassword} -h${DBLocalHostName} ${DBName}
   fi

   CMD="rm  $TEMPHEAD.m2v   $TEMPAUDIO "  # Large; usually best to remove these. 
   echo "running: "${CMD}""
   ${CMD}

   ################

else

   ################
   # Use pyscript, which isn't restricted to mpeg2; but PjX can repair some mpeg2 defects.
   echo "Running : " 
   cat pyscript$$
   echo
   echo
   ./pyscript$$
   echo
   ###############
fi

# Cutting completed.  Now clean up.
CMD="ionice -c3 mythutil  --clearcutlist  --chanid "$chanid" --starttime "$starttime" -q "
echo "running: "${CMD}""
${CMD}
echo -e "Cutlist has been cleared.\n" 

#rebuild seek table; this now also resets filesize in DB 
CMD="ionice -c3 mythcommflag -q --rebuild --file $OUTFILE "
echo "running: "${CMD}""
${CMD}
#echo -e "Seek table has been rebuilt.\n"

#echo  "The cutlist was applied in ** "$CUTMODE" ** mode."
echo -e "Output file is $OUTFILE. \n" 

# Get tech details of output file into the log.
echo -e "\nRunning:  mythffmpeg -i "$OUTFILE" 2>&1 | grep -C 4 Video" | tee -a log$$
echo
mythffmpeg -i "$OUTFILE" 2>&1 | grep -C 4 Video | tee -a log$$
echo

CMD="grep -A 2 Switch log$$ " # put outfile switchpoint info onto terminal
echo "running: "${CMD}""
${CMD}

# The PjX-listed switchpoints are usually 2 frames further apart than those calculated above.
# The editor often sees them about 1 GOP earlier.  Dropped frames and/or seektable problems?  
echo "outfile switchpoints listed by PjX"
grep 'cut-' ${TEMPHEAD}_log.txt 
cat log$$ >> ${TEMPHEAD}_log.txt

echo -e "\nWhile the .old file still exists you can examine it with\n
hexdump -C -n 40 -s {byteoffset} ${RECDIR}/$BN.old\n
On my 32-bit installation, hexdump misinterprets start offsets > 2 GiB.\n "

CMD="mv ${TEMPHEAD}_log.txt  ${LOGDIR}/$OUTFILE.PXcut$$.txt"
echo "running: "${CMD}""
${CMD}

rm bytelist$$
rm edlist$$
rm keylist$$
rm revedlist$$
rm pyscript$$
rm projx$$
rm temp$$.txt
rm log$$ 
cd ~

# mythpreviewgen isn't essential here so put it where failure won't cause other problems.
# Creates a blank frame from .mkv file.

CMD="ionice -c3 mythpreviewgen --chanid "$chanid" --starttime "$starttime" -q "
echo "running: "${CMD}""
${CMD}
echo -e "Preview created.\n"

exit 0



Script.png pycut_20130305.py


#! /usr/bin/env python
# -*- coding: utf-8 -*-

# Cut out and concatenate sections of a file
# access as pycut.py from mythDVBcut.sh

import sys, os
#print sys.argv


######################
## For tests
##
## echo "0123456789A123456789B123456789C123456789D123456789E123456789F123456789" > ~/test.txt 
## 
## fn1 = './test.txt'
## fn2 = './temp.txt'
## chunks = [ 3, 12, 35, 47, 53, 68  ]
## buflen = 5

## Doesn't recognise '~/test.txt' but this, or full path, seems ok
## python pycut.py './test.txt' './temp.txt' 3 12 35 47 53 68  

# Zambesi HD

#./printcutlist /home/john/Mythrecs/1054_20120328222600.mpg
# Generates byte-mode cutlist for use with Project-X  - and here
#CollectionPanel.CutMode=0

#fn1 = '/mnt/f10store/myth/reca/1054_20120323002600old.mpg'
#fn2 = '/mnt/sam1/recb/1054_20120323002600.mpg'
#chunks = [ 390284804, 4556742872 ]
#buflen = 1024*1024
#
########################

fn1 = sys.argv[1]  # input file
fn2 = sys.argv[2]  # output file
chunks = map( int, sys.argv [ 3 : ] )  # start and end bytes of chunks in infile
buflen = 1024*1024
#bignum = 10000000000                   # for use as EOF if needed
# less likely to be surprised if we use the actual filesize here

print "infile        ", fn1
print "outfile       ", fn2
print "switchpoints  ", chunks

#######################

# sanity checks

chunklen = len(chunks)
if chunklen != 2 * ( chunklen / 2 ) :
#    chunks.append(bignum)
    chunks.append( 1 + os.path.getsize(fn1))
    chunklen = len(chunks)

# adjust chunk-endpoints in the hope of keeping chain linkage in the data intact
n = 1
while n < chunklen :
  chunks[n] += -1
  n += 2
  
n=0
while n < chunklen - 2 :
   if chunks[n] > chunks[n+1] :
      print "Quitting: switchpoints out of order"
      sys.exit(98)
   n += 1

print "Adjusted switchpoints  ", chunks

n = 0
m = 0
offset = [ 0 ]
while n < chunklen - 1 :
   m += 1 + chunks[ n+1 ] - chunks[ n ]
   offset.append( m )
   n += 2

print
print "Byte offsets of cutpoints in output file: ",  offset
print "DB table is recordedseek, mark (framecount) is type 9."
##################################
# Don't touch stuff below here 
## byte numbering starts at 0 and output includes both chunk-endpoints
i=0
j=0
imax = 40     # buffers per star
jmax = 25     # stars per line 
print         # for progress display

chnklim = len(chunks) - 1 
nchnk = 0
chstart=chunks[nchnk]
chend=chunks[nchnk + 1]
bufstart = 0

f1 = open(fn1, 'rb')
f2 = open(fn2, 'wb')

while True :
  data = f1.read(buflen)
  lendat = len(data)
  if lendat == 0 :
       break
  bufend = bufstart + lendat 
  while chstart < bufend :
       if chend <  bufend :
           f2.write(data[chstart - bufstart : chend - bufstart + 1 ])
           nchnk += 2
           if nchnk > chnklim :             # job done
               chstart = bufend + buflen*2  # kill further looping
               break
           
           chstart = chunks[nchnk]
           chend   = chunks[nchnk + 1]
       else :
           f2.write(data[chstart - bufstart :  ])
           chstart = bufend 
 
  bufstart += lendat
  i += 1           # progress display          
  if i > imax :
     sys.stdout.write("*")
     sys.stdout.flush()
     i = 0
     j += 1
     if j > jmax :
        print
        j = 0
 
f1.close()
f2.close()
print



Script.png mythTScut_20180414.sh




#!/bin/bash

set -e    # exit on error.  I have only become aware of these options recently, and they have not been used.
# set -x  # display command lines
# set -ex # both of the above

# by John Pilkington, developed using ideas from 
# https://www.mythtv.org/wiki/H264_commercial_remover_and_remuxer
# by Ian Thiele  (icthiele@gmail.com) and from my own
# https://www.mythtv.org/wiki/MythDVBcut
#
# This script is offered as work-in-progress.  No guarantees given and no redress or support implied. 
#
# Working directories and database access must be edited (in the first page) to suit your installation.
# Other parameters scattered through the script may not be optimum.
#
# This version uses tools from mythffmpeg, which is usually part of a MythTV installation.
# It was originally intended for cutting h264-encoded recordings but may also work with other TS formats.
# 
# It does not use mkvmerge or Project-X.  The output can either:
#
#     replace the original recording, which is renamed.  Cutlist and bookmarks are cleared and the seektable rebuilt
#        or
#     be written to the specified 'video' directory.  For playback by mythfrontend it will need to be 'scanned in' and preferably given a seektable.
# 
#  Suggested work sequence is:
#     Preliminary test:  ~/mythTScut.sh /path/to/infile T 1
#         which will read the database and process a cutlist but should not touch the recording.
#
#     ionice -c3 ~/mythTScut.sh /path/to/infile R 1 NC
#       It's best to establish the cutlist here, or at least to confirm its accuracy.
#     ionice -c3 ~/mythTScut.sh /path/to/infile V 1
#     scan for new videos from the mythfrontend 'Watch Videos' screen
#     ionice -c3 mythutil --rebuild --video /path/to/the/video/outfile
#
#  and you should then have before-and-after versions available for comparison.  The logfile has the cut positions.
# 
# Cuts are made at video keyframes near to positions taken from a list typically created or adjusted by the interactive cutlist editor.
# In the UK the spacing of keyframes is usually around 1 second, but their positions often coincide with edit-points at source.
# I have the impression that the keyframe positions used by MythTV do not exactly coincide with those found and used by ffmpeg tools:
#    (The editor often finds a keyframe around 7 frames after a scene change)

# I usually place edits at the first or last keyframe for which the editor shows a wanted image.  
# You may prefer to move ends-of-cut a few seconds early to allow for better lipsync recovery. 

# Usage:  /path/$0 /path/to/recording_known_to_MythTV_database (R or V or T)outmode  (1 or 2)streamreadmode  (NC don't cut)

# Use $HOSTNAME (as given by running 'hostname') to find out which machine is being used and assign working directories.

HOST1="HP_Fed"  # f26
HOST2="HP_Box"  # sl7

if [ $HOSTNAME = ${HOST1} ] ; then
  VIDDIR="/mnt/sdb1/Vids" 
  WORKDIR="/mnt/sdb1/TScut_work"
  LOGDIR="/mnt/sdb1/TScut_logs"
elif [ $HOSTNAME = ${HOST2} ] ; then
  VIDDIR="/mnt/dat1/VSG2"
  WORKDIR="/mnt/dat1/TScut_work"
  LOGDIR="/mnt/dat1/TScut_logs" 
else
   echo "Hostname $HOSTNAME not recognised."
   exit 1
fi 

OUTDIR=${VIDDIR}
chunkdir=${WORKDIR}

#RECDIR is obtained here from the commandline

FFCALL="ionice -c3 mythffmpeg -hide_banner -ignore_unknown "

# keyline for connection to DB; copy entries from /etc/mysql.xml
#mysqlconnect="mysql -N -h$DBHostName -u$DBUserName -p$DBPassword $DBName"

mysqlconnect="mysql -N -hlocalhost -umythtv -pmythtv mythconverg"
export mysqlconnect

TIMEOUT=20 # sec: pause for interactive confirmation

TESTRUN=false

echo "WORKDIR is ${WORKDIR}"
echo "LOGDIR is ${LOGDIR}"

cd ${WORKDIR}

INFILE=$1

if [ $2 = R ] ; then 
   tovideo=false 
elif [ $2 = V ] ; then
   tovideo=true
elif [ $2 = T ] ; then
   TESTRUN=true
else  
   echo "Second parameter needed, must be R, V, T"
   echo "R will move the infile to $1.old, replace it and clear its cutlist, bookmarks etc."
   echo "V will just create a new file in $VIDDIR"
   echo "T will Test and exit without making changes."
 
   exit 1
fi 

# Third parameter on the command line:
# Either read video and audio streams 'independently'
# Or  extend the duration of the input segment instead

if [ $3 -eq 1 ] ; then 
   TWINREAD=false 
   skewms=0   # empirically based on a few tests.  Is there a standard value?
elif [ $3 -eq 2 ] ; then
   TWINREAD=true
   skewms=0  # 1600  Not currently used in the code?
else  
   echo " Third parameter needed, must be 1 or 2; read video and audio as 1 or 2 streams."
   echo " 1 seems more robust and usually allows repeat runs;  2 has given bad sync on repeats."
   exit 1
fi

nocut=false
if [ $# -ge 4 ] ; then
  if [ $4 = NC ] ; then
     nocut=true
  else
     echo " Fourth parameter NC will ignore an existing cutlist and work on the entire file."
     exit 1
  fi
fi

#let's make sure we have a sane environment
if [ -z "`which mythffmpeg`" ]; then
    echo "mythffmpeg not present in the path. Adjust environment or install mythffmpeg"
    exit 1
fi

if [ -z "`ls ${WORKDIR}`" ]; then
    mkdir -p ${WORKDIR}
fi
if [ ! -f ${INFILE} ]; then
    echo " ${INFILE} doesn't exist, aborting."
    exit 1
fi

if  [ -f ${INFILE}.old ] ; then 
    echo " ${INFILE}.old exists: giving up." 
    exit 1
fi

echo -e "\n Working on ${INFILE} \n"

#determine directory and filename
RECDIR=`dirname ${INFILE}`
BASENAME=`basename ${INFILE}`

bnhead=$(basename $BASENAME .ts )

logfile=${LOGDIR}/${bnhead}_$$.log
>${logfile}

echo "Processing ${RECDIR}/${BASENAME} run number  $$ " | tee -a ${logfile}

# recordedid is the prime search key in recent versions of MythTV

query="select recordedid from recorded where basename=\"$BASENAME\";"
echo ${query} | tee -a ${logfile}
recordedid=$(echo ${query} | $mysqlconnect)
echo -e  "\n recordedid: ${recordedid} \n " | tee -a ${logfile}

# but some tables and utils still need chanid and starttime.
query="select chanid from recorded where recordedid = '$recordedid' ;"
echo ${query} | tee -a ${logfile}
chanid=$(echo ${query} | $mysqlconnect)
echo -e "\n chanid: ${chanid} \n " | tee -a ${logfile}

query="select starttime from recorded where recordedid = '$recordedid' ;"
echo ${query} | tee -a ${logfile}
starttime=$(echo ${query} | $mysqlconnect)
echo -e "\n starttime: ${starttime} \n " | tee -a ${logfile}

if [ -z "$chanid" ] || [ -z "$starttime" ]
then
    echo "Recording not found in MythTV database, script aborted."
    exit 1
fi

# some myth tools don't like the original format of starttime
starttime=$(echo  ${starttime} | tr -d ': -')
echo -e "\n reformatted starttime: ${starttime} \n " | tee -a ${logfile}

echo -e "\n chanid ${chanid}   starttime ${starttime}  recordedid ${recordedid} \n" | tee -a ${logfile}

#query="select data from recordedmarkup where chanid=$chanid and starttime='$starttime' and type=33 ; "
#echo ${query} | tee -a ${logfile}
#totaldurationms=$(echo ${query} | $mysqlconnect)
# use a nominal value rather than expecting this to be set
totaldurationms=36000000
echo -e "\n totaldurationms:  ${totaldurationms} \n " | tee -a ${logfile}

#query="select data from recordedmarkup where chanid=$chanid and starttime='$starttime' and type=34 ; "
#echo ${query} | tee -a ${logfile}
#totalframes=$(echo ${query} | $mysqlconnect)
totalframes=$(( totaldurationms / 40 ))
echo -e "\n totalframes:  ${totalframes} \n " | tee -a ${logfile}

CMD="mythutil --getcutlist --chanid  $chanid --starttime $starttime -q"
echo " Running: "${CMD}"" | tee -a ${logfile}
mythutilcutlist=$($(echo ${CMD}))
echo "${mythutilcutlist}" | tee -a ${logfile}

#exit

shortlist=$(echo ${mythutilcutlist} | sed 's/Cutlist://' )
echo "${shortlist}" | tee -a ${logfile}

echo -e "\n If you want to reset this cutlist, after restoring the .old file 
and rebuilding its seektable, you could try: \n
mythutil --setcutlist ${shortlist} --chanid  $chanid --starttime $starttime \n
-but seektables from different tools may differ!!\n" | tee -a ${logfile}

if $nocut ; then
  echo -e "\n You have elected to IGNORE the cutlist. \n" | tee -a ${logfile}
fi  

echo -e "\nTo continue, press \"a\", or wait....\n"  
echo  "If you want to select other values, or to quit and think about it, press another key within $TIMEOUT seconds."
echo -e "If you go on, one video and one audio stream will be retained.\n"
 
read -t $TIMEOUT -n 1 RESP
if  [ $? -gt 128 ] ; then    
  RESP="a"
fi

if [ "$RESP" != "a" ] ; then
    echo -e "Quitting"
    cd ~
    exit 1
fi


if [ "${shortlist}" = "" ] ; then
  echo -e "\n Cutlist was empty: working on the complete file \n" | tee -a ${logfile}
  nocut=true
fi

if $nocut ; then
  shortlist=""
  echo -e " 0  $totaldurationms " | tee switchpointsms$$  | tee -a ${logfile}
  echo -e " 0  $totalframes " | tee keylist$$ | tee - ${logfile}  # dummies
  
else

  echo -e "\n Cutframe list as supplied: " | tee -a ${logfile}
  echo "${shortlist}" | tr -d [:alpha:] | tr [:punct:] " " | tee  edlist$$ | tee -a ${logfile}
  echo
  
  #exit
  #Reverse the sense of the cutlist; carried over from mythcutprojectX 
  #
  echo -n > revedlist$$
  for i in  $(cat edlist$$) ; 
  do
    if  [ $i -eq 0 ] ;  then
      for j in  $(cat edlist$$) ; 
      do
         if  [ $j != 0 ] ; then echo -n "$j " >> revedlist$$
         fi
      done
    else
      echo -n " 0 " >> revedlist$$
      for j in  $(cat edlist$$) ; 
      do
          echo -n "$j " >> revedlist$$
      done
    fi
    break
  done
  echo >> revedlist$$
  echo -e "\n Passframe (reversed cutframe) list from editor: " | tee -a ${logfile}
  cat revedlist$$ | tee -a ${logfile}
 
  # For a byte-count cutlist, (ProjectX CutMode=0)
  # mark is the frame count in the seektable and is compared here with the frame count defined by the cutlist editor.
  # mark type 9 is MARK_GOP_BYFRAME (see eg trac ticket #1088) with adjacent values typically separated by 12 or more,
  # so that is presumably the cutpoint frame granularity in bytecount mode. 
  # Subjectively the granularity seems smaller, but this may not apply to locally encoded recordings; 
  # In dvb-t and similar systems the position of keyframes often appears to depend on pre-transmission edits of content. 
  #
  # Find the next keyframe (mark type 9) after a position slightly before the frame identified by the cutlist editor.
  # The original version, which I used originally, for several years, effectively had lag=0.
  # The 'recordedseek' table doesn't know about recordedid.

  #### This version, for use with ffmpeg tools, uses 'type=33' to get times from the start of recording in ms  ####

  # In HD recordings from UK DVB-T2 the MythTV editor finds a typical keyframe separation of about 24 frames
  # A change of scene is usually followed by an editor keyframe about 7 frames later
  
  lead=44      #  In frames.  Best value might depend on recording source, MythTV version and seektable history.   
  scope=200    #  Sometimes h264 keyframes in the wild (or from oldversiond of mythcommflag) are much more widely spaced than expected. 
               #  
               #  This might only have been true for buggy 'rebuilt' seektables, now fixed, but the large value should do no harm.
  go=0         # go=0 indicates the start of a playing segment.  go=1 indicates a segment endpoint
  
  for frame in $(cat revedlist$$) 
  do
    i=$((${frame} )) 
    if [ $i = 0 ] ; then
       echo -e "0 \t0 " 
    else   
      j=$((${i} + ${lead} )) 
      k=$((${j} - ${scope}))
      if [ $k -lt 0 ]; then k=0 
      fi
    
      query="select offset, mark from recordedseek where chanid=$chanid and starttime='$starttime' and type=33 
      and mark >= ${k} and mark < ${j}  order by offset desc limit ${go}, 1 ;" 
    
      echo  ${query} | $mysqlconnect 
    fi
    
    if [ ${go} = 1 ]; then
      go=0
    else
      go=1
    fi
    
  done > tmp1$$

  echo "Full results of DB read:"  | tee -a ${logfile}
  cat tmp1$$  | tee -a ${logfile}
  echo | tee -a ${logfile}
 
  # write the byte offset and frame number into one-line cutlists.

  echo -e "\n Active keyframe passlist via DB.  First is a cut-in:\n" | tee -a ${logfile}
  cut -f2 tmp1$$ | tr "\n" "  "  | tee keylist$$ | tee -a ${logfile}
  
  echo -e "\n Millisecond switchpoints in original file, via DB.  First is a cut-in:\n" | tee -a ${logfile}
  cut -f1 tmp1$$ |  tr "\n" "  "  | tee mslist$$ | tee -a ${logfile}

  # echo -e "\nByte offsets of 2nd keyframes, via DB.  Compare with PjX log." >> log$$
  # cut -f1 tmp2$$ |  tr "\n" " "  | tee -a ${logfile}

  mv tmp1$$ tmp$$  # for use.  The discarded values were collected during development.

  echo -e "\n\n Creating the cutlist" | tee -a ${logfile}
  cut -f1 tmp$$ | tee -a ${logfile} > PXraw$$

  rm tmp$$

  # The PXraw and EOF below are intended to reveal unexpected extra lines from non-printing field separators

  echo "PXraw$$"
  cat PXraw$$
  echo "EOF"

  # Add a limiting filesize value for use if EOF will be reached
  J=$( cat PXraw$$ | wc -l )
  echo -e "\nPXraw$$ has $J lines" | tee -a ${logfile}
  if [ $(( $J % 2 )) -eq 1 ] ; then
    echo " an odd number:  Adding EOF endmark" | tee -a ${logfile}
    echo "$totaldurationms" >> PXraw$$ | tee -a ${logfile}
  fi

  echo -e "\n ###########  hexdump of start of recording \n " | tee -a ${logfile}
  CMD="hexdump -C -n 80 -s 0 ${1} "
  echo ${CMD} | tee -a ${logfile}
  # ${CMD}

  echo -e "\n  getting byte positions of switchpoints in original file - for possible examination only\n" | tee -a ${logfile}       
  for frame in $(cat keylist$$) 
  do
    query="select offset from recordedseek where chanid=$chanid and starttime='$starttime' and mark = $frame and type=9 ; "
#    echo ${query} | tee -a ${logfile}
#    echo
    bytepos=$(echo ${query} | $mysqlconnect ) 
#    echo  $bytepos
    CMD="hexdump -C -n 40 -s $bytepos ${1} "
    echo ${CMD} | tee -a ${logfile}
    echo
    # ${CMD}
  done

  echo -e "\n Millisecond switchpoints to be used \n" | tee -a ${logfile}
  cat PXraw$$ | tr "\n" "   " | tee switchpointsms$$ | tee -a ${logfile}

  rm PXraw$$

# reentry point if thecutlist was blank 
fi
echo
echo "SWPTSMS"
cat switchpointsms$$ 
echo
echo "KEYLIST"
cat keylist$$ 

ms2HMSf()  
  { # convert $1, time interval in ms to hh:mm:ss.xxxx    HMSf is global
     local s=$(( $1 / 1000 )) ; local h=$(( ( $s / 3600 ) % 24  )) ; h=$(printf "%02d" $h )
     local m=$(( ( $s / 60 ) % 60 )) ; m=$(printf "%02d" $m )
     s=$( echo "scale=4; $(( $1 % 60000 )) / 1000.0 " | bc -l ) ; s=$(printf "%07.4f\n" $s )
     HMSf=" $h":"$m":"$s "
  }

#exit
# These calculations work only if no  frames have been dropped

  echo -e "\n\n Millisecond positions of joins in new file:\n" | tee -a ${logfile}
  J=0
  S=0                           # 0 or 1 for cut or pass lists
  for i in  $(cat switchpointsms$$) ;  
  do 
    if [ $S -eq 0 ] ; then
        J=$((J - i))           
        S=1
    else
        J=$((J + i))
        S=0
        ms2HMSf $J ; echo " $HMSf    $J ms" | tee -a ${logfile}
     fi
  done 
  echo | tee -a ${logfile}
    
  echo -e "\n Frame positions of joins in new file:\n" | tee -a ${logfile}
  J=0
  S=0                           # 0 or 1 for cut or pass lists
  for i in  $(cat keylist$$) ;  
  do
    if [ $S -eq 0 ] ; then
        J=$((J - i))           
        S=1
    else
        J=$((J + i))
        S=0
        echo -n "$J " | tee -a ${logfile}
    fi
  done 
  echo | tee -a ${logfile}
   
  switchlist="$(cat switchpointsms$$)"
  echo -e "\n Millisec switchlist for action: ${switchlist} \n" | tee -a ${logfile}
  

if $TESTRUN ; then
   echo -e "\n Quitting because TESTRUN is ${TESTRUN}"
   rm -f cutlist$$
   rm -f temp$$   
   cd ~
   exit 0
fi


# save the input file and replace it as a 'recording' with the product of the merge
# It seems best not to meddle with filename extensions, in case ffmpeg uses them internally
if [ ! $tovideo ] ; then
    OUTDIR=${RECDIR}
else
    OUTDIR=${VIDDIR}
fi
OUTFILE=${OUTDIR}/${bnhead}_$$.ts

swarray=( ${switchlist} ) 

switchcount=${#swarray[@]}
echo -e  "\n Number of switchpoints passed to cutter:  $switchcount \n" | tee -a ${logfile}

if [ $switchcount -lt 3 ] ; then # only one segment, no need to concat
  doconcat=false
  chunkdir=${OUTDIR}
else
  doconcat=true
  chunkdir=${WORKDIR}
fi 
chunkhead=${chunkdir}/${bnhead}_$$

clipcount=0   # initialise tag for file segments

#exit

ms2sf()
{  # input $1 millisec 
# output sf, seconds sssss.xxxx formatted with leading zero for ffmpeg; global
local s 
s=$(echo "scale=4;  $1  / 1000.0 " | bc -l )
sf=$(printf "%.4f\n" $s )
}
 
i=0
until [ $i -ge $switchcount ]
do
   #Real keyframe is around 7 frames before the editor value; go back abit further.  Make duration half a frame longer. 
   startms="${swarray[$i]}"   ; startms=$(( $startms - 20 ))  # don't land on the fence
   endms="${swarray[$i + 1]}" ; endms=$(( $endms + 20 ))      # 6 frames early
   
   ms2sf $(( $startms ))                     ; startsecf=$sf
   ms2sf $(( $endms - $startms ))            ; duration2f=$sf
   ms2sf $(( $endms - $startms + $skewms ))  ; duration1f=$sf
   ms2sf $(( $skewms ))                      ; skewsecf=$sf
   
   clipcount=$(( ++clipcount ))

######  https://trac.ffmpeg.org/wiki/Seeking
    
#  CMD="ffmpeg -hide_banner-ss $startsecf -i ${RECDIR}/${bnhead} -t $durationf -map 0:0 -map 0:1 -c:v copy -c:a copy  \
#  -avoid_negative_ts 1   ${chunkhead}_${clipcount}.ts"
#  It's not clear what options might be best here.  Non-obvious interactions are common.

# https://superuser.com/questions/702645/ffmpeg-sync-when-extracting-parts-of-video

if ! ${TWINREAD} ; then 
    # selected by $3 -eq 1
    ##### duration extended here to allow for skew between video and audio streams
    echo -e  "\n skew applied in non-TWINREAD mode:  $skewms ms \n" | tee -a ${logfile}
    # read using video-frame timestamps only (?)
    
    CMD="$FFCALL \
    -ss $startsecf -noaccurate_seek -t $duration1f -i ${RECDIR}/${BASENAME}  \
    -map 0:0 -c:v  copy \
    -map 0:1 -c:a  copy \
    -avoid_negative_ts 1   ${chunkhead}_${clipcount}.ts"
else
    # selected by $3 -eq 2
    echo -e  "\n skew applied once in TWINREAD mode:  $skewms  ms \n" | tee -a ${logfile}
    # read each stream using its own timestamps  (?). -ignore_unknown here is a per-etream repeat.
    
    CMD="$FFCALL   -ss $startsecf -t $duration2f  -i ${RECDIR}/${BASENAME}  \
   -ignore_unknown -ss $startsecf -t $duration2f  -i ${RECDIR}/${BASENAME}  \
   -map 0:0 -c:v  copy \
   -map 1:1 -c:a  copy -copyts \
   -avoid_negative_ts 1   ${chunkhead}_${clipcount}.ts"
fi
    echo -e "${CMD} \n " | tee -a ${logfile}
    ${CMD} | tee -a ${logfile}

    i=$(( $i + 2 ))
done

#exit
# ##### file segments have been created.  Now prepare to merge them ########

######  https://trac.ffmpeg.org/wiki/Concatenate
if  $doconcat  ; then
  mergestr=""
  for i in $(ls ${chunkhead}_* | sort)
  do
    if [ -z "$mergestr" ]; then
	mergestr="concat:$i"
	continue
    fi
    mergestr="$mergestr|$i"
  done

  CMD="$FFCALL -i $mergestr -c copy  ${OUTFILE}" 
  echo -e "${CMD} \n " | tee -a ${logfile}
  ${CMD} | tee -a ${logfile}
else
  mv ${OUTDIR}/${bnhead}_$$_1.ts ${OUTFILE}
fi

# Here the outfile name is $bnhead_$$.ts, while the input was $bnhead.ts

#if [ $tovideo ] ; then
# This needs myth to be aware of the new video.  Skip for now
#   CMD="ionice -c3 mythcommflag -q --video ${OUTFILE}"
#fi
 
if ! $tovideo ; then

  # move the original out of the way, replace it, and update the DB
  mv ${INFILE} ${INFILE}_$$in.ts
  mv ${OUTFILE}  ${INFILE}
  OUTFILE=${INFILE}
  
  # Rebuilding the seek table also resets filesize in the DB 
  CMD="ionice -c3 mythcommflag -q --rebuild --file ${OUTFILE} "
  echo "running: "${CMD}"" | tee -a ${logfile}
  ${CMD} # | tee -a ${logfile}

  CMD="ionice -c3 mythutil  --clearcutlist  --chanid "$chanid" --starttime "$starttime" -q "
  echo "running: "${CMD}""  | tee -a ${logfile}
  ${CMD}   | tee -a ${logfile}
  echo -e "Cutlist has been cleared.\n" 

  # This command was introduced at 0.28-pre-3055.  See Ticket #11713
  CMD="ionice -c3 mythutil  --clearbookmarks  --chanid "$chanid" --starttime "$starttime" -q "
  echo "running: "${CMD}""  | tee -a ${logfile}
  ${CMD}  | tee -a ${logfile}
  echo -e "Bookmarks have been cleared.\n" 

  CMD="ionice -c3 mythpreviewgen --chanid "$chanid" --starttime "$starttime" -q "
  echo "running: "${CMD}""  | tee -a ${logfile}
  ${CMD} 
  echo -e "Preview created.\n"
fi

echo -e "Output file is $OUTFILE. \n" 

echo -e "\nWhile the .old file still exists you can examine it with\n
hexdump -C -n 40 -s {byteoffset} ${INFILE}_$$in.ts\n
32-bit hexdump may misinterpret start offsets > 2 GiB.\n "

# Get tech details of output file into the log.
CMD="  mythffprobe -hide_banner -v error -show_streams -show_format -of json   $OUTFILE "
echo "${CMD}" | tee -a ${logfile}
${CMD} 2>&1 | tee -a ${logfile}
echo

rm edlist$$
rm keylist$$
rm revedlist$$
rm mslist$$
rm switchpointsms$$

# and if it all worked these may no longer be needed
# rm ${WORKDIR}/$bnhead_$$*
# rm ${INFILE}.old

cd ~
exit 0