Difference between revisions of "MythNetvision Grabber Script Format"

From MythTV Official Wiki
Jump to: navigation, search
m
m (MythNetvision Version Output)
 
(47 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== Grabbers ==
+
= Grabber Script Standards =
 +
 
 +
* All grabbers must support a search and/or a tree view option.  Supporting both is optional.
 +
* Grabbers expect that the environment is set to UTF-8.
 +
* A search grabber by default should return a maximum of 20 video items per page. This usually keeps the response time to a reasonable duration.
 +
* A search grabber must support paging with a specific page number specified through the "-p" command line option. The grabber must interpret the page number into what the source api requires. Sometimes it is a page number and sometimes a video index (e.g. 21st video of the search results).
 +
* A grabber's file name must have a matching image file name for an icon to be displayed by the MythNetvision plugin.
 +
 
 +
 
 +
== Grabber Script Command Line Options ==
 +
 
 
{| border=1 cellspacing=0 cellpadding=5
 
{| border=1 cellspacing=0 cellpadding=5
| '''Grabber'''
+
| '''Command Line Option'''
| '''API'''
+
| '''Arguments'''
| '''Search'''
+
| '''Example'''
| '''Tree view'''
+
| '''Description'''
| '''Channel icon'''
 
 
|-
 
|-
| [http://blip.tv/ bliptv.py]
+
| -v
| [http://blip.tv/about/api/]
+
|''None''
|
+
|/usr/share/mythtv/mythnetvision/scripts/youtube.py -v
| √
+
|
| bliptv.png
+
* This option is used by MythNetvision to identify the grabber's name and its capabilities, which can include search and/or tree view.  See below for details.
 
|-
 
|-
| [http://www.dailymotion.com dailymotion.py]
+
| -p
| [http://www.dailymotion.com/ca-en/doc/api/player/javascript_api#DailymotionPlayerTools-Overview]
+
| Unsigned Integer
|
+
|/usr/share/mythtv/mythnetvision/scripts/youtube.py -p 2 -S "A Search Term"
| √
+
|
| dailymotion.jpg
+
* Page number.  Always includes an integer and is only used with the search option (-S). When not provided the page number defaults to one.
 
|-
 
|-
| [http://www.joost.com joost.py]
+
| -S
| [http://www.joost.com/test/doc/api/]
+
| Text String
 +
|/usr/share/mythtv/mythnetvision/scripts/youtube.py -p 2 -S "A Search Term"
 
|
 
|
| √
+
* Search option. Requires a search term made of one or more words.
| joost.png
+
* Only required if the grabber supports a Search view.
|-
 
| [http://www.mtv.com/ mtv.py]
 
| [http://developer.mtvnservices.com/docs]
 
| √
 
| √
 
| mtv.png
 
 
|-
 
|-
| [http://www.themoviedb.org/ tmdb_nv.py]
+
| -T
| [http://api.themoviedb.org/2.1/]
+
|''None''
| √
+
|/usr/share/mythtv/mythnetvision/scripts/youtube.py -T
 
|
 
|
| tmdb_nv.png
+
* Tree view. Returns a properly formatted tree view RSS feed.
|-
+
* Only required if a grabber supports a Tree View.
| [http://www.vimeo.com vimeo.py]
 
| [http://www.vimeo.com/api/]
 
| √
 
| √
 
| vimeo.jpg
 
|-
 
| [http://www.youtube.com youtube.py]
 
| [http://code.google.com/apis/youtube/overview.html]
 
| √
 
| √
 
| youtube.png
 
 
|}
 
|}
* If you develop your own grabber please add it's details to this table
+
 
* Grabbers posted here must not violate the source site's TOS (Terms of Service)
+
 
* Grabbers do not have to be based on an API and can be a screen scraper as long as they do not violate TOS. Just be prepared for more frequent support requests as site changes will often break a screen scraping grabber.
+
= MythNetvision Version Output =
<BR>
+
 
== Grabber Script Standards ==
+
All Internet Content scripts must return their capabilities in the following format:
* All grabbers must at least support a search and/or a tree view option, supporting both is optional
+
 
* Grabbers expect that the environment is set to UTF-8
+
'''Command Line:'''
* A search grabber by default should return a maximum of 20 video items per page. This usually keeps the response time to a reasonable duration.
+
 
* A search grabber must support paging with a specific page number specified through the "-p" or "--pagenumber" command line option. The grabber must interpret the page number into what the source api requires. Sometimes it is a page number and sometimes a video index (e.g. 21st video of the search results).
+
<pre>./youtube.py -v</pre>
* A grabber's file name must have a matching image file name for an icon to be displayed by the Netvision plug-in
+
 
=== Command line options ===
+
'''Returned XML:'''
"-v", "--version"
+
 
* This option is used by Netvision to identify the grabber's name and it's supported options search and/or tree view
+
<pre>
* The format is grabber title then a "|" character followed by the grabber's supported options search and/or tree view
+
<grabber>
* Example: "YouTube|ST" says that the "YouTube" grabber supports both Netvision search and tree view options
+
  <name>YouTube</name>
"-h:, "--help"
+
  <command>youtube.py</command>
* Display all options and the command line format
+
  <author>R.D. Vaughan</author>
* Include script version number and author's name
+
  <thumbnail>youtube.png</thumbnail>
"-u", "--usage"
+
  <type>video</type>
* Command line examples on how to invoke the grabber and an abridged example of the RSS results
+
  <description>Share your videos with friends, family, and the world.</description>
"-d", "--debug"
+
  <version>0.22</version>
* Debug is optional but recommended
+
  <search>true</search>
* Debug when supported should at least display the URL used to access a source's video data
+
  <tree>true</tree>
* Debug when supported can also display the raw data returned by the source. This is useful when a source uses a one-time or time specific URL that cannot be used directly in a browser (e.g. vimeo)
+
</grabber>
* The debug data is displayed to standard out (stdout) or in a log
+
</pre>
"-p". "--pagenumber"
+
 
* Always includes an integer and is only used with the search option (-S). When not provided the page number defaults to one.
+
''Name'' is the "plain english" name of your script as displayed in the UI.  ''Author'' is your name or pseudonym.  ''Command'' is the basename command name for your grabber ''Thumbnail'' is the image you distribute to represent your grabber or source.  ''Type'' can presently only be ''video''.  ''Description'' is a short explanation of the source. ''Version'' is a ''double'' representing the version number (ie 0.11, 1.00, 2.10, etc). ''Search'' and ''tree'' are booleans expressing whether or not the script is capable of searching, tree view, or both.
"-l", "--language"
+
 
* Optional parameter but when used must include an two character ISO compliant language code. At this time the support for language codes in searches and tree views are limited. If the source site does not support language filtering base on a language code then this option is ignored.
+
= MythNetvision Search Scripts =
"-S", "--search"
+
 
* The search option requires a search term made of one or more words.
+
Let's examine the following example of a search script:
"-T", "--treeview"
+
 
* Tree view does not require or support any additional options
+
'''Command Line:'''
<BR>
+
 
=== Netvision RSS standard format ===
+
<pre>./youtube.py -p 1 -S "Peep Show"</pre>
==== XML and RSS tag format ====
+
 
Both Tree view and Search results start with:
+
'''Returned XML:'''
 +
 
 +
<pre>
 +
<?xml version="1.0" encoding="UTF-8"?>
 +
<rss version="2.0"
 +
xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
 +
xmlns:content="http://purl.org/rss/1.0/modules/content/"
 +
xmlns:cnettv="http://cnettv.com/mrss/"
 +
xmlns:creativeCommons="http://backend.userland.com/creativeCommonsRssModule"
 +
xmlns:media="http://search.yahoo.com/mrss/"
 +
xmlns:atom="http://www.w3.org/2005/Atom"
 +
xmlns:amp="http://www.adobe.com/amp/1.0"
 +
xmlns:dc="http://purl.org/dc/elements/1.1/"
 +
xmlns:mythtv="http://www.mythtv.org/wiki/MythNetvision_Grabber_Script_Format">
 +
    <channel>
 +
        <title>YouTube</title>
 +
        <link>http://www.youtube.com/</link>
 +
        <description>Share your videos with friends, family, and the world.</description>
 +
        <numresults>2</numresults>
 +
        <returned>2</returned>
 +
        <startindex>1</startindex>
 +
        <item>
 +
            <title>Peep Show S02E02 P01</title>
 +
            <author>babyblue8five</author>
 +
            <pubDate>Sun, 06 Jan 2008 21:44:36 GMT</pubDate>
 +
            <description>Peep Show season 2, episode 2, part 1/3</description>
 +
            <link>http://www.youtube.com/watch?v=gCWmiXEkAGE&amp;feature=youtube_gdata</link>
 +
            <media:group>
 +
                <media:thumbnail url='http://i.ytimg.com/vi/gCWmiXEkAGE/0.jpg'/>
 +
                <media:content url='http://www.youtube.com/watch?v=gCWmiXEkAGE&amp;feature=youtube_gdata' duration='' width='' height='' lang=''/>
 +
            </media:group>
 +
            <rating>4.988506</rating>
 +
        </item>
 +
        <item>
 +
            <title>Peep Show S02E01 P01</title>
 +
            <author>babyblue8five</author>
 +
            <pubDate>Sun, 06 Jan 2008 20:13:10 GMT</pubDate>
 +
            <description>Peep Show Season 2, episode 1, part 1/3</description>
 +
            <link>http://www.youtube.com/watch?v=38EhKzJ0zS8&amp;feature=youtube_gdata</link>
 +
            <media:group>
 +
                <media:thumbnail url='http://i.ytimg.com/vi/38EhKzJ0zS8/0.jpg'/>
 +
                <media:content url='http://www.youtube.com/watch?v=38EhKzJ0zS8&amp;feature=youtube_gdata' duration='' width='' height='' lang=''/>
 +
            </media:group>
 +
            <rating>4.931818</rating>
 +
        </item>
 +
    </channel>
 +
</rss>
 +
</pre>
 +
 
 +
The above is a search command which returns two items.  Let's break down the results so that you can see how it works.
 +
 
 +
 
 +
== Establishing The Namespace ==
 +
 
 +
<pre>
 
  <?xml version="1.0" encoding="UTF-8"?>
 
  <?xml version="1.0" encoding="UTF-8"?>
 
  <rss version="2.0"
 
  <rss version="2.0"
Line 95: Line 142:
 
  xmlns:atom="http://www.w3.org/2005/Atom"
 
  xmlns:atom="http://www.w3.org/2005/Atom"
 
  xmlns:amp="http://www.adobe.com/amp/1.0"
 
  xmlns:amp="http://www.adobe.com/amp/1.0"
  xmlns:dc="http://purl.org/dc/elements/1.1/">
+
  xmlns:dc="http://purl.org/dc/elements/1.1/"
 +
xmlns:mythtv="http://www.mythtv.org/wiki/MythNetvision_Grabber_Script_Format">
 +
</pre>
 +
 
 +
All MythNetvision returns (Search and Tree) start with the above.  This is called setting the XML ''namespace''.  Namespaces are custom definitions of RSS/XML tags that can be used in your document.  Presently the mythtv, media, amp, and itunes tags are used to some extent, but others may be supported in the future so the above should tentatively be used on all returns.
  
The last line in the RSS results must be:
+
== An RSS "Channel" ==
</rss>
 
  
==== Channel tag format ====
+
<pre>
* Searches are organised into channels
 
* Tree views are organised into channels and directories
 
* A channel is usually the description of the source of the videos
 
 
     <channel>
 
     <channel>
 
         <title>YouTube</title>
 
         <title>YouTube</title>
 
         <link>http://www.youtube.com/</link>
 
         <link>http://www.youtube.com/</link>
 
         <description>Share your videos with friends, family, and the world.</description>
 
         <description>Share your videos with friends, family, and the world.</description>
         <numresults>2092</numresults>
+
         <numresults>2</numresults>
         <returned>20</returned>
+
         <returned>2</returned>
         <startindex>40</startindex>
+
         <startindex>1</startindex>
 +
</pre>
 +
 
 +
All RSS returns in MythNetvision are segmented into "channels."  Although it's not generally necessary to have more than one channel in a return, it is possible and acceptable to do so.  A channel is a logical grouping of results with a title, description, and some information to allow control of paging in search scripts.  Let's take a look at what each of these tags means.
  
 
{| border=1 cellspacing=0 cellpadding=5
 
{| border=1 cellspacing=0 cellpadding=5
Line 117: Line 167:
 
|-
 
|-
 
| channel
 
| channel
| Tag identifies the start of a channel. The channel must eventually be closed with a "</channel>"
+
| Tag identifies the start of a channel.
 
|-
 
|-
 
| title
 
| title
| The title of the source of the meta data which will be displayed by the Netvision plug-in
+
| The title of the source of the metadata which will be displayed by the MythNetvision plugin.
 
|-
 
|-
 
| link
 
| link
| URL of the home page for the source of the meta data
+
| URL of the home page for the source of the metadata.
 
|-
 
|-
 
| description
 
| description
| Descriptive text relevant for the source of the meta data
+
| Descriptive text about the metadata source.
 
|-
 
|-
 
| numresults
 
| numresults
| The number of results returned from a search. Usually this is total items or total pages of items. This value is not used in a tree view.
+
| The number of results returned from a search. Usually this is total items or total pages of items. ''This tag is only used in search mode''.
 
|-
 
|-
 
| returned
 
| returned
| This number reflects the maximum number of items per page or the number returned when a search does not find a full page worth of video items. This value is not used in a tree view.
+
| This number reflects the maximum number of items per page or the number returned when a search does not find a full page worth of video items. ''This tag is only used in search mode''.
 
|-
 
|-
 
| startindex
 
| startindex
| The start index of the next page or next video item. If this number is less than or equal to the number of results the Netvision plug-in will not allow the user to request a next page. This value is not used in a tree view.
+
| The start index of the next page or next video item. If this number is greater than or equal to the number of results the MythNetvision plugin will not allow the user to request a next page. ''This tag is only used in search mode''.
 
|}
 
|}
<BR>
+
 
==== Directory tag format ====
+
 
* Directories include a label and icon image used to organise the tree view.
+
== A MythNetvision Video Item ==
* Directories can be nested
+
 
* This is the method for the grabber to telling Netvision how to display/organize the video items
+
<pre>
<directory name="Feeds" thumbnail="/usr/local/share/mythtv/mythnetvision/icons/youtube.png">
+
        <item>
{| border=1 cellspacing=0 cellpadding=5
+
            <title>Peep Show S02E02 P01</title>
| '''Tag name'''
+
            <mythtv:subtitle>Jeremy Makes It</mythtv:subtitle>
| '''Description'''
+
            <author>babyblue8five</author>
|-
+
            <pubDate>Sun, 06 Jan 2008 21:44:36 GMT</pubDate>
| directory
+
            <description>Peep Show season 2, episode 2, part 1/3</description>
| Identifies the start of a directory. The directory must eventually be closed with a "</directory>"
+
            <link>http://www.youtube.com/watch?v=gCWmiXEkAGE&amp;feature=youtube_gdata</link>
|-
+
            <media:group>
| name
+
                <media:thumbnail url='http://i.ytimg.com/vi/gCWmiXEkAGE/0.jpg'/>
| Descriptive label for the directory. This description will be displayed by the Netvision plug-in
+
                <media:content url='http://www.youtube.com/watch?v=gCWmiXEkAGE&amp;feature=youtube_gdata' length='' duration='' width='' height='' lang=''/>
|-
+
            </media:group>
| thumbnail
+
            <rating>4.988506</rating>
| An image URL or absolute file path for the image icon that will be displayed by the Netvision plug-in
+
            <mythtv:country>us</mythtv:country>
|}
+
            <mythtv:country>uk</mythtv:country>
<BR>
+
            <mythtv:season>2</mythtv:season>
==== Item tag format ====
+
            <mythtv:episode>2</mythtv:episode>
Each item contains meta data exclusive to a single video<BR>
+
        </item>
    <item>
+
</pre>
        <title>Project for Awesome - My Public Access Channel!</title>
+
 
        <author>peron75</author>
+
The above represents a single viewable video in MythNetvision.  By manipulating the format of the information, you can express that the item is downloadable, web view only, and present different types of metadata.
        <pubDate>Thu, 17 Dec 2009 13:51:34 GMT</pubDate>
 
        <description>Please support all the Project for Awesome videos today with ratings/comments! Thank you! Thank you to Hank and John Green, Dan Brown and everyone involved!!!  I chose the public access station where I began What the Buck! They are fundraising to help with their new building project.  You can help buy simply signing up for this site and then when you shop, they get donations from that! Yay! Thanks if you can sign up! (its Free!) LOL xoxo Michael  Please sign up: http://igive.com/wpaa</description>
 
        <link>http://www.youtube.com/v/tdBHzkoXB_8?f=standard&amp;app=youtube_gdata&amp;autoplay=1</link>
 
        <media:group>
 
            <media:thumbnail url='http://i.ytimg.com/vi/tdBHzkoXB_8/hqdefault.jpg'/>
 
            <media:content url='http://www.youtube.com/v/tdBHzkoXB_8?f=standard&amp;app=youtube_gdata&amp;autoplay=1' duration='259' width='640' height='480' lang='en'/>
 
        </media:group>
 
        <rating>4.972514</rating>
 
    </item>
 
  
 
{| border=1 cellspacing=0 cellpadding=5
 
{| border=1 cellspacing=0 cellpadding=5
Line 177: Line 218:
 
|-
 
|-
 
| item
 
| item
| Identifies the start of a video item. The video item must eventually be closed with a "</item>"
+
| Identifies the start of a video item.  
 
|-
 
|-
 
| title
 
| title
| Descriptive label for the video
+
| The title text for the video.
 +
|-
 +
| mythtv:subtitle
 +
| The subtitle text for the video.
 
|-
 
|-
 
| author
 
| author
| The author who submitted the video to the source web site. Often this is submitter's user id.
+
| The author who submitted the video to the source web site. Often this is submitter's user id or username.
 
|-
 
|-
 
| pubDate
 
| pubDate
| The date that the video was submitted to the site. The format of "Thu, 17 Dec 2009 13:51:34 GMT" must always be used. If you do not know the GMT time then set it to all zeros "00:00:00".
+
| The date that the video was submitted to the site. The format of "Thu, 17 Dec 2009 13:51:34 GMT" must always be used. If you cannot retrieve the timezone, simply specify the approximate timezone in which the source exists.  If you do not know the time then set it to all zeros "00:00:00".  pubDate follows the [http://asg.web.cmu.edu/rfc/rfc822.html#sec-5 RFC-822] date-time format.
 
|-
 
|-
 
| description
 
| description
Line 192: Line 236:
 
|-
 
|-
 
| link
 
| link
| A URL to the web page containing the video or directly to the video player. If this link is not identical to the "media:content url" tag then that indicates to Netvision that the "media:content url" file can be downloaded. Unless the source site TOS allows a video file to be downloaded then the "link" and "media:content url" URLs should be identical.
+
| A URL to the web page containing the video or directly to the video player. If this link is not identical to the "media:content url" tag then that indicates to MythNetvision that the "media:content url" file can be downloaded. Unless the source site TOS allows a video file to be downloaded then the "link" and "media:content url" URLs should be identical (Web View Only).
 +
|-
 +
| player
 +
|  '''Note''': Do not populate this field if you wish to use MythTV's built-in player! The executable name to an external player application.  An example might be something like:
 +
<pre><player>mplayer</player></pre>
 +
|-
 +
| playerargs
 +
| '''Note''': Do not populate this field if you wish to use MythTV's built-in player!  The arguments passed to the external binary in the <player> tags.  An example might be something like:
 +
<pre><playerargs>-fs -zoom mms://123.45.6.78/</playerargs></pre>
 +
 
 +
You can insert variables here and MythNetvision will dynamically insert the correct values before playback.  They are as follows:
 +
<pre>
 +
%URL% - The <link> value.
 +
%MEDIAURL% - The <media:content> url= value
 +
</pre>
 +
|-
 +
| download
 +
| '''Note''': Do not populate this field if you wish to use MythNetvision's built-in download handling!  The executable name to an external downloader application.  An example might be something like:
 +
<pre><download>mydownloadscript</download></pre>
 +
An external download script must return a single line upon a successful result:  The fully qualified path to the downloaded file.  MythNetvision then reads in this value and uses it to play back the item with the internal player and (in the future) to perform any insertions into the database.
 +
 
 +
Example download script use:
 +
 
 +
<pre>./mydownloadscript -outpath %HOMEDIR% -outfile %FILE% http://my.ip.add.ress/file.html</pre>
 +
 
 +
Example script output:
 +
 
 +
<pre>/home/someguy/download_123_456.mp4</pre>
 +
 
 +
Note that you may output logging and errors to stderr, but only the output filename must be emitted to stdout.  The above example shows the use of userjob-style variables, which are discussed below in <downloadargs>.
 +
 
 +
'''Not yet fully implemented.'''
 +
|-
 +
| downloadargs
 +
|  '''Note''': Do not populate this field if you wish to use MythNetvision's built-in download handling!  The arguments passed to the external binary in the <download> tags.  An example might be something like:
 +
<pre><downloadargs>-outpath /home/someguy/ -outfile somefile.flv http://path.to.a/thing.html</downloadargs></pre>
 +
 
 +
In addition to defining static locations, you can insert the following userjob-style variables in your downloader arguments, which MythNetvision will then replace with the local values:
 +
 
 +
'''%HOMEDIR%''' The frontend user's home directory.
 +
 
 +
'''%FILE%''' The output filename suggestion in MythNetvision format.
 +
 
 +
'''Not yet fully implemented.'''
 
|-
 
|-
 
| media:group
 
| media:group
| Indicates the start of a video's thumbnail and meta data. The item must eventually be closed with a "</media:group>"
+
| Indicates the start of a video's thumbnail and metadata. The item must eventually be closed with a "</media:group>"
 
|-
 
|-
 
| media:thumbnail url
 
| media:thumbnail url
Line 201: Line 288:
 
|-
 
|-
 
| media:content
 
| media:content
| The main tag for the video link and video meta data
+
| The main tag for the video link and video metadata.
 
|-
 
|-
 
| url
 
| url
 
| A URL to the video file/player/web page. This link should only differ from the URL "link" tag when the URL points to a downloadable video file.
 
| A URL to the video file/player/web page. This link should only differ from the URL "link" tag when the URL points to a downloadable video file.
 +
|-
 +
| length
 +
| Unlike what it appears to be, length is actually the filesize in bytes.  This is only used for downloadable media.
 
|-
 
|-
 
| duration
 
| duration
Line 210: Line 300:
 
|-
 
|-
 
| width
 
| width
| The width in pixels of the video
+
| The width in pixels of the video.
 
|-
 
|-
 
| height
 
| height
| The height in pixels of the video
+
| The height in pixels of the video.
 
|-
 
|-
 
| lang
 
| lang
| The ISO language code of the video. [http://en.wikipedia.org/wiki/List_of_ISO_639-1_codes Two character language codes]
+
| The ISO-639 language code of the video. [http://en.wikipedia.org/wiki/List_of_ISO_639-1_codes Two character language codes]
 
|-
 
|-
 
| rating
 
| rating
 
| A floating point number representing a numerical rating for the video. There is no consistency for this value between the source sites. The most common values are X out or 5 or 10.
 
| A floating point number representing a numerical rating for the video. There is no consistency for this value between the source sites. The most common values are X out or 5 or 10.
 +
|-
 +
| mythtv:country
 +
| A country in which the video is playable.  If the video is playable anywhere, return no country.  If the video is only playable in certain locales, or the Terms of Service only allow playback in certain locales, return the country code for the playable countries here.  You may add as many country items as you like.
 +
|-
 +
| mythtv:season
 +
| An unsigned integer representing the season number of the item.  Optional.
 +
|-
 +
| mythtv:episode
 +
| An unsigned integer representing the episode number of the item.  Optional.
 
|}
 
|}
<BR>
+
 
=== Meta data standards ===
+
== Clean This Mess Up, Mister! ==
* When a search returns no results then no RSS data is returned, just exit with a return code of zero
+
 
* Do not include any video items or directories that do not have at least one video link
+
<pre>
 +
    </channel>
 +
</rss>
 +
</pre>
 +
 
 +
For those unfamiliar with XML, RSS, or HTML, *all* tags must be closed or parsing the document will fail.
 +
 
 +
= MythNetvision Tree Grabber Scripts =
 +
 
 +
Let's examine the following example of a Site Tree building script:
 +
 
 +
'''Command Line:'''
 +
 
 +
<pre>/youtube.py -T</pre>
 +
 
 +
'''Returned XML:'''
 +
 
 +
<pre>
 +
<?xml version="1.0" encoding="UTF-8"?>
 +
<rss version="2.0"
 +
xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
 +
xmlns:content="http://purl.org/rss/1.0/modules/content/"
 +
xmlns:cnettv="http://cnettv.com/mrss/"
 +
xmlns:creativeCommons="http://backend.userland.com/creativeCommonsRssModule"
 +
xmlns:media="http://search.yahoo.com/mrss/"
 +
xmlns:atom="http://www.w3.org/2005/Atom"
 +
xmlns:amp="http://www.adobe.com/amp/1.0"
 +
xmlns:dc="http://purl.org/dc/elements/1.1/">
 +
    <channel>
 +
        <title>YouTube</title>
 +
        <link>http://www.youtube.com/</link>
 +
        <description>Share your videos with friends, family, and the world.</description>
 +
            <directory name="Feeds" thumbnail="/usr/share/mythtv/mythnetvision/icons/youtube.png">
 +
                <directory name="Highest Rated" thumbnail="/usr/share/mythtv/mythnetvision/icons/directories/topics/rated.png">
 +
                    <item>
 +
                        <title>Project for Awesome - My Public Access Channel!</title>
 +
                        <author>peron75</author>
 +
                        <pubDate>Thu, 17 Dec 2009 13:51:34 GMT</pubDate>
 +
                        <description>Please support all the Project for Awesome videos today with ratings/comments! Thank you! Thank you to Hank and John Green, Dan Brown and everyone
 +
involved!!!  I chose the public access station where I began What the Buck! They are fundraising to help with their new building project.  You can help buy simply signing up for th
 +
is site and then when you shop, they get donations from that! Yay! Thanks if you can sign up! (its Free!) LOL xoxo Michael  Please sign up: http://igive.com/wpaa</description>
 +
                        <link>http://www.youtube.com/v/tdBHzkoXB_8?f=standard&amp;app=youtube_gdata&amp;autoplay=1</link>
 +
                        <media:group>
 +
                            <media:thumbnail url='http://i.ytimg.com/vi/tdBHzkoXB_8/hqdefault.jpg'/>
 +
                            <media:content url='http://www.youtube.com/v/tdBHzkoXB_8?f=standard&amp;app=youtube_gdata&amp;autoplay=1' duration='259' width='' height='' lang=''/>
 +
                        </media:group>
 +
                        <rating>4.9695706</rating>
 +
                    </item>
 +
                </directory>
 +
            </directory>
 +
    </channel>
 +
</rss>
 +
</pre>
 +
 
 +
Wow, this is remarkably familiar, isn't it?  The format of a site tree is very similar to the search view, and extremely similar to any old RSS feed.  Let's only look at the items which differ from a search script.
 +
 
 +
 
 +
== Directories/Tree Hierarchy ==
 +
 
 +
<pre>
 +
            <directory name="Feeds" thumbnail="/usr/local/share/mythtv/mythnetvision/icons/youtube.png">
 +
                <directory name="Highest Rated" thumbnail="/usr/local/share/mythtv/mythnetvision/icons/directories/topics/rated.png">
 +
</pre>
 +
 
 +
Shocking as it might seem, the ''only'' new element added in a tree grabber is the directory tag, which allows you to create a filesystem-like structure for your tree.  The above example contains two nested directories, meaning the represented path would be:
 +
 
 +
<pre>
 +
YouTube/Feeds/Highest Rated/
 +
</pre>
 +
 
 +
Directories include a name, which is the directory name, and a thumbnail attribute, which is an option URL or absolute path to a folder icon.
 +
 
 +
{| border=1 cellspacing=0 cellpadding=5
 +
| '''Tag name'''
 +
| '''Description'''
 +
|-
 +
| directory
 +
| Identifies the start of a directory. The directory must eventually be closed with a "</directory>"
 +
|-
 +
| name
 +
| Descriptive label for the directory. This description will be displayed by the MythNetvision plugin.
 +
|-
 +
| thumbnail
 +
| An image URL or absolute file path for the image icon that will be displayed by the MythNetvision plugin.
 +
|}
 +
 
 +
= Metadata standards =
 +
 
 +
Consider the following "rules of thumb" when writing your grabber:
 +
 
 +
* When a search returns no results then no RSS data is returned, exit with a return code of zero.
 +
* Do not include any video items or directories that do not have at least one video link.
 
* Always include every tag even if the source does not have any data for that tag. Leave the content empty.
 
* Always include every tag even if the source does not have any data for that tag. Leave the content empty.
 
  width=""
 
  width=""
 
  <description></description>
 
  <description></description>
* Whenever possible always use a video link that auto starts the video and displayed in full screen
+
* Whenever possible always use a video link that auto starts the video and displayed in full screen.
 
** Sometimes the method to auto start or display in full screen is not included in the API documentation. The video's HTML Web page source can sometimes provide this information.
 
** Sometimes the method to auto start or display in full screen is not included in the API documentation. The video's HTML Web page source can sometimes provide this information.
* Replace any ampersand characters "&" with "<code>&amp;amp;</code>"
+
* Replace any ampersand characters "&" with "<code>&amp;amp;</code>".
 
* Remove line feed characters and HTML tags from any text fields. This is usually an issue with the description and title text of a video.
 
* Remove line feed characters and HTML tags from any text fields. This is usually an issue with the description and title text of a video.
* Do not use the word "video" in any of your Tree view directory descriptions as it is redundant
+
* Do not use the word "video" in any of your Tree view directory descriptions or grabber names as it is redundant (MythNetvision is a video plugin, after all).
* Additional tags may by included by a grabber but they will be ignored by the Netvision plug-in
+
* Additional tags may by included by a grabber but they will be ignored by the MythNetvision plugin.
=== Tree view thumbnail icon images ===
+
 
* Each directory tag should have a thumbnail image
+
 
* The default is a channel URL image or the grabber specific icon image found in ".../mythtv/mythnetvision/icons"
+
= Tree view thumbnail icon images =
* Included with Netvision are a set of generic directory icon images that can be used. See ".../mythtv/mythnetvision/icons/directories". You can contribute additional directory icon images but please be consistent with the provided icon image theme.
+
* Each directory tag can include a thumbnail image.
=== Errors handling ===
+
* The thumbnail may be an absolute path or a URL.
* Standard out (stdout) is reserved exclusively for valid RSS data read by the MythTV Netvision plug-in
+
* The MythNetvision core icon set is found in "/usr{/local}/share/mythtv/mythnetvision/icons"
* Standard error (stderr) is reserved exclusively for error or grabber processing messages
+
* Included with MythNetvision are a set of generic directory icon images that can be used. See "/usr{/local}/share/mythtv/mythnetvision/icons/directories".
* Logging is optional
+
 
 +
 
 +
= Error handling =
 +
* Standard out (stdout) is reserved exclusively for valid RSS data read by the MythNetvision plugin.
 +
* Standard error (stderr) is reserved exclusively for error or grabber processing messages.
 +
* Logging is optional.
 
* Return code is zero for any processing that completed without a script abort. This includes a search or treeview that did not return any results.
 
* Return code is zero for any processing that completed without a script abort. This includes a search or treeview that did not return any results.
* Return code of one if an error forced the premature termination of the grabber
+
* Return code of one if an error forced the premature termination of the grabber.
 
* If a grabber script skips bad data during process an error about the bad data should be sent to stderr but a return code of zero would still be returned indicating the grabber completed its processing successfully.
 
* If a grabber script skips bad data during process an error about the bad data should be sent to stderr but a return code of zero would still be returned indicating the grabber completed its processing successfully.
== Python grabber development ==
+
 
Although a grabber can be developed in any computer language that can support the grabber standards a framework for grabber development is available for python.
+
[[Category:Grabber Documentation]]
If the framework is adopted by your python grabber you be able to focus on source site specific processing without having to recode generic grabber functions.
 
See this wiki link for details [http://www.mythtv.org/wiki?title=MythNetvision_Python_Grabber_Development&action=edit&redlink=1 Netvision python grabber development]
 

Latest revision as of 21:02, 25 May 2010

Grabber Script Standards

  • All grabbers must support a search and/or a tree view option. Supporting both is optional.
  • Grabbers expect that the environment is set to UTF-8.
  • A search grabber by default should return a maximum of 20 video items per page. This usually keeps the response time to a reasonable duration.
  • A search grabber must support paging with a specific page number specified through the "-p" command line option. The grabber must interpret the page number into what the source api requires. Sometimes it is a page number and sometimes a video index (e.g. 21st video of the search results).
  • A grabber's file name must have a matching image file name for an icon to be displayed by the MythNetvision plugin.


Grabber Script Command Line Options

Command Line Option Arguments Example Description
-v None /usr/share/mythtv/mythnetvision/scripts/youtube.py -v
  • This option is used by MythNetvision to identify the grabber's name and its capabilities, which can include search and/or tree view. See below for details.
-p Unsigned Integer /usr/share/mythtv/mythnetvision/scripts/youtube.py -p 2 -S "A Search Term"
  • Page number. Always includes an integer and is only used with the search option (-S). When not provided the page number defaults to one.
-S Text String /usr/share/mythtv/mythnetvision/scripts/youtube.py -p 2 -S "A Search Term"
  • Search option. Requires a search term made of one or more words.
  • Only required if the grabber supports a Search view.
-T None /usr/share/mythtv/mythnetvision/scripts/youtube.py -T
  • Tree view. Returns a properly formatted tree view RSS feed.
  • Only required if a grabber supports a Tree View.


MythNetvision Version Output

All Internet Content scripts must return their capabilities in the following format:

Command Line:

./youtube.py -v

Returned XML:

<grabber>
  <name>YouTube</name>
  <command>youtube.py</command>
  <author>R.D. Vaughan</author>
  <thumbnail>youtube.png</thumbnail>
  <type>video</type>
  <description>Share your videos with friends, family, and the world.</description>
  <version>0.22</version>
  <search>true</search>
  <tree>true</tree>
</grabber>

Name is the "plain english" name of your script as displayed in the UI. Author is your name or pseudonym. Command is the basename command name for your grabber Thumbnail is the image you distribute to represent your grabber or source. Type can presently only be video. Description is a short explanation of the source. Version is a double representing the version number (ie 0.11, 1.00, 2.10, etc). Search and tree are booleans expressing whether or not the script is capable of searching, tree view, or both.

MythNetvision Search Scripts

Let's examine the following example of a search script:

Command Line:

./youtube.py -p 1 -S "Peep Show"

Returned XML:

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
xmlns:content="http://purl.org/rss/1.0/modules/content/"
xmlns:cnettv="http://cnettv.com/mrss/"
xmlns:creativeCommons="http://backend.userland.com/creativeCommonsRssModule"
xmlns:media="http://search.yahoo.com/mrss/"
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:amp="http://www.adobe.com/amp/1.0"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:mythtv="http://www.mythtv.org/wiki/MythNetvision_Grabber_Script_Format">
    <channel>
        <title>YouTube</title>
        <link>http://www.youtube.com/</link>
        <description>Share your videos with friends, family, and the world.</description>
        <numresults>2</numresults>
        <returned>2</returned>
        <startindex>1</startindex>
        <item>
            <title>Peep Show S02E02 P01</title>
            <author>babyblue8five</author>
            <pubDate>Sun, 06 Jan 2008 21:44:36 GMT</pubDate>
            <description>Peep Show season 2, episode 2, part 1/3</description>
            <link>http://www.youtube.com/watch?v=gCWmiXEkAGE&feature=youtube_gdata</link>
            <media:group>
                <media:thumbnail url='http://i.ytimg.com/vi/gCWmiXEkAGE/0.jpg'/>
                <media:content url='http://www.youtube.com/watch?v=gCWmiXEkAGE&feature=youtube_gdata' duration='' width='' height='' lang=''/>
            </media:group>
            <rating>4.988506</rating>
        </item>
        <item>
            <title>Peep Show S02E01 P01</title>
            <author>babyblue8five</author>
            <pubDate>Sun, 06 Jan 2008 20:13:10 GMT</pubDate>
            <description>Peep Show Season 2, episode 1, part 1/3</description>
            <link>http://www.youtube.com/watch?v=38EhKzJ0zS8&feature=youtube_gdata</link>
            <media:group>
                <media:thumbnail url='http://i.ytimg.com/vi/38EhKzJ0zS8/0.jpg'/>
                <media:content url='http://www.youtube.com/watch?v=38EhKzJ0zS8&feature=youtube_gdata' duration='' width='' height='' lang=''/>
            </media:group>
            <rating>4.931818</rating>
        </item>
    </channel>
</rss>

The above is a search command which returns two items. Let's break down the results so that you can see how it works.


Establishing The Namespace

 <?xml version="1.0" encoding="UTF-8"?>
 <rss version="2.0"
 xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
 xmlns:content="http://purl.org/rss/1.0/modules/content/"
 xmlns:cnettv="http://cnettv.com/mrss/"
 xmlns:creativeCommons="http://backend.userland.com/creativeCommonsRssModule"
 xmlns:media="http://search.yahoo.com/mrss/"
 xmlns:atom="http://www.w3.org/2005/Atom"
 xmlns:amp="http://www.adobe.com/amp/1.0"
 xmlns:dc="http://purl.org/dc/elements/1.1/"
 xmlns:mythtv="http://www.mythtv.org/wiki/MythNetvision_Grabber_Script_Format">

All MythNetvision returns (Search and Tree) start with the above. This is called setting the XML namespace. Namespaces are custom definitions of RSS/XML tags that can be used in your document. Presently the mythtv, media, amp, and itunes tags are used to some extent, but others may be supported in the future so the above should tentatively be used on all returns.

An RSS "Channel"

    <channel>
        <title>YouTube</title>
        <link>http://www.youtube.com/</link>
        <description>Share your videos with friends, family, and the world.</description>
        <numresults>2</numresults>
        <returned>2</returned>
        <startindex>1</startindex>

All RSS returns in MythNetvision are segmented into "channels." Although it's not generally necessary to have more than one channel in a return, it is possible and acceptable to do so. A channel is a logical grouping of results with a title, description, and some information to allow control of paging in search scripts. Let's take a look at what each of these tags means.

Tag name Description
channel Tag identifies the start of a channel.
title The title of the source of the metadata which will be displayed by the MythNetvision plugin.
link URL of the home page for the source of the metadata.
description Descriptive text about the metadata source.
numresults The number of results returned from a search. Usually this is total items or total pages of items. This tag is only used in search mode.
returned This number reflects the maximum number of items per page or the number returned when a search does not find a full page worth of video items. This tag is only used in search mode.
startindex The start index of the next page or next video item. If this number is greater than or equal to the number of results the MythNetvision plugin will not allow the user to request a next page. This tag is only used in search mode.


A MythNetvision Video Item

        <item>
            <title>Peep Show S02E02 P01</title>
            <mythtv:subtitle>Jeremy Makes It</mythtv:subtitle>
            <author>babyblue8five</author>
            <pubDate>Sun, 06 Jan 2008 21:44:36 GMT</pubDate>
            <description>Peep Show season 2, episode 2, part 1/3</description>
            <link>http://www.youtube.com/watch?v=gCWmiXEkAGE&feature=youtube_gdata</link>
            <media:group>
                <media:thumbnail url='http://i.ytimg.com/vi/gCWmiXEkAGE/0.jpg'/>
                <media:content url='http://www.youtube.com/watch?v=gCWmiXEkAGE&feature=youtube_gdata' length='' duration='' width='' height='' lang=''/>
            </media:group>
            <rating>4.988506</rating>
            <mythtv:country>us</mythtv:country>
            <mythtv:country>uk</mythtv:country>
            <mythtv:season>2</mythtv:season>
            <mythtv:episode>2</mythtv:episode>
        </item>

The above represents a single viewable video in MythNetvision. By manipulating the format of the information, you can express that the item is downloadable, web view only, and present different types of metadata.

Tag name Description
item Identifies the start of a video item.
title The title text for the video.
mythtv:subtitle The subtitle text for the video.
author The author who submitted the video to the source web site. Often this is submitter's user id or username.
pubDate The date that the video was submitted to the site. The format of "Thu, 17 Dec 2009 13:51:34 GMT" must always be used. If you cannot retrieve the timezone, simply specify the approximate timezone in which the source exists. If you do not know the time then set it to all zeros "00:00:00". pubDate follows the RFC-822 date-time format.
description The text description of the video. A description should not contain any HTML or line-feeds.
link A URL to the web page containing the video or directly to the video player. If this link is not identical to the "media:content url" tag then that indicates to MythNetvision that the "media:content url" file can be downloaded. Unless the source site TOS allows a video file to be downloaded then the "link" and "media:content url" URLs should be identical (Web View Only).
player Note: Do not populate this field if you wish to use MythTV's built-in player! The executable name to an external player application. An example might be something like:
<player>mplayer</player>
playerargs Note: Do not populate this field if you wish to use MythTV's built-in player! The arguments passed to the external binary in the <player> tags. An example might be something like:
<playerargs>-fs -zoom mms://123.45.6.78/</playerargs>

You can insert variables here and MythNetvision will dynamically insert the correct values before playback. They are as follows:

%URL% - The <link> value.
%MEDIAURL% - The <media:content> url= value
download Note: Do not populate this field if you wish to use MythNetvision's built-in download handling! The executable name to an external downloader application. An example might be something like:
<download>mydownloadscript</download>

An external download script must return a single line upon a successful result: The fully qualified path to the downloaded file. MythNetvision then reads in this value and uses it to play back the item with the internal player and (in the future) to perform any insertions into the database.

Example download script use:

./mydownloadscript -outpath %HOMEDIR% -outfile %FILE% http://my.ip.add.ress/file.html

Example script output:

/home/someguy/download_123_456.mp4

Note that you may output logging and errors to stderr, but only the output filename must be emitted to stdout. The above example shows the use of userjob-style variables, which are discussed below in <downloadargs>.

Not yet fully implemented.

downloadargs Note: Do not populate this field if you wish to use MythNetvision's built-in download handling! The arguments passed to the external binary in the <download> tags. An example might be something like:
<downloadargs>-outpath /home/someguy/ -outfile somefile.flv http://path.to.a/thing.html</downloadargs>

In addition to defining static locations, you can insert the following userjob-style variables in your downloader arguments, which MythNetvision will then replace with the local values:

%HOMEDIR% The frontend user's home directory.

%FILE% The output filename suggestion in MythNetvision format.

Not yet fully implemented.

media:group Indicates the start of a video's thumbnail and metadata. The item must eventually be closed with a "</media:group>"
media:thumbnail url A URL to the thumbnail image for the video file. The source site does not always have a thumbnail for a video. When there are multiple thumbnails choose the first one over 200 pixels wide as it will display well but keep download times reasonable.
media:content The main tag for the video link and video metadata.
url A URL to the video file/player/web page. This link should only differ from the URL "link" tag when the URL points to a downloadable video file.
length Unlike what it appears to be, length is actually the filesize in bytes. This is only used for downloadable media.
duration The duration of the video in seconds. Some sources use a format of "MM:SS". This value must be converted to seconds.
width The width in pixels of the video.
height The height in pixels of the video.
lang The ISO-639 language code of the video. Two character language codes
rating A floating point number representing a numerical rating for the video. There is no consistency for this value between the source sites. The most common values are X out or 5 or 10.
mythtv:country A country in which the video is playable. If the video is playable anywhere, return no country. If the video is only playable in certain locales, or the Terms of Service only allow playback in certain locales, return the country code for the playable countries here. You may add as many country items as you like.
mythtv:season An unsigned integer representing the season number of the item. Optional.
mythtv:episode An unsigned integer representing the episode number of the item. Optional.

Clean This Mess Up, Mister!

    </channel>
</rss>

For those unfamiliar with XML, RSS, or HTML, *all* tags must be closed or parsing the document will fail.

MythNetvision Tree Grabber Scripts

Let's examine the following example of a Site Tree building script:

Command Line:

/youtube.py -T

Returned XML:

<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
xmlns:content="http://purl.org/rss/1.0/modules/content/"
xmlns:cnettv="http://cnettv.com/mrss/"
xmlns:creativeCommons="http://backend.userland.com/creativeCommonsRssModule"
xmlns:media="http://search.yahoo.com/mrss/"
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:amp="http://www.adobe.com/amp/1.0"
xmlns:dc="http://purl.org/dc/elements/1.1/">
    <channel>
        <title>YouTube</title>
        <link>http://www.youtube.com/</link>
        <description>Share your videos with friends, family, and the world.</description>
            <directory name="Feeds" thumbnail="/usr/share/mythtv/mythnetvision/icons/youtube.png">
                <directory name="Highest Rated" thumbnail="/usr/share/mythtv/mythnetvision/icons/directories/topics/rated.png">
                    <item>
                        <title>Project for Awesome - My Public Access Channel!</title>
                        <author>peron75</author>
                        <pubDate>Thu, 17 Dec 2009 13:51:34 GMT</pubDate>
                        <description>Please support all the Project for Awesome videos today with ratings/comments! Thank you! Thank you to Hank and John Green, Dan Brown and everyone 
involved!!!  I chose the public access station where I began What the Buck! They are fundraising to help with their new building project.  You can help buy simply signing up for th
is site and then when you shop, they get donations from that! Yay! Thanks if you can sign up! (its Free!) LOL xoxo Michael   Please sign up: http://igive.com/wpaa</description>
                        <link>http://www.youtube.com/v/tdBHzkoXB_8?f=standard&app=youtube_gdata&autoplay=1</link>
                        <media:group>
                            <media:thumbnail url='http://i.ytimg.com/vi/tdBHzkoXB_8/hqdefault.jpg'/>
                            <media:content url='http://www.youtube.com/v/tdBHzkoXB_8?f=standard&app=youtube_gdata&autoplay=1' duration='259' width='' height='' lang=''/>
                        </media:group>
                        <rating>4.9695706</rating>
                    </item>
                </directory>
            </directory>
    </channel>
</rss>

Wow, this is remarkably familiar, isn't it? The format of a site tree is very similar to the search view, and extremely similar to any old RSS feed. Let's only look at the items which differ from a search script.


Directories/Tree Hierarchy

            <directory name="Feeds" thumbnail="/usr/local/share/mythtv/mythnetvision/icons/youtube.png">
                <directory name="Highest Rated" thumbnail="/usr/local/share/mythtv/mythnetvision/icons/directories/topics/rated.png">

Shocking as it might seem, the only new element added in a tree grabber is the directory tag, which allows you to create a filesystem-like structure for your tree. The above example contains two nested directories, meaning the represented path would be:

YouTube/Feeds/Highest Rated/

Directories include a name, which is the directory name, and a thumbnail attribute, which is an option URL or absolute path to a folder icon.

Tag name Description
directory Identifies the start of a directory. The directory must eventually be closed with a "</directory>"
name Descriptive label for the directory. This description will be displayed by the MythNetvision plugin.
thumbnail An image URL or absolute file path for the image icon that will be displayed by the MythNetvision plugin.

Metadata standards

Consider the following "rules of thumb" when writing your grabber:

  • When a search returns no results then no RSS data is returned, exit with a return code of zero.
  • Do not include any video items or directories that do not have at least one video link.
  • Always include every tag even if the source does not have any data for that tag. Leave the content empty.
width=""
<description></description>
  • Whenever possible always use a video link that auto starts the video and displayed in full screen.
    • Sometimes the method to auto start or display in full screen is not included in the API documentation. The video's HTML Web page source can sometimes provide this information.
  • Replace any ampersand characters "&" with "&amp;".
  • Remove line feed characters and HTML tags from any text fields. This is usually an issue with the description and title text of a video.
  • Do not use the word "video" in any of your Tree view directory descriptions or grabber names as it is redundant (MythNetvision is a video plugin, after all).
  • Additional tags may by included by a grabber but they will be ignored by the MythNetvision plugin.


Tree view thumbnail icon images

  • Each directory tag can include a thumbnail image.
  • The thumbnail may be an absolute path or a URL.
  • The MythNetvision core icon set is found in "/usr{/local}/share/mythtv/mythnetvision/icons"
  • Included with MythNetvision are a set of generic directory icon images that can be used. See "/usr{/local}/share/mythtv/mythnetvision/icons/directories".


Error handling

  • Standard out (stdout) is reserved exclusively for valid RSS data read by the MythNetvision plugin.
  • Standard error (stderr) is reserved exclusively for error or grabber processing messages.
  • Logging is optional.
  • Return code is zero for any processing that completed without a script abort. This includes a search or treeview that did not return any results.
  • Return code of one if an error forced the premature termination of the grabber.
  • If a grabber script skips bad data during process an error about the bad data should be sent to stderr but a return code of zero would still be returned indicating the grabber completed its processing successfully.