Use Slicebot to monitor one or more directories for new files. Once a file is detected, it is automatically sliced and uploaded to the cloud.
Slicebot is included with the Slicer installation.
Slicebot is not officially supported on Windows.
Setting up Slicebot consists of the following steps:
Start Slicebot by running the following command:
This section explains:
Configuration settings may be defined through the following mechanisms:
Defines a default configuration, identifies the set of directories to be monitored, and defines directory-specific configurations.
File-specific Configuration File
Defines how a specific file will be processed.
Defines how a specific file will be processed.
Settings will be applied in the following order:
As a result, settings defined within a file name will take precedence over any conflicting settings defined elsewhere.
Slicebot requires a configuration file that contains settings that define:
Key information:
Create this configuration file using a text editor.
Categorize settings into the following sections:
Directory-specific Settings: A section must be created for each desired monitored directory.
The header for this type of section should be set to the full path to the desired directory.
Example:
The settings defined in this type of section:
A section may be left blank. A blank section indicates that the specified directory should be monitored and processed according to the configuration defined in:
Settings must be defined using the following format:
A sample configuration file is provided below.
[global]
enc_wait = 1
mail_host = smtp.gmail.com:587
done_emails = sys@mycompany.com, me@mycompany.com
fail_emails = me@mycompany.com
[~/t/slice/a/]
[~/t/slice/b/]
enc_wait = 0
username = me@email.com
apikey = abcDEFghiJKLmnoPQRtuvWXYz123ABCdefGHIJKL
Instructions on when and how a specific file is processed may be defined by creating a configuration file that contains the desired configuration settings.
Although most configuration settings may be defined within a file-specific configuration, there are exceptions (e.g., halt_on_error and ignore). The description associated with each setting indicates any limitations on where it may be implemented.
Key information:
File-specific settings may be defined in a text file that meets the following conditions:
The file extension for this file should be "cfg" in lower-case (e.g., myvideo.mp4.cfg).
Use the helper_noext setting to exclude the source media's file extension (e.g., mp4).
A sample configuration file is provided below.
enc_wait = 0
done_emails = marketing@mycompany.com, joe@mycompany.com
fail_emails = jane@mycompany.com
Instructions on when and how a specific file is processed may be defined by including the desired settings within the source media's file name.
Slicebot will only honor settings defined within a file name when the allow_inline setting has been enabled via either the main or a file-specific configuration file.
Although most configuration settings may be defined within a file name, there are exceptions (e.g., halt_on_error and ignore). The description associated with each setting indicates any limitations on where it may be implemented.
Syntax:
Example:
Original file name:
New file name:
Follow these guidelines when defining a setting within a file name:
A file name should never contain spaces.
Convert space characters within a setting's value to the plus (+) symbol.
Example:
In the above example, the asset's name will be set "Standard Trailer."
A matching file name must still be assigned to any supporting files (e.g., closed captioning or file-specific configuration file).
Closed Captioning Example:
The helper_noext setting determines whether source media's file extension (e.g., mp4) needs to be included when naming supporting files.
Avoid creating file names longer than those supported by the OS.
Use file-specific configuration files to avoid OS limitations.
Use --dry-run option to test your configuration without processing files.
Slicebot runs indefinitely until one of the following conditions is true:
It is manually interrupted. The recommended method of interrupting Slicebot is through the SIGINT signal. This signal allows Slicebot to finish the current slicing job before shutting down. Pass this signal to Slicebot by running the following command:
This section describes the supported configuration settings that can be used to control how slicebot processes files.
Key information:
Setting values support basic value substitution:
[FILENAME] - replaced with the current source filename, minus the path. Example, with a source file of /mnt/files/ad.mp4:
external_id=foo[FILENAME]bar
would become
external_id=fooad.mp4bar
[FILENAME_SHORT] - replaced with the current source filename, minus the path and file extension. Example, with a source file of /mnt/files/ad.mp4:
external_id=foo[FILENAME_SHORT]bar
would become
external_id=fooadbar
The following configuration settings may be leveraged when defining:
Setting |
Description |
Example |
||||||
---|---|---|---|---|---|---|---|---|
Directory Monitoring Determines whether the source media's file name may contain configuration settings. Valid values are:
Default value: 1 |
allow_inline = 0 |
|||||||
Required |
Requires Slicer version 17111500 or higher Authorization Sets the secret API key through which Slicebot authenticates to the CMS. Bypass this requirement by setting the SLICER_USER and the SLICER_APIKEY environment variables. This API key should correspond to the user defined by the username setting. Where can I find my API key(s)?
Syntax: |
apikey = abc...apikey = abcDEFghiJKLmnoPQRtuvWXYz123ABCdefGHIJKL |
||||||
Asset Replacement (Library) Determines whether asset(s) with the same external ID will be automatically replaced. Valid values are:
|
auto_replace = 1 |
|||||||
Asset Retention (Library) Marks an asset for deletion once the specified number of hours have elapsed. The sample configuration, shown to the right, will delete assets 48 hours after they have been encoded. |
autoexpire = 48 |
|||||||
Ad Breaks (Asset) Inserts an ad break for each specified time range. Key information:
Single Ad Break Syntax: breaks = StartTimeReplace this variable with the start time of the desired ad break.-StopTimeReplace this variable with the stop time of the desired ad break. Multiple Ad Breaks Syntax: breaks = StartTime1-StopTime1,StartTime2-StopTime2,StartTimeN-StopTimeN |
breaks = 10.1-30.58,105-151.332 |
|||||||
Asset Overlays an image on the video at the specified start time for the given duration. Key information:
Syntax: bug = PNGFileReplace this variable with the path to the desired PNG image.,StartTimeReplace this variable with the start time, in milliseconds, of the image overlay effect.,DurationReplace this variable with the duration, in milliseconds, of the image overlay effect.
The sample configuration, shown to the right, will add overlay1.png at the start of the asset for 5 seconds and overlay2.png five seconds before the end of the asset. If these two overlay effects overlap, then overlay2.png would appear on top of overlay1.png. |
bug = overlay1.png,0,5000,,,overlay2.png,-5000,5000 |
|||||||
Asset (Library) Assigns a name to the asset generated from the source media. This setting is typically used when defining file-specific configurations. By default, assets are named after the source media's file name. |
description = Marketing Event 2016 |
|||||||
Post-Processing Defines the directory to which the source media will be moved after it has been successfully processed. Key information:
|
done_directory = ~/media/success |
|||||||
Notifications Defines a comma-separated list of email addresses to which an email notification will be sent when a file is successfully processed. This setting requires the mail_host setting. |
done_emails = fred@gmail.com, george@example.com |
|||||||
Notifications Defines the URL to which a HTTP POST request will be submitted when a file is successfully processed. |
done_url = http://myserver.net/myservice |
|||||||
Slicebot Determines whether Slicebot will wait until after the cloud finishes encoding an asset before it starts processing the next source media file. Valid values are:
Default value: 0 |
enc_wait = 1 |
|||||||
Asset (Library) Assigns an external ID to the asset generated from the source media. By default, the same external ID may be assigned to multiple assets. Override this behavior by adding the "_replace:" prefix to the desired external ID. This will cause the new asset to replace existing assets with the same external ID. |
external_id = ep20115e_12 external_id = _replace:ep20115e_12 |
|||||||
Asset Fades audio and/or video in/out from the specified start time. Key information:
Syntax: fade = MethodReplace this variable with either "in" or "out" to fade in or out, respectively.,MediaTypeReplace this variable with one of the following values: audio, video, or both.,StartTimeReplace this variable with the start time, in milliseconds, of the fade effect.,DurationReplace this variable with the duration, in milliseconds, of the fade effect.
The sample configuration, shown to the right, will fade in audio/video at the start of the asset for 5 seconds and then fade out 5 seconds before the end of the asset. |
fade = in,both,0,5000,,,out,both,-5000,5000 |
|||||||
Post-Processing Defines the directory to which the source media will be moved when it is not successfully processed. Key information:
|
fail_directory = ~/media/failure |
|||||||
Notifications Defines a comma-separated list of email addresses to which an email notification will be sent when a file is unsuccessfully processed. This setting requires the mail_host setting. |
fail_emails = ops@company.org |
|||||||
Notifications Defines a URL to which a HTTP POST request will be submitted when a file is unsuccessfully processed. |
fail_url = http://myserver.net/myservice2?status=fail |
|||||||
Asset Forces the video to the specified aspect ratio. Syntax: |
force_aspect_ratio = 16:9 |
|||||||
Source Media Processing Determines whether processing will stop when an issue occurs. This setting may only be defined within the main configuration file. Valid values are:
Default value: 0 |
halt_on_error = 1 |
|||||||
Directory Monitoring Determines whether supporting files must include the source media's file extension. Valid values are:
|
helper_noext = 1 |
|||||||
Configuration File Determines whether a directory will be monitored for new files. This setting should only be defined under a directory-specific section within the main configuration file. Valid values are:
|
ignore = 1 |
|||||||
Directory Monitoring Ignores files in a monitored directory with the specified file name. Key information:
|
ignoreName = thumbs.db,.DS_Store |
|||||||
Directory Monitoring Ignores files in a monitored directory with the specified file extension(s). Key information:
|
ignoreSuffix = db,txt,jpg |
|||||||
Asset (Library) Adds an asset to one or more shared libraries. Key information:
Add to Library by Name Syntax: libraries = LibraryName Add to Library by GUID Syntax: libraries = LibraryGUID Add to Multiple Libraries Syntax: |
||||||||
Notifications Determines the sender email address (i.e., from) for email notifications. |
mail_from = automation@company.org |
|||||||
Notifications Defines the hostname for the mail server through which email notifications will be sent. A port number may be appended to the specified host. Mail servers typically use the following ports:
If a mail server requires authorization before sending out emails, then the mail_password and mail_username settings must also be defined. Syntax: |
mail_host = smtp.company.com:587 |
|||||||
Notifications Defines the password for the user defined by the mail_username setting. This setting is only required when a mail server requires authorization before sending emails. |
mail_password = G183hIU39331f |
|||||||
Notifications Defines the name of the user account on the mail server through which email notifications will be sent. This setting is only required when a mail server requires authorization before sending emails. |
mail_username = robo-emails@example.com |
|||||||
Slicing Limits the number of temporal slices to generate from the source media. Key information:
|
max_slices = 20 |
|||||||
Asset (Library) Adds metadata to the asset. Key information:
Syntax: meta = FieldReplace this variable with the name of the desired metadata field.=ValueReplace this variable with the value that will be assigned to the specified metadata field.
An alternative method for assigning metadata is through a JSON file. |
meta = rating=TV-13 meta = rating=TV-13,,,air_date=20160105 |
|||||||
Asset (Library) Adds metadata containing an integer value to the asset. Key information:
Syntax: |
meta_int = is_ad=1 meta_int = is_ad=1,,,year=2013 |
|||||||
Asset Mixes multiple audio tracks together into a single track. A sample use case is when the source audio has separate left and right audio tracks instead of a single stereo audio track. Key information:
Syntax: mixatracks = Track1Replace this variable with a 0-based audio track number.=Channel1Replace this variable with the desired channel. Valid values are: L,R,C,RL,RR,SL,SR,and LFE.,Track2=Channel2=,TrackN=ChannelN
|
mix_atracks = 0=L,1=R,2=C,3=RL,4=RR,5=LFE |
|||||||
Directory Monitoring Indicates whether multiple instances of Slicebot are monitoring the same directory. Valid values are:
Default value: 1 |
multibot = 1 |
|||||||
Notifications Determines the maximum number of attempts to send out a HTTP notification before proceeding to the next file. Notifications are defined through the following settings: Default value: 3 |
notify_retries = 10 |
|||||||
Notifications Determines the number of seconds that must pass after a failed notification before reattempting to send out a notification. Default value: 10 |
notify_retry_wait = 30 |
|||||||
password Deprecated |
This setting and the SLICER_PASS environment variable have been deprecated and should not be used. Please use the apikey setting or the SLICER_APIKEY environment variable instead. Defines the password that corresponds to the user defined by the username setting. |
password = 1@D208PneG63f |
||||||
Asset Generates a poster image from the video frame corresponding to the specified time. This parameter is ignored when poster_file has been defined. Syntax: |
poster = 5000 |
|||||||
Asset Sets the asset's poster image to the specified file. Syntax: |
poster_file = /images/poster.png |
|||||||
Asset Sets the size of the poster image. Key information:
Syntax: |
poster_size = 512x512 |
|||||||
Processing Prerequisite Determines whether a closed captioning file is required. Valid values are:
|
require_captions = 1 |
|||||||
Processing Prerequisite Determines whether a file-specific configuration file is required. Valid values are:
This is an invalid setting for file-specific configuration files. |
require_config = 1 |
|||||||
Asset Determines whether the asset's playback URLs must be signed. Valid values are:
|
skip_drm = 1 |
|||||||
Defines the URL to which a HTTP POST request will be submitted whenever slicing is initiated on a file. |
start_url = http://myserver.net/myservice |
|||||||
Asset Adds one or more thumbnails. Key information:
Syntax: thumbnails = PrefixReplace this variable with the prefix that will be assigned to the thumbnails generated by this option.:WidthReplace this variable with the desired thumbnail width in pixels.xHeightReplace this variable with the desired thumbnail height in pixels.,Prefix2:Width2xHeight2,PrefixN:WidthNxHeightN
|
thumbnails = tiny:50x50,med:400x400 |
|||||||
Asset Adds timed metadata at the specified time. Single Field Syntax: timedmeta = TimeReplace this variable with the time, in milliseconds, at which the metadata will be inserted. :FieldReplace this variable with the name of the desired metadata field.=ValueReplace this variable with the value that will be assigned to the specified metadata field.
Multiple Fields Syntax: timedmeta = Time1:Field1=Value1,Time2:Field2=Value2,TimeN:FieldN=ValueN
|
timedmeta = 0:TIT2=MyTitle, 2000:TCOM=Myself |
|||||||
Asset Trims the content from the source media that occurs prior to the specified start time. Optionally, duration may be specified to trim content that exceeds the specified time period. Specify StartTime and Duration in milliseconds. The source media's file type may affect video frame accuracy by a few milliseconds. Trim Content Prior to Start Time Syntax: Trim Content by Start Time and Duration Syntax: trim = StartTimeReplace this variable with the time at which the asset will start.,DurationReplace this variable with the maximum duration of the asset. The sample configuration, shown to the right, will exclude the first 3 seconds of the source media and truncate it so that the asset's total duration is 52.5 seconds. |
trim = 3000,52500 |
|||||||
Required |
Authorization Defines the user name corresponding to the account to which the asset will be uploaded. Bypass this requirement by setting the SLICER_USER and SLICER_APIKEY environment variables. |
username = joe@example.com |
You can provide slicebot with closed caption information by placing closed caption data in a TTML file in the same directory as the media file. TTML files are simple text files that can be created with third party tools or with any text editor:
<?xml version="1.0" encoding="utf-8"?> <tt xml:lang="en" xmlns="http://www.w3.org/2006/04/ttaf1" xmlns:tts="http://www.w3.org/2006/04/ttaf1#styling"> <head> <styling> <style id="defaultSpeaker" tts:fontSize="24" tts:fontFamily="Arial" tts:fontWeight="normal" tts:fontStyle="normal" tts:textDecoration="none" tts:color="white" tts:backgroundColor="black" tts:textAlign="center" /> <style id="defaultCaption" tts:fontSize="24" tts:fontFamily="Arial" tts:fontWeight="normal" tts:fontStyle="normal" tts:textDecoration="none" tts:color="white" tts:backgroundColor="black" tts:textAlign="center" /> </styling> </head> <body style="defaultCaption" id="thebody"> <div xml:lang="en"> <p begin="00:00:01.420" end="00:00:02.620"><metadata ccrow="14" cccol="3" /><span tts:fontStyle="italic">THIS IS SOME TEXT...</span></p> <p begin="00:00:06.620" end="00:00:09.790"><metadata ccrow="13" cccol="1" />(Doorbell rings)</p> <p begin="00:00:11.100" end="00:00:13.000"><metadata ccrow="14" cccol="8" />♪ LA LA LA SINGING ♪</p> <p begin="00:00:20.460" end="00:00:22.170"><metadata ccrow="14" cccol="0" />THIS IS MORE TEXT</p> </div> </body> </tt>
The require_captions config setting can be used to ensure that slicebot will skip a file if it does not have a corresponding TTML file. As with other settings, it can be used in the global, directory, or per-file context.
The TTML files use a naming convention similar to config files: take the name of the associated media file and add a ".ttml" extension to the filename. For example, if slicebot is processing a file named /tmp/foo/film.mp4, it will look to see if there is a file named /tmp/foo/film.mp4.ttml and, if so, use it for the source of closed caption data. If no corresponding TTML file exists, the video will not have any closed caption data. Note that the helper_noext setting can alter the naming convention for TTML files.
Only hh:mm:ss.sss time formats are currently supported in the TTML file.
You can assign metadata key-value pairs to an asset using individual settings; for larger amounts of metadata, however, this can be cumbersome, so it may be more convenient to make use of a metadata file. This file is a JSON text file that has the metadata you want associated with the media file. As with caption and config files, the name of the metadata file is the media filename with an additional extension: .meta.json. For example, if a media file is named video123.mp4, then an associated metadata file for it would be named video123.mp4.meta.json.
The contents of the metadata file must be a valid JSON object, or processing the file will produce an error. Structured metadata is supported, although presently it is not fully displayed or editable in the CMS UI.
{ "genre" : "comedy", "network" : "ZNN", "season" : 1, "episode" : 13 }
In general, a given media file will use individual metadata settings or a metadata file, but not both. However, in the event that both are specified for a particular file, metadata from a file is applied first, followed by any metadata settings.
HTTP notifications provide an additional way to integrate a system with your content production workflow. This type of notification allows Slicebot to submit a HTTP POST request whenever one of the following events takes place:
Enable HTTP notifications by setting one or more of the above settings to the URL that will be requested.
Email notifications may also be sent whenever slicing completes (done_emails) or when an error occurs (fail_emails).
Slicebot will submit a HTTP POST request using the default Internet media type (i.e., application/x-www-form-urlencoded). The body of this request will consist of a single line with the following parameters:
Parameter | Description |
---|---|
filename |
Indicates the full path and the filename of the file. |
asset_id |
Indicates the system-generated unique ID of the new asset. |
external_id |
Indicates the external ID defined in the configuration. If one has not been defined, then it will be set to an empty string. |
description |
Indicates the file's description defined in the configuration. If one has not been defined, then it will be set to an empty string. |
status |
Indicates the result of the slicing operation. Valid values are either "ok" or "error." |
In addition to the above parameters, any custom query string parameters defined in the configuration setting (i.e., start_url, done_url, and fail_url) will be automatically stripped from the request and inserted into the request body. This allows custom parameters to be passed to the remote server as needed.
Custom query string parameters whose names match one of the above system-defined parameters (e.g., filename or description) will be ignored.
To allow for future expansion, the server process that receives the HTTP POST should silently ignore unrecognized parameters.
Slicebot will log an error under the following conditions:
After which, it will reattempt the HTTP notification several times as determined by the notify_retries setting. After each attempt, it will pause for a few seconds as determined by the notify_retry_wait setting. If it is unable to successfully submit the HTTP POST request, then it will move on to the next file.
This sample HTTP notification scenario assumes that the done_url setting is configured to:
The following sample HTTP POST request will be submitted once a file called "video.m4v" has been successfully sliced.
POST /slicing/handler HTTP/1.1
Host: www.example.com
Accept-Encoding: identity
Content-Length: 161
Content-Type: application/x-www-form-urlencoded
Connection: close
User-Agent: upLynk-slicebot/1.0
status=ok&other=5678&description=The%20file&filename=%2Fopt%2Ft%2Fmyvideos%2Fvideo.m4v&external_id=dummy_id&custom=1234&asset_id=a9119d7afc5242319d028794ae283ea7
Notice that the custom query string parameters (i.e., custom and other) defined in the done_url setting were automatically stripped from the request and inserted into the request body.