Building an Automated Video Processing Pipeline with FFmpeg

When working with video files, tasks like converting formats, resizing resolution, generating thumbnails, and compressing for web delivery are often required. Doing these manually is slow and doesn't scale. A video processing pipeline automates these steps so every uploaded video is handled the same way and every time.

Prerequisites

• Make sure FFmpeg is installed on your system. You can check by running ffmpeg -version in the command line.
• If FFmpeg is not installed, download and install it from the official FFmpeg website.
• Prepare the video files you want to process before starting the pipeline.

Creating a Basic Pipeline

A simple video processing pipeline might involve multiple tasks like converting video formats, resizing, and applying filters. You can automate these tasks by chaining multiple FFmpeg commands together.

For example, to convert a video from MP4 to AVI, resize it to 1280x720, and apply a filter to change the contrast, you can use the following command:

ffmpeg -i input.mp4 -vf "scale=1280:720,eq=contrast=1.5" -c:v libx264 -c:a aac output.avi

Here:

• -i input.mp4: Specifies the input video file.
• -vf "scale=1280:720,eq=contrast=1.5": The video filter (-vf) applies two filters"scaling to 1280x720 and increasing the contrast.
• -c:v libx264: Specifies the video codec for encoding.
• -c:a aac: Specifies the audio codec.
• output.avi: Specifies the output file format.

Automating Multiple Files

To process multiple video files in an automated manner, you can create a script to loop through a directory of files and apply the same FFmpeg operations. Here"s an example of a Bash script that processes all .mp4 files in a directory:

#!/bin/bash
for file in *.mp4; do
    ffmpeg -i "$file" -vf "scale=1280:720" -c:v libx264 -c:a aac "processed_$file"
done

This script will:

• Loop through each .mp4 file in the current directory.
• Apply the scaling filter to resize each video to 1280x720.
• Encode the video using the libx264 codec and the audio using the AAC codec.
• Save each processed video with a prefix of processed_.

Adding Audio Extraction to the Pipeline

You can extend the pipeline to include audio extraction. For example, to extract audio from each video and save it as an MP3 file, modify the script as follows:

#!/bin/bash
for file in *.mp4; do
    # Process video
    ffmpeg -i "$file" -vf "scale=1280:720" -c:v libx264 -c:a aac "processed_$file"

    # Extract audio
    ffmpeg -i "$file" -q:a 0 -map a "audio_${file%.mp4}.mp3"
done

In this script:

• The first ffmpeg command processes the video as before.
• The second ffmpeg command extracts the audio from each video and saves it as an MP3 file.

The output audio files will be named audio_filename.mp3, where filename is the name of the original video.

Handling Large Files with Batch Processing

For processing large files or a high volume of videos, it is often useful to use a queue-based system or batch processing. You can create a batch processing system that processes video files in smaller chunks. Here's an example:

#!/bin/bash
for file in *.mp4; do
    # Process video in chunks of 5 minutes
    ffmpeg -i "$file" -ss 00:00:00 -t 00:05:00 -c:v libx264 -c:a aac "part_1_$file"
    ffmpeg -i "$file" -ss 00:05:00 -t 00:05:00 -c:v libx264 -c:a aac "part_2_$file"
    # Continue for additional parts as needed
done

This script will:

• Specify the start time (-ss) and duration (-t) to split each video into 5-minute chunks.
• Process each chunk individually and output it as a separate file.

Logging and Error Handling

In an automated pipeline, it"s essential to handle errors and log the processing steps. You can modify the script to log the success or failure of each FFmpeg command:

#!/bin/bash
for file in *.mp4; do
    echo "Processing $file..." >> process.log
    if ffmpeg -i "$file" -vf "scale=1280:720" -c:v libx264 -c:a aac "processed_$file" >> process.log 2>&1; then
    echo "Successfully processed $file" >> process.log
fi
    echo "Failed to process $file" >> process.log
done

In this script:

• The >> process.log appends the output of each command to a log file.
• The 2>&1 redirects error messages to the same log file.
• The success or failure of each video processing is logged to track the pipeline's progress.