What is Node.js? A Beginner???s Guide

JavaScript is used in browsers to build interactive user interfaces, but you can use it on servers to handle backend processes. Node.js is a runtime environment that allows developers to run JavaScript outside the browser, on servers, and to use a single language for client-side and server-side development.

Developers use Node.js to write both front-end and back-end code in JavaScript, so they don"t need to learn different languages for each part of an application. With Node.js, developers can create fast & scalable applications that handle real-time data, such as chat applications.

Core Characteristics

Single-Threaded Event Loop

Node.js uses a non-blocking and event-driven architecture. While it runs on a single thread, it can handle concurrent connections efficiently via asynchronous callbacks. This architecture avoids thread management overhead typical in multi-threaded servers.

Asynchronous I/O

I/O operations (such as file reading, network requests, and database access) are handled atthe same time. This prevents blocking and improves throughput in I/O-heavy applications.

Example:

const fs = require('fs');

fs.readFile('file.txt', 'utf8', (err, data) => {

if (err) throw err;

console.log(data);

});

Here, readFile does not block the thread. Execution continues while the file is being read.

Modules and npm

Node.js uses the CommonJS module system and is integrated with npm (Node Package Manager). This provides access to a vast ecosystem of third-party libraries. This code creates a basic HTTP server without external dependencies.

Example Of Module Usage:

const http = require('http');

http.createServer((req, res) => {

res.end('Hello World');

}).listen(3000);

Built-in Modules

Node.js provides a set of built-in modules for developers to interact with the system and manage files, HTTP requests, etc. without external dependencies. Some core modules include:

fs (File System): Provides an API for interacting with the file system, enabling tasks like reading, writing, and managing files.

http: Allows for the creation of HTTP servers. This module builds web applications and APIs.

crypto: Provides cryptographic functionalities (such as hashing and encryption) for security in video-related operations.

child_process: Executes system commands and allows for running shell commands from Node.js.

How Node.js Handles Execution

Node.js uses the libuv library to implement the event loop. This loop checks for completed tasks and triggers their callbacks. This model keeps applications responsive even with high concurrency.

Timer Phase

This phase is responsible for executing callbacks scheduled by setTimeout() and setInterval() once their delay has elapsed. The timers are not guaranteed to run on schedule; they are queued once the delay is over and run in the next iteration of the event loop.

I/O Callbacks

In this phase, the event loop processes callbacks from completed I/O operations such as disk or network tasks. These include some system-level operations like TCP errors or asynchronous file access for timely handling of external responses.

Poll

The poll phase actively waits for new I/O events to arrive and processes them if available. If no timers are due and there are no immediate callbacks, the event loop can idle here while waiting for incoming data or I/O events to process.

Check

This phase is specifically designed to execute setImmediate() callbacks. They are scheduled to run after the poll phase. It gives developers predictable scheduling control, as setImmediate() callbacks always run before any additional timers.

Close

The close phase handles cleanup for resources such as sockets or file descriptors that have been closed. If an object emits a 'close' event (like a stream), the associated cleanup callbacks are run in this final step of the loop.

Node.js in Video Streaming and Processing Workflows

Uploading and Validating Video Files

When handling video files in a web application, manage memory to prevent the app from being overloaded with large files. Node.js supports middleware, which helps with memory-efficient file uploads by streaming files instead of loading them into memory. Multer uses a streaming approach to handle file uploads for applications where large files are uploaded.

const multer = require('multer');

const upload = multer({ dest: 'uploads/' });

app.post('/upload', upload.single('video'), (req, res) => {

// req.file contains uploaded video metadata

res.send('Upload complete');

});

Explanation:

• const multer = require('multer');: Imports the multer library, which is used for handling file uploads in a Node.js application.
• const upload = multer({ dest: 'uploads/' });: Configures multer to store uploaded files in the 'uploads' directory.
• app.post('/upload', upload.single('video'), (req, res) => {...});: Sets up a POST route that listens for file uploads at the /upload endpoint and processes a single file named 'video'.
• res.send('Upload complete');: Sends a response back to the client indicating the file upload is complete.

Serving Video Content with Byte-Range Streaming

For large video files, streaming them in small chunks is efficient. Node.js allows you to serve partial content through byte-range requests, which are useful for media applications like video streaming services.

app.get('/video', (req, res) => {

const path = 'videos/sample.mp4';

const stat = fs.statSync(path);

const fileSize = stat.size;

const range = req.headers.range;

if (range) {

const [start, end] = range.replace(/bytes=/, '').split('-');

const chunkStart = parseInt(start, 10);

const chunkEnd = end ? parseInt(end, 10) : fileSize - 1;

const stream = fs.createReadStream(path, { start: chunkStart, end: chunkEnd });

res.writeHead(206, {

'Content-Range': `bytes ${chunkStart}-${chunkEnd}/${fileSize}`,

'Accept-Ranges': 'bytes',

'Content-Length': chunkEnd - chunkStart + 1,

'Content-Type': 'video/mp4'

});

stream.pipe(res);

} else {

res.writeHead(200, { 'Content-Length': fileSize, 'Content-Type': 'video/mp4' });

fs.createReadStream(path).pipe(res);

}

});

Explanation:

• app.get('/video', (req, res) => {...});: Defines a route that handles GET requests to the /video endpoint, which serves the video file.
• const range = req.headers.range;: Extracts the range of the video requested by the client from the HTTP request headers.
• if (range) {...} else {...}: Checks if the client requested a specific range of the video (for partial content) or the entire file.
• const chunkStart = parseInt(start, 10);: Converts the start byte position from a string to an integer.
• const chunkEnd = end ? parseInt(end, 10) : fileSize - 1;: Converts the end byte position from a string to an integer, using the file size if no end is provided.
• res.writeHead(206, {...});: Sends an HTTP status code 206 (Partial Content) along with headers indicating the byte range being sent.
• fs.createReadStream(path).pipe(res);: Streams the entire video file to the client when no range is requested.

Transcoding with FFmpeg and Node.js

Transcoding involves converting a video from one format to another, such as converting a .mp4 file into an .m3u8 playlist for HTTP Live Streaming (HLS). FFmpeg (a powerful multimedia processing tool) can be used in Node.js for this task.

const { exec } = require('child_process');

const input = 'uploads/input.mp4';

const output = 'outputs/playlist.m3u8';

exec(`ffmpeg -i ${input} -codec: copy -start_number 0 -hls_time 10 -hls_list_size 0 -f hls ${output}`, (err, stdout, stderr) => {

if (err) {

console.error('Transcoding failed:', err);

return;

}

console.log('HLS playlist generated');

});

Explanation:

• const { exec } = require('child_process');: Imports the exec function from the child_process module.
• const input = 'uploads/input.mp4';: Defines the path to the input video file that will be transcoded.
• const output = 'outputs/playlist.m3u8';: Defines the path where the output HLS (HTTP Live Streaming) playlist file will be saved.
• (err, stdout, stderr) => {...}: Defines the callback function that handles the execution result, logging errors or success.

Extracting Video Metadata and Segment Details

Metadata extraction is a common requirement in video processing. Node.js can use ffprobe (part of FFmpeg) or fluent-ffmpeg to extract details such as duration, resolution, codec information, and more.

const ffmpeg = require('fluent-ffmpeg');

ffmpeg.ffprobe('uploads/input.mp4', (err, metadata) => {

if (err) throw err;

console.log(metadata.format.duration);

});

Explanation:

• const ffmpeg = require('fluent-ffmpeg');: Imports the fluent-ffmpeg library to interact with ffmpeg commands in Node.js.
• ffmpeg.ffprobe('uploads/input.mp4', (err, metadata) => {...});: Uses ffmpeg.ffprobe to retrieve metadata about the specified video file.
• (err, metadata) => {...}: Defines the callback function that handles the result, with err for any errors and metadata containing the video file's metadata.
• if (err) throw err;: Throws an error if the ffprobe command fails.
• console.log(metadata.format.duration);: Logs the duration of the video from the metadata object.

Generating Pre-Signed URLs for Secure CDN Access

In cloud-based applications, you might store large video files in services like Amazon S3. Signed URLs are temporary URLs that allow clients to access these files securely. They are used for scenarios where you want to restrict access to certain files or segments.

const AWS = require('aws-sdk');

const s3 = new AWS.S3();

const params = {

Bucket: 'video-bucket',

Key: 'segments/video.ts',

Expires: 300

};

const url = s3.getSignedUrl('getObject', params);

Explanation:

• const AWS = require('aws-sdk');: Imports the AWS SDK, which provides a set of tools to interact with Amazon Web Services, including S3.
• const s3 = new AWS.S3();: Creates a new instance of the S3 service to interact with Amazon S3 storage.
• const params = {...};: Defines the parameters needed for generating a signed URL, including the S3 bucket name, file path (Key), and expiration time.
• Key: 'segments/video.ts': Defines the path to the file within the bucket, in this case, a video segment.
• const url = s3.getSignedUrl('getObject', params);: Generates a signed URL that allows temporary access to the specified S3 object (video file).