Category Archives: Linux

Stress Test CPUs on Linux machines

I came across a scenario where I needed to stress test some new servers CPU and didn’t want to install any program to do it so I wrote a script to stress test CPU’s.

Features:

  • Customizable duration: Set any positive integer duration with -t.
  • Customizable core count: Stress a specific number of cores with -c, up to the maximum available.
  • Quiet mode: Use -q for minimal output, ideal for scripting or automated runs.
  • Input validation: Checks for valid duration and core count, with clear error messages.
  • Verbose reporting: Logs start/end times, core count, PIDs, and a countdown timer (unless in quiet mode).
  • Reliable cleanup: Stores PIDs and uses kill -9 to ensure yes processes are terminated.
  • Cleanup verification: Checks for lingering yes processes and warns if any remain.

Options:

  • -t duration: Set duration in seconds (default: 30).
  • -c cores: Specify number of CPU cores to stress (default: all available cores, as detected by nproc).
  • -q: Quiet mode, suppresses detailed output (only shows warnings or errors).

How to use:

  1. Save as max_cpu.sh.
  2. Make executable: chmod +x max_cpu.sh.
  3. Run with options, e.g.:
    • ./max_cpu.sh (default: 30 seconds, all CPU cores, verbose output)
    • ./max_cpu.sh -t 60 (run for 60 seconds)
    • ./max_cpu.sh -t 45 -c 2 (run for 45 seconds, stress 2 cores)
    • ./max_cpu.sh -t 20 -q (run for 20 seconds, minimal output)

Example output (verbose mode, ./max_cpu.sh -t 5 -c 2):

Starting CPU stress test at Fri May 30 22:25:47 NZST 2025
Using 2 CPU core(s) for 5 seconds
Launching 'yes' processes to max out CPU...
Started 'yes' process with PID 12345
Started 'yes' process with PID 12346
Running CPU at 100% for 5 seconds...
Time remaining: 5 seconds
Time remaining: 4 seconds
Time remaining: 3 seconds
Time remaining: 2 seconds
Time remaining: 1 seconds
Stopping 'yes' processes...
Terminated process with PID 12345
Terminated process with PID 12346
All 'yes' processes terminated successfully
CPU stress test completed at Fri May 30 22:25:52 NZST 2025

Example output (quiet mode, ./max_cpu.sh -t 5 -q):

All 'yes' processes terminated successfully

Script:

#!/bin/bash

# Default values
DURATION=30
CORES=$(nproc)
VERBOSE=1  # 1 for verbose output, 0 for minimal

# Function to display usage
usage() {
    echo "Usage: $0 [-t duration] [-c cores] [-q]"
    echo "  -t duration: Duration in seconds (default: 30)"
    echo "  -c cores: Number of CPU cores to stress (default: all, $CORES)"
    echo "  -q: Quiet mode (minimal output)"
    exit 1
}

# Parse command-line options
while getopts "t:c:q" opt; do
    case $opt in
        t) DURATION=$OPTARG ;;
        c) CORES=$OPTARG ;;
        q) VERBOSE=0 ;;
        *) usage ;;
    esac
done

# Validate inputs
if ! [[ "$DURATION" =~ ^[0-9]+$ ]] || [ "$DURATION" -le 0 ]; then
    echo "Error: Duration must be a positive integer"
    usage
fi
if ! [[ "$CORES" =~ ^[0-9]+$ ]] || [ "$CORES" -le 0 ] || [ "$CORES" -gt "$(nproc)" ]; then
    echo "Error: Number of cores must be a positive integer up to $(nproc)"
    usage
fi

# Function to log messages (respects verbose mode)
log() {
    if [ "$VERBOSE" -eq 1 ]; then
        echo "$1"
    fi
}

log "Starting CPU stress test at $(date)"
log "Using $CORES CPU core(s) for $DURATION seconds"

# Array to store process IDs
pids=()

# Start 'yes' processes for specified number of cores
log "Launching 'yes' processes to max out CPU..."
for i in $(seq $CORES); do
    yes > /dev/null &
    pids+=($!)
    log "Started 'yes' process with PID ${pids[$i-1]}"
done

# Wait for specified duration, showing progress if verbose
if [ "$VERBOSE" -eq 1 ]; then
    log "Running CPU at 100% for $DURATION seconds..."
    for ((i=$DURATION; i>0; i--)); do
        echo "Time remaining: $i seconds"
        sleep 1
    done
else
    sleep $DURATION
fi

# Terminate all 'yes' processes
log "Stopping 'yes' processes..."
for pid in "${pids[@]}"; do
    kill -9 "$pid" 2>/dev/null
    log "Terminated process with PID $pid"
done

# Wait briefly to ensure cleanup
wait 2>/dev/null

# Verify no 'yes' processes remain
if ! pgrep -x "yes" > /dev/null; then
    log "All 'yes' processes terminated successfully"
else
    echo "Warning: Some 'yes' processes may still be running. Run 'sudo killall -9 yes' manually."
fi

log "CPU stress test completed at $(date)"

 

Backup Docker Container configuration script

A while ago I looked into a good way to backup Docker configurations for each container.  This is useful if you have many containers and want a backup strategy that includes container settings.

The below script is from https://github.com/007revad/Docker_Autocompose and does rely on the image red5d/docker-autocompose which the script will download if not already present on your Docker server.

Save the following script as docker-autocompose.sh and run with the commands shown at the top.

#!/usr/bin/env sh
#--------------------------------------------------------------------------------------
# A script to create docker-compose.yml files from docker containers.
#
# Script can be run with a container name parameter to only process that container:
# sudo -s docker-autocompose.sh plex
#
# Or with no parameter, or the "all" parameter, to process all containers:
# sudo -s docker-autocompose.sh all
#
# https://github.com/007revad/Docker_Autocompose
# Adapted from: https://www.synoforum.com/threads/docker-autocompose.4644/#post-20341
#--------------------------------------------------------------------------------------
# REQUIRED:
#
# Needs Red5d/docker-autocompose installed in docker.
# Red5d/docker-autocompose should not be started in docker.
#--------------------------------------------------------------------------------------

# Set the path where you want to .yml files saved to. If blank will save in your home.
saveto="/opt/backup-docker"

# Set to yes to include hostname in the folder name.
# Handy if you have multiple devices that backup to the same location.
IncludeHostname=yes

# Set to yes to backup all containers (running or stopped), or no to backup only running containers.
BackupAllContainers=yes

#--------------------------------------------------------------------------------------

autocompose="red5d/docker-autocompose"

# Check script is running as root (or docker.sock won't run)
if [ $( whoami ) != "root" ]; then
    echo "Script needs to run as root. Aborting."
    exit 1
fi

# Check our saveto path exists (if saveto is set)
if [ ! -d "${saveto}" ]; then
    echo "saveto path not found. Files will be saved in your home."
    saveto=
fi 

# Get hostname if IncludeHostname is yes
echo "IncludeHostname is set to: '${IncludeHostname}'"  # Debugging
if [ "${IncludeHostname}" = "yes" ]; then
    host="$(hostname)_"
fi

# Create subfolder with date
if [ -d "${saveto}" ]; then
    Now=$( date '+%Y%m%d')
    if mkdir -p "${saveto}/${Now}_${host}docker-autocompose"; then
        path="${saveto}/${Now}_${host}docker-autocompose/"
    fi
fi

# Function to process a single container
process_container() {
    container_name="${1}"
    container_id=$(docker container ls -a -q --filter name="${container_name}")
    
    # Check if container exists
    if [ -z "${container_id}" ]; then
        echo "Container '${container_name}' not found. Skipping."
        return
    fi

    # Skip non-running containers if BackupAllContainers=no
    if [ "${BackupAllContainers}" != "yes" ]; then
        is_running=$(docker container inspect -f '{{.State.Running}}' "${container_id}")
        if [ "${is_running}" != "true" ]; then
            echo "Container '${container_name}' is not running. Skipping (BackupAllContainers=no)."
            return
        fi
    fi

    # Check if container is running
    is_running=$(docker container inspect -f '{{.State.Running}}' "${container_id}")
    
    if [ "${is_running}" != "true" ] && [ "${BackupAllContainers}" = "yes" ]; then
        echo "Starting container '${container_name}' temporarily..."
        if docker start "${container_id}" > /dev/null 2>&1; then
            # Generate docker-compose.yml
            docker run --rm -v /var/run/docker.sock:/var/run/docker.sock "${autocompose}" "${container_id}" > "${path}${container_name}-compose.yml"
            # Stop the container after generating the file
            docker stop "${container_id}" > /dev/null 2>&1
            echo "Generated docker-compose.yml for '${container_name}' and stopped it."
        else
            echo "Failed to start container '${container_name}'. Skipping."
            return
        fi
    else
        # Container is already running, generate docker-compose.yml directly
        docker run --rm -v /var/run/docker.sock:/var/run/docker.sock "${autocompose}" "${container_id}" > "${path}${container_name}-compose.yml"
        echo "Generated docker-compose.yml for running container '${container_name}'."
    fi
}

# Do the magic
case "${1}" in
    all|"")
        # Create a docker-compose.yml file for each container
        # Clear existing arguments
        while [ "${1}" ]; do
            shift
        done
        # Create array of container names based on BackupAllContainers setting
        if [ "${BackupAllContainers}" = "yes" ]; then
            set -- $(docker container ls -a --format '{{.Names}}')
            echo "Backing up all containers (running and stopped)."
        else
            set -- $(docker ps --format '{{.Names}}')
            echo "Backing up only running containers."
        fi
        while [ "${1}" ]; do
            process_container "${1}"
            shift
        done
        ;;
    *)
        # Only process specified container
        process_container "${1}"
        ;;
esac

echo "All done"
exit 0

 

Python Script for Analyzing Media File Audio Languages

This Python script scans a specified directory for media files, analyzes their audio streams using ffprobe (part of FFmpeg), and generates a report categorizing files based on their audio language. It separates files with non-English audio, other language audio (or no audio), and provides a detailed report of all audio streams. The script is designed to run on Debian 12.
Key Features
  • Comprehensive Language Detection: Identifies English (‘eng’), specific non-English languages, and undefined (‘und’) audio streams.
  • Recursive Scanning: Processes all media files in the specified directory and its subdirectories.
  • Detailed Reporting: Provides both a summary of non-English/undefined files and a detailed breakdown of all audio streams.
  • Robust Error Handling: Skips problematic files and continues processing, with clear error messages.
  • Customizable: Media file extensions can be modified in the media_extensions set.
Script Summary:
  • Uses ffprobe to analyze media file streams
  • Supports common video formats (.mp4, .mkv, .avi, .mov, .wmv, .flv, .m4v)
  • Recursively scans all subdirectories
  • Creates a text file (no_english_audio.txt) with results
  • Handles errors gracefully
  • Prompts for directory path (defaults to current directory if Enter is pressed)
The output file will contain:
  • A list of file paths for media files without English audio streams
  • Or a message indicating all files have English audio if none are found without it
Install Dependencies
First, install the required dependencies:
sudo apt update
sudo apt install ffmpeg python3
Save the script to a file (e.g., check_audio.py)
Script:
#!/usr/bin/env python3

import os
import subprocess
import json
import sys
from pathlib import Path

def get_audio_streams(file_path):
    """
    Get detailed information about audio streams in a media file
    Returns list of dictionaries containing stream info
    """
    try:
        cmd = [
            'ffprobe',
            '-v', 'error',
            '-show_streams',
            '-print_format', 'json',
            str(file_path)
        ]
        result = subprocess.run(cmd, capture_output=True, text=True, check=True)
        
        data = json.loads(result.stdout)
        audio_streams = []
        
        for stream in data.get('streams', []):
            if stream.get('codec_type') == 'audio':
                stream_info = {
                    'index': stream.get('index', 'unknown'),
                    'codec': stream.get('codec_name', 'unknown'),
                    'language': stream.get('tags', {}).get('language', 'und'),
                    'channels': stream.get('channels', 'unknown')
                }
                audio_streams.append(stream_info)
        
        return audio_streams
    
    except (subprocess.CalledProcessError, json.JSONDecodeError) as e:
        print(f"Error processing {file_path}: {e}")
        return []

def scan_directory(directory_path, output_file):
    """
    Scan directory for media files and analyze audio stream languages
    """
    media_extensions = {'.mp4', '.mkv', '.avi', '.mov', '.wmv', '.flv', '.m4v'}
    directory = Path(directory_path)
    
    if not directory.is_dir():
        print(f"Error: {directory_path} is not a valid directory")
        return
    
    # Lists for results
    no_english_files = []
    undefined_lang_files = []
    detailed_report = []
    
    # Scan directory
    for file_path in directory.rglob('*'):
        if file_path.is_file() and file_path.suffix.lower() in media_extensions:
            print(f"Analyzing: {file_path}")
            audio_streams = get_audio_streams(file_path)
            
            # Build detailed report entry
            file_entry = f"File: {file_path}\n"
            if audio_streams:
                file_entry += f"  Found {len(audio_streams)} audio stream(s):\n"
                has_english = False
                all_undefined = True
                
                for stream in audio_streams:
                    lang = stream['language'].lower()
                    file_entry += f"    Stream {stream['index']}: {lang} ({stream['codec']}, {stream['channels']} channels)\n"
                    if lang == 'eng':
                        has_english = True
                    if lang != 'und':
                        all_undefined = False
                
                # Categorize the file
                if not has_english:
                    if all_undefined:
                        undefined_lang_files.append(str(file_path))
                    else:
                        no_english_files.append(str(file_path))
            else:
                file_entry += "  No audio streams found\n"
                undefined_lang_files.append(str(file_path))  # Treat no audio as undefined
            
            detailed_report.append(file_entry)
    
    # Write results
    try:
        with open(output_file, 'w') as f:
            # Summary of files without English (specific non-English languages)
            f.write("=== Files With Non-English Audio (Excluding Undefined) ===\n")
            if no_english_files:
                f.write(f"Found {len(no_english_files)} file(s) with specific non-English audio:\n")
                f.write("\n".join(no_english_files))
                f.write("\n\n")
            else:
                f.write("No files found with specific non-English audio.\n\n")
            
            # Summary of files with undefined language
            f.write("=== Files With Undefined Language Audio (or No Audio) ===\n")
            if undefined_lang_files:
                f.write(f"Found {len(undefined_lang_files)} file(s) with undefined language audio:\n")
                f.write("\n".join(undefined_lang_files))
                f.write("\n\n")
            else:
                f.write("No files found with undefined language audio.\n\n")
            
            # Detailed report
            f.write("=== Detailed Audio Stream Report ===\n")
            f.write("\n".join(detailed_report))
        
        print(f"Results written to {output_file}")
    except IOError as e:
        print(f"Error writing to output file: {e}")

def main():
    if len(sys.argv) != 2:
        print("Usage: ./check_audio.py <directory_path>")
        print("Example: ./check_audio.py /path/to/media")
        sys.exit(1)
    
    directory_path = sys.argv[1]
    output_file = "audio_language_report.txt"
    
    try:
        subprocess.run(['ffprobe', '-version'], capture_output=True, check=True)
    except (subprocess.CalledProcessError, FileNotFoundError):
        print("Error: FFmpeg is not installed. Please install it using 'sudo apt install ffmpeg'")
        sys.exit(1)
    
    scan_directory(directory_path, output_file)

if __name__ == "__main__":
    main()
Make the script executable:
chmod +x check_audio.py
Run the script:
./check_audio.py /path/to/media
Example output in audio_language_report.txt:
=== Files With Non-English Audio (Excluding Undefined) ===
Found 1 file(s) with specific non-English audio:
/path/to/video1.mkv

=== Files With Undefined Language Audio (or No Audio) ===
Found 2 file(s) with undefined language audio:
/path/to/video2.mp4
/path/to/video4.avi

=== Detailed Audio Stream Report ===
File: /path/to/video1.mkv
  Found 1 audio stream(s):
    Stream 1: spa (aac, 2 channels)

File: /path/to/video2.mp4
  Found 1 audio stream(s):
    Stream 1: und (mp3, 2 channels)

File: /path/to/video3.mkv
  Found 2 audio stream(s):
    Stream 1: eng (aac, 6 channels)
    Stream 2: jpn (aac, 2 channels)

File: /path/to/video4.avi
  Found 0 audio stream(s):
    No audio streams found
Step-by-Step Breakdown
Script Initialization and Dependencies
  • The script uses Python 3, which is included with Debian 12.
  • Requires FFmpeg (ffprobe) to analyze media files. Install it with:
sudo apt update
sudo apt install ffmpeg
  • Imports necessary Python modules: os, subprocess, json, sys, and pathlib.Path.
  • The script is executed with a command-line argument specifying the directory to scan.
Command-Line Argument Handling
  • The script expects a single command-line argument: the path to the directory to scan.
  • Usage example:
./check_audio.py /path/to/media
  • If no or incorrect arguments are provided, it displays usage instructions and exits:
Usage: ./check_audio.py <directory_path>
Example: ./check_audio.py /path/to/media
  • The output report is saved to a file named audio_language_report.txt.
FFmpeg Availability Check
  • Verifies that ffprobe is installed by running:
ffprobe -version
  • If FFmpeg is not installed, the script exits with an error message instructing the user to install it.
Audio Stream Analysis (get_audio_streams Function)
  • Uses ffprobe to extract stream information from a media file in JSON format.
  • Command executed:
ffprobe -v error -show_streams -print_format json <file_path>
  • Parses the JSON output to identify audio streams.
  • For each audio stream, collects:

    • Stream index
    • Codec name (e.g., aac, mp3)
    • Language tag (defaults to ‘und’ if undefined)
    • Number of channels
  • Returns a list of dictionaries containing stream details or an empty list if an error occurs (e.g., file corruption or invalid format).
Directory Scanning (scan_directory Function)
  • Accepts the directory path and output file name as parameters.
  • Supports common media file extensions: .mp4, .mkv, .avi, .mov, .wmv, .flv, .m4v.
  • Recursively scans the directory using Path.rglob to find all media files.
  • For each file:
    • Calls get_audio_streams to retrieve audio stream details.
    • Builds a detailed report entry listing all audio streams, including their language, codec, and channels.
    • Categorizes the file based on its audio streams:
      • Files with English audio: If any stream has language ‘eng’, the file is excluded from summary lists.
      • Files with non-English audio: If no ‘eng’ stream exists and at least one stream has a specific language (e.g., ‘spa’, ‘fre’), the file is added to no_english_files.
      • Files with undefined language or no audio: If all streams are ‘und’ (undefined) or no audio streams exist, the file is added to undefined_lang_files.
Output Generation
Writes results to audio_language_report.txt in three sections:
  • Files With Non-English Audio (Excluding Undefined):
    • Lists files with specific non-English languages (e.g., Spanish, French).
    • Example:
Found 1 file(s) with specific non-English audio:
/path/to/video1.mkv
  • Files With Undefined Language Audio (or No Audio)
    • Lists files with only ‘und’ language tags or no audio streams.
    • Example:
Found 2 file(s) with undefined language audio:
/path/to/video2.mp4
/path/to/video4.avi
  • Detailed Audio Stream Report:
    • Lists all files with their audio stream details.
    • Example:
File: /path/to/video1.mkv
  Found 1 audio stream(s):
    Stream 1: spa (aac, 2 channels)
File: /path/to/video2.mp4
  Found 1 audio stream(s):
    Stream 1: und (mp3, 2 channels)
File: /path/to/video3.mkv
  Found 2 audio stream(s):
    Stream 1: eng (aac, 6 channels)
    Stream 2: jpn (aac, 2 channels)
File: /path/to/video4.avi
  Found 0 audio stream(s):
    No audio streams found
  • Handles IO errors gracefully, printing an error message if the output file cannot be written.
Error Handling
  • Checks for valid directory input; exits if the directory is invalid.
  • Handles ffprobe errors (e.g., corrupted files) by skipping problematic files and logging errors.
  • Manages JSON parsing errors, ensuring the script continues processing other files.