OpenMQTT-Gateway/.github/workflows/README.md

GitHub Actions Workflows Documentation

This document provides an overview of all GitHub Actions workflows in the OpenMQTTGateway project.

Architecture Overview

The workflow system is organized in two layers:

Main Workflows (User-facing triggers)

Entry points triggered by user actions, schedules, or events:

• build.yml - CI validation on push/PR
• build_and_docs_to_dev.yml - Daily development builds
• release.yml - Production releases
• manual_docs.yml - Documentation deployment
• security-scan.yml - Security vulnerability scanning
• stale.yml - Issue management

Task Workflows (Reusable components)

Parameterized building blocks called by main workflows:

• task-build.yml - Configurable firmware build
• task-docs.yml - Configurable documentation build
• task-lint.yml - Configurable code formatting check
• task-security-scan.yml - Configurable security scanning

Workflow Overview Table

Workflow	Trigger	Purpose	Artifacts
build.yml	Push, Pull Request	CI Build Validation	Firmware binaries (7 days)
build_and_docs_to_dev.yml	Daily Cron, Manual	Development Builds + Docs	Firmware + Docs deployment
release.yml	Release Published	Production Release	Release assets + Docs
manual_docs.yml	Manual, Workflow Call	Documentation Only	GitHub Pages docs
security-scan.yml	Weekly Cron, Manual	Security Vulnerability Scanning	SARIF, SBOM reports
stale.yml	Daily Cron	Issue Management	None
task-build.yml	Workflow Call	Reusable Build Logic	Configurable
task-docs.yml	Workflow Call	Reusable Docs Logic	GitHub Pages
task-lint.yml	Workflow Call	Reusable Lint Logic	None
task-security-scan.yml	Workflow Call	Reusable Security Scan	SARIF, SBOM, Reports

Workflow Dependencies and Call Chain

flowchart TD
    %% Triggers
    subgraph triggers ["🎯 Triggers"]
        push["Push"]
        pr["Pull Request"]
        release["Release Published"]
         cron1["Cron: Daily 00:00 UTC"]
        manual["Manual Trigger"]
       cron3["Cron: Weekly Monday 02:00 UTC"]
        cron2["Cron: Daily 00:30 UTC"]
        
    end

subgraph github_workflows ["📋 GitHub Workflows"]
    %% Main Workflows
    subgraph main ["📋 Main Workflows"]
        build["build.yml<br/>CI Build"]
        release_wf["release.yml<br/>Production Release"]
        build_dev["build_and_docs_to_dev.yml<br/>Dev Builds"]
        security_scan["security-scan.yml<br/>Security Scan"]
        stale["stale.yml<br/>Issue Management"]
        manual_docs["manual_docs.yml<br/>Docs Only"]
    end

    %% Task Workflows
    subgraph tasks ["⚙️ Task Workflows"]
        task_build["task-build.yml<br/>Build Firmware"]
        task_lint["task-lint.yml<br/>Code Format"]
        task_security["task-security-scan.yml<br/>Security Scan"]
        task_docs["task-docs.yml<br/>Build & Deploy Docs"]

    end
end

subgraph ci_scripts ["🔧 CI Scripts"]
    %% CI Scripts Layer
    subgraph bash ["🔧 Orchestrator"]
        ci_main["ci.sh<br/>(main dispatcher)"]
        ci_build_script["ci_build.sh<br/>(build orchestrator)"]
        ci_site_script["ci_site.sh<br/>(docs orchestrator)"]
        ci_qa_script["ci_qa.sh<br/>(lint orchestrator)"]
        ci_security_script["ci_security.sh<br/>(security orchestrator)"]
    end

    %% Sub-Scripts Layer
    subgraph sub_scripts ["⚙️ Workers"]
        ci_build_fw["ci_build_firmware.sh<br/>(PlatformIO build)"]
        ci_prep_art["ci_prepare_artifacts.sh<br/>(artifact packaging)"]
        gen_board["generate_board_docs<br/>(npm package)"]
        gen_wu["gen_wu<br/>(npm package)"]
        clang_fmt["clang-format<br/>(code formatter)"]
        trivy["Trivy<br/>(vulnerability scanner)"]
    end
end

    %% Trigger connections
    push -->|"all or subset<br/>(depends on branch)"| build
    pr -->|"all<br/>(always full)"| build
    
    release --> release_wf
    manual --> manual_docs
    cron1 --> build_dev
    cron2 --> stale
    cron3 --> security_scan

    %% Main workflow to task workflow connections
    build --> task_build
    build_dev --> task_build
    build_dev --> task_docs
    release_wf --> task_build
    release_wf --> task_docs
    manual_docs --> task_docs
    security_scan --> task_security

    %% Task workflow internal dependencies
    task_build --> task_lint
    task_build --> task_security

    %% Task workflows to CI scripts
    task_build -->|"ci.sh build ..."| ci_main
    task_docs -->|"ci.sh site ..."| ci_main
    task_lint -->|"ci.sh qa ..."| ci_main
    task_security -->|"ci.sh security ..."| ci_main

    %% CI main dispatcher to orchestrators
    ci_main -->|"route: build"| ci_build_script
    ci_main -->|"route: site"| ci_site_script
    ci_main -->|"route: qa"| ci_qa_script
    ci_main -->|"route: security"| ci_security_script

    %% Orchestrators to workers
    ci_build_script --> ci_build_fw
    ci_build_script --> ci_prep_art
    
    ci_site_script --> gen_board
    ci_site_script --> gen_wu
    
    ci_qa_script --> clang_fmt

    ci_security_script --> trivy

    %% Styling
    classDef triggerStyle fill:#e1f5ff,stroke:#0066cc,stroke-width:2px
    classDef mainStyle fill:#fff4e6,stroke:#ff9900,stroke-width:2px
    classDef taskStyle fill:#e6f7e6,stroke:#00aa00,stroke-width:2px
    classDef ciStyle fill:#ffe6f0,stroke:#cc0066,stroke-width:2px
    classDef subStyle fill:#f0e6ff,stroke:#9933ff,stroke-width:2px
    
    class push,pr,release,manual,cron1,cron2,cron3 triggerStyle
    class build,release_wf,manual_docs,build_dev,stale,security_scan mainStyle
    class task_build,task_docs,task_lint,task_security taskStyle
    class ci_main,ci_build_script,ci_site_script,ci_qa_script,ci_security_script ciStyle
    class ci_build_fw,ci_prep_art,gen_board,gen_wu,clang_fmt,trivy subStyle

    
    style github_workflows stroke:#6A7BD8,stroke-dasharray:6 4,stroke-width:1.8px,fill:#fbfbfc
    style main stroke:#6A7BD8,stroke-dasharray:6 4,stroke-width:0.6px,fill:#fcfdff
    style tasks stroke:#6A7BD8,stroke-dasharray:6 4,stroke-width:0.6px,fill:#fcfdff

    style ci_scripts stroke:#FF9A3C,stroke-dasharray:6 4,stroke-width:1.8px,fill:#fffaf5
    style bash stroke:#FF9A3C,stroke-dasharray:6 4,stroke-width:0.6px,fill:#fffaf5
    style sub_scripts stroke:#FF9A3C,stroke-dasharray:6 4,stroke-width:0.6px,fill:#fffaf5

    style triggers fill:none,stroke:none

    linkStyle 0,1,2,3,4,5,6 stroke:#0066cc,stroke-width:3px;
    linkStyle 7,8,9,10,11,12,13,14,15 stroke:orange, stroke-width:2px;
    linkStyle 16,17,18,19,20,21,22,23,24,25,26,27,28,29 stroke:red, stroke-width:2px;
    

Workflow Relationships

Main → Task Mapping:

• build.yml → calls task-build.yml (also contains inline documentation job)
• build_and_docs_to_dev.yml → calls task-build.yml + task-docs.yml
• release.yml → calls task-build.yml + task-docs.yml
• manual_docs.yml → calls task-docs.yml
• security-scan.yml → calls task-security-scan.yml
• stale.yml → standalone (no dependencies)

Task → CI Script Mapping:

• task-build.yml → ci.sh build --version --mode --deploy-ready
  • Routes to: ci_build.sh → ci_build_firmware.sh, ci_prepare_artifacts.sh
  • Output: generated/artifacts/firmware_build/
• task-docs.yml → ci.sh site --mode --version --url-prefix
  • Routes to: ci_site.sh → generate_board_docs (npm), gen_wu (npm), VuePress
  • Output: generated/site/
• task-lint.yml → ci.sh qa --check --source --extensions --clang-format-version
  • Routes to: ci_qa.sh → clang-format
• task-security-scan.yml → ci.sh security --scan-type --severity --generate-sbom
  • Routes to: ci_security.sh → Trivy (vulnerability scanner)
  • Output: generated/reports/ (SARIF, JSON, SBOM)

Job Dependencies:

• build_and_docs_to_dev.yml: prepare → build (task) → deploy & documentation (task)
• release.yml: prepare → build (task) → deploy → documentation (task)

Script Execution Flow:

GitHub Action (task-*.yml)
    ↓
./scripts/ci.sh <command> [OPTIONS]  ← Main dispatcher
    ↓
./scripts/ci_<command>.sh            ← Command orchestrator
    ↓
./scripts/ci_*.sh / *.py             ← Worker scripts

---

Detailed Workflow Documentation

1. build.yml - Continuous Integration Build

Purpose: Validates that code changes compile successfully with intelligent environment selection based on branch importance.

Triggers:

• Push: Every commit pushed to any branch
• Pull Request: Every PR creation or update

What it does:

1. Determine build scope: Selects environment list based on branch name
   • Full build (all environment): All PRs and Push on important branches (development, master, edge, stable, release/, hotfix/)
   • Quick build (ci subset environment): All PRs and Push on non-critical branches
2. Build job: Calls task-build.yml with appropriate environment set
   • Builds firmware in parallel
3. Documentation job: Inline job that validates docs build (doesn't deploy)
   • Downloads common config from theengs.io
   • Runs npm install and npm run docs:build
   • Uses Node.js 14.x

Technical Details:

• Calls: task-build.yml only (documentation is inline)
• Python version: 3.13 (for build job)
• Build strategy: Parallel matrix via task workflow
• Artifact retention: 7 days
• Development OTA: Enabled (enable-dev-ota: true)
• Environment selection logic:
  • Full build (all): All Pull Requests + branches: development, master, edge, stable, release/, hotfix/
  • Quick build (ci subset): All other feature branches

Outputs:

• Firmware binaries for selected environments
• No documentation deployment (validation only)

Use Case: Ensures no breaking changes before merge. Fast feedback for feature branches (~10 min), comprehensive validation for PRs and critical branches (~40 min).

Execution Context: Runs for ALL contributors on ALL branches with smart scaling based on branch importance.

---

2. build_and_docs_to_dev.yml - Development Deployment Pipeline

Purpose: Creates nightly development builds and deploys documentation to the /dev subdirectory for testing.

Triggers:

• Schedule: Daily at midnight UTC (0 0 * * *)
• Manual: Via workflow_dispatch button

What it does:

1. Prepare job: Generates 6-character short SHA
2. Handle-firmwares job: Calls task-build.yml with development parameters
   • Builds firmware for all environments in parallel
   • Enables development OTA updates with SHA commit version
   • Artifact retention: 1 day
3. Handle-documentation job: Calls task-docs.yml with development parameters
   • Deploys to /dev subdirectory on GitHub Pages
   • Uses short SHA as version identifier
   • Runs PageSpeed Insights on dev site

Technical Details:

• Calls: task-build.yml + task-docs.yml
• Repository restriction: Hardcoded to 1technophile owner only
• Version: 6-character short SHA (e.g., abc123)
• Documentation URL prefix: /dev/
• GitHub Pages destination: dev subdirectory
• PageSpeed URL: https://docs.openmqttgateway.com/dev/

Workflow Parameters:

• Build: enable-dev-ota: true, version-tag: <short-sha>, artifact-retention-days: 1
• Docs: mode: "dev", version: <short-sha>, url-prefix: "/dev/", destination-dir: "dev", run-pagespeed: true

Outputs:

• Firmware binaries with -firmware.bin suffix (1 day retention)
• Bootloader and partition binaries
• Documentation deployed to docs.openmqttgateway.com/dev/

Version Labeling:

• Git SHA (6 chars) injected into firmware via version-tag
• Docs display short SHA as version

Use Case: Daily bleeding-edge builds for early adopters and testing. Preview documentation changes before production release.

Execution Context: Only runs on 1technophile repository owner. Forks will skip this workflow automatically.

---

3. release.yml - Production Release Pipeline

Purpose: Creates official release builds when a new version is published.

Triggers:

• Release: When a GitHub release is published (tagged)

What it does:

1. Prepare job: Extracts version tag and release info from GitHub event
2. Build job: Calls task-build.yml with production parameters

• Builds firmware for all environments in parallel
• Injects release tag version into firmware
• Artifact retention: 90 days

3. Deploy job: Downloads and uploads release assets

• Downloads all firmware artifacts from build job
• Uploads binaries to GitHub Release

4. Documentation job: Calls task-docs.yml for production docs

• Deploys to root (/) of GitHub Pages
• Uses release tag as version

Technical Details:

• Calls: task-build.yml + task-docs.yml
• Build flag: Standard (no DEVELOPMENTOTA)
• Artifact retention: 90 days
• Deploy uses bgpat/release-asset-action to attach assets to GitHub Release

Workflow Parameters:

• Build: enable-dev-ota: false, version-tag: <git-tag>, artifact-retention-days: 90
• Docs: mode: "prod", version: <git-tag>, url-prefix: "/", destination-dir: "."

Outputs:

• Production firmware binaries attached to GitHub Release (all prepared by task-build.yml with --deploy-ready)
• Production documentation at docs.openmqttgateway.com/

Version Labeling:

• Git tag (e.g., v1.2.3) injected into firmware

Workflow Chain:

prepare → build (task-build.yml) → deploy → documentation (task-docs.yml)

Use Case: Official releases for end users. Stable, versioned firmware.

Execution Context: Triggered by repository maintainers creating releases.

---

4. manual_docs.yml - Documentation Deployment

Purpose: Entry point for standalone documentation deployment to GitHub Pages.

Triggers:

• Manual: Via workflow_dispatch button
• Workflow Call: Can be called by other workflows (legacy compatibility)

What it does:

1. Calls task-docs.yml with production parameters
2. Deploys to root directory (/) of GitHub Pages

Technical Details:

• Calls: task-docs.yml
• Mode: prod
• Version: Uses latest release tag by default (or provided input)
• URL prefix: /
• Destination: Root of GitHub Pages
• PageSpeed: Optional (disabled by default)

Outputs:

• Production documentation at docs.openmqttgateway.com/
• Custom domain: docs.openmqttgateway.com (via CNAME)

Use Case: Standalone documentation updates without full release process.

Execution Context: Manual trigger or legacy workflow calls.

---

5. security-scan.yml - Security Vulnerability Scanning

Purpose: Scans the project for security vulnerabilities and generates Software Bill of Materials (SBOM) for supply chain security.

Triggers:

• Schedule: Weekly on Monday at 02:00 UTC (0 2 * * 1)
• Manual: Via workflow_dispatch button with input parameters

Manual Trigger Inputs:

• severity: Severity levels to scan (choices: UNKNOWN, LOW, MEDIUM, HIGH, CRITICAL; default: HIGH,CRITICAL)
• Allows filtering results to specific severity levels

What it does:

1. Prepare job: Sets up environment for scanning
2. Security scan job: Calls task-security-scan.yml with parameters
   • Runs Trivy vulnerability scanner on filesystem
   • Filters results by severity level
   • Generates SARIF format for GitHub Security tab integration
   • Creates SBOM in CycloneDX and SPDX formats
   • Uploads findings to GitHub Security tab (code scanning dashboard)
3. Artifact upload job: Stores generated reports and SBOM
   • SARIF results for GitHub integration
   • SBOM files for supply chain tracking
   • Retention: 90 days

Technical Details:

• Calls: task-security-scan.yml
• Vulnerability scanner: Trivy (vulnerability database updated automatically)
• Report formats: SARIF (GitHub), JSON (detailed), Markdown (summary)
• SBOM formats: CycloneDX and SPDX (standard formats)
• Failure behavior: Does NOT fail the workflow on vulnerabilities (exit-code: 0)
• GitHub Security tab: Auto-uploads SARIF for code scanning dashboard visibility

Workflow Parameters:

• scan-type: "fs" (filesystem scan)
• severity: "HIGH,CRITICAL" (default, or manual input)
• generate-sbom: true (always enabled)
• upload-to-security-tab: true (GitHub Security integration)

Outputs:

• SARIF report: generated/reports/trivy-results.sarif (GitHub Security tab)
• JSON report: generated/reports/trivy-results.json (detailed findings)
• Markdown summary: generated/reports/security-summary.md
• SBOM: generated/reports/sbom/sbom.cyclonedx.json + sbom.spdx.json
• Artifacts retained for 90 days

Use Case: Regular security audits, compliance tracking, vulnerability management, supply chain security.

Execution Context: Weekly automated scans + manual on-demand scanning for developers.

---

6. stale.yml - Issue and PR Management

Purpose: Automatically closes inactive issues and pull requests to reduce maintenance burden.

Triggers:

• Schedule: Daily at 00:30 UTC (30 0 * * *)

What it does:

1. Marks issues/PRs as stale after 90 days of inactivity
2. Closes stale issues/PRs after 14 additional days
3. Exempts issues labeled "enhancement"

Configuration:

• Stale after: 90 days
• Close after: 14 days (104 days total)
• Stale label: stale
• Exempt labels: enhancement

Messages:

• Stale: "This issue is stale because it has been open for 90 days with no activity."
• Close: "This issue was closed because it has been inactive for 14 days since being marked as stale."

Use Case: Housekeeping. Reduces backlog of abandoned issues.

Execution Context: Automated maintenance by GitHub bot.

---

Task Workflows (Reusable Components)

7. task-build.yml - Reusable Build Workflow

Purpose: Parameterized firmware build logic used by multiple workflows.

Trigger: workflow_call only (called by other workflows)

Parameters:

• python-version: Python version to use (default: '3.13')
• pio-version: PlatformIO version to use (default: 'v6.1.19')
• environment-set: Which set of environments to build: 'all' or 'ci' (default: 'all')
• enable-dev-ota: Enable development OTA builds (default: false)
• version-tag: Optional version tag to pass to ci.sh build - omitted if empty (default: '')
• artifact-retention-days: Number of days to retain build artifacts (default: 7)

What it does:

1. Lint code: Runs task-lint.yml to check code formatting (main directory, .h and .ino files)
2. Load environments: Reads environment list from environments.json based on environment-set input (all or ci)
3. Install PlatformIO: Uses uv to install the pio-version input (custom pioarduino/platformio-core fork)
4. Matrix build: Builds selected environments in parallel, blocking on lint job completion
5. Build execution: Calls unified ci.sh build <environment> [OPTIONS]:
   • <environment>: Target hardware (e.g., esp32dev-ble)
   • --version <tag>: Version to inject (SHA for dev, tag for prod)
   • --mode <dev|prod>: Build mode (enables/disables OTA)
   • --deploy-ready: Prepare artifacts for deployment
   • --output <dir>: Output directory for artifacts (default: generated/artifacts/)

Command Flow:

./scripts/ci.sh build esp32dev-ble --version v1.8.0 --mode prod --deploy-ready
    ↓
    ├─→ ci_build.sh (orchestrator)
    │   ├─→ ci_build_firmware.sh esp32dev-ble [--dev-ota]
    │   └─→ ci_prepare_artifacts.sh esp32dev-ble [--deploy] → outputs to generated/artifacts/

Technical Details:

• Runs on: Ubuntu latest
• PlatformIO version: Configurable via pio-version input (default: v6.1.19, custom fork: pioarduino/platformio-core)
• Python package manager: uv (astral-sh/setup-uv@v6)
• Environment sets: all (complete set) or ci (subset for quick validation)
• Strategy: Matrix with fail-fast: false (builds complete even if one environment fails)
• Dependencies: Lint job must pass before build matrix starts
• Main orchestrator: ci.sh → ci_build.sh → sub-scripts

Callers:

• build.yml (CI validation)
• build_and_docs_to_dev.yml (development builds)
• release.yml (production releases)

---

8. task-docs.yml - Reusable Documentation Workflow

Purpose: Parameterized documentation build and deployment logic.

Trigger: workflow_call only (called by other workflows)

Parameters:

• mode: Documentation mode (prod or dev, default: prod)
• version: Version string for docs (default: auto → tag or short SHA depending on caller)
• url-prefix: Base URL path (default: / for prod, /dev/ for dev callers)
• destination-dir: Deploy directory on GitHub Pages (default: .)
• run-pagespeed: Run PageSpeed Insights after deploy (default: false)
• pagespeed-url: URL to test with PageSpeed (default: https://docs.openmqttgateway.com/)

What it does:

1. Build documentation: Calls unified ci.sh site --mode <mode> --version <ver> --url-prefix <path>
2. Deploy: Publishes to GitHub Pages using peaceiris/actions-gh-pages@v3
3. PageSpeed test: Optionally runs performance audit on the provided URL

Command Flow:

./scripts/ci.sh site --mode prod --version v1.2.3 --url-prefix /
  ↓
  └─→ ci_site.sh (orchestrator)
    └─→ npm run docs:build (VuePress compilation)

Callers:

• build_and_docs_to_dev.yml (dev docs to /dev)
• release.yml (production docs to /)
• manual_docs.yml (manual production docs)

---

9. task-lint.yml - Reusable Lint Workflow

Purpose: Parameterized code formatting validation for consistent code style.

Trigger: workflow_call only (called by other workflows)

Parameters:

• source: Source directory to lint (default: 'main')
• extensions: File extensions to check, comma-separated (default: 'h,ino')
• clang-format-version: clang-format version to use (default: '9')

What it does:

1. Checks out code
2. Installs clang-format (specified version)
3. Runs unified ci.sh qa [OPTIONS]:
   • --check: Validation mode (exit on violations)
   • --fix: Auto-fix formatting issues
   • --source <dir>: Directory to lint
   • --extensions <list>: File extensions (comma-separated)
   • --clang-format-version <ver>: Formatter version
4. Fails if formatting violations found

Command Flow:

./scripts/ci.sh qa --check --source main --extensions h,ino --clang-format-version 9
    ↓
    └─→ ci_qa.sh (formatter)
        └─→ clang-format (checks/fixes code style)

Technical Details:

• Runs on: Ubuntu latest
• Script: ci_qa.sh (custom formatting check script)
• Formatter: clang-format-$version installed via apt-get
• Default scope: main directory only (not lib)
• Default file types: .h and .ino (not .cpp)
• Strategy: Single sequential job (not parallelized)

Callers:

• build.yml (inline lint check before build)
• Can be called by other workflows as needed

Default Behavior: If called without parameters, lints main directory for .h and .ino files only.

10. task-security-scan.yml - Reusable Security Scan Workflow

Purpose: Parameterized security vulnerability scanning and SBOM generation logic.

Trigger: workflow_call only (called by other workflows)

Parameters:

• scan-type: Type of scan: 'fs' (filesystem), 'config' (configuration), or 'image' (container) (default: 'fs')
• severity: Severity levels to report (comma-separated: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL) (default: 'HIGH,CRITICAL')
• scan-path: Path to scan (default: '.')
• exit-code: Exit code when vulnerabilities found (0=continue, 1=fail) (default: '0')
• upload-to-security-tab: Upload SARIF to GitHub Security tab (default: true)
• generate-sbom: Generate SBOM artifacts (default: true)

What it does:

1. Install Trivy: Retrieves and installs Trivy vulnerability scanner
2. Run security scan: Calls unified ci.sh security with parameters:
   • --scan-type <fs|config|image>: Target type
   • --severity <levels>: Filter by severity
   • --scan-path <path>: Directory to scan
   • --generate-sbom: Generate SBOM in CycloneDX and SPDX formats
   • --exit-code <0|1>: Fail behavior on critical vulnerabilities
   • --upload-to-security-tab: Upload SARIF to GitHub
3. Upload artifacts: Stores reports and SBOM for later download
4. GitHub Security integration: SARIF automatically appears in Security tab

Command Flow:

./scripts/ci.sh security --scan-type fs --severity HIGH,CRITICAL --generate-sbom --upload-to-security-tab
    ↓
    └─→ ci_security.sh (security orchestrator)
        └─→ Trivy (vulnerability scanner)
            ├─→ Generate SARIF, JSON, summary
            ├─→ Generate SBOM (CycloneDX, SPDX)
            └─→ Upload to GitHub Security tab

Technical Details:

• Runs on: Ubuntu latest
• Scanner: Trivy (latest version auto-installed)
• Report formats: SARIF (GitHub integration), JSON (detailed), Markdown (summary)
• SBOM formats: CycloneDX and SPDX (industry standards)
• GitHub Security tab: Auto-uploads SARIF for code scanning dashboard
• Strategy: Single sequential job (not parallelized)
• Artifact retention: As configured by caller

Output Files:

• generated/reports/trivy-results.sarif - SARIF format (GitHub Security tab upload)
• generated/reports/trivy-results.json - JSON format (detailed results)
• generated/reports/security-summary.md - Human-readable summary
• generated/reports/sbom/sbom.cyclonedx.json - CycloneDX SBOM
• generated/reports/sbom/sbom.spdx.json - SPDX SBOM

Callers:

• security-scan.yml (weekly + manual scanning)
• Can be called by other workflows for custom security workflows

---

Environment Configuration

Centralized Environment Management

All build environments are defined in .github/workflows/environments.json:

{
  "environments": {
    "all": [ ...all environments ],
    "ci": [...a subset of environments]
    }
  }
}

Environment Sets:

• all: Complete production set for releases and comprehensive validation
• ci: Representative subset for fast CI feedback

Environment Categories

ESP32 Family (~50 environments):

• Standard: esp32dev-* variants
• ESP32-S3: esp32s3-* variants
• ESP32-C3: esp32c3-* variants
• Specialized boards: M5Stack, Heltec, LilyGO, Theengs

ESP8266 Family (~20 environments):

• NodeMCU: nodemcuv2-* variants
• Sonoff: sonoff-* variants
• Generic: esp8266-* variants

Specialized Boards (~13 environments):

• Theengs Plug
• Theengs Bridge
• RF Bridge variants
• Custom board configurations

---

Configuration Variables

Repository Restrictions

Development Builds (build_and_docs_to_dev.yml):

• Hardcoded restriction: github.repository_owner == '1technophile'
• Only runs for the main repository owner
• Prevents accidental deployments from forks
• No configuration variable needed

Release Builds (release.yml):

• No repository restrictions
• Runs on any fork when a release is published
• Deploy step requires proper GitHub token permissions

Documentation (manual_docs.yml):

• No repository restrictions
• Can be triggered manually from any fork
• Requires GitHub Pages to be configured

---

Glossary

• Environment: A specific hardware board + gateway combination (e.g., esp32dev-ble)
• Matrix Build: Parallel execution of builds across multiple environments
• Artifact: Build output stored temporarily for download (firmware binaries)
• workflow_call: GitHub Actions feature for calling one workflow from another
• workflow_dispatch: Manual trigger button for workflows
• Task Workflow: Reusable workflow component with parameterized inputs
• Main Workflow: Entry point workflow triggered by events or schedules
• CNAME: Custom domain configuration for GitHub Pages
• OTA: Over-The-Air firmware update capability
• SHA: Git commit hash used for version identification in dev builds

---

Maintenance Notes

CI/CD Script Architecture

Main Entry Point: ci.sh (unified interface)

• Commands: build, site, qa, all
• Routes to specialized orchestrators
• Provides consistent CLI across all operations

Build System (ci.sh build):

• PlatformIO 6.1.19 from custom fork: pioarduino/platformio-core
• Python package manager: uv for fast dependency installation
• Orchestrator: ci_build.sh
  • Worker: ci_build_firmware.sh (PlatformIO compilation)
  • Worker: ci_prepare_artifacts.sh (artifact packaging)

Documentation System (ci.sh site):

• Documentation framework: VuePress
• Orchestrator: ci_site.sh
  • Worker: generate_board_docs (npm package - auto-generate board pages)
  • Worker: gen_wu (npm package - WebUpdater manifest)
  • External: Common config from theengs.io

Code Quality (ci.sh qa):

• Orchestrator: ci_qa.sh
  • Worker: clang-format version 9
  • Worker: shellcheck for shell scripts
  • Default scope: main directory, .h and .ino files

Security Scanning System (ci.sh security):

• Vulnerability scanner: Trivy (container vulnerability database)
• Orchestrator: ci_security.sh
  • Worker: Trivy (filesystem, configuration, and container image scanning)
  • Report formats: SARIF (GitHub Security tab), JSON (detailed), Markdown (summary)
  • SBOM generation: CycloneDX and SPDX formats
  • GitHub Security integration: Auto-uploads SARIF for code scanning dashboard
  • Output: generated/reports/ (security findings and artifacts)

Configuration:

• Environment list: .github/workflows/environments.json
• Task workflows: task-*.yml (reusable GitHub Actions components)
• Repository owner restriction: Hardcoded to 1technophile for dev deployments
• All scripts located in: ./scripts/

Local Development:

# Build firmware locally
./scripts/ci.sh build esp32dev-ble --mode dev --version test

# Build documentation locally
./scripts/ci.sh site --mode dev --version test

# Check code format
./scripts/ci.sh qa --check --source main --extensions h,ino --clang-format-version 9

# Security scanning
./scripts/ci.sh security --scan-type fs --severity HIGH,CRITICAL --generate-sbom

# Run complete pipeline
./scripts/ci.sh all --mode dev

---

Document Version: 2.3
Last Updated: 01/14/2026
Maintainer: OpenMQTTGateway Development Team