[CI] Refactor GitHub Actions workflows for build, documentation, and linting (#2260)
* Refactor GitHub Actions workflows for build, documentation, and linting - Consolidated build logic into reusable workflows (`task-build.yml` and `task-docs.yml`) to reduce duplication across multiple workflows. - Introduced `environments.json` to centralize the list of PlatformIO build environments, improving maintainability and clarity. - Updated `build.yml` and `build_and_docs_to_dev.yml` to utilize the new reusable workflows and environment definitions. - Enhanced `release.yml` to streamline the release process and integrate documentation generation. - Created reusable linting workflow (`task-lint.yml`) to standardize code formatting checks across the repository. - Simplified manual documentation workflow by leveraging the new reusable documentation workflow. - Improved artifact management and retention policies across workflows. - Updated dependencies and versions in workflows to ensure compatibility and performance. * CI/CD pipeline agnostic of Workflow Engine and integrated on github actions - Implemented ci.sh for orchestrating the complete build pipeline. - Created ci_00_config.sh for centralized configuration of build scripts. - Created ci_build_firmware.sh for building firmware for specified PlatformIO environments. - Created ci_prepare_artifacts.sh for preparing firmware artifacts for upload or deployment. - Created ci_set_version.sh for updating version tags in firmware configuration files. - Created ci_build.sh to orchestrate the complete build pipeline. - Created ci_qa.sh for code linting and formatting checks using clang-format. - Created ci_site.sh for building and deploying VuePress documentation with version management. - Implemented checks for required tools and dependencies in the new scripts. - Updated common_wu.py, compressFirmware.py, gen_wu.py, generate_board_docs.py, and prepare_deploy.sh with descriptive comments. Refactor CI/CD scripts for improved modularity and clarity - Consolidated build steps in task-build.yml to utilize ci.sh for version tagging, building, and artifact preparation. - Updated task-lint.yml to use ci.sh for code formatting checks instead of ci_qa.sh. - Enhanced CI_SCRIPTS.md documentation to reflect changes in script usage, command structure, and output organization. - Improved internal scripts for better error handling and logging. - Streamlined the output structure for build artifacts and documentation.
This commit is contained in:
Alessandro Staniscia authored and GitHub committed
1 parent
fd433c220e
commit
134c03362c
27 files changed
+4151
-346
No files matched your search
@@ -0,0 +1,575 @@
|
||||
# CI/CD Scripts Documentation
|
||||
|
||||
This documentation describes the CI/CD scripts used to build OpenMQTTGateway firmware and documentation. These scripts work in GitHub Actions, locally, and in any CI/CD environment.
|
||||
|
||||
- [Overview](#cicd-scripts-documentation)
|
||||
- [Quick Reference](#quick-reference)
|
||||
- [Script Hierarchy](#script-hierarchy)
|
||||
- [Script Description](#script-description)
|
||||
- [Output Structure](#output-structure)
|
||||
- [Commands](#commands)
|
||||
- [ci.sh - Main Entry Point](#commands)
|
||||
- [ci.sh build - Build Firmware](#cish-build---build-firmware)
|
||||
- [ci.sh site - Build Documentation](#cish-site---build-documentation)
|
||||
- [ci.sh qa - Code Formatting Check](#cish-qa---code-formatting-check)
|
||||
- [Internal Scripts](#internal-scripts)
|
||||
- [ci_set_version.sh](#ci_set_versionsh)
|
||||
- [ci_build_firmware.sh](#ci_build_firmwaresh)
|
||||
- [ci_prepare_artifacts.sh](#ci_prepare_artifactssh)
|
||||
- [ci_00_config.sh](#ci_00_configsh)
|
||||
- [Python Helper Scripts](#python-helper-scripts)
|
||||
- [generate_board_docs.py](#generate_board_docspy)
|
||||
- [gen_wu.py](#gen_wupy)
|
||||
- [Environment Variables](#environment-variables)
|
||||
- [Exit Codes](#exit-codes)
|
||||
- [Environment Detection](#environment-detection)
|
||||
|
||||
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Script Hierarchy
|
||||
|
||||
```
|
||||
ci.sh (dispatcher)
|
||||
├── build → ci_build.sh → ci_set_version.sh
|
||||
│ → ci_build_firmware.sh
|
||||
│ → ci_prepare_artifacts.sh
|
||||
├── site → ci_site.sh → generate_board_docs.py
|
||||
│ → gen_wu.py
|
||||
├── qa → ci_qa.sh → clang-format
|
||||
└── all → qa + build + site (sequential)
|
||||
```
|
||||
|
||||
### Script Description
|
||||
|
||||
| Script | Purpose | Called By |
|
||||
|--------|---------|-----------|
|
||||
| `ci.sh` | Main command dispatcher | User/GitHub Actions |
|
||||
| `ci_build.sh` | Build firmware orchestrator | ci.sh build |
|
||||
| `ci_site.sh` | Documentation build orchestrator | ci.sh site |
|
||||
| `ci_qa.sh` | Code formatting checker | ci.sh qa |
|
||||
| `ci_set_version.sh` | Version injection in firmware | ci_build.sh |
|
||||
| `ci_build_firmware.sh` | PlatformIO build execution | ci_build.sh |
|
||||
| `ci_prepare_artifacts.sh` | Artifact packaging | ci_build.sh |
|
||||
| `ci_00_config.sh` | Shared configuration and functions | All scripts |
|
||||
|
||||
|
||||
### Output Structure
|
||||
|
||||
Build outputs are organized in the project root:
|
||||
|
||||
```
|
||||
.pio/build/<environment>/ # PlatformIO build outputs
|
||||
├── firmware.bin # Main firmware binary
|
||||
├── bootloader.bin # ESP32 bootloader
|
||||
└── partitions.bin # ESP32 partition table
|
||||
|
||||
generated/
|
||||
├── artifacts/ # Packaged firmware artifacts
|
||||
└── site/ # Built documentation (VuePress output)
|
||||
|
||||
scripts/
|
||||
├── latest_version.json # Production version metadata
|
||||
└── latest_version_dev.json # Development version metadata
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
`ci.sh` is the main Entry Point. Command dispatcher that routes to specialized scripts.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
./scripts/ci.sh <command> [OPTIONS]
|
||||
```
|
||||
|
||||
**Commands:**
|
||||
- `build` - Build firmware for specified environment
|
||||
- `site` or `docs` - Build documentation website
|
||||
- `qa` or `lint` - Run code formatting checks
|
||||
- `all` or `pipeline` - Run complete pipeline (qa + build + site)
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Get Help
|
||||
./scripts/ci.sh build --help
|
||||
./scripts/ci.sh qa --help
|
||||
./scripts/ci.sh site --help
|
||||
|
||||
# Build firmware
|
||||
./scripts/ci.sh build esp32dev-ble --mode dev
|
||||
./scripts/ci.sh build esp32dev-all-test --version v1.8.0 --deploy-ready
|
||||
|
||||
# Build documentation
|
||||
./scripts/ci.sh site --mode prod
|
||||
./scripts/ci.sh site --mode dev --url-prefix /dev/
|
||||
|
||||
# Check code formatting
|
||||
./scripts/ci.sh qa --check
|
||||
./scripts/ci.sh qa --fix
|
||||
|
||||
# Run complete pipeline
|
||||
./scripts/ci.sh all esp32dev-ble --version v1.8.0
|
||||
./scripts/ci.sh all esp32dev-ble --no-site
|
||||
```
|
||||
|
||||
**Options for `all` command:**
|
||||
- `--no-site` - Skip documentation build step
|
||||
- All options from `build` command are passed through
|
||||
|
||||
---
|
||||
|
||||
### ci.sh build - Build Firmware
|
||||
|
||||
Orchestrates complete firmware build: version injection, compilation, artifact packaging.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
./scripts/ci.sh build <environment> [OPTIONS]
|
||||
```
|
||||
|
||||
**Required Arguments:**
|
||||
- `<environment>` - PlatformIO environment name (e.g., esp32dev-ble, nodemcuv2-rf)
|
||||
|
||||
**Options:**
|
||||
- `--version <tag>` - Version string to inject (default: auto-generated from git)
|
||||
- `--mode <dev|prod>` - Build mode (default: prod)
|
||||
- `dev` - Enables development OTA, sets DEVELOPMENTOTA=true
|
||||
- `prod` - Standard production build
|
||||
- `--deploy-ready` - Package artifacts with deployment naming (env-firmware.bin)
|
||||
- `--output <dir>` - Output directory for artifacts (default: generated/artifacts)
|
||||
- `--skip-verification` - Skip build tools verification
|
||||
- `--clean` - Clean previous build before starting
|
||||
- `--verbose` - Enable verbose PlatformIO output
|
||||
- `--list-envs` - List all available PlatformIO environments
|
||||
- `--help` - Show help message
|
||||
|
||||
**Execution Flow:**
|
||||
```
|
||||
ci.sh build esp32dev-ble --version v1.8.0 --mode prod --deploy-ready
|
||||
│
|
||||
├─> ci_build.sh (orchestrator)
|
||||
│ ├─> verify_build_tools() - Check python3, platformio, git
|
||||
│ ├─> ci_set_version.sh v1.8.0 - Inject version in User_config.h
|
||||
│ ├─> ci_build_firmware.sh esp32dev-ble - Execute PlatformIO build
|
||||
│ └─> ci_prepare_artifacts.sh esp32dev-ble --deploy - Package binaries
|
||||
│
|
||||
└─> Outputs in generated/artifacts/
|
||||
├─ esp32dev-ble-firmware.bin
|
||||
├─ esp32dev-ble-bootloader.bin
|
||||
└─ esp32dev-ble-partitions.bin
|
||||
```
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Development build
|
||||
./scripts/ci.sh build esp32dev-ble --mode dev
|
||||
|
||||
# Production build with version
|
||||
./scripts/ci.sh build esp32dev-ble --version v1.8.0 --mode prod
|
||||
|
||||
# Deploy-ready build
|
||||
./scripts/ci.sh build esp32dev-all-test --version v1.8.0 --deploy-ready
|
||||
|
||||
# Clean build with verbose output
|
||||
./scripts/ci.sh build nodemcuv2-rf --clean --verbose
|
||||
|
||||
# List available environments
|
||||
./scripts/ci.sh build --list-envs
|
||||
```
|
||||
|
||||
**Environment Variables:**
|
||||
- `CI` - Set to 'true' in CI/CD environments
|
||||
- `BUILD_NUMBER` - Build number from CI/CD system
|
||||
- `GIT_COMMIT` - Git commit hash for auto-versioning
|
||||
- `PLATFORMIO_BUILD_FLAGS` - Additional PlatformIO flags (set by script when --mode dev)
|
||||
|
||||
**Output Files:**
|
||||
- Standard mode: `firmware.bin`, `partitions.bin` in generated/artifacts/
|
||||
- Deploy mode: `<env>-firmware.bin`, `<env>-bootloader.bin`, `<env>-partitions.bin`
|
||||
|
||||
---
|
||||
|
||||
### ci.sh site - Build Documentation
|
||||
|
||||
Builds VuePress documentation website with version management and WebUploader manifest generation.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
./scripts/ci.sh site [OPTIONS]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--mode <dev|prod>` - Documentation mode (default: prod)
|
||||
- `dev` - Development documentation with watermark
|
||||
- `prod` - Production documentation
|
||||
- `--version-source <release|custom>` - Version source (default: release)
|
||||
- `release` - Use git tag as version
|
||||
- `custom` - Use custom version string
|
||||
- `--custom-version <version>` - Custom version string (requires --version-source custom)
|
||||
- `--url-prefix <path>` - Base URL path (default: /)
|
||||
- Example: `/dev/` for development subdirectory
|
||||
- `--no-webuploader` - Skip WebUploader manifest generation
|
||||
- `--webuploader-args <args>` - Additional arguments for gen_wu.py
|
||||
- `--preview` - Open browser after build (local development)
|
||||
- `--help` - Show help message
|
||||
|
||||
**Execution Flow:**
|
||||
```
|
||||
ci.sh site --mode prod --version-source release
|
||||
│
|
||||
├─> ci_site.sh (orchestrator)
|
||||
│ ├─> check_requirements() - Verify node, npm, python3, pip3
|
||||
│ ├─> install_dependencies() - npm install, pip3 install packages
|
||||
│ ├─> download_common_config() - Fetch from theengs.io
|
||||
│ ├─> get_version() - Extract from git tag or use custom
|
||||
│ ├─> set_version() - Update VuePress config and JSON files
|
||||
│ ├─> set_url_prefix() - Set base path in config
|
||||
│ ├─> generate_board_docs.py - Auto-generate board documentation
|
||||
│ ├─> npm run docs:build - Build VuePress site
|
||||
│ └─> gen_wu.py - Generate WebUploader manifest
|
||||
│
|
||||
└─> Outputs in generated/site/
|
||||
├─ index.html
|
||||
├─ assets/
|
||||
└─ [board documentation pages]
|
||||
```
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Production documentation
|
||||
./scripts/ci.sh site --mode prod
|
||||
|
||||
# Development documentation with custom version
|
||||
./scripts/ci.sh site --mode dev --version-source custom --custom-version "DEVELOPMENT SHA:abc123"
|
||||
|
||||
# Documentation for /dev/ subdirectory
|
||||
./scripts/ci.sh site --mode dev --url-prefix /dev/
|
||||
|
||||
# Skip WebUploader manifest
|
||||
./scripts/ci.sh site --no-webuploader
|
||||
|
||||
# Local preview
|
||||
./scripts/ci.sh site --preview
|
||||
```
|
||||
|
||||
**Required Tools:**
|
||||
- Node.js (for VuePress)
|
||||
- npm (for package management)
|
||||
- Python 3 (for board docs generator)
|
||||
- pip3 (for Python dependencies: requests, pandas, markdown, pytablereader, tabulate)
|
||||
|
||||
**Output Files:**
|
||||
- `generated/site/` - Complete static website
|
||||
- `scripts/latest_version.json` - Production version metadata (updated)
|
||||
- `scripts/latest_version_dev.json` - Development version metadata (updated)
|
||||
|
||||
---
|
||||
|
||||
### ci.sh qa - Code Formatting Check
|
||||
|
||||
Checks and fixes code formatting using clang-format.
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
./scripts/ci.sh qa [OPTIONS]
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--check` - Check formatting without modifying files (default)
|
||||
- `--fix` - Automatically fix formatting issues
|
||||
- `--source <dir>` - Source directory to check (default: main)
|
||||
- `--extensions <list>` - File extensions to check, comma-separated (default: h,ino,cpp)
|
||||
- `--clang-format-version <ver>` - clang-format version to use (default: 9)
|
||||
- `--verbose` - Show detailed output for each file
|
||||
- `--help` - Show help message
|
||||
|
||||
**Execution Flow:**
|
||||
```
|
||||
ci.sh qa --check --source main --extensions h,ino
|
||||
│
|
||||
├─> ci_qa.sh (orchestrator)
|
||||
│ ├─> check_clang_format() - Find clang-format-9 or clang-format
|
||||
│ ├─> find_files() - Locate files matching extensions in source dir
|
||||
│ └─> check_formatting() - Run clang-format --dry-run --Werror
|
||||
│ └─> Report files with formatting issues
|
||||
│
|
||||
└─> Exit code: 0 (pass) or 1 (formatting issues found)
|
||||
```
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Check formatting (CI mode)
|
||||
./scripts/ci.sh qa --check
|
||||
|
||||
# Fix formatting automatically
|
||||
./scripts/ci.sh qa --fix
|
||||
|
||||
# Check specific directory
|
||||
./scripts/ci.sh qa --check --source lib
|
||||
|
||||
# Check only .h and .ino files
|
||||
./scripts/ci.sh qa --check --extensions h,ino
|
||||
|
||||
# Check with verbose output
|
||||
./scripts/ci.sh qa --check --verbose
|
||||
|
||||
# Use different clang-format version
|
||||
./scripts/ci.sh qa --check --clang-format-version 11
|
||||
```
|
||||
|
||||
**Required Tools:**
|
||||
- clang-format (version specified, default: 9)
|
||||
- Install: `sudo apt-get install clang-format-9`
|
||||
|
||||
**Output:**
|
||||
- Check mode: Lists files with formatting issues and shows diffs
|
||||
- Fix mode: Modifies files in-place and reports changes
|
||||
- Exit code 0: All files properly formatted
|
||||
- Exit code 1: Formatting issues found (in check mode)
|
||||
|
||||
---
|
||||
|
||||
## Internal Scripts
|
||||
|
||||
These scripts are called by the main orchestrators. It can be invoked directly but is not raccomanded.
|
||||
|
||||
### ci_set_version.sh
|
||||
|
||||
Injects version string into firmware configuration files.
|
||||
|
||||
**Called By:** `ci_build.sh`
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
./scripts/ci_set_version.sh <version> [--dev]
|
||||
```
|
||||
|
||||
**Arguments:**
|
||||
- `<version>` - Version string to inject (e.g., v1.8.0 or abc123)
|
||||
- `--dev` - Development mode (updates latest_version_dev.json)
|
||||
|
||||
**Files Modified:**
|
||||
- `main/User_config.h` - Replaces "version_tag" with actual version
|
||||
- `scripts/latest_version.json` - Production version metadata
|
||||
- `scripts/latest_version_dev.json` - Development version metadata (--dev mode)
|
||||
|
||||
**Behavior:**
|
||||
- Creates .bak backup files before modification
|
||||
- Replaces all occurrences of "version_tag" string
|
||||
- Validates version string (must not be empty or "version_tag")
|
||||
- Cleans up backup files on success
|
||||
|
||||
---
|
||||
|
||||
### ci_build_firmware.sh
|
||||
|
||||
Executes PlatformIO build for specified environment.
|
||||
|
||||
**Called By:** `ci_build.sh`
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
./scripts/ci_build_firmware.sh <environment> [OPTIONS]
|
||||
```
|
||||
|
||||
**Arguments:**
|
||||
- `<environment>` - PlatformIO environment name
|
||||
|
||||
**Options:**
|
||||
- `--dev-ota` - Enable development OTA (sets PLATFORMIO_BUILD_FLAGS)
|
||||
- `--clean` - Clean before build
|
||||
- `--verbose` - Verbose PlatformIO output
|
||||
|
||||
**Environment Variables Set:**
|
||||
- `PYTHONIOENCODING=utf-8`
|
||||
- `PYTHONUTF8=1`
|
||||
- `PLATFORMIO_BUILD_FLAGS="-DDEVELOPMENTOTA=true"` (when --dev-ota)
|
||||
|
||||
**PlatformIO Command:**
|
||||
```bash
|
||||
platformio run -e <environment> [--verbose]
|
||||
```
|
||||
|
||||
**Output Location:**
|
||||
- `.pio/build/<environment>/firmware.bin`
|
||||
- `.pio/build/<environment>/bootloader.bin` (ESP32 only)
|
||||
- `.pio/build/<environment>/partitions.bin` (ESP32 only)
|
||||
|
||||
---
|
||||
|
||||
### ci_prepare_artifacts.sh
|
||||
|
||||
Packages firmware binaries from PlatformIO build directory.
|
||||
|
||||
**Called By:** `ci_build.sh`
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
./scripts/ci_prepare_artifacts.sh <environment> [OPTIONS]
|
||||
```
|
||||
|
||||
**Arguments:**
|
||||
- `<environment>` - PlatformIO environment name
|
||||
|
||||
**Options:**
|
||||
- `--deploy` - Use deployment naming (prefix with environment name)
|
||||
- `--output <dir>` - Output directory (default: generated/artifacts)
|
||||
|
||||
**Behavior:**
|
||||
|
||||
Standard mode (no --deploy):
|
||||
- Copies: `firmware.bin`, `partitions.bin`
|
||||
- Does NOT copy: `bootloader.bin`
|
||||
|
||||
Deploy mode (with --deploy):
|
||||
- Copies and renames:
|
||||
- `firmware.bin` → `<env>-firmware.bin`
|
||||
- `bootloader.bin` → `<env>-bootloader.bin`
|
||||
- `partitions.bin` → `<env>-partitions.bin`
|
||||
|
||||
**Source Location:**
|
||||
- `.pio/build/<environment>/`
|
||||
|
||||
**Output Location:**
|
||||
- Specified by `--output` or default `generated/artifacts/`
|
||||
|
||||
---
|
||||
|
||||
### ci_00_config.sh
|
||||
|
||||
Shared configuration and helper functions for all CI scripts.
|
||||
|
||||
**Sourced By:** All ci_*.sh scripts
|
||||
|
||||
**Provides:**
|
||||
- Color codes for terminal output (BLUE, GREEN, RED, YELLOW, NC)
|
||||
- Logging functions: `log_info()`, `log_warn()`, `log_error()`, `log_success()`
|
||||
- Path constants: `BUILD_DIR`, `ARTIFACTS_DIR`, `SITE_DIR`
|
||||
- Common utility functions
|
||||
|
||||
**Constants Defined:**
|
||||
- `BUILD_DIR=".pio/build"` - PlatformIO build directory
|
||||
- `ARTIFACTS_DIR="generated/artifacts"` - Artifact output directory
|
||||
- `SITE_DIR="generated/site"` - Documentation output directory
|
||||
|
||||
**Logging Functions:**
|
||||
```bash
|
||||
log_info "message" # Blue [INFO] prefix
|
||||
log_warn "message" # Yellow [WARN] prefix
|
||||
log_error "message" # Red [ERROR] prefix
|
||||
log_success "message" # Green [SUCCESS] prefix
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Python Helper Scripts
|
||||
|
||||
Other scripts are present and used as internal scripts and it's used as retrocompatibility. Below the lists:
|
||||
- `generate_board_docs.py`
|
||||
- `gen_wu.py`
|
||||
|
||||
### generate_board_docs.py
|
||||
|
||||
Auto-generates board-specific documentation pages from platformio.ini.
|
||||
|
||||
**Called By:** `ci_site.sh`
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
python3 ./scripts/generate_board_docs.py
|
||||
```
|
||||
|
||||
**Input:**
|
||||
- `platformio.ini` - Board configurations
|
||||
- `environments.ini` - Additional environments
|
||||
|
||||
**Output:**
|
||||
- Markdown files in `docs/` directory for each board configuration
|
||||
|
||||
**Purpose:**
|
||||
- Creates documentation pages for each hardware board
|
||||
- Extracts configuration details from PlatformIO environment definitions
|
||||
- Formats technical specifications and pin mappings
|
||||
|
||||
---
|
||||
|
||||
### gen_wu.py
|
||||
|
||||
Generates WebUploader manifest for OTA firmware updates.
|
||||
|
||||
**Called By:** `ci_site.sh`
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
python3 ./scripts/gen_wu.py [--dev] [repository]
|
||||
```
|
||||
|
||||
**Arguments:**
|
||||
- `--dev` - Generate development manifest
|
||||
- `repository` - GitHub repository name (e.g., 1technophile/OpenMQTTGateway)
|
||||
|
||||
**Input:**
|
||||
- `.pio/build/<env>/firmware.bin` - Compiled firmware files
|
||||
- `scripts/latest_version.json` or `scripts/latest_version_dev.json`
|
||||
|
||||
**Output:**
|
||||
- WebUploader manifest JSON file in `docs/.vuepress/public/`
|
||||
|
||||
**Purpose:**
|
||||
- Creates manifest for web-based firmware updater
|
||||
- Lists available firmware files with metadata
|
||||
- Used by documentation site for OTA updates
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Scripts respect these environment variables:
|
||||
|
||||
- `PYTHONIOENCODING=utf-8`: Python encoding
|
||||
- `PYTHONUTF8=1`: UTF-8 mode
|
||||
- `PLATFORMIO_BUILD_FLAGS`: Custom build flags
|
||||
- `ESP32_PLATFORM_VERSION`: Extracted automatically
|
||||
|
||||
---
|
||||
|
||||
## Exit Codes
|
||||
|
||||
All scripts use standard exit codes:
|
||||
|
||||
- `0` - Success
|
||||
- `1` - General error or failure
|
||||
- `2` - Missing required tools or dependencies
|
||||
|
||||
Scripts use `set -euo pipefail` for strict error handling:
|
||||
- `-e` - Exit on error
|
||||
- `-u` - Exit on undefined variable
|
||||
- `-o pipefail` - Exit on pipe failure
|
||||
|
||||
---
|
||||
|
||||
## Environment Detection
|
||||
|
||||
Scripts automatically detect if running in CI/CD:
|
||||
|
||||
```bash
|
||||
if [[ "${CI:-false}" == "true" ]]; then
|
||||
# Running in CI/CD
|
||||
# Disable interactive prompts
|
||||
# Use different output formatting
|
||||
fi
|
||||
```
|
||||
|
||||
CI/CD environments typically set:
|
||||
- `CI=true`
|
||||
- `GITHUB_ACTIONS=true` (GitHub Actions)
|
||||
- `BUILD_NUMBER` (build number)
|
||||
- `GIT_COMMIT` (commit hash)
|
||||
|
||||
|
||||
---
|
||||
|
||||
This documentation reflects the current implementation of CI/CD scripts. All scripts are located in `./scripts/` directory.
|
||||
|
||||
For GitHub Actions workflow documentation, see `.github/workflows/README.md`.
|
||||
@@ -1,3 +1,5 @@
|
||||
# Adds compiler flags to suppress warnings during PlatformIO build
|
||||
# Used by: PlatformIO environments (esp32dev-pilight*, esp32-m5stick-c*)
|
||||
Import("env")
|
||||
|
||||
|
||||
|
||||
Executable
+174
@@ -0,0 +1,174 @@
|
||||
#!/bin/bash
|
||||
# CI/CD Main Entry Point - Command Dispatcher
|
||||
# Routes commands to specialized scripts for build, site, qa, and all
|
||||
# Usage: ./scripts/ci.sh <command> [OPTIONS]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Constants
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
# Load shared configuration (colors, logging functions, paths)
|
||||
if [[ -f "${SCRIPT_DIR}/ci_00_config.sh" ]]; then
|
||||
source "${SCRIPT_DIR}/ci_00_config.sh"
|
||||
else
|
||||
echo "ERROR: ci_00_config.sh not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Function to print banner
|
||||
print_banner() {
|
||||
echo "╔════════════════════════════════════════╗"
|
||||
echo "║ OpenMQTTGateway CI/CD Pipeline ║"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# Show usage
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 <command> [OPTIONS]
|
||||
|
||||
OpenMQTTGateway CI/CD Pipeline - Main Entry Point
|
||||
|
||||
Commands:
|
||||
build Build firmware for specified environment
|
||||
site Build and deploy documentation/website
|
||||
qa Run quality assurance checks (linting, formatting)
|
||||
all Run complete pipeline (qa + build + site)
|
||||
|
||||
Examples:
|
||||
# Build firmware
|
||||
$0 build esp32dev-all-test --mode dev
|
||||
$0 build esp32dev-bt --version v1.8.0 --deploy-ready
|
||||
|
||||
# Build and deploy documentation
|
||||
$0 site --mode prod --deploy
|
||||
$0 site --mode dev --preview
|
||||
|
||||
# Run quality checks
|
||||
$0 qa --check
|
||||
$0 qa --fix
|
||||
|
||||
# Run complete pipeline
|
||||
$0 all esp32dev-bt --version v1.8.0
|
||||
|
||||
Get help for specific commands:
|
||||
$0 build --help
|
||||
$0 site --help
|
||||
$0 qa --help
|
||||
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
# Function to run build pipeline
|
||||
run_build_pipeline() {
|
||||
log_info "Executing build pipeline..."
|
||||
"${SCRIPT_DIR}/ci_build.sh" "$@"
|
||||
}
|
||||
|
||||
# Function to run site pipeline
|
||||
run_site_pipeline() {
|
||||
log_info "Executing site pipeline..."
|
||||
"${SCRIPT_DIR}/ci_site.sh" "$@"
|
||||
}
|
||||
|
||||
# Function to run QA pipeline
|
||||
run_qa_pipeline() {
|
||||
log_info "Executing QA pipeline..."
|
||||
"${SCRIPT_DIR}/ci_qa.sh" "$@"
|
||||
}
|
||||
|
||||
# Function to run complete pipeline
|
||||
run_all_pipeline() {
|
||||
local start_time
|
||||
start_time=$(date +%s)
|
||||
|
||||
log_info "Starting complete CI/CD pipeline..."
|
||||
echo ""
|
||||
|
||||
# Step 1: Quality Assurance
|
||||
log_info "═══ Step 1/3: Quality Assurance ═══"
|
||||
run_qa_pipeline --check || {
|
||||
log_error "QA checks failed. Pipeline aborted."
|
||||
return 1
|
||||
}
|
||||
echo ""
|
||||
|
||||
# Step 2: Build Firmware
|
||||
log_info "═══ Step 2/3: Build Firmware ═══"
|
||||
run_build_pipeline "$@" || {
|
||||
log_error "Build failed. Pipeline aborted."
|
||||
return 1
|
||||
}
|
||||
echo ""
|
||||
|
||||
# Step 3: Build Site (only if not in --no-site mode)
|
||||
if [[ ! " $* " =~ " --no-site " ]]; then
|
||||
log_info "═══ Step 3/3: Build Documentation ═══"
|
||||
run_site_pipeline --mode prod || {
|
||||
log_warn "Site build failed, but continuing..."
|
||||
}
|
||||
else
|
||||
log_info "Skipping site build (--no-site flag)"
|
||||
fi
|
||||
|
||||
local end_time
|
||||
end_time=$(date +%s)
|
||||
local duration=$((end_time - start_time))
|
||||
|
||||
echo ""
|
||||
echo "╔════════════════════════════════════════╗"
|
||||
echo "║ Complete Pipeline Summary ║"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
echo " Total Duration: ${duration}s"
|
||||
echo " Status: SUCCESS ✓"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
}
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
# Check if no arguments provided
|
||||
if [[ $# -eq 0 ]]; then
|
||||
print_banner
|
||||
usage
|
||||
fi
|
||||
|
||||
# Get command
|
||||
local command="$1"
|
||||
shift || true
|
||||
|
||||
# Handle help flags
|
||||
if [[ "$command" == "--help" || "$command" == "-h" ]]; then
|
||||
print_banner
|
||||
usage
|
||||
fi
|
||||
|
||||
print_banner
|
||||
|
||||
# Route to appropriate pipeline
|
||||
case "$command" in
|
||||
build)
|
||||
run_build_pipeline "$@"
|
||||
;;
|
||||
site|docs)
|
||||
run_site_pipeline "$@"
|
||||
;;
|
||||
qa|lint)
|
||||
run_qa_pipeline "$@"
|
||||
;;
|
||||
all|pipeline)
|
||||
run_all_pipeline "$@"
|
||||
;;
|
||||
*)
|
||||
log_error "Unknown command: $command"
|
||||
echo ""
|
||||
usage
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Execute main function
|
||||
main "$@"
|
||||
Executable
+58
@@ -0,0 +1,58 @@
|
||||
# Build Scripts Configuration
|
||||
# Used by: All build scripts for centralized configuration
|
||||
|
||||
# Python Configuration
|
||||
PYTHON_VERSION="3.13"
|
||||
PLATFORMIO_VERSION="https://github.com/pioarduino/platformio-core/archive/refs/tags/v6.1.18.zip"
|
||||
|
||||
# Centralized Output Directory Structure
|
||||
# All CI/CD generated files go under generated/
|
||||
GENERATED_BASE_DIR="generated"
|
||||
ARTIFACTS_DIR="${GENERATED_BASE_DIR}/artifacts"
|
||||
SITE_DIR="${GENERATED_BASE_DIR}/site"
|
||||
REPORTS_DIR="${GENERATED_BASE_DIR}/reports"
|
||||
|
||||
# PlatformIO Directory Configuration
|
||||
BUILD_DIR=".pio/build"
|
||||
SCRIPTS_DIR="scripts"
|
||||
|
||||
# Build Configuration
|
||||
DEFAULT_ENVIRONMENT="esp32dev-all-test"
|
||||
ENABLE_VERBOSE_BUILD="false"
|
||||
ENABLE_BUILD_CACHE="true"
|
||||
|
||||
# Artifact Configuration
|
||||
ARTIFACT_RETENTION_DAYS="7"
|
||||
CREATE_MANIFEST="true"
|
||||
COMPRESS_ARTIFACTS="false"
|
||||
|
||||
# Version Configuration
|
||||
VERSION_FILE_PROD="scripts/latest_version.json"
|
||||
VERSION_FILE_DEV="scripts/latest_version_dev.json"
|
||||
USER_CONFIG_FILE="main/User_config.h"
|
||||
|
||||
# Logging Configuration
|
||||
ENABLE_COLOR_OUTPUT="true"
|
||||
LOG_LEVEL="INFO" # DEBUG, INFO, WARN, ERROR
|
||||
|
||||
# ============================================================================
|
||||
# Colors - ANSI color codes for terminal output
|
||||
# ============================================================================
|
||||
readonly RED='\033[0;31m'
|
||||
readonly GREEN='\033[0;32m'
|
||||
readonly YELLOW='\033[1;33m'
|
||||
readonly BLUE='\033[0;34m'
|
||||
readonly NC='\033[0m' # No Color
|
||||
|
||||
# ============================================================================
|
||||
# Logging Functions - Standardized logging across all build scripts
|
||||
# ============================================================================
|
||||
log_info() { echo -e "${GREEN}[INFO]${NC} $*" >&2; }
|
||||
log_warn() { echo -e "${YELLOW}[WARN]${NC} $*" >&2; }
|
||||
log_error() { echo -e "${RED}[ERROR]${NC} $*" >&2; }
|
||||
log_step() { echo -e "${BLUE}[STEP]${NC} $*" >&2; }
|
||||
|
||||
# Advanced Options
|
||||
ENABLE_CCACHE="false"
|
||||
CCACHE_DIR=".ccache"
|
||||
MAX_BUILD_JOBS="4"
|
||||
Executable
+384
@@ -0,0 +1,384 @@
|
||||
#!/bin/bash
|
||||
# CI/CD agnostic wrapper for complete build pipeline
|
||||
# Orchestrates all build scripts with a single command
|
||||
# Usage: ./scripts/ci.sh <environment> [OPTIONS]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Constants
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
# Load shared configuration (colors, logging functions, paths)
|
||||
if [[ -f "${SCRIPT_DIR}/ci_00_config.sh" ]]; then
|
||||
source "${SCRIPT_DIR}/ci_00_config.sh"
|
||||
else
|
||||
echo "ERROR: ci_00_config.sh not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Function to print banner
|
||||
print_banner() {
|
||||
echo "╔════════════════════════════════════════╗"
|
||||
echo "║ OpenMQTTGateway CI/CD Build ║"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# Function to print summary
|
||||
print_summary() {
|
||||
local env="$1"
|
||||
local version="$2"
|
||||
local start_time="$3"
|
||||
local end_time
|
||||
end_time=$(date +%s)
|
||||
local duration=$((end_time - start_time))
|
||||
|
||||
echo ""
|
||||
echo "╔════════════════════════════════════════╗"
|
||||
echo "║ Build Summary ║"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
echo " Environment: $env"
|
||||
echo " Version: $version"
|
||||
echo " Duration: ${duration}s"
|
||||
echo " Status: SUCCESS ✓"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
}
|
||||
|
||||
# Function to check if command exists
|
||||
command_exists() {
|
||||
command -v "$1" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
# Function to get command version
|
||||
get_command_version() {
|
||||
local cmd="$1"
|
||||
case "$cmd" in
|
||||
platformio)
|
||||
platformio --version 2>&1 | head -n1 | grep -oP '\d+\.\d+\.\d+' || echo "unknown"
|
||||
;;
|
||||
python|python3)
|
||||
python3 --version 2>&1 | grep -oP '\d+\.\d+' || echo "unknown"
|
||||
;;
|
||||
*)
|
||||
echo "unknown"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Function to verify required tools
|
||||
verify_build_tools() {
|
||||
log_info "Verifying required build tools..."
|
||||
|
||||
local missing_tools=()
|
||||
local version_mismatch=()
|
||||
|
||||
# Check Python
|
||||
if ! command_exists python3; then
|
||||
missing_tools+=("python3")
|
||||
else
|
||||
local python_version
|
||||
python_version=$(get_command_version python3)
|
||||
log_info "✓ Python ${python_version} found"
|
||||
fi
|
||||
|
||||
# Check PlatformIO
|
||||
if ! command_exists platformio; then
|
||||
missing_tools+=("platformio")
|
||||
else
|
||||
local pio_version
|
||||
pio_version=$(get_command_version platformio)
|
||||
log_info "✓ PlatformIO ${pio_version} found"
|
||||
fi
|
||||
|
||||
# Check git (for version auto-generation)
|
||||
if ! command_exists git; then
|
||||
log_warn "git not found (optional, but recommended)"
|
||||
else
|
||||
log_info "✓ git found"
|
||||
fi
|
||||
|
||||
# Report missing tools
|
||||
if [[ ${#missing_tools[@]} -gt 0 ]]; then
|
||||
log_error "Missing required tools: ${missing_tools[*]}"
|
||||
log_error ""
|
||||
log_error "Please install missing tools:"
|
||||
for tool in "${missing_tools[@]}"; do
|
||||
case "$tool" in
|
||||
python3)
|
||||
log_error " - Python 3: https://www.python.org/downloads/"
|
||||
;;
|
||||
platformio)
|
||||
log_error " - PlatformIO: pip3 install platformio"
|
||||
log_error " or: pip3 install ${PLATFORMIO_VERSION:-platformio}"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
log_error ""
|
||||
log_error "Or skip this check with: --skip-verification"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "All required tools are available"
|
||||
return 0
|
||||
}
|
||||
|
||||
# Function to cleanup on error
|
||||
cleanup_on_error() {
|
||||
log_error "Build failed, cleaning up..."
|
||||
# Restore any backups
|
||||
find . -name "*.bak" -type f -exec bash -c 'mv "$1" "${1%.bak}"' _ {} \; 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Function to list available environments
|
||||
list_environments() {
|
||||
log_info "Available PlatformIO environments:"
|
||||
echo ""
|
||||
|
||||
local env_files=("${PROJECT_ROOT}/platformio.ini" "${PROJECT_ROOT}/environments.ini")
|
||||
local envs=()
|
||||
|
||||
for file in "${env_files[@]}"; do
|
||||
if [[ -f "$file" ]]; then
|
||||
while IFS= read -r line; do
|
||||
if [[ "$line" =~ ^\[env:([^\]]+)\] ]]; then
|
||||
local env_name="${BASH_REMATCH[1]}"
|
||||
# Skip test environments
|
||||
if [[ ! "$env_name" =~ -test$ && ! "$env_name" =~ -all- ]]; then
|
||||
envs+=("$env_name")
|
||||
fi
|
||||
fi
|
||||
done < "$file"
|
||||
fi
|
||||
done
|
||||
|
||||
# Sort and display unique environments
|
||||
if [[ ${#envs[@]} -gt 0 ]]; then
|
||||
printf '%s\n' "${envs[@]}" | sort -u | column -c 80
|
||||
else
|
||||
log_warn "No environments found in configuration files"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
log_info "Total: $(printf '%s\n' "${envs[@]}" | sort -u | wc -l) environments"
|
||||
}
|
||||
|
||||
# Show usage
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 <environment> [OPTIONS]
|
||||
|
||||
Complete CI/CD build pipeline wrapper.
|
||||
|
||||
Arguments:
|
||||
environment PlatformIO environment name
|
||||
|
||||
Options:
|
||||
--version [TAG] Set version tag (if TAG omitted, auto-generated)
|
||||
--mode MODE Build mode: 'prod' or 'dev' [default: prod]
|
||||
'dev' enables OTA and development version
|
||||
--deploy-ready Prepare for deployment (renamed artifacts)
|
||||
--output DIR Output directory for artifacts [default: generated/artifacts/]
|
||||
--skip-verification Skip build tools verification
|
||||
--clean Clean build before starting
|
||||
--verbose Enable verbose output
|
||||
--list-envs List all available PlatformIO environments
|
||||
--help Show this help message
|
||||
|
||||
Environment Variables:
|
||||
CI Set to 'true' in CI/CD environments
|
||||
BUILD_NUMBER Build number from CI/CD
|
||||
GIT_COMMIT Git commit hash for versioning
|
||||
|
||||
Examples:
|
||||
# List available environments
|
||||
$0 --list-envs
|
||||
|
||||
# Local development build
|
||||
$0 esp32dev-all-test --mode dev
|
||||
|
||||
# Production release build
|
||||
$0 esp32dev-bt --version v1.7.0 --mode prod --deploy-ready
|
||||
|
||||
# CI/CD build (auto-detects version)
|
||||
$0 theengs-bridge --version --mode dev
|
||||
|
||||
EOF
|
||||
}
|
||||
|
||||
# Main pipeline
|
||||
main() {
|
||||
local environment=""
|
||||
local version=""
|
||||
local set_version=false
|
||||
local mode=""
|
||||
local deploy=false
|
||||
local output_dir=""
|
||||
local skip_verification=false
|
||||
local clean=false
|
||||
local verbose=false
|
||||
|
||||
# Parse arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--version)
|
||||
set_version=true
|
||||
# Check if next argument is a version tag or another option
|
||||
if [[ $# -gt 1 && ! "$2" =~ ^-- ]]; then
|
||||
version="$2"
|
||||
shift 2
|
||||
else
|
||||
shift
|
||||
fi
|
||||
;;
|
||||
--mode)
|
||||
if [[ $# -lt 2 ]]; then
|
||||
log_error "--mode requires an argument: 'prod' or 'dev'"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
if [[ "$2" != "prod" && "$2" != "dev" ]]; then
|
||||
log_error "Invalid mode: $2. Must be 'prod' or 'dev'"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
mode="$2"
|
||||
shift 2
|
||||
;;
|
||||
--deploy-ready)
|
||||
deploy=true
|
||||
shift
|
||||
;;
|
||||
--output)
|
||||
if [[ $# -lt 2 ]]; then
|
||||
log_error "--output requires a directory argument"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
output_dir="$2"
|
||||
shift 2
|
||||
;;
|
||||
--skip-verification)
|
||||
skip_verification=true
|
||||
shift
|
||||
;;
|
||||
--clean)
|
||||
clean=true
|
||||
shift
|
||||
;;
|
||||
--verbose)
|
||||
verbose=true
|
||||
shift
|
||||
;;
|
||||
--list-envs)
|
||||
list_environments
|
||||
exit 0
|
||||
;;
|
||||
--help|-h)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
-*)
|
||||
log_error "Unknown option: $1"
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
*)
|
||||
environment="$1"
|
||||
shift
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Set default mode if not specified
|
||||
if [[ -z "$mode" ]]; then
|
||||
mode="prod"
|
||||
log_info "Mode not specified, defaulting to production"
|
||||
fi
|
||||
|
||||
# Validate environment
|
||||
if [[ -z "$environment" ]]; then
|
||||
log_error "Environment name is required"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Auto-generate version if --version flag is set but no tag provided
|
||||
if [[ "$set_version" == "true" && -z "$version" ]]; then
|
||||
if [[ "${CI:-false}" == "true" ]]; then
|
||||
# CI/CD environment
|
||||
version="${BUILD_NUMBER:-${GIT_COMMIT:-unknown}}"
|
||||
else
|
||||
# Local development
|
||||
version="local-$(date +%Y%m%d-%H%M%S)"
|
||||
fi
|
||||
log_info "Auto-generated version: $version"
|
||||
fi
|
||||
|
||||
# Setup error handling
|
||||
trap cleanup_on_error ERR
|
||||
|
||||
# Change to project root
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Start timer
|
||||
local start_time
|
||||
start_time=$(date +%s)
|
||||
|
||||
# Print banner
|
||||
print_banner
|
||||
|
||||
# Step 1: Verify build tools
|
||||
if [[ "$skip_verification" == "false" ]]; then
|
||||
log_step "1/4 Verifying build tools..."
|
||||
verify_build_tools || exit 1
|
||||
echo ""
|
||||
else
|
||||
log_warn "Skipping build tools verification (--skip-verification)"
|
||||
echo ""
|
||||
fi
|
||||
|
||||
# Step 2: Set version (only if --version flag was provided)
|
||||
if [[ "$set_version" == "true" ]]; then
|
||||
log_step "2/4 Setting version: $version"
|
||||
if [[ "$mode" == "dev" ]]; then
|
||||
"${SCRIPT_DIR}/ci_set_version.sh" "$version" --dev || exit 1
|
||||
else
|
||||
"${SCRIPT_DIR}/ci_set_version.sh" "$version" --prod || exit 1
|
||||
fi
|
||||
echo ""
|
||||
else
|
||||
log_info "Skipping version setting (use --version to set version)"
|
||||
echo ""
|
||||
fi
|
||||
|
||||
# Step 3: Build firmware
|
||||
log_step "3/4 Building firmware for: $environment"
|
||||
local build_opts=()
|
||||
[[ "$mode" == "dev" ]] && build_opts+=(--dev-ota)
|
||||
[[ "$clean" == "true" ]] && build_opts+=(--clean)
|
||||
[[ "$verbose" == "true" ]] && build_opts+=(--verbose)
|
||||
|
||||
"${SCRIPT_DIR}/ci_build_firmware.sh" "$environment" "${build_opts[@]}" || exit 1
|
||||
echo ""
|
||||
|
||||
# Step 4: Prepare artifacts
|
||||
log_step "4/4 Preparing artifacts..."
|
||||
local artifact_opts=()
|
||||
[[ "$deploy" == "true" ]] && artifact_opts+=(--deploy) || artifact_opts+=(--standard)
|
||||
artifact_opts+=(--manifest)
|
||||
[[ -n "$output_dir" ]] && artifact_opts+=(--output "$output_dir")
|
||||
|
||||
"${SCRIPT_DIR}/ci_prepare_artifacts.sh" "$environment" "${artifact_opts[@]}" || exit 1
|
||||
echo ""
|
||||
|
||||
# Print summary
|
||||
print_summary "$environment" "$version" "$start_time"
|
||||
|
||||
log_info "✓ Complete build pipeline finished successfully"
|
||||
}
|
||||
|
||||
# Run main if executed directly
|
||||
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
|
||||
main "$@"
|
||||
fi
|
||||
Executable
+281
@@ -0,0 +1,281 @@
|
||||
#!/bin/bash
|
||||
# Builds firmware for specified PlatformIO environment
|
||||
# Used by: CI/CD pipelines and local development
|
||||
# Usage: ./build_firmware.sh <environment> [OPTIONS]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Constants
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
# Load shared configuration (colors, logging functions, paths)
|
||||
if [[ -f "${SCRIPT_DIR}/ci_00_config.sh" ]]; then
|
||||
source "${SCRIPT_DIR}/ci_00_config.sh"
|
||||
else
|
||||
echo "ERROR: ci_00_config.sh not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Set absolute path for BUILD_DIR
|
||||
BUILD_DIR="${PROJECT_ROOT}/${BUILD_DIR}"
|
||||
|
||||
# Script-specific logging function
|
||||
log_build() { echo -e "${BLUE}[BUILD]${NC} $*"; }
|
||||
|
||||
# Function to validate environment name
|
||||
validate_environment() {
|
||||
local env="$1"
|
||||
|
||||
if [[ -z "$env" ]]; then
|
||||
log_error "Environment name is required"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Check if environment exists in platformio.ini or environments.ini
|
||||
if ! grep -q "^\[env:${env}\]" "${PROJECT_ROOT}/platformio.ini" "${PROJECT_ROOT}/environments.ini" 2>/dev/null; then
|
||||
log_warn "Environment '${env}' not found in configuration files"
|
||||
log_warn "Proceeding anyway (PlatformIO will validate)"
|
||||
fi
|
||||
|
||||
log_info "Building environment: $env"
|
||||
}
|
||||
|
||||
# Function to setup build environment variables
|
||||
setup_build_env() {
|
||||
local enable_dev_ota="${1:-false}"
|
||||
|
||||
export PYTHONIOENCODING=utf-8
|
||||
export PYTHONUTF8=1
|
||||
|
||||
if [[ "$enable_dev_ota" == "true" ]]; then
|
||||
export PLATFORMIO_BUILD_FLAGS='"-DDEVELOPMENTOTA=true"'
|
||||
log_info "Development OTA enabled"
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to check PlatformIO availability
|
||||
check_platformio() {
|
||||
if ! command -v platformio >/dev/null 2>&1; then
|
||||
log_error "PlatformIO not found. Run setup_build_env.sh first"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to clean build artifacts
|
||||
clean_build() {
|
||||
local env="$1"
|
||||
local env_dir="${BUILD_DIR}/${env}"
|
||||
|
||||
if [[ -d "$env_dir" ]]; then
|
||||
log_info "Cleaning previous build artifacts for: $env"
|
||||
rm -rf "$env_dir"
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to run PlatformIO build
|
||||
run_build() {
|
||||
local env="$1"
|
||||
local clean="${2:-false}"
|
||||
local verbose="${3:-false}"
|
||||
|
||||
log_build "Starting build for environment: $env"
|
||||
|
||||
local build_cmd="platformio run -e $env"
|
||||
|
||||
if [[ "$clean" == "true" ]]; then
|
||||
build_cmd="platformio run -e $env --target clean && $build_cmd"
|
||||
fi
|
||||
|
||||
if [[ "$verbose" == "true" ]]; then
|
||||
build_cmd="$build_cmd --verbose"
|
||||
fi
|
||||
|
||||
# Execute build with timing
|
||||
local start_time
|
||||
start_time=$(date +%s)
|
||||
|
||||
if eval "$build_cmd"; then
|
||||
local end_time
|
||||
end_time=$(date +%s)
|
||||
local duration=$((end_time - start_time))
|
||||
|
||||
log_build "Build completed successfully in ${duration}s"
|
||||
return 0
|
||||
else
|
||||
log_error "Build failed for environment: $env"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to verify build artifacts
|
||||
verify_artifacts() {
|
||||
local env="$1"
|
||||
local env_dir="${BUILD_DIR}/${env}"
|
||||
|
||||
log_info "Verifying build artifacts..."
|
||||
|
||||
local artifacts_found=0
|
||||
local firmware="${env_dir}/firmware.bin"
|
||||
local partitions="${env_dir}/partitions.bin"
|
||||
local bootloader="${env_dir}/bootloader.bin"
|
||||
|
||||
if [[ -f "$firmware" ]]; then
|
||||
local size
|
||||
size=$(stat -f%z "$firmware" 2>/dev/null || stat -c%s "$firmware" 2>/dev/null)
|
||||
log_info "✓ firmware.bin (${size} bytes)"
|
||||
((artifacts_found++))
|
||||
else
|
||||
log_warn "✗ firmware.bin not found"
|
||||
fi
|
||||
|
||||
if [[ -f "$partitions" ]]; then
|
||||
log_info "✓ partitions.bin"
|
||||
((artifacts_found++))
|
||||
fi
|
||||
|
||||
if [[ -f "$bootloader" ]]; then
|
||||
log_info "✓ bootloader.bin"
|
||||
((artifacts_found++))
|
||||
fi
|
||||
|
||||
if [[ $artifacts_found -eq 0 ]]; then
|
||||
log_error "No build artifacts found"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "Found ${artifacts_found} artifact(s)"
|
||||
}
|
||||
|
||||
# Function to show build summary
|
||||
show_build_summary() {
|
||||
local env="$1"
|
||||
local env_dir="${BUILD_DIR}/${env}"
|
||||
|
||||
echo ""
|
||||
echo "═══════════════════════════════════════"
|
||||
echo " Build Summary: $env"
|
||||
echo "═══════════════════════════════════════"
|
||||
|
||||
if [[ -d "$env_dir" ]]; then
|
||||
find "$env_dir" -name "*.bin" -o -name "*.elf" | while read -r file; do
|
||||
local size
|
||||
size=$(stat -f%z "$file" 2>/dev/null || stat -c%s "$file" 2>/dev/null)
|
||||
local size_kb=$((size / 1024))
|
||||
echo " $(basename "$file"): ${size_kb} KB"
|
||||
done
|
||||
fi
|
||||
|
||||
echo "═══════════════════════════════════════"
|
||||
}
|
||||
|
||||
# Show usage
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 <environment> [OPTIONS]
|
||||
|
||||
Build firmware for a specific PlatformIO environment.
|
||||
|
||||
Arguments:
|
||||
environment PlatformIO environment name (e.g., esp32dev-all-test)
|
||||
|
||||
Options:
|
||||
--dev-ota Enable development OTA build flags
|
||||
--clean Clean build artifacts before building
|
||||
--verbose Enable verbose build output
|
||||
--no-verify Skip artifact verification
|
||||
--help Show this help message
|
||||
|
||||
Examples:
|
||||
$0 esp32dev-all-test
|
||||
$0 esp32dev-bt --dev-ota
|
||||
$0 theengs-bridge --clean --verbose
|
||||
|
||||
EOF
|
||||
}
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
local environment=""
|
||||
local enable_dev_ota=false
|
||||
local clean_build_flag=false
|
||||
local verbose=false
|
||||
local verify=true
|
||||
|
||||
# Parse arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--dev-ota)
|
||||
enable_dev_ota=true
|
||||
shift
|
||||
;;
|
||||
--clean)
|
||||
clean_build_flag=true
|
||||
shift
|
||||
;;
|
||||
--verbose)
|
||||
verbose=true
|
||||
shift
|
||||
;;
|
||||
--no-verify)
|
||||
verify=false
|
||||
shift
|
||||
;;
|
||||
--help|-h)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
-*)
|
||||
log_error "Unknown option: $1"
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
*)
|
||||
environment="$1"
|
||||
shift
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Validate inputs
|
||||
if [[ -z "$environment" ]]; then
|
||||
log_error "Environment name is required"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Change to project root
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Check prerequisites
|
||||
check_platformio || exit 1
|
||||
|
||||
# Validate environment
|
||||
validate_environment "$environment" || exit 1
|
||||
|
||||
# Setup build environment
|
||||
setup_build_env "$enable_dev_ota"
|
||||
|
||||
# Clean if requested
|
||||
if [[ "$clean_build_flag" == "true" ]]; then
|
||||
clean_build "$environment"
|
||||
fi
|
||||
|
||||
# Run build
|
||||
run_build "$environment" "$clean_build_flag" "$verbose" || exit 1
|
||||
|
||||
# Verify artifacts
|
||||
if [[ "$verify" == "true" ]]; then
|
||||
verify_artifacts "$environment" || exit 1
|
||||
fi
|
||||
|
||||
# Show summary
|
||||
show_build_summary "$environment"
|
||||
|
||||
log_info "Build process completed successfully"
|
||||
}
|
||||
|
||||
# Run main if executed directly
|
||||
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
|
||||
main "$@"
|
||||
fi
|
||||
Executable
+318
@@ -0,0 +1,318 @@
|
||||
#!/bin/bash
|
||||
# Prepares firmware artifacts for upload or deployment
|
||||
# Used by: CI/CD pipelines for artifact packaging
|
||||
# Usage: ./prepare_artifacts.sh <environment> [OPTIONS]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Constants
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
# Load shared configuration (colors, logging functions, paths)
|
||||
if [[ -f "${SCRIPT_DIR}/ci_00_config.sh" ]]; then
|
||||
source "${SCRIPT_DIR}/ci_00_config.sh"
|
||||
else
|
||||
echo "ERROR: ci_00_config.sh not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Set absolute paths
|
||||
BUILD_DIR="${PROJECT_ROOT}/${BUILD_DIR}"
|
||||
DEFAULT_OUTPUT_DIR="${PROJECT_ROOT}/${ARTIFACTS_DIR}"
|
||||
|
||||
# Function to create output directory
|
||||
create_output_dir() {
|
||||
local output_dir="$1"
|
||||
|
||||
if [[ -d "$output_dir" ]]; then
|
||||
log_warn "Output directory already exists: $output_dir"
|
||||
log_info "Cleaning existing artifacts..."
|
||||
rm -rf "$output_dir"
|
||||
fi
|
||||
|
||||
mkdir -p "$output_dir"
|
||||
log_info "Created output directory: $output_dir"
|
||||
}
|
||||
|
||||
# Function to copy artifact with optional renaming
|
||||
copy_artifact() {
|
||||
local source="$1"
|
||||
local dest="$2"
|
||||
local artifact_type="$3"
|
||||
|
||||
if [[ ! -f "$source" ]]; then
|
||||
log_warn "${artifact_type} not found: $source"
|
||||
return 1
|
||||
fi
|
||||
|
||||
if cp "$source" "$dest"; then
|
||||
local size
|
||||
size=$(stat -f%z "$dest" 2>/dev/null || stat -c%s "$dest" 2>/dev/null)
|
||||
local size_kb=$((size / 1024))
|
||||
log_info "✓ Copied ${artifact_type}: $(basename "$dest") (${size_kb} KB)"
|
||||
return 0
|
||||
else
|
||||
log_error "Failed to copy ${artifact_type}: $source"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to prepare standard artifacts (no renaming)
|
||||
prepare_standard_artifacts() {
|
||||
local env="$1"
|
||||
local output_dir="$2"
|
||||
local env_dir="${BUILD_DIR}/${env}"
|
||||
|
||||
log_info "Preparing STANDARD artifacts for: $env"
|
||||
|
||||
local copied=0
|
||||
|
||||
# Copy firmware.bin (required)
|
||||
if copy_artifact "${env_dir}/firmware.bin" "${output_dir}/firmware.bin" "firmware"; then
|
||||
((copied++))
|
||||
fi
|
||||
|
||||
# Copy partitions.bin (optional)
|
||||
copy_artifact "${env_dir}/partitions.bin" "${output_dir}/partitions.bin" "partitions" && ((copied++)) || true
|
||||
|
||||
# Note: bootloader.bin is NOT copied in standard mode (only needed for deployment)
|
||||
|
||||
if [[ $copied -eq 0 ]]; then
|
||||
log_error "No artifacts were copied"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "Copied ${copied} artifact(s) in standard mode"
|
||||
}
|
||||
|
||||
# Function to prepare deployment artifacts (with renaming)
|
||||
prepare_deployment_artifacts() {
|
||||
local env="$1"
|
||||
local output_dir="$2"
|
||||
local env_dir="${BUILD_DIR}/${env}"
|
||||
|
||||
log_info "Preparing DEPLOYMENT artifacts for: $env"
|
||||
|
||||
local copied=0
|
||||
|
||||
# Copy and rename firmware.bin
|
||||
if copy_artifact "${env_dir}/firmware.bin" "${output_dir}/${env}-firmware.bin" "firmware"; then
|
||||
((copied++))
|
||||
fi
|
||||
|
||||
# Copy and rename partitions.bin (optional)
|
||||
copy_artifact "${env_dir}/partitions.bin" "${output_dir}/${env}-partitions.bin" "partitions" && ((copied++)) || true
|
||||
|
||||
# Copy and rename bootloader.bin (optional)
|
||||
copy_artifact "${env_dir}/bootloader.bin" "${output_dir}/${env}-bootloader.bin" "bootloader" && ((copied++)) || true
|
||||
|
||||
# Copy boot_app0.bin if exists (ESP32 specific)
|
||||
copy_artifact "${env_dir}/boot_app0.bin" "${output_dir}/${env}-boot_app0.bin" "boot_app0" && ((copied++)) || true
|
||||
|
||||
if [[ $copied -eq 0 ]]; then
|
||||
log_error "No artifacts were copied"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "Copied ${copied} artifact(s) in deployment mode"
|
||||
}
|
||||
|
||||
# Function to create manifest file
|
||||
create_manifest() {
|
||||
local env="$1"
|
||||
local output_dir="$2"
|
||||
local manifest="${output_dir}/manifest.txt"
|
||||
|
||||
log_info "Creating artifact manifest..."
|
||||
|
||||
{
|
||||
echo "Environment: $env"
|
||||
echo "Build Date: $(date -u '+%Y-%m-%d %H:%M:%S UTC')"
|
||||
echo "Build Host: $(hostname)"
|
||||
echo ""
|
||||
echo "Artifacts:"
|
||||
|
||||
find "$output_dir" -type f -name "*.bin" | sort | while read -r file; do
|
||||
local size
|
||||
size=$(stat -f%z "$file" 2>/dev/null || stat -c%s "$file" 2>/dev/null)
|
||||
local size_kb=$((size / 1024))
|
||||
local md5sum_val
|
||||
md5sum_val=$(md5sum "$file" 2>/dev/null | cut -d' ' -f1 || md5 -q "$file" 2>/dev/null)
|
||||
echo " - $(basename "$file"): ${size_kb} KB (MD5: ${md5sum_val})"
|
||||
done
|
||||
} > "$manifest"
|
||||
|
||||
log_info "Manifest created: $manifest"
|
||||
}
|
||||
|
||||
# Function to compress artifacts (optional)
|
||||
compress_artifacts() {
|
||||
local output_dir="$1"
|
||||
local archive_name="$2"
|
||||
|
||||
log_info "Compressing artifacts..."
|
||||
|
||||
local archive="${output_dir}/${archive_name}.tar.gz"
|
||||
|
||||
if tar -czf "$archive" -C "$output_dir" .; then
|
||||
local size
|
||||
size=$(stat -f%z "$archive" 2>/dev/null || stat -c%s "$archive" 2>/dev/null)
|
||||
local size_kb=$((size / 1024))
|
||||
log_info "Archive created: ${archive_name}.tar.gz (${size_kb} KB)"
|
||||
else
|
||||
log_error "Failed to create archive"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to list artifacts
|
||||
list_artifacts() {
|
||||
local output_dir="$1"
|
||||
|
||||
echo ""
|
||||
echo "═══════════════════════════════════════"
|
||||
echo " Prepared Artifacts"
|
||||
echo "═══════════════════════════════════════"
|
||||
|
||||
if [[ -d "$output_dir" ]]; then
|
||||
find "$output_dir" -type f | sort | while read -r file; do
|
||||
local size
|
||||
size=$(stat -f%z "$file" 2>/dev/null || stat -c%s "$file" 2>/dev/null)
|
||||
local size_kb=$((size / 1024))
|
||||
echo " $(basename "$file"): ${size_kb} KB"
|
||||
done
|
||||
else
|
||||
echo " No artifacts found"
|
||||
fi
|
||||
|
||||
echo "═══════════════════════════════════════"
|
||||
}
|
||||
|
||||
# Show usage
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 <environment> [OPTIONS]
|
||||
|
||||
Prepare firmware artifacts for upload or deployment.
|
||||
|
||||
Arguments:
|
||||
environment PlatformIO environment name
|
||||
|
||||
Options:
|
||||
--deploy Prepare for deployment (rename with environment prefix)
|
||||
--standard Prepare standard artifacts (no renaming) [default]
|
||||
--output DIR Output directory [default: generated/artifacts/]
|
||||
--manifest Create manifest file with artifact metadata
|
||||
--compress Compress artifacts into tar.gz archive
|
||||
--help Show this help message
|
||||
|
||||
Examples:
|
||||
$0 esp32dev-all-test
|
||||
$0 esp32dev-bt --deploy --manifest
|
||||
$0 theengs-bridge --output build/artifacts --compress
|
||||
|
||||
EOF
|
||||
}
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
local environment=""
|
||||
local mode="standard"
|
||||
local output_dir="$DEFAULT_OUTPUT_DIR"
|
||||
local create_manifest_flag=false
|
||||
local compress_flag=false
|
||||
|
||||
# Parse arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--deploy)
|
||||
mode="deploy"
|
||||
shift
|
||||
;;
|
||||
--standard)
|
||||
mode="standard"
|
||||
shift
|
||||
;;
|
||||
--output)
|
||||
output_dir="$2"
|
||||
shift 2
|
||||
;;
|
||||
--manifest)
|
||||
create_manifest_flag=true
|
||||
shift
|
||||
;;
|
||||
--compress)
|
||||
compress_flag=true
|
||||
shift
|
||||
;;
|
||||
--help|-h)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
-*)
|
||||
log_error "Unknown option: $1"
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
*)
|
||||
environment="$1"
|
||||
shift
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Validate inputs
|
||||
if [[ -z "$environment" ]]; then
|
||||
log_error "Environment name is required"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Change to project root
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Check if build directory exists
|
||||
if [[ ! -d "${BUILD_DIR}/${environment}" ]]; then
|
||||
log_error "Build directory not found for environment: $environment"
|
||||
log_error "Run build_firmware.sh first"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Create output directory
|
||||
create_output_dir "$output_dir"
|
||||
|
||||
# Prepare artifacts based on mode
|
||||
case "$mode" in
|
||||
standard)
|
||||
prepare_standard_artifacts "$environment" "$output_dir" || exit 1
|
||||
;;
|
||||
deploy)
|
||||
prepare_deployment_artifacts "$environment" "$output_dir" || exit 1
|
||||
;;
|
||||
*)
|
||||
log_error "Unknown mode: $mode"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
# Create manifest if requested
|
||||
if [[ "$create_manifest_flag" == "true" ]]; then
|
||||
create_manifest "$environment" "$output_dir"
|
||||
fi
|
||||
|
||||
# Compress if requested
|
||||
if [[ "$compress_flag" == "true" ]]; then
|
||||
compress_artifacts "$output_dir" "$environment"
|
||||
fi
|
||||
|
||||
# Show summary
|
||||
list_artifacts "$output_dir"
|
||||
|
||||
log_info "Artifact preparation completed successfully"
|
||||
}
|
||||
|
||||
# Run main if executed directly
|
||||
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
|
||||
main "$@"
|
||||
fi
|
||||
Executable
+409
@@ -0,0 +1,409 @@
|
||||
#!/bin/bash
|
||||
# CI/CD Quality Assurance (QA) - Code Linting and Formatting
|
||||
# Checks and fixes code formatting using clang-format
|
||||
# Usage: ./scripts/ci_qa.sh [OPTIONS]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Constants
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
# Load shared configuration
|
||||
if [[ -f "${SCRIPT_DIR}/ci_00_config.sh" ]]; then
|
||||
source "${SCRIPT_DIR}/ci_00_config.sh"
|
||||
else
|
||||
echo "ERROR: ci_00_config.sh not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Default values
|
||||
CHECK_MODE=true
|
||||
FIX_MODE=false
|
||||
FORMAT_ONLY=false
|
||||
SOURCE_DIR="main"
|
||||
EXTENSIONS="h,ino,cpp"
|
||||
CLANG_FORMAT_VERSION="9"
|
||||
VERBOSE=false
|
||||
|
||||
# Function to check if clang-format is available
|
||||
check_clang_format() {
|
||||
local version="$1"
|
||||
local cmd="clang-format-${version}"
|
||||
|
||||
if command -v "$cmd" >/dev/null 2>&1; then
|
||||
echo "$cmd"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Try without version suffix
|
||||
if command -v clang-format >/dev/null 2>&1; then
|
||||
echo "clang-format"
|
||||
return 0
|
||||
fi
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
# Function to find files to check
|
||||
find_files() {
|
||||
local source="$1"
|
||||
local extensions="$2"
|
||||
|
||||
if [[ ! -d "${PROJECT_ROOT}/${source}" ]]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
local find_patterns=()
|
||||
IFS=',' read -ra exts <<< "$extensions"
|
||||
for ext in "${exts[@]}"; do
|
||||
find_patterns+=(-name "*.${ext}" -o)
|
||||
done
|
||||
# Remove last -o
|
||||
unset 'find_patterns[-1]'
|
||||
|
||||
local files
|
||||
files=$(find "${PROJECT_ROOT}/${source}" -type f \( "${find_patterns[@]}" \) 2>/dev/null || true)
|
||||
|
||||
if [[ -z "$files" ]]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
echo "$files"
|
||||
}
|
||||
|
||||
# Function to check formatting
|
||||
check_formatting() {
|
||||
local clang_format_cmd="$1"
|
||||
local files="$2"
|
||||
|
||||
log_info "Checking code formatting..."
|
||||
|
||||
local failed_files=()
|
||||
local checked_count=0
|
||||
local has_issues=false
|
||||
|
||||
while IFS= read -r file; do
|
||||
if [[ -z "$file" ]]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
checked_count=$((checked_count + 1))
|
||||
|
||||
if [[ "$VERBOSE" == true ]]; then
|
||||
log_info "Checking: $file"
|
||||
fi
|
||||
|
||||
# Check if file needs formatting and capture diff
|
||||
local diff_output
|
||||
diff_output=$("$clang_format_cmd" --dry-run --Werror "$file" 2>&1)
|
||||
local format_result=$?
|
||||
|
||||
if [[ $format_result -ne 0 ]]; then
|
||||
failed_files+=("$file")
|
||||
has_issues=true
|
||||
|
||||
# Show the actual formatting differences
|
||||
echo ""
|
||||
log_warn "⚠ Formatting issues in: $file"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
|
||||
# Generate and show diff with colors
|
||||
local actual_diff
|
||||
actual_diff=$(diff -u "$file" <("$clang_format_cmd" "$file") 2>/dev/null || true)
|
||||
|
||||
if [[ -n "$actual_diff" ]]; then
|
||||
echo "$actual_diff" | head -50 | while IFS= read -r line; do
|
||||
if [[ "$line" =~ ^-[^-] ]]; then
|
||||
echo -e "\033[31m$line\033[0m" # Red for removed lines
|
||||
elif [[ "$line" =~ ^+[^+] ]]; then
|
||||
echo -e "\033[32m$line\033[0m" # Green for added lines
|
||||
elif [[ "$line" =~ ^@@ ]]; then
|
||||
echo -e "\033[36m$line\033[0m" # Cyan for line numbers
|
||||
else
|
||||
echo "$line"
|
||||
fi
|
||||
done
|
||||
else
|
||||
echo "$diff_output"
|
||||
fi
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
else
|
||||
if [[ "$VERBOSE" == true ]]; then
|
||||
log_info " ✓ OK"
|
||||
fi
|
||||
fi
|
||||
done <<< "$files"
|
||||
|
||||
echo ""
|
||||
log_info "Checked ${checked_count} files"
|
||||
|
||||
if [[ $has_issues == true ]]; then
|
||||
echo ""
|
||||
log_error "Found ${#failed_files[@]} files with formatting issues:"
|
||||
for file in "${failed_files[@]}"; do
|
||||
log_error " - $file"
|
||||
done
|
||||
echo ""
|
||||
log_error "To fix these issues automatically, run:"
|
||||
log_error " $0 --fix"
|
||||
echo ""
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "✓ All files are properly formatted"
|
||||
return 0
|
||||
}
|
||||
|
||||
# Function to fix formatting
|
||||
fix_formatting() {
|
||||
local clang_format_cmd="$1"
|
||||
local files="$2"
|
||||
|
||||
log_info "Fixing code formatting..."
|
||||
|
||||
local fixed_count=0
|
||||
local total_count=0
|
||||
|
||||
while IFS= read -r file; do
|
||||
if [[ -z "$file" ]]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
total_count=$((total_count + 1))
|
||||
|
||||
if [[ "$VERBOSE" == true ]]; then
|
||||
log_info "Processing: $file"
|
||||
fi
|
||||
|
||||
# Apply formatting in-place
|
||||
if "$clang_format_cmd" -i "$file" 2>/dev/null; then
|
||||
fixed_count=$((fixed_count + 1))
|
||||
if [[ "$VERBOSE" == true ]]; then
|
||||
log_info " ✓ Formatted"
|
||||
fi
|
||||
else
|
||||
if [[ "$VERBOSE" == true ]]; then
|
||||
log_warn " ✗ Failed to format"
|
||||
fi
|
||||
fi
|
||||
done <<< "$files"
|
||||
|
||||
echo ""
|
||||
log_info "Processed ${total_count} files"
|
||||
log_info "✓ Formatting applied to ${fixed_count} files"
|
||||
|
||||
if [[ $fixed_count -gt 0 ]]; then
|
||||
log_warn ""
|
||||
log_warn "Files have been modified. Please review and commit the changes:"
|
||||
log_warn " git diff"
|
||||
log_warn " git add -u"
|
||||
log_warn " git commit -m 'style: apply clang-format'"
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to run all QA checks
|
||||
run_all_checks() {
|
||||
log_info "Running all quality assurance checks..."
|
||||
|
||||
local all_passed=true
|
||||
|
||||
# Format check
|
||||
log_info "═══ Code Formatting ═══"
|
||||
if ! run_format_check; then
|
||||
all_passed=false
|
||||
fi
|
||||
echo ""
|
||||
|
||||
# Future: Add more checks here
|
||||
# - cppcheck static analysis
|
||||
# - code complexity metrics
|
||||
# - TODO/FIXME detection
|
||||
# - license header validation
|
||||
|
||||
if [[ "$all_passed" == false ]]; then
|
||||
log_error "Some QA checks failed"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "✓ All QA checks passed"
|
||||
return 0
|
||||
}
|
||||
|
||||
# Function to run format check
|
||||
run_format_check() {
|
||||
log_info "Checking for clang-format version ${CLANG_FORMAT_VERSION}..."
|
||||
|
||||
local clang_format_cmd
|
||||
clang_format_cmd=$(check_clang_format "$CLANG_FORMAT_VERSION")
|
||||
|
||||
if [[ $? -ne 0 ]] || [[ -z "$clang_format_cmd" ]]; then
|
||||
log_error "clang-format not found"
|
||||
log_error "Please install clang-format:"
|
||||
log_error " Ubuntu/Debian: sudo apt-get install clang-format-${CLANG_FORMAT_VERSION}"
|
||||
log_error " macOS: brew install clang-format"
|
||||
return 1
|
||||
fi
|
||||
|
||||
if [[ "$clang_format_cmd" == "clang-format-${CLANG_FORMAT_VERSION}" ]]; then
|
||||
log_info "✓ clang-format-${CLANG_FORMAT_VERSION} found"
|
||||
else
|
||||
local installed_version
|
||||
installed_version=$(clang-format --version | grep -oP '\d+\.\d+' | head -1 || echo "unknown")
|
||||
log_warn "clang-format-${CLANG_FORMAT_VERSION} not found, using clang-format (version ${installed_version})"
|
||||
fi
|
||||
|
||||
log_info "Finding files in '${SOURCE_DIR}' with extensions: ${EXTENSIONS}"
|
||||
|
||||
local files
|
||||
files=$(find_files "$SOURCE_DIR" "$EXTENSIONS")
|
||||
|
||||
if [[ $? -ne 0 ]] || [[ -z "$files" ]]; then
|
||||
log_error "Source directory not found: ${PROJECT_ROOT}/${SOURCE_DIR}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
local file_count
|
||||
file_count=$(echo "$files" | wc -l)
|
||||
log_info "Found ${file_count} files to check"
|
||||
|
||||
if [[ "$FIX_MODE" == true ]]; then
|
||||
fix_formatting "$clang_format_cmd" "$files"
|
||||
else
|
||||
check_formatting "$clang_format_cmd" "$files"
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to show usage
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 [OPTIONS]
|
||||
|
||||
Run quality assurance checks on OpenMQTTGateway code.
|
||||
|
||||
Options:
|
||||
--check Check formatting only (CI mode) [default]
|
||||
--fix Fix formatting issues automatically
|
||||
--format Run only format checks
|
||||
--all Run all QA checks [default]
|
||||
--source DIR Source directory to check [default: main]
|
||||
--extensions EXTS File extensions (comma-separated) [default: h,ino,cpp]
|
||||
--clang-format-version V clang-format version [default: 9]
|
||||
--verbose Enable verbose output
|
||||
--help Show this help message
|
||||
|
||||
Examples:
|
||||
# Check formatting (CI mode)
|
||||
$0 --check
|
||||
|
||||
# Fix formatting issues
|
||||
$0 --fix
|
||||
|
||||
# Check specific directory
|
||||
$0 --check --source lib/LEDManager
|
||||
|
||||
# Check with custom extensions
|
||||
$0 --check --extensions h,cpp
|
||||
|
||||
# Verbose output
|
||||
$0 --check --verbose
|
||||
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
# Parse command line arguments
|
||||
parse_args() {
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--check)
|
||||
CHECK_MODE=true
|
||||
FIX_MODE=false
|
||||
shift
|
||||
;;
|
||||
--fix)
|
||||
FIX_MODE=true
|
||||
CHECK_MODE=false
|
||||
shift
|
||||
;;
|
||||
--format)
|
||||
FORMAT_ONLY=true
|
||||
shift
|
||||
;;
|
||||
--all)
|
||||
FORMAT_ONLY=false
|
||||
shift
|
||||
;;
|
||||
--source)
|
||||
SOURCE_DIR="$2"
|
||||
shift 2
|
||||
;;
|
||||
--extensions)
|
||||
EXTENSIONS="$2"
|
||||
shift 2
|
||||
;;
|
||||
--clang-format-version)
|
||||
CLANG_FORMAT_VERSION="$2"
|
||||
shift 2
|
||||
;;
|
||||
--verbose)
|
||||
VERBOSE=true
|
||||
shift
|
||||
;;
|
||||
--help)
|
||||
usage
|
||||
;;
|
||||
*)
|
||||
log_error "Unknown option: $1"
|
||||
usage
|
||||
;;
|
||||
esac
|
||||
done
|
||||
}
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
local start_time
|
||||
start_time=$(date +%s)
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
log_info "Starting QA pipeline..."
|
||||
|
||||
if [[ "$FIX_MODE" == true ]]; then
|
||||
log_info "Mode: FIX (will modify files)"
|
||||
else
|
||||
log_info "Mode: CHECK (read-only)"
|
||||
fi
|
||||
|
||||
# Run checks
|
||||
local result=0
|
||||
if [[ "$FORMAT_ONLY" == true ]]; then
|
||||
run_format_check || result=$?
|
||||
else
|
||||
run_all_checks || result=$?
|
||||
fi
|
||||
|
||||
local end_time
|
||||
end_time=$(date +%s)
|
||||
local duration=$((end_time - start_time))
|
||||
|
||||
echo ""
|
||||
echo "╔════════════════════════════════════════╗"
|
||||
echo "║ QA Pipeline Summary ║"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
echo " Duration: ${duration}s"
|
||||
|
||||
if [[ $result -eq 0 ]]; then
|
||||
echo " Status: SUCCESS ✓"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
return 0
|
||||
else
|
||||
echo " Status: FAILED ✗"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Execute main function
|
||||
main "$@"
|
||||
Executable
+197
@@ -0,0 +1,197 @@
|
||||
#!/bin/bash
|
||||
# Updates version tags in firmware configuration and JSON files
|
||||
# Used by: CI/CD pipelines for versioning builds
|
||||
# Usage: ./set_version.sh <version_tag> [--dev]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Constants
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
readonly USER_CONFIG="${PROJECT_ROOT}/main/User_config.h"
|
||||
readonly LATEST_VERSION_PROD="${SCRIPT_DIR}/latest_version.json"
|
||||
readonly LATEST_VERSION_DEV="${SCRIPT_DIR}/latest_version_dev.json"
|
||||
|
||||
# Load shared configuration (colors, logging functions, paths)
|
||||
if [[ -f "${SCRIPT_DIR}/ci_00_config.sh" ]]; then
|
||||
source "${SCRIPT_DIR}/ci_00_config.sh"
|
||||
else
|
||||
echo "ERROR: ci_00_config.sh not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Function to validate version tag
|
||||
validate_version() {
|
||||
local version="$1"
|
||||
|
||||
if [[ -z "$version" ]] || [[ "$version" == "version_tag" ]]; then
|
||||
log_error "Invalid version tag: '$version'"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "Version tag validated: $version"
|
||||
}
|
||||
|
||||
# Function to backup files
|
||||
backup_file() {
|
||||
local file="$1"
|
||||
|
||||
if [[ -f "$file" ]]; then
|
||||
cp "$file" "${file}.bak"
|
||||
log_info "Backed up: $file"
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to replace version in file
|
||||
replace_version() {
|
||||
local file="$1"
|
||||
local version="$2"
|
||||
|
||||
if [[ ! -f "$file" ]]; then
|
||||
log_warn "File not found: $file (skipping)"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Backup before modification
|
||||
backup_file "$file"
|
||||
|
||||
# Replace version_tag placeholder
|
||||
if sed -i "s/version_tag/${version}/g" "$file"; then
|
||||
log_info "Updated version in: $file"
|
||||
else
|
||||
log_error "Failed to update version in: $file"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to set production version
|
||||
set_production_version() {
|
||||
local version="$1"
|
||||
|
||||
log_info "Setting PRODUCTION version: $version"
|
||||
|
||||
replace_version "$USER_CONFIG" "$version" || return 1
|
||||
replace_version "$VERSION_JSON" "$version" || return 1
|
||||
}
|
||||
|
||||
# Function to set development version
|
||||
set_development_version() {
|
||||
local version="$1"
|
||||
|
||||
log_info "Setting DEVELOPMENT version: $version"
|
||||
|
||||
replace_version "$USER_CONFIG" "$version" || return 1
|
||||
replace_version "$VERSION_DEV_JSON" "$version" || return 1
|
||||
}
|
||||
|
||||
# Function to restore backups
|
||||
restore_backups() {
|
||||
log_warn "Restoring backups..."
|
||||
|
||||
for file in "$USER_CONFIG" "$VERSION_JSON" "$VERSION_DEV_JSON"; do
|
||||
if [[ -f "${file}.bak" ]]; then
|
||||
mv "${file}.bak" "$file"
|
||||
log_info "Restored: $file"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# Function to clean backups
|
||||
clean_backups() {
|
||||
for file in "$USER_CONFIG" "$VERSION_JSON" "$VERSION_DEV_JSON"; do
|
||||
if [[ -f "${file}.bak" ]]; then
|
||||
rm "${file}.bak"
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# Show usage
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 <version_tag> [OPTIONS]
|
||||
|
||||
Update version tags in firmware configuration files.
|
||||
|
||||
Arguments:
|
||||
version_tag Version string to inject (e.g., v1.2.3, abc123, dev-20230101)
|
||||
|
||||
Options:
|
||||
--dev Use development version files (latest_version_dev.json)
|
||||
--prod Use production version files (latest_version.json) [default]
|
||||
--help Show this help message
|
||||
|
||||
Examples:
|
||||
$0 v1.2.3 # Production release
|
||||
$0 abc123 --dev # Development build
|
||||
$0 \${{ github.sha }} --dev # CI/CD with commit SHA
|
||||
|
||||
EOF
|
||||
}
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
local version=""
|
||||
local is_dev=false
|
||||
|
||||
# Parse arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--dev)
|
||||
is_dev=true
|
||||
shift
|
||||
;;
|
||||
--prod)
|
||||
is_dev=false
|
||||
shift
|
||||
;;
|
||||
--help|-h)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
-*)
|
||||
log_error "Unknown option: $1"
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
*)
|
||||
version="$1"
|
||||
shift
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Validate version
|
||||
if [[ -z "$version" ]]; then
|
||||
log_error "Version tag is required"
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
validate_version "$version" || exit 1
|
||||
|
||||
# Change to project root
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Set version with error handling
|
||||
if [[ "$is_dev" == true ]]; then
|
||||
set_development_version "$version" || {
|
||||
restore_backups
|
||||
exit 1
|
||||
}
|
||||
else
|
||||
set_production_version "$version" || {
|
||||
restore_backups
|
||||
exit 1
|
||||
}
|
||||
fi
|
||||
|
||||
# Clean up backups on success
|
||||
clean_backups
|
||||
|
||||
log_info "Version update completed successfully"
|
||||
}
|
||||
|
||||
# Run main if executed directly
|
||||
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
|
||||
main "$@"
|
||||
fi
|
||||
Executable
+505
@@ -0,0 +1,505 @@
|
||||
#!/bin/bash
|
||||
# CI/CD Site/Documentation Builder
|
||||
# Builds and deploys VuePress documentation with version management
|
||||
# Usage: ./scripts/ci_site.sh [OPTIONS]
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# Constants
|
||||
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
readonly PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
readonly DOCS_DIR="${PROJECT_ROOT}/docs"
|
||||
readonly VUEPRESS_CONFIG="${DOCS_DIR}/.vuepress/config.js"
|
||||
readonly VUEPRESS_BUILD_DIR="${DOCS_DIR}/.vuepress/dist"
|
||||
|
||||
# Load shared configuration
|
||||
if [[ -f "${SCRIPT_DIR}/ci_00_config.sh" ]]; then
|
||||
source "${SCRIPT_DIR}/ci_00_config.sh"
|
||||
else
|
||||
echo "ERROR: ci_00_config.sh not found" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Final output directory for site
|
||||
readonly SITE_OUTPUT_DIR="${PROJECT_ROOT}/${SITE_DIR}"
|
||||
|
||||
# Default values
|
||||
MODE="prod"
|
||||
URL_PREFIX="/"
|
||||
CUSTOM_VERSION=""
|
||||
VERSION_SOURCE="release"
|
||||
PREVIEW=false
|
||||
GENERATE_WEBUPLOADER=true
|
||||
WEBUPLOADER_ARGS=""
|
||||
DEPLOY_DIR="."
|
||||
RUN_PAGESPEED=false
|
||||
PAGESPEED_URL="https://docs.openmqttgateway.com/"
|
||||
|
||||
# Function to check required tools
|
||||
check_requirements() {
|
||||
log_info "Checking required tools..."
|
||||
|
||||
local missing_tools=()
|
||||
|
||||
if ! command -v node >/dev/null 2>&1; then
|
||||
missing_tools+=("node")
|
||||
else
|
||||
local node_version
|
||||
node_version=$(node --version)
|
||||
log_info "✓ Node.js ${node_version} found"
|
||||
fi
|
||||
|
||||
if ! command -v npm >/dev/null 2>&1; then
|
||||
missing_tools+=("npm")
|
||||
else
|
||||
local npm_version
|
||||
npm_version=$(npm --version)
|
||||
log_info "✓ npm ${npm_version} found"
|
||||
fi
|
||||
|
||||
if ! command -v python3 >/dev/null 2>&1; then
|
||||
missing_tools+=("python3")
|
||||
else
|
||||
local python_version
|
||||
python_version=$(python3 --version | grep -oP '\d+\.\d+' || echo "unknown")
|
||||
log_info "✓ Python ${python_version} found"
|
||||
fi
|
||||
|
||||
if ! command -v pip3 >/dev/null 2>&1; then
|
||||
missing_tools+=("pip3")
|
||||
else
|
||||
local pip_version
|
||||
pip_version=$(pip3 --version | grep -oP '\d+\.\d+' || echo "unknown")
|
||||
log_info "✓ pip ${pip_version} found"
|
||||
fi
|
||||
|
||||
if [[ ${#missing_tools[@]} -gt 0 ]]; then
|
||||
log_error "Missing required tools: ${missing_tools[*]}"
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "All required tools are available"
|
||||
return 0
|
||||
}
|
||||
|
||||
# Function to install dependencies
|
||||
install_dependencies() {
|
||||
log_info "Installing dependencies..."
|
||||
|
||||
# Upgrade pip first (if pip module is available)
|
||||
if python3 -m pip --version >/dev/null 2>&1; then
|
||||
log_info "Upgrading pip..."
|
||||
python3 -m pip install --upgrade pip --quiet || {
|
||||
log_warn "Failed to upgrade pip, continuing with existing version..."
|
||||
}
|
||||
fi
|
||||
|
||||
# Install Python dependencies (without --user if in virtualenv)
|
||||
log_info "Installing Python dependencies..."
|
||||
pip3 install requests pandas markdown pytablereader tabulate || {
|
||||
log_error "Failed to install Python dependencies"
|
||||
return 1
|
||||
}
|
||||
|
||||
# Install Node dependencies
|
||||
log_info "Installing Node.js dependencies..."
|
||||
cd "${PROJECT_ROOT}"
|
||||
npm install --quiet || {
|
||||
log_error "Failed to install Node.js dependencies"
|
||||
return 1
|
||||
}
|
||||
|
||||
log_info "Dependencies installed successfully"
|
||||
}
|
||||
|
||||
# Function to download common config
|
||||
download_common_config() {
|
||||
log_info "Downloading common configuration..."
|
||||
|
||||
local config_url="https://www.theengs.io/commonConfig.js"
|
||||
local config_dest="${DOCS_DIR}/.vuepress/public/commonConfig.js"
|
||||
|
||||
mkdir -p "$(dirname "$config_dest")"
|
||||
|
||||
if curl -sSf -o "$config_dest" "$config_url"; then
|
||||
log_info "✓ Common config downloaded"
|
||||
else
|
||||
log_warn "Failed to download common config, continuing anyway..."
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to get version
|
||||
get_version() {
|
||||
local version=""
|
||||
|
||||
if [[ "$VERSION_SOURCE" == "custom" && -n "$CUSTOM_VERSION" ]]; then
|
||||
version="$CUSTOM_VERSION"
|
||||
log_info "Using custom version: $version"
|
||||
elif [[ "$VERSION_SOURCE" == "release" ]]; then
|
||||
# Try to get latest git tag (simulating GitHub release)
|
||||
if command -v git >/dev/null 2>&1 && [[ -d "${PROJECT_ROOT}/.git" ]]; then
|
||||
version=$(git describe --tags --abbrev=0 2>/dev/null || echo "development")
|
||||
log_info "Using release version from git: $version"
|
||||
else
|
||||
version="development"
|
||||
log_warn "Git not available, using version: $version"
|
||||
fi
|
||||
else
|
||||
# Auto-detect from git
|
||||
if command -v git >/dev/null 2>&1 && [[ -d "${PROJECT_ROOT}/.git" ]]; then
|
||||
version=$(git describe --tags --abbrev=0 2>/dev/null || echo "development")
|
||||
log_info "Using git version: $version"
|
||||
else
|
||||
version="development"
|
||||
log_warn "Git not available, using version: $version"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "$version"
|
||||
}
|
||||
|
||||
# Function to set version in config files
|
||||
set_version() {
|
||||
local version="$1"
|
||||
|
||||
log_info "Setting version: $version"
|
||||
|
||||
# Update VuePress config
|
||||
if [[ -f "$VUEPRESS_CONFIG" ]]; then
|
||||
sed -i "s|version_tag|${version}|g" "$VUEPRESS_CONFIG"
|
||||
fi
|
||||
|
||||
# Update version JSON file based on version source
|
||||
if [[ "$VERSION_SOURCE" == "custom" ]]; then
|
||||
# Custom version updates dev file
|
||||
local version_file="${SCRIPT_DIR}/latest_version_dev.json"
|
||||
if [[ -f "$version_file" ]]; then
|
||||
sed -i "s|version_tag|${version}|g" "$version_file"
|
||||
fi
|
||||
else
|
||||
# Release version updates production file
|
||||
local version_file="${SCRIPT_DIR}/latest_version.json"
|
||||
if [[ -f "$version_file" ]]; then
|
||||
sed -i "s|version_tag|${version}|g" "$version_file"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to set URL prefix (base path)
|
||||
set_url_prefix() {
|
||||
local url_prefix="$1"
|
||||
|
||||
if [[ "$url_prefix" != "/" ]]; then
|
||||
log_info "Setting URL prefix: $url_prefix"
|
||||
sed -i "s|base: '/'|base: '${url_prefix}'|g" "$VUEPRESS_CONFIG"
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to generate board documentation
|
||||
generate_board_docs() {
|
||||
log_info "Generating board documentation..."
|
||||
|
||||
local generator="${SCRIPT_DIR}/generate_board_docs.py"
|
||||
|
||||
if [[ -f "$generator" ]]; then
|
||||
cd "${PROJECT_ROOT}"
|
||||
python3 "$generator" || {
|
||||
log_warn "Board documentation generation failed, continuing..."
|
||||
}
|
||||
else
|
||||
log_warn "Board documentation generator not found, skipping..."
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to generate WebUploader manifest
|
||||
generate_webuploader() {
|
||||
if [[ "$GENERATE_WEBUPLOADER" != true ]]; then
|
||||
log_info "Skipping WebUploader generation"
|
||||
return 0
|
||||
fi
|
||||
|
||||
log_info "Generating WebUploader manifest..."
|
||||
|
||||
local generator="${SCRIPT_DIR}/gen_wu.py"
|
||||
|
||||
if [[ -f "$generator" ]]; then
|
||||
cd "${PROJECT_ROOT}"
|
||||
python3 "$generator" $WEBUPLOADER_ARGS || {
|
||||
log_warn "WebUploader generation failed, continuing..."
|
||||
}
|
||||
else
|
||||
log_warn "WebUploader generator not found, skipping..."
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to build documentation
|
||||
build_docs() {
|
||||
log_info "Building documentation..."
|
||||
|
||||
cd "${PROJECT_ROOT}"
|
||||
|
||||
# Set Node options for compatibility with newer Node.js versions
|
||||
export NODE_OPTIONS="--openssl-legacy-provider"
|
||||
|
||||
npm run docs:build || {
|
||||
log_error "Documentation build failed"
|
||||
return 1
|
||||
}
|
||||
|
||||
if [[ -d "$VUEPRESS_BUILD_DIR" ]]; then
|
||||
log_info "✓ Documentation built successfully"
|
||||
|
||||
# Copy to centralized output directory
|
||||
log_info "Copying site to: $SITE_OUTPUT_DIR"
|
||||
mkdir -p "$SITE_OUTPUT_DIR"
|
||||
rm -rf "${SITE_OUTPUT_DIR}"/*
|
||||
cp -r "${VUEPRESS_BUILD_DIR}"/* "${SITE_OUTPUT_DIR}/"
|
||||
|
||||
log_info " VuePress output: $VUEPRESS_BUILD_DIR"
|
||||
log_info " Final output: $SITE_OUTPUT_DIR"
|
||||
else
|
||||
log_error "Build output directory not found: $VUEPRESS_BUILD_DIR"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Function to preview documentation
|
||||
preview_docs() {
|
||||
log_info "Starting documentation preview..."
|
||||
|
||||
if [[ ! -d "$SITE_OUTPUT_DIR" ]]; then
|
||||
log_error "Build output not found. Run build first."
|
||||
return 1
|
||||
fi
|
||||
|
||||
log_info "Preview server starting at: http://localhost:8080"
|
||||
log_info "Press Ctrl+C to stop"
|
||||
|
||||
cd "$SITE_OUTPUT_DIR"
|
||||
python3 -m http.server 8080
|
||||
}
|
||||
|
||||
# Show usage
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 [OPTIONS]
|
||||
|
||||
Build and deploy OpenMQTTGateway documentation site using VuePress.
|
||||
|
||||
This script handles the complete documentation build pipeline including:
|
||||
- Installing Node.js and Python dependencies
|
||||
- Downloading shared configuration from theengs.io
|
||||
- Version management (from git tags or custom version)
|
||||
- Generating board documentation from environments
|
||||
- Creating WebUploader manifest for firmware updates
|
||||
- Building VuePress static site
|
||||
- Optional local preview or deployment to GitHub Pages
|
||||
|
||||
OPTIONS:
|
||||
--mode MODE
|
||||
Build mode: 'prod' or 'dev' [default: prod]
|
||||
- prod: Production documentation with release version
|
||||
- dev: Development documentation with custom version tag
|
||||
|
||||
--url-prefix PATH
|
||||
URL prefix for documentation routing (VuePress base path) [default: /]
|
||||
Controls how URLs are generated in the built site.
|
||||
Examples:
|
||||
--url-prefix / # Production site at root URL
|
||||
--url-prefix /dev/ # Development site at /dev/ URL path
|
||||
Note: Should match --deploy-dir for correct link generation
|
||||
|
||||
--custom-version TAG
|
||||
Override version tag displayed in documentation
|
||||
Example: --custom-version "v1.8.0-beta"
|
||||
Note: Automatically sets --version-source to 'custom'
|
||||
|
||||
--version-source SOURCE
|
||||
Source for version information: 'release' or 'custom' [default: release]
|
||||
- release: Use latest git tag from repository
|
||||
- custom: Use version from --custom-version parameter
|
||||
|
||||
--preview
|
||||
Start local HTTP server to preview built documentation
|
||||
Server will run at http://localhost:8080
|
||||
Press Ctrl+C to stop the preview server
|
||||
|
||||
--deploy-dir DIR
|
||||
Deployment directory on GitHub Pages [default: .]
|
||||
Controls where files are copied in the gh-pages branch.
|
||||
Examples:
|
||||
--deploy-dir . # Deploy to root of gh-pages branch
|
||||
--deploy-dir dev # Deploy to dev/ folder in gh-pages
|
||||
Note: Should match --url-prefix for correct site structure
|
||||
|
||||
--no-webuploader
|
||||
Skip WebUploader manifest generation
|
||||
By default, generates manifest for web-based firmware updates
|
||||
|
||||
--webuploader-args ARGS
|
||||
Additional arguments passed to gen_wu.py script
|
||||
Example: --webuploader-args "--dev"
|
||||
|
||||
--run-pagespeed
|
||||
Run Google PageSpeed Insights after deployment
|
||||
Requires APIKEY to be configured in workflow
|
||||
|
||||
--pagespeed-url URL
|
||||
URL to test with PageSpeed Insights
|
||||
Default: https://docs.openmqttgateway.com/
|
||||
|
||||
--help
|
||||
Show this help message and exit
|
||||
|
||||
EXAMPLES:
|
||||
|
||||
# Build production documentation with latest release version
|
||||
$0 --mode prod
|
||||
|
||||
# Build and preview development documentation locally
|
||||
$0 --mode dev --url-prefix /dev/ --preview
|
||||
|
||||
# Build with custom version tag
|
||||
$0 --custom-version "v1.8.0-beta" --version-source custom
|
||||
|
||||
# Build without WebUploader manifest
|
||||
$0 --mode prod --no-webuploader
|
||||
|
||||
# Build for dev environment with custom WebUploader args
|
||||
$0 --mode dev --url-prefix /dev/ --deploy-dir dev --webuploader-args "--dev"
|
||||
|
||||
WORKFLOW:
|
||||
1. Check requirements (Node.js, npm, Python)
|
||||
2. Install dependencies (npm packages, Python libraries)
|
||||
3. Download common configuration from theengs.io
|
||||
4. Determine version (from git tags or custom)
|
||||
5. Generate board documentation from PlatformIO environments
|
||||
6. Generate WebUploader manifest (optional)
|
||||
7. Build VuePress static site
|
||||
8. Preview (optional)
|
||||
|
||||
OUTPUT:
|
||||
Built documentation will be in: generated/site/
|
||||
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
# Parse command line arguments
|
||||
parse_args() {
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--mode)
|
||||
MODE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--url-prefix|--base-path)
|
||||
URL_PREFIX="$2"
|
||||
shift 2
|
||||
;;
|
||||
--deploy-dir|--destination-dir)
|
||||
DEPLOY_DIR="$2"
|
||||
shift 2
|
||||
;;
|
||||
--custom-version)
|
||||
CUSTOM_VERSION="$2"
|
||||
VERSION_SOURCE="custom"
|
||||
shift 2
|
||||
;;
|
||||
--version-source)
|
||||
VERSION_SOURCE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--preview)
|
||||
PREVIEW=true
|
||||
shift
|
||||
;;
|
||||
|
||||
--no-webuploader)
|
||||
GENERATE_WEBUPLOADER=false
|
||||
shift
|
||||
;;
|
||||
--webuploader-args)
|
||||
WEBUPLOADER_ARGS="$2"
|
||||
shift 2
|
||||
;;
|
||||
--run-pagespeed)
|
||||
RUN_PAGESPEED=true
|
||||
shift
|
||||
;;
|
||||
--pagespeed-url)
|
||||
PAGESPEED_URL="$2"
|
||||
shift 2
|
||||
;;
|
||||
--help)
|
||||
usage
|
||||
;;
|
||||
*)
|
||||
log_error "Unknown option: $1"
|
||||
usage
|
||||
;;
|
||||
esac
|
||||
done
|
||||
}
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
local start_time
|
||||
start_time=$(date +%s)
|
||||
|
||||
parse_args "$@"
|
||||
|
||||
log_info "Starting site build pipeline..."
|
||||
log_info "Mode: $MODE"
|
||||
log_info "URL prefix: $URL_PREFIX"
|
||||
log_info "Deploy directory: $DEPLOY_DIR"
|
||||
|
||||
# Check requirements
|
||||
check_requirements || exit 1
|
||||
|
||||
# Install dependencies
|
||||
install_dependencies || exit 1
|
||||
|
||||
# Download common config
|
||||
download_common_config
|
||||
|
||||
# Get and set version
|
||||
local version
|
||||
version=$(get_version)
|
||||
set_version "$version"
|
||||
|
||||
# Set URL prefix
|
||||
set_url_prefix "$URL_PREFIX"
|
||||
|
||||
# Generate board documentation
|
||||
generate_board_docs
|
||||
|
||||
# Generate WebUploader manifest
|
||||
generate_webuploader
|
||||
|
||||
# Build documentation
|
||||
build_docs || exit 1
|
||||
|
||||
# Preview if requested
|
||||
if [[ "$PREVIEW" == true ]]; then
|
||||
preview_docs
|
||||
fi
|
||||
|
||||
local end_time
|
||||
end_time=$(date +%s)
|
||||
local duration=$((end_time - start_time))
|
||||
|
||||
echo ""
|
||||
echo "╔════════════════════════════════════════╗"
|
||||
echo "║ Site Build Summary ║"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
echo " Mode: $MODE"
|
||||
echo " Version: $version"
|
||||
echo " Duration: ${duration}s"
|
||||
echo " Output: $SITE_OUTPUT_DIR"
|
||||
echo " Status: SUCCESS ✓"
|
||||
echo "╚════════════════════════════════════════╝"
|
||||
}
|
||||
|
||||
# Execute main function
|
||||
main "$@"
|
||||
@@ -1,3 +1,5 @@
|
||||
# Common templates and constants for web installer manifest generation
|
||||
# Used by: scripts/gen_wu.py
|
||||
import string
|
||||
|
||||
mf_temp32 = string.Template('''{
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
# Compresses firmware binaries with gzip for OTA updates during build
|
||||
# Used by: PlatformIO environments (optional, commented in environments.ini)
|
||||
import gzip
|
||||
import shutil
|
||||
import os
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
# Creates web installer manifests for ESP Web Tools firmware installation
|
||||
# Used by: .github/workflows/task-docs.yml
|
||||
import os
|
||||
import requests
|
||||
import json
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
# Generates board documentation table from platformio.ini environments
|
||||
# Used by: .github/workflows/task-docs.yml
|
||||
import pytablereader as ptr
|
||||
import pandas as pd
|
||||
import os
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
#!/bin/bash
|
||||
# Prepares firmware binaries and libraries for GitHub release deployment
|
||||
# Used by: .github/workflows/release.yml
|
||||
set -e
|
||||
echo "renaming bin files with the environment name"
|
||||
rename -v 's:/:-:g' .pio/build/*/*.bin
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
|
||||
# Replaces BLE library with custom version during PlatformIO build
|
||||
# Used by: Currently unused (utility script for BLE library replacement)
|
||||
import shutil
|
||||
import os
|
||||
import hashlib
|
||||
|
||||
Reference in new issue
Block a user