When the network drops at runtime and the WiFi stack is wedged (e.g. the
ESP32 driver getting stuck after a long uptime, see #2276), the main loop
kept calling wifi_reconnect_bypass() forever with no recovery: the device
stayed stale until a manual reboot. The reboot watchdog only existed in the
ESPWifiManualSetup path, which the default WiFiManager build doesn't compile.
Count consecutive reconnection failures in the runtime loop and ESPRestart(2)
once they exceed maxRetryWatchDog, mirroring the existing manual-setup
watchdog. Gated behind the new WifiReconnectWatchDog flag (default on) so
environments where an unexpected reboot is undesirable can opt out; disabled
for theengs-plug to avoid interrupting the relay state.
Co-authored-by: Claude <noreply@anthropic.com>
Home Assistant rejects an MQTT discovery payload that carries `options`
(an enum sensor) together with `state_class` or `unit_of_measurement`:
these fields are mutually exclusive. createDiscovery() was emitting all
three for Mi Body Composition Scale (XMTZC0x) weighing_mode/unit sensors,
so HA discarded the discovery message.
Gate the stat_cla and unit_of_meas writes on enum_options == nullptr so
enum sensors no longer carry the conflicting fields.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
iBeacon uuid/mfid (and any decoder property with unit "hex") were built
as HA sensors with state_class "measurement" and unit "hex". Home
Assistant rejects a measurement sensor whose value is non-numeric, so it
logged "non-numeric value" errors on every iBeacon decode. Route "hex"
units through the same path as "string" units: plain text sensor, no
state_class, no unit_of_measurement.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
immediateBTAction() sets BTProcessLock=true and stops the scan before an
outbound BLE action, and resets it on the success path and the
"BLE busy" path — but the branch where semaphoreCreateOrUpdateDevice
can't be taken left it true. Under scan contention that wedges the BLE
scanner/connect subsystem until a restart (same BTProcessLock-mishandling
family as the long-uptime NimBLE scanner hang). Reset it on that branch
too.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [SYS] Don't clobber MQTT/portal defaults with empty form fields
The /wifisave handler unconditionally strcpy'd every form value into the
persisted config, even when WiFiManager surfaced an empty string for a
field the caller omitted. A partial provisioning POST (e.g. only s/p)
silently wiped the compile-time MQTT defaults so the DUT joined WiFi but
never reached the broker.
Mirror the mqtt_pass sentinel guard with strlen() > 0 checks for the
remaining textual fields (server, port, user, topic, name, ota). The
boolean checkboxes and certificate fields already had non-empty guards.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [DOCS] Document /wifisave provisioning portal API
Add a reference page enumerating the HTTP form fields accepted by the
on-device configuration portal so companion apps and provisioning scripts
can integrate without reading firmware source. Documents the empty-field
ignore semantics introduced in the previous commit and the AP-teardown
response race that callers must tolerate.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [SYS] Also length-guard the MQTT password sentinel branch
The sentinel-only check (form-value != MQTT_PASS) was meant to protect
the stored password against accidental re-saves of the form default,
but it also persists an empty string from a partial / programmatic POST
that omits the `pass` field — empty differs from the sentinel, so the
broker password gets wiped. Add a strlen() > 0 precondition so partial
POSTs preserve the existing password.
Verified on the HIL bench (test_sys/test_wifimanager_portal_partial):
without this commit the DUT joined WiFi but never reached the broker;
with it, LWT-online lands within the reconnect budget.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [DOCS] Reflect length-guard in /wifisave mqtt_pass semantics
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [BT] Default BLE AES key empty, skip decryption without a key
Shipping a non-empty default key caused fresh installs to attempt
decryption of every encrypted PVVX/BTHome v2/Victron advertisement with
the placeholder string from User_config.h. Frames that didn't happen to
share that key were silently dropped at the CCM auth step, masking
otherwise-decodable devices (notably Victron MPPT) until users found and
entered a per-MAC key.
Initialise ble_aes[] to "" and treat an empty default as "no default
configured" in the decryption path - mirroring Theengs Gateway, which
has no default and skips frames whose MAC has no bindkey. Per-MAC keys
in ble_aes_keys still take precedence; the BLE_AES macro remains as the
WebUI placeholder hint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [BT] clang-format: move BLE_AES default comment above #define
clang-format-9 rejected the trailing-comment continuation on the empty
BLE_AES default line. Move the explanation to a leading // block above
the #ifndef so the directive line is comment-free.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [BT] Use THEENGS_LOG_TRACE for missing-AES-key skip log
Previous commit referenced THEENGS_LOG_DEBUG, which OMG does not define
(only VERBOSE/TRACE/NOTICE/WARNING/ERROR exist), breaking every BLE
firmware build. Switch to THEENGS_LOG_TRACE — same level as the adjacent
"Custom AES key" / "Default AES key" trace lines.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Switch Ethernet RMII clock from GPIO16 to GPIO0 output mode,
freeing GPIO16 for PSRAM chip select (SPICS1) on WROVER-IE
- Enable PSRAM with QIO flash mode and cache workaround flags
- Add PHY pre-power support for ETH_CLOCK_GPIO0_IN mode
- Report PSRAM status and free memory in SYS MQTT topic
Requires Theengs lib-builder update: remove LIBTIME from IRAM
cache workaround (CONFIG_SPIRAM_CACHE_LIBTIME_IN_IRAM=n) to
free ~3.5KB IRAM for the -mfix-esp32-psram-cache-issue flag.
Tested on Theengs Bridge v1.1: 4MB PSRAM detected, Ethernet
100Mbps on GPIO0, BLE scanning nominal.
Co-authored-by: Florian <1technophile@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
createDiscovery sits at the bottom of the discovery call chain
(loop → launchBTDiscovery → <device>Discovery → createDiscoveryFromList
(9-col, VLA) → createDiscoveryFromList(13-col) → here). Keeping
2×1024 B StaticJsonDocuments on the stack burned loopTask headroom
down to ~100 B free under a full BLE + WebUI load. Moving the two
buffers to the heap drops that 2 KB off the stack peak.
Measured over the full test_perf + test_webui suite on
esp32dev-ble-hil-fastsys (dev HEAD):
freestck_min_bytes: 292 → 2324 B (+2032, target was ≥1500)
minmem_soak_bytes: 18200 → 20652 B (+2452)
all 17 tests still pass
The docs are function-scoped, so net heap usage is unchanged long-term
(free-on-return), and the soak heap floor actually improved.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ble_script is defined as "" (empty string). Passing it to
server.sendContent("") signals end of HTTP chunked transfer,
closing the response prematurely. All subsequent chunks then
fail with errno 104 "Connection reset by peer".
Guard the call with a length check so the empty string is
never sent as a chunk terminator.
Co-authored-by: Florian <1technophile@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Expand the BLE config page from encryption-only (BLEDecryptor) to a
comprehensive settings page available for all ZgatewayBT builds, covering
scan, publish, presence, external decoder, and encryption settings.
Add MQTT Discovery checkbox to the gateway configuration page.
Co-authored-by: Florian <1technophile@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Refactor GitHub Actions workflows for build, documentation, and linting
- Consolidated build logic into reusable workflows (`task-build.yml` and `task-docs.yml`) to reduce duplication across multiple workflows.
- Introduced `environments.json` to centralize the list of PlatformIO build environments, improving maintainability and clarity.
- Updated `build.yml` and `build_and_docs_to_dev.yml` to utilize the new reusable workflows and environment definitions.
- Enhanced `release.yml` to streamline the release process and integrate documentation generation.
- Created reusable linting workflow (`task-lint.yml`) to standardize code formatting checks across the repository.
- Simplified manual documentation workflow by leveraging the new reusable documentation workflow.
- Improved artifact management and retention policies across workflows.
- Updated dependencies and versions in workflows to ensure compatibility and performance.
CI/CD pipeline agnostic of Workflow Engine and integrated on github actions
- Implemented ci.sh for orchestrating the complete build pipeline.
- Created ci_00_config.sh for centralized configuration of build scripts.
- Created ci_build_firmware.sh for building firmware for specified PlatformIO environments.
- Created ci_prepare_artifacts.sh for preparing firmware artifacts for upload or deployment.
- Created ci_set_version.sh for updating version tags in firmware configuration files.
- Created ci_build.sh to orchestrate the complete build pipeline.
- Created ci_qa.sh for code linting and formatting checks using clang-format.
- Created ci_site.sh for building and deploying VuePress documentation with version management.
- Implemented checks for required tools and dependencies in the new scripts.
- Improved internal scripts for better error handling and logging.
UPDATE the web installer manifest generation and update documentation structure
- Enhanced ci_list-env.sh to list environments from a JSON file.
- Replaced common_wu.py and gen_wu.py scripts with new npm scripts for site generation and previewing on docsgen/gen_wu.js
- Replaced generate_board_docs.py with docsgen/generated_board_docs.js
- Added new npm scripts for integration of site generation on build phase.
- Created preview_site.js to serve locally generated site over HTTPS with improved error handling.
- Added new CI environments for CI builds in environments.json.
- Deleted lint.yml as part of workflow cleanup.
- Enhanced task-build.yml to include linting as a job and added support for specifying PlatformIO version.
- Improved task-docs.yml to handle versioning more effectively and added clean option.
Enhance documentation
- ADD CLEAR Mark of development version of site
- Updated README.md to include detailed workflow dependencies and relationships using mermaid diagrams.
- Improved development.md with a quick checklist for contributors and clarified the code style guide.
- Enhanced quick_start.md with tips for contributors and streamlined the workflow explanation.
LINT FIX
- Refined User_config.h for better formatting consistency.
- Adjusted blufi.cpp and gatewayBT.cpp for improved code readability and consistency in formatting.
- Updated gatewaySERIAL.cpp and mqttDiscovery.cpp to enhance logging error messages.
- Improved sensorDS1820.cpp for better logging of device information.
Add security scan workflows for vulnerability detection
Add SBOM generation and upload to release workflow; update security scan summary handling
Add shellcheck suppor + FIX shellcheck warning
Enhance documentation for CI/CD scripts and workflows, adding details for security scanning and SBOM generation processes
Fix formatting and alignment in BLE connection handling
Reviewed the full web board presentation and the ESP32 web upload. The project uses a modern pattern where data is divided from the presentation layer.
- Removed the `generate_board_docs` script.
- Updated the `gen_wu` script in order to generate `boards-info.json`: the fail that containe all information about the configuration
- Created and isolate the file `boards-info.js` to streamline the parsing of PlatformIO dependencies, modules, environments and improve the handling of library information.
- Introduced vuepress component `BoardEnvironmentTable.vue` that render `boards-info.json` as UI card component
- Introduced vuepress component `FlashEnvironmentSelector.vue` that render a selectred environment from `boards-info.json` and provide esp-web-upload feature on it
- Introduced a new board page `board-selector.md` for improved firmware selection.
- Updated `web-install.md` to enhance the firmware upload process, including a new board environment table.
- Enhanced custom descriptions in `environments.ini` to include HTML links for better user guidance and board image link
Add CC1101 initialization improvements and logging enhancements
Add installation step for PlatformIO dependencies in documentation workflow
Remove ci_set_version.sh script and associated versioning functionality
* Fix comment provisined
Fix PlatformIO version input reference in documentation workflow
Remove outdated Squeezelite-ESP32 installer documentation
HASS_UNIT_PPM is defined in config_mqttDiscovery.h but was missing from
the availableHASSUnits validation array in mqttDiscovery.cpp. This caused
unit_of_measurement to be silently dropped for CO2 sensors (like the
Govee H5140) during MQTT discovery, resulting in Home Assistant warnings
about invalid units.
* Fix: Set WiFi hostname to gateway name
Add WiFi.setHostname() calls to ensure the ESP's WiFi hostname matches
the configured gateway name instead of using the default ESP hostname.
- Add wifiManager.setHostname(gateway_name) in setupWiFiManager() for
WiFiManager-based setups
- Add WiFi.setHostname(gateway_name) in setupWiFiFromBuild() for manual
WiFi setup configurations (ESPWifiManualSetup)
This matches the existing behavior for Ethernet connections where
ETH.setHostname(gateway_name) is already called.
Fixes#2150
Co-authored-by: Florian <1technophile@users.noreply.github.com>
* Change WiFi mode position
* Fix setHostname when using WiFi Manager
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Florian <1technophile@users.noreply.github.com>
The bug was a race condition where saved configuration was applied
to the LoRa hardware BEFORE LoRa.begin() was called, causing settings
like syncword to be lost after reset/power cycle.
Changes:
- Created LORAConfig_apply() to centralize hardware configuration
- Removed hardware application from LORAConfig_fromJson() (now only
updates struct values)
- Call LORAConfig_apply() after LoRa.begin() in setupLORA()
- Call LORAConfig_apply() after LORAConfig_fromJson() in runtime
handlers (MQTT/WebUI) since hardware is already initialized
- Fixed syncword logging to properly show 'unchanged' vs 'changed'
Fixes#2270
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Florian <1technophile@users.noreply.github.com>
When a saved RF configuration exists in NVS, loadFromStorage() was loading
the config but not calling iRFReceiver.enable(), which meant initCC1101()
was never invoked during boot. This caused the CC1101 to not receive signals
until the user manually toggled the RF receiver via the WebUI.
This fix adds a reinitReceiver parameter to loadFromStorage() that controls
whether to disable/enable the receiver after loading config. By default it's
true (for boot-time initialization), but loadFromMessage() passes false to
avoid double initialization when loading config from MQTT messages.
Fixes#2264
Co-authored-by: Florian <1technophile@users.noreply.github.com>
Co-authored-by: Odyno <odyno@users.noreply.github.com>
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
- Introduced RFConfiguration class to encapsulate RF settings and operations.
- Replaced direct usage of RFConfig structure with iRFConfig instance across multiple files.
- Updated frequency handling in actuatorSomfy, gatewayPilight, gatewayRF, and gatewayRF2 to use iRFConfig.
- Modified webUI to interact with iRFConfig for RF settings management.
- Removed deprecated RFConfig structure and related functions.
- Enhanced JSON handling for RF configuration loading and saving.
- Improved logging for RF configuration operations.
Fix duplicate W and keep units and classes in flash
Use macro for discovery classes and units
[BREAKING] change gateway sensor unique id
module - parameter
Standardize the use of createDiscoveryFromList
Co-authored-by: Florian <1technophile@users.noreply.github.com>
* Changing WebUI to include display device name, and change it to select drop down rather than checkbox
* Fix mqttDiscovery to require WebUI and ESP32 for displayDeviceName
* Fix mqttDiscovery to require WebUI and ESP32 and ESP8266 for displayDeviceName and ForceDeviceName
* Changing WebUI to include display device name, and change it to select drop down rather than checkbox
* Fixes for WebUI and BT for supporting custom setting Display name
* Fixes for WebUI and BT for supporting custom setting Display name
* Move DISPLAY_DEVICE_NAME to User_config.h
* Update docs to include change for Display temperature
* Update docs to include change for Display temperature
* Fix minor cosmetic bug where devices were not linking in HA to the gateway using via_device as it should be the gateway mac address not name
* Add support for decrypting BTHome v2 frames
* Add support for decrypting BTHome v2 frames
* Add support for decrypting BTHome v2 frames
* BTHome fix issue with theengs-plug
* BTHome fix issue with theengs-plug
* Adding support for all BLE encrypted methods, support in UI and gatewayBT for specific MACAddress AES Keys
* Fix lint
* Fix build issue with theengs-bridge-v11 and esp32dev-all-test and revert the documentation to Units of measurement displayed
* Revert docs
* Revert displayDeviceName and Units of measurement
* Revert displayDeviceName and Units of measurement
* Revert displayDeviceName and Units of measurement
* Revert minor typo
* Revert minor typo
* Revert minor typo
* Bug in Victron as nonce should be 16 bytes
* Shortened the client side javascript for BLE key validation that is commented out due to image constrains on theengs-bridge-v11
* blufi status
* Make the manufacturer ID configurable at build time.
* fix missing break.
* Update blufi gateway state when changed
---------
Co-authored-by: h2zero <powellperalata@gmail.com>
* Refactor announceDeviceTrigger function to improve parameter handling and documentation for optional device information
Fix null pointer references in RFtoMQTTdiscovery and update logging levels for RF signal handling
Refactor RF to MQTT discovery functions for improved clarity and parameter handling
* Update documentation for RF auto discovery and MQTT integration, correcting terminology and enhancing clarity
* Refactor enableRFReceive function to accept parameters for frequency and GPIOs, enhancing flexibility
Removal of openHAB specific MQTT discovery patch "ohdisc", as it is no longer required with openHAB 4.3 and later, and with openHAB before 4.3 MQTT discovery never was fully implemented anyway.
