chriz/OpenMQTT-Gateway
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

[SYS] Multiple connections management (#1947)

Browse Source 
Enable the management of different connections at runtime with a set of certs for each.
Add parameters to the WiFi Manager onboarding, client certificates, and certificate validation
This commit is contained in:
Florian authored and GitHub committed 2024-05-20 09:30:49 -05:00
1 parent 743083cbfe
commit f98474bed0
14 files changed
+808 -448
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/build.yml
-1
View File
@@ -62,7 +62,6 @@ jobs:
- "nodemcuv2-rf-cc1101"
- "nodemcuv2-somfy-cc1101"
- "manual-wifi-test"
- "nodemcuv2-mqtt-fw-test"
- "rf-wifi-gateway"
- "nodemcuv2-rf2"
- "nodemcuv2-rf2-cc1101"
docs/upload/advanced-configuration.md
+13 -5
View File
@@ -10,16 +10,17 @@ The configuration of the broker is not covered here, you should look into the do
### Prerequisites
The MQTT broker is configured for TLS and you have access to the CA certificate which was used to sign the MQTT broker certificate.
You are using ESP8266 or ESP32, for other boards TLS is not supported.
You are using ESP8266 or ESP32.
### Configure secure connection in the gateway
To enable the secure connection and use TLS set the `#define MQTT_DEFAULT_SECURE` to true.
To enable the secure connection and use TLS set the `#define MQTT_DEFAULT_SECURE` to true at build time, or the `mqtt secure` parameter with WiFi Manager or `mqtt_secure` with MQTT.
Set `MQTT_SERVER` to the Common Name (CN) of the certificate of the broker.
This can be the hostname or the IP of the broker.
The server identity can be verified against a certificate or not, if you don't want to use a certicate to verify the server you can uncheck the option `validate cert` with WiFi Manager or set `mqtt_validate` to `false` with MQTT
The CA certificate should be in PEM ascii format.
If your CA certificate is not in the correct format or you don't know the format, use `openssl` to convert the certificate to the correct format.
In `User_config.h` replace the `...` with the content of your certificate which is between the `-----BEGIN CERTIFICATE-----` and `-----END CERTIFICATE-----` lines:
At build time, in `main/certs/default_server_cert.h` replace the `...` with the content of your certificate which is between the `-----BEGIN CERTIFICATE-----` and `-----END CERTIFICATE-----` lines:
```cpp
const char* certificate CERT_ATTRIBUTE = R"EOF("
-----BEGIN CERTIFICATE-----
@@ -28,7 +29,14 @@ const char* certificate CERT_ATTRIBUTE = R"EOF("
")EOF";
```
You can know compile and upload to your board and the gateway should connect with TLS to your broker.
With WiFi Manager copy your certificate from `-----BEGIN CERTIFICATE-----` to `-----END CERTIFICATE-----` (included) and paste it into the input field:
```
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
```
This process can also be used for the other certificates, OTA, client key, client certificate if necessary.
## Add the received "value" at the end of the topic
For the gateways that publish a "value" parameter on the json (RF, IR...), it is possible to concatenate this parameter at the end of the topic.
docs/upload/portal.md
+4 -2
View File
@@ -29,8 +29,6 @@ If the board is connected by ethernet, the Wifi and password can be empty. If yo
* Set your MQTT Server IP or domain name (for Home Assistant you can enter `homeassistant.local` if your server is configured with mDNS)
* Set your MQTT Server Port (default: 1883)
* Set the MQTT secure connection box to select whether or not the connection should be secure
* Copy/paste the MQTT server certificate in the MQTT server cert box (only required if using a secure connection), be sure to include the `-----BEGIN CERTIFICATE-----` and `-----END CERTIFICATE-----` markers
* Set your MQTT Server username (optional, 64 characters maximum)
* Set your MQTT Server password (optional, 64 characters maximum)
* Set your MQTT base topic if you need to change it (you must keep the / at the end) (default: home/, 64 characters maximum)
@@ -39,6 +37,10 @@ If the board is connected by ethernet, the Wifi and password can be empty. If yo
* Local and remote OTA
* WebUI login (login:"admin", password: "")
* WiFi Manager Access Point password after initial config
* Set the MQTT secure connection box to select whether or not the connection should be secure
* Set the certificate validation to select wether or not you want to validate the server identity with a certificate
* Copy/paste the MQTT server certificate in the MQTT server cert box (only required if using a secure connection), be sure to include the `-----BEGIN CERTIFICATE-----` and `-----END CERTIFICATE-----` markers
* Add your certificates
* Click on save
![WiFi manager save](../img/OpenMQTTGateway_Wifi_Manager_save.png)
docs/use/gateway.md
+100 -34
View File
@@ -69,9 +69,11 @@ mosquitto_pub -t "home/OpenMQTTGateway/commands/MQTTtoSYS/config" -m
"mqtt_pass": "password",
"mqtt_server": "host",
"mqtt_port": "port",
"mqtt_secure": "false"
"mqtt_validate": false,
"mqtt_secure": false
}'
```
::: tip INFO
By default this function is not available on the pre built binary of RFBridge, in order to have less code size and enable to have OTA update working properly. So as to enable it remove from the rf bridge env:
```
@@ -80,7 +82,6 @@ build_flags = '-UMQTTsetMQTT'
:::
::: tip
Server, port, and secure_flag are only required if changing connection to another broker.
If the new connection fails the gateway will fallback to the previous connection.
:::
@@ -96,13 +97,13 @@ mosquitto_pub -t "home/OpenMQTTGateway/commands/MQTTtoSYS/config" -m
This will change the subscribed and published mqtt_topic/gateway_name that the gateway uses. No parameters are mandatory, the current topic or gateway name will be used if not supplied.
:::
## Switching brokers and using self signed and client certificates
## Switching brokers and using signed and client certificates
In the `user_config.h` file it is possible to specify multiple MQTT brokers and client certificates. These are commonly self signed and are supported by defining `MQTT_SECURE_SELF_SIGNED` as true or 1.
In the `user_config.h` file it is possible to specify multiple MQTT brokers and client certificates.
Additionally, support for multiple brokers and client certificates has been added. To use this, it is required that the server certificate, client certificate, and client key are provided as their own constant string value as demonstrated in the file.
To add more than one broker and switch between them it is necessary to provide all of the relevant certificates/keys and add their respective variable names in the `certs_array` structure, as shown in `user_config.h`, and changing the array size to the number of different connections -1.
To add more than one broker and switch between them it is necessary to provide all of the relevant certificates/keys and add their respective variable names in the `cnt_parameters_array` structure, as shown in `user_config.h`..
To switch between these servers with an MQTT command message, the format is as follows:
To switch between these connections with an MQTT command message, the format is as follows:
```
mosquitto_pub -t "home/OpenMQTTGateway/commands/MQTTtoSYS/config" -m
'{
@@ -110,14 +111,89 @@ mosquitto_pub -t "home/OpenMQTTGateway/commands/MQTTtoSYS/config" -m
"mqtt_pass": "password",
"mqtt_server": "host",
"mqtt_port": "port",
"mqtt_secure": "true",
"mqtt_cert_index":0
"mqtt_secure": true,
"mqtt_validate": true,
"cnt_index":1,
"save_cnt": true
}'
```
::: tip
The `mqtt_cert_index` value corresponds to the 0 to X index of the `certs_array` in `user_config.h`.
The `cnt_index` value corresponds to the 0 to 2 index of the `cnt_parameters_array` in `user_config.h`.
0 being the default index, containing the onboarding parameters.
:::
To read the connection parameters:
```
mosquitto_pub -t "home/OpenMQTTGateway/commands/MQTTtoSYS/config" -m
'{
"cnt_index":1,
"read_cnt": true
}'
```
To test a connection change without saving:
```
mosquitto_pub -t "home/OpenMQTTGateway/commands/MQTTtoSYS/config" -m
'{
"cnt_index":1,
"test_cnt": true
}'
```
## Saving/Loading connection parameters/certificates at runtime
This chapter details the process for managing certificates/connections parameters used for secure MQTT communication with OpenMQTTGateway
### Storing and Loading Certificates
* Flash Memory Storage:
Certificates can be saved to the flash memory using specific indices. Valid indices for storing certificates are 1 and 2, as 0 is reserved for the default certificate.
* RAM Memory Loading:
Certificates can be loaded from RAM, where valid indices range from 0 to 2. The device publishes a hash of the certificate to the broker to verify its identity. If the connection using the current certificate fails, the device will revert to the previous certificate.
### Use Case: Changing a Group of Certificates
When updating certificates, follow these steps to ensure that the new certificates are correctly loaded and used:
1. Push Certificates via MQTT:
Send the new certificates one by one through MQTT, using indices 1 or 2. Replace newline characters (\n) in the certificates with spaces.
```json
{
"cnt_index": 1,
"mqtt_server_cert": "-----BEGIN CERTIFICATE----- MIIDQTCC----END CERTIFICATE-----"
}
```
Accepted certificates are:
* `mqtt_server_cert`
* `mqtt_client_cert`
* `mqtt_client_key`
* `ota_server_cert`
2. Verify Certificates in RAM:
After pushing the certificates, verify that they have been correctly loaded into RAM.
```json
{
"cnt_index": 1,
"read_cnt": true
}
```
3. Test and Save Certificates:
Once verification is complete, test the connection using the new certificates. If the connection is successful, send the command to save the certificates to flash.
```json
{
"cnt_index": 1,
"save_cnt": true
}
```
4. Broker Connection:
The broker will attempt to use the newly received certificates for the connection.
5. Successful Connection Handling:
If the connection is successful, the certificates are permanently stored in the flash memory at the specified index.
6. Handling Connection Failures:
If the connection fails, the device will revert to the previously used certificate index, and the new certificates will not be saved.
# Firmware update from MQTT (ESP only)
When the gateway used is from a standard ESP32 environment [listed and defined here](https://github.com/1technophile/OpenMQTTGateway/blob/development/environments.ini), it can be updated through a simple MQTT command:
@@ -144,7 +220,7 @@ OpenMQTTGateway checks at start and every hour if an update is available.
Alternatively if you want to choose the update URL you can use the command below (ESP32 and ESP8266):
Without certificate, in this case we will use the root certificate defined in User_config.h
Without certificate, in this case the gateway will use the ota_server_cert certificate defined in default_ota_cert.h
```
mosquitto_pub -t "home/OpenMQTTGateway_ESP32_BLE/commands/MQTTtoSYS/firmware_update" -m '{
"version": "test",
@@ -153,39 +229,29 @@ mosquitto_pub -t "home/OpenMQTTGateway_ESP32_BLE/commands/MQTTtoSYS/firmware_upd
}'
```
With certificate:
With certificate (replace the \n in the certificate by spaces to publish it easily):
```
mosquitto_pub -t "home/OpenMQTTGateway_ESP32_BLE/commands/MQTTtoSYS/firmware_update" -m '{
"version": "test",
"password": "OTAPASSWORD",
"url": "https://github.com/1technophile/OpenMQTTGateway/releases/download/v0.9.12/esp32dev-ble-firmware.bin",
"server_cert": "-----BEGIN CERTIFICATE-----
MIIDrzCCApegAwIBAgIQCDvgVpBCRrGhdWrJWZHHSjANBgkqhkiG9w0BAQUFADBh
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBD
QTAeFw0wNjExMTAwMDAwMDBaFw0zMTExMTAwMDAwMDBaMGExCzAJBgNVBAYTAlVT
MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5j
b20xIDAeBgNVBAMTF0RpZ2lDZXJ0IEdsb2JhbCBSb290IENBMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4jvhEXLeqKTTo1eqUKKPC3eQyaKl7hLOllsB
CSDMAZOnTjC3U/dDxGkAV53ijSLdhwZAAIEJzs4bg7/fzTtxRuLWZscFs3YnFo97
nh6Vfe63SKMI2tavegw5BmV/Sl0fvBf4q77uKNd0f3p4mVmFaG5cIzJLv07A6Fpt
43C/dxC//AH2hdmoRBBYMql1GNXRor5H4idq9Joz+EkIYIvUX7Q6hL+hqkpMfT7P
T19sdl6gSzeRntwi5m3OFBqOasv+zbMUZBfHWymeMr/y7vrTC0LUq7dBMtoM1O/4
gdW7jVg/tRvoSSiicNoxBN33shbyTApOB6jtSj1etX+jkMOvJwIDAQABo2MwYTAO
BgNVHQ8BAf8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUA95QNVbR
TLtm8KPiGxvDl7I90VUwHwYDVR0jBBgwFoAUA95QNVbRTLtm8KPiGxvDl7I90VUw
DQYJKoZIhvcNAQEFBQADggEBAMucN6pIExIK+t1EnE9SsPTfrgT1eXkIoyQY/Esr
hMAtudXH/vTBH1jLuG2cenTnmCmrEbXjcKChzUyImZOMkXDiqw8cvpOp/2PV5Adg
06O/nVsJ8dWO41P0jmP6P6fbtGbfYmbW0W5BjfIttep3Sp+dWOIrWcBAI+0tKIJF
PnlUkiaY4IBIqDfv8NZ5YBberOgOzW6sRBc4L0na4UU+Krk2U886UAb3LujEV0ls
YSEY1QSteDwsOoBrp+uvFRTp2InBuThs4pFsiv9kuXclVzDAGySj4dzp30d8tbQk
CAUw7C29C79Fv1C5qfPrmAESrciIxpg0X40KPMbp1ZWVbd4=
-----END CERTIFICATE-----"}'
"ota_server_cert": "-----BEGIN CERTIFICATE----- MIIDrzCCApegAwIBAgIQCDvgVpBCRrGhdWrJWZHHSjANBgkqhkiG9w0BAQUFADBh CAUw7C29C79Fv1C5qfPrmAESrciIxpg0X40KPMbp1ZWVbd4= -----END CERTIFICATE-----"}'
```
A bash script is available [here](ota_command_cert.zip) to simplify the use of the `server_cert` parameter.
A bash script is available [here also](ota_command_cert.zip) to simplify the use of the `server_cert` parameter.
Alternatively the OTA certificate can also be saved with the cnt_index for future use:
```
mosquitto_pub -t "home/OpenMQTTGateway_ESP32_BLE/commands/MQTTtoSYS/config" -m '{
"cnt_index": 1,
"save_cnt":true,
"ota_server_cert": "-----BEGIN CERTIFICATE----- MIIDrzCCApegAwIBAgIQCDvgVpBCRrGhdWrJWZHHSjANBgkqhkiG9w0BAQUFADBh CAUw7C29C79Fv1C5qfPrmAESrciIxpg0X40KPMbp1ZWVbd4= -----END CERTIFICATE-----"
}'
```
The other connection parameters corresponding to the index need to be valid for the save function to work. This command will switch to connection parameters of index 1.
To enable this functionality, `MQTT_HTTPS_FW_UPDATE` will need to be defined or the line that defines in in user_config.h will need to be uncommented.
::: tip
environments.ini
+3 -13
View File
@@ -328,9 +328,10 @@ build_flags =
'-DZgatewayBT="BT"'
'-DLED_SEND_RECEIVE=2'
'-DLED_SEND_RECEIVE_ON=0'
'-DARDUINO_LOOP_STACK_SIZE=15000'
'-DMQTT_SECURE_DEFAULT=true'
'-DMQTT_SECURE_SELF_SIGNED'
'-DMQTT_CERT_VALIDATE_DEFAULT=true'
'-DMQTT_SECURE_SIGNED_CLIENT=true'
'-DMQTT_SERVER="xxxxxxxxxxxxx-ats.iot.eu-west-2.amazonaws.com"'
'-DMQTT_PORT="8883"'
'-DMQTT_USER=""'
@@ -1366,18 +1367,6 @@ build_flags =
'-DGateway_Name="OMG_TEST_MANUAL_WIFI"'
board_build.flash_mode = dout
[env:nodemcuv2-mqtt-fw-test]
platform = ${com.esp8266_platform}
board = nodemcuv2
lib_deps =
${com-esp.lib_deps}
${libraries.wifimanager8266}
build_flags =
${com-esp.build_flags}
'-DMQTT_SECURE_SELF_SIGNED'
'-DGateway_Name="OMG_TEST_MQTT_FW"'
board_build.flash_mode = dout
[env:rf-wifi-gateway]
platform = ${com.esp8266_platform}
board = nodemcuv2
@@ -1748,4 +1737,5 @@ build_flags =
'-DLED_SEND_RECEIVE_ON=0'
'-DGateway_Name="OMG_ESP32_BLE"'
'-DUSE_BLUFI=1'
'-DARDUINO_LOOP_STACK_SIZE=15000'
custom_description = Regular BLE gateway based on esp-idf with adaptive scanning activated, automatically adapts the scan parameters depending on your devices
main/User_config.h
+112 -109
View File
@@ -147,17 +147,30 @@ const byte mac[] = {0xDE, 0xED, 0xBA, 0xFE, 0x54, 0x95}; //W5100 ethernet shield
//MQTT Parameters definition
#define parameters_size 65
#define mqtt_topic_max_size 150
#ifndef mqtt_max_packet_size
# ifdef MQTT_HTTPS_FW_UPDATE
# ifndef CHECK_OTA_UPDATE
# define CHECK_OTA_UPDATE true // enable to check for the presence of a new version for your environment on Github
# endif
# define mqtt_max_packet_size 2560
# else
# define mqtt_max_packet_size 1024
#define mqtt_key_max_size 20
#ifdef MQTT_HTTPS_FW_UPDATE
# ifndef CHECK_OTA_UPDATE
# define CHECK_OTA_UPDATE true // enable to check for the presence of a new version for your environment on Github
# endif
#endif
#ifndef JSON_MSG_BUFFER
# if defined(ESP32)
# define JSON_MSG_BUFFER 816 // adjusted to minimum size covering largest Theengs device JSON properties (RuuviTag_RAWv2)
# elif defined(ESP8266)
# define JSON_MSG_BUFFER 512 // Json message max buffer size, don't put 768 or higher it is causing unexpected behaviour on ESP8266, certificates handling with ESP8266 is not tested
# endif
# if MQTT_SECURE_DEFAULT
# define JSON_MSG_BUFFER_MAX 2048 // Json message buffer size increased to handle certificate changes through MQTT, used for the queue and the coming MQTT messages
# else
# define JSON_MSG_BUFFER_MAX JSON_MSG_BUFFER
# endif
#endif
#ifndef mqtt_max_topic_size
# define mqtt_max_packet_size JSON_MSG_BUFFER_MAX + mqtt_topic_max_size + 10 // maximum size of the MQTT packet
#endif
#ifndef MQTT_USER
# define MQTT_USER "your_username"
#endif
@@ -181,103 +194,103 @@ const byte mac[] = {0xDE, 0xED, 0xBA, 0xFE, 0x54, 0x95}; //W5100 ethernet shield
# define QueueSemaphoreTimeOutLoop 100 // time out for semaphore retrieval from the loop
#endif
#if defined(ESP8266) || defined(ESP32)
// Uncomment to use a device running TheengsGateway to decode BLE data. (https://github.com/theengs/gateway)
// Set the topic to the subscribe topic configured in the TheengGateway
// #define MQTTDecodeTopic "MQTTDecode"
// The root ca certificate used for validating the MQTT broker
// The certificate must be in PEM ascii format
const char* certificate PROGMEM = R"EOF("
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
")EOF";
#define ATTEMPTS_BEFORE_BG 10 // Number of wifi connection attempts before going to BG protocol
#define ATTEMPTS_BEFORE_B 20 // Number of wifi connection attempts before going to B protocol
# define ATTEMPTS_BEFORE_BG 10 // Number of wifi connection attempts before going to BG protocol
# define ATTEMPTS_BEFORE_B 20 // Number of wifi connection attempts before going to B protocol
#ifndef NTP_SERVER
# define NTP_SERVER "pool.ntp.org"
#endif
# ifndef NTP_SERVER
# define NTP_SERVER "pool.ntp.org"
# endif
#ifndef MQTT_SECURE_DEFAULT
# define MQTT_SECURE_DEFAULT false
#endif
# ifndef MQTT_SECURE_DEFAULT
# define MQTT_SECURE_DEFAULT false
# endif
#ifndef MQTT_CERT_VALIDATE_DEFAULT
# define MQTT_CERT_VALIDATE_DEFAULT false
#endif
# ifndef MQTT_CERT_VALIDATE_DEFAULT
# define MQTT_CERT_VALIDATE_DEFAULT false
# endif
#ifndef AWS_IOT
# define AWS_IOT false
#endif
# ifndef AWS_IOT
# define AWS_IOT false
# endif
# if AWS_IOT
#if AWS_IOT
// Enable the use of ALPN for AWS IoT Core with the port 443
const char* alpnProtocols[] = {"x-amzn-mqtt-ca", NULL};
# endif
#endif
//# define MQTT_HTTPS_FW_UPDATE //uncomment to enable updating via MQTT message.
# ifdef MQTT_HTTPS_FW_UPDATE
#ifdef MQTT_HTTPS_FW_UPDATE
// If used, this should be set to the root CA certificate of the server hosting the firmware.
# ifdef PRIVATE_CERTS
# include "certs/private_ota_cert.h"
# else
# include "certs/default_ota_cert.h"
# endif
# ifndef MQTT_HTTPS_FW_UPDATE_USE_PASSWORD
# define MQTT_HTTPS_FW_UPDATE_USE_PASSWORD 1 // Set this to 0 if not using TLS connection to MQTT broker to prevent clear text passwords being sent.
# endif
# if DEVELOPMENTOTA
# define OTA_JSON_URL "https://github.com/1technophile/OpenMQTTGateway/raw/gh-pages/dev/firmware_build/latest_version_dev.json" //OTA url used to discover new versions of the firmware from development nightly builds
# else
# define OTA_JSON_URL "https://github.com/1technophile/OpenMQTTGateway/raw/gh-pages/firmware_build/latest_version.json" //OTA url used to discover new versions of the firmware
# endif
# define ENTITY_PICTURE "https://github.com/1technophile/OpenMQTTGateway/raw/development/docs/img/Openmqttgateway_logo_mini_margins.png"
# define RELEASE_LINK_DEV "https://github.com/1technophile/OpenMQTTGateway/raw/gh-pages/dev/firmware_build/"
# define RELEASE_LINK "https://github.com/1technophile/OpenMQTTGateway/releases/download/"
# ifdef PRIVATE_CERTS
# include "certs/private_ota_cert.h"
# else
# include "certs/default_ota_cert.h"
# endif
# ifndef MQTT_SECURE_SELF_SIGNED
# define MQTT_SECURE_SELF_SIGNED 0
# ifndef MQTT_HTTPS_FW_UPDATE_USE_PASSWORD
# define MQTT_HTTPS_FW_UPDATE_USE_PASSWORD 1 // Set this to 0 if not using TLS connection to MQTT broker to prevent clear text passwords being sent.
# endif
# ifndef MQTT_SECURE_SELF_SIGNED_CLIENT
# define MQTT_SECURE_SELF_SIGNED_CLIENT 1 // If using a self signed certificate for the broker and not using client certificates set this to false or 0
# if DEVELOPMENTOTA
# define OTA_JSON_URL "https://github.com/1technophile/OpenMQTTGateway/raw/gh-pages/dev/firmware_build/latest_version_dev.json" //OTA url used to discover new versions of the firmware from development nightly builds
# else
# define OTA_JSON_URL "https://github.com/1technophile/OpenMQTTGateway/raw/gh-pages/firmware_build/latest_version.json" //OTA url used to discover new versions of the firmware
# endif
# define ENTITY_PICTURE "https://github.com/1technophile/OpenMQTTGateway/raw/development/docs/img/Openmqttgateway_logo_mini_margins.png"
# define RELEASE_LINK_DEV "https://github.com/1technophile/OpenMQTTGateway/raw/gh-pages/dev/firmware_build/"
# define RELEASE_LINK "https://github.com/1technophile/OpenMQTTGateway/releases/download/"
#else
const char* OTAserver_cert = "";
#endif
# ifndef MQTT_SECURE_SELF_SIGNED_INDEX_DEFAULT
# define MQTT_SECURE_SELF_SIGNED_INDEX_DEFAULT 0
# endif
#ifndef MQTT_SECURE_SIGNED_CLIENT
# define MQTT_SECURE_SIGNED_CLIENT 0 // If using a signed certificate for the broker and using client certificate/key set this to true or 1
#endif
# if MQTT_SECURE_SELF_SIGNED
# ifdef PRIVATE_CERTS
# include "certs/private_client_cert.h"
# include "certs/private_client_key.h"
# include "certs/private_server_cert.h"
# else
# include "certs/default_client_cert.h"
# include "certs/default_client_key.h"
# include "certs/default_server_cert.h"
# endif
#ifndef CNT_DEFAULT_INDEX
# define CNT_DEFAULT_INDEX 0 // Default set of connection parameters
#endif
struct ss_certs {
const char* server_cert;
const char* client_cert;
const char* client_key;
#ifdef PRIVATE_CERTS
# include "certs/private_client_cert.h"
# include "certs/private_client_key.h"
# include "certs/private_server_cert.h"
#else
# include "certs/default_client_cert.h"
# include "certs/default_client_key.h"
# include "certs/default_server_cert.h"
#endif
#include <string>
struct ss_cnt_parameters {
std::string server_cert;
std::string client_cert;
std::string client_key;
std::string ota_server_cert;
char mqtt_server[parameters_size];
char mqtt_port[6];
char mqtt_user[parameters_size];
char mqtt_pass[parameters_size];
bool isConnectionSecure;
bool isCertValidate;
bool validConnection;
};
struct ss_certs certs_array[2] = {
{ss_server_cert, ss_client_cert, ss_client_key},
{ss_server_cert, ss_client_cert, ss_client_key}};
// Index 0 is used for connection parameters provided in the build that can be overloaded by WiFi Manager/Onboarding/WebUI,MQTT
#define CNT_DEFAULT_INDEX 0
// Index 1 and more are used for connection parameters provided at runtime by MQTT
static_assert(MQTT_SECURE_SELF_SIGNED_INDEX_DEFAULT < (sizeof(certs_array) / sizeof(ss_certs)),
"Invalid MQTT self signed default index");
# endif
#endif
ss_cnt_parameters cnt_parameters_array[3] = {
{ss_server_cert, ss_client_cert, ss_client_key, OTAserver_cert, MQTT_SERVER, MQTT_PORT, MQTT_USER, MQTT_PASS, MQTT_SECURE_DEFAULT, MQTT_CERT_VALIDATE_DEFAULT, true},
{"", "", "", "", MQTT_SERVER, MQTT_PORT, MQTT_USER, MQTT_PASS, MQTT_SECURE_DEFAULT, MQTT_CERT_VALIDATE_DEFAULT, false},
{"", "", "", "", MQTT_SERVER, MQTT_PORT, MQTT_USER, MQTT_PASS, MQTT_SECURE_DEFAULT, MQTT_CERT_VALIDATE_DEFAULT, false}};
#define MIN_CERT_LENGTH 200 // Minimum length of a certificate to be considered valid
/**
* Deep-sleep for the ESP8266.
@@ -655,16 +668,6 @@ Adafruit_NeoPixel leds2(ANEOPIX_IND_NUM_LEDS, ANEOPIX_IND_DATA_GPIO2, ANEOPIX_IN
# define valueAsATopic false // define true to integrate msg value into the subject when receiving
#endif
#if defined(ESP32)
# define JSON_MSG_BUFFER 816 // adjusted to minimum size covering largest Theengs device JSON properties (RuuviTag_RAWv2)
# define SIGNAL_SIZE_UL_ULL uint64_t
# define STRTO_UL_ULL strtoull
#elif defined(ESP8266)
# define JSON_MSG_BUFFER 512 // Json message max buffer size, don't put 768 or higher it is causing unexpected behaviour on ESP8266
# define SIGNAL_SIZE_UL_ULL uint64_t
# define STRTO_UL_ULL strtoull
#endif
#if defined(ZgatewayRF) || defined(ZgatewayIR) || defined(ZgatewaySRFB) || defined(ZgatewayWeatherStation) || defined(ZgatewayRTL_433)
// variable to avoid duplicates
# ifndef time_avoid_duplicate
@@ -715,9 +718,6 @@ char mqtt_topic[parameters_size + 1] = Base_Topic;
char gateway_name[parameters_size + 1] = Gateway_Name;
void connectMQTT();
#ifndef ESPWifiManualSetup
void saveConfig();
#endif
unsigned long uptime();
bool cmpToMainTopic(const char*, const char*);
@@ -731,40 +731,43 @@ void pub(const char*, const char*);
Preferences preferences;
#endif
#ifdef ZmqttDiscovery
unsigned long lastDiscovery = 0; // Time of the last discovery to trigger automaticaly to off after DiscoveryAutoOffTimer
#ifndef DEFAULT_DISCOVERY
# define DEFAULT_DISCOVERY true
#endif
#if defined(ESP8266) || defined(ESP32)
# include <vector>
#include <vector>
// Flags definition for white list, black list, discovery management
# define device_flags_init 0 << 0
# define device_flags_isDisc 1 << 0
# define device_flags_isWhiteL 1 << 1
# define device_flags_isBlackL 1 << 2
# define device_flags_connect 1 << 3
# define isWhite(device) device->isWhtL
# define isBlack(device) device->isBlkL
# define isDiscovered(device) device->isDisc
#define device_flags_init 0 << 0
#define device_flags_isDisc 1 << 0
#define device_flags_isWhiteL 1 << 1
#define device_flags_isBlackL 1 << 2
#define device_flags_connect 1 << 3
#define isWhite(device) device->isWhtL
#define isBlack(device) device->isBlkL
#define isDiscovered(device) device->isDisc
/*--------------------Minimum freeHeap--------------------*/
// Below this parameter we trigger a restart, this avoid stuck boards like seen in https://github.com/1technophile/OpenMQTTGateway/issues/1693
# define MinimumMemory 40000
#define MinimumMemory 40000
/*----------------CONFIGURABLE PARAMETERS-----------------*/
struct SYSConfig_s {
bool XtoMQTT; // if true the gateway will publish the received data on the MQTT broker
bool discovery; // HA discovery convention
bool ohdiscovery; // OH discovery specificities
# ifdef RGB_INDICATORS
#ifdef RGB_INDICATORS
int rgbbrightness; // brightness of the RGB LED
# endif
#endif
};
#ifndef DEFAULT_XtoMQTT
# define DEFAULT_XtoMQTT true
#endif
#if defined(ZgatewayRF) || defined(ZgatewayIR) || defined(ZgatewaySRFB) || defined(ZgatewayWeatherStation) || defined(ZgatewayRTL_433)
bool isAduplicateSignal(SIGNAL_SIZE_UL_ULL);
void storeSignalValue(SIGNAL_SIZE_UL_ULL);
bool isAduplicateSignal(uint64_t);
void storeSignalValue(uint64_t);
#endif
// Origin topics
main/Zblufi.ino
+45 -17
View File
@@ -57,38 +57,45 @@ static void example_event_callback(esp_blufi_cb_event_t event, esp_blufi_cb_para
* now, as a example, we do it more simply */
switch (event) {
case ESP_BLUFI_EVENT_INIT_FINISH:
Log.notice(F("BLUFI init finish" CR));
Log.trace(F("BLUFI init finish" CR));
esp_blufi_adv_start();
break;
case ESP_BLUFI_EVENT_DEINIT_FINISH:
Log.notice(F("BLUFI deinit finish" CR));
Log.trace(F("BLUFI deinit finish" CR));
NimBLEDevice::deinit(true);
break;
case ESP_BLUFI_EVENT_BLE_CONNECT:
Log.trace(F("BLUFI BLE connect" CR));
omg_blufi_ble_connected = true;
esp_blufi_adv_stop();
blufi_security_init();
break;
case ESP_BLUFI_EVENT_BLE_DISCONNECT:
Log.trace(F("BLUFI BLE disconnect" CR));
omg_blufi_ble_connected = false;
blufi_security_deinit();
if (WiFi.isConnected()) {
esp_blufi_deinit();
# ifndef ESPWifiManualSetup
wifiManager.stopConfigPortal();
# endif
} else {
esp_blufi_adv_start();
}
break;
case ESP_BLUFI_EVENT_REQ_CONNECT_TO_AP:
Log.notice(F("BLUFI requset wifi connect to AP" CR));
Log.trace(F("BLUFI request wifi connect to AP" CR));
WiFi.begin((char*)gl_sta_ssid, (char*)gl_sta_passwd);
gl_sta_is_connecting = true;
blufiConnectAP = true;
break;
case ESP_BLUFI_EVENT_REQ_DISCONNECT_FROM_AP:
Log.notice(F("BLUFI requset wifi disconnect from AP\n" CR));
Log.trace(F("BLUFI request wifi disconnect from AP\n" CR));
WiFi.disconnect();
blufiConnectAP = false;
break;
case ESP_BLUFI_EVENT_REPORT_ERROR:
Log.notice(F("BLUFI report error, error code %d\n" CR), param->report_error.state);
Log.trace(F("BLUFI report error, error code %d\n" CR), param->report_error.state);
esp_blufi_send_error_info(param->report_error.state);
break;
case ESP_BLUFI_EVENT_GET_WIFI_STATUS: {
@@ -109,6 +116,7 @@ static void example_event_callback(esp_blufi_cb_event_t event, esp_blufi_cb_para
break;
}
case ESP_BLUFI_EVENT_RECV_SLAVE_DISCONNECT_BLE:
Log.trace(F("BLUFI recv slave disconnect a ble connection" CR));
esp_blufi_disconnect();
break;
case ESP_BLUFI_EVENT_RECV_STA_SSID:
@@ -137,20 +145,40 @@ static void example_event_callback(esp_blufi_cb_event_t event, esp_blufi_cb_para
}
if (!json.isNull()) {
Log.trace(F("\nparsed json, size: %u" CR), json.memoryUsage());
if (json.containsKey("mqtt_server"))
strcpy(mqtt_server, json["mqtt_server"]);
if (json.containsKey("mqtt_port"))
strcpy(mqtt_port, json["mqtt_port"]);
if (json.containsKey("mqtt_user"))
strcpy(mqtt_user, json["mqtt_user"]);
if (json.containsKey("mqtt_pass"))
strcpy(mqtt_pass, json["mqtt_pass"]);
if (json.containsKey("mqtt_topic"))
strcpy(mqtt_topic, json["mqtt_topic"]);
if (json.containsKey("mqtt_broker_secure"))
mqtt_secure = json["mqtt_broker_secure"].as<bool>();
if (json.containsKey("mqtt_server") && json["mqtt_server"].is<const char*>() && json["mqtt_server"].as<String>().length() > 0 && json["mqtt_server"].as<String>().length() < parameters_size)
strcpy(cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_server, json["mqtt_server"]);
if (json.containsKey("mqtt_port") && json["mqtt_port"].is<const char*>() && json["mqtt_port"].as<String>().length() > 0 && json["mqtt_port"].as<String>().length() < parameters_size)
strcpy(cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_port, json["mqtt_port"]);
if (json.containsKey("mqtt_user") && json["mqtt_user"].is<const char*>() && json["mqtt_user"].as<String>().length() > 0 && json["mqtt_user"].as<String>().length() < parameters_size)
strcpy(cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_user, json["mqtt_user"]);
if (json.containsKey("mqtt_pass") && json["mqtt_pass"].is<const char*>() && json["mqtt_pass"].as<String>().length() > 0 && json["mqtt_pass"].as<String>().length() < parameters_size)
strcpy(cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_pass, json["mqtt_pass"]);
if (json.containsKey("mqtt_broker_secure") && json["mqtt_broker_secure"].is<bool>())
cnt_parameters_array[CNT_DEFAULT_INDEX].isConnectionSecure = json["mqtt_broker_secure"].as<bool>();
if (json.containsKey("mqtt_iscertvalid") && json["mqtt_iscertvalid"].is<bool>())
cnt_parameters_array[CNT_DEFAULT_INDEX].isCertValidate = json["mqtt_iscertvalid"].as<bool>();
if (json.containsKey("mqtt_broker_cert") && json["mqtt_broker_cert"].is<const char*>() && json["mqtt_broker_cert"].as<String>().length() > MIN_CERT_LENGTH)
cnt_parameters_array[CNT_DEFAULT_INDEX].server_cert = json["mqtt_broker_cert"].as<const char*>();
if (json.containsKey("mqtt_client_cert") && json["mqtt_client_cert"].is<const char*>() && json["mqtt_client_cert"].as<String>().length() > MIN_CERT_LENGTH)
cnt_parameters_array[CNT_DEFAULT_INDEX].client_cert = json["mqtt_client_cert"].as<const char*>();
if (json.containsKey("mqtt_client_key") && json["mqtt_client_key"].is<const char*>() && json["mqtt_client_key"].as<String>().length() > MIN_CERT_LENGTH)
cnt_parameters_array[CNT_DEFAULT_INDEX].client_key = json["mqtt_client_key"].as<const char*>();
if (json.containsKey("ota_server_cert") && json["ota_server_cert"].is<const char*>() && json["ota_server_cert"].as<String>().length() > MIN_CERT_LENGTH)
cnt_parameters_array[CNT_DEFAULT_INDEX].ota_server_cert = json["ota_server_cert"].as<const char*>();
if (json.containsKey("gateway_name"))
strcpy(gateway_name, json["gateway_name"]);
if (json.containsKey("mqtt_topic"))
strcpy(mqtt_topic, json["mqtt_topic"]);
if (json.containsKey("ota_pass"))
strcpy(ota_pass, json["ota_pass"]);
if (json.containsKey("cnt_index") && json["cnt_index"].is<int>() && json["cnt_index"].as<int>() > 0 && json["cnt_index"].as<int>() < 3) {
cnt_index = json["cnt_index"].as<int>();
} else {
cnt_index = CNT_DEFAULT_INDEX;
}
saveConfig();
}
break;
main/ZgatewayBT.ino
+2 -2
View File
@@ -800,7 +800,7 @@ void stopProcessing() {
xSemaphoreGive(semaphoreBLEOperation);
}
}
Log.notice(F("BLE gateway stopped %T, free heap: %d" CR), ESP.getFreeHeap());
Log.notice(F("BLE gateway stopped, free heap: %d" CR), ESP.getFreeHeap());
}
void coreTask(void* pvParameters) {
@@ -906,7 +906,7 @@ void setupBTTasksAndBLE() {
procBLETask, /* Function to implement the task */
"procBLETask", /* Name of the task */
# ifdef USE_BLUFI
13000,
13500,
# else
8500, /* Stack size in bytes */
# endif
main/ZgatewayIR.ino
+4 -4
View File
@@ -114,7 +114,7 @@ void IRtoMQTT() {
# ifdef ESP32
Log.trace(F("IR Task running on core :%d" CR), xPortGetCoreID());
# endif
IRdata["value"] = (SIGNAL_SIZE_UL_ULL)(results.value);
IRdata["value"] = uint64_t(results.value);
IRdata["protocol"] = (int)(results.decode_type);
IRdata["bits"] = (int)(results.bits);
IRdata["hex"] = resultToHexidecimal(&results);
@@ -141,7 +141,7 @@ void IRtoMQTT() {
Log.trace(F("raw redirected" CR));
# endif
irrecv.resume(); // Receive the next value
SIGNAL_SIZE_UL_ULL MQTTvalue = IRdata["value"].as<SIGNAL_SIZE_UL_ULL>();
uint64_t MQTTvalue = IRdata["value"].as<uint64_t>();
//trc(MQTTvalue);
if ((pubIRunknownPrtcl == false && IRdata["protocol"].as<int>() == -1)) { // don't publish unknown IR protocol
Log.notice(F("--no pub unknwn prt--" CR));
@@ -159,7 +159,7 @@ void IRtoMQTT() {
}
}
bool sendIdentifiedProtocol(const char* protocol_name, SIGNAL_SIZE_UL_ULL data, const char* hex, unsigned int valueBITS, uint16_t valueRPT);
bool sendIdentifiedProtocol(const char* protocol_name, uint64_t data, const char* hex, unsigned int valueBITS, uint16_t valueRPT);
# if jsonReceiving
void MQTTtoIR(char* topicOri, JsonObject& IRdata) {
@@ -254,7 +254,7 @@ void MQTTtoIR(char* topicOri, JsonObject& IRdata) {
}
# endif
bool sendIdentifiedProtocol(const char* protocol_name, SIGNAL_SIZE_UL_ULL data, const char* hex, unsigned int valueBITS, uint16_t valueRPT) {
bool sendIdentifiedProtocol(const char* protocol_name, uint64_t data, const char* hex, unsigned int valueBITS, uint16_t valueRPT) {
uint8_t dataarray[valueBITS];
if (hex) {
const char* ptr = NULL;
main/ZgatewayRF.ino
+5 -5
View File
@@ -82,7 +82,7 @@ static char* dec2binWzerofill(unsigned long Dec, unsigned int bitLength) {
# if defined(ZmqttDiscovery) && !defined(RF_DISABLE_TRANSMIT) && defined(RFmqttDiscovery)
void RFtoMQTTdiscovery(SIGNAL_SIZE_UL_ULL MQTTvalue) {
void RFtoMQTTdiscovery(uint64_t MQTTvalue) {
//on the fly switch creation from received RF values
char val[11];
sprintf(val, "%lu", MQTTvalue);
@@ -114,11 +114,11 @@ void RFtoMQTT() {
# ifdef ESP32
Log.trace(F("RF Task running on core :%d" CR), xPortGetCoreID());
# endif
SIGNAL_SIZE_UL_ULL MQTTvalue = mySwitch.getReceivedValue();
uint64_t MQTTvalue = mySwitch.getReceivedValue();
int length = mySwitch.getReceivedBitlength();
const char* binary = dec2binWzerofill(MQTTvalue, length);
RFdata["value"] = (SIGNAL_SIZE_UL_ULL)MQTTvalue;
RFdata["value"] = (uint64_t)MQTTvalue;
RFdata["protocol"] = (int)mySwitch.getReceivedProtocol();
RFdata["length"] = (int)mySwitch.getReceivedBitlength();
RFdata["delay"] = (int)mySwitch.getReceivedDelay();
@@ -168,7 +168,7 @@ void MQTTtoRF(char* topicOri, char* datacallback) {
# endif
mySwitch.disableReceive();
mySwitch.enableTransmit(RF_EMITTER_GPIO);
SIGNAL_SIZE_UL_ULL data = STRTO_UL_ULL(datacallback, NULL, 10); // we will not be able to pass values > 4294967295 on Arduino boards
uint64_t data = strtoull(datacallback, NULL, 10); // we will not be able to pass values > 4294967295 on Arduino boards
// RF DATA ANALYSIS
//We look into the subject to see if a special RF protocol is defined
@@ -227,7 +227,7 @@ void MQTTtoRF(char* topicOri, char* datacallback) {
void MQTTtoRF(char* topicOri, JsonObject& RFdata) { // json object decoding
if (cmpToMainTopic(topicOri, subjectMQTTtoRF)) {
Log.trace(F("MQTTtoRF json" CR));
SIGNAL_SIZE_UL_ULL data = RFdata["value"];
uint64_t data = RFdata["value"];
if (data != 0) {
int valuePRT = RFdata["protocol"] | 1;
int valuePLSL = RFdata["delay"] | 350;
main/ZmqttDiscovery.ino
+2
View File
@@ -1300,4 +1300,6 @@ void pubMqttDiscovery() {
# endif
# endif
}
#else
void pubMqttDiscovery() {}
#endif
main/ZwebUI.ino
+8 -7
View File
@@ -708,34 +708,34 @@ void handleMQ() {
if (server.hasArg("mh")) {
WEBtoSYS["mqtt_server"] = server.arg("mh");
if (strncmp(mqtt_server, server.arg("mh").c_str(), parameters_size)) {
if (strncmp(cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_server, server.arg("mh").c_str(), parameters_size)) {
update = true;
}
}
if (server.hasArg("ml")) {
WEBtoSYS["mqtt_port"] = server.arg("ml");
if (strncmp(mqtt_port, server.arg("ml").c_str(), 6)) {
if (strncmp(cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_port, server.arg("ml").c_str(), 6)) {
update = true;
}
}
if (server.hasArg("mu")) {
WEBtoSYS["mqtt_user"] = server.arg("mu");
if (strncmp(mqtt_user, server.arg("mu").c_str(), parameters_size)) {
if (strncmp(cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_user, server.arg("mu").c_str(), parameters_size)) {
update = true;
}
}
if (server.hasArg("mp")) {
WEBtoSYS["mqtt_pass"] = server.arg("mp");
if (strncmp(mqtt_pass, server.arg("mp").c_str(), parameters_size)) {
if (strncmp(cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_pass, server.arg("mp").c_str(), parameters_size)) {
update = true;
}
}
// SC - Secure Connection argument is only present when true
if (mqtt_secure != server.hasArg("sc")) {
if (cnt_parameters_array[CNT_DEFAULT_INDEX].isConnectionSecure != server.hasArg("sc")) {
update = true;
}
WEBtoSYS["mqtt_secure"] = server.hasArg("sc");
@@ -764,7 +764,8 @@ void handleMQ() {
# ifndef ESPWifiManualSetup
if (update) {
Log.warning(F("[WebUI] Save MQTT and Reconnect" CR));
WEBtoSYS["cnt_index"] = 1; // For the moment we only enable to change index 1 with WebUI
WEBtoSYS["save_cnt"] = true;
char jsonChar[100];
serializeJson(modules, jsonChar, measureJson(modules) + 1);
char buffer[WEB_TEMPLATE_BUFFER_MAX_SIZE];
@@ -801,7 +802,7 @@ void handleMQ() {
response += String(script);
response += String(style);
// mqtt server (mh), mqtt port (ml), mqtt username (mu), mqtt password (mp), secure connection (sc), server certificate (msc), topic (mt)
snprintf(buffer, WEB_TEMPLATE_BUFFER_MAX_SIZE, config_mqtt_body, jsonChar, gateway_name, mqtt_server, mqtt_port, mqtt_user, (mqtt_secure ? "checked" : ""), gateway_name, mqtt_topic);
snprintf(buffer, WEB_TEMPLATE_BUFFER_MAX_SIZE, config_mqtt_body, jsonChar, gateway_name, cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_server, cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_port, cnt_parameters_array[CNT_DEFAULT_INDEX].mqtt_user, (cnt_parameters_array[CNT_DEFAULT_INDEX].isConnectionSecure ? "checked" : ""), gateway_name, mqtt_topic);
response += String(buffer);
snprintf(buffer, WEB_TEMPLATE_BUFFER_MAX_SIZE, footer, OMG_VERSION);
response += String(buffer);
main/main.ino
+510 -248
View File
File diff suppressed because it is too large. Load diff
platformio.ini
-1
View File
@@ -84,7 +84,6 @@ extra_configs =
;default_envs = nodemcuv2-rf-cc1101
;default_envs = nodemcuv2-somfy-cc1101
;default_envs = manual-wifi-test
;default_envs = nodemcuv2-mqtt-fw-test
;default_envs = rf-wifi-gateway
;default_envs = nodemcuv2-rf2
;default_envs = nodemcuv2-rf2-cc1101