⬆️ Update emqx/emqx to v5.7.0

[renovate]

This PR contains the following updates:

Package	Update	Change
emqx/emqx	minor	v5.6.1 -> v5.7.0

---

Release Notes

emqx/emqx (emqx/emqx)

v5.7.0: EMQX v5.7.0

Compare Source

Enhancements

MQTT

Implemented Durable Sessions, which persists MQTT Persistent Sessions and their messages to disk, and continuously replicates session metadata and MQTT messages among multiple nodes in the EMQX cluster. This achieves effective failover and recovery mechanisms, ensuring service continuity and high availability, thereby enhancing system reliability.
Added metrics related to EMQX durable storage to Prometheus:

• emqx_ds_egress_batches
• emqx_ds_egress_batches_retry
• emqx_ds_egress_batches_failed
• emqx_ds_egress_messages
• emqx_ds_egress_bytes
• emqx_ds_egress_flush_time
• emqx_ds_store_batch_time
• emqx_ds_builtin_next_time
• emqx_ds_storage_bitfield_lts_counter_seek
• emqx_ds_storage_bitfield_lts_counter_next
• emqx_ds_storage_bitfield_lts_counter_collision

Note: these metrics are only visible when session persistence is enabled.
The number of persisted messages has also been added to the Dashboard.

Security

• #​12947 For JWT authentication, support new disconnect_after_expire option. When enabled, the client will be disconnected after the JWT token expires.

Note: This is a breaking change. This option is enabled by default, so the default behavior is changed. Previously, the clients with actual JWTs could connect to the broker and stay connected even after the JWT token expired. Now, the client will be disconnected after the JWT token expires. To preserve the previous behavior, set disconnect_after_expire to false.

Data Processing and Integration

• #​12671 An unescape function has been added to the rule engine SQL language to handle the expansion of escape sequences in strings. This addition has been done because string literals in the SQL language don't support any escape codes (e.g., \n and \t). This enhancement allows for more flexible string manipulation within SQL expressions.

Extensibility

• #​12872 Implemented Client Attributes feature. It allows setting additional properties for each client using key-value pairs. Property values can be generated from MQTT client connection information (such as username, client ID, TLS certificate) or set from data accompanying successful authentication returns. Properties can be used in EMQX for authentication, authorization, data integration, and MQTT extension functions. Compared to using static properties like client ID directly, client properties offer greater flexibility in various business scenarios, simplifying the development process and enhancing adaptability and efficiency in development work.
  Initialization of client_attrs
  The client_attrs fields can be initially populated from one of the following clientinfo fields:

  • cn: The common name from the TLS client's certificate.
  • dn: The distinguished name from the TLS client's certificate, that is, the certificate "Subject".
  • clientid: The MQTT client ID provided by the client.
  • username: The username provided by the client.
  • user_property: Extract a property value from 'User-Property' of the MQTT CONNECT packet.

  Extension through Authentication Responses
  Additional attributes may be merged into client_attrs from authentication responses. Supported
  authentication backends include:

  • HTTP: Attributes can be included in the JSON object of the HTTP response body through a
    client_attrs field.
  • JWT: Attributes can be included via a client_attrs claim within the JWT.

  Usage in Authentication and Authorization
  If client_attrs is initialized before authentication, it can be used in external authentication
  requests. For instance, ${client_attrs.property1} can be used within request templates
  directed at an HTTP server for authenticity validation.

  • The client_attrs can be utilized in authorization configurations or request templates, enhancing
    flexibility and control. Examples include: In acl.conf, use {allow, all, all, ["${client_attrs.namespace}/#"]} to apply permissions based on the namespace attribute.
  • In other authorization backends, ${client_attrs.namespace} can be used within request templates to dynamically include client attributes.

• #​12910 Added plugin configuration management and schema validation. For EMQX enterprise edition, one can also annotate the schema with metadata to facilitate UI rendering in the Dashboard. See more details in the plugin template and plugin documentation.

Operations and Management

• #​12923 Provided more specific error when importing wrong format into builtin authenticate database.

• #​12940 Added ignore_readonly argument to PUT /configs API.
  Before this change, EMQX would return 400 (BAD_REQUEST) if the raw config included read-only root keys (cluster, rpc, and node).
  After this enhancement it can be called as PUT /configs?ignore_readonly=true, EMQX will in this case ignore readonly root config keys, and apply the rest. For observability purposes, an info level message is logged if any readonly keys are dropped.
  Also fixed an exception when config has bad HOCON syntax (returns 500). Now bad syntax will cause the API to return 400 (BAD_REQUEST).

• #​12957 Started building packages for macOS 14 (Apple Silicon) and Ubuntu 24.04 Noble Numbat (LTS).

Bug Fixes

Security

• #​12887 Fixed MQTT enhanced auth with sasl scram.

• #​12962 TLS clients can now verify server hostname against wildcard certificate. For example, if a certificate is issued for host *.example.com, TLS clients is able to verify server hostnames like srv1.example.com.

MQTT

• #​12996 Fixed process leak in emqx_retainer application. Previously, client disconnection while receiving retained messages could cause a process leak.

Data Processing and Integration

• #​12653 The rule engine function bin2hexstr now supports bitstring inputs with a bit size that is not divisible by 8. Such bitstrings can be returned by the rule engine function subbits.

• #​12657 The rule engine SQL-based language previously did not allow putting any expressions as array elements in array literals (only constants and variable references were allowed). This has now been fixed so that one can use any expressions as array elements.
  The following is now permitted, for example:

  select
  [21 + 21, abs(-abs(-2)), [1 + 1], 4] as my_array
  from "t/#"

• #​12932 Previously, if a HTTP action request received a 503 (Service Unavailable) status, it was marked as a failure and the request was not retried. This has now been fixed so that the request is retried a configurable number of times.

• #​12948 Fixed an issue where sensitive HTTP header values like Authorization would be substituted by ****** after updating a connector.

• #​13118 Fix a performance issue in the rule engine template rendering.

Observability

• #​12765 Make sure stats subscribers.count subscribers.max contains shared-subscribers. It only contains non-shared subscribers previously.

Operations and Management

• #​12812 Made resource health checks non-blocking operations. This means that operations such as updating or removing a resource won't be blocked by a lengthy running health check.

• #​12830 Made channel (action/source) health checks non-blocking operations. This means that operations such as updating or removing an action/source data integration won't be blocked by a lengthy running health check.

• #​12993 Fixed listener config update API when handling an unknown zone.
  Before this fix, when a listener config is updated with an unknown zone, for example {"zone": "unknown"}, the change would be accepted, causing all clients to crash whens connected.
  After this fix, updating the listener with an unknown zone name will get a "Bad request" response.

• #​13012 The MQTT listerners config option access_rules has been improved in the following ways:

  • The listener no longer crash with an incomprehensible error message if a non-valid access rule is configured. Instead a configuration error is generated.
  • One can now add several rules in a single string by separating them by comma (for example, "allow 10.0.1.0/24, deny all").

• #​13041 Improved HTTP authentication error log message. If HTTP content-type header is missing for POST method, it now emits a meaningful error message instead of a less readable exception with stack trace.

• #​13077 This fix makes EMQX only read action configurations from the global configuration when the connector starts or restarts, and instead stores the latest configurations for the actions in the connector. Previously, updates to action configurations would sometimes not take effect without disabling and enabling the action. This means that an action could sometimes run with the old (previous) configuration even though it would look like the action configuration has been updated successfully.

• #​13090 Attempting to start an action or source whose connector is disabled will no longer attempt to start the connector itself.

Gateways

• #​12909 Fixed UDP listener process handling on errors or closure, The fix ensures the UDP listener is cleanly stopped and restarted as needed if these error conditions occur.

• #​13001 Fixed an issue where the syskeeper forwarder would never reconnect when the connection was lost.

• #​13010 Fixed the issue where the JT/T 808 gateway could not correctly reply to the REGISTER_ACK message when requesting authentication from the registration service failed.

Breaking Changes

• #​12947 For JWT authentication, a new boolean option disconnect_after_expire has been added with default value set to true. When enabled, the client will be disconnected after the JWT token expires.

  Previously, the clients with actual JWTs could connect to the broker and stay connected even after the JWT token expired. Now, the client will be disconnected after the JWT token expires. To preserve the previous behavior, set disconnect_after_expire to false.

• #​12957 Stopped building packages for macOS 12.

---

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR has been generated by Mend Renovate. View repository job log here.

[coderabbitai]

Important

Review Skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to full the review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.