Commits on Nov 29, 2022

1. feat!: add new kafka backend and refactor metrics sender engine …

   The SNMPCollector is currently focused and oriented to send metrics to
   InfluxDB, using internally specific functions of InfluxDB client to
   manage and send those metrics.
   
   This PR tries to break with this dependency on metric internal management
   adopting the current telegraf metric definitions.
   With this abstracion, a new engine is created in order to send those
   generic metrics to specific backends in a centralized way.
   
   The new metric sender engine also adopts the current telegraf buffer. The metrics
   are now stored on the buffer instead of being written each time that
   a measurement produces a small set of them.
   
   The proposed engine is changed and it is based on the current telegraf
   way: a ticker is defined and the metrics are flushed on several batches,
   reducing drastically the number of request that are being done and with
   the change to control the current buffer size and make operations on top
   the output.
   
   Since all the metrics are stored on a top-level buffer, the send process
   is being done by the backend implementation. This PR redefines the
   current influxdb to be a backend that only performs the connection, write
   close.
   
   The internal selfmon metrics -measurement, runtime, outdb stats- are
   refactored to follow the same generic metric definition, using, as always,
   the 'default' backend defined, now, as an Output.
   
   With this behaviour, a new Kafka Backend has been created to be able to
   write metrics to several brokers using JSON output format. The code is
   copied/modified from the current telegraf.output Kafka
   
   The SNMPCollector OutDB now is related with an Output instead of
   InfluxDB Server. All the management of Outputs and new Kafka Server
   can be done on the UI and via API. The output relation with the backend
   is, right now 1:1, and a Device can have only 1 output attached
   
   BREAKING CHANGE: the metrics are now stored on internal buffer and
   written to the final backend based on time. The write process is being
   done each FlushInterval and splitted into several request based on
   the buffer length and the MetricsBatchSize.
   
   BREAKING CHANGE: the InfluxDB UI component is being moved to a new
   section and the relation with the SNMPDevice is break and changed to an
   Output. The field BufferSize is being removed from API request and new
   DB initializtion, but mantained on old configurations to perform a
   migration process
   
   fix #343

   

   sbengo committed Nov 29, 2022
   Configuration menu

   • View commit details
   • Copy full SHA for c71c091
   • Browse repository at this point

   Copy the full SHA

   c71c091 View commit details

   Browse the repository at this point in the history