feat: add batching to otelcol log collector

batch_size is set to the same value as the metadata enricher.

timeout is set to the same value as fluent-bit's Flush.

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- feat: add batching to otelcol log collector [#2018][#2018]
 - feat: add experimental otelcol log collector [#1986][#1986]
 - feat: add option to disable pod owners enrichment [#1959][#1959]
 
@@ -33,6 +34,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 [#1992]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/1992
 [#1993]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/1993
 [#2013]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2013
+[#2018]: https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/2018
 
 ## [v2.3.1][v2_3_1] - 2021-12-14
 

deploy/helm/sumologic/values.yaml

@@ -4178,6 +4178,8 @@ otellogs:
             - filelog/containers
           exporters:
             - otlphttp
+          processors:
+            - batch
     receivers:
       filelog/containers:
         include:
@@ -4322,6 +4324,15 @@ otellogs:
     exporters:
       otlphttp:
         endpoint: http://${LOGS_METADATA_SVC}.${NAMESPACE}.svc.cluster.local:4318
+    processors:
+      ## The batch processor accepts spans and places them into batches grouped by node and resource
+      batch:
+        ## Number of spans after which a batch will be sent regardless of time
+        ## This is set to the same value as in the metadata enricher
+        ## TODO: Figure out what the optimal values should be for different configurations
+        send_batch_size: 256
+        ## Time duration after which a batch will be sent regardless of size
+        timeout: 1s
   daemonset:
     ## Set securityContext for containers running in pods in log collector daemonset
     securityContext:

tests/helm/logs_otc/static/basic.output.yaml

@@ -20,6 +20,10 @@ data:
         directory: /var/lib/storage/otc
         timeout: 10s
       health_check: {}
+    processors:
+      batch:
+        send_batch_size: 256
+        timeout: 1s
     receivers:
       filelog/containers:
         include:
@@ -60,6 +64,8 @@ data:
         logs/containers:
           exporters:
           - otlphttp
+          processors:
+          - batch
           receivers:
           - filelog/containers
       telemetry: