Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

Filebeat: Readiness probe failed: kafka output doesn't support testing #316

Closed
tunguyen9889 opened this issue Oct 7, 2019 · 2 comments · Fixed by #420
Closed

Filebeat: Readiness probe failed: kafka output doesn't support testing #316

tunguyen9889 opened this issue Oct 7, 2019 · 2 comments · Fixed by #420
Labels
bug Something isn't working

Comments

@tunguyen9889
Copy link

tunguyen9889 commented Oct 7, 2019

Chart version: filebeat 7.0.1

Kubernetes version: v1.14.6-eks-5047ed

Kubernetes provider: EKS

Helm Version: v2.12.3

Example filebeat.yaml config in one filebeat pod

helm list | grep filebeat
filebeat        2               Mon Sep 30 17:34:16 2019        DEPLOYED        filebeat-3.0.0                  7.0.1 
$ kubectl exec -it filebeat-filebeat-4mz72 -c filebeat bash
[root@filebeat-filebeat-4mz72 filebeat]# pwd
/usr/share/filebeat
[root@filebeat-filebeat-4mz72 filebeat]# ls
LICENSE.txt  NOTICE.txt  README.md  data  fields.yml  filebeat  filebeat.reference.yml  filebeat.yml  kibana  logs  module  modules.d  prospectors.d
[root@filebeat-filebeat-4mz72 filebeat]# cat filebeat.yml
filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false
  prospectors:
    path: ${path.config}/prospectors.d/*.yml
    reload.enabled: false
filebeat.prospectors:
- enabled: true
  fields:
    log_type: infra_log
  paths:
  - /var/log/*.log
  type: log
- containers.ids:
  - '*'
  fields:
    log_type: container_log
  processors:
  - add_kubernetes_metadata:
      in_cluster: true
  - drop_event:
      when:
        equals:
          kubernetes.container.name: filebeat
  type: docker
http.enabled: true
http.port: 5066
output.kafka:
  bulk_max_size: 8192
  compression: gzip
  hosts:
  - kafka.endpoint:9093
  max_message_bytes: 10000000
  partition.random:
    reachable_only: false
  required_acks: 1
  topics:
  - topic: cluster-container-logs
    when:
      equals:
        fields.log_type: container_log
  - topic: cluster-infra-logs
    when:
      equals:
        fields.log_type: infra_log
xpack.monitoring:
  elasticsearch:
    hosts:
    - elasticsearch.endpoint:9200
  enabled: true

[root@filebeat-filebeat-4mz72 filebeat]# filebeat test output
kafka output doesn't support testing

Describe the bug:

Filebeat pod readiness failed if we define output.kafka and pod becomes Unhealthy.

Steps to reproduce:

  • Deploy Filebeat Helm chart with output.kafka.

Expected behavior:

Pod readiness status should pass.

Provide logs and/or server output (if relevant):

Events:
  Type     Reason     Age                     From                                                       Message
  ----     ------     ----                    ----                                                       -------
  Normal   Scheduled  8m56s                   default-scheduler                                          Successfully assigned addons/filebeat-filebeat-q54cc to ip-10-55-111-114.ap-southeast-1.compute.internal
  Normal   Pulled     8m55s                   kubelet, ip-10-55-111-114.ap-southeast-1.compute.internal  Container image "docker.elastic.co/beats/filebeat-oss:6.3.2" already present on machine
  Normal   Created    8m55s                   kubelet, ip-10-55-111-114.ap-southeast-1.compute.internal  Created container filebeat
  Normal   Started    8m55s                   kubelet, ip-10-55-111-114.ap-southeast-1.compute.internal  Started container filebeat
  Warning  Unhealthy  3m53s (x30 over 8m43s)  kubelet, ip-10-55-111-114.ap-southeast-1.compute.internal  Readiness probe failed: kafka output doesn't support testing
@jmlrt jmlrt added the bug Something isn't working label Oct 10, 2019
@VladyslavTokar
Copy link

Is there any way to make "filebeat test output" configurable? We faced the same issue with redis output.

@nativo-mshivaswamy
Copy link

Ran into the same issue.
Also tried with readinessProbe: {} and still creates the readinessProbe.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants