in_http: use 'tag_key' option when json array is received

[imankurpatel000]

When a json array is received by http input, it doesn't use the 'tag_key' option and always sets the tag to 'http.N'. So this fixes the bug and also adds test cases to test for both json object and json array.

Fixes #9589

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A] Run local packaging test showing all targets (including any new ones) build.
• [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• [N/A] Documentation required for this feature

Backporting

• Backport to latest stable release.
  We definitely want to backport this but I would like to confirm first if the PR looks ok before creating new release PR.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[imankurpatel000]

Example configuration:

[INPUT]
    name http
    listen 0.0.0.0
    port 8888
    tag_key key1

[OUTPUT]
    name stdout
    match *

Log output from testing the change:

> fluent-bit -c fluent-bit.conf 
Fluent Bit v3.2.0
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _           _____  _____ 
|  ___| |                | |   | ___ (_) |         |____ |/ __  \
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __   / /`' / /'
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / /   \ \  / /  
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /.___/ /./ /___
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/ \____(_)_____/


[2024/11/13 09:25:48] [ info] [fluent bit] version=3.2.0, commit=50512ed56a, pid=68056
[2024/11/13 09:25:48] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/11/13 09:25:48] [ info] [simd    ] disabled
[2024/11/13 09:25:48] [ info] [cmetrics] version=0.9.9
[2024/11/13 09:25:48] [ info] [ctraces ] version=0.5.7
[2024/11/13 09:25:48] [ info] [input:http:http.0] initializing
[2024/11/13 09:25:48] [ info] [input:http:http.0] storage_strategy='memory' (memory only)
[2024/11/13 09:25:48] [ info] [output:stdout:stdout.0] worker #0 started
[2024/11/13 09:25:48] [ info] [sp] stream processor started
[0] value1: [[1731489957.918200469, {}], {"key1"=>"value1", "key2"=>"value2"}]

Json array sent to http input:

curl -d '[{"key1":"value1","key2":"value2"}]' -XPOST -H "content-type: application/json" http://localhost:8888

[imankurpatel000]

Valgrind output

fluent-bit ➜ /workspaces/fluent-bit/build (master) $ valgrind --leak-check=full ./bin/fluent-bit -c fluent-bit.conf
==77861== Memcheck, a memory error detector
==77861== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==77861== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info
==77861== Command: ./bin/fluent-bit -c fluent-bit.conf
==77861== 
Fluent Bit v3.2.0
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _           _____  _____ 
|  ___| |                | |   | ___ (_) |         |____ |/ __  \
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __   / /`' / /'
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / /   \ \  / /  
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /.___/ /./ /___
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/ \____(_)_____/


[2024/11/13 10:27:43] [ info] [fluent bit] version=3.2.0, commit=50512ed56a, pid=77861
[2024/11/13 10:27:43] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/11/13 10:27:43] [ info] [simd    ] disabled
[2024/11/13 10:27:43] [ info] [cmetrics] version=0.9.9
[2024/11/13 10:27:43] [ info] [ctraces ] version=0.5.7
[2024/11/13 10:27:43] [ info] [input:http:http.0] initializing
[2024/11/13 10:27:43] [ info] [output:stdout:stdout.0] worker #0 started
[2024/11/13 10:27:43] [ info] [input:http:http.0] storage_strategy='memory' (memory only)
[2024/11/13 10:27:43] [ info] [sp] stream processor started
[0] value1: [[1731493675.531346176, {}], {"key1"=>"value1", "key2"=>"value2"}]
[0] value1: [[1731493678.814109220, {}], {"key1"=>"value1", "key2"=>"value2"}]
^C[2024/11/13 10:28:04] [engine] caught signal (SIGINT)
[2024/11/13 10:28:04] [ warn] [engine] service will shutdown in max 5 seconds
[2024/11/13 10:28:04] [ info] [engine] service has stopped (0 pending tasks)
[2024/11/13 10:28:04] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2024/11/13 10:28:04] [ info] [output:stdout:stdout.0] thread worker #0 stopped
==77861== 
==77861== HEAP SUMMARY:
==77861==     in use at exit: 0 bytes in 0 blocks
==77861==   total heap usage: 2,027 allocs, 2,027 frees, 2,056,491 bytes allocated
==77861== 
==77861== All heap blocks were freed -- no leaks are possible
==77861== 
==77861== For lists of detected and suppressed errors, rerun with: -s
==77861== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

[milon619]

Any chance this can be reviewed and the bugfix released? This is kind of blocking from going to the latest version of FluentBit.

[cosmo0920]

Basically, this change is reasonable.
I found a tiny needed to improve point.
Could you follow this suggestion?

[ajschmidt8]

@imankurpatel000, thanks for working on this!

@cosmo0920, thank you for reviewing the PR quickly. I'm hopeful that this fix can be merged and deployed soon. my team really needs this fix published.