Fluentd is not able to send json logs to logstash

Elastic Stack Logstash
1

Hi,
I am trying to send json logs from application pods to logstash through fluentd agent. But i am getting one error and unable to send json logs to logstash. We mentioned the file extension in fluentd config is .json but when this extension is .log or .txt it is working fine(means the path file).
I am sharing getting error.
2024-08-21 07:30:44 +0000 [warn]: #0 pattern not matched: "{"applicationName":"abc"}}"
2024-08-21 07:30:44 +0000 [warn]: #0 dump an error event: error_class=ArgumentError error="log does not exist" location=nil tag="fluent.warn" time=2024-08-21 07:30:44.830543276 +0000 record={"message"=>"pattern not matched: "{\"applicationName\":\"abc\"}}""}
2024-08-21 07:30:44.830543276 +0000 fluent.warn: {"message":"pattern not matched: "{\"applicationName\":\"abc\"}}""}

sample json log:-
{"applicationName":"abc"}

fluentd config:-

@type tail path /opt/jenkins/deploymentlogs/output.json, pos_file /var/log/td-agent/httpd-access.log.pos, format json, time_format %Y-%m-%dT%H:%M:%S, tag *, read_from_head true, #exclude_path ["/var/log/fluent*"] #from_encoding UTF-8 #encoding UTF-8 @type json

<filter **>
@type parser
format json
key_name log
reserve_data true

<match **>
@type stdout

<match **>
@type http
time_as_integer true
http_method post
tls_client_cert_path "/etc/fluent/ssl/client.pem"
tls_private_key_path "/etc/fluent/ssl/client.key"
tls_ca_cert_path "/etc/fluent/ssl/ca.pem"
tls_verify_mode "peer"
tls_version TLSv1_2
endpoint xxxxxxxxxxxxxxxx

Could you please guide me on this?
Thanks

2

Hello,

This seems to be an issue with fluentd, which is not an Elastic tool and not supported here.

Do you have any issue on logstash side? Any log error from logstash?

For issues with fluentd you need to look for some fluentd forum.