This error seems to be related to mapping issues in Elasticsearch, below error found in logstash

Elastic Stack Logstash
1

2023-11-04T10:33:48,353][WARN ][logstash.outputs.elasticsearch][main]
Could not index event to Elasticsearch. {:status=>400, :action=>["index",
{:_id=>nil, :_index=>"staging-2023.11.04", :_type=>"_doc", :routing=>nil},
#LogStash::Event:0x4fdc7ef6], :response=>{"index"=>{"_index"=>"staging-2023.11.04",
"_type"=>"_doc", "_id"=>"adjhghASjfv4456", "status"=>400, "error"=>
{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field
[kubernetes.labels.app] of type [text] in document with id 'adjhghASjfv4456'.
Preview of field's value: '{kubernetes={io/instance=cert-manager, io/component=cainjector,
io/name=cainjector, io/version=v1.5.4}, value=cainjector}'", "
caused_by"=>{"type"=>"illegal_state_exception",
"reason"=>"Can't get text on a START_OBJECT at 1:1022"}}}}}
/var/log/logstash$ PUT staging-2023.11.04

2

Welcome to the community.

Check you .conf and the template, especially the kubernetes.labels.app field, you are writing the text type for example: "[kubernetes][labels][app]" => "pod1" instead of kubernetes.labels. as JSON. Check ECS. Also check the template, might be a wrong data type dynamically created during testing.

kubernetes.labels.*

Kubernetes labels map

type: object

3

here it is my conf file

input {
redis {
host => "elk-XXXX-XXX.amazonaws.com"
id => "staging"
data_type => "list"
key => "staging"
tags => [ "staging" ]
}
}

filter {
mutate {
gsub => ["message", "\x1B[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]", ""]
}
mutate {
remove_field => [ "@version", "_score", "_type"]
}
}

4

Can you show the full message? Should be the field named: "kubernetes".

5

no we dont have that filed

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.