Remove field after parsing json

Elastic Stack Logstash
1

Hello

i need help please

this is my logstash.conf

<
filter {

if "jaeger" in [message] {
	drop { }
}


mutate {
	gsub => ["message", '"message":', '"containerlogs":']
	remove_tag => [ "pfd", "middle", "fluentd", "beats_input_codec_plain_applied" ]
}

grok {
	match => { "message" => "^%{SYSLOGTIMESTAMP:integration_date} %{HOSTNAME:host_caas} fluentd %{GREEDYDATA:json_message}$"}
}

mutate {
	remove_field => [ "message" ]
}

json {
	source => "json_message"
	target => "message_json"
}


if ("" in [message_json][containerlogs]){
	mutate { add_tag => "text" }   
	mutate {
		rename => {"[message_json][containerlogs]" => "[message_json][containerlogstext]"}
	}
}else{
	mutate { add_tag => "json" }
}

if [message_json][containerlogs][application] == "ApiGateway-Hydra" {
	grok {
		match => { "[message_json][containerlogs][msg]" => "%{IP:access.http_request.host} %{USER:access.http_request.identite} %{USER:access.http_request.userid} \[%{HTTPDATE:access.http_request.timestamp}\] \"%{WORD:access.http_request.method} %{NOTSPACE:access.http_request.path} %{NOTSPACE:access.http_request.http_version}\" %{NUMBER:access.http_response.status} %{NUMBER:access.http_response.size} %{NUMBER:access.http_response.time}"} 

		add_tag => ["access", "apigateway-hydra" ]
		}
	if "_grokparsefailure" in [tags] {

	grok {	
		match => { "[message_json][containerlogs][msg]" => "%{GREEDYDATA:application_message}$"}
		
		add_tag => ["application", "apigateway-hydra" ]
		remove_tag => "_grokparsefailure"
			}
	}
	}

}
/>

after the filed result in kibana is something like that:

message_json.containerlogs.logger
message_json.containerlogs.level
message_json.kubernetes.container_name

i would like to remove field message_json.containerlogs and message_json.kubernetes just keep logger, level and container_name

How can i do?

Thanks

2

Just use another mutate filter.

mutate {
    remove_field => [ "[message_json][containerlogs][logger]",  "[message_json][containerlogs][level]",  "[message_json][kubernetes][container_name]"]
}
3

Thanks for your answer.
I don't want remove my filed but i want to rename it.

i have a lot line who begin by message_json.containerlogs or message_json.kubernetes.container_name

i don't want rename one by one.

i hope they have a command to do that

Thanks

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.