Json filter message filed

Elastic Stack Logstash
1

I currently have 2 different type of log formats coming in Json format.

message{
id:scenario1
ip:0.0.0.
}
message
{
id:scenario2
message: "This is a message field coming from application"
}
json {
		source => "message"		
		skip_on_invalid_json => true
		 }

Situation 1:
Does not include any message field in it.
For this: I want to remove the message field as it contains the raw json message.
Situation 2:
Includes a message field with some application data in it. For this scenario the above code works fine. It replaces the message field with the data that is coming in.

For situation 1 the raw data is getting published to elastic search. I tried writing some thing like this

json {
		source => "message"
		tartget => parsedJson
		skip_on_invalid_json => true
		 }
if ![parsedJson][message]
{
mutate {
remove_field => ["message"] ---This is raw data
}

But every thing is getting tagged with parsedjson.id and so on..
Any inputs are appreciated.

2

are both messages coming from single source ? if so , you will need a unique identifier that only exists in one of the message.

if they come from different sources, you can tag them on ingestion

3

No, both are coming from different sources to which I don’t have control over.

4

then you can tag them on ingest based on the source host, depending on how you receive them. for example if you receive with syslog, you can tag them in the input section

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.