Useruuid,tracingId and correlationId field is not comming in logstash

[vikascateina (vikas)]

Not able to see Useruuid,tracingId and correlationId field in logs in logstash which is comming from mule but it is comming in message field in logstash.Below is my logstash.conf file and I have attached the screenshot also.
input {
tcp {
port => 4560
codec => json
}
}

filter {
date {
match => [ "timeMillis", "UNIX_MS" ]
}
}

output {
elasticsearch {
hosts => [ "elasticsearch-url" ]
index => "%{[application_id]}-%{[environment]}-%{+YYYY.MM.dd}"
user => "xxx"
password => "xxxxx"
}
}

[Badger]

In addition to your json codec, you have a [message] field which is JSON, so you should parse that using a json filter.

[vikascateina (vikas)]

input {
tcp {
port => 4560
codec => json
}
}

filter {
date {
match => [ "timeMillis", "UNIX_MS" ]
}

json {
source => "message"
target => "correlationId"
}
}

output {
elasticsearch {
hosts => [ "xxx" ]
index => "%{[application_id]}-%{[environment]}-%{+YYYY.MM.dd}"
user => "xxx"
password => "xxx"
}
}
The above is logstash.conf .I got correlationId.correlationId field but I only want correlationId then I have added json {
source => "message"
target => "correlationId"
}

mutate {
add_field => { "correlationId" => "%{[correlationId][correlationId]}" }
}

mutate {
remove_field => ["correlationId"]
}
}
but then also it doesnt work it was not giving value of correlationId Id.In value it is giving %{[correlationId][correlationId]}

[leandrojmp (Leandro Pereira)]

These filters are confusing:

json {
source => "message"
target => "correlationId"
}

mutate {
add_field => { "correlationId" => "%{[correlationId][correlationId]}" }
}

mutate {
remove_field => ["correlationId"]
}
}

First you re parsing the message field using the correlationId field as the target, so the json document in the message field will be nested under the correlationId.

Then you are adding a field named correlationId, which already exists as this is the target of your json filter, with the value %{[correlationId][correlationId]}, and after that you are removing the entire correlationId field.

If those filters work, you will not have any field named correlationId in your document because you are removing it.

Can you share a sample of your message so the pipeline can be replicated?

[vikascateina (vikas)]

{
"correlationId" : "xxxx",
"tracePoint" : "OUTBOUND_REQUEST_SCOPE_AFTER",
"priority" : "INFO",
"elapsed" : 111,
"scopeElapsed" : 16,
"locationInfo" : {
"lineInFile" : "1757",
"component" : "json-logger:logger-scope",
"fileName" : "xx",
"rootContainer" : "xx"
},
"timestamp" : "2024-02-12T12:47:51.550Z",
"applicationName" : "xxx",
"applicationVersion" : "1.0.3",
"environment" : "dev",
"threadName" : "xxx"
}
The above is my message which I am getting in elasticsearch but all the fields I wanted as a seperate field which I am getting correlationId.correlationId as know but I only wanted correlationId.Can you tell me what configurations should I change in logstash.conf to get this.