Bulk messages sent through Logstash are stored a message

Elastic Stack Logstash
1

Hello Everyone,

I developed a powershell script to send message in bulk to Elastiseach using the Invoke-RestMethod command.

When I send a bulk request directly to Elasticsearch, I can see the properties of my json object stored in the source field.

However, when I use logstash in the middle, the json object gets stored in the message field as string.

I`ve already read this discussing here, but I am still confused.

How I can send bulk messages through Logstash so that they I will get correctly store on ElasticSearch?

Thank for you help

2

I forgot to mention how I am sending data through logstash

Here is the powershell command I am using to send messages:

Invoke-RestMethod -Uri ($uri + '/_bulk') -Method POST -ContentType 'application/x-ndjson' -Body $jsonObjectToSend

3

I'm assuming you're using an http input. Set its codec option to json.

4

Yes, I am. But I have already solved this problem.

input {
http {
port => "5001"
codec => "json"
}
}
filter{
json {
source => "message"
remove_field => ["message"]
}

    mutate{
            remove_field => ["headers","message"]
    }

}

output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "logstash-windows-filereport-%{+YYYY.MM.dd}"
user => "username"
password => "password"
}
#stdout{}
}

1 Like
5

Thank you

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.