Bulk messages sent through Logstash are stored a message

(Paulo Leao) #1

Hello Everyone,

I developed a powershell script to send message in bulk to Elastiseach using the Invoke-RestMethod command.

When I send a bulk request directly to Elasticsearch, I can see the properties of my json object stored in the source field.

However, when I use logstash in the middle, the json object gets stored in the message field as string.

I`ve already read this discussing here, but I am still confused.

How I can send bulk messages through Logstash so that they I will get correctly store on ElasticSearch?

Thank for you help

(Paulo Leao) #2

I forgot to mention how I am sending data through logstash

Here is the powershell command I am using to send messages:

Invoke-RestMethod -Uri ($uri + '/_bulk') -Method POST -ContentType 'application/x-ndjson' -Body $jsonObjectToSend

(Magnus B├Ąck) #3

I'm assuming you're using an http input. Set its codec option to json.

(Paulo Leao) #4

Yes, I am. But I have already solved this problem.

input {
http {
port => "5001"
codec => "json"
json {
source => "message"
remove_field => ["message"]

            remove_field => ["headers","message"]


output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "logstash-windows-filereport-%{+YYYY.MM.dd}"
user => "username"
password => "password"

(Paulo Leao) #5

Thank you

(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.