Simple way to test output of data getting written to local Kibana REST endpoint


#1

I have what i think a simple Docker logstash setup and im trying to troubleshoot why data isnt getting to my endpoint (Kibana)...I am running in debug mode and not seeing any errors so wanted to get some direction on how i could troubleshoot this.

  • config/logstash.yml

node.name: test_1
pipeline.id: main
pipeline.workers: 1
pipeline.batch.size: 15
pipeline.batch.delay: 50
path.config: /usr/share/logstash/pipeline
log.level: debug
log.format: json

  • pipeline/logstash.conf

input {
generator {
lines => [
"line 1",
"line 2",
"line 3"
]
# Emit all lines 3 times.
count => 20
}
file {
path => ["/usr/share/logstash/log/test_json.log"]
}
}

output {
http {
url => "https://in.datalens.nordstrom.com/nonprod/kafka/topics/esb-proton-logs"
http_method => "post"
format => "json"
headers => {"X-API-Key" => "Mk8hzwid84cZfWARfPAhsubZ3f7_Q96aFXwm"}
}
}

  • test_json.log
    {"employees":[
    { "firstName":"John", "lastName":"Doe" },
    { "firstName":"Anna", "lastName":"Smith" },
    { "firstName":"Peter", "lastName":"Jones" }
    ]}

#2

switched to the generator input and now seeing some connection to endpoint failures which gets me on a path to troubleshoot...good to go.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.