Device with json log - how to import

Elastic Stack Logstash
1

I have a device on my network that creates a json log file. Lets call this device device1. So if you curl https://device1/log.json you will get an output in json format. I want to be able to "pull" that data into elasticsearch via some method. I could do a curl and dump the file but then in 1 minute more data is added (and eventually the file becomes quite large). The device claims to have a json API but I am new to this and can not find good documentation as to how to connect to the device and grab the last updates to the log files. Any thoughts or direction would be awesome.

Thanks,
Brad

2

Hi,

You may want to use the http poller input:

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.html

But you'll need to determine what kind of url you specify which returns the "latest" log based on your device.

3

The log is continuous. I don't know that it ever rolls over so getting the latest isn't the issue but when the log file is huge and the poller is having to load the whole thing every time it gets a bit excessive. I was hoping to be able to somehow "talk" with the device and have it send only the logging information for the last X minutes/hours/days. I have an e-mail into the manufacturer asking for more information about the API but have not heard anything back. Hopefully they have a better way to do this.

Brad