Hi,
we´re using servicenow.com system and I want to have stats of service-now tickets (INC|REQ) in our ELK. Service-now have an API, but how can I retrieve only new tickets since last sync? Anybody got an idea?
Firstly, you cannot use an http_poller input since you cannot modify the request parameters, so unless you are willing to fetch the entire database on each run you will have no way of identifying new tickets.
I would use an exec input running /bin/true with a schedule so that you can get an event down the pipeline every 10 minutes (or however often works for you). Then configure an http filter to run the query against the SNOW API.
If your query risks producing duplicates, then use the incident id as the document_id in elasticsearch, so that those duplicates overwrite the existing documents for those incidents.
If you are running the request every 10 minutes you could run a query that requests tickets for the last 10 minutes. This will result in missed data when logstash is down, but it is pretty simple to do. Use a ruby filter to get Time.now and strftime into whatever format SNOW uses. Then add a 'date after' item to the query filter.
Alternatively, you can do it properly and persist the last value you requested. You could borrow code for this from an existing input plugin. I would recommend against trying to borrow from the jdbc plugin because that is complicated and spread across several files. Perhaps the s3 input where the functionality is basic and the code is all in one place.
Any approach to this is going to require you to write ruby code.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.