I'm currently figuring out if we can use Logstash for collecting specific data from external services we're calling with our API. Eventually to be used for alarm triggers.
I'll try to explain the case first.
Customers of ours can interact with our API, which in return calls appropriate (SOAP / REST) webservices from external companies. Depending on the need our API calls different methods from the same webservice. See image below. Our API is a java based system using the spring framework. Currently our logfiles are logged using the log4j framework.
Eventually I want to be able to do the following for every of the external webservice.
- Measure response times from the external webservice, and if the response time is exceeding x time, an alarm should be triggered (i.e email, slack). Measuring should be based on a certain operation to be called.
- Detecting if too many failure responses are returned by an external webservice. This can be Error 500 with error messages, or a certain soap response with.
So we are already using Elasticsearch just for searching through the application. And are in experimental phase of using logstash in combination with kibana for other logfiles. My main point of investigation now is to see wether or not logstash/ elastic is the appropriate way to go.
So for point 1 I was thinking that I can use the beats framework, i.e PacketBeat? With this to measure the response times with a specific operation from an external webservice. But i'm not sure if PacketBeats is able to do this out of the box?
For point 2 I was thinking about logging the error response to a logfile, a logfile for each specific operation. These logfiles can then be sent out to logstash and then transform the specific data to filter on things like http error codes, maybe an application specific error. In either way, wether REST or SOAP
For the actual alarming part I'm thinking I'm able to use Alerting to specify the rules.
I'm also curious if somebody has a similar case. And how you solved it.