I have a requirement. Events will be listening on some TCP ports in Logstash. using a filter plugin I will extract the username field from the event. After that, I have to query another API server providing that username and get the output back and store it in Elastic. What are the things that I have to look to build this pipepline? It can be done through logstash?
Yes, this can be achieved using LogStash without having to use pipeline to pipeline communication. you need:
- TCP input plugin to get the logs
- Grok filter to extract the username
- HTTP filter to call the API server
- optionally, Mutate filter to remove uninteresting fields
- Elasticsearch Output to send the results to ElasticSearch
Thank you for your reply. I thought the HTTP plugin is an output plugin. I was mistaken. Thanks for your suggestion.