Output conditions with logstash plugin http_poller

Elastic Stack Logstash
1

I would like to implement a conditional output to Elasticsearch index using logstash http_poller input plugin. I am using a GET request and sending the response to the index only if the response is not already in the index:

Read LastMsg every 10 seconds
If LastMsg not in Index:
  write LastMsg in Index

For now, I implemented the input section and I would like to develop the output section

input {
  http_poller {
    urls =>   {
    api_read_msg ==> {
    method ==> get
    url ==> "https://api.sendgrid.com/v3/messages?limit=1"
    headers ==> {
      Authorization ==> "Bearer ${API_KEY}",
      Content-Type ==> "application/json"
      }
    }
    tags ==> "sendgrid"
    request_timeout ==> 60
    schedue ==> { cron ==> "*/10 * * * * * UTC"}
    coedc => "json"
    metadata_target ==> "http_poller_metadata"
    }
  }
}
output {
  elasticsearch {
    hosts    => [ '${ES_HOST_ADDRESS}' ]
    user     => '${ES_USER}'
    password => '${ES_PASSWORD}'
    ssl => "true"
    index => "my_index"
  }
  stdout {
    id => "Pipeline_read_api"
    codec => json
  }
}

Has any one tried to perform such action ?
Thanks,

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.