Http_poller_plugin: Index document issue

[manojkrishna561994 (manojkrishna)]

I'm using http_poller_plugin in order to fetch the logs for every 2 mins.

Index is getting created and the data is also seen when the visualization is created, however, after completing every 1.30 mins (or so..) the data is getting vanished for almost 20 seconds considering the data size, again it is repopulating the data and the respective visualizations created with that data is also getting loaded after the mentioned time.

Is there a way to not let this data getting vanished for 20-30 sec (or so..).

Here is my config file

  http_poller {
  urls => {
      test1 => {
        method => get
        user => "xxxxx"
        password => "xxxxx"
        url => "http://0.0.0.0:3000/Message/0012365ff30af9497ebb88c35a6df756"
        headers => {
          Accept => "application/json"
        }
     }
    }
    keepalive => false
    request_timeout => 180
    connect_timeout => 60
    socket_timeout => 180
    schedule => { cron => "/2 * * * * UTC"}
    codec => "json"
    metadata_target => "http_poller_metadata"
  }
}

output {
elasticsearch{
        hosts => ["localhost:9200"]
        index => "one_7apr"
        document_id => "%{doc_id}"
        failure_type_logging_whitelist => [409]
        sniffing => false
        doc_as_upsert => true
}
  stdout {
    codec => rubydebug
  }
}```