How to mark an event as "Looked At" in Logstash

Elastic Stack Logstash
1

So I am querying Logstash with this query, which returns all the things off of the top of the stack:

{
"query": {
"match_all": { }
},
"size": 7,
"_source": {
"includes": [ "transport", "dest", "packet_source", "id_orig_p", "id_orig_p", "id_orig_h", "conn_state", "id_resp_h", "id_resp_p", "service", "proto" ]
} ,
"sort": [
  {
    "@timestamp": {
    "order": "desc"
  }
}
] 
}

However, as I query the data then I get duplicates if there is no new data coming in. Is there a way to mark each unique event id as "LookedAt" with an integer field or something?

I need to do this to reduce the garbage collection of my queries in C#, but storing all the ID's and doing string comparisons is pretty heavy, and seems unnecessary when I have Logstash.

2

Hmmm. Even though your question is Logstash-related, this is more appropriately asked in the Elasticsearch discussion group, as it is about an Elasticsearch query.

The short answer is no, however. What you are describing would require either:

  1. Writing an extra boolean field to each "read" event, which is a delete-then-reindex operation for each document altered, or
  2. Pulling all results to a temporary space, and then tracking what has been read locally, by whatever means are available in that local space.