Pipeline: copy unique value to a new index

I guys, I need an help!
I have an index called s1v6_new_cleaninq2 with more then 1 million of documents. Inside each documents I have a field called "num_pratica" and this field could be repeated for a couple of documents.
Here you can find a screenshot of my "Discover" page:

I'm trying to create a pipeline to copy and paste (in a new index called "s1v6_new_cleaninq2_prtuniche") the documents with unique "num_pratica".
So I would like to find only one document for each num_pratica different value.

This is my pipeline:

input {
  elasticsearch {
      hosts       => ["http://localhost:9200"]
      index       => "s1v6_new_cleaninq2"
  }
}

filter {
  elasticsearch {
    hosts                     => ["http://localhost:9200"]
    index                     => "s1v6_new_cleaninq2_prtuniche"
    query                     => "num_pratica:%{num_pratica}"
    add_tag                   => "not_insert"
  }
}

output {
  if "not_insert" not in [tags] {
    elasticsearch {
      hosts       => ["http://localhost:9200"]
      index       => "s1v6_new_cleaninq2_prtuniche"
    }
  }
}

But I can find several hits for same "num_pratica" as you can see in the screenshot:

Could you please help me?

Thanks!!
Salvo

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.