I guys, I need an help!
I have an index called s1v6_new_cleaninq2 with more then 1 million of documents. Inside each documents I have a field called "num_pratica" and this field could be repeated for a couple of documents.
Here you can find a screenshot of my "Discover" page:
I'm trying to create a pipeline to copy and paste (in a new index called "s1v6_new_cleaninq2_prtuniche") the documents with unique "num_pratica".
So I would like to find only one document for each num_pratica different value.
This is my pipeline:
input {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "s1v6_new_cleaninq2"
}
}
filter {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "s1v6_new_cleaninq2_prtuniche"
query => "num_pratica:%{num_pratica}"
add_tag => "not_insert"
}
}
output {
if "not_insert" not in [tags] {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "s1v6_new_cleaninq2_prtuniche"
}
}
}
But I can find several hits for same "num_pratica" as you can see in the screenshot:
Could you please help me?
Thanks!!
Salvo