Removal of duplicate based on some condition


(Sundara Senthil) #1

We have millions of rows as part of elastic, in that we found many duplicate IDs, we want to clean it up
We use Go Elastic library https://github.com/olivere/elastic

We have query to first 10000 record to return duplicate
agg := elastic.NewTermsAggregation().Field("_id").MinDocCount(2).Size(10000)
search = client.Search("employee")
search = search.Aggregation("myterm", agg)
sr, err := search.Do(context.TODO())

but we want this query to run for all millions of record we have and we want to get use of scroll API of elastic for it....

Can any one please help us


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.