Intersect documents with same id in two indices (5.6)

Elastic Stack Elasticsearch
1

I have two indices with documents, each document assigned with a unique id.

My goal is to find duplicate documents among the indices.

Is there any sort of an intersection aggregation that i can make in order to determine whether my indices share common id documents?

2

You could use transform, we have an example in the docs for comparing 2 indices.

1 Like
3

I forgot to mention that im using elastic 5.6

4

ok, 5.6 is fairly old and EOL since 2019-03-11, please check if an upgrade is possible:

If you can't go to 7.x:

  • >= 6.1 would give you access to composite aggs, which would let you page through aggs, in a nutshell you could write your own transform
  • >= 6.8 would allow you to use transform + CCS from a >= 7.6 cluster that queries your >=6.8 cluster
1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.