Hey, I am using pagination and i want to filter duplicated doc related to one field. For the moment i am trying it with Collapse functionality to filter duplicated and with Cardinality aggregation to get the total unique files. But it is not working as expected.(Since i have docs with same name and different values for isValid flag. I get the correct total number of unique docs from cardinality, but latest pages of pagination have no results(Probably Collapse provide less results). Is there a way how to fix it to have the same number of doc from collapse and the same number of dos from cardinality ? Or is there another way, ( even without using Collapse ) to get the unique docs with pagination ?
const coreSearch = {
index: 'images',
body: {
query: {
bool: {
must: [
{ term: { type: 'png' } },
{ term: { isValid: false } }
]
},
},
collapse: { field: 'name.keyword' },
from: from,
size: 50,
aggs: {
total_files: { cardinality: { field: 'name.keyword' } }
}
},
}