Best approach to use to match new data against already existing indexed data


#1

I have an items database with an elastic search index used for simple matching. I would like to leverage elastic search so that before new items are added, it can be determined if an existing, similar item already exists. An item has a name. New items being added generally have much longer and more descriptive names than existing items. Once a new item is matched to an existing item, I would like the new item's description to be included in attempts to match future items.


(Doug Turnbull) #2

Have you looked into more like this?

Do you just want to put this on autopilot? Or is ther a human in the loop? On autopilot, more like this could be risky. There's always a chance that the candidate document is not the same with minor edits. However you CAN tune more like this to be very strict (see docs above).


#3

Thanks for the pointer Doug! No, I have not seen the more like this
query. It looks promising. Yes, there will be a human in the loop to
double check the mapping and select from prospective candidates. However,
where the relevance is high, I am hoping we can get to an auto pilot
approach.


(system) #4