Adding fulltext search to mongodb

Elastic Stack Elasticsearch
1

We are trying to develop a strategy for using elasticsearch for full-
text searching on our mongodb instance. It would appear that every key
that we want to use as a filter must be included in elastics index.
Potentially we could want to use every key in mongo as a filter - i.e.
full-text search on description, filter by date and telephone number.

Does anyone have any real-world experiences of adding full-text to
mongo with elasticsearch that they can share?

Maybe we could just use elastic for storage as well, we would
potentially need to update records as well as insert though

2

We use elasticsearch as an index for our MongoDB instance ... or
alternatively: we use MongoDB as a backing store for our elasticsearch
index :slight_smile:

I think I gave some details of what we did in a different thread a
couple of months back (not that it's particularly noteworthy/clever,
but it works fairly nicely of our purposes), you could search for
that.

But basically we convert the BSON in MongoDB into Java beans using
Gson and then write the JSON into elasticsearch (with an optional
mapping object that transforms any fields for eg performance reasons).
There is a river that will automatically sync the two from the MongoDB
log, though we don't use that for various (good and bad) reasons.
(We're open-sourcing our software soon; it might be possible to
extract out the ES/MongoDB layer as an alternative to the river then,
I'm not sure if it's worth it though).

When searching we just get an ids arrays back from elasticsearch and
turn that into in a { $in: ids } query to MongoDB. The limiting factor
(for us) is the time is takes to get back the objects from MongoDB (by
x5, we get the top 1000 documents), not the es or MongoDB searches; so
for searching for documents, it is pretty much the same as MongoDB
having a full text search.

Per-query aggregations are more problematic - for simple ones we use
elasticsearch facets and bypass MongoDB completely; for more complex
ones (eg that depend on commonly varying fields) there isn't a
particularly good solution at the moment - you just have to limit the
number of MonogDB records you aggregate over depending on required
accuracy vs performance.

Hope this helps!

On Feb 7, 11:03 am, stew s...@tripledub.co.uk wrote:

We are trying to develop a strategy for using elasticsearch for full-
text searching on our mongodb instance. It would appear that every key
that we want to use as a filter must be included in elastics index.
Potentially we could want to use every key in mongo as a filter - i.e.
full-text search on description, filter by date and telephone number.

Does anyone have any real-world experiences of adding full-text to
mongo with elasticsearch that they can share?

Maybe we could just use elastic for storage as well, we would
potentially need to update records as well as insert though