I'm using couchdb for storage for a Node.js app I've written. I was using
the couchriver plugin to get data into Elasticsearch from couch, but I
wanted to perform some substantial manipulation of the different document
types before they were sent to ES so I wrote my own 'indexer' that listens
to the couchdb changes feed, munges the documents and then sends them to ES.
One area of my application queries ES to display a table of users. Each row
in the table has a 'delete' link to delete the user. When I click the link
I'm taken to a separate confirmation page. When I confirm the deletion my
application performs a 'soft delete' in couchdb. As soon as the 'soft
delete' is recorded by couchdb, my application redirects to the page that
displays the table of users again.
I see a console message telling me my indexer ran immediately after the
soft delete and before any data is sent to my application. That should
mean the updated copy of the document has been put into ES, but I can't be
certain on the timing of everything. When my application queries ES for a
list of users, it filters out documents that have been soft deleted so I
shouldn't get the soft deleted document back.
However, when my application redirects to the table of users and queries ES
immediately after a soft delete, ES sends the document I soft deleted back
for some reason. All subsequent queries to ES do not return the soft
deleted document though. I suspect what's happening is the indexer is
sending the change to ES and ES is still processing the change when the
request for the updated user list comes in. ES answers the query, finishes
processing the change, and all future requests work properly. I'm not sure
if that's right though.
Have you run into this before? I'm considering my options for getting
around this. Fundamentally, having two sets of data, one in couch and one
in ES, is less than ideal. Right now the best solution seems to be to get
rid of the indexer and have my application manually record changes in the
database and ES before it redirects. Does the river have a way of dealing
with this issue?
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to email@example.com.
For more options, visit https://groups.google.com/groups/opt_out.