This may or may not help, but the following worked well for me. Just as any
database-backed application, the business logic (such as what you
described) is best implemented outside of the database. Since ES is a
first-class Java citizen and its Java API is clean and superb
(documentation shortcomings are more than made up for by the excellent
advice from this newsgroup), this is what I did:
Create my business logic, and use the TransportClient to contact
Elasticsearch. It's easy to test it in a stand-alone driver.
Create a server using Netty and the LMAX Disruptor. Netty is awesome,
and I could craft the interface to accept the questions that made sense to
the business, not to Elasticsearch. Then I instantiated an LMAX Disruptor
in multi-producer mode, and each Netty thread simply created an event that
contained 2 objects: The decoded HTTP request parameter, and the reference
to the response output channel.
The Disruptor contained a fixed number of Worker handler threads (to
match those in Elasticsearch) that actually converted the request into one
or more Elasticsearch requests, performing whatever functions the business
logic required, and then writing the responses back to the response channel.
Even if my server got a huge burst of requests, they were quickly queued
which means that the Netty handler thread count stays low. And then the
back-end Disruptor thread max will ensure that Elasticsearch is never
This is kind of the hard way using Java, but it suited me fine. Other
solutions could be similarly implemented with comperable performance but
perhaps a lot less development time in Python with Django, or Node.js, or
Ruby on Rails. But Java worked well for me; Netty is fast and easy to work
with, and the LMAX Disruptor offers a near-zero latency nearly lockless
queuing, and together they keep the thread count low while still handling
many many more requests at one time.
This sounds like a lot of work, but the hard work is a one-off kind of
thing, and then the business logic can be easily tuned, fixed, and extended
without much effort. Even things that Elasticsearch does not do and will
never do (such as foreign keys and enforced consistency across types or
even indices) are relatively easily done within the custom business logic
of a front-end server.
Again, I don't offer this as a strong suggestion but rather as a case study
of what worked well for me.
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to email@example.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a5011573-970d-4d4c-839d-a274e4d4cc34%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.