"LinkedIn built our early search engines on Lucene. As we grew, we evolved
the search stack by adding layers on top of Lucene. Our approach to
scaling the system was reactive, often narrowly focused, and led to
stacking new components to our architecture, each to solve a particular
problem without thinking holistically about the overall system needs. This
incremental evolution eventually hit a wall requiring us to spend a lot of
time keeping systems running, and performing scalability hacks to stretch
the limits of the system.
Around a year ago, we decided to completely redesign our platform given our
growth needs and our direction towards realizing the world’s first economic
graph. The result was Galene, our new search architecture, which has since
been implemented and successfully powering multiple search products at
LinkedIn. Galene has helped us improve our development culture and forced
us to incorporate new development processes. For example, the ability to
build new indices every week with changes in the offline algorithms
requires us to adopt a more agile testing and release process. Galene has
also helped us clearly separate infrastructure tasks from relevance tasks.
For example, relevance engineers no longer have to worry about writing
multi-threaded code, perform RPCs, or worry about scaling the system."
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firstname.lastname@example.org.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4ee6c0b9-3221-4a38-aa38-0955216cb235%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.