We currently have a lot of data sitting in numerous tables that
describe the entity that is queried, or paged (like a catalog would
be). The querying part is however very brute force and I felt it could
benefit from the use of a search engine like ES.
The one decision I'm trying to make however is whether to further
extrapolate the existing SQL tables and 'massage' the somewhat raw
format of the data into other tables that would ease the burden on SQL
queries, or whether ES would be completely up to the task itself, of
being used to search, and also to simply page through a list of
entities based on various filtering criteria (so somewhat like a
search... essentially a filter is like a 'saved search' I guess).
But I'm not sure whether to introduce the intermediate SQL tables as
well as to use ES, or whether to simply just use ES and the SQL tables
are there as backup (if you will) of the data, and to go directly to
certain data once ES has produced its results.
I'm not sure if I've explained myself well, but if I have, does
anybody have any insights/advice on this?
Thanks in advance,