Hi there,
I have primarily web development background and now in business development. I like to equip myself with more knowledge in technologies I am not aware of in order to put together better solutions for my customers.
So typically we build customised solutions using django and postgres. Each postgres dump is typically ~ 20mb of data.
Now we need to deal with scenarios such as product data where the user wants the flexibility to add new attributes to the product data over time.
Product data is also split into different entities. Imagine a single SKU has Shape, Pattern, and Label as the parent entities.
SO if each entity has 3 records, that means the universe of SKU entities can go up to 27. Right now, other information being stored is the inventory of the SKU instances.
When a report needs to be done, the user wants the flexibility to pick and choose different attributes of Shape, Label, and Pattern.
For e.g.
imagine a report with 8 columns where the last column represents number of pieces of a SKU.
Each data row in the report represents 1 unique SKU.
The users want to the see in terms of the columns from left to right:
SKU Serial Number, Shape Name, Shape Serial Number, Label Name, Label Serial Number, Pattern Name, Pattern Serial Number, Quantity
and they want to be able to perform searches on this report.
We can build search using LIKE clauses.
We haven't figured out a good way to do this as well as allowing users to create new attributes over time. E.g. they might want to add a Shape Japanese Name to the Shape entity
Also the users want to be able to pick and choose what columns they want to see when they build their own reports.
Is this a good use case for ElasticSearch as a database?
if so, is it wholly move all present postgres data into elasticsearch or only partially?
By partially, I mean something like: keep the product related primary keys in postgres rdbms so that serves as the source of truth for primary keys. But the attribute data is stored into the elastic search?
We havent figured out a good way to allow users to add product related attributes on the fly as that goes against the paradigm of django and postgres. Was wondering if elasticsearch can help with that as well.
Thanks for your time. Apologies if these high level use-case questions have been asked and answered elsewhere. I have tried searching around but not found what I am looking for.