Complete MySQL database to Elasticsearch

Elastic Stack Elasticsearch
1

Hi Team,
I am working on a data analysis project on MySQL Database which contains around 100 tables.

Details are as below:-
Database Type :- RDBMS - MySQL
Total No. of Tables:- 100
The total Data size of Current MySQL Database:- 600 GB
Maximum number of columns in the table:- 12 Billion in Product Sales Table

I want to index my MySQL to Elasticseach where I can use Kibana for Data Visualization as well as Data Analysis platform where I can able to do report / Chart generation too.

Please tell me how to index all 100 MySQL tables to Elasticsearch to make sure everything should be work perfect as well as please let me know other scenarios where I have to take cautions.

Thanks,

2

You're asking quite a lot here, basically for us to architect an entire project for you. We're happy to help with things, but I think this might be a bit much.

Have you considered engaging an organisation that can help?

1 Like
3

If my question is bothering you then I will drop it from here.
Thanks for your earlier notice.

4

I don't think you are bothering anyone here and @warkolm's answer is accurate.
We are happy to help on any specific question you might have but we can't really answer a so generic one which is basically "how do I move from a relational model to a document oriented model?" and "how do I move from a datastore to a search engine?".

One silly answer would be: just index each table as a new index. But you will see that does not work well with the search engine and document oriented paradigm.

So, you have to ask yourself some questions that I covered at: Which is the best way to index the data from relational database

Then, when you know more about your use case, features, data model, etc. we can help about sizing and other stuff. Feel free to ask.

1 Like
5

Yes I am agree. May be the way I express my question projected wrong meaning, but My motto was to just get the idea about how to handle MYSQL DB relationship in ES to make sure Analysis should be work perfectly.

Even I am working from my side. Hope Will find some solution then will update here..

Happy together learning. :grinning::blush:

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.