# Data Storage **URL:** \ **Category:** Elasticsearch\ **Created:** [September 21, 2022, 9:23am UTC](https://discuss.elastic.co/t/data-storage/314846 "2022-09-21T09:23:35Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![ibtissem\_Ben\_lafi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibtissem_ben_lafi/32/110401_2.png) [@ibtissem\_Ben\_lafi](https://discuss.elastic.co/u/ibtissem_Ben_lafi)\ **Post date:** [September 21, 2022, 9:23am UTC](https://discuss.elastic.co/t/data-storage/314846/1 "2022-09-21T09:23:35Z")
Hello eveyone, I have an issue: The size of the data is not the same in MySQL and Elasticsearch . What can I do for this problem? Someone help me, please ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 21, 2022, 9:51am UTC](https://discuss.elastic.co/t/data-storage/314846/2 "2022-09-21T09:51:26Z")
Welcome to our community! 😃 Why is that a problem? ---
**Author:** ![ibtissem\_Ben\_lafi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibtissem_ben_lafi/32/110401_2.png) [@ibtissem\_Ben\_lafi](https://discuss.elastic.co/u/ibtissem_Ben_lafi)\ **Post date:** [September 21, 2022, 10:04am UTC](https://discuss.elastic.co/t/data-storage/314846/3 "2022-09-21T10:04:09Z")
thank you for reply That is a problm because the size of data in mysql 500 gb in 6 months (in streaming) but in the elasticsearch just a week the size of data is 418gb that is too much ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 21, 2022, 10:07am UTC](https://discuss.elastic.co/t/data-storage/314846/4 "2022-09-21T10:07:17Z")
The data structures are completely different though. To get a better idea of what is happening; - What is the mapping of the index? - What is the output of `_cat/indices/yourindexname?v`? - What version of Elasticsearch are you running?? ---
**Author:** ![ibtissem\_Ben\_lafi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibtissem_ben_lafi/32/110401_2.png) [@ibtissem\_Ben\_lafi](https://discuss.elastic.co/u/ibtissem_Ben_lafi)\ **Post date:** [September 21, 2022, 10:24am UTC](https://discuss.elastic.co/t/data-storage/314846/5 "2022-09-21T10:24:32Z")
> [@warkolm](#): > > What is the mapping of the index? The mapping of the index are : MYSQL =\> logstash =\> elasticsearch=\> kibana. The ouput just for today (because I delete the indexes yesterday ) : HmMins8VQcWMhf1FhDfCaw 1 1 66205606 0 47.2gb 47.2gb The version of elasticsearch is version" : { "number" : "8.4.0", ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 21, 2022, 10:31am UTC](https://discuss.elastic.co/t/data-storage/314846/6 "2022-09-21T10:31:23Z")
Please don't post pictures of text, logs or code. They are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them 🙂 ---
**Author:** ![ibtissem\_Ben\_lafi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibtissem_ben_lafi/32/110401_2.png) [@ibtissem\_Ben\_lafi](https://discuss.elastic.co/u/ibtissem_Ben_lafi)\ **Post date:** [September 21, 2022, 10:41am UTC](https://discuss.elastic.co/t/data-storage/314846/7 "2022-09-21T10:41:35Z")
Thank you for help me The mapping of the index are : MYSQL =\> logstash =\> elasticsearch=\> kibana. The ouput just for today (because I delete the indexes yesterday ) : HmMins8VQcWMhf1FhDfCaw 1 1 66205606 0 47.2gb 47.2gb The version of elasticsearch is version" : { "number" : "8.4.0", ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 22, 2022, 4:25am UTC](https://discuss.elastic.co/t/data-storage/314846/8 "2022-09-22T04:25:59Z")
What does your Logstash pipeline look like? Does the document count in the index match what you would expect? ---
**Author:** ![ibtissem\_Ben\_lafi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibtissem_ben_lafi/32/110401_2.png) [@ibtissem\_Ben\_lafi](https://discuss.elastic.co/u/ibtissem_Ben_lafi)\ **Post date:** [September 22, 2022, 8:29am UTC](https://discuss.elastic.co/t/data-storage/314846/9 "2022-09-22T08:29:29Z")
![image.png](https://us1.discourse-cdn.com/elastic/original/3X/9/5/958ce0316bb474bb4fbb4385e40dc050511ec361.png) ---
**Author:** ![ibtissem\_Ben\_lafi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibtissem_ben_lafi/32/110401_2.png) [@ibtissem\_Ben\_lafi](https://discuss.elastic.co/u/ibtissem_Ben_lafi)\ **Post date:** [September 22, 2022, 8:30am UTC](https://discuss.elastic.co/t/data-storage/314846/10 "2022-09-22T08:30:04Z")
Thank you for replay ![image.png](https://us1.discourse-cdn.com/elastic/original/3X/9/5/958ce0316bb474bb4fbb4385e40dc050511ec361.png) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 22, 2022, 8:52am UTC](https://discuss.elastic.co/t/data-storage/314846/11 "2022-09-22T08:52:51Z")
Please do not post images of text as it can be hard to read and impossible to search. Is your SELECT statement using the tracking column? It is not possible to tell from the image. I also see that you are indexing the data twice into two eparate indices. This will naturally take up more space. Have you optimised your mappings along with [these guidelines](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/tune-for-disk-usage.html)? ---
**Author:** ![ibtissem\_Ben\_lafi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ibtissem_ben_lafi/32/110401_2.png) [@ibtissem\_Ben\_lafi](https://discuss.elastic.co/u/ibtissem_Ben_lafi)\ **Post date:** [September 22, 2022, 1:49pm UTC](https://discuss.elastic.co/t/data-storage/314846/12 "2022-09-22T13:49:02Z")
My SELECT : SELECT DISTINCT log.ID\_CALL as idcall, log.\* ,cus.NAME as cus\_clientname, cus.CODE\_CLIENT as cus\_client\_code, cus.CODE\_CLIENT\_EXTERNE as cus\_client\_code\_externe, cus.COUNTRY as cus\_client\_country, cus.COUNTRY\_CODE as cus\_client\_country\_code, pl.PREFIXE as pl\_prefixe, pl.COUNTRY as pl\_country, pl.COUNTRY\_ZONE as pl\_country\_zone, cm.MODE as cli\_mode\_name FROM LOG\_CDR\_TODAY AS log, CONFIG\_CUSTOMER as cus , CONFIG\_PRICE\_LIST as pl, CLI\_MODE AS cm WHERE cus.ID\_CUSTOMER=log.ID\_CUSTOMER AND log.ID\_PRICE\_LIST=pl.ID\_PRICE\_LIST AND cm.ID=log.CLI\_MODE TRAKING COLUMN : "idcall" yeah I have two outputs the first for the history and the second is just for today with cronjob ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [September 22, 2022, 2:01pm UTC](https://discuss.elastic.co/t/data-storage/314846/13 "2022-09-22T14:01:19Z")
I have not used this plugin in quite some time, but it seems to me like you have configured a numeric tracking column but not added any condition in the WHERE clause where you only select `idcall > :sql_last_value` as shown in the example [here](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html#_predefined_parameters). I therefore believe you might be index all the data every time the plugin runs. As you are not specifying document IDs in the Elasticsearch output plugins you should be able to search for a `idcall` value and see how many documents you have in Elasticsearch. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 20, 2022, 2:01pm UTC](https://discuss.elastic.co/t/data-storage/314846/14 "2022-10-20T14:01:52Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.