# Mysql database shows only latest record **URL:** \ **Category:** Elasticsearch\ **Created:** [May 22, 2018, 1:20pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831 "2018-05-22T13:20:08Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![mamta](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@mamta](https://discuss.elastic.co/u/mamta)\ **Post date:** [May 22, 2018, 1:20pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/1 "2018-05-22T13:20:08Z")
Hi, I am using ELK 5.6.8 version. My ELK is running on RedHat machine. When I first run logstash.conf for MySQL database then I was able to see all the records of one table but now I am not able to see all records of one able. Kibana shows only last/latest row data. When I run logstash.conf file from the terminal on terminal I am able to see all the records of one table but on kibana only last records. When I used curl command as shown as below for elasticsearch to see MySQL index data then It shows only that last records which show on kibana. curl -XGET '[http://localhost:9200/mysql/\_search?pretty](http://localhost:9200/mysql/_search?pretty)' How can I Identify whether elasticsearch is able to store all the records or it is storing only last records? ---
**Author:** ![eedugon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eedugon/32/45404_2.png) [@eedugon](https://discuss.elastic.co/u/eedugon)\ **Post date:** [May 22, 2018, 7:51pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/2 "2018-05-22T19:51:21Z")
Hi @mamta, By default, the \_search API will show you only 10 hits if you don't specify otherwise. But in the response you should also be able to see the total number of `hits` that the search returned. Take a look to search URI and BODY requests in the documentation. [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-uri-request.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-uri-request.html) And you can always take a look to the stats API to see general information about the index. [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html) In your case it would be something like :9200/mysql/\_stats There you should be able to see the amount of docs of your index for example, and you could compare it with your expectations from logstash. Another possible explanation for not having all the documents in elasticsearch could be this: - If you have ever deleted your index in elasticsearch maybe logstash hasn't sent all the information back (as the information was already sent). in such case I would recommend to force logstash to send again all the info. Regards! Eduardo ---
**Author:** ![mamta](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@mamta](https://discuss.elastic.co/u/mamta)\ **Post date:** [May 23, 2018, 11:10am UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/3 "2018-05-23T11:10:27Z")
Okay, let me rephrase the question. I have a config file to connect to a MySQL database and retrieve all the records from a table. The table only contains 15 records. However, only the latest record (only one record) is being pulled and displayed from the table . ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 23, 2018, 11:13am UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/4 "2018-05-23T11:13:32Z")
What does your config file look like? Are you by any chance setting the document ID based on a parameter that does not exists, causing multiple updates of the same document? (You can check this by looking at the ID of the document you can see through Kibana.) ---
**Author:** ![mamta](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@mamta](https://discuss.elastic.co/u/mamta)\ **Post date:** [May 23, 2018, 12:15pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/5 "2018-05-23T12:15:45Z")
I am uploading my logstash.conf file.I am not able to understand the problem. When I run conf file first time then kibana was able to show all the 15 records but now when I run it, it shows only latest record. Thank you. ![conf](https://us1.discourse-cdn.com/elastic/original/3X/d/d/ddb675e9fbd830c2f8acd80f16bca92e00931937.png) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 23, 2018, 12:34pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/6 "2018-05-23T12:34:13Z")
When you look at the output to `stdout` are fields in the document upper-case? ---
**Author:** ![mamta](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@mamta](https://discuss.elastic.co/u/mamta)\ **Post date:** [May 23, 2018, 12:40pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/7 "2018-05-23T12:40:58Z")
I am not able to get your question? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 23, 2018, 12:43pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/8 "2018-05-23T12:43:14Z")
Can you show me what the record you see looks like in Kibana? ---
**Author:** ![mamta](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@mamta](https://discuss.elastic.co/u/mamta)\ **Post date:** [May 23, 2018, 12:44pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/9 "2018-05-23T12:44:17Z")
sure!! May I ask you one help from you? ---
**Author:** ![mamta](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@mamta](https://discuss.elastic.co/u/mamta)\ **Post date:** [May 23, 2018, 12:47pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/10 "2018-05-23T12:47:22Z")
Is it possible you to join me on zoom? That will help me to understand the problem and will give you more clearification. Thank you ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 23, 2018, 12:49pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/11 "2018-05-23T12:49:47Z")
> [@mamta](#): > > Is it possible you to join me on zoom? No, that is not possible. I prefer discussions to be public so other users can benefit. ---
**Author:** ![mamta](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@mamta](https://discuss.elastic.co/u/mamta)\ **Post date:** [May 23, 2018, 12:52pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/12 "2018-05-23T12:52:16Z")
Ok. I am uploading kibana output of mysql database. ![kibana_output](https://us1.discourse-cdn.com/elastic/original/3X/0/2/02365a38cfb630f822a20503299ae425b23083b4.png) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 23, 2018, 1:03pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/13 "2018-05-23T13:03:54Z")
As you can see there, all field names in the document are lower-cased, which is why `document_id => "%{LOG_ID}"` is not working, which causes the same document to be updated repeatedly. Change this to `document_id => "%{log_id}"` and it should work. ---
**Author:** ![mamta](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@mamta](https://discuss.elastic.co/u/mamta)\ **Post date:** [May 23, 2018, 1:10pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/14 "2018-05-23T13:10:39Z")
Let me try now. Will you let you know. Thank you. ---
**Author:** ![mamta](https://avatars.discourse-cdn.com/v4/letter/m/73ab20/32.png) [@mamta](https://discuss.elastic.co/u/mamta)\ **Post date:** [May 23, 2018, 1:26pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/15 "2018-05-23T13:26:16Z")
Hey, That's worked for me. Thank you so much. I have one more question. I have 4-5 conf file for different logs so every time I have to run logstash conf file for current log. I want to continuously run the conf file in the backend. I have tried to run from bin#./logstash -f logstash.conf and also tried to run from /etc/logstash/confd directory but not successful. I also tried nohup and crontab to run conf file in the backend but at a time only one file is running continuously. Thank you so much. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 20, 2018, 1:26pm UTC](https://discuss.elastic.co/t/mysql-database-shows-only-latest-record/132831/16 "2018-06-20T13:26:19Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.