Hello, I have loaded 6lakhs+ of data from database into elasticsearch using logstash. Observed the count of the number of rows in database is equal to the number of records in elasticsearch(as highlighted in yellow color below ). But observes some data is missing when validating group by query wit…

[image] balumurari1: I have 28 columns in a table. Does this table not have a primary key (even a compound primary key)? If not, then as I stated before, the only way to guarantee order is to specify all selected columns in your ORDER BY clause.

Data mismatch in elasticsearch when loading data from database using logstash

Elastic Stack Logstash

balumurari1 January 4, 2019, 9:28am 17

Hi @yaauie,

I have tried by specifying all the 28 columns in ORDER BY clause but observes data mismatch is occurring again.
Can I add a column to SQL statement to get row number and use it to order by rownum.

can you please suggest me to eliminate the issue.

Waiting for your response,
Thanks inadvance.

Why I am getting this error when using JDBC input plugin of Logstash

Topic		Replies	Views
Log stash not loading exact number of records in elasticsearch and on every hit results are changing Logstash	2	479	February 17, 2021
Elastic document count mismatch - SQL to ES Elasticsearch	4	1408	December 13, 2017
Logstash to elasticsearch Logstash	4	372	May 18, 2018
Logstash JDBC missing many documents Logstash	0	88	June 5, 2024
Logstash - not pushing all data into elasticsearch Logstash	3	754	October 5, 2020

Data mismatch in elasticsearch when loading data from database using logstash

Related Topics