Hi,
I have used the configration for logstash to dump data from MYSQL 5.5 to Elastic Search 6.1.2.
input {
jdbc {
jdbc_driver_library => "/home/ubuntu/mysql-connector-java-5.1.45/mysql-connector-java-5.1.45-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://IP:PORT/DATABASE"
jdbc_user => "username"
jdbc_password => "password"
statement_filepath => "/queries/abc.sql"
lowercase_column_names => false
}
}
output {
stdout { codec => json_lines }
elasticsearch {
index => "accounts"
document_type => "account"
document_id => "%{id}"
hosts => ["elastic_ip:9200"]
}
}
When i run the logstash on my local machine(input database present on a Test server and ES on local machine), the results are accurate.
But when I try to run logstash on Test Server( both database and ES on Test Server), some records are missing randomly.
I am stuck at this point. Please help me out.
For other indexes which have same configration with more than 50000 records are ingested correctly, but account index is missing the documents.