Logstash output Performance


(Elsayed Awd Elsayed Ali) #1

I 'am using Elasticsearch 5.1.1 , logstash 5.1.1 ,I imported 3 millions rows from sqlserver into elastic via logstash in 2 hours

I have 1 single windows machine with 4GB Ram , core I 3 :slight_smile: is there any additional configurations should I add to speed up the importing ?

Logstash Configurations

input {
jdbc {
jdbc_driver_library => "D:\Usefull_Jars\sqljdbc4-4.0.jar"
jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
jdbc_connection_string => "jdbc:sqlserver://192.168.5.14:1433;databaseName=DataSource;integratedSecurity=false;user=;password=;"
jdbc_user => ""
jdbc_password => "
"
statement => "SELECT * FROM RawData"
jdbc_fetch_size => 1000
}
}
output {
elasticsearch {
hosts => "localhost"
index => "testdata"
document_type => "testfeed"
document_id => "%{id}"
flush_size => 512
}
}


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.