Currently I am using Ubuntu 14.04 with 4GB RAM.
My Data size is more than 5GB to index. I am getting below error
Settings: Default filter workers: 1
Logstash startup completed
java.lang.OutOfMemoryError: Java heap space
Dumping heap to /opt/logstash/heapdump.hprof ...
Unable to create /opt/logstash/heapdump.hprof: File exists
Exception when executing JDBC query {:exception=>#<Sequel::DatabaseError: Java::JavaLang::OutOfMemoryError: Java heap space>, :level=>:warn}
Logstash shutdown completed
Even I tried to increase the HEAP SIZE to 2 or 3GB it would work. I thought of doing Alias for same Index Multiple db select query and create 2 or there Index with Alias to single Index Name .
As I am new how to do it in logstash jdbc input/output ? setting Alias in output , my codes below
input {
jdbc {
jdbc_driver_library =>"/usr/share/java/mysql-connector-java.jar"
jdbc_driver_class =>"com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://xxxxx.rds.amazonaws.com:3306/dbname"
jdbc_user => "username"
jdbc_password => "password"
jdbc_validate_connection => true
statement => "SELECT * from table "
}
}
output {
elasticsearch {
index =>"my_index"
document_type => "xyz"
document_id => "%{id}"
hosts => "xx.xxx.xxx.xx"
}
}
Please suggest how to use Alias or other way to fix my issue
