Hello,
Very new to Logstash and ElasticSearch (2.4.1), so am just experimenting at this stage.
We are trying to create two indexes in ES, which are sourced from MySQL in Logstash, then outputted to ES.
At first, it seemed to be working, but then we have realized that the two indexes in ES appear to be identical in size and seem to be "merged" somehow (even though they both have their unique names). Are we misunderstanding how we should create these two separate indexes?
Logstash conf file 1:
"
input {
jdbc {
jdbc_connection_string => "jdbc:mysql://localhost:3306/masdata"
jdbc_user => "username"
jdbc_password => "userpass"
jdbc_driver_library => "/usr/share/java/mysql-connector-java.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
schedule => "*/30 * * * mon-fri"
statement => "SELECT item_code,part_no,part_stock_no FROM tbl_products WHERE brand_id < 999"
}
}
output {
elasticsearch {
hosts => "10.11.1.223:9200"
index => "idx_md-partnos"
}
}
"
Logstash conf file 2:
"
input {
jdbc {
jdbc_connection_string => "jdbc:mysql://localhost:3306/masdata"
jdbc_user => "username"
jdbc_password => "userpass"
jdbc_driver_library => "/usr/share/java/mysql-connector-java.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
schedule => "*/30 * * * mon-fri"
statement => "SELECT item_code,item_description,brand_name FROM tbl_products p LEFT JOIN tbl_brands b ON b.brand_id = p.brand_id WHERE p.brand_id < 999"
}
}
output {
elasticsearch {
hosts => "10.11.1.223:9200"
index => "idx_md-descriptions"
}
}
"
Can some kind soul put us on the right path?
Thank you!
Elliot