Hi,we are planning to implement elasticsearch for retrieving records,we have already installed elasticsearch which is working fine
abhishek@CASSANDRA02 star_sd_24jan]$ curl -X GET 'http://localhost:9200'
{
"name" : "node-1",
"cluster_name" : "my-application",
"cluster_uuid" : "5VEbDq8NQ4ufkbDEmsuAYw",
"version" : {
"number" : "6.1.3",
"build_hash" : "af51318",
"build_date" : "2018-01-26T18:22:55.523Z",
"build_snapshot" : false,
"lucene_version" : "7.1.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
and we have to select records from table name:events in database name:test at faster rate.We have also installed logstash for communicating between elasticsearch and Mysql.Please find below configuration in logstash.conf
input {
jdbc {
jdbc_connection_string => "jdbc:mysql://localhost:3306/test"
The user we wish to execute our statement as
jdbc_user => ""
jdbc_password => ""
The path to our downloaded jdbc driver
jdbc_driver_library => "/home/abhishek/Downloads/mysql-connector-java-5.1.45/mysql-connector-java-5.1.45-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
our query
statement => "SELECT * FROM events"
}
}
output {
stdout { codec => json }
elasticsearch {
"hosts" => "localhost:9200"
"index" => "test-migrate"
"document_type" => "data"
}
}
but running curl -XPOST -H 'Content-Type: application/json' 'http://localhost:9200/test-migrate/_search?pretty=true' -d '{}',we are getting this output
{
"took" : 229,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 3,
"max_score" : 1.0,
"hits" : [
{
"_index" : "test-migrate",
"_type" : "pretty=true",
"_id" : "kOG5WmEBiAKb5bnD-zow",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "test-migrate",
"_type" : "pretty=true",
"_id" : "keG7WmEBiAKb5bnDUDqN",
"_score" : 1.0,
"_source" : { }
},
{
"_index" : "test-migrate",
"_type" : "pretty=true",
"_id" : "kuH8WmEBiAKb5bnDwTqV",
"_score" : 1.0,
"_source" : { }
}
]
}
}
whereas we needed each row as a separate document with the columns as field