How to import mysql data into elastic search


(NETHAJI) #1

hi,
i need to know about how to import the mysql data into elasticsearch,

can u people please explain about what are software required for kibana visualize.

i'm not clearly understand about procedures,

what are steps i need to follow?

please explain about the flows, how the data is imported in elasticsearch and how it goes to kibana, i googled many times, but i'm facing many issues.

please provide some sample example from installation to kibana visuals.


(David Pilato) #2

You can use something like logstash and its jdbc input to read from your DB and push json to elasticsearch.

but i'm facing many issues.

Which are?


(NETHAJI) #3

Error: Could not find or load main class folder\logstash-6.4.1\logstash-6.4.1\logstash-core\lib\jars\commons-codec-1.11.jar;


(NETHAJI) #4

input {
jdbc {
jdbc_connection_string => "jdbc:mysql://192.168.9.231:3306/sam_sit3"
# The user we wish to execute our statement as
jdbc_user => "Test"
jdbc_password => "*****"
# The path to our downloaded jdbc driver
jdbc_driver_library => "D:\New folder\mysql-connector-java-5.1.47-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
# our query
statement => "SELECT * FROM tbl_softwarelicenseinventory;"
}
}
output {
elasticsearch {
hosts => "192.168.7.231:9200"
index => "softwares"
document_type => "software"
document_id => "%{SoftwareLincenseInvetory_Id}"

}
}

these are the inputs given in config file, while running it i'm facing errors,.


(David Pilato) #6

This is probably a bad installation of Logstash. Not sure what you did...
Make sure you are following the instructions here: https://www.elastic.co/guide/en/logstash/current/installing-logstash.html


(NETHAJI) #7

Hi,
I imported data from mysql to kibana, but in kibana it shows only one row.
how to get the full data ?


(David Pilato) #8

Probably you are using the same _id for all documents... That's a blind guess though as I have no idea of what you are doing.


(NETHAJI) #9

No, i'm used different id


(NETHAJI) #10

i took the primary key in every mysql table as id