Loading IBM DB2 data using Logstash into Elasticsearch


(Neeraj Rajagopalan Iyer) #1

Hi,

I have 56 views to be loaded into elasticsearch. So i created a config file which has 56 jdbc blocks to load 56 views. Everytime it gets loaded but in Kibana i see only one table randomly.
Below is my config file. Request you to help me out.

#file: db2.conf

input{
jdbc{

IBM DB2 jdbc connection string to database BIIPODST

jdbc_connection_string => "jdbc:db2://jade002:3936/BIIPODST"

user

jdbc_user => "IYE8288"

Password

jdbc_password => "#####"

path to downloaded jdbc driver

jdbc_driver_library => "C:\Users\IYE8288\Downloads\logstash-6.3.1\logstash-6.3.1\Drivers\sqljdbc4.jar"

#name of driver class for DB2
jdbc_driver_class => "com.ibm.db2.jcc.DB2Driver"

Query

statement => "SELECT * from BIIPODS.BIIPMETA_CLT_ADDRESS_TYPE_VW fetch first 50 rows only"
}

#jdbc2
jdbc{
jdbc_connection_string => "jdbc:db2://jade002:3936/BIIPODST"
jdbc_user => "IYE8288"
jdbc_password => "#####"
jdbc_driver_library => "C:\Users\IYE8288\Downloads\logstash-6.3.1\logstash-6.3.1\Drivers\sqljdbc4.jar"
jdbc_driver_class => "com.ibm.db2.jcc.DB2Driver"
statement => "SELECT * from BIIPODS.BIIPMETA_CLT_ADDRESS_VW fetch first 50 rows only"
}

#jdbc3
jdbc{
jdbc_connection_string => "jdbc:db2://jade002:3936/BIIPODST"
jdbc_user => "IYE8288"
jdbc_password => "#####"
jdbc_driver_library => "C:\Users\IYE8288\Downloads\logstash-6.3.1\logstash-6.3.1\Drivers\sqljdbc4.jar"
jdbc_driver_class => "com.ibm.db2.jcc.DB2Driver"
statement => "SELECT * from BIIPODS.BIIPMETA_CLT_ASSOCIATED_LE_VW fetch first 50 rows only"
}
...........#up to 56 blocks#
}

output {
elasticsearch {

protocol => http

index => "biipodst"
document_type => "BIIPMETA_CLT_PERSON_VW"
document_id => "%{KEY_COLUMN_ID_NUM}"
hosts => "localhost"
}

}


(Mark Walkom) #2

As you have another topic on this I am closing this one.

As suggested in the previous topic, it helps greatly if you properly format things.


(Mark Walkom) #3