Issue Transfer Data to ElasticSearch

[Bernard.Thoven (Bernard Thoven Lukito)]

Hi,
I'm new using ELK and i'm trying to connect logstash to oracle DB.

I try to run logstash.conf and create the index, but not all the data got send to kibana.

Any suggestion ?

Regards,
Bernard

[dadoonet (David Pilato)]

Hi Bernard

I moved your question to #logstash.
BTW How do you know that not all the data has been sent?

[Bernard.Thoven (Bernard Thoven Lukito)]

Hi David,

Because in my oracleDB the query return 33 row , and i manually count the data in logstash window there are 33 data.
But in elasticsearch only have 30 data.

Thanks

[Robo]

can you post here your logstash pipeline configuration?

[Bernard.Thoven (Bernard Thoven Lukito)]

You mean this ?

input{
jdbc{
clean_run => true
jdbc_driver_library => "D:/Elastic Stack/ojdbc8.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "jdbc:oracle:thin:username/passwd@hostname:port/SID"
jdbc_user => "apps"
jdbc_password => "apps"
clean_run => true
schedule => "* * * * *"
statement =>
"
select
aia.invoice_id ,
aia.invoice_num,
aia.invoice_amount,
aia.invoice_date,
aia.invoice_type_lookup_code,
aila.item_description,
aila.amount
from ap_invoices_all aia,
ap_invoice_lines_all aila
where 1=1
and to_char(invoice_date,'RRRR') IN ('2018')
and aia.invoice_id = aila.invoice_id
and aila.line_type_lookup_code = 'ITEM'
and aila.discarded_flag = 'N'
and aia.invoice_id > :sql_last_value
order by invoice_id
"
use_column_value => true
tracking_column => "invoice_id"
}
}

output{
elasticsearch {
hosts => ["http://localhost:9200"]
index => "logstash_oracle_1"
}

stdout{codec => rubydebug}

}