Help! Problem output using jdbc input

Elastic Stack Logstash
1

Hello! i'm using jdbc input to access my first Mysql data, my schedule is around 30 minutes, after that i'm using jdbc streaming to upload an other database , then i use simple output to see my data as stdout ruby , the problem is that the process is very slow and after running it in mode debug i see my first statement and after 20 minutes the same statement again and i have nothing in my output

2

I have removed the schedule, even though after 10 minutes the statement of my first database is repeating , please help

3

Start by posting your Logstash configuration.

4

input {
jdbc {
jdbc_driver_library => "/etc/logstash/mysql-connector-java-5.1.36.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql:*"
jdbc_validate_connection => true
jdbc_user => ""
jdbc_password => "*****"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
#schedule => "/40 * * * *"
statement => "SELECT id1, id2, Id3, id4 from base1 where id3 > :sql_last_value AND id2 REGEXP '' order by id1 ASC Limit 1000"
record_last_run => "true"
use_column_value => "true"
tracking_column => "id2"
tracking_column_type => "numeric"
clean_run => "false"
last_run_metadata_path => "/etc/logstash/.logstash_jdbc_last_run"
type => "base"
}
}

5

filter {
date {
match => [ "id2","UNIX_MS" ]
target => "@timestamp"
}
mutate {
replace => { "id2" => "%{@timestamp}" }
lowercase => ["id2"]
}
jdbc_streaming {
jdbc_driver_library => "/etc/logstash/mysql-connector-java-5.0.8-bin.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql:*"
jdbc_user => ""
jdbc_password => ""
statement => "SELECT `` From base2
parameters => {"variable"=>""}
target => ""
cache_expiration => 43200.0
cache_size => 520000
}

6

split {
field =>""
}
alter {
coalesce => ["" , "%{[][]}"]
}

mutate {
remove_field => [""]
}
}
output {
stdout {codec => rubydebug}
}

8

Okay. So you're saying that every 10 minutes the query in your jdbc input is run and produces a number of events, even though the schedule option is clearly commented out? Have you checked that you don't have any extra files in /etc/logstash/conf.d that Logstash might be reading?

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.