JDBC update issue


(Igor Gubaidullin) #1

Hi,

At the moment I have following configuration

input {
  jdbc {
    jdbc_driver_library => "/usr/share/java/mysql-connector-java.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://localhost:3306/xf"
    jdbc_user => "xxxx"
    jdbc_password => "xxxxx"
    schedule => "0 * * * *"
    statement => "select xf_post.username, count(*) as posts, count(xf_thread.best_answer_id) as best_answers, sum(xf_post.likes) as likes from xf_post left join xf_thread on (xf_post.post_id = xf_thread.best_answer_id and xf_thread.best_answer_id > 0) where xf_post.post_date between UNIX_TIMESTAMP('2018-01-01 00:00:00') and UNIX_TIMESTAMP('2018-02-01 00:00:00') group by xf_post.username order by posts desc limit 25"
}
}
output {
 elasticsearch {
   hosts => ["http://xxxx.com:9200/"]
   user => logstash_internal
   password => xxxx
   index => "forum_stat"
   document_type => "forum_stat"
   document_id => "%{id}"
 }
# stdout { codec => rubydebug }
}

Problem in that records in ES index are duplicated and the second problem in that I want to use 1 month period in my SQL query (xf_post.post_date between UNIX_TIMESTAMP('2018-01-01 00:00:00') and UNIX_TIMESTAMP('2018-02-01 00:00:00')) and I don't know how to make it automatically without changing this period in config manually and removing index each month.

Please help me with it.

Thanks


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.