JDBC update issue

Hi,

At the moment I have following configuration

input {
  jdbc {
    jdbc_driver_library => "/usr/share/java/mysql-connector-java.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://localhost:3306/xf"
    jdbc_user => "xxxx"
    jdbc_password => "xxxxx"
    schedule => "0 * * * *"
    statement => "select xf_post.username, count(*) as posts, count(xf_thread.best_answer_id) as best_answers, sum(xf_post.likes) as likes from xf_post left join xf_thread on (xf_post.post_id = xf_thread.best_answer_id and xf_thread.best_answer_id > 0) where xf_post.post_date between UNIX_TIMESTAMP('2018-01-01 00:00:00') and UNIX_TIMESTAMP('2018-02-01 00:00:00') group by xf_post.username order by posts desc limit 25"
}
}
output {
 elasticsearch {
   hosts => ["http://xxxx.com:9200/"]
   user => logstash_internal
   password => xxxx
   index => "forum_stat"
   document_type => "forum_stat"
   document_id => "%{id}"
 }
# stdout { codec => rubydebug }
}

Problem in that records in ES index are duplicated and the second problem in that I want to use 1 month period in my SQL query (xf_post.post_date between UNIX_TIMESTAMP('2018-01-01 00:00:00') and UNIX_TIMESTAMP('2018-02-01 00:00:00')) and I don't know how to make it automatically without changing this period in config manually and removing index each month.

Please help me with it.

Thanks

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.