Hi,
At the moment I have following configuration
input {
jdbc {
jdbc_driver_library => "/usr/share/java/mysql-connector-java.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/xf"
jdbc_user => "xxxx"
jdbc_password => "xxxxx"
schedule => "0 * * * *"
statement => "select xf_post.username, count(*) as posts, count(xf_thread.best_answer_id) as best_answers, sum(xf_post.likes) as likes from xf_post left join xf_thread on (xf_post.post_id = xf_thread.best_answer_id and xf_thread.best_answer_id > 0) where xf_post.post_date between UNIX_TIMESTAMP('2018-01-01 00:00:00') and UNIX_TIMESTAMP('2018-02-01 00:00:00') group by xf_post.username order by posts desc limit 25"
}
}
output {
elasticsearch {
hosts => ["http://xxxx.com:9200/"]
user => logstash_internal
password => xxxx
index => "forum_stat"
document_type => "forum_stat"
document_id => "%{id}"
}
# stdout { codec => rubydebug }
}
Problem in that records in ES index are duplicated and the second problem in that I want to use 1 month period in my SQL query (xf_post.post_date between UNIX_TIMESTAMP('2018-01-01 00:00:00') and UNIX_TIMESTAMP('2018-02-01 00:00:00')) and I don't know how to make it automatically without changing this period in config manually and removing index each month.
Please help me with it.
Thanks