Some problems about logstash5 with non time based jdbc input

Elastic Stack Logstash
1

1./ With logstash jdbc plugin, I got postgresql db pipeline to Elasticsearch, but I noticed the index size in elasticsearch doubled every time I restart logstash, and kibana showed doubled data counts as well, why? and how to deal with this issue?

2./ As I had none time based data from postgresql, the logstash process pushes the data only once, and "stopping pipline" automatically after about 10 minutes and jps showed the process of logstash dead, what's wrong it could be? and how to keep logstash from keeping dying? PS: I noticed from the log which showed 4.xxxxs "select * from test".

2
  1. What does the query look like? How are you using the sql_last_value parameter to avoid processing old data?
  2. If you want Logstash to periodically make the SQL query and process any results you need to define the desired execution schedule in the schedule option.
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.