Randomly missing events

Elastic Stack Logstash
1

Hi
i have one pipeline processing 16 .conf file
in each file i have 2 jdbc
the issue is sometimes randomly i see logstash miss some jdbc !
i check the logstash-plain.log but nothing abnormal !

i tired to update logstash to latest version but still the issue appear
please your help :frowning:

2

What does this mean? Logstash miss some rows in SQL?
Can you share the pipelien?

3

thanks for your reply
The scenario is sometimes all input run, and sometimes some of the input run and the other input not
there is no pattern for input running

also if you can see from my dashboard there is gaps
there is no issue in the elastic, because the input didn't get triggered in the first place !

CS

here one of my file

input {
  jdbc {
    jdbc_driver_library => "/opt/ojdbc7.jar"
    jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
    jdbc_connection_string => "jdbc:oracle:thin:@*:*/*"
    jdbc_user => "*"
              jdbc_password => "*"
    schedule => "* * * * *"
    statement => "select * from my table"

  }
}
##############################################
input {
jdbc {
    jdbc_driver_library => "/opt/ojdbc7.jar"
    jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
    jdbc_connection_string => "jdbc:oracle:thin:@*/*"
    jdbc_user => "*"
              jdbc_password => "*"
    schedule => "* * * * *"
    statement => "select * from my table"

  }
}

my pipeline file as default

- pipeline.id: my-pipeline_1
  path.config: "my config path"

i hope my issue clear

4

Isn't it just because there is no data in the daytime histogram buckets? Does data in gaps exist in the SQL?

5

no it's not exist in sql
if i run the sql query i will have a result
but the issue logstash didn't execute the sql query

6

It is possibly something related to this topic: "Dealing With Large Result-sets". That SQL query returns all the rows?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.