Systemd restarts logstash when using jdbc input

[arisbanach]

Not sure if I've configured something wrong since this is my first time working with JDBC. I am trying to import a large database into Elasticsearch using Logstash and the logstash-input-jdbc plugin. Here is my pipeline:

input {
  jdbc {
    jdbc_driver_library => "/home/ubuntu/postgresql-42.1.4.jar"
    jdbc_driver_class => "org.postgresql.Driver"
    jdbc_connection_string => "jdbc:postgresql://myhost.com:5432/dbname?user=myuser&password=mypassword"
    jdbc_user => "myuser"
    jdbc_password => "mypassword"
#    schedule => "* * * * *"                                                                                                 
    statement => "SELECT * from tablename LIMIT 1"
    jdbc_paging_enabled => "true"
    #jdbc_fetch_size => "1000"                                                                                               
  }
}
output {
  stdout { codec => json_lines }
  #elasticsearch {                                                                                                           
  #  hosts => localhost                                                                                                      
  #  index => "db-import"                                                                                                   
  #}                                                                                                                         
}

I commented out a couple lines because I wasn't sure if they were causing issues or not. Right now when I run this and do journalctl -f I see:

Dec 01 15:01:59 elastisearch systemd[1]: logstash.service: Service hold-off time over, scheduling restart.
Dec 01 15:01:59 elastisearch systemd[1]: Stopped logstash.
Dec 01 15:01:59 elastisearch systemd[1]: Started logstash.
Dec 01 15:02:35 elastisearch logstash[11884]: Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties

and then the next log line is one row from the database. Then, it repeats: logstash has the Service hold-off time over, scheduling restart error, restarts, and imports the same row.

Why is it stuck in a loop? Shouldn't it just import the one row and then stop the job? And why is Logstash itself restarting? Does that imply there is a config error somewhere?

Thank you

[arisbanach]

I also noticed that when I put a LIMIT on the statement, it does return results quickly, but if I'm running this with systemd it will return the results, stop Logstash, start it again, then process the pipeline again, and repeat until I stop the service.

[magnusbaeck (Magnus Bäck)]

Unless you use the schedule option Logstash will shut down after the jdbc input has processed all rows returned by the query. systemd has obviously been configured to keep Logstash alive by restarting it when it shuts down.

[arisbanach]

Ah, that makes sense then! Is there a reason Logstash shuts down after the pipeline finishes? I'm relatively new to Logstash but expected that if I start it with systemd that it would only stop if there is either an error or I tell systemd to stop it. Wouldn't it make more sense to just have the pipeline finish and then Logstash to stay running and just output "finished pipeline" to the console or something?

[magnusbaeck (Magnus Bäck)]

Is there a reason Logstash shuts down after the pipeline finishes?

Why keep running if you only have a jdbc input that won't run more than once?

I'm relatively new to Logstash but expected that if I start it with systemd that it would only stop if there is either an error or I tell systemd to stop it. Wouldn't it make more sense to just have the pipeline finish and then Logstash to stay running and just output "finished pipeline" to the console or something?

That would make some sense when run as a service as in your case but would be undesirable if run interactively. In fact, a common complaint about the file input is that it doesn't shut Logstash down once it has reached the end of the file.

Running Logstash with just non-scheduled jdbc inputs is a rather unusual use case, I'd say. I'd expect you to be able to adjust the unit file to not respawn Logstash when it shuts down.

[arisbanach]

Okay, thank you!