Hi,
I'm trying to keep postsgresql synchronized with elasticsearch using logstash.
To do this, I tried to insert the scheduling parameter into the input configuration pipeline:
file.conf
input {
jdbc {
# Postgres jdbc connection string to our database, mydb
jdbc_connection_string => "jdbc:postgresql://localhost:5432/my_db"
# The user we wish to execute our statement as
jdbc_user => "myuser"
# The path to our downloaded jdbc driver
jdbc_driver_library => "/etc/logstash/postgresql-42.2.5.jar"
# The name of the driver class for Postgresql
jdbc_driver_class => "org.postgresql.Driver"
#password marambal
jdbc_password => "mypassword"
# our query
statement => "SELECT document::text,updated_at from snapshots"
schedule => " 0 7 * * * "
use_column_value => true
tracking_column => "updated_at"
}
}
filter{
json{
source => "document"
#target => "json_doc"
remove_field => ["document"]
}
}
output {
stdout { codec => json_lines }
elasticsearch {
#codec => json
index => "snapshots"
#document_type => ""
document_id => "%{updated_at}"
hosts => ["localhost"]
}
}
The moment I start logstash with this command:
// sudo bin/logstash -f /etc/logstash/conf.d/big-data.conf --path.settings /etc/logstash/
the following output comes out:
[2019-06-21T13:54:38,756][INFO ][logstash.javapipeline ] Pipeline started {"pipeline.id"=>"main"}
[2019-06-21T13:54:38,998][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[2019-06-21T13:54:40,108][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/cronline.rb:77: warning: constant ::Fixnum is deprecated
and remains pending...it doesn't work.
I expect logstash to upload new data to elasticsearch every day at 7 AM, isn't it?