I'm trying to pull in DynamoDB data into ElasticSearch using the logstash-input-dynamodb plugin. When I try to start Logstash after setting up my .conf file, I get a LogStash::ConfigurationError. How can I debug this further? Thanks!
Pipeline aborted due to error {:exception=>"LogStash::ConfigurationError", :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-dynamodb-2.0.5-java/lib/logstash/inputs/dynamodb.rb:155:inregister'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:330:in start_inputs'", "org/jruby/RubyArray.java:1613:ineach'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:329:in start_inputs'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:180:instart_workers'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/pipeline.rb:136:in run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.0-java/lib/logstash/agent.rb:491:instart_pipeline'"], :level=>:error}`
When I run Logstash with the --debug flag, I can see that this error occurs after this debug line:
Checkpointer: logstash_input_dynamodb_cptr {:level=>:info, :file=>"logstash/inputs/dynamodb.rb", :line=>"149", :method=>"register"}
My .conf file:
input { dynamodb{ endpoint => "dynamodb.us-west-2.amazonaws.com" streams_endpoint => "streams.dynamodb.us-west-2.amazonaws.com" view_type => "new_and_old_images" aws_access_key_id => "ACCESSKEY" aws_secret_access_key => "SECRET" table_name => "Users" read_ops => "0.1" log_format => "json_drop_binary" } } output { elasticsearch { host => "search-MYDOMAIN.us-west-2.es.amazonaws.com" protocol => "http" } stdout { } }