Setting up logstash with mongodb not starting

Elastic Stack Logstash
1

hi community,
need your expertise / advise in this issue im facing when starting logstash which im trying to connect wth mongodb.

input {
        uri => 'mongodb://userhere:passwordhere@1234.234.123.12:8081'
        placeholder_db_dir => '/opt/logstash-mongodb/'
        placeholder_db_name => 'logstash.db'
        collection => 'mtg_stuff'
        batch_size => 5000
}
filter {

}
output {
        stdout {
            codec => rubydebug
        }
        elasticsearch {
            action => "index"
            index => "index0123"
            hosts => "https://cloud-blahblah:9243"
            user => "elastic"
            password => "xxxxxxxxxxxxxx"
        }
}```

[ERROR] 2021-08-11 08:52:19.676 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, 
:exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"{\" at line 2, column 13 (byte 21) after input {\n       
 uri ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'",
 "org/logstash/execution/AbstractPipelineExt.java:187:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in `initialize'",
 "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'",
 "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:391:in `block in converge_state'"]}
2

You don't have any reference to the mongodb plugin

Your input need to be like this :

input {
  mongodb {
        uri => 'mongodb://userhere:passwordhere@1234.234.123.12:8081'
        placeholder_db_dir => '/opt/logstash-mongodb/'
        placeholder_db_name => 'logstash.db'
        collection => 'mtg_stuff'
        batch_size => 5000
  }
}
3

great! yes worked and able to start logstash..
e.. eventhough it started, it did not create the indeces in elastic.
am i missing anyting else?

[INFO ] 2021-08-11 11:23:27.251 [[main]-pipeline-manager] javapipeline - Pipeline Java execution initialization time {"seconds"=>1.78}
[INFO ] 2021-08-11 11:23:29.525 [[main]-pipeline-manager] mongodb - Registering MongoDB input
[INFO ] 2021-08-11 11:23:33.103 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
[INFO ] 2021-08-11 11:23:33.196 [Agent thread] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}

4

If you don't have data in elasticsearch, i think it's because logstash fail to acces to the values in mongodb. The error can't come from the communication between logstash and elasticsearch because you don't have any error in your logs.

I never use mongodg before so i'm can't help about his configuration.

5

I would start Logstash using --debug to get more details on what's going on.

Also this plugin is a 3rd party and if you suspect the input plugin is having issues you can ask questions on their github.

6

thank you.. yes when i ran as debug, shows this:
[DEBUG] 2021-08-12 12:10:57.411 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.
[DEBUG] 2021-08-12 12:10:58.159 [pool-6-thread-1] jvm - collector name {:name=>"ParNew"}
[DEBUG] 2021-08-12 12:10:58.159 [pool-6-thread-1] jvm - collector name {:name=>"ConcurrentMarkSweep"}
[DEBUG] 2021-08-12 12:10:58.537 [[main]<mongodb] mongodb - Updating watch collections
[DEBUG] 2021-08-12 12:10:58.552 [[main]<mongodb] mongodb - No new rows. Sleeping. {:time=>5}
[DEBUG] 2021-08-12 12:11:02.411 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto p

seems not fetching any data. any idea on this?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.