Mongodb input plugin for logstash does not get any document


(Rodolpho Gheleri) #1

I'm trying to use Logstash to get data from Mongodb and insert into Elasticsearch, using the logstash-input-mongodb plugin. Aparently I can connect in mongo with success but no document is stored in elasticsearch, not even shows in stdout.

My conf file:

input {
  mongodb {
    uri => 'mongodb://username:password@my-server-ip:27017/database'
    placeholder_db_dir => '/home/rodolpho/Documentos/Projetos/relatorios/temp'
    placeholder_db_name => 'logstash_sqlite.db'
    collection => 'sale_notifications'
    batch_size => 5000
  }
}

output {
    elasticsearch { 
         hosts => "localhost:9200"
         index => "myIndex"
         document_id => "%{_id}"
         document_type => "myDocType"
    }
        stdout { codec => rubydebug }
}

My mongo collection:
Captura de tela de 2017-11-29 16-12-27

Logstash response:

[2017-11-29T15:58:26,802][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
D, [2017-11-29T15:58:27.213000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | SUCCEEDED | 0.433s
D, [2017-11-29T15:58:27.229000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | STARTED | {"find"=>"sale_notifications", "filter"=>{"_id"=>{"$gt"=>BSON::ObjectId('5a12cb0100b15f0f6138e2f7')}}, "limit"=>5000}
D, [2017-11-29T15:58:27.502000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | SUCCEEDED | 0.271s
D, [2017-11-29T15:58:27.515000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.listCollections | STARTED | {"listCollections"=>1, "cursor"=>{}, "filter"=>{:name=>{"$not"=>/system.|$/}}}
D, [2017-11-29T15:58:27.702000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.listCollections | SUCCEEDED | 0.188s
D, [2017-11-29T15:58:27.758000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | STARTED | {"find"=>"sale_notifications", "filter"=>{"_id"=>{"$gt"=>BSON::ObjectId('5a1837a000b15f6cfc205669')}}, "limit"=>5000}
D, [2017-11-29T15:58:28.040000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | SUCCEEDED | 0.281s
D, [2017-11-29T15:58:28.047000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | STARTED | {"find"=>"sale_notifications", "filter"=>{"_id"=>{"$gt"=>BSON::ObjectId('5a12cb0100b15f0f6138e2f7')}}, "limit"=>5000}
D, [2017-11-29T15:58:28.441000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | SUCCEEDED | 0.394s
D, [2017-11-29T15:58:28.447000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.listCollections | STARTED | {"listCollections"=>1, "cursor"=>{}, "filter"=>{:name=>{"$not"=>/system.|$/}}}
D, [2017-11-29T15:58:28.854000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.listCollections | SUCCEEDED | 0.405s

After a while, the execution doesn't end and not even elasticsearch index is created. Please, I really would appreciate any help.

Versions:

ES: 5.6.2
Logstash: 5.6.2
MongoDB: 3.2.8
logstash-input-mongodb: 0.4.1


(Zak Maq) #2

Hi ! :grinning:
I have the same problem..

Did you find any solution ??

Thx


(Rodolpho Gheleri) #3

Sorry, but no...


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.