I'm trying to use Logstash to get data from Mongodb and insert into Elasticsearch, using the logstash-input-mongodb plugin. Aparently I can connect in mongo with success but no document is stored in elasticsearch, not even shows in stdout.
My conf file:
input { mongodb { uri => 'mongodb://username:password@my-server-ip:27017/database' placeholder_db_dir => '/home/rodolpho/Documentos/Projetos/relatorios/temp' placeholder_db_name => 'logstash_sqlite.db' collection => 'sale_notifications' batch_size => 5000 } } output { elasticsearch { hosts => "localhost:9200" index => "myIndex" document_id => "%{_id}" document_type => "myDocType" } stdout { codec => rubydebug } }
My mongo collection:
Logstash response:
[2017-11-29T15:58:26,802][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
D, [2017-11-29T15:58:27.213000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | SUCCEEDED | 0.433s
D, [2017-11-29T15:58:27.229000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | STARTED | {"find"=>"sale_notifications", "filter"=>{"_id"=>{"$gt"=>BSON::ObjectId('5a12cb0100b15f0f6138e2f7')}}, "limit"=>5000}
D, [2017-11-29T15:58:27.502000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | SUCCEEDED | 0.271s
D, [2017-11-29T15:58:27.515000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.listCollections | STARTED | {"listCollections"=>1, "cursor"=>{}, "filter"=>{:name=>{"$not"=>/system.|$/}}}
D, [2017-11-29T15:58:27.702000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.listCollections | SUCCEEDED | 0.188s
D, [2017-11-29T15:58:27.758000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | STARTED | {"find"=>"sale_notifications", "filter"=>{"_id"=>{"$gt"=>BSON::ObjectId('5a1837a000b15f6cfc205669')}}, "limit"=>5000}
D, [2017-11-29T15:58:28.040000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | SUCCEEDED | 0.281s
D, [2017-11-29T15:58:28.047000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | STARTED | {"find"=>"sale_notifications", "filter"=>{"_id"=>{"$gt"=>BSON::ObjectId('5a12cb0100b15f0f6138e2f7')}}, "limit"=>5000}
D, [2017-11-29T15:58:28.441000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.find | SUCCEEDED | 0.394s
D, [2017-11-29T15:58:28.447000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.listCollections | STARTED | {"listCollections"=>1, "cursor"=>{}, "filter"=>{:name=>{"$not"=>/system.|$/}}}
D, [2017-11-29T15:58:28.854000 #24740] DEBUG -- : MONGODB | my-server-ip:27017 | database.listCollections | SUCCEEDED | 0.405s
After a while, the execution doesn't end and not even elasticsearch index is created. Please, I really would appreciate any help.
Versions:
ES: 5.6.2
Logstash: 5.6.2
MongoDB: 3.2.8
logstash-input-mongodb: 0.4.1
