Hi,
tried using the unofficial logstash-input-mongodb, but im having problem importing MongoDB to ES.
Is there something that I am missing in the config file? TIA
MongoDB Name: test
Table: restaurants
Configuration:
input {
mongodb {
uri => 'mongodb://user:pass@192.168.1.50:27017/test'
placeholder_db_dir => '/opt/logstash-mongodb/'
placeholder_db_name => 'logstash_sqlite.db'
collection => '/restaurants/'
batch_size => 5000
}
}
output {
elasticsearch {
hosts => "localhost:9200"
index => "restaurants"
document_type => "restaurant"
document_id => "%{restaurant_id}"
manage_template => true
}
stdout { codec => rubydebug }
}
This is what I got:
logstash-2.4.0\bin>logstash -w8 -f mongodb.conf
Settings: User set pipeline workers: 8, Default pipeline workers: 4
D, [2016-10-03T07:23:26.095000 #12324] DEBUG -- : MONGODB | Adding 192.168.1.50:
27017 to the cluster. | runtime: 0.0000ms
D, [2016-10-03T07:23:26.124000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
admin.$cmd selector={:ismaster=>1} flags=[] limit=-1 skip=0 project=nil | runtim
e: 27.0000ms
D, [2016-10-03T07:23:26.231000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:saslStart=>1, :autoAuthorize=>1, :payload=><BSON::Binary:0x
13480 type=generic data=0x6e2c2c6e3d746573...>, :mechanism=>"SCRAM-SHA-1"} flags
=[] limit=-1 skip=0 project=nil | runtime: 4.9999ms
D, [2016-10-03T07:23:26.306000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:saslContinue=>1, :payload=><BSON::Binary:0x13482 type=gener
ic data=0x633d626977732c72...>, :conversationId=>1} flags=[] limit=-1 skip=0 pro
ject=nil | runtime: 6.0000ms
D, [2016-10-03T07:23:26.315000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:saslContinue=>1, :payload=><BSON::Binary:0x13484 type=gener
ic data=0x...>, :conversationId=>1} flags=[] limit=-1 skip=0 project=nil | runti
me: 5.9998ms
D, [2016-10-03T07:23:26.322000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 99.000
2ms
D, [2016-10-03T07:23:26.348000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 17.000
0ms
D, [2016-10-03T07:23:26.385000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 10.999
9ms
D, [2016-10-03T07:23:26.436000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 9.0001
ms
D, [2016-10-03T07:23:26.530000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 10.000
0ms
D, [2016-10-03T07:23:26.702000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 7.0000
ms
D, [2016-10-03T07:23:27.034000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 7.9999
ms
Pipeline main started
D, [2016-10-03T07:23:27.682000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 5.9998
ms
D, [2016-10-03T07:23:28.970000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 5.0001
ms
D, [2016-10-03T07:23:31.537000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 5.0001
ms
D, [2016-10-03T07:23:36.135000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
admin.$cmd selector={:ismaster=>1} flags=[] limit=-1 skip=0 project=nil | runtim
e: 7.0000ms
D, [2016-10-03T07:23:36.546000 #12324] DEBUG -- : MONGODB | COMMAND | namespace=
test.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>
/system.|$/}}} flags=[:slave_ok] limit=-1 skip=0 project=nil | runtime: 5.0001
ms