# Logstash from Mongo To Elastic - Array Problem
**URL:** https://discuss.elastic.co/t/logstash-from-mongo-to-elastic-array-problem/124340
**Category:** Logstash
**Created:** [March 16, 2018, 5:46pm UTC](https://discuss.elastic.co/t/logstash-from-mongo-to-elastic-array-problem/124340 "2018-03-16T17:46:37Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Pierpaolo\_Rai](https://discuss.elastic.co/u/Pierpaolo_Rai)
#### Post date: [March 16, 2018, 5:46pm UTC](https://discuss.elastic.co/t/logstash-from-mongo-to-elastic-array-problem/124340/1 "2018-03-16T17:46:38Z")
Hello guys, I'm running Logstash to sync my ElasticSearch with my MongoDB operation.
I used the mongdb plugin to read from mongo, this is my initial config file:
input {
mongodb {
uri =\> 'mongodb://localhost:27017/db'
placeholder\_db\_dir =\> '/opt/logstash-mongodb/'
placeholder\_db\_name =\> 'logstash\_sqlite.db'
collection =\> 'catalog'
}
}
output {
elasticsearch {
hosts =\> ['localhost:9200']
user =\> 'elastic'
password =\> 'xxxxx'
index =\> 'catalog'
}
}
I have a problem on syncronizind the json containing arrays.
This is my inserto on mongo:
db.getCollection('catalog').insert({
"schema" : {
"first" : {
"name" : "name",
"type" : "type",
"namespace" : "namespace",
"fields" : [
{
"name" : "first\_field",
"type" : "string"
},
{
"name" : "second\_field",
"type" : "string"
}
]
}
}
})
and this is the result on elasticsearch:
{
"\_index": "catalog",
"\_type": "doc",
"\_id": "Jd\_lL2IBwDdr4Pihholf",
"\_score": 1,
"\_source": {
"@version": "1",
"@timestamp": "2018-03-16T17:39:52.899Z",
"logdate": "2018-03-16T17:35:10+00:00",
"schema\_first\_name": "name",
"schema\_first\_namespace": "namespace",
"log\_entry": """{"\_id"=\>BSON::ObjectId('5aac004edf7f789a5cbf6fe5'), "schema"=\>{"first"=\>{"name"=\>"name", "type"=\>"type", "namespace"=\>"namespace", "fields"=\>[{"name"=\>"first\_field", "type"=\>"string"}, {"name"=\>"second\_field", "type"=\>"string"}]}}}""",
"host": "",
"schema\_first\_type": "type",
"mongo\_id": "5aac004edf7f789a5cbf6fe5"
}
}
i've lost the "fields" object.
I tried different solution, for example:
filter {
split {
field =\> "fields"
}
}
but I always received the follow error:
[logstash.filters.split] Only String and Array types are splittable. field:fields is of type = NilClass\_
Any suggestion?
Thanks in advance.
PPL
---
### Author:  [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [March 17, 2018, 4:17am UTC](https://discuss.elastic.co/t/logstash-from-mongo-to-elastic-array-problem/124340/2 "2018-03-17T04:17:10Z")
It looks like you're probably using [Phillip Hutchins' community-contributed plugin](https://github.com/phutchins/logstash-input-mongodb/blob/master/README.md), which was designed for a specific use-case and may or may not fit your needs.
From what I can tell, by default the input "flatten"s the structured object; it can also be configured to "dig" out specific bits, and has a "simple" option that does surprising things like discarding the sign of negative numbers.
It's also unclear to me what exactly the local database does, or how that database is kept from growing unbounded.
---
### Author:  [@Pierpaolo\_Rai](https://discuss.elastic.co/u/Pierpaolo_Rai)
#### Post date: [March 17, 2018, 8:18am UTC](https://discuss.elastic.co/t/logstash-from-mongo-to-elastic-array-problem/124340/3 "2018-03-17T08:18:47Z")
Thanks yaauie, you are right, but about you wich is the best way to sync MongoDB and ES?
(I need for a runtime sync) I tried also via jdbc plugin but I didn't find a good jdbc driver for mongodb.
---
### Author:  [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [March 17, 2018, 8:31am UTC](https://discuss.elastic.co/t/logstash-from-mongo-to-elastic-array-problem/124340/4 "2018-03-17T08:31:35Z")
If you need a runtime sync, you'll need to dual-write in your application since MongoDB doesn't expose a transaction log -- there is no way to get the creates, updates, and deletes in the order that they happened so that we can keep them in sync other than hooking into the application that is performing those actions.
---
### Author:  [@Pierpaolo\_Rai](https://discuss.elastic.co/u/Pierpaolo_Rai)
#### Post date: [March 17, 2018, 9:32am UTC](https://discuss.elastic.co/t/logstash-from-mongo-to-elastic-array-problem/124340/5 "2018-03-17T09:32:08Z")
And what about running mongo in replica-set mode? I read in this article that is possible to sync via the mongo oplog:
> **[5 Different ways to synchronize data from MongoDB to ElasticSearch](https://code.likeagirl.io/5-different-ways-to-synchronize-data-from-mongodb-to-elasticsearch-d8456b83d44f?gi=d1d6eba3e118)**
>
> MongoDB + ElasticSearch
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 14, 2018, 9:32am UTC](https://discuss.elastic.co/t/logstash-from-mongo-to-elastic-array-problem/124340/6 "2018-04-14T09:32:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.