# Can logstash handle array of nested objects?

**URL:** <https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558>\
**Category:** Logstash\
**Created:** [June 21, 2016, 7:24pm UTC](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558 "2016-06-21T19:24:18Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ChrisR](https://avatars.discourse-cdn.com/v4/letter/c/ecd19e/32.png) [@ChrisR](https://discuss.elastic.co/u/ChrisR)\
**Post date:** [June 21, 2016, 7:24pm UTC](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558/1 "2016-06-21T19:24:18Z")

</div>

I am unable to configure logstash to load an array of nested objects as described in the [elasticsearch guide](https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-objects.html).

I can map and load the array of nested objects via the elasticsearch REST API (as shown below), but I am unable to do the same via logstash when there is an unknown number of nested objects in the array (in this case, an unknown number of comments).

Any suggestions?

```auto
PUT /my_index/blogpost/1
{
  "title": "Nest eggs",
  "body": "Making your money work...",
  "tags": ["cash", "shares"],
  "comments": [ 
    {
      "name": "John Smith",
      "comment": "Great article",
      "age": 28,
      "stars": 4,
      "date": "2014-09-01"
    },
    {
      "name": "Alice White",
      "comment": "More like this please",
      "age": 31,
      "stars": 5,
      "date": "2014-10-22"
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 21, 2016, 7:44pm UTC](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558/2 "2016-06-21T19:44:17Z")

</div>

What problems are you having, specifically?

---

<div class="post-metadata">

**Author:** ![ChrisR](https://avatars.discourse-cdn.com/v4/letter/c/ecd19e/32.png) [@ChrisR](https://discuss.elastic.co/u/ChrisR)\
**Post date:** [June 21, 2016, 9:29pm UTC](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558/3 "2016-06-21T21:29:27Z")

</div>

If I try to load the comments field as a stringified json object, I get this error:

"object mapping for [comments] tried to parse field [comments] as object, but found a concrete value"

I cannot use mutate to rename the field to object properties because there is an unknown number of comments.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 22, 2016, 5:36am UTC](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558/4 "2016-06-22T05:36:37Z")

</div>

But why would you want to store that field as serialized JSON? The error message above means that the `comments` field has been mapped as an object but you're trying to index a document with that field being a non-object, in your case probably a string. If you really want to store a string (it sounds ill-advised to me) you need to reindex and make sure `comments` is mapped as a string.

It would probably be helpful if you showed us an example Logstash event. Then we can see if we can turn that into the kind of document you indexed manually with a REST call.

---

<div class="post-metadata">

**Author:** ![ChrisR](https://avatars.discourse-cdn.com/v4/letter/c/ecd19e/32.png) [@ChrisR](https://discuss.elastic.co/u/ChrisR)\
**Post date:** [June 22, 2016, 11:18am UTC](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558/5 "2016-06-22T11:18:57Z")

</div>

I do not want to store the field as serialized JSON. I want to store it as a nested object. Posting the string via the REST API stores the object perfectly, which is why I attempted to pass it in as a string via logstash.

The event is the same as the example above where originally each comment is a separate row in a database. There may be an unknown number of comments per document.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 22, 2016, 11:52am UTC](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558/6 "2016-06-22T11:52:46Z")

</div>

It's still not clear to me what the problem is. You're pulling multiple rows from a database but can't join them into a single event that you can pass to ES? Or you already have the comments in a single event (like in the example)? We'll save everyone's time if you can be explicit about your configuration and what your events currently look like (we know what you _want_ them to look like).

---

<div class="post-metadata">

**Author:** ![ChrisR](https://avatars.discourse-cdn.com/v4/letter/c/ecd19e/32.png) [@ChrisR](https://discuss.elastic.co/u/ChrisR)\
**Post date:** [June 22, 2016, 12:44pm UTC](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558/7 "2016-06-22T12:44:44Z")

</div>

The events currently look the same as the example above where a "blog" table has a row with the title, body and tags and a "comments" table contains name, comment, age, stars and date and may have multiple rows associated with each blog row. I can flatten the multiple rows into a single event row using SQL. The question is how do I configure logstash to load this into ES, if possible, when there is an unknown number of nested comment objects.

I can do this using the ES java client. I can do this with a custom REST client. I would prefer to use logstash if possible.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:51am UTC](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558/8 "2017-07-06T04:51:12Z")

</div>


