# Basic nested JSON parse failure **URL:** \ **Category:** Logstash\ **Created:** [February 4, 2016, 11:10pm UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013 "2016-02-04T23:10:47Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Patrick\_Barker](https://avatars.discourse-cdn.com/v4/letter/p/ecc23a/32.png) [@Patrick\_Barker](https://discuss.elastic.co/u/Patrick_Barker)\ **Post date:** [February 4, 2016, 11:10pm UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/1 "2016-02-04T23:10:47Z")
Hey, I am new to log stash and surely I must be doing someone wrong. When I sending basic nested json from python 3 it fails: `Message=json.dumps({'default': 'default', 'time': 'again', 'this': {'go': 'by'}})` My log stash config is: ``` input { tcp { port => 5000 } sqs { queue => "QCB_logstash" region => "us-east-1" } } output { elasticsearch { hosts => "elasticsearch:9200" } } ``` Super simple, but it fails with: `{:timestamp=>"2016-02-04T22:54:03.980000+0000", :message=>"Failed action. ", :status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2016.02.04", :_type=>"logs", :_routing=>nil}, #, @cancelled=false, @data={"default"=>"default", "this"=>{"go"=>"by"}, "time"=>"again", "@version"=>"1", "@timestamp"=>"2016-02-04T22:54:03.533Z"}, @metadata={}, @accessors=#"default", "this"=>{"go"=>"by"}, "time"=>"again", "@version"=>"1", "@timestamp"=>"2016-02-04T22:54:03.533Z"}, @lut={"type"=>[{"default"=>"default", "this"=>{"go"=>"by"}, "time"=>"again", "@version"=>"1", "@timestamp"=>"2016-02-04T22:54:03.533Z"}, "type"]}>>], :response=>{"create"=>{"_index"=>"logstash-2016.02.04", "_type"=>"logs", "_id"=>"AVKufhLIk85JQl8a5Oze", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [this]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"unknown property [go]"}}}}, :level=>:warn}` This seems so simple but I've tried a bunch of configs and can't get a nested structure to work. Any help would be great, thanks ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 4, 2016, 11:24pm UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/2 "2016-02-04T23:24:20Z")
Let's establish exactly what Logstash sees. Replace the elasticsearch output with `stdout { codec => rubydebug }` and show the results that you get to stdout. ---
**Author:** ![Patrick\_Barker](https://avatars.discourse-cdn.com/v4/letter/p/ecc23a/32.png) [@Patrick\_Barker](https://discuss.elastic.co/u/Patrick_Barker)\ **Post date:** [February 4, 2016, 11:36pm UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/3 "2016-02-04T23:36:09Z")
> [@magnusbaeck](#): > > codec =\> rubydebug Hey Magnus, that was exactly where I was heading. Heres the output: ``` { "time" => "again", "default" => "default", "this" => { "go" => "by" }, "@version" => "1", "@timestamp" => "2016-02-04T23:34:21.157Z" } ``` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 5, 2016, 6:37am UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/4 "2016-02-05T06:37:25Z")
Okay, that looks good. What does the index's mapping look like? I suspect that `this` has been mapped as some data type and how you're suddenly sending something else, hence mapping\_parsing\_exception. ---
**Author:** ![Patrick\_Barker](https://avatars.discourse-cdn.com/v4/letter/p/ecc23a/32.png) [@Patrick\_Barker](https://discuss.elastic.co/u/Patrick_Barker)\ **Post date:** [February 5, 2016, 4:14pm UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/5 "2016-02-05T16:14:22Z")
If your are referring to the index in ES, it doesn't appear to be indexed. If I change `"this" => { "go" => "by"}` to `json.dumps( "this" => { "go" => "by"})` It will make it to ES and Kibana looks like: ![](https://us1.discourse-cdn.com/elastic/original/2X/9/96fcb486c8fbdc31bb2f7a3a55fea9b27f7e0237.png) ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 5, 2016, 4:16pm UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/6 "2016-02-05T16:16:46Z")
I don't understand what you mean -- you were using json.dumps() all along weren't you? I was referring to the mapping of the index, and the error message indicates that the current mapping isn't compatible with the document you were trying to index. ---
**Author:** ![Patrick\_Barker](https://avatars.discourse-cdn.com/v4/letter/p/ecc23a/32.png) [@Patrick\_Barker](https://discuss.elastic.co/u/Patrick_Barker)\ **Post date:** [February 5, 2016, 4:19pm UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/7 "2016-02-05T16:19:00Z")
Hm, I guess I don't understand what index you are referring to or how to access that. In my example I was using a second nested `json.dumps` for the nested structure. Sorry for the confusion. i.e. `Message=json.dumps({'default': 'default', 'time': 'again', json.dumps('this': {'go': 'by'})})` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [February 6, 2016, 11:30am UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/8 "2016-02-06T11:30:12Z")
Your original Python code was fine. Your second attempt with a second json.dump() doesn't make sense and results in, as you've seen, {"go": "by"}` being indexed as a string which I can't imagine is what you want. The index I'm talking about is the Elasticsearch index, i.e. the datastore. It's similar to a table in a relational database. An ES index has one or more fields, and fields have mappings which are similar to the data types of the columns (fields) in a relational database table. As opposed to a relational database the mapping of a field isn't necessarily declared beforehand but detected by ES based on the input it receives. My guess is that the `this` field at some point was mapped as a string since the first document you sent into the index had a string value in that field. Now you're trying to index another document where the `this` field isn't a string but an object (a dictionary in Python-land) containing a string field. Such an object can't be stored into a string field, hence the mapper\_parsing\_exception error. Please consult the ES documentation to get the full story on how this works. You can use the [get mapping API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html) to obtain the current mappings of an index and thereby check how the `this` field actually is mapped. Mappings of fields can't be changed after the fact. The data needs to be reindexed. ---
**Author:** ![Patrick\_Barker](https://avatars.discourse-cdn.com/v4/letter/p/ecc23a/32.png) [@Patrick\_Barker](https://discuss.elastic.co/u/Patrick_Barker)\ **Post date:** [February 8, 2016, 4:47pm UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/9 "2016-02-08T16:47:23Z")
Awesome deleting the indexes worked, I thought you were referring to that index. I guess my confusion arises that when I am not sending the output to ES, but rather just stdout with the codec set to ruby debug and it fails I assumed that was not connected to the ES indexes. Either way, thanks for the help! ---
**Author:** ![streamn](https://avatars.discourse-cdn.com/v4/letter/s/e8c25b/32.png) [@streamn](https://discuss.elastic.co/u/streamn)\ **Post date:** [August 8, 2016, 12:22am UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/10 "2016-08-08T00:22:53Z")
I have seen this before, but I'm still confused about the statement: > "Mappings of fields can't be changed after the fact. The data needs to be reindexed." I have tried changing mapping, but it doesn't work, as stated above, but how would I go about re-indexing existing data, without deleting the index? In short, errors are human, and that happens. If somehow the type is wrong once, the whole index is screwed? I don't mind re-indexing if that means I can re-index existing data, but I do mind losing the data! Can someone explain how this works in real life? Thanks ---
**Author:** ![JeremyinNC](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeremyinnc/32/44868_2.png) [@JeremyinNC](https://discuss.elastic.co/u/JeremyinNC)\ **Post date:** [January 19, 2017, 4:24pm UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/11 "2017-01-19T16:24:14Z")
There is no "reindexing" the way you're thinking. There is exporting it & importing it into a new index with a template defining the new mappings. If I have a nested JSON object do I need to define all layers as "object"? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:29am UTC](https://discuss.elastic.co/t/basic-nested-json-parse-failure/41013/12 "2017-07-06T04:29:15Z")