# Json::ParserError: Unrecognized token

**URL:** <https://discuss.elastic.co/t/json-unrecognized-token/215006>\
**Category:** Logstash\
**Created:** [January 14, 2020, 3:17pm UTC](https://discuss.elastic.co/t/json-unrecognized-token/215006 "2020-01-14T15:17:12Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![akuninja](https://avatars.discourse-cdn.com/v4/letter/a/85f322/32.png) [@akuninja](https://discuss.elastic.co/u/akuninja)\
**Post date:** [January 14, 2020, 3:17pm UTC](https://discuss.elastic.co/t/json-unrecognized-token/215006/1 "2020-01-14T15:17:12Z")

</div>

//////////////////////////////////THIS PART I SOLVED MYSELF ///////////////////////////////////////  
I am still getting a broken token within my logstash output, and I do not know where this timestamp (possibly) token comes from. Could someone offer some help?

here is my logstash error message:

[2020-01-14T16:11:31,022][ERROR][logstash.codecs.json][main] JSON parse error, original data now in message field {:error=\>#\<LogStash::Json::ParserError: Unrecognized token 'mestamp': was expecting ('true', 'false' or 'null')

Here is what I am sending to logstash via python script:  
LOGGER = logging.getLogger('python-logstash-logger')  
LOGGER.setLevel(logging.DEBUG)

# LOGGER.addHandler(logstash.LogstashHandler(127.0.0.1, 5000, version=1))

print(dir(logstash))  
LOGGER.addHandler(logstash.TCPLogstashHandler('127.0.0.1', 5000, version=1))  
LOGGER.error('python-logstash: test logstash error message.')  
LOGGER.info('python-logstash: test logstash info message.')  
LOGGER.warning('python-logstash: test logstash warning message.')

# add extra field to logstash message

extra = {  
'test\_string': 'python version: ' + repr(sys.version\_info),  
'test\_boolean': True,  
'test\_dict': {'a': 1, 'b': 'c'},   
'test\_float': 1.23,  
'test\_integer': 1238888888,  
'test\_list': [1, 2, '3'],  
}

LOGGER.info("python-logstash: test extra fields", extra=extra)

and here is what I see in kibana (as you can see all the fields land in the 'message' due to the error with that token):

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/0/208dc596a1338c3bf9ddbc2f5b4a124f139ae0ff.png)

**HOW to fix that token???**

I tried to filter it out with mutate, but the issue is still there:

filter {  
mutate {  
add\_field =\> { "test\_string" =\> "Python version 1" }  
remove\_field =\> {"timestamp"}  
}  
}

or

filter {  
mutate {  
add\_field =\> { "test\_string" =\> "Python version 1" }  
remove\_field =\> {"mestamp"}  
}  
}

//////////////////////////////////////////////"solution"///////////////////////////////////////////////////////////////////////OK I AM PAST THAT PROBLEM NOW, the reason was too long integer in one of the extra fields created in py file, namely: 'test\_integer': 1238888888,

///////////////////////////////////////N E W P R O B L E M////////////////////////////////////////  
the new issue is with the mapping, when i reduced the length of that 'test\_integer' to 12, well yesterday all worked great, but today I am getting logstash error:

[2020-01-15T13:25:57,356][WARN][logstash.outputs.elasticsearch][main] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"mylogstash", :\_type=\>"\_doc", :routing=\>nil}, #LogStash::Event:0x5c5416ed], :response=\>{"index"=\>{"\_index"=\>"mylogstash", "\_type"=\>"\_doc", "\_id"=\>"dycqqW8BjlS\_m8VD4TZl", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"mapper [test\_list] of different type, current\_type [text], merged\_type [long]"}}}}

I suppose the mapping involved that too long integer already, or?  
How to solve the current issue?  
\*\*\*\*\*\*\*\*\*\*\*\* my logstash mapping is: \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

```
{
  "mapping": {
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "@version": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "host": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "level": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "logger_name": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "message": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "path": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "port": {
        "type": "long"
      },
      "tags": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "test_boolean": {
        "type": "boolean"
      },
      "test_dict": {
        "properties": {
          "a": {
            "type": "long"
          },
          "b": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "test_float": {
        "type": "float"
      },
      "test_string": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "type": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [January 15, 2020, 12:51pm UTC](https://discuss.elastic.co/t/json-unrecognized-token/215006/2 "2020-01-15T12:51:59Z")

</div>

Can you provide full index mapping in markdown ? to make this readable.

Also i think you just have te re-create a new index with test\_list of type list.

---

<div class="post-metadata">

**Author:** ![akuninja](https://avatars.discourse-cdn.com/v4/letter/a/85f322/32.png) [@akuninja](https://discuss.elastic.co/u/akuninja)\
**Post date:** [January 15, 2020, 1:11pm UTC](https://discuss.elastic.co/t/json-unrecognized-token/215006/3 "2020-01-15T13:11:20Z")

</div>

OK, I did that markdown.

What you mean would basically mean adding to mapping this??:

```
 "test_list": {
        "type": "list",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },

```

My thoughts are that if I got the error in my opinion connected to "mylogstash" index from logstash config file, I should somehow change the mapping in regards to that, no?

Here is my current logstash error:

[2020-01-15T13:58:15,340][WARN][logstash.outputs.elasticsearch][main] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"mylogstash", :\_type=\>"\_doc", :routing=\>nil}, #LogStash::Event:0x15996dd7], :response=\>{"index"=\>{"\_index"=\>"mylogstash", "\_type"=\>"\_doc", "\_id"=\>"gdZIqW8BgEpJs2Q7c-Od", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"mapper [test\_list] of different type, current\_type [text], merged\_type [long]"}}}}

And my current logstash file:

```
input {
  tcp {
    port => 5000
    codec => json
  }
}

filter {
	mutate {
		add_field => { "test_string" => "Python version 1" }
	}
}

	
output {
	elasticsearch{
		hosts => ["localhost:9200"]
		index => "mylogstash"
	}	
  
}

```

Is this the source of my problem : index =\> "mylogstash" ?  
Or at which step is that index assigned to \_index with the wrong type?

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [January 15, 2020, 1:21pm UTC](https://discuss.elastic.co/t/json-unrecognized-token/215006/4 "2020-01-15T13:21:31Z")

</div>

All i can see is that you have a type error with the "test\_list" field that apparently does not exist in your current mapping.

Try recreating the index with the field and type **text**

Reading the [docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html) the list datatype does not exist 😉

---

<div class="post-metadata">

**Author:** ![akuninja](https://avatars.discourse-cdn.com/v4/letter/a/85f322/32.png) [@akuninja](https://discuss.elastic.co/u/akuninja)\
**Post date:** [January 15, 2020, 1:54pm UTC](https://discuss.elastic.co/t/json-unrecognized-token/215006/5 "2020-01-15T13:54:15Z")

</div>

You are so enigmatic, Sir! 🙂  
Anyway, thank you.  
I hope I should do it like so:

```
"test_list": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "text",
            "ignore_above": 256
          }
        }
      },

```

One more thing however is where exactly can I find the .json responsible for that mapping?  
[I have a lot of mapping.json i the kibana plugin folder: .\kibana\_elasticsearch\kibana-7.4.1-windows-x86\_64\x-pack\legacy\plugins]

I do not have any console where I could get the mapping, as described [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html)

my cmd where I start ES and Kibana are not writable, same for logstash (for the current tcp input).  
Where is that json file and that console you talked about in the docs?

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [January 15, 2020, 2:06pm UTC](https://discuss.elastic.co/t/json-unrecognized-token/215006/6 "2020-01-15T14:06:05Z")

</div>

Thanks,

You're trying to edit the mappings.json ?

Index mappings cannot be edited/updated you have to create a new indices with "custom mapping"

In order to do this you have to query from kibana "Dev Tools"

and use the [API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html)

```auto
PUT new-example
{
  "mappings": {
    "properties": {
    	"@timestamp": {
        "type": "date" 
   [... Insert your mapping ...]
}

```

Then run the command using the button ( top right ) dont forget you have to delete the old one before.

To see the current mapping of a indices from the console :

```auto
GET example/_mapping
```

---

<div class="post-metadata">

**Author:** ![akuninja](https://avatars.discourse-cdn.com/v4/letter/a/85f322/32.png) [@akuninja](https://discuss.elastic.co/u/akuninja)\
**Post date:** [January 15, 2020, 2:37pm UTC](https://discuss.elastic.co/t/json-unrecognized-token/215006/7 "2020-01-15T14:37:18Z")

</div>

Ahaaa, now I understand! Thank you!

Still asking for the current mapping gives me this error:

```
{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index [example]",
        "resource.type" : "index_or_alias",
        "resource.id" : "example",
        "index_uuid" : "_na_",
        "index" : "example"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index [example]",
    "resource.type" : "index_or_alias",
    "resource.id" : "example",
    "index_uuid" : "_na_",
    "index" : "example"
  },
  "status" : 404
}

```

Although I can see the (different) mapping in the Kibana-\>IndexPatterns but I maybe I will try to grasp it myself from now on 🙂

Anyway, you have helped me a tone!  
BTW I love those Le gendarme se marie or Le Gendarme de St. Tropez, I did not know someone still does! 😃

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 12, 2020, 2:37pm UTC](https://discuss.elastic.co/t/json-unrecognized-token/215006/8 "2020-02-12T14:37:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.