# Logstash Kafka input failed to parse JSON

**URL:** <https://discuss.elastic.co/t/logstash-kafka-input-failed-to-parse-json/51620>\
**Category:** Logstash\
**Created:** [June 1, 2016, 10:00pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-failed-to-parse-json/51620 "2016-06-01T22:00:08Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hongwei\_Liu](https://avatars.discourse-cdn.com/v4/letter/h/a587f6/32.png) [@Hongwei\_Liu](https://discuss.elastic.co/u/Hongwei_Liu)\
**Post date:** [June 1, 2016, 10:00pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-failed-to-parse-json/51620/1 "2016-06-01T22:00:08Z")

</div>

hi experts,

i am using logstash to read data from kafka cluster and output to elasticsearch. but i encountered this error - in the message it contains hex characters, so failed to parse the json.

{:timestamp=\>"2016-06-01T21:36:03.222000+0000", :message=\>"JSON parse failure. Falling back to plain-text", :error=\>#\<LogStash::Json::ParserError: Unexpected character ('' (code 92)): was expecting double-quote to start field name  
at [Source: [B@4e292087; line: 1, column: 3]\>, :data=\>"{\"Operator\":\"tls\",\"Environment\":\"proda\",\"Site\":\"A\",\"Slot\":\"S58\",\"SessionId\":\"Null Client InstrumentationContext\",\"Association\":null,\"ServerOrigin\":\"InstrumentationContextExtensions\",\"ListFields\":\"{\\\"titleId\\\":\\\"1257\_907395\_Live\\\",\\\"matchStatus\\\":\\\"PartialMatchFound\\\",\\\"title\\\":\\\"Il **\xE9tait une fois... les** Explorateurs\\\",\\\"episodeTitle\\\":null,\\\"releaseYear\\\":null,\\\"originalAirDate\\\":\\\"1996-12-10T00:00:00Z\\\",\\\"matchedProvider\\\":\\\"tms\\\",\\\"matchedProgramId\\\":null,\\\"matchedSeriesId\\\":\\\"SH002351360000\\\",\\\"matchedCandidates\\\":\\\"SH001587420000,SH018724030000,SH017853210000,SH023917590000,SH000742910000\\\",\\\"frmt\\\":\\\"Matching response for title {titleId}: {matchStatus}.\\\\nRequest:\\\\n\\\\tTitle: {title}\\\\n\\\\tEpisodeTitle: {episodeTitle}\\\\n\\\\tReleaseYear: {releaseYear}\\\\n\\\\tOriginalAirDate: {originalAirDate}\\\\nResponse:\\\\n\\\\tProvider: {matchedProvider}\\\\n\\\\tProgramId: {matchedProgramId}\\\\n\\\\tSeriesId: {matchedSeriesId}\\\\n\\\\tCandidates: {matchedCandidates}\\\"}\",\"RoleName\":\"catalogworker\",\"InstanceNumber\":\"2\",\"DeploymentId\":\"catalogworker-s58-a-proda-tls\_1.0.1591.1\",\"LogTime\":\"2016-05-29T18:28:57.2095678Z\",\"Group\":8,\"Name\":\"UniversalMatchingManager\",\"esIndex\":\"logs-2016.05.29\",\"esType\":\"tls\_proda\_A\"}", :level=\>:info}  
{:timestamp=\>"2016-06-01T21:36:03.224000+0000", :message=\>"Received an event that has a different character encoding than you configured.", :text=\>"{\"Operator\":\"tls\",\"Environment\":\"proda\",\"Site\":\"A\",\"Slot\":\"S58\",\"SessionId\":\"Null Client InstrumentationContext\",\"Association\":null,\"ServerOrigin\":\"InstrumentationContextExtensions\",\"ListFields\":\"{\\\"titleId\\\":\\\"1257\_626316\_Live\\\",\\\"matchStatus\\\":\\\"MatchFound\\\",\\\"title\\\":\\\"Hors s\xE9rie\\\",\\\"episodeTitle\\\":null,\\\"releaseYear\\\":null,\\\"originalAirDate\\\":\\\"2012-11-09T00:00:00Z\\\",\\\"matchedProvider\\\":\\\"tms\\\",\\\"matchedProgramId\\\":\\\"EP016354030001\\\",\\\"matchedSeriesId\\\":\\\"SH016354030000\\\",\\\"matchedCandidates\\\":\\\"EP016354030002,EP016354030003,EP016354030004,EP016354030005,EP016354030006\\\",\\\"frmt\\\":\\\"Matching response for title {titleId}: {matchStatus}.\\\\nRequest:\\\\n\\\\tTitle: {title}\\\\n\\\\tEpisodeTitle: {episodeTitle}\\\\n\\\\tReleaseYear: {releaseYear}\\\\n\\\\tOriginalAirDate: {originalAirDate}\\\\nResponse:\\\\n\\\\tProvider: {matchedProvider}\\\\n\\\\tProgramId: {matchedProgramId}\\\\n\\\\tSeriesId: {matchedSeriesId}\\\\n\\\\tCandidates: {matchedCandidates}\\\"}\",\"RoleName\":\"catalogworker\",\"InstanceNumber\":\"3\",\"DeploymentId\":\"catalogworker-s58-a-proda-tls\_1.0.1591.1\",\"LogTime\":\"2016-05-29T18:28:46.1512684Z\",\"Group\":8,\"Name\":\"UniversalMatchingManager\",\"esIndex\":\"logs-2016.05.29\",\"esType\":\"tls\_proda\_A\"}", :expected\_charset=\>"UTF-8", :level=\>:warn}

here is my config  
input {  
kafka {  
topic\_id =\> "elasticsearchtopiclogs"  
group\_id =\> "elasticsearchtopiclogs-tls-a-debug"  
zk\_connect =\> "xxx"  
auto\_offset\_reset =\> "smallest"  
reset\_beginning =\> "true"  
consumer\_threads =\> 3  
codec =\> json {  
charset =\> "UTF-8"  
}  
}  
}

filter {}

output {  
if "\_jsonparsefailure" in [tags] {  
stdout { codec =\> rubydebug { metadata =\> true }}  
}

elasticsearch {  
hosts =\> ["[https://xxx:9200](https://xxx:9200)"]  
ssl =\> true  
index =\> "debug-logs-%{+YYYY.MM.dd}"  
user =\> "xxx"  
password =\> "xxx"  
document\_type =\> "%{estype}"  
template\_name =\> "logs-logstash-template"  
workers =\> 5  
}  
}  
can someone help me to correctly handle this ? how to ignore the error without block the whole pipeline ?  
Thanks  
Hongwei

---

<div class="post-metadata">

**Author:** ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)\
**Post date:** [June 9, 2016, 3:32am UTC](https://discuss.elastic.co/t/logstash-kafka-input-failed-to-parse-json/51620/2 "2016-06-09T03:32:28Z")

</div>

Seems like an encoding issue. Check out [https://github.com/elastic/logstash/issues/1695](https://github.com/elastic/logstash/issues/1695) and [https://www.elastic.co/guide/en/logstash/current/plugins-filters-urldecode.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-urldecode.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:53am UTC](https://discuss.elastic.co/t/logstash-kafka-input-failed-to-parse-json/51620/3 "2017-07-06T04:53:58Z")

</div>