# ES isn't properly handling unicode ? advice for debugging this problem?

**URL:** <https://discuss.elastic.co/t/es-isnt-properly-handling-unicode-advice-for-debugging-this-problem/22651>\
**Category:** Elasticsearch\
**Created:** [March 13, 2015, 4:31am UTC](https://discuss.elastic.co/t/es-isnt-properly-handling-unicode-advice-for-debugging-this-problem/22651 "2015-03-13T04:31:09Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![burtonator](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/burtonator/32/44790_2.png) [@burtonator](https://discuss.elastic.co/u/burtonator)\
**Post date:** [March 13, 2015, 4:31am UTC](https://discuss.elastic.co/t/es-isnt-properly-handling-unicode-advice-for-debugging-this-problem/22651/1 "2015-03-13T04:31:09Z")

</div>

I have unit tests setup to test using transport client to write unicode  
data into ES and then read it back out.

It's using the standard ElasticsearchIntegrationTest that ES recommends.

I'm using MY JSON encoder... and then I write my JSON to the  
TransportClient, and read it back out, and it's correct!

The problem is that IN PRODUCTION it doesn't work and all my data is  
garbled. I _think_ it's treading the data either as ASCII or ISO-8859-1  
(which are the usual defaults).

What's the best way to test this.

I imagine I could sue ethereal to look at the raw protocol and verify that  
the data is being sent properly.

The only other thing I can think of is to step through ES directly in a  
debugger.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/fa5bb1a1-9e1e-4376-b947-5bffeab8efee%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/fa5bb1a1-9e1e-4376-b947-5bffeab8efee%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:27am UTC](https://discuss.elastic.co/t/es-isnt-properly-handling-unicode-advice-for-debugging-this-problem/22651/2 "2017-07-06T00:27:05Z")

</div>