# Persistent queue breaks UTF-8 international language

**URL:** <https://discuss.elastic.co/t/persistent-queue-breaks-utf-8-international-language/130175>\
**Category:** Logstash\
**Created:** [May 2, 2018, 2:30am UTC](https://discuss.elastic.co/t/persistent-queue-breaks-utf-8-international-language/130175 "2018-05-02T02:30:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ilpc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ilpc/32/48756_2.png) [@ilpc](https://discuss.elastic.co/u/ilpc)\
**Post date:** [May 2, 2018, 2:30am UTC](https://discuss.elastic.co/t/persistent-queue-breaks-utf-8-international-language/130175/1 "2018-05-02T02:30:55Z")

</div>

my.conf:  
input {  
file { path =\> ["/var/log/logstash/input.log"] }  
}  
output {  
file { path =\> "/var/log/logstash/output.log" }  
}

input.log:  
1가2나3다4

It works fine on memory queue.

logstash.yml:  
path.data: /var/lib/logstash  
#queue.type: persisted  
path.logs: /var/log/logstash  
xpack.monitoring.elasticsearch.url: [http://xxxxx:9200](http://xxxxx:9200)

output.log:  
{"@timestamp":"2018-05-02T02:02:53.649Z","message":"1가2나3다4","@version":"1","path":"/var/log/logstash/input.log","host":"xxxxx"}

But it looks like persisted queue breaks international language encoded by UTF-8.

logstash.yml:  
path.data: /var/lib/logstash  
queue.type: persisted  
path.logs: /var/log/logstash  
xpack.monitoring.elasticsearch.url: [http://xxxxx:9200](http://xxxxx:9200)

output.log:  
{"host":"xxxxx","@version":"1","path":"/var/log/logstash/input.log","@timestamp":"2018-05-02T02:01:20.209Z","message":"1가2나3다4"}

Here is the hex code of input/output.

![input](https://us1.discourse-cdn.com/elastic/original/3X/c/e/cee6936fcbfcc43407514f600cf5606b5b9fdd2b.PNG)  
 ![output2](https://us1.discourse-cdn.com/elastic/original/3X/0/d/0d7361fbc905382225a3ab592eabcca8b6bacabe.PNG)

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [May 2, 2018, 6:36am UTC](https://discuss.elastic.co/t/persistent-queue-breaks-utf-8-international-language/130175/2 "2018-05-02T06:36:54Z")

</div>

This is a known issue that should be fixed with 6.2.4.

---

<div class="post-metadata">

**Author:** ![ilpc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ilpc/32/48756_2.png) [@ilpc](https://discuss.elastic.co/u/ilpc)\
**Post date:** [May 2, 2018, 8:46am UTC](https://discuss.elastic.co/t/persistent-queue-breaks-utf-8-international-language/130175/3 "2018-05-02T08:46:48Z")

</div>

Fantastic! Thank you for the response.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 30, 2018, 8:46am UTC](https://discuss.elastic.co/t/persistent-queue-breaks-utf-8-international-language/130175/4 "2018-05-30T08:46:50Z")

</div>

Closed on May 30, 2018, 8:46am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.