# UTF-16 Broken since logstash 6

**URL:** <https://discuss.elastic.co/t/utf-16-broken-since-logstash-6/135558>\
**Category:** Logstash\
**Created:** [June 12, 2018, 2:24pm UTC](https://discuss.elastic.co/t/utf-16-broken-since-logstash-6/135558 "2018-06-12T14:24:09Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![maigel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maigel/32/81104_2.png) [@maigel](https://discuss.elastic.co/u/maigel)\
**Post date:** [June 12, 2018, 2:24pm UTC](https://discuss.elastic.co/t/utf-16-broken-since-logstash-6/135558/1 "2018-06-12T14:24:09Z")

</div>

Hello,

I recently migrated our cluster from ELK2 to ELK6.  
One of our logstash configs is used to parse some powershell CSV output, which is in UTF-16LE  
(Little-endian UTF-16 Unicode text, with very long lines, with CRLF line terminators)

In logstash 2 I had the following simple config, which worked fine:

```
input {
  file { 
    path => ["/data/*.csv"]
    start_position => "beginning"
    codec => plain { charset => "UTF-16LE"}
    type => "office365"
  }
}

filter {
  if [type] == "office365" {
    csv {
      columns => ["PSComputerName","RunspaceId","PSShowComputerName","RecordType","CreationDate","UserIds","Operations","AuditData","ResultIndex","ResultCount","Identity","IsValid","ObjectState"]
    }
  }
}

output {
  if [type] == "office365" {
    stdout { codec => rubydebug }
  }
}

```

However, since logstash 6 the UTF-16 support seems to be broken. The CSV fails to parse and the output message I get seems to only contain what seems to be Chinese letters.  
Were there any changes to the UTF-16 decoder in logstash 6?

When I convert the input file to UTF-8 first using:  
iconv -f UTF-16LE -t UTF-8 input.csv \> output.csv  
And remove the codec UTF-8 line, the file gets parsed fine

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 10, 2018, 2:24pm UTC](https://discuss.elastic.co/t/utf-16-broken-since-logstash-6/135558/2 "2018-07-10T14:24:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.