# S3 CSV Codec - Using a non printable character as separator

**URL:** <https://discuss.elastic.co/t/s3-csv-codec-using-a-non-printable-character-as-separator/282262>\
**Category:** Logstash\
**Created:** [August 23, 2021, 4:36pm UTC](https://discuss.elastic.co/t/s3-csv-codec-using-a-non-printable-character-as-separator/282262 "2021-08-23T16:36:16Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hammond95](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hammond95/32/93591_2.png) [@Hammond95](https://discuss.elastic.co/u/Hammond95)\
**Post date:** [August 23, 2021, 4:36pm UTC](https://discuss.elastic.co/t/s3-csv-codec-using-a-non-printable-character-as-separator/282262/1 "2021-08-23T16:36:16Z")

</div>

Is it possible to use a non printable character as a separator in the csv codec?

```auto
codec => csv {
          columns => ["col1", "col2", "col3"]
          charset => "UTF-8"
          separator => "\\u001F"
}

```

I have tried the following:  
`\u001f`, `\u001F`, `\\u001f`, `\\u001F`, `\u{001F}`

some ideas?

If this is not possible, how would I have to modify the code to make it possible?

> <https://github.com/logstash-plugins/logstash-codec-csv/blob/master/lib/logstash/codecs/csv.rb>

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 23, 2021, 5:16pm UTC](https://discuss.elastic.co/t/s3-csv-codec-using-a-non-printable-character-as-separator/282262/2 "2021-08-23T17:16:04Z")

</div>

It's not the codec that you would modify to accept those UTF-16 strings, it is the compiler for the logstash configuration language.

However, there is no need to do that, you can just use the literal character. When logged into a UNIX host from a Windows environment I would use Ctrl/V Alt031 to generate it. od shows that as a 'us' character.

---

<div class="post-metadata">

**Author:** ![Hammond95](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hammond95/32/93591_2.png) [@Hammond95](https://discuss.elastic.co/u/Hammond95)\
**Post date:** [August 23, 2021, 5:31pm UTC](https://discuss.elastic.co/t/s3-csv-codec-using-a-non-printable-character-as-separator/282262/3 "2021-08-23T17:31:51Z")

</div>

Yeah I know that I can copy paste it, but I wanted to avoid this option and stick with the \u representation or \x repr.

I have tried but my YAML is not happy with that character...

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/1/111c11625ae26704ef0be886b3ecb74d1f058cd9.png)

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 24, 2021, 3:30am UTC](https://discuss.elastic.co/t/s3-csv-codec-using-a-non-printable-character-as-separator/282262/4 "2021-08-24T03:30:30Z")

</div>

An interesting related post is [here](https://discuss.elastic.co/t/logstash-http-poller-body-escape-double-and-single-quotation-marks/282039/2). In that case \u00xx works inside a string, but not because logstash decodes it. I would guess something in Manticore decodes it, or else on the receiving end in elasticsearch. It will not work here.

---

<div class="post-metadata">

**Author:** ![Hammond95](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hammond95/32/93591_2.png) [@Hammond95](https://discuss.elastic.co/u/Hammond95)\
**Post date:** [August 25, 2021, 12:55am UTC](https://discuss.elastic.co/t/s3-csv-codec-using-a-non-printable-character-as-separator/282262/5 "2021-08-25T00:55:47Z")

</div>

In the end I did my fork of the plugin to use that specific character.

> **[GitHub - Hammond95/logstash-codec-csv: This codec takes care of decoding and...](https://github.com/Hammond95/logstash-codec-csv)**
>
> This codec takes care of decoding and encoding csv data - GitHub - Hammond95/logstash-codec-csv: This codec takes care of decoding and encoding csv data

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 22, 2021, 12:56am UTC](https://discuss.elastic.co/t/s3-csv-codec-using-a-non-printable-character-as-separator/282262/6 "2021-09-22T00:56:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.