# Read CSV without introducing, and having to remove, a message field?

**URL:** <https://discuss.elastic.co/t/read-csv-without-introducing-and-having-to-remove-a-message-field/78809>\
**Category:** Logstash\
**Created:** [March 16, 2017, 7:56am UTC](https://discuss.elastic.co/t/read-csv-without-introducing-and-having-to-remove-a-message-field/78809 "2017-03-16T07:56:53Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![GrahamHannington](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grahamhannington/32/4404_2.png) [@GrahamHannington](https://discuss.elastic.co/u/GrahamHannington)\
**Post date:** [March 16, 2017, 7:56am UTC](https://discuss.elastic.co/t/read-csv-without-introducing-and-having-to-remove-a-message-field/78809/1 "2017-03-16T07:56:53Z")

</div>

I can use Logstash to read a file of JSON Lines from `stdin` without introducing—and, hence, without needing to remove—a `message` field:

```
input {
  stdin {
   codec => json_lines
  }
}

```

(No filter section required; the config outputs to Elasticsearch.)

Is there some combination of codec and filter that I can use to read a CSV file from `stdin`, without having to explicitly remove the `message` field?

Here’s what I’m currently using to read a CSV file from `stdin`:

```
input {
  stdin {
  }
}
filter {
  csv {
    columns => [...]
    remove_field => ["message"]
  }
}

```

I think the answer is “No”, because the `csv` filter requires a `source` field name (and the default value is `"message"`), but perhaps there’s a way to do it that I can’t see.

I’m currently testing in Logstash 2.4, where the default codec for `stdin` is `line`. I notice that, from Logstash 5.1 (I’m about to start using 5.2), the default codec for `stdin` is `plain`. I can’t tell from the user documentation for any of these codecs whether they output a `message` field: in practice, `json_lines` doesn’t, `line` does. I’d appreciate tips on that. “Read the codec source” would be a fair comment 🙂.

---

<div class="post-metadata">

**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\
**Post date:** [March 16, 2017, 1:03pm UTC](https://discuss.elastic.co/t/read-csv-without-introducing-and-having-to-remove-a-message-field/78809/2 "2017-03-16T13:03:21Z")

</div>

Yep, you guessed it. Its not possible.  
You would need a `csv_lines` codec.  
No one has written it yet.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 13, 2017, 1:03pm UTC](https://discuss.elastic.co/t/read-csv-without-introducing-and-having-to-remove-a-message-field/78809/3 "2017-04-13T13:03:21Z")

</div>

Closed on April 13, 2017, 1:03pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.