# Logstash - csv parsing failure

**URL:** <https://discuss.elastic.co/t/logstash-csv-parsing-failure/195415>\
**Category:** Logstash\
**Created:** [August 16, 2019, 2:49am UTC](https://discuss.elastic.co/t/logstash-csv-parsing-failure/195415 "2019-08-16T02:49:20Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![cheriemilk](https://avatars.discourse-cdn.com/v4/letter/c/c37758/32.png) [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\
**Post date:** [August 16, 2019, 2:49am UTC](https://discuss.elastic.co/t/logstash-csv-parsing-failure/195415/1 "2019-08-16T02:49:20Z")

</div>

Hi Team,

I have below data in a csv file which has two columns only. As you see, the content in msg columns actually are a few of key-vlaue pairs, and I want logstash to parse these key-value pairs as well.  
indent preformatted text by 4 spaces

> ```
> @timestamp,msg
> "August 15th 2019, 06:41:24.789","module=SCM fa=TS at=SCM.TS.SEARCH si=4C3D8709E51DDC4EE879A9E30729B512.mo-5692ea7ca ci=SCMStella cn=SCMStella cs=qacandrot_SCMStella. pi=dbPool1 ui=cgrant1 locale=en_US ktf1=[C,E,X,H,M] bf1=false bf2=false ktf2=Keyword if1=0"
> "August 15th 2019, 06:41:20.318","module=SCM fa=TS at=SCM.TS.MODIFY_SEARCH si=4C3D8709E51DDC4EE879A9E30729B512.mo-5692ea7ca ci=SCMStella cn=SCMStella cs=qacandrot_SCMStella. pi=dbPool1 ui=cgrant1 locale=en_US ktf1=[C,E,X,H,M]".
> 
> ```

Below is the filter plug-in part  
indent preformatted text by 4 spaces

> ```
> filter { 
> csv { columns => [ "@timestamp",
> "msg"]
> separator => ","
> skip_header => "true"
> } 
> 
> if [field] == "msg" {
> 
> kv {
> field_split => " "
> value_split => "="
> trim_key => " "
> trim_value => " "
> include_brackets => false
>                
> }
> }
>            
> }
> 
> ```

But after I run command logstash -f ../config/logstash.conf, I got below error.  
indent preformatted text by 4 spaces

```
[2019-08-15T16:41:32,040][DEBUG][logstash.filters.csv] Running csv filter {:event=>#<LogStash::Event:0x9edd36>}
[2019-08-15T16:41:32,169][DEBUG][logstash.filters.csv] Running csv filter {:event=>#<LogStash::Event:0x6c264f05>}
[2019-08-15T16:41:32,199][WARN][logstash.filters.csv] Error parsing csv {:field=>"message", :source=>"\"August 15th 2019, 06:41:24.789\",\"module=SCM fa=TS at=SCM.TS.SEARCH si=4C3D8709E51DDC4EE879A9E30729B512.mo-5692ea7ca ci=SCMStella cn=SCMStella cs=qacandrot_SCMStella. pi=dbPool1 ui=cgrant1 locale=en_US ktf1=[C,E,X,H,M] bf1=false bf2=false ktf2=Keyword if1=0\"\r", :exception=>#<TypeError: wrong argument type String (expected LogStash::Timestamp)>}
[2019-08-15T16:41:32,202][DEBUG][logstash.filters.csv] Running csv filter {:event=>#<LogStash::Event:0xe366c5a>}
[2019-08-15T16:41:32,206][WARN][logstash.filters.csv] Error parsing csv {:field=>"message", :source=>"\"August 15th 2019, 06:41:20.318\",\"module=SCM fa=TS at=SCM.TS.MODIFY_SEARCH si=4C3D8709E51DDC4EE879A9E30729B512.mo-5692ea7ca ci=SCMStella cn=SCMStella cs=qacandrot_SCMStella. pi=dbPool1 ui=cgrant1 locale=en_US ktf1=[C,E,X,H,M]\"\r", :exception=>#<TypeError: wrong argument type String (expected LogStash::Timestamp)>}
C:/elkstack/logstash-7.0.1/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated

```

What is the issue here?

---

<div class="post-metadata">

**Author:** ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)\
**Post date:** [August 16, 2019, 8:38pm UTC](https://discuss.elastic.co/t/logstash-csv-parsing-failure/195415/2 "2019-08-16T20:38:53Z")

</div>

> [@cheriemilk](#):
>
> csv { columns =\> ["@timestamp", "msg"]

This is your problem.  
@timestamp is of type Timestamp. You need to first ingest the first column as your csv as a string and then use the [date filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html) to parse it into @timestamp.

Like this:  
Note: This may not work as I'm not sure if the "th" in the date will translate properly. If it does not, then you'll need a GROK filter first.

> ```
> csv { columns => [ "string_timestamp",
> "msg"]
> separator => ","
> skip_header => "true"
> } 
> date {
> target => "@timestamp"
> match => ["string_timestamp","MMMMM ddth yyyy, HH:mm:ss.SSS"]
> #August 15th 2019, 06:41:24.789
> }
> 
> ```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 16, 2019, 8:52pm UTC](https://discuss.elastic.co/t/logstash-csv-parsing-failure/195415/3 "2019-08-16T20:52:22Z")

</div>

You can use mutate+gsub to remove the ordinal indicator.

```
mutate { gsub => ["timestamp", "(\d)(st|nd|rd|th) ", "\1 "] }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 13, 2019, 8:57pm UTC](https://discuss.elastic.co/t/logstash-csv-parsing-failure/195415/4 "2019-09-13T20:57:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.