# Csv parsing through logstash

**URL:** <https://discuss.elastic.co/t/csv-parsing-through-logstash/130249>\
**Category:** Logstash\
**Created:** [May 2, 2018, 12:52pm UTC](https://discuss.elastic.co/t/csv-parsing-through-logstash/130249 "2018-05-02T12:52:47Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Raj\_Kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raj_kumar/32/25420_2.png) [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\
**Post date:** [May 2, 2018, 12:52pm UTC](https://discuss.elastic.co/t/csv-parsing-through-logstash/130249/1 "2018-05-02T12:52:49Z")

</div>

Hi All,

Iam trying to parse my csv file to elasticsearch through logstash

my config looks like this

```auto
input {
      file {
          path => "/home/raj/Uge 16 Spyware-malware total7.csv"
          type => "trend_micro_spyware"
          start_position => "beginning"
  }
}

filter {
   if [type] == "trend_micro_spyware" {
    csv {
        columns => ["Received", "Generated","Product Entity/Endpoint", "Product", "Spyware/Grayware", "Endpoint","Source Host", "User", "Result","Detections", "Channel"]
        skip_empty_columns => "true"
        skip_empty_rows => "true"
        skip_header => "true"
    }
}
}

```

but am getting it in kibana like this

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/7/6740e054bd158b3f98aa2a5dc874b2c1ab25b352.png)

which includes a separate document with csv headers

and when i see the message its like this

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/6/6696010683a83b3b1921c86e4d4986d5632bdfcd.png)

Please help me to figure out this issue.

Thanks,  
Raj

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 2, 2018, 1:05pm UTC](https://discuss.elastic.co/t/csv-parsing-through-logstash/130249/2 "2018-05-02T13:05:41Z")

</div>

Whilst the file may be called .csv, it appears to be either space or tab separated, so you need to supply the [separator](https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html#plugins-filters-csv-separator) option to the csv filter.

---

<div class="post-metadata">

**Author:** ![Raj\_Kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raj_kumar/32/25420_2.png) [@Raj\_Kumar](https://discuss.elastic.co/u/Raj_Kumar)\
**Post date:** [May 2, 2018, 1:46pm UTC](https://discuss.elastic.co/t/csv-parsing-through-logstash/130249/3 "2018-05-02T13:46:01Z")

</div>

Thank you Badger for the info , i tried to use the separator

```auto
    csv {
        columns => ["Received", "Generated","Product Entity/Endpoint", "Product", "Spyware/Grayware", "Endpoint","Source Host", "User", "Result","Detections", "Channel"]
        separator => " "
        skip_empty_columns => "true"
        skip_empty_rows => "true"
        skip_header => "true"
    }

![image|690x345](upload://v2pyDEvzpEIh3Z6OU7EyvxT6eti.png)

1.First thing it splitted even the values, for example in the screenshot its one value like 'host details'
since it has space it splitted in to two different values like host and detail separately
2. Secondly, header was not removed ,empty columns and rows was not removed

Thanks,
Raj
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 2, 2018, 1:54pm UTC](https://discuss.elastic.co/t/csv-parsing-through-logstash/130249/4 "2018-05-02T13:54:23Z")

</div>

Check out the skip\_header and skip\_empty\_rows options for the filter.

If your fields really are space separated and contain embedded spaces without quotes then the format is ambiguous and csv will not be able to parse it. However, the message looks like it is either tab separated (which csv can handle) or fixed-width (which grok would be better for).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 30, 2018, 1:54pm UTC](https://discuss.elastic.co/t/csv-parsing-through-logstash/130249/5 "2018-05-30T13:54:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.