# Unable to parse CSV input from Filebeat -\> Logstash

**URL:** <https://discuss.elastic.co/t/unable-to-parse-csv-input-from-filebeat-logstash/174952>\
**Category:** Logstash\
**Created:** [April 2, 2019, 9:14am UTC](https://discuss.elastic.co/t/unable-to-parse-csv-input-from-filebeat-logstash/174952 "2019-04-02T09:14:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![avj1986](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/avj1986/32/42478_2.png) [@avj1986](https://discuss.elastic.co/u/avj1986)\
**Post date:** [April 2, 2019, 9:14am UTC](https://discuss.elastic.co/t/unable-to-parse-csv-input-from-filebeat-logstash/174952/1 "2019-04-02T09:14:21Z")

</div>

Hi,

I am trying to read a CSV file using Filebeat and sending the data to Logstash.  
On Logstash, I need to parse this CSV data and load it into ElasticSearch.  
I am able to ship the data from Filebeat to Logstash to Elastic search using CSV plugin,  
however, the event data is not split into respective columns. Index doesn't have fields pertaining to CSV column headers.  
Entire data seems to be dumped under "message" field.  
I tried multiple options, but no luck.  
Here are my configurations for reference:

**FileCsv.csv**

```
ProcessorTime, Rate, Interval, Dontwantthis, WantThis, DontWantThisOneToo
Sample1, 2, 3, A, 4, B
Sample2, 2, 3, Y, 4, Z

```

**Filebeat.conf:**

```
filebeat.inputs:
- type: log
  paths: 
        - /Users/amit_joshi/ElasticData/spool/FileReader*.log
  fields: 
        document_type: detectionLogs

- type: log 
  paths:
        - /Users/amit_joshi/ElasticData/spool/FileCsv*.csv
  fields:
        document_type: perfMonLogs

include_lines: ['^FINEST']
exclude_lines: ['^ProcessorTime']

output.logstash:
  hosts: ["localhost:5044"]

```

**Logstash.conf:**

```
input {
  beats {
    port => 5044
  }
}
filter {

if ([fields][log_type] == "detectionLogs") 
    {
        #Some processing statements
    }
else if ([fields][log_type] == "perfMonLogs") 
    {
        csv {
                columns => ["ProcessorTime", "Rate","Interval","Dontwantthis","WantThis","DontWantThisOneToo"]
                separator => ","
            }
    }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "mock_20march_2019"
    #user => "elastic"
    #password => "changeme"
  }
stdout { codec => rubydebug }

}
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 2, 2019, 1:15pm UTC](https://discuss.elastic.co/t/unable-to-parse-csv-input-from-filebeat-logstash/174952/2 "2019-04-02T13:15:16Z")

</div>

You add a field called document\_type, but you test a field called log\_type. Neither the if nor the else if will match.

---

<div class="post-metadata">

**Author:** ![avj1986](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/avj1986/32/42478_2.png) [@avj1986](https://discuss.elastic.co/u/avj1986)\
**Post date:** [April 2, 2019, 1:37pm UTC](https://discuss.elastic.co/t/unable-to-parse-csv-input-from-filebeat-logstash/174952/3 "2019-04-02T13:37:06Z")

</div>

@Badger, That's really bad on my part. Changing the field name in filebeat config to "log\_type", resolved the issue.  
Thanks a ton!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 30, 2019, 1:43pm UTC](https://discuss.elastic.co/t/unable-to-parse-csv-input-from-filebeat-logstash/174952/4 "2019-04-30T13:43:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.