# Csv -\> filebeat -\> logstash not parsing?

**URL:** https://discuss.elastic.co/t/csv-filebeat-logstash-not-parsing/134317
**Category:** Beats
**Tags:** filebeat
**Created:** [June 3, 2018, 2:47pm UTC](https://discuss.elastic.co/t/csv-filebeat-logstash-not-parsing/134317 "2018-06-03T14:47:38Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![TestCandidate](https://avatars.discourse-cdn.com/v4/letter/t/8dc957/32.png) [@TestCandidate](https://discuss.elastic.co/u/TestCandidate)
#### Post date: [June 3, 2018, 2:47pm UTC](https://discuss.elastic.co/t/csv-filebeat-logstash-not-parsing/134317/1 "2018-06-03T14:47:38Z")

</div>

I have configured a server using elk stack with filebeat.  
In a folder, i have multiple CSV files. I have managed to get filebeat to read them using:

```
  paths:
     - /var/capture/dpi-tcp/*.csv

```

Now for logstash, I have 4 configuration files in conf.d as following:

**02-beats-input.conf**

```
input {
  beats {
port => 5044
  }
}

```

**06-tcp-filter.conf (THIS IS SUPPOSED TO PARSE THE CSV)**

```
filter {
  if [path] =~ "tcp" {
   csv {
      columns => [
          "logdate",
          "ppoeip",
          "userip",
          "protocol",
          "logserverip",
          "destinationip",
          "remove1",
          "remove2",
          "sourceport",
          "destinationport",
          "description-url"
        ]
        separator => ";"
        remove_field => ["remove1", "remove2"]
   }
  }
}

```

**10-syslog-filter.conf**

```
filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => ["received_at", "%{@timestamp}"]
      add_field => ["received_from", "%{host}"]
    }
    syslog_pri { }
    date {
      match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
    }
  }
}

```

**30-elasticsearch-output.conf**

```
output {
  elasticsearch {
    hosts => ["localhost:9200"]
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

```

**The Problem**  
My issue is, the data are being sent. I can see them in Kibana. However,  
All the CSV file is not being parsed, instead, the records are appearing in a filebeat object inside a "message field"

I'm not sure what I'm doing wrong.  
Any advise would be helpful.  
Thanks

---

<div class="post-metadata">

### Author: ![TestCandidate](https://avatars.discourse-cdn.com/v4/letter/t/8dc957/32.png) [@TestCandidate](https://discuss.elastic.co/u/TestCandidate)
#### Post date: [June 3, 2018, 3:59pm UTC](https://discuss.elastic.co/t/csv-filebeat-logstash-not-parsing/134317/2 "2018-06-03T15:59:21Z")

</div>

NVM.

Stupidly enough, I was outputting directly to elasticsearch instead of logstash.

Thanks

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 1, 2018, 3:59pm UTC](https://discuss.elastic.co/t/csv-filebeat-logstash-not-parsing/134317/3 "2018-07-01T15:59:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.