# Top-level configuration not working

**URL:** <https://discuss.elastic.co/t/top-level-configuration-not-working/216961>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 29, 2020, 9:03am UTC](https://discuss.elastic.co/t/top-level-configuration-not-working/216961 "2020-01-29T09:03:08Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ELK4Life](https://avatars.discourse-cdn.com/v4/letter/e/f4b2a3/32.png) [@ELK4Life](https://discuss.elastic.co/u/ELK4Life)\
**Post date:** [January 29, 2020, 9:03am UTC](https://discuss.elastic.co/t/top-level-configuration-not-working/216961/1 "2020-01-29T09:03:09Z")

</div>

Dear Logstash community,

I have challenged myself to capture my network data using TShark and make custom dashboards in Kibana.  
I have the following setup:

- 1 Ubuntu 18.04 VM with ElasticSearch, Logstash and Kibana dockerized
- 1 Ubuntu 18.04 VM with TShark and Filebeat running on the host

I run TShark using the specified `-T ek` flag and export the capture to a rolling CSV file.  
Using Filebeat I send the CSV file to Logstash. Logstash will then send the file to Elastic Search which will be queried by Kibana.

**My problem is: Kibana shows the Logstash logs as it's main input and per event includes one whole CSV capture line in a single field. How do I solve this that the CSV becomes the top-level input?**

 ![Kibana view](https://us1.discourse-cdn.com/elastic/original/3X/b/5/b56718828fd715f8e24dfce1dd45cecff1f6e2b8.png)

This is my logstash.conf:

```auto
input {
  beats {
    port => 5044
  }
}

filter {
	csv {
		source => "message"
		columns => ["col.Time", "col.Source", "col.Destination", "ip.src", "ip.dst", "tcp.srcport", "tcp.dstport", "col.Protocol", "ip.len", "col.Info"]
		}
   
	mutate {
      convert => ["ip-len", "integer"]
	}
	date {
	   match => ["col.time", "YYYY-MM-DD HH:mm:ss.SSSSSSSSS"]
       target => "@timestamp"
	}
}  
  
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" 
    document_type => "%{[@metadata][type]}" 
    user => elastic
    password => changeme
  }
}

```

This is my filebeat.yml:

```auto
filebeat.modules:
- module: system
  syslog:
    enabled: false
  auth:
    enabled: true
    var.paths: ["/home/user/Documents/tsharkcap/tshark.csv"]
output.logstash:
  hosts: ["192.168.234.134:5044"]

```

I run ElasticSearch with default config:

```auto
sudo docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.5.2

```

I run Kibana with default config:

```auto
sudo docker run --link docker-cont:elasticsearch -p5601:5601 docker.elastic.co/kibana/kibana:7.5.2

```

Thanks in advance,  
ELK4Life

---

<div class="post-metadata">

**Author:** ![inhinyera16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inhinyera16/32/61625_2.png) [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\
**Post date:** [January 29, 2020, 10:17am UTC](https://discuss.elastic.co/t/top-level-configuration-not-working/216961/2 "2020-01-29T10:17:04Z")

</div>

You need to enable multiline configuration in your filebeat.yml

Example:  
multiline.pattern: ',\d+,[^",]+$'  
multiline.negate: true  
multiline.match: before

See this link for multiline configuration =\> [https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html)

You can also check your pattern here =\>  
[https://play.golang.org/p/uAd5XHxscu](https://play.golang.org/p/uAd5XHxscu)

---

<div class="post-metadata">

**Author:** ![ELK4Life](https://avatars.discourse-cdn.com/v4/letter/e/f4b2a3/32.png) [@ELK4Life](https://discuss.elastic.co/u/ELK4Life)\
**Post date:** [January 29, 2020, 2:33pm UTC](https://discuss.elastic.co/t/top-level-configuration-not-working/216961/3 "2020-01-29T14:33:38Z")

</div>

Solved!

I was using the wrong cmd for tshark, thus outputting json data instead of CSV data.  
@inhinyera16 correctly identified this mistake.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 26, 2020, 2:33pm UTC](https://discuss.elastic.co/t/top-level-configuration-not-working/216961/4 "2020-02-26T14:33:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.