# Filebeat to logstash using csv filter not working

**URL:** <https://discuss.elastic.co/t/filebeat-to-logstash-using-csv-filter-not-working/206955>\
**Category:** Logstash\
**Created:** [November 7, 2019, 12:26pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-using-csv-filter-not-working/206955 "2019-11-07T12:26:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![nareshreddyp](https://avatars.discourse-cdn.com/v4/letter/n/e56c9b/32.png) [@nareshreddyp](https://discuss.elastic.co/u/nareshreddyp)\
**Post date:** [November 7, 2019, 12:26pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-using-csv-filter-not-working/206955/1 "2019-11-07T12:26:22Z")

</div>

Hello,

I have a csv file with following entries (Each column is already with double quotes)  
"2019/11/07 13:19:18.538";"138.106.61.153";"sxisl";"sxist0";"mxobjecttype:(""BUSINESSOBJECT"")";"xx";"0";"2";"1";"1";"118820";

**When filebeat sends to logstash it is sending with ", so the csv filter is not working. How to filter the data coming from csv file using filebeat \_**

Logstash ruby output

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/c/3c519ccf6bee11223f0e01d3a16eb4d41aa231cb.png)

My file beat config

filebeat.inputs:

- input\_type: log  
paths:
  - /data/3dspaceindex/cv/run/searchserver-ss0/search-reporting/\*  
tags: ["otb,3dspaceindex"]  
output.logstash:  
hosts: ["localhost:5065"]

logstash config

input {  
beats {  
port =\> 5065  
}  
}

filter {  
if "3dspaceindex" in [tags]  
{  
mutate {  
gsub =\> ["message", """, ""]  
}  
csv {  
skip\_empty\_columns =\> "true"  
skip\_header =\> "true"  
separator =\> ";"  
columns =\> [""#timeStamp","apiclient\_ip","query\_logic","query\_target","query\_querystring","query\_language","query\_start","query\_hf","answer\_nmatches","answer\_nhits","time\_total","query\_full","query\_id","query\_origin","answer\_status""]  
}  
date  
{  
match =\> ["#timeStamp", "yyyy/MM/dd HH:mm:ss.SSS"]  
remove\_field =\> ["timestamp"]  
remove\_field =\> ["message"]  
}  
mutate {  
convert =\> {  
"query\_start" =\> "integer"  
"query\_hf" =\> "integer"  
"answer\_nmatches" =\> "integer"  
"answer\_nhits" =\> "integer"  
"time\_total" =\> "integer"  
"query\_id" =\> "integer"  
}  
}  
if [message] =~ /^#timestamp/ {  
drop { }  
}  
}

}

output {  
stdout { codec =\> rubydebug }  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 7, 2019, 4:33pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-using-csv-filter-not-working/206955/2 "2019-11-07T16:33:01Z")

</div>

> [@nareshreddyp](#):
>
> mutate {  
> gsub =\> ["message", """, ""]  
> }

I think you should remove this.

---

<div class="post-metadata">

**Author:** ![nareshreddyp](https://avatars.discourse-cdn.com/v4/letter/n/e56c9b/32.png) [@nareshreddyp](https://discuss.elastic.co/u/nareshreddyp)\
**Post date:** [November 8, 2019, 7:48am UTC](https://discuss.elastic.co/t/filebeat-to-logstash-using-csv-filter-not-working/206955/3 "2019-11-08T07:48:45Z")

</div>

Hi,

I have tried removing that but the result was always with ' **"**'  
logstash is not filtering correctly.

Regards,  
Naresh

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 6, 2019, 7:48am UTC](https://discuss.elastic.co/t/filebeat-to-logstash-using-csv-filter-not-working/206955/4 "2019-12-06T07:48:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.