### Author:  [@fxiger](https://discuss.elastic.co/u/fxiger)
#### Post date: [May 8, 2017, 1:03pm UTC](https://discuss.elastic.co/t/csv-output-not-working-properly-after-upgrade-from-5-3-1-to-5-4-0/84936/1 "2017-05-08T13:03:57Z")
Hi,
i'm using an ELK stack (now) at **5.4.0** on a debian machine and had before version 5.3.1 running.
**my usecase:** read from rabbitMQ and depending on filter either put events directly into an ES index or output specific events into CSV file. logstash is running as a service.
the ES output part seems to be fine. but the CSV output does not look (_anymore_) as it should. With version 5.3.1 running, everything was fine.
* * *
plugins used:
**logstash-input-rabbitmq (5.2.2)**
**logstash-filter-grok (3.3.1)**
**logstash-filter-mutate (3.1.3)**
**logstash-output-csv (3.0.2)**
**logstash-output-elasticsearch (6.3.0)**
**logstash-output-stdout (3.1.0)**
* * *
my config looks like:
#input {
rabbitmq {
host =\> "xxx"
# queue =\> "xxx"
durable =\> false
key =\> ""
exchange =\> "traces"
prefetch\_count =\> 50
port =\> 5672
user =\> "xxx"
password =\> "xxxx"
type =\> "some\_event"
vhost =\> "symphony"
exclusive =\> true
auto\_delete =\> true
codec =\> "plain"
}
}
#filter {
if [type] == "some\_event" {
mutate {
gsub =\> ['message','"', ""]
gsub =\> ['message','/', "-"]
}
grok {
match =\> { "message" =\> "%{DATE:date}-%{TIME:time} {source:%{DATA:source},timestamp:%{NUMBER:unix\_timestamp},name:%{DATA:name},toolname:%{DATA:toolname},type:%{WORD:event\_type},key:%{DATA:key}} " }
patterns\_dir =\> ["/usr/share/logstash/patterns"]
}
mutate {
add\_field =\> ["date\_time","%{date} %{time}"]
add\_field =\> ["event\_source","raw\_some\_event"]
}
if "dafuq" == [source] {
mutate {
add\_tag =\> ["dastool\_install"]
}
}
else if [source] != "dafuq" {
mutate {
add\_tag =\> ["dastool\_other"]
}
}
}
}
#output {
if [type] == "dastool\_event" {
if "dastool\_install" in [tags] {
csv {
csv\_options =\> {"col\_sep" =\> ";"}
fields =\> ["@version","@timestamp","date\_time","type","source","unix\_timestamp","name","toolname","event\_type","key"]
path =\> "/xxxx/logfilename\_%{+yyyy.MM.dd}.log"
}
}
```
elasticsearch {
action => "index"
hosts => ["localhost"]
index => "tool_raw-%{+YYYY.MM.dd}"
}
stdout {
codec => rubydebug
}
}
```
}
* * *
**previous output looked like the following:**
> ```
> 1;2017-05-08T10:55:32.202Z;2017-05-08 12:55:39;some_event;dafuq;1494240939.354620;xxxxx;einrichtezeit;post;xxxx
> 1;2017-05-08T10:55:40.578Z;2017-05-08 12:55:47;some_event;dafuq;1494240947.705;xxxx;einrichtezeit;pre;xxxx
> 1;2017-05-08T10:55:59.278Z;2017-05-08 12:56:06;some:event;dafuq;1494240966.335118;xxxx;einrichtezeit;post;xxxx
>
> ```
**output now looks like the following** (_without linebreaks and it seems not to write my specific fields_)
> 2017-05-08T11:06:11.871Z %{host} 2017-05-08-13:06:19 {source:dafuq,timestamp:1494241579.345582,xxxxx,toolname:einrichtezeit,type:post,key:xxxx} 2017-05-08T11:06:11.871Z %{host} 2017-05-08-13:06:19 {source:dafuq,timestamp:1494241579.411888,xxxx,toolname:einrichtezeit,type:post,key:xxxx} 2017-05-08T11:06:13.139Z %{host} 2017-05-08-13:06:20 {source:dafuq,timestamp:1494241580.418222,xxxxx,toolname:einrichtezeit,type:post,key:xxxx}
**When i just have a look at the incoming message, it seems to be fine so far:**
14:21:38.602 [[main]-pipeline-manager] INFO logstash.pipeline - Pipeline main started
14:21:38.695 [Api Webserver] INFO logstash.agent - Successfully started Logstash API endpoint {:port=\>9600}
14:21:38.825 [[main]\ "2017-05-08",
"toolname" =\> "einrichtezeit",
"event\_source" =\> "raw\_some\_event",
"source" =\> "dafuq",
"message" =\> "2017-05-08-14:23:01 {source:dafuq,timestamp:1494246181.760,name:xxxx,toolname:einrichtezeit,type:pre,key:xxxxx} ",
"type" =\> "some\_event",
"tags" =\> [
[0] "dastool\_install"
],
"@timestamp" =\> 2017-05-08T12:22:54.295Z,
"event\_type" =\> "pre",
"date\_time" =\> "2017-05-08 14:23:01",
"@version" =\> "1",
"name" =\> "xxxxx",
"time" =\> "14:23:01",
"unix\_timestamp" =\> "1494246181.760",
"key" =\> "xxxxx"
}
So, does anybody have an idea what has changed since 5.3.1? I'm puzzled
---
### Author:  [@fxiger](https://discuss.elastic.co/u/fxiger)
#### Post date: [May 8, 2017, 2:47pm UTC](https://discuss.elastic.co/t/csv-output-not-working-properly-after-upgrade-from-5-3-1-to-5-4-0/84936/2 "2017-05-08T14:47:36Z")
When i add the line
> codec =\> line { }
to the output section, it does at least the linebreak after each line, but it still ignores the fields-option 😢
---
### Author:  [@fxiger](https://discuss.elastic.co/u/fxiger)
#### Post date: [May 8, 2017, 3:19pm UTC](https://discuss.elastic.co/t/csv-output-not-working-properly-after-upgrade-from-5-3-1-to-5-4-0/84936/3 "2017-05-08T15:19:50Z")
When i use the file-output plugin instead, i get a proper output (_just as i want_)
```
file {
codec => line { format => "%{[@version]};%{[@timestamp]};%{[date_time]};%{[type]};%{[source]};%{[unix_timestamp]};%{[name]};%{[toolname]};%{[event_type]};%{[key]}" }
path => "/xxxxl/logfilename_%{+yyyy.MM.dd}.log"
}
```
but, i'm still wondering why the csv-output-plugin does not work properly 😐
---
### Author:  [@fxiger](https://discuss.elastic.co/u/fxiger)
#### Post date: [May 8, 2017, 3:36pm UTC](https://discuss.elastic.co/t/csv-output-not-working-properly-after-upgrade-from-5-3-1-to-5-4-0/84936/4 "2017-05-08T15:36:58Z")
Okay, now i performed an additional update on my logstash-plugins :
> root@xxx:/usr/share/logstash# bin/logstash-plugin update
> Updated logstash-codec-fluent 3.0.2 to 3.1.1
> Updated logstash-codec-netflow 3.3.0 to 3.4.0
> Updated logstash-filter-date 3.1.3 to 3.1.5
> Updated logstash-filter-fingerprint 3.0.2 to 3.0.3
> Updated logstash-filter-grok 3.3.1 to 3.4.0
> Updated logstash-filter-urldecode 3.0.2 to 3.0.3
> Updated logstash-input-beats 3.1.12 to 3.1.15
> Updated logstash-input-couchdb\_changes 3.1.0 to 3.1.1
> Updated logstash-input-elasticsearch 4.0.2 to 4.0.3
> Updated logstash-input-http 3.0.3 to 3.0.4
> Updated logstash-input-jdbc 4.1.3 to 4.2.0
> Updated logstash-input-kafka 5.1.6 to 5.1.7
> Updated logstash-input-log4j 3.0.3 to 3.0.5
> Updated logstash-input-rabbitmq 5.2.2 to 5.2.3
> Updated logstash-input-s3 3.1.2 to 3.1.4
> Updated logstash-input-sqs 3.0.2 to 3.0.3
> Updated logstash-input-twitter 3.0.3 to 3.0.4
> Updated logstash-input-unix 3.0.2 to 3.0.3
> Updated logstash-output-csv 3.0.2 to 3.0.3
> Updated logstash-output-elasticsearch 6.3.0 to 7.3.1
> Updated logstash-output-http 4.1.0 to 4.2.0
> Updated logstash-output-kafka 5.1.5 to 5.1.6
> Updated logstash-output-rabbitmq 4.0.6 to 4.0.7
> Updated logstash-output-s3 4.0.5 to 4.0.7
> Updated logstash-output-sns 4.0.2 to 4.0.3
and it seems, my setup is now working again as intended 😮
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 5, 2017, 3:50pm UTC](https://discuss.elastic.co/t/csv-output-not-working-properly-after-upgrade-from-5-3-1-to-5-4-0/84936/5 "2017-06-05T15:50:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.