# Csv file with multiline values getting parsed as single message **URL:** \ **Category:** Logstash\ **Created:** [September 22, 2016, 5:42am UTC](https://discuss.elastic.co/t/csv-file-with-multiline-values-getting-parsed-as-single-message/61200 "2016-09-22T05:42:08Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Immortal](https://avatars.discourse-cdn.com/v4/letter/i/f08c70/32.png) [@Immortal](https://discuss.elastic.co/u/Immortal)\ **Post date:** [September 22, 2016, 5:42am UTC](https://discuss.elastic.co/t/csv-file-with-multiline-values-getting-parsed-as-single-message/61200/1 "2016-09-22T05:42:08Z")
Hi again, I am facing problem to parse a csv file that has values spread across multiline. While trying with csv plugin ,the file is getting parsed as a single message(i.e: after the mentioned column names in csv plugin fields are getting parsed as column 15=\>... ,column 16=\>... ,column 17=\>....,...... etc, So my problem is how to write multiline pattern for csv to parse the field as different events!!!!???? Pls suggest something!! ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 22, 2016, 5:44am UTC](https://discuss.elastic.co/t/csv-file-with-multiline-values-getting-parsed-as-single-message/61200/2 "2016-09-22T05:44:03Z")
Without knowing what the file looks like and what you expect Logstash to produce it's impossible to help. ---
**Author:** ![Immortal](https://avatars.discourse-cdn.com/v4/letter/i/f08c70/32.png) [@Immortal](https://discuss.elastic.co/u/Immortal)\ **Post date:** [September 22, 2016, 7:04am UTC](https://discuss.elastic.co/t/csv-file-with-multiline-values-getting-parsed-as-single-message/61200/3 "2016-09-22T07:04:39Z")
Thanks for the reply, I am sharing the configuration , log events and output i am getting: ## csv Log events : 1654996090,3,22303-2306-22e6-8d34-12a456e152c0 ,100, AppRequest/Exception, 1, 10.1.1.30, FormattedMessage="Service SessionService: Raised Exception: Invalid token. --- End of stack trace from previous location where exception was thrown --- ,AppRequest 2345160901,3,1w4ab1-1234-11e6-8d28-6e152c0, 20000, AppNotification/Notification, 2,10.1.1.56, FormattedMessage="When:Activation was called with parameters ReferenceException --- End of stack trace from previous location where exception was thrown --- ,AppNotification ## Configuration: csv { ``` columns => ["time1","split","time2","id","name","level","ip","msg",task"] separator => "," ``` } ........... ## Output : ``` "time1" => "1654996090", "split" => "3", "time2" => "22303-2306-22e6-8d34-12a456e152c0", "id" => "100", "name" => "AppRequest/ExceptionRaised", "level" => "1", "ip" => "10.1.1.30", "msg" => "FormattedMessage=\\'Service SessionService: Raised Exception: Invalid token. --- End of stack trace from previou ``` s location where exception was thrown --- ", ``` "task" => "AppRequest 2345160901", "column14" => "3", "column15" => "1w4ab1-1234-11e6-8d28-6e152c0", "column16" => "20000", "column17" => "AppNotification/NotificationFailure", "column18" => "2", "column19" => "10.1.1.56", "column20" => "FormattedMessage=\\'When:Activation was called with parameters ReferenceException --- End of stack trace from ``` previous location where exception was thrown --- ", ``` "column21" => "AppNotification" ``` ## required output: Here task field is taking next event minute column also(task" =\> "AppRequest 2345160901) After AppRequest it should take as 2nd event, But it is expanding with column 14..... So please suggest something so that it parse this multiline csv cinto different events!! 😕 ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [September 22, 2016, 7:14am UTC](https://discuss.elastic.co/t/csv-file-with-multiline-values-getting-parsed-as-single-message/61200/4 "2016-09-22T07:14:18Z")
The configuration you posted is invalid (there's a double quote missing) and the log snippet is incomplete somehow because Logstash complains that it can't parse it. ```nohighlight $ cat data 1654996090,3,22303-2306-22e6-8d34-12a456e152c0 ,100, AppRequest/Exception, 1, 10.1.1.30, FormattedMessage=\"Service SessionService: Raised Exception: Invalid token. --- End of stack trace from previous location where exception was thrown --- ,AppRequest 2345160901,3,1w4ab1-1234-11e6-8d28-6e152c0, 20000, AppNotification/Notification, 2,10.1.1.56, FormattedMessage=\"When:Activation was called with parameters ReferenceException --- End of stack trace from previous location where exception was thrown --- ,AppNotification $ cat test.config input { stdin { } } output { stdout { codec => rubydebug } } filter { csv { columns => ["time1","split","time2","id","name","level","ip","msg","task"] separator => "," } } $ /opt/logstash/bin/logstash -f test.config < data Settings: Default pipeline workers: 8 Pipeline main started Error parsing csv {:field=>"message", :source=>"1654996090,3,22303-2306-22e6-8d34-12a456e152c0 ,100, AppRequest/Exception, 1, 10.1.1.30, FormattedMessage=\\\"Service SessionService: Raised Exception: Invalid token. --- End of stack trace from previous location where exception was thrown --- ,AppRequest", :exception=>#, :level=>:warn} Error parsing csv {:field=>"message", :source=>"2345160901,3,1w4ab1-1234-11e6-8d28-6e152c0, 20000, AppNotification/Notification, 2,10.1.1.56, FormattedMessage=\\\"When:Activation was called with parameters ReferenceException --- End of stack trace from previous location where exception was thrown --- ,AppNotification", :exception=>#, :level=>:warn} { "message" => "1654996090,3,22303-2306-22e6-8d34-12a456e152c0 ,100, AppRequest/Exception, 1, 10.1.1.30, FormattedMessage=\\\"Service SessionService: Raised Exception: Invalid token. --- End of stack trace from previous location where exception was thrown --- ,AppRequest", "@version" => "1", "@timestamp" => "2016-09-22T07:12:45.601Z", "host" => "lnxolofon", "tags" => [ [0] "_csvparsefailure" ] } { "message" => "2345160901,3,1w4ab1-1234-11e6-8d28-6e152c0, 20000, AppNotification/Notification, 2,10.1.1.56, FormattedMessage=\\\"When:Activation was called with parameters ReferenceException --- End of stack trace from previous location where exception was thrown --- ,AppNotification", "@version" => "1", "@timestamp" => "2016-09-22T07:12:45.639Z", "host" => "lnxolofon", "tags" => [ [0] "_csvparsefailure" ] } Pipeline main has been shutdown stopping pipeline {:id=>"main"} ``` Please edit your post so that it reflects reality and so that someone else can reproduce the problem. ---
**Author:** ![Immortal](https://avatars.discourse-cdn.com/v4/letter/i/f08c70/32.png) [@Immortal](https://discuss.elastic.co/u/Immortal)\ **Post date:** [September 22, 2016, 8:43am UTC](https://discuss.elastic.co/t/csv-file-with-multiline-values-getting-parsed-as-single-message/61200/5 "2016-09-22T08:43:16Z")
Ok sorry there was a missing double quote in task column so i have added it : csv { columns =\> ["time1","split","time2","id","name","level","ip","msg","task"] // i changed from task" to "task" separator =\> "," } Following is snippet for input multine codec : input { codec =\> multiline { ``` pattern =>'^("time1")("split")("time2")("id")("name")("level")("ip")("msg")("task")' negate=>true what => "next" } ``` } ## Output: ``` "time1" => "1654996090", "split" => "3", "time2" => "22303-2306-22e6-8d34-12a456e152c0 ", "id" => "100", "name" => " AppRequest/Exception", "level" => " 1", "ip" => " 10.1.1.30", "msg" => " FormattedMessage=\\'Service SessionService: Raised Exception: Invalid token. --- End of stack trace from previous lo ``` cation where exception was thrown --- ", "task" =\> "AppRequest 2345160901", "column10" =\> "3", "column11" =\> "1w4ab1-1234-11e6-8d28-6e152c0", "column12" =\> " 20000", "column13" =\> " AppNotification/Notification", "column14" =\> " 2", "column15" =\> "10.1.1.56", "column16" =\> " FormattedMessage=\'When:Activation was called with parameters ReferenceException --- End of stack trace from previo us location where exception was thrown ---" Thanks again & please suggest why multiline is not working!!! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:37am UTC](https://discuss.elastic.co/t/csv-file-with-multiline-values-getting-parsed-as-single-message/61200/6 "2017-07-06T04:37:25Z")