# Logstash csv output plugins giving problem **URL:** https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969 **Category:** Logstash **Created:** [April 3, 2017, 8:45am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969 "2017-04-03T08:45:27Z") **Posts on this page:** 13 **Page:** 1
### Author: ![Nikparab](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikparab/32/17493_2.png) [@Nikparab](https://discuss.elastic.co/u/Nikparab) #### Post date: [April 3, 2017, 8:45am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/1 "2017-04-03T08:45:27Z")
I write output-csv.conf file and run it under bin folder of logstash. input { elasticsearch { hosts =\> ["localhost:9200"] index =\> "mongo3" query =\> '{ "query": { "bool":{ "must":{ "match":{ "content":"recognition" } } } } }' } } output { csv { fields =\> ["@timestamp","content","component","duration","host"] path =\> "D:/elasticsearch/excel/mongo3.csv" } } this is input ---
### Author: ![Nikparab](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikparab/32/17493_2.png) [@Nikparab](https://discuss.elastic.co/u/Nikparab) #### Post date: [April 3, 2017, 8:49am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/2 "2017-04-03T08:49:14Z")
output- 13:53:40.533 [[main]\>worker3] DEBUG logstash.outputs.csv - File, writing event to file. {:filename=\>"D:/elasticsearch/excel/mongo3.csv"} 13:53:40.533 [[main]\>worker3] DEBUG logstash.outputs.csv - File, writing event to file. {:filename=\>"D:/elasticsearch/excel/mongo3.csv"} 13:53:40.549 [[main]\>worker3] DEBUG logstash.outputs.csv - File, writing event to file. {:filename=\>"D:/elasticsearch/excel/mongo3.csv"} 13:53:40.549 [[main]\>worker3] DEBUG logstash.outputs.csv - File, writing event to file. {:filename=\>"D:/elasticsearch/excel/mongo3.csv"} 13:53:40.549 [[main]\>worker3] DEBUG logstash.outputs.csv - File, writing event to file. {:filename=\>"D:/elasticsearch/excel/mongo3.csv"} 13:53:41.526 [[main]\["localhost:9200"], index=\>"mongo3", query=\>"{\n "query": {\n "bool":{\n\t\t "must":{\n\t\t\t "ma tch":{\n\t\t "content":"recognition"\n\t\t }\n\t\t\t\t\t}\n\t\t\t\t}\n }\n }", id=\>"ec055bcc4495cb3c048db0ccab7a85a61d32de52-1", e nable\_metric=\>true, codec=\>\"json\_00a077e7-c90b-4455-8c0a-8e2aec3f1211", enable\_metric=\>true, charset=\>"UTF-8"\>, size=\>1000, scroll=\>"1m", docinfo =\>false, docinfo\_target=\>"@metadata", docinfo\_fields=\>["\_index", "\_type", "\_id"], ssl=\>false\> Error: [400] {"error":{"root\_cause":[{"type":"illegal\_argument\_exception","reason":"Failed to parse request body"}],"type":"illegal\_argument\_exception","reason":"Failed to parse request body","caused\_by":{"type":"json\_parse\_exception","reason":"Unrecognized token 'DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAMnFjNVazlMeGJOUXUtWTdsR0pROXUyNHcAAAAAAAADJhYzVWs 5THhiTlF1LVk3bEdKUTl1MjR3AAAAAAAAAygWM1VrOUx4Yk5RdS1ZN2xHSlE5dTI0dwAAAAAAAAMpFjNVazlMeGJOUXUtWTdsR0pROXUyNHcAAAAAAAADKhYzVWs5THhiTlF1LVk3bEdKUTl1MjR3': was expecting ('true' , 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@40904328; line: 1, column: 457]"}},"status":400} Exception: Elasticsearch::Transport::Transport::Errors::BadRequest Stack: D:/elasticsearch/logstash-5.3.0/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-5.0.3/lib/elasticsearch/transport/transport/base.rb:201:in `__raise_transport_e rror' D:/elasticsearch/logstash-5.3.0/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-5.0.3/lib/elasticsearch/transport/transport/base.rb:318:in`perform\_request' D:/elasticsearch/logstash-5.3.0/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-5.0.3/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request' D:/elasticsearch/logstash-5.3.0/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-5.0.3/lib/elasticsearch/transport/client.rb:131:in`perform\_request' D:/elasticsearch/logstash-5.3.0/vendor/bundle/jruby/1.9/gems/elasticsearch-api-5.0.3/lib/elasticsearch/api/actions/scroll.rb:62:in `scroll' D:/elasticsearch/logstash-5.3.0/vendor/bundle/jruby/1.9/gems/logstash-input-elasticsearch-4.0.2/lib/logstash/inputs/elasticsearch.rb:187:in`scroll\_request' D:/elasticsearch/logstash-5.3.0/vendor/bundle/jruby/1.9/gems/logstash-input-elasticsearch-4.0.2/lib/logstash/inputs/elasticsearch.rb:156:in `process_next_scroll' D:/elasticsearch/logstash-5.3.0/vendor/bundle/jruby/1.9/gems/logstash-input-elasticsearch-4.0.2/lib/logstash/inputs/elasticsearch.rb:148:in`run' D:/elasticsearch/logstash-5.3.0/logstash-core/lib/logstash/pipeline.rb:425:in `inputworker' D:/elasticsearch/logstash-5.3.0/logstash-core/lib/logstash/pipeline.rb:419:in`start\_input' 13:53:41.541 [[main]\>worker3] DEBUG logstash.pipeline - filter received {"event"=\>{"severity"=\>"I", "duration"=\>"893", "path"=\>"C:\Data\log\filter-mongologs-2017-03-18\m ongodb3.log", "component"=\>"COMMAND", "@timestamp"=\>2017-03-31T09:17:34.583Z, "host"=\>"Admin-PC", "context"=\>"conn274726", "@version"=\>"1", "content"=\>"command hgthanka.Reco gnition command: aggregate { aggregate: "Recognition", pipeline: [ { $match: { Status: "Active", Template.Type: { $in: [ "Recognition", "KeyResult", "Objective", "Milestone", "Award", "Context", "Thanks", "Quick", "Congrats", "Newsfeed", "ProductItem", "PollResult", "GoalKeyResultUpdate" ] }, SuppressInFeed: false, RecipientMember.Location.hgId: "32a336e0-e27f-11e6-9a31-c1daaeaf8e28" } }, { $group: { \_id: "$BatchId", max: { $max: "$ModifiedDate" } } }, { $sort: { max: -1 } }, { $skip: 0 }, { $limit: 20 }, { $project: { \_id: 0, BatchId: "$\_id" } } ] } keyUpdates:0 writeConflicts:0 numYields:2284 reslen:1238 locks:{ Global: { acquireCount: { r: 457 4 } }, Database: { acquireCount: { r: 2287 } }, Collection: { acquireCount: { r: 2287 } } } protocol:op\_query "}} 13:53:41.604 [[main]\>worker3] DEBUG logstash.pipeline - filter received {"event"=\>{"severity"=\>"I", "duration"=\>"899", "path"=\>"C:\Data\log\filter-mongologs-2017-03-18\m ongodb3.log", "component"=\>"COMMAND", "@timestamp"=\>2017-03-31T09:17:34.552Z, "host"=\>"Admin-PC", "context"=\>"conn275532", "@version"=\>"1", "content"=\>"command hgthanka.Reco gnition command: aggregate { aggregate: "Recognition", pipeline: [ { $match: { Status: "Active", Template.Type: { $in: [ "Recognition", "KeyResult", "Objective", "Milestone", "Award", "Context", "Thanks", "Quick", "Congrats", "Newsfeed", "ProductItem", "PollResult", "GoalKeyResultUpdate" ] }, SuppressInFeed: false, RecipientMember.Location.hgId: "8a245490-aa93-11e6-9402-abce09847bec" } }, { $group: { \_id: "$BatchId", max: { $max: "$ModifiedDate" } } }, { $sort: { max: -1 } }, { $skip: 0 }, { $limit: 20 }, { $project: { \_id: 0, BatchId: "$\_id" } } ] } keyUpdates:0 writeConflicts:0 numYields:2284 reslen:1238 locks:{ Global: { acquireCount: { r: 457 4 } }, Database: { acquireCount: { r: 2287 } }, Collection: { acquireCount: { r: 2287 } } } protocol:op\_query "}} 13:53:41.619 [[main]\>worker3] DEBUG logstash.pipeline - filter received {"event"=\>{"severity"=\>"I", "duration"=\>"887", "path"=\>"C:\Data\log\filter-mongologs-2017-03-18\m ongodb3.log", "component"=\>"COMMAND", "@timestamp"=\>2017-03-31T09:17:34.583Z, "host"=\>"Admin-PC", "context"=\>"conn276785", "@version"=\>"1", "content"=\>"command hgthanka.Reco It gives continuous & repeated output. It gives error of pipeline input elasticsearch stoping. How to resolve it.I m using ELK version 5.3.0 ---
### Author: ![Andrei\_Stefan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrei_stefan/32/47533_2.png) [@Andrei\_Stefan](https://discuss.elastic.co/u/Andrei_Stefan) #### Post date: [April 6, 2017, 1:16pm UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/3 "2017-04-06T13:16:25Z")
Identified as bug: [https://github.com/logstash-plugins/logstash-input-elasticsearch/issues/62](https://github.com/logstash-plugins/logstash-input-elasticsearch/issues/62) ---
### Author: ![Nikparab](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikparab/32/17493_2.png) [@Nikparab](https://discuss.elastic.co/u/Nikparab) #### Post date: [April 7, 2017, 7:13am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/4 "2017-04-07T07:13:32Z")
Thanks @Andrei_Stefan I install elasticsearch input plugin of version 4.0.1 and run the same command to load elasticsearch data into csv file but it is giving same output. I got the continuous and repeating output like above i mentioned. ---
### Author: ![Nikparab](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikparab/32/17493_2.png) [@Nikparab](https://discuss.elastic.co/u/Nikparab) #### Post date: [April 7, 2017, 7:15am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/5 "2017-04-07T07:15:09Z")
Is there any upgraded version of logstash-output-csv plugin. I m using version 3.0.3 ---
### Author: ![Andrei\_Stefan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrei_stefan/32/47533_2.png) [@Andrei\_Stefan](https://discuss.elastic.co/u/Andrei_Stefan) #### Post date: [April 7, 2017, 7:40am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/6 "2017-04-07T07:40:27Z")
The fix will go in es input plugin version 4.0.3. ---
### Author: ![Nikparab](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikparab/32/17493_2.png) [@Nikparab](https://discuss.elastic.co/u/Nikparab) #### Post date: [April 7, 2017, 8:50am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/7 "2017-04-07T08:50:39Z")
@Andrei_Stefan Thanks for reply I tried this also but it giving following error D:\Elastic\logstash-5.3.0\bin\>logstash-plugin install --version 4.0.3 logstash-input-elasticsearch Validating logstash-input-elasticsearch-4.0.3 Plugin logstash-input-elasticsearch version 4.0.3 does not exist ERROR: Installation aborted, verification failed for logstash-input-elasticsearch 4.0.3 ---
### Author: ![Andrei\_Stefan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrei_stefan/32/47533_2.png) [@Andrei\_Stefan](https://discuss.elastic.co/u/Andrei_Stefan) #### Post date: [April 7, 2017, 9:35am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/8 "2017-04-07T09:35:39Z")
The error is self explanatory, almost. That version hasn't been released yet. ---
### Author: ![Nikparab](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikparab/32/17493_2.png) [@Nikparab](https://discuss.elastic.co/u/Nikparab) #### Post date: [April 7, 2017, 9:38am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/9 "2017-04-07T09:38:32Z")
Then how can i resolve the error of continuous output? Can u please give some solution for that?/😌 ---
### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje) #### Post date: [April 7, 2017, 10:17am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/10 "2017-04-07T10:17:50Z")
Try version 4.0.0 of the elasticsearch input. The issue Andrei linked says the versions greater or equal to 4.0.1 have the bug. ---
### Author: ![Nikparab](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikparab/32/17493_2.png) [@Nikparab](https://discuss.elastic.co/u/Nikparab) #### Post date: [April 7, 2017, 10:25am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/11 "2017-04-07T10:25:12Z")
Thanks @guyboertje I will try it and let you know ---
### Author: ![Nikparab](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikparab/32/17493_2.png) [@Nikparab](https://discuss.elastic.co/u/Nikparab) #### Post date: [April 7, 2017, 10:31am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/12 "2017-04-07T10:31:12Z")
@guyboertje Thank You. It works ---
### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system) #### Post date: [May 5, 2017, 10:44am UTC](https://discuss.elastic.co/t/logstash-csv-output-plugins-giving-problem/80969/13 "2017-05-05T10:44:19Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.