# Error in logstash reading a csv file **URL:** \ **Category:** Logstash\ **Created:** [May 27, 2019, 2:12pm UTC](https://discuss.elastic.co/t/error-in-logstash-reading-a-csv-file/182928 "2019-05-27T14:12:14Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![decodeit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/decodeit/32/46023_2.png) [@decodeit](https://discuss.elastic.co/u/decodeit)\ **Post date:** [May 27, 2019, 2:12pm UTC](https://discuss.elastic.co/t/error-in-logstash-reading-a-csv-file/182928/1 "2019-05-27T14:12:14Z")
I am trying to read a csv file with logstash . Here is my config file: input { file{ path =\> ["/home/abhinavkumar.gurung/Applications/csv/devian/data/autos.csv"] type =\> "cars" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { csv{ columns =\> ["dateCrawled","name","seller", "offerType","price","abtest","vechicletype","yearOfRegistration","gearbox","powerPS","model","kilometer","monthOfRegistration","fuelType","brand","notRepairedDamage","dateCreated","nrOfPictures","postalCode","lastSeen"] separator =\>"," convert =\> {"price" =\> "integer" "yearOfRegistration" =\> "integer" "powerPS" =\> "integer" "kilometer" =\> "integer" "nrOfPictures" =\> "integer" "postalCode" =\> "integer" } remove\_field =\>["message","dateCrawled","monthOfRegistration","dateCreated","lastSeen"] skip\_header =\>"true" } } output { elasticsearch { hosts =\> ["elasticsearch:9200"] user =\> elastic password =\> changeme index =\> "cars" } stdout { codec =\> rubydebug } } My error: ``` `Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 24, column 8 (byte 696) after filter {\n csv{\n\tcolumns => [\"dateCrawled\",\"name\",\"seller\",\n\t\"offerType\",\"price\",\"abtest\",\"vechicletype\",\"yearOfRegistration\",\"gearbox\",\"powerPS\",\"model\",\"kilometer\",\"monthOfRegistration\",\"fuelType\",\"brand\",\"notRepairedDamage\",\"dateCreated\",\"nrOfPictures\",\"postalCode\",\"lastSeen\"]\n \tseparator =>\",\"\n\tconvert => {\"price\" => \"integer\"\n\t\t\t\t\t\t\t\"yearOfRegistration\" => \"integer\"\n\t\t\t\t\t\t\t\"powerPS\" => \"integer\"\n\t\t\t\t\t\t\t\"kilometer\" => \"integer\"\n\t\t\t\t\t\t\t\"nrOfPictures\" => \"integer\"\n\t\t\t\t\t\t\t\"postalCode\" => \"integer\"\n\t\n\n\t}\n\tmutate", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:41:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:49:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:10:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:151:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:23:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:325:in `block in converge_state'"]}` ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 27, 2019, 4:05pm UTC](https://discuss.elastic.co/t/error-in-logstash-reading-a-csv-file/182928/2 "2019-05-27T16:05:26Z")
The log message indicates it fails when it reaches a mutate after the csv convert option, but the configuration you post does not have a mutate, so it is not the configuration you are running with. ---
**Author:** ![decodeit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/decodeit/32/46023_2.png) [@decodeit](https://discuss.elastic.co/u/decodeit)\ **Post date:** [May 27, 2019, 4:10pm UTC](https://discuss.elastic.co/t/error-in-logstash-reading-a-csv-file/182928/3 "2019-05-27T16:10:02Z")
So I changed my csv file (much smaller and simpler) input { file{ path =\> ["/home/abhinavkumar.gurung/Applications/csv/devian/data/Admission\_Predict.csv"] start\_position =\> "beginning" sincedb\_path =\> "/dev/null" codec =\> plain {charset =\> "UTF-8"} } } filter { csv{ columns =\> ["Serial No","GRE Score","TOEFL Score","University Rating","SOP","LOR","CGPA","Research","Chance of Admit"] convert =\> {"Serial No" =\> "integer" "GRE Score" =\> "integer" "TOEFL Score" =\> "integer" "University Rating" =\> "integer" "SOP" =\> "float" "LOR" =\> "float" "CGPA" =\> "float" "Research" =\> "integer" "Chance of Admit" =\> "float" } remove\_field =\> ["message"] separator =\>"," skip\_header =\>"true" } } output { elasticsearch { hosts =\> ["elasticsearch:9200"] document\_type =\> "\_doc" user =\> elastic password =\> changeme index =\> "cars" } stdout { codec =\> rubydebug } } However I don't see any kind of error. But there is nothing stored in es. I doubt if logstash is even reading it. if it is , isnt it supposed to output in console? How would it output in console a csv file, any thing I need to do to convert to json. But main issue is nothing is stored in elastic search. ---
**Author:** ![decodeit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/decodeit/32/46023_2.png) [@decodeit](https://discuss.elastic.co/u/decodeit)\ **Post date:** [May 27, 2019, 4:18pm UTC](https://discuss.elastic.co/t/error-in-logstash-reading-a-csv-file/182928/4 "2019-05-27T16:18:21Z")
partial logs ``` [2019-05-27T15:24:01,182][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}} ``` [2019-05-27T15:24:01,214][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=\>"[http://elasticsearch:9200/](http://elasticsearch:9200/)"} [2019-05-27T15:24:01,242][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es\_version=\>7} [2019-05-27T15:24:01,242][WARN][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>7} [2019-05-27T15:24:01,316][INFO][logstash.outputs.elasticsearch] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearch", :hosts=\>["[http://elasticsearch:9200](http://elasticsearch:9200)"]} [2019-05-27T15:24:01,320][INFO][logstash.javapipeline] Starting pipeline {:pipeline\_id=\>".monitoring-logstash", "pipeline.workers"=\>1, "pipeline.batch.size"=\>2, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>2, :thread=\>"#\"} [2019-05-27T15:24:01,459][INFO][logstash.javapipeline] Pipeline started {"pipeline.id"=\>".monitoring-logstash"} [2019-05-27T15:24:01,528][INFO][logstash.agent] Pipelines running {:count=\>2, :running\_pipelines=\>[:main, :".monitoring-logstash"], :non\_running\_pipelines=\>} [2019-05-27T15:24:02,661][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600} [2019-05-27T15:30:07,456][WARN][logstash.runner] SIGTERM received. Shutting down. [2019-05-27T15:30:07,804][INFO][filewatch.observingtail] QUIT - closing all files and shutting down. [2019-05-27T15:30:08,581][INFO][logstash.javapipeline] Pipeline terminated {"pipeline.id"=\>"main"} [2019-05-27T15:30:08,840][INFO][logstash.javapipeline] Pipeline terminated {"pipeline.id"=\>".monitoring-logstash"} [2019-05-27T15:30:09,430][INFO][logstash.runner] Logstash shut down. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 24, 2019, 4:18pm UTC](https://discuss.elastic.co/t/error-in-logstash-reading-a-csv-file/182928/5 "2019-06-24T16:18:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.