# Csv parse failure

**URL:** https://discuss.elastic.co/t/csv-parse-failure/333049
**Category:** Logstash
**Created:** [May 10, 2023, 8:28am UTC](https://discuss.elastic.co/t/csv-parse-failure/333049 "2023-05-10T08:28:29Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![Hanni](https://avatars.discourse-cdn.com/v4/letter/h/ee7513/32.png) [@Hanni](https://discuss.elastic.co/u/Hanni)
#### Post date: [May 10, 2023, 8:28am UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/1 "2023-05-10T08:28:29Z")

</div>

Hello, I'm trying to parse a CSV file with Logstash, but I'm encountering a CSV parse failure. Can you please help me?

---

<div class="post-metadata">

### Author: ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)
#### Post date: [May 10, 2023, 3:45pm UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/2 "2023-05-10T15:45:01Z")

</div>

Hi Hanni,

Can you share the full error you are getting?

---

<div class="post-metadata">

### Author: ![Hanni](https://avatars.discourse-cdn.com/v4/letter/h/ee7513/32.png) [@Hanni](https://discuss.elastic.co/u/Hanni)
#### Post date: [May 11, 2023, 8:41am UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/3 "2023-05-11T08:41:11Z")

</div>

```auto
, :exception=>#<CSV::MalformedCSVError: Unclosed quoted field in line 1.>}
[WARN] 2023-05-11 10:38:31.439 [[main]>worker0] csv - Error parsing csv {:field=>"message", :source=>"", :exception=>#<NoMethodError: undefined method `each_index' for nil:NilClass>}
[WARN] 2023-05-11 10:38:31.460 [[main]>worker0] csv - Error parsing csv {:field=>"message", :source=>"\",2023/05/09", :exception=>#<CSV::MalformedCSVError: Unclosed quoted field in line 1.>}
[WARN] 2023-05-11 10:38:31.463 [[main]>worker0] csv - Error parsing csv {:field=>"message", :source=>"11154,No,No,No,la,tcp,65,Unknown Service Detection: Banner Retrieval,There is an unknown service running on the remote host.,Nessus was unable to identify a service on the remote host even though it returned a banner of some type.,No,No,\"", :exception=>#<CSV::MalformedCSVError: Unclosed quoted field in line 1.>}
[WARN] 2023-05-11 10:38:31.474 [[main]>worker0] csv - Error parsing csv {:field=>"message", :source=>"", :exception=>#<NoMethodError: undefined method `each_index' for nil:NilClass>}
[WARN] 2023-05-11 10:38:31.495 [[main]>worker0] csv - Error parsing csv {:field=>"message", :source=>"\",2023/05/09", :exception=>#<CSV::MalformedCSVError: Unclosed quoted field in line 1.>}
[WARN] 2023-05-11 10:38:31.498 [[main]>worker0] csv - Error parsing csv {:field=>"message", :source=>"11154,No,No,No,la,tcp,65,Unknown Service Detection: Banner Retrieval,There is an unknown service running on the remote host.,Nessus was unable to identify a service on the remote host even though it returned a banner of some type.,No,No,\"", :exception=>#<CSV::MalformedCSVError: Unclosed quoted field in line 1.>}
[WARN] 2023-05-11 10:38:31.507 [[main]>worker0] csv - Error parsing csv {:field=>"message", :source=>"", :exception=>#<NoMethodError: undefined method `each_index' for nil:NilClass>}
[WARN] 2023-05-11 10:38:31.531 [[main]>worker0] csv - Error parsing csv {:field=>"message", :source=>"\",2023/05/09", :exception=>#<CSV::MalformedCSVError: Unclosed quoted field in line 1.>}
[WARN] 2023-05-11 10:38:31.535 [[main]>worker0] csv - Error parsing csv {:field=>"message", :source=>"11154,No,No,No,la,tcp,65,Unknown Service Detection: Banner Retrieval,There is an unknown service running on the remote host.,Nessus was unable to identify a service on the remote host even though it returned a banner of some type.,No,No,\"", :exception=>#<CSV::MalformedCSVError: Unclosed quoted field in line 1.>}
[WARN] 2023-05-11 10:39:09.720 [[main]>worker1] csv - Error parsing csv {:field=>"message", :source=>"", :exception=>#<NoMethodError: undefined method `each_index' for nil:NilClass>}

```

---

<div class="post-metadata">

### Author: ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)
#### Post date: [May 11, 2023, 11:29am UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/4 "2023-05-11T11:29:36Z")

</div>

> [@Hanni](#):
>
> `<CSV::MalformedCSVError: Unclosed quoted field in line 1.>}`

Can you check the format of your CSV? It looks like you might be missing a quote or an escaped character somewhere on the first line of your CSV file.

---

<div class="post-metadata">

### Author: ![Hanni](https://avatars.discourse-cdn.com/v4/letter/h/ee7513/32.png) [@Hanni](https://discuss.elastic.co/u/Hanni)
#### Post date: [May 11, 2023, 1:18pm UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/5 "2023-05-11T13:18:52Z")

</div>

I checked it and didin't see any problem. I tried import my csv manually in elasticsearch and it worked fine. So I don't understand why it doesn't work with logstash

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 11, 2023, 3:22pm UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/6 "2023-05-11T15:22:54Z")

</div>

> [@Hanni](#):
>
> [WARN] 2023-05-11 10:38:31.463 [[main]\>worker0] csv - Error parsing csv {:field=\>"message", :source=\>"11154,No,No,No,la,tcp,65,Unknown Service Detection: Banner Retrieval,There is an unknown service running on the remote host.,Nessus was unable to identify a service on the remote host even though it returned a banner of some type.,No,No,"", :exception=\>#\<CSV::MalformedCSVError: Unclosed quoted field in line 1.\>}

The line has a trailing double quote. That is not allowed.

---

<div class="post-metadata">

### Author: ![Hanni](https://avatars.discourse-cdn.com/v4/letter/h/ee7513/32.png) [@Hanni](https://discuss.elastic.co/u/Hanni)
#### Post date: [May 16, 2023, 8:51am UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/7 "2023-05-16T08:51:28Z")

</div>

finally found that my file some data in his columns that Logstash is unable to parse. Is there a way to fix this?"

---

<div class="post-metadata">

### Author: ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)
#### Post date: [May 16, 2023, 8:53am UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/8 "2023-05-16T08:53:08Z")

</div>

Are you able to use a `mutate` filter to change the value? I found an older topic that recommends that [here](https://discuss.elastic.co/t/csv-filter-quote-character-causing-csvparsefailure/121273/3).

---

<div class="post-metadata">

### Author: ![Hanni](https://avatars.discourse-cdn.com/v4/letter/h/ee7513/32.png) [@Hanni](https://discuss.elastic.co/u/Hanni)
#### Post date: [May 16, 2023, 11:40am UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/9 "2023-05-16T11:40:54Z")

</div>

I made this configuration

```auto
mutate {
     gsub => ["Solution", "[."]", " "]
 }

```

and I got this error

```auto
[ERROR] 2023-05-16 13:37:54.477 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [\\t\\r\\n], \"#\", \"}\" at line 15, column 33 (byte 375) after filter {\n csv {\n separator => \",\"\n skip_header => \"true\"\n columns => [\"Plugin_ID\",\"CVE\",\"CVSS_v2.0_Base_Score\",\"Risk\",\"Host\",\"Protocol\",\"Port\",\"Solution\",\"date\"]\n }\n mutate {\n gsub => [\"Solution\", \"[.\"]", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:239:in `initialize'", "org/logstash/execution/AbstractPipelineExt.java:173:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:48:in `initialize'", "org/jruby/RubyClass.java:911:in `new'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:50:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:386:in `block in converge_state'"]}
[INFO] 2023-05-16 13:37:54.517 [LogStash::Runner] runner - Logstash shut down.

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 16, 2023, 2:45pm UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/10 "2023-05-16T14:45:10Z")

</div>

> [@Hanni](#):
>
> `gsub => ["Solution", "[."]", " "]`

You cannot have an unquoted double quote inside double quotes. Perhaps `gsub => ["Solution", '[."]', " "]`, but I am not sure what you are trying to do.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 13, 2023, 2:45pm UTC](https://discuss.elastic.co/t/csv-parse-failure/333049/11 "2023-06-13T14:45:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.