# Csv filter parsing error

**URL:** https://discuss.elastic.co/t/csv-filter-parsing-error/386766
**Category:** Logstash
**Created:** [June 8, 2026, 9:41pm UTC](https://discuss.elastic.co/t/csv-filter-parsing-error/386766 "2026-06-08T21:41:05Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![carellevbt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carellevbt/32/139298_2.png) [@carellevbt](https://discuss.elastic.co/u/carellevbt)
#### Post date: [June 8, 2026, 9:41pm UTC](https://discuss.elastic.co/t/csv-filter-parsing-error/386766/1 "2026-06-08T21:41:06Z")

</div>

Hello,

I have multiple csv files I need to ingest. Among the csv files I have 13 column header names variations. I would like to use the `autodetect_column_names` to avoid manually configuring these column names. Also please note each csv file has two empty column headers at the end.

Looking at previous post I saw a suggested fix using the below configuration, which helped with my error `exception: #<RuntimeError: Invalid FieldReference: `[]`\>`.

`mutate { gsub => ["message", "[[]]", " "] }`

I am no longer receiving the `_csvparsefailure` and logs are getting parsed a bit. However, they are not aligned. For example, I have:  
`"GeoIP City": "12/24/2024 08:14:23"` when it should be a city  
`"first_name": null` when there should be a name.

My full filter is as below

\<filter {  
mutate {  
gsub =\> ["message", "[\[\]]", " "]  
}

```
csv {
    separator => ","
    autodetect_column_names => true
    #skip_empty_columns => true

```

}}\>

Here is a sample output of a row data with the additional error I get

`Error parsing csv {field: "message", source: "[test@email.com](mailto:test@email.com),"",FY26 Q3 ALL,"","",No,"",No,"",No,"","","",,unknown,unknown,unknown,2.0.0,"Apr 23 18:46:21 postfix-smtp postfix/smtp[5969]: 4: [to=test@email.com](mailto:to=test@email.com), [relay=aspmx.l.google.com](http://relay=aspmx.l.google.com)[x.x.x.x]:25, delay=1.2, delays=0.12/0/0.29/0.83, dsn=2.0.0, status=sent (250 2.0.0 OK 177-12.53 - gsmtp)",04/23/2026 14:46:21,,,,,,Test,Name", exception: #\<RuntimeError: Invalid FieldReference: `[Apr 23 18:41:38 postfix-smtp postfix/smtp[5601]: 4: to=<other@email.com>, ``relay=aspmx.l.google.com``[y.y.y.y]:25, delay=1.1, delays=0.12/0/0.3/0.67, dsn=x.0.0, status=sent (250 2.0.0 OK 177-2e.16 - gsmtp)]`\>}`

Any help you could provide will be much appreciated.

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [June 10, 2026, 7:17am UTC](https://discuss.elastic.co/t/csv-filter-parsing-error/386766/2 "2026-06-10T07:17:40Z")

</div>

Can you put a sample, the header and data lines?

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [June 10, 2026, 9:45am UTC](https://discuss.elastic.co/t/csv-filter-parsing-error/386766/3 "2026-06-10T09:45:20Z")

</div>

You have commas inside the field in the column 19.

Try something like this:

```auto
input {
  generator {
       message => 'test@email.com,"",FY26 Q3 ALL,"","",No,"",No,"",No,"","","",,unknown,unknown,unknown,2.0.0,"Apr 23 18:46:21 postfix-smtp postfix/smtp[5969]: 4: to=test@email.com, relay=aspmx.l.google.com[x.x.x.x]:25, delay=1.2, delays=0.12/0/0.29/0.83, dsn=2.0.0, status=sent (250 2.0.0 OK 177-12.53 - gsmtp)",04/23/2026 14:46:21,,,,,,Test,Name'
       count => 1
  }
}
filter{
    mutate{
	 copy => { "message" => "[@metadata][temp]" }
	}
	
	# replace ,\s with something will not show up i.e "=>" in copied temp field
    mutate{
	 gsub => ["[@metadata][temp]", ", ", "=>"]
	}
	
	# parse
	csv{
	source => "[@metadata][temp]"
	separator => ","
	#skip_empty_columns => false
	}
	
	#return back the field separator
     mutate{
	 gsub => ["column19", "=>", ", "]
	}
    mutate{
	remove_field => ["event", "host", "@timestamp", "@version", "message"] 
	}

}
output {
    stdout { codec => rubydebug{ metadata => true}} 
}

```

Output:

```auto
{
      "column1" => "test@email.com",
     "column23" => nil,
...
    "@metadata" => {
        "temp" => "test@email.com,\"\",FY26 Q3 ALL,\"\",\"\",No,\"\",No,\"\",No,\"\",\"\",\"\",,unknown,unknown,unknown,2.0.0,\"Apr 23 18:46:21 postfix-smtp postfix/smtp[5969]: 4: to=test@email.com=>relay=aspmx.l.google.com[x.x.x.x]:25=>delay=1.2=>delays=0.12/0/0.29/0.83=>dsn=2.0.0=>status=sent (250 2.0.0 OK 177-12.53 - gsmtp)\",04/23/2026 14:46:21,,,,,,Test,Name"
    },
     "column19" => "Apr 23 18:46:21 postfix-smtp postfix/smtp[5969]: 4: to=test@email.com, relay=aspmx.l.google.com[x.x.x.x]:25, delay=1.2, delays=0.12/0/0.29/0.83, dsn=2.0.0, status=sent (250 2.0.0 OK 177-12.53 - gsmtp)",
      "message" => "test@email.com,\"\",FY26 Q3 ALL,\"\",\"\",No,\"\",No,\"\",No,\"\",\"\",\"\",,unknown,unknown,unknown,2.0.0,\"Apr 23 18:46:21 postfix-smtp postfix/smtp[5969]: 4: to=test@email.com, relay=aspmx.l.google.com[x.x.x.x]:25, delay=1.2, delays=0.12/0/0.29/0.83, dsn=2.0.0, status=sent (250 2.0.0 OK 177-12.53 - gsmtp)\",04/23/2026 14:46:21,,,,,,Test,Name",
...
}

```