# Using other than comma as separator in csv filter
**URL:** https://discuss.elastic.co/t/using-other-than-comma-as-separator-in-csv-filter/240067
**Category:** Logstash
**Created:** [July 6, 2020, 11:31pm UTC](https://discuss.elastic.co/t/using-other-than-comma-as-separator-in-csv-filter/240067 "2020-07-06T23:31:54Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Shipper01](https://discuss.elastic.co/u/Shipper01)
#### Post date: [July 6, 2020, 11:31pm UTC](https://discuss.elastic.co/t/using-other-than-comma-as-separator-in-csv-filter/240067/1 "2020-07-06T23:31:55Z")
I have files that has either a semi-colon or pipe as separator. The values within the fields can have commas, so I can't use a commas as separator. I have tried using the csv filter and altering the separator. My conf file and error message are below.
```auto
input {
file {
path => "C:/Users/dclar/Documents/ESTK/material/*.csv"
start_position => "beginning"
sincedb_path => "NUL"
}
}
filter {
csv {
separator => "|"
columns => ["Host"|"AC"|"UUID"|"GUID"|"Bn_Number"|"ID"|"INCREMENT"|"NUMBER"|"UTC"|"DESCRIPTION"|"need"|"Reported"|"Percentage"|"Percentage"]
}
}
output {
elasticsearch {
action => "index"
hosts => "http://localhost:9200"
index => "brute_force"
}
}
```
```auto
C:\logstash-7.8.0\bin>logstash
Sending Logstash logs to C:/logstash-7.8.0/logs which is now configured via log4j2.properties
[2020-07-06T19:25:53,353][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.8.0", "jruby.version"=>"jruby 9.2.11.1 (2.5.7) 2020-03-25 b1f55b1a40 Java HotSpot(TM) Client VM 25.251-b08 on 1.8.0_251-b08 +indy +jit [mswin32-i386]"}
[2020-07-06T19:25:54,098][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:af8, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [\\t\\r\\n], \"#\", \"{\", \",\", \"]\" at line 11, column 21 (byte 223) after filter {\r\n csv {\r\n\t separator => \"|\"\r\n\t columns => [\"Host\"", :backtrace=>["C:/logstash-7.8.0/logstash-core/lib/logstash/compiler.rb:58:in `compile_imperative'", "C:/logstash-7.8.0/logstash-core/lib/logstash/compiler.rb:66:in `compile_graph'", "C:/logstash-7.8.0/logstash-core/lib/logstash/compiler.rb:28:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in `map'", "C:/logstash-7.8.0/logstash-core/lib/logstash/compiler.rb:27:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:181:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:67:in `initialize'", "C:/logstash-7.8.0/logstash-core/lib/logstash/java_pipeline.rb:43:in `initialize'", "C:/logstash-7.8.0/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "C:/logstash-7.8.0/logstash-core/lib/logstash/agent.rb:342:in `block in converge_state'"]}
[2020-07-06T19:25:54,360][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2020-07-06T19:25:59,398][INFO][logstash.runner] Logstash shut down.
C:\logstash-7.8.0\bin>
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 6, 2020, 11:40pm UTC](https://discuss.elastic.co/t/using-other-than-comma-as-separator-in-csv-filter/240067/2 "2020-07-06T23:40:49Z")
> [@Shipper01](#):
>
> `columns => ["Host"|"AC"|"UUID"|"GUID"|"Bn_Number"|"ID"|"INCREMENT"|"NUMBER"|"UTC"|"DESCRIPTION"|"need"|"Reported"|"Percentage"|"Percentage"]`
The separator option sets the separator expected in each event. So you would set that if the lines look like
```
someHost|Foo,bar|baz|someGuid...
```
The columns option of a csv filter expects an array of strings, and that means they have to be comma separated.
```
columns => ["Host", "AC", "UUID", "GUID" ...]
```
---
### Author:  [@Shipper01](https://discuss.elastic.co/u/Shipper01)
#### Post date: [July 7, 2020, 12:30am UTC](https://discuss.elastic.co/t/using-other-than-comma-as-separator-in-csv-filter/240067/3 "2020-07-07T00:30:31Z")
Thank you.
Will column headers be picked up automatically then if I don't list them in the array, but they are in the first line of the csv file?
---
### Author:  [@Shipper01](https://discuss.elastic.co/u/Shipper01)
#### Post date: [July 7, 2020, 12:45am UTC](https://discuss.elastic.co/t/using-other-than-comma-as-separator-in-csv-filter/240067/4 "2020-07-07T00:45:35Z")
Badger,
Thank you, sorry, it took a moment to sink in what you said. The columns option is an array. It is not for defining the columns using the chosen separator. Many thanks.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 7, 2020, 12:53am UTC](https://discuss.elastic.co/t/using-other-than-comma-as-separator-in-csv-filter/240067/5 "2020-07-07T00:53:01Z")
> [@Shipper01](#):
>
> Will column headers be picked up automatically then if I don't list them in the array, but they are in the first line of the csv file?
If your file has the column names in the first line and that uses the same seperator as the other events then the csv filter can handle that using the [autodetect\_column\_names](https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html#plugins-filters-csv-autodetect_column_names) option.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 4, 2020, 12:53am UTC](https://discuss.elastic.co/t/using-other-than-comma-as-separator-in-csv-filter/240067/6 "2020-08-04T00:53:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.