### Author:  [@cilzzz](https://discuss.elastic.co/u/cilzzz)
#### Post date: [November 27, 2017, 9:09am UTC](https://discuss.elastic.co/t/csv-parse-error/109187/1 "2017-11-27T09:09:31Z")
Hello,
i am trying to parse a csv file but when logstash piupeline started this error is showing
my filter in logstash is :
```
filter {
if [type] == "soe_phys_tsm" {
csv {
columns => ["ParentCiNum","Parent_Id","Parent_Objet","Parent_CiAppartenanceCmdb","Parent_Classe","Parent_Type","Parent_SousType","Parent_Proprietaire","Parent_Environnement","Parent_Nom","Parent_ComplementUnicite","Relation","EnfantCiNum","Enfant_Id","Enfant_Objet","Enfant_CiAppartenanceCmdb","Enfant_Classe","Enfant_Type","Enfant_SousType","Enfant_Proprietaire","Enfant_Environnement","Enfant_Nom","Enfant_ComplementUnicite","DateCreation","DateModification","Parent_CiNumOasis","Enfant_CiNumOasis","Relation_Verbe"]
separator => ","
quote_char => "`"
}
}
}
```
and the error is :
{:timestamp=\>"2017-11-27T09:47:19.113000+0100", :message=\>"Error parsing csv", :field=\>"message", :source=\>"", :exception=\>#\, :level=\>:warn}
so no data in showing in kibana can anyone help please
Thank you
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 27, 2017, 9:27am UTC](https://discuss.elastic.co/t/csv-parse-error/109187/2 "2017-11-27T09:27:52Z")
What does the input look like?
---
### Author:  [@cilzzz](https://discuss.elastic.co/u/cilzzz)
#### Post date: [November 27, 2017, 9:34am UTC](https://discuss.elastic.co/t/csv-parse-error/109187/3 "2017-11-27T09:34:53Z")
actually filebeat is sending logs to logstash so my filebeat.yml file is
```
-
paths:
- /app/list/logs/cmdb/soe-cici-relations.csv
ignore_older: 1m
document_type: soe_phys_tsm
fields:
env: hors-production
client: silca
filebeat_v: 1.0
```
and my logstash input is:
input {
beats {
port =\> 8080
}
}
it worked on other csv files but i don't know what this error means
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [November 27, 2017, 12:36pm UTC](https://discuss.elastic.co/t/csv-parse-error/109187/4 "2017-11-27T12:36:48Z")
What Christian meant was, what does the _input file_ look like?
---
### Author:  [@cilzzz](https://discuss.elastic.co/u/cilzzz)
#### Post date: [November 27, 2017, 3:04pm UTC](https://discuss.elastic.co/t/csv-parse-error/109187/5 "2017-11-27T15:04:59Z")
it is 175150 lines log file and it is generated by a machine. i think the error is that the csv file contains quotations marks on each term like that
## "Header1","Header2","Header3" "aaa,"bbb","ccc"
-
-
-
-
-
how can i remove the all the "" from my csv file in logstash config file please
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 27, 2017, 3:06pm UTC](https://discuss.elastic.co/t/csv-parse-error/109187/6 "2017-11-27T15:06:24Z")
> [@cilzzz](#):
>
> quote\_char =\> "`"
Then it does seem like this does not match you data. based on the sample you provided the default option should work, so try removing this parameter.
---
### Author:  [@cilzzz](https://discuss.elastic.co/u/cilzzz)
#### Post date: [November 27, 2017, 3:17pm UTC](https://discuss.elastic.co/t/csv-parse-error/109187/7 "2017-11-27T15:17:32Z")
thank you i will try it
---
### Author:  [@cilzzz](https://discuss.elastic.co/u/cilzzz)
#### Post date: [November 28, 2017, 9:01am UTC](https://discuss.elastic.co/t/csv-parse-error/109187/8 "2017-11-28T09:01:25Z")
it didn't work is there a method to eliminate all the double quotes ina csv file in logstash config file ?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 28, 2017, 10:01am UTC](https://discuss.elastic.co/t/csv-parse-error/109187/9 "2017-11-28T10:01:42Z")
What does your current config look like? Can you show a full raw input event?
---
### Author:  [@cilzzz](https://discuss.elastic.co/u/cilzzz)
#### Post date: [November 28, 2017, 10:56am UTC](https://discuss.elastic.co/t/csv-parse-error/109187/10 "2017-11-28T10:56:12Z")
i have a large csv file in this format
"header1";"header2";header3".... "
aaaaaaa";"bbbbbbb";ccccccc"...
..............................
..............................
it is the input file in logstash
what should i write in logstash config file to eliminate all the double quotes
it is saying this error:
{\> :timestamp=\>"2017-11-27T16:56:31.108000+0100", :message=\>"Pipeline main started"}
> {:timestamp=\>"2017-11-27T16:56:47.940000+0100", :message=\>"Error parsing csv", :field=\>"message", :source=\>"rentCiNum";"Parent\_Id";"Parent\_Objet";"Parent\_CiAppartenanceCmdb";"Parent\_Classe";"Parent\_Type";"Parent\_SousType";"Parent\_Proprietaire";"Parent\_Environnement";"Parent\_Nom";"Parent\_ComplementUnicite";"Relation";"EnfantCiNum";"Enfant\_Id";"Enfant\_Objet";"Enfant\_CiAppartenanceCmdb";"Enfant\_Classe";"Enfant\_Type";"Enfant\_SousType";"Enfant\_Proprietaire";"Enfant\_Environnement";"Enfant\_Nom";"Enfant\_ComplementUnicite";"DateCreation";"DateModification";"Parent\_CiNumOasis";"Enfant\_CiNumOasis";"Relation\_Verbe"", :exception=\>#, :level=\>:warn}
> exception=\>#\, :level=\>:warn}
my logstash config file is:
> filter {
>
> if [type] == "soe\_phys\_tsm" {
> csv {
> columns =\> ["ParentCiNum","Parent\_Id","Parent\_Objet","Parent\_CiAppartenanceCmdb","Parent\_Classe","Parent\_Type","Parent\_SousType","Parent\_Proprietaire","Parent\_Environnement","Parent\_Nom","Parent\_ComplementUnicite","Relation","EnfantCiNum","Enfant\_Id","Enfant\_Objet","Enfant\_CiAppartenanceCmdb","Enfant\_Classe","Enfant\_Type","Enfant\_SousType","Enfant\_Proprietaire","Enfant\_Environnement","Enfant\_Nom","Enfant\_ComplementUnicite","DateCreation","DateModification","Parent\_CiNumOasis","Enfant\_CiNumOasis","Relation\_Verbe"]
> separator =\> ";"
>
> ```
> }
> }
>
> ```
>
> }
thank you
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 28, 2017, 11:11am UTC](https://discuss.elastic.co/t/csv-parse-error/109187/11 "2017-11-28T11:11:20Z")
It does look like your test data is missing a few double quotes. I ran the following config using a corrected copy of your test data:
```auto
input {
generator {
lines => ['"aaaaaaa";"bbbbbbb";"ccccccc"']
count => 1
}
}
filter {
csv {
columns => ["ParentCiNum","Parent_Id","Parent_Objet","Parent_CiAppartenanceCmdb","Parent_Classe","Parent_Type","Parent_SousType","Parent_Proprietaire","Parent_Environnement","Parent_Nom","Parent_ComplementUnicite","Relation","EnfantCiNum","Enfant_Id","Enfant_Objet","Enfant_CiAppartenanceCmdb","Enfant_Classe","Enfant_Type","Enfant_SousType","Enfant_Proprietaire","Enfant_Environnement","Enfant_Nom","Enfant_ComplementUnicite","DateCreation","DateModification","Parent_CiNumOasis","Enfant_CiNumOasis","Relation_Verbe"]
separator => ";"
skip_empty_columns => true
}
}
output {
stdout { codec => rubydebug }
}
```
Which generated:
```auto
{
"sequence" => 0,
"@timestamp" => 2017-11-28T11:09:04.220Z,
"Parent_Objet" => "ccccccc",
"@version" => "1",
"host" => "whitenode",
"Parent_Id" => "bbbbbbb",
"ParentCiNum" => "aaaaaaa",
"message" => "\"aaaaaaa\";\"bbbbbbb\";\"ccccccc\""
}
```
Seems fine to me.
---
### Author:  [@cilzzz](https://discuss.elastic.co/u/cilzzz)
#### Post date: [November 28, 2017, 11:33am UTC](https://discuss.elastic.co/t/csv-parse-error/109187/12 "2017-11-28T11:33:36Z")
thank you in fact the inputand output are in a a config file apart and cannot be modified bcz i am using filebeat to send logs to logstash . and for each log i should create a filter in a config file apart. there some large logs file generated by a machine and i should not get rid of the double quotes manually. some log file are without double quotes like this format:
a;b;c
d;e;f
which it worked
and some
"a";"b";"c"
"d";"e";"f"
i need to get rid of all the double quotes it is 175150 lines log file!!!!!
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 28, 2017, 11:42am UTC](https://discuss.elastic.co/t/csv-parse-error/109187/13 "2017-11-28T11:42:04Z")
The csv filter by default removes the double quotes when it parses a line, as you can see in my example.
---
### Author:  [@cilzzz](https://discuss.elastic.co/u/cilzzz)
#### Post date: [November 28, 2017, 1:24pm UTC](https://discuss.elastic.co/t/csv-parse-error/109187/14 "2017-11-28T13:24:03Z")
thanks
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 26, 2017, 1:24pm UTC](https://discuss.elastic.co/t/csv-parse-error/109187/15 "2017-12-26T13:24:30Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.