# Csv parsing error, like in an out of buffer error \[solved\]

**URL:** <https://discuss.elastic.co/t/csv-parsing-error-like-in-an-out-of-buffer-error-solved/91566>\
**Category:** Logstash\
**Created:** [July 2, 2017, 4:41pm UTC](https://discuss.elastic.co/t/csv-parsing-error-like-in-an-out-of-buffer-error-solved/91566 "2017-07-02T16:41:26Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dfermiumlabs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dfermiumlabs/32/17197_2.png) [@dfermiumlabs](https://discuss.elastic.co/u/dfermiumlabs)\
**Post date:** [July 2, 2017, 4:41pm UTC](https://discuss.elastic.co/t/csv-parsing-error-like-in-an-out-of-buffer-error-solved/91566/1 "2017-07-02T16:41:26Z")

</div>

Hy guys, i'm having a really hard time diagnosing this.

I'm getting tons of tons of parsing errors where it appears logstash is sending to ES completely out of bound fields. For example, it's supposed to send a number between the two "," in a csv but it's just rending random jibberish, some of which i can't find anywhere in the logs.

I think this could be a bug, the csv filter reading where it's not supposed to.

Example config:

```
input {
  file {
path => "/home/cdrs/*.CDR"
start_position => "beginning"
# sincedb_path => "/home/cdrs/sincedb/sincedb"
sincedb_path => "/dev/null"

ignore_older => 86400
  }
}
filter
{
  mutate {
   gsub => ["message", '\\"', '']
   gsub => ['message','\"','']
   gsub => ['message','"',""]
   remove_field => ["null*"]
  }
  csv {
  separator => ";"
 columns => ["start-time","start-time_ts", "another 200 or so columns here..."]
 periodic_flush => true
 skip_empty_columns => true
  }
}
output {
   elasticsearch {
 hosts => "http://localhost:9200"
 index => "cdr"
  }
}

```

here is an error, from elasticsearch as an example, from a similar config:

```
[2017-07-02T17:24:49,290][DEBUG][o.e.a.b.TransportShardBulkAction] [OHshd_3] [callstat][3] failed to execute bulk item (index) BulkShardRequest [[callstat][3]] containing [22] requests
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [egress_peak_concurrent_calls]
	at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:298) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:450) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.DocumentParser.parseValue(DocumentParser.java:576) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.DocumentParser.innerParseObject(DocumentParser.java:396) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrNested(DocumentParser.java:373) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.DocumentParser.internalParseDocument(DocumentParser.java:93) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:66) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:277) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:536) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:513) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.action.bulk.TransportShardBulkAction.prepareIndexOperationOnPrimary(TransportShardBulkAction.java:450) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:458) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:143) [elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:113) [elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:69) [elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:939) [elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:908) [elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:113) [elasticsearch-5.4.1.jar:5.4.1]

[...]

	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_73]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_73]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]
Caused by: java.lang.NumberFormatException: For input string: "ingress-reg-p;;;;;;;;;PCMA;PCMA;"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_73]
	at java.lang.Long.parseLong(Long.java:589) ~[?:1.8.0_73]
	at java.lang.Long.parseLong(Long.java:631) ~[?:1.8.0_73]
	at org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:172) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.NumberFieldMapper$NumberType$7.parse(NumberFieldMapper.java:740) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.NumberFieldMapper$NumberType$7.parse(NumberFieldMapper.java:719) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:1058) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:287) ~[elasticsearch-5.4.1.jar:5.4.1]
	... 36 more

```

The funny thing? `ingress-reg-p;;;;;;;;;PCMA;PCMA;` should not be anywhere in that place, it's definetly a parsing bug, but it's parsing the wrong file. from another configuration.

What do you think?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 2, 2017, 5:08pm UTC](https://discuss.elastic.co/t/csv-parsing-error-like-in-an-out-of-buffer-error-solved/91566/2 "2017-07-02T17:08:42Z")

</div>

Do you by any chance have multiple configuration files that Logstash picks up and concatenation, leading data from other inputs being processed by the csv filter?

---

<div class="post-metadata">

**Author:** ![dfermiumlabs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dfermiumlabs/32/17197_2.png) [@dfermiumlabs](https://discuss.elastic.co/u/dfermiumlabs)\
**Post date:** [July 2, 2017, 5:43pm UTC](https://discuss.elastic.co/t/csv-parsing-error-like-in-an-out-of-buffer-error-solved/91566/3 "2017-07-02T17:43:04Z")

</div>

yeah. that's the problem. it's a bit counterintuitive.

Thank you very much

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 30, 2017, 5:43pm UTC](https://discuss.elastic.co/t/csv-parsing-error-like-in-an-out-of-buffer-error-solved/91566/4 "2017-07-30T17:43:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.