# Csv character limitation per row

**URL:** <https://discuss.elastic.co/t/csv-character-limitation-per-row/135309>\
**Category:** Logstash\
**Created:** [June 11, 2018, 8:30am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309 "2018-06-11T08:30:22Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![bobbytan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bobbytan/32/32720_2.png) [@bobbytan](https://discuss.elastic.co/u/bobbytan)\
**Post date:** [June 11, 2018, 8:30am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/1 "2018-06-11T08:30:22Z")

</div>

Hello,

I am new to the ELK,i have noticed that certain rows of my CSV file that exceed about 250 characters are having issues being visualized in kibana after importing using logstash.

Is there a way to solve this?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 11, 2018, 8:34am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/2 "2018-06-11T08:34:02Z")

</div>

What, exactly, do you mean by "having issues being visualized"?

---

<div class="post-metadata">

**Author:** ![bobbytan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bobbytan/32/32720_2.png) [@bobbytan](https://discuss.elastic.co/u/bobbytan)\
**Post date:** [June 11, 2018, 9:08am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/3 "2018-06-11T09:08:59Z")

</div>

thank you for the quick response!

 ![Capture](https://us1.discourse-cdn.com/elastic/original/3X/1/4/141accffafe4f240020a021d4143919c36892418.PNG)  
The image is an example.  
As you can see there are only 2 visible rows.I have imported an csv file consisting of 1 column and 3 rows of data using logstash.The row that is not visible consists of about 280 characters in the csv file.Only after reducing the amount of characters in that particular row would be data be visible in kibana.

How do i solve this?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 11, 2018, 10:02am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/4 "2018-06-11T10:02:46Z")

</div>

Start by taking Elasticsearch out of the equation and use a simple `stdout { codec => rubydebug }` output. Are you getting all expected events?

---

<div class="post-metadata">

**Author:** ![bobbytan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bobbytan/32/32720_2.png) [@bobbytan](https://discuss.elastic.co/u/bobbytan)\
**Post date:** [June 12, 2018, 2:01am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/5 "2018-06-12T02:01:14Z")

</div>

yes im getting all expected events in the shell running logstash

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 12, 2018, 3:16am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/6 "2018-06-12T03:16:42Z")

</div>

Can you show us the raw input (especially the longest line) and configuration you are using to consume it?

---

<div class="post-metadata">

**Author:** ![bobbytan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bobbytan/32/32720_2.png) [@bobbytan](https://discuss.elastic.co/u/bobbytan)\
**Post date:** [June 12, 2018, 4:27am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/7 "2018-06-12T04:27:52Z")

</div>

**raw input longest line:**

WebContainer : 4 - 2018-01-10 18:00:00.168 INFO c.i.g.s.s.e.l.SPLogger:63 - com.ida.gov.sg.sp.eai.interceptor.SingpassInterceptor | | UnAuthenticated URL list | [authnlogin, common, eservicelogout, tamoperationhandler, eservloginpage, loginpage, gettasconnection, errorpage]

**config file:**

input {

file {  
path =\> "C:data\sui\_testing.csv"  
start\_position =\>"beginning"  
sincedb\_path =\> "/dev/null"  
}  
}

filter{  
csv{  
separator =\> ","  
columns =\> ["logs"]

}  
}

output{  
elasticsearch {  
hosts =\> ["localhost:9200"]  
index=\> "logs\_testing"

}

}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 12, 2018, 5:24am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/8 "2018-06-12T05:24:02Z")

</div>

I think this is related to the fact that message.keyword does not created for documents longer than 256 characters, but someone who actually understands anything about elasticsearch would be in a better position to explain that. I just do logstash. Magnus?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 12, 2018, 5:57am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/9 "2018-06-12T05:57:34Z")

</div>

It's most likely the `ignore_above` option in the mapping.

> <https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/v9.2.0/lib/logstash/outputs/elasticsearch/elasticsearch-template-es6x.json#L19-L28>

It's not clear why the OP wants to aggregate over the whole `message` field in the first place. That's probably a mistake, and it's possible that the problem disappears if the line is correctly parsed into fields.

---

<div class="post-metadata">

**Author:** ![bobbytan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bobbytan/32/32720_2.png) [@bobbytan](https://discuss.elastic.co/u/bobbytan)\
**Post date:** [June 12, 2018, 6:56am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/10 "2018-06-12T06:56:44Z")

</div>

ok thank you for the feedback

---

<div class="post-metadata">

**Author:** ![bobbytan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bobbytan/32/32720_2.png) [@bobbytan](https://discuss.elastic.co/u/bobbytan)\
**Post date:** [June 12, 2018, 7:15am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/11 "2018-06-12T07:15:40Z")

</div>

thanks for the solution.Just a follow up to this,  
Is it possible to use logstash to parse a textfile consisting a large of number logs?  
if so,should the parsing be done within the configuration file?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 12, 2018, 8:02am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/13 "2018-06-12T08:02:53Z")

</div>

> Is it possible to use logstash to parse a textfile consisting a large of number logs?

Yes, of course.

> if so,should the parsing be done within the configuration file?

You could use an Elasticsearch ingest pipeline, but apart from that I'm not sure where the parsing would otherwise take place.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 10, 2018, 8:02am UTC](https://discuss.elastic.co/t/csv-character-limitation-per-row/135309/14 "2018-07-10T08:02:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
