# ES 32kb Field Limit - Logstash Ruby Plugin help
**URL:** https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152
**Category:** Logstash
**Created:** [April 30, 2019, 9:08pm UTC](https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152 "2019-04-30T21:08:08Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@EmFalcon](https://discuss.elastic.co/u/EmFalcon)
#### Post date: [April 30, 2019, 9:08pm UTC](https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152/1 "2019-04-30T21:08:08Z")
Using Elastic Stack 6.3. Workflow : Filebeat (input logfile) -\> Logstash -\> ES
A specific log file we have generates a individual message that exceeds 32kb which from what I am reading the limit of lucene for index and searching.
Is it possible to use the ruby filter plugin for logstash to split the field and send to 2 different fields based on size or length (of say 8100 chars)? My ruby skills are non-existent and if anyone can help me it would be greatly appreciated but below is what I THINK is possible. If there is a better way I am all ears.
Help!
filter
{
if ([entity\_type] == "type\_log") {
grok { id =\> "filter\_grok\_type\_log"
match =\> { "message" =\> ("%{GREEDYDATA:message}") # ignore that this isn't exactly my filter
}
ruby {
code =\> message = event["message"].split(0..8100)
message2 = event["message"].split(8101.. ??) # ?? should be end of message field
}
mutate {
replace =\> { "message", "%[message]" }
add\_field =\> { "message2", "%[message2]" }
}
}
}
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [April 30, 2019, 9:41pm UTC](https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152/2 "2019-04-30T21:41:01Z")
The following will chop up a string into 150 character chunks...
```
mutate { add_field => { "someField" => "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex
ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.
Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum." } }
ruby {
code => '
part = 1
s = event.get("someField")
while s != ""
event.set("part#{part}", s[0..150])
s[0..150] = ""
part += 1
end
'
}
```
---
### Author:  [@EmFalcon](https://discuss.elastic.co/u/EmFalcon)
#### Post date: [May 1, 2019, 12:15pm UTC](https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152/3 "2019-05-01T12:15:30Z")
Badger thank you but then my question becomes how do I take that and assign new fields to each chunk? Sorry if I seem obtuse, I feel that way on this problem.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 1, 2019, 1:08pm UTC](https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152/4 "2019-05-01T13:08:17Z")
The filter does that.
```
"part1" => "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor\nincididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, qu",
"part2" => "is nostrud exercitation ullamco laboris nisi ut aliquip ex\nea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum ",
"part3" => "dolore eu fugiat nulla pariatur.\nExcepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.",
```
---
### Author:  [@EmFalcon](https://discuss.elastic.co/u/EmFalcon)
#### Post date: [May 3, 2019, 7:07pm UTC](https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152/5 "2019-05-03T19:07:16Z")
Thank you badger this works well enough that I can run with it.
---
### Author:  [@EmFalcon](https://discuss.elastic.co/u/EmFalcon)
#### Post date: [May 3, 2019, 9:27pm UTC](https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152/6 "2019-05-03T21:27:16Z")
Note that I actually have adjusted the code to read as such now.
```
ruby {
code => '
if event.get("message").length > #value then
part = 1
s = event.get("message")
while s != ""
event.set("message#{part}", s[0..#value])
s[0..#value] = ""
part += 1
event.set("message", "Split fields")
end
end
'
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 31, 2019, 9:27pm UTC](https://discuss.elastic.co/t/es-32kb-field-limit-logstash-ruby-plugin-help/179152/7 "2019-05-31T21:27:16Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.