Hi,
I'm currently looking at storing a somewhat large amount of information inside 1 field, I read about the max number of bytes ES could handle so before I started inserting data I used the below template to create the index, set doc values to false and index to false :
PUT /test-index-2018.02.19?pretty
{
"mappings": {
"doc" : {
"properties" : {
"message" : {
"type" : "keyword",
"index" : false,
"doc_values": false
}
}
}
}
}
--
Once this was created without any issues, I then use filebeat to ship the log to logstash (where some of the log file lines have groks to extract things such as time taken field and application exit status.)
Below is a snippit of the message that is passed into ES and visualised in Kibana, the issue is that it only passes through 500 total lines in the message field:
02-02-2018 07:30:01 GMT - NOTICE - Cron started.
02-02-2018 07:30:01 GMT - INFO - array (
0 => '20140506',
1 => '20140703',
2 => '20140704',
3 => '20140705',
4 => '20140706',
5 => '20140707',
6 => '20140708',
7 => '20140709',
8 => '20140710',
9 => '20140711',
10 => '20140712',
+ 488 other lines of similar content.
The log file changes at about 800 lines in (there is about 1500 lines in total) to something like:
01-02-2018 11:00:04 GMT - INFO - Deleted file s3://test-bucket/testfile.csv
But this isn't shown in Kibana, I've looked in the ES logs but nothing (only me creating the index).
Is there a way to be able to see all of the lines in the message field rather than just what appears to be a limit of 500?
Many thanks!