***Logstash 7.2 in docker container.
For test I used simple configuration.
input {
tcp {
port => 5959
codec => json
}
}
output {
amazon_es
}
And my logs in Kibana has tag: _jsonparsefailure .
When I used udp instead tcp
input {
udp {
port => 5959
codec => json
}
}
output {
amazon_es
}
My logs in kibana without some carapted data. I linked it whith statment that udp has:
bufer_size and quequ_size
May when I sent log by one big file (500Kb ~ 18000 rows) to tcp input, it can not get time for parsing all file, and did corrupted json with clipped messages and tags: _jsonparsefailure.
Perhaps for udp in this time used bufer and quequ which allowed take all file and processed.
For avoid situation with clipped measges and tags: _jsonparsefailure I tried to used logstesh.yml
queue.type: persisted
queue.max_bytes: 100mb
but it not help and how i noticed, if I got info from apiport 9600:
curl -XGET 'logstash:9600/_node/stats'
"queue" : {
**"events"** **:** 0 **,**
**"capacity"** **: {**
**"max_queue_size_in_bytes"** **:** 104857600 **,**
**"queue_size_in_bytes"** **:** 47839 **,**
**"page_capacity_in_bytes"** **:** 67108864 **,**
**"max_unread_events"** **:** 0
queue_size_in_bytes" - increase after every receive logs. Maybe quequ not work?