I'm having trouble using the UDP input in the same way the file input does with Json files.
I would like automatic field recognition for the structured data it receives.
Here's my test:
I'm using version 6.2.1 (amd64), libbeat 6.2.1.
Bare minimum config file:
filebeat.prospectors:
- type: udp
enabled: true
json.add_error_key: true
json.keys_under_root: true
output.console:
pretty: true
processors:
- decode_json_fields:
fields: ['reads', 'writes']
process_array: false
max_depth: 1
basic test program in golang:
package main
import (
"net"
"fmt"
)
func main() {
//Connect udp
conn, err := net.Dial("udp", "127.0.0.1:8080")
if err != nil {
//return err
}
fmt.Print("connected")
defer conn.Close()
//simple write
conn.Write(byte("{"reads":3,"writes":1}\n"))
}
the json data are simply included in the message field and not split in singular values.
2018-02-27T13:47:32.209Z DEBUG [publish] pipeline/processor.go:275 Publish event: {
"@timestamp": "2018-02-27T13:47:32.209Z",
"@metadata": {
"beat": "filebeat",
"type": "doc",
"version": "6.2.1"
},
"beat": {
"hostname": "eurvlii06649",
"version": "6.2.1",
"name": "eurvlii06649"
},
"message": "{"reads":3,"writes":1}\n",
"prospector": {
"type": "udp"
}
}
{
"@timestamp": "2018-02-27T13:47:32.209Z",
"@metadata": {
"beat": "filebeat",
"type": "doc",
"version": "6.2.1"
},
"message": "{"reads":3,"writes":1}\n",
"prospector": {
"type": "udp"
},
"beat": {
"hostname": "eurvlii06649",
"version": "6.2.1",
"name": "eurvlii06649"
}
}
the message is correct as you can see with this test using shell:
[lc20583@eurvlii06649 filebeat-6.2.1-linux-x86_64]$ sudo nc -ul 127.0.0.1 8080 | jq
{
"reads": 3,
"writes": 1
}
I also tried to remove the whole processor block, but got the same result.
The source code for the udp input has far less function/lines of code compared to the log file, is json parsing not available?
Thanks for any reply!
