Pulled down the latest alpha (filebeat-5.0.0-alpha3-darwin-x64).
Below is the config to ship .json logs
- input_type: log
# Paths that should be crawled and fetched. Glob based paths.
paths:
- ./test/test.json
#- c:\programdata\elasticsearch\logs\*
json.message_key: name
json.keys_under_root: false
json.overwrite_keys: false
json.add_error_key: true`
Here is the input file
{"name":"first","count":100}
{"name":"second","count":200}
{"name":"third","count":300}
{"name":"4th","count":400}
{"name":"5th","count":0}
{"name":"6th","count":99}
{"name":"7th","count":992}
I got 7 lines to ship, below is the console log when I run ./filebeat -e
**==> ./filebeat -e**
{
"@timestamp": "2016-06-12T17:03:49.223Z",
"beat": {
"hostname": "Sahass-MacBook-Pro.local",
"name": "Sahass-MacBook-Pro.local"
},
"input_type": "log",
"json": {
"count": 100,
"name": "first"
},
"offset": 29,
"source": "./test/test.json",
"type": "log"
}
{
"@timestamp": "2016-06-12T17:03:49.223Z",
"beat": {
"hostname": "Sahass-MacBook-Pro.local",
"name": "Sahass-MacBook-Pro.local"
},
"input_type": "log",
"json": {
"count": 200,
"name": "second"
},
"offset": 59,
"source": "./test/test.json",
"type": "log"
}
{
"@timestamp": "2016-06-12T17:03:49.223Z",
"beat": {
"hostname": "Sahass-MacBook-Pro.local",
"name": "Sahass-MacBook-Pro.local"
},
"input_type": "log",
"json": {
"count": 300,
"name": "third"
},
"offset": 88,
"source": "./test/test.json",
"type": "log"
}
{
"@timestamp": "2016-06-12T17:03:49.223Z",
"beat": {
"hostname": "Sahass-MacBook-Pro.local",
"name": "Sahass-MacBook-Pro.local"
},
"input_type": "log",
"json": {
"count": 400,
"name": "4th"
},
"offset": 115,
"source": "./test/test.json",
"type": "log"
}
{
"@timestamp": "2016-06-12T17:03:49.223Z",
"beat": {
"hostname": "Sahass-MacBook-Pro.local",
"name": "Sahass-MacBook-Pro.local"
},
"input_type": "log",
"json": {
"count": 0,
"name": "5th"
},
"offset": 140,
"source": "./test/test.json",
"type": "log"
}
{
"@timestamp": "2016-06-12T17:03:49.223Z",
"beat": {
"hostname": "Sahass-MacBook-Pro.local",
"name": "Sahass-MacBook-Pro.local"
},
"input_type": "log",
"json": {
"count": 99,
"name": "6th"
},
"offset": 166,
"source": "./test/test.json",
"type": "log"
}
Its is consistently missing the last line when lines are added incrementally
When I run filbeat with preset input file, it does ship all 7 log entries..
Am I missing something obvious?
Thanks !