I would like to index 2 Json files (one event line per file) directly to Elastic.
Here is my Filebeat configuraion:
filebeat.modules:
filebeat.prospectors:
- input_type: log
paths:
encoding: utf-8
fields:
document_type: report
fields_under_root: false
max_bytes: 52428800
json.keys_under_root: true
json.overwrite_keys: true
json.add_error_key: true
output.elasticsearch:
hosts: ["elastic:9200"]
index: "inmaze-reports-%{+yyyy.MM}"
bulk_max_size: 10
timeout: 30
template.enabled: true
template.name: "inmaze-reports"
template.path: "${path.config}/reports.template.json"
template.overwrite: true
template.versions.2x.enabled: false
template.versions.6x.enabled: false
logging.level: debug
logging.to_files: true
logging.files:
rotateeverybytes: 10485760
keepfiles: 7
my files both have the following json:
{test1: "test1"}
Here is the debug log of my Filebeats:
End of file reached: c:\temp\reports\test2.json; Backoff now.
2017-11-19T22:47:00+02:00 DBG End of file reached: c:\temp\reports\test1.json; Backoff now.
2017-11-19T22:47:02+02:00 DBG Flushing spooler because of timeout. Events flushed: 2
2017-11-19T22:47:02+02:00 DBG No events to publish
2017-11-19T22:47:02+02:00 DBG Events sent: 2
2017-11-19T22:47:02+02:00 DBG Processing 2 events
2017-11-19T22:47:02+02:00 DBG New state added for c:\temp\reports\test2.json
2017-11-19T22:47:02+02:00 DBG New state added for c:\temp\reports\test1.json
2017-11-19T22:47:02+02:00 DBG Registrar states cleaned up. Before: 2, After: 2
Why there are no events to publish?