Hello everyone,
filebeat.prospectors:
- type: log
paths:
- "C:/logs/info/*registry2json.json"
json.keys_under_root: true
json.add_error_key: true
json.messsage_key: message
tags: registry_json
clean_removed: true
output.file:
path: 'C:/logs/temp'
filename: filebeat
codec.json:
pretty: true
Here is a sample of my single line JSON file:
{"ComputerName":"computer1","alignedServer":"server1","appVersion":"18.0.27.255","MSWindowsVersion":"10.0","MAC":"70-00-00-00-00-00","date":"04/13/2018 10:08:30","HKLM":{"HKLM\\Software\\company\\hrbu":{"Habitat":"QA","LANTaxServer":"server1", }}
The file I am trying to prospect is From all that I have read in the documentation, this should be working.
Any help would be greatly appreciated
The JSON you posted in invalid. But even so Filebeat will send an event saying:
"error": {
"message": "Error decoding JSON: EOF",
"type": "json"
},
What Filebeat version are you testing this with? Each time you run the test you'll likely want to remove the registry file (./data/registry) that gets created so that it re-reads the input file from the start.
Thank you for your reply Andrew.
Currently using 6.2.3.
The JSON I posted is just a very small sample of the complete file. I have verified that the full file is is valid in JSONlint.
I forgot to add, as part of a process that runs on our systems, the JSON file is deleted and recreated every 90 minutes. That is why I have the clean_removed enabled. Does that not do the same as delete the registry file but just for that specific file?
Thanks
As a troubleshooting step I would do a test to see if the output file is populated at all after removing the registry manually. You could also try the console output to see if that works (since you said that it is harvesting but there's no output).
Adding some debug logging could be useful too. In your config file:
logging.level: debug
Just to be sure, the registry file was deleted and the filebeat service started again. Still no luck.
I have also tried the console route, again, no luck.
I have debug mode on, and the filebeat log shows:
wash, rinse, repeat.
What is strange is I can harvest other log files with no issue from the same directory with the same permissions, but this file will not.
Thanks
steffens
April 16, 2018, 5:03pm
7
Are the json events split by newlines? Filebeat still requires a newline symbol per event. The log messages say the offset is still 0, while the reader is at the end of the file. It might be the case the file contents is still buffered, waiting for the newline in order to forward the raw contents to the JSON parser.
Steffens,
No, there is no new lines as the file is all one line JSON. The file does not include a new line at the end of the file.
I just added the new line, and boom, data was sent.
Thanks for the help everyone!
system
May 14, 2018, 7:52pm
9
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.