# Filebeat ignoring json created by ruby
**URL:** https://discuss.elastic.co/t/filebeat-ignoring-json-created-by-ruby/191073
**Category:** Beats
**Tags:** filebeat
**Created:** [July 17, 2019, 6:54pm UTC](https://discuss.elastic.co/t/filebeat-ignoring-json-created-by-ruby/191073 "2019-07-17T18:54:46Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@DevopsNewb](https://discuss.elastic.co/u/DevopsNewb)
#### Post date: [July 17, 2019, 6:54pm UTC](https://discuss.elastic.co/t/filebeat-ignoring-json-created-by-ruby/191073/1 "2019-07-17T18:54:46Z")
Thanks in advance for any help - I've been fighting with this thing all day and I don't feel like I've made much progress.
Here's the situation. Using Ansible facts, we are pulling out a list of all installed packages, then using ruby to parse out the values we need and create a (valid) json file. This part works great.
For whatever reason, filebeat will completely ignore this file no matter what you do (short of changing a single character in the file manually). I have tried creating it in a different folder and moving it to the prospector path, I've tried to copy, cat, echo, rename, chown to the local user, chmod to 755, etc, etc, etc, this file and nothing makes filebeat see it besides opening it in vim and changing even a single character and saving it. I guess saving it from vim causes it to be streamed, which causes filebeat to see it.
Further, I can see filebeat actually LOOKING at and actively ignoring the file. I see line after line of the file with "offset 0" and "backoff now" so it's clear filebeat thinks this is an old file. We are not using ignore\_older.
Everything seems to be configured correctly, but it's not picking up the file until the offset goes to \>0.
Oh, and I almost forgot to mention, I was putting the Ansible dump.json file (pre-parsed) in the same folder and IT got picked up just fine, but the one created by ruby does not.
Any help is greatly appreciated.
---
### Author:  [@Michal\_Pristas](https://discuss.elastic.co/u/Michal_Pristas)
#### Post date: [July 18, 2019, 3:13pm UTC](https://discuss.elastic.co/t/filebeat-ignoring-json-created-by-ruby/191073/2 "2019-07-18T15:13:04Z")
hey @DevopsNewb, welcome to discuss.
can you please share your FB config?
please make sure indentation is correct when you paste it and use `` button to format
---
### Author:  [@DevopsNewb](https://discuss.elastic.co/u/DevopsNewb)
#### Post date: [July 18, 2019, 3:21pm UTC](https://discuss.elastic.co/t/filebeat-ignoring-json-created-by-ruby/191073/3 "2019-07-18T15:21:53Z")
```
filebeat.prospectors:
- input_type: log
paths:
- "/home/ec2-user/*.json"
document_type: package-log-json
fields:
host_type: "{{ app_name }}-{{ bld_env }}"
beats: true
output.logstash:
hosts: ["logstash-int:5044"]
timeout: 45
ssl.certificate_authorities: ["/etc/filebeat/logstash-forwarder.crt"]
logging.to_files: true
logging.files:
path: /var/log/filebeat
name: filebeat.log
logging.selectors: ["*"]
logging.level: debug
```
---
### Author:  [@DevopsNewb](https://discuss.elastic.co/u/DevopsNewb)
#### Post date: [July 18, 2019, 3:23pm UTC](https://discuss.elastic.co/t/filebeat-ignoring-json-created-by-ruby/191073/4 "2019-07-18T15:23:39Z")
I removed the extraneous prospectors. I know the yml is in good working order, or ansible would pitch a fit and crash the deployment. Also, this works just fine, if the file is edited in place.
---
### Author:  [@DevopsNewb](https://discuss.elastic.co/u/DevopsNewb)
#### Post date: [July 31, 2019, 1:13pm UTC](https://discuss.elastic.co/t/filebeat-ignoring-json-created-by-ruby/191073/5 "2019-07-31T13:13:58Z")
Alright, I never got it to work, but I did figure out a workaround. Since I'm modifying the JSON on write anyway, I just had it leave off the closing brace, then inserted it with Ansible (echo '}' \>\> file) and Filebeat catches it now. Weird, but we can mark it complete.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 28, 2019, 1:14pm UTC](https://discuss.elastic.co/t/filebeat-ignoring-json-created-by-ruby/191073/6 "2019-08-28T13:14:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.