# JSON logs not being uploaded
**URL:** https://discuss.elastic.co/t/json-logs-not-being-uploaded/150646
**Category:** Beats
**Tags:** filebeat
**Created:** [October 2, 2018, 5:46am UTC](https://discuss.elastic.co/t/json-logs-not-being-uploaded/150646 "2018-10-02T05:46:12Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@etoews](https://discuss.elastic.co/u/etoews)
#### Post date: [October 2, 2018, 5:46am UTC](https://discuss.elastic.co/t/json-logs-not-being-uploaded/150646/1 "2018-10-02T05:46:12Z")
I've configured Filebeat to harvest some OpenShift Audit logs. The logs are in JSON format so I'm using the [JSON Log input](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-config-json).
However, the logs simply aren't getting uploaded to ElasticSearch (via Logstash). Well...I'm unable to see them in Kibana when I search for "auditID", which is one of the fields as you can see from the Log output below. I've also included as much debugging info as I can think of below.
Can anyone spot any misconfiguration or anything else I might be missing?
Any other debugging tips for this?
Config
```
# grep -A 9 filebeat.inputs /etc/filebeat/filebeat.yml
filebeat.inputs:
- type: docker
containers.ids: '*'
processors:
- add_docker_metadata: ~
- add_host_metadata: ~
- type: log
paths:
- /var/log/audit/openshift.log
json.keys_under_root: true
```
Log output
```
# tail -f /var/log/audit/openshift.log
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":"2018-10-02T05:27:49Z"},"level":"Metadata","timestamp":"2018-10-02T05:27:49Z","auditID":"d65a7719-d789-4dbb-9a36-0944113259b3","stage":"RequestReceived","requestURI":"xxx","verb":"get","user":{},"sourceIPs":["x.x.x.x"],"requestReceivedTimestamp":"2018-10-02T05:27:49.192712Z","stageTimestamp":"2018-10-02T05:27:49.192712Z"}
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":"2018-10-02T05:27:49Z"},"level":"Metadata","timestamp":"2018-10-02T05:27:49Z","auditID":"d65a7719-d789-4dbb-9a36-0944113259b3","stage":"ResponseComplete","requestURI":"xxx","verb":"get","user":{},"sourceIPs":["x.x.x.x"],"responseStatus":{"metadata":{},"code":302},"requestReceivedTimestamp":"2018-10-02T05:27:49.192712Z","stageTimestamp":"2018-10-02T05:27:49.194596Z"}
...
```
Version
```
# filebeat version
filebeat version 6.4.1 (amd64), libbeat 6.4.1 [37b5f2d2a20f2734b2373a454b4b4cbb2627e841 built 2018-09-13 21:25:47 +0000 UTC]
```
Debug Mode
```
# filebeat -c /etc/filebeat/filebeat.yml -e -d *
2018-10-02T18:27:07.604+1300 INFO instance/beat.go:544 Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2018-10-02T18:27:07.604+1300 INFO instance/beat.go:551 Beat UUID: 2cafaf0a-c529-4f84-822f-67c74e45e052
2018-10-02T18:27:07.605+1300 INFO [seccomp] seccomp/seccomp.go:116 Syscall filter successfully installed
2018-10-02T18:27:07.605+1300 INFO [beat] instance/beat.go:768 Beat info {"system_info": {"beat": {"path": {"config": "/etc/filebeat", "data": "/var/lib/filebeat", "home": "/usr/share/filebeat", "logs": "/var/log/filebeat"}, "type": "filebeat", "uuid": "2cafaf0a-c529-4f84-822f-67c74e45e052"}}}
2018-10-02T18:27:07.605+1300 INFO [beat] instance/beat.go:777 Build info {"system_info": {"build": {"commit": "37b5f2d2a20f2734b2373a454b4b4cbb2627e841", "libbeat": "6.4.1", "time": "2018-09-13T21:25:47.000Z", "version": "6.4.1"}}}
2018-10-02T18:27:07.605+1300 INFO [beat] instance/beat.go:780 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.10.3"}}}
...
2018-10-02T18:27:07.643+1300 INFO log/input.go:138 Configured paths: [/var/log/audit/openshift.log]
...
```
Connectivity
```
# filebeat test output
logstash: foo1.com:5044...
connection...
parse host... OK
dns lookup... OK
addresses: x.x.x.x
dial up... OK
TLS... OK
talk to server... OK
logstash: foo2.com:5044...
connection...
parse host... OK
dns lookup... OK
addresses: x.x.x.x
dial up... OK
TLS... OK
talk to server... OK
```
---
### Author:  [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [October 2, 2018, 8:39am UTC](https://discuss.elastic.co/t/json-logs-not-being-uploaded/150646/2 "2018-10-02T08:39:31Z")
For testing purposes could you please set a different registry file for Filebeat and try to send the logs again with debug logging turned on. Please share the ouput of at least the first 20-30 seconds, so we can get an idea of what is happening in your instance.
You can set the registry file to a different path:
```auto
filebeat.registry: ${path.data}/test-registry
```
By setting it to a different path we can try to send events without losing where you left off. You can also delete the registry file, so Filebeat will reread everything it has encountered before. But for a first debug run it's not necessary.
First you could try sending it to a file input, to see if Filebeat is able to find and process your logs.
---
### Author:  [@etoews](https://discuss.elastic.co/u/etoews)
#### Post date: [October 2, 2018, 10:18am UTC](https://discuss.elastic.co/t/json-logs-not-being-uploaded/150646/3 "2018-10-02T10:18:16Z")
Sorry, I'm not very familiar with Beats at all. I don't really know what "try sending it to a file input" means. How would I do that?
Thanks!
---
### Author:  [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [October 2, 2018, 10:24am UTC](https://discuss.elastic.co/t/json-logs-not-being-uploaded/150646/4 "2018-10-02T10:24:16Z")
Apparently, I am not familiar with the English language... I am sorry I meant file output. 🙂
Just add the following snippet to your config and comment out the existing output config.
```auto
output.file:
path: "/path/to/the/input/filebeat"
filename: filebeat
```
See more: [https://www.elastic.co/guide/en/beats/filebeat/current/file-output.html](https://www.elastic.co/guide/en/beats/filebeat/current/file-output.html)
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 30, 2018, 10:24am UTC](https://discuss.elastic.co/t/json-logs-not-being-uploaded/150646/5 "2018-10-30T10:24:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.