Filebeat getting error when parsing json

Elastic Stack Beats
1

Hi there,

Really new to the filebeat.

I have a beat config like below.

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /tmp/api_json.log
  fields:
    log_group: gitlab_test
    log_id: api_json
  json.keys_under_root: true


output.logstash:
  # Array of hosts to connect to.
  hosts: ["192.168.1.1:5044"]
  ssl.certificate_authorities: ["/etc/filebeat/exam-cirunner1.example.au.crt"]
  ssl.certificate: "/etc/filebeat/exam-cirunner1.example.au.crt"
  ssl.key: "/etc/filebeat/exam-cirunner1.example.au.key"

The /tmp/api_json.log file is like this:

{"time":"2021-03-12T00:40:57.016Z","severity":"INFO","duration_s":0.02124,"db_duration_s":0.00624,"view_duration_s":0.015,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"13.9.0","revision":"2ebc4dc4","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null}}},{"key":"token","value":"[FILTERED]"},{"key":"last_update","value":"850530bed1cfb46fc50f5ac9d0279288"}],"host":"gitlab.cecs.anu.edu.au","remote_ip":"150.203.186.174, 150.203.186.174","ua":"gitlab-runner 13.9.0 (13-9-stable; go1.13.8; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.00724,"redis_calls":5,"redis_duration_s":0.002841,"correlation_id":"62YEwxh9m9a","meta.caller_id":"/api/:version/jobs/request"}
{"time":"2021-03-12T03:01:06.140Z","severity":"INFO","duration_s":0.00966,"db_duration_s":0.00341,"view_duration_s":0.00625,"status":403,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"12.7.0","revision":"58272c27","platform":"linux","architecture":"amd64","executor":"shell","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null}}},{"key":"token","value":"[FILTERED]"}],"host":"gitlab.cecs.anu.edu.au","remote_ip":"150.203.211.228, 150.203.211.228","ua":"gitlab-runner 12.7.0 (12-7-stable; go1.13.5; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.016515,"redis_calls":2,"redis_duration_s":0.002369,"correlation_id":"n1sFXP3hpv3","meta.caller_id":"/api/:version/jobs/request"}

{"time":"2021-03-12T03:02:28.476Z","severity":"INFO","duration_s":0.03199,"db_duration_s":0.0141,"view_duration_s":0.01789,"status":204,"method":"POST","path":"/api/v4/jobs/request","params":[{"key":"info","value":{"name":"gitlab-runner","version":"13.9.0","revision":"2ebc4dc4","platform":"linux","architecture":"amd64","executor":"docker","shell":"bash","features":{"variables":"[FILTERED]","image":null,"services":null,"artifacts":null,"cache":null,"shared":null,"upload_multiple_artifacts":null,"upload_raw_artifacts":null,"session":null,"terminal":null,"refspecs":null,"masking":null,"proxy":null,"raw_variables":"[FILTERED]","artifacts_exclude":null,"multi_build_steps":null,"trace_reset":"[FILTERED]","trace_checksum":"[FILTERED]","trace_size":"[FILTERED]","vault_secrets":"[FILTERED]","cancelable":null,"return_exit_code":null}}},{"key":"token","value":"[FILTERED]"},{"key":"last_update","value":"84da15a4efb5d40762a8de767e9773cc"}],"host":"gitlab.cecs.anu.edu.au","remote_ip":"150.203.186.143, 150.203.186.143","ua":"gitlab-runner 13.9.0 (13-9-stable; go1.13.8; linux/amd64)","route":"/api/:version/jobs/request","queue_duration_s":0.006998,"redis_calls":5,"redis_duration_s":0.005071,"correlation_id":"2Mu1bM9SzF6","meta.caller_id":"/api/:version/jobs/request"}

Now I can't get any event in kabana, so I checked the log file at file beat like below.

ERROR        [reader_json]        readjson/json.go:57        Error decoding JSON: EOF

I copied the log into JSON validation website and it said this is a valid JSON so not quite understand what's going on there.

The version of filebeat is 7.10.2.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.