Connection marked as failed because the onConnect callback failed

Sebastian_Busek · January 11, 2017, 1:51pm

Hello, I'm trying use Filebeat to send json log to elasticsearch, but without success.

By Structured logging with Filebeat blog post, I'd saved sample json and used following configuration in filebeat.yml

filebeat.prospectors:
- input_type: log
  paths:
    - 'C:\Projects\Utils\FileBeats\sample\log.json'

output.elasticsearch:
  hosts: ["http://10.0.10.209:9200/"]
  template.name: filebeat
  template.path: filebeat.template.json

But when I run

.\filebeat.exe -c filebeat.yml -e -v

I got this errors:

2017/01/11 13:48:14.193332 beat.go:267: INFO Home path: [C:\Projects\Utils\FileBeats] Config path: [C:\Projects\Utils\FileBeats] Data path: [C:\Projects\Utils\FileBeats\data] Logs path: [C:\Projects\Utils\FileBeats\logs]
2017/01/11 13:48:14.193332 logp.go:219: INFO Metrics logging every 30s
2017/01/11 13:48:14.194326 beat.go:177: INFO Setup Beat: filebeat; Version: 5.1.1
2017/01/11 13:48:14.195251 output.go:167: INFO Loading template enabled. Reading template file: C:\Projects\Utils\FileBeats\filebeat.template.json
2017/01/11 13:48:14.195251 output.go:178: INFO Loading template enabled for Elasticsearch 2.x. Reading template file: C:\Projects\Utils\FileBeats\filebeat.template-es2x.json
2017/01/11 13:48:14.196243 client.go:120: INFO Elasticsearch url: http://10.0.10.209:9200/
2017/01/11 13:48:14.196243 outputs.go:106: INFO Activated elasticsearch as output plugin.
2017/01/11 13:48:14.196243 publish.go:291: INFO Publisher name: BCZBICSBU2
2017/01/11 13:48:14.200264 async.go:63: INFO Flush Interval set to: 1s
2017/01/11 13:48:14.200264 async.go:64: INFO Max Bulk Size set to: 50
2017/01/11 13:48:14.201251 beat.go:207: INFO filebeat start running.
2017/01/11 13:48:14.202320 registrar.go:85: INFO Registry file set to: C:\Projects\Utils\FileBeats\data\registry
2017/01/11 13:48:14.206245 registrar.go:106: INFO Loading registrar data from C:\Projects\Utils\FileBeats\data\registry
2017/01/11 13:48:14.206245 registrar.go:131: INFO States Loaded from registrar: 0
2017/01/11 13:48:14.206245 registrar.go:230: INFO Starting Registrar
2017/01/11 13:48:14.206245 crawler.go:34: INFO Loading Prospectors: 1
2017/01/11 13:48:14.206245 spooler.go:63: INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2017/01/11 13:48:14.206245 sync.go:41: INFO Start sending events to output
2017/01/11 13:48:14.206245 prospector_log.go:57: INFO Prospector with previous states loaded: 0
2017/01/11 13:48:14.207244 crawler.go:46: INFO Loading Prospectors completed. Number of prospectors: 1
2017/01/11 13:48:14.207244 crawler.go:61: INFO All prospectors are initialised and running with 0 states to persist
2017/01/11 13:48:14.207244 prospector.go:111: INFO Starting prospector of type: log
2017/01/11 13:48:14.208244 log.go:84: INFO Harvester started for file: C:\Projects\Utils\FileBeats\sample\log.json
2017/01/11 13:48:19.392773 client.go:652: INFO Connected to Elasticsearch version 5.1.1
2017/01/11 13:48:19.392773 output.go:214: INFO Trying to load template for client: http://10.0.10.209:9200/
2017/01/11 13:48:19.399745 single.go:140: ERR Connecting error publishing events (retrying): Connection marked as failed because the onConnect callback failed: Could not load template: couldn't load template: couldn't load json. Error: 500 Internal Server Error
2017/01/11 13:48:20.406677 client.go:652: INFO Connected to Elasticsearch version 5.1.1
2017/01/11 13:48:20.406677 output.go:214: INFO Trying to load template for client: http://10.0.10.209:9200/
2017/01/11 13:48:20.412675 single.go:140: ERR Connecting error publishing events (retrying): Connection marked as failed because the onConnect callback failed: Could not load template: couldn't load template: couldn't load json. Error: 500 Internal Server Error
2017/01/11 13:48:21.892998 filebeat.go:168: INFO Stopping filebeat
2017/01/11 13:48:21.892998 crawler.go:67: INFO Stopping Crawler
2017/01/11 13:48:21.893998 crawler.go:73: INFO Stopping 1 prospectors
2017/01/11 13:48:21.893998 prospector.go:181: INFO Stopping Prospector
2017/01/11 13:48:21.893998 prospector.go:128: INFO Prospector channel stopped
2017/01/11 13:48:21.893998 log.go:105: INFO Reader was closed: C:\Projects\Utils\FileBeats\sample\log.json. Closing.
2017/01/11 13:48:21.893998 prospector.go:152: INFO Prospector ticker stopped
2017/01/11 13:48:21.895001 crawler.go:80: INFO Crawler stopped
2017/01/11 13:48:21.895001 spooler.go:101: INFO Stopping spooler
2017/01/11 13:48:21.895001 registrar.go:285: INFO Stopping Registrar
2017/01/11 13:48:21.895001 registrar.go:242: INFO Ending Registrar
2017/01/11 13:48:21.897001 logp.go:245: INFO Total non-zero values:  registrar.writes=1 filebeat.harvester.started=1 filebeat.harvester.closed=1 libbeat.es.publish.read_bytes=1372 libbeat.es.publish.write_bytes=3144 libbeat.publisher.published_events=1
2017/01/11 13:48:21.897001 logp.go:246: INFO Uptime: 7.7087189s
2017/01/11 13:48:21.897001 beat.go:211: INFO filebeat stopped.

Can you give me some advices? Thank you and have a nice day!

steffens · January 11, 2017, 3:29pm

This is your actual error:

 Error: 500 Internal Server Error

Have you checked elasticsearch logs?

Sebastian_Busek · January 12, 2017, 8:03am

Thank you for reply, but in my case problem was '/' in the end of hosts: ["http://10.0.10.209:9200/"]; so current configuration is following:

filebeat.prospectors:
- input_type: log
  paths:
    - 'C:\Projects\Utils\FileBeats\sample\log.json'

output.elasticsearch:
  hosts: ["http://10.0.10.209:9200"]
  template.name: filebeat
  template.path: filebeat.template.json

system · February 9, 2017, 8:03am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.