ERR failed to initialize elasticsearch plugin as output: missing condition

sunny1 · March 30, 2018, 7:12pm

did as you mentioned. Opened my file in notepad and at end of line one pressed enter and saved both the files with new name and made changes accordingly in yml config. But i got error this time saying error decoding json.

Output log:

2018-03-30T12:10:32-07:00 DBG Disable stderr logging
2018-03-30T12:10:32-07:00 INFO Home path: [C:\Users\sgorasia\Downloads\filebeat] Config path: [C:\Users\sgorasia\Downloads\filebeat] Data path: [C:\ProgramData\filebeat] Logs path: [C:\Users\sgorasia\Downloads\filebeat\logs]
2018-03-30T12:10:32-07:00 INFO Setup Beat: filebeat; Version: 5.5.1
2018-03-30T12:10:32-07:00 DBG Processors:
2018-03-30T12:10:32-07:00 DBG Initializing output plugins
2018-03-30T12:10:32-07:00 INFO Loading template enabled. Reading template file: C:\Users\sgorasia\Downloads\filebeat\filebeat.template.json
2018-03-30T12:10:32-07:00 INFO Loading template enabled for Elasticsearch 2.x. Reading template file: C:\Users\sgorasia\Downloads\filebeat\filebeat.template-es2x.json
2018-03-30T12:10:32-07:00 INFO Loading template enabled for Elasticsearch 6.x. Reading template file: C:\Users\sgorasia\Downloads\filebeat\filebeat.template-es6x.json
2018-03-30T12:10:32-07:00 INFO Elasticsearch url: http://localhost:9200
2018-03-30T12:10:32-07:00 INFO Activated elasticsearch as output plugin.
2018-03-30T12:10:32-07:00 DBG Create output worker
2018-03-30T12:10:32-07:00 DBG No output is defined to store the topology. The server fields might not be filled.
2018-03-30T12:10:32-07:00 INFO Publisher name: 5CG722343K
2018-03-30T12:10:32-07:00 INFO Flush Interval set to: 1s
2018-03-30T12:10:32-07:00 INFO Max Bulk Size set to: 50
2018-03-30T12:10:32-07:00 DBG create bulk processing worker (interval=1s, bulk size=50)
2018-03-30T12:10:32-07:00 INFO filebeat start running.
2018-03-30T12:10:32-07:00 INFO Registry file set to: C:\ProgramData\filebeat\registry
2018-03-30T12:10:32-07:00 INFO Loading registrar data from C:\ProgramData\filebeat\registry
2018-03-30T12:10:32-07:00 DBG Windows is interactive: false
2018-03-30T12:10:32-07:00 INFO States Loaded from registrar: 10
2018-03-30T12:10:32-07:00 INFO Loading Prospectors: 2
2018-03-30T12:10:32-07:00 INFO Starting Registrar
2018-03-30T12:10:32-07:00 INFO Start sending events to output
2018-03-30T12:10:32-07:00 DBG File Configs: [C:\TestRail\TestRailResults14.json]
2018-03-30T12:10:32-07:00 INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2018-03-30T12:10:32-07:00 DBG exclude_files: []
2018-03-30T12:10:32-07:00 DBG New state added for C:\TestRail\TestRailResults14.json
2018-03-30T12:10:32-07:00 INFO Prospector with previous states loaded: 1
2018-03-30T12:10:32-07:00 INFO Starting prospector of type: log; id: 3140866685714624168
2018-03-30T12:10:32-07:00 DBG Start next scan
2018-03-30T12:10:32-07:00 DBG File Configs: [C:\TestRail\TestRailResults15.json]
2018-03-30T12:10:32-07:00 DBG exclude_files: []
2018-03-30T12:10:32-07:00 DBG New state added for C:\TestRail\TestRailResults15.json
2018-03-30T12:10:32-07:00 INFO Prospector with previous states loaded: 1
2018-03-30T12:10:32-07:00 INFO Starting prospector of type: log; id: 10374277926638004162
2018-03-30T12:10:32-07:00 DBG Check file for harvesting: C:\TestRail\TestRailResults14.json
2018-03-30T12:10:32-07:00 INFO Loading and starting Prospectors completed. Enabled prospectors: 2
2018-03-30T12:10:32-07:00 DBG Start next scan
2018-03-30T12:10:32-07:00 DBG Update existing file for harvesting: C:\TestRail\TestRailResults14.json, offset: 261
2018-03-30T12:10:32-07:00 DBG Resuming harvesting of file: C:\TestRail\TestRailResults14.json, offset: 261
2018-03-30T12:10:32-07:00 DBG Check file for harvesting: C:\TestRail\TestRailResults15.json
2018-03-30T12:10:32-07:00 DBG Update existing file for harvesting: C:\TestRail\TestRailResults15.json, offset: 267
2018-03-30T12:10:32-07:00 DBG Resuming harvesting of file: C:\TestRail\TestRailResults15.json, offset: 267
2018-03-30T12:10:32-07:00 DBG Set previous offset for file: C:\TestRail\TestRailResults14.json. Offset: 261
2018-03-30T12:10:32-07:00 DBG Setting offset for file: C:\TestRail\TestRailResults14.json. Offset: 261
2018-03-30T12:10:32-07:00 DBG Prospector states cleaned up. Before: 1, After: 1
2018-03-30T12:10:32-07:00 INFO Harvester started for file: C:\TestRail\TestRailResults14.json
2018-03-30T12:10:32-07:00 DBG Set previous offset for file: C:\TestRail\TestRailResults15.json. Offset: 267
2018-03-30T12:10:32-07:00 DBG Setting offset for file: C:\TestRail\TestRailResults15.json. Offset: 267
2018-03-30T12:10:32-07:00 DBG Prospector states cleaned up. Before: 1, After: 1
2018-03-30T12:10:32-07:00 ERR Error decoding JSON: EOF
2018-03-30T12:10:32-07:00 INFO Harvester started for file: C:\TestRail\TestRailResults15.json
2018-03-30T12:10:32-07:00 ERR Error decoding JSON: EOF
2018-03-30T12:10:32-07:00 DBG End of file reached: C:\TestRail\TestRailResults15.json; Backoff now.
2018-03-30T12:10:32-07:00 DBG End of file reached: C:\TestRail\TestRailResults14.json; Backoff now.
2018-03-30T12:10:33-07:00 DBG End of file reached: C:\TestRail\TestRailResults15.json; Backoff now.
2018-03-30T12:10:33-07:00 DBG End of file reached: C:\TestRail\TestRailResults14.json; Backoff now.
2018-03-30T12:10:35-07:00 DBG End of file reached: C:\TestRail\TestRailResults14.json; Backoff now.
2018-03-30T12:10:35-07:00 DBG End of file reached: C:\TestRail\TestRailResults15.json; Backoff now.
2018-03-30T12:10:37-07:00 DBG Flushing spooler because of timeout. Events flushed: 6
2018-03-30T12:10:37-07:00 DBG Publish: {
"@timestamp": "2018-03-30T19:10:32.917Z",
"beat": {
"hostname": "5CG722343K",
"name": "5CG722343K",
"version": "5.5.1"
},
"fields": {
"document_type": "testrailresults15"
},
"input_type": "log",
"json_error": "Error decoding JSON: EOF",
"offset": 269,
"source": "C:\TestRail\TestRailResults15.json",
"type": "log"
}
2018-03-30T12:10:37-07:00 DBG Publish: {
"@timestamp": "2018-03-30T19:10:32.916Z",
"beat": {
"hostname": "5CG722343K",
"name": "5CG722343K",
"version": "5.5.1"
},
"fields": {
"document_type": "testrailresults14"
},
"input_type": "log",
"json_error": "Error decoding JSON: EOF",
"offset": 263,
"source": "C:\TestRail\TestRailResults14.json",
"type": "log"
}

sunny1 · March 30, 2018, 7:13pm

Remaining log:

2018-03-30T12:10:37-07:00 DBG output worker: publish 2 events
2018-03-30T12:10:37-07:00 DBG ES Ping(url=http://localhost:9200, timeout=1m30s)
2018-03-30T12:10:37-07:00 DBG Ping status code: 200
2018-03-30T12:10:37-07:00 INFO Connected to Elasticsearch version 5.5.1
2018-03-30T12:10:37-07:00 INFO Trying to load template for client: http://localhost:9200
2018-03-30T12:10:37-07:00 DBG HEAD http://localhost:9200/_template/filebeat
2018-03-30T12:10:37-07:00 INFO Template already exists and will not be overwritten.
2018-03-30T12:10:37-07:00 DBG PublishEvents: 2 events have been published to elasticsearch in 60.006ms.
2018-03-30T12:10:37-07:00 DBG send completed
2018-03-30T12:10:37-07:00 DBG Events sent: 6
2018-03-30T12:10:37-07:00 DBG Processing 6 events
2018-03-30T12:10:37-07:00 DBG Registrar states cleaned up. Before: 10, After: 10
2018-03-30T12:10:37-07:00 DBG Write registry file: C:\ProgramData\filebeat\registry
2018-03-30T12:10:37-07:00 DBG Registry file updated. 10 states written.
2018-03-30T12:10:38-07:00 DBG Received svc stop/shutdown request
2018-03-30T12:10:38-07:00 INFO Stopping filebeat
2018-03-30T12:10:38-07:00 INFO Stopping Crawler
2018-03-30T12:10:38-07:00 INFO Stopping 2 prospectors
2018-03-30T12:10:38-07:00 INFO Prospector ticker stopped
2018-03-30T12:10:38-07:00 INFO Stopping Prospector: 10374277926638004162
2018-03-30T12:10:38-07:00 INFO Reader was closed: C:\TestRail\TestRailResults15.json. Closing.
2018-03-30T12:10:38-07:00 DBG Stopping harvester for file: C:\TestRail\TestRailResults15.json
2018-03-30T12:10:38-07:00 INFO Prospector channel stopped because beat is stopping.
2018-03-30T12:10:38-07:00 DBG Closing file: C:\TestRail\TestRailResults15.json
2018-03-30T12:10:38-07:00 INFO Prospector ticker stopped
2018-03-30T12:10:38-07:00 DBG Update state: C:\TestRail\TestRailResults15.json, offset: 269
2018-03-30T12:10:38-07:00 INFO Stopping Prospector: 3140866685714624168
2018-03-30T12:10:38-07:00 INFO Prospector channel stopped because beat is stopping.
2018-03-30T12:10:38-07:00 INFO Reader was closed: C:\TestRail\TestRailResults14.json. Closing.
2018-03-30T12:10:38-07:00 DBG Stopping harvester for file: C:\TestRail\TestRailResults14.json
2018-03-30T12:10:38-07:00 DBG Closing file: C:\TestRail\TestRailResults14.json
2018-03-30T12:10:38-07:00 DBG Update state: C:\TestRail\TestRailResults14.json, offset: 263
2018-03-30T12:10:38-07:00 INFO Crawler stopped
2018-03-30T12:10:38-07:00 INFO Stopping spooler
2018-03-30T12:10:38-07:00 DBG Spooler has stopped
2018-03-30T12:10:38-07:00 DBG Shutting down sync publisher
2018-03-30T12:10:38-07:00 INFO Stopping Registrar
2018-03-30T12:10:38-07:00 INFO Ending Registrar
2018-03-30T12:10:38-07:00 DBG Write registry file: C:\ProgramData\filebeat\registry
2018-03-30T12:10:38-07:00 DBG Registry file updated. 10 states written.
2018-03-30T12:10:38-07:00 INFO Total non-zero values: filebeat.harvester.closed=2 filebeat.harvester.started=2 libbeat.es.call_count.PublishEvents=1 libbeat.es.publish.read_bytes=823 libbeat.es.publish.write_bytes=1179 libbeat.es.published_and_acked_events=2 libbeat.publisher.published_events=2 publish.events=6 registrar.states.current=10 registrar.states.update=6 registrar.writes=2
2018-03-30T12:10:38-07:00 INFO Uptime: 5.9135913s
2018-03-30T12:10:38-07:00 INFO filebeat stopped.

sunny1 · March 30, 2018, 7:27pm

It worked. Thanks so much for your help.

sunny1 · March 30, 2018, 7:27pm

Really appreciate all your help/time.

pierhugues · March 30, 2018, 7:29pm

Glad it worked!

sunny1 · March 30, 2018, 7:30pm

Thanks much.

system · April 27, 2018, 7:30pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Filebeat Config Failure Beats filebeat	2	475	June 2, 2021
If condition in logstash filter - not working Logstash	7	1944	February 14, 2019
Not able to create /see any indexs in elasticsearch Beats filebeat	2	333	November 13, 2018
Filebeat with Logstash failed to create Index Beats filebeat	9	1305	May 28, 2021
Filebeat: Select index based on prospector Beats filebeat	2	2179	August 16, 2017

ERR failed to initialize elasticsearch plugin as output: missing condition

Related topics