Failed to upload data in Elasticsearch

Elastic Stack Beats
1

Hi, i have this responds when i start filebeat.

metrics.go:34: INFO No non-zero metrics in the last 30s
metrics.go:34: INFO No non-zero metrics in the last 30s
metrics.go:34: INFO No non-zero metrics in the last 30s

My filebeat.yml config

filebeat.prospectors:
-input_type: log
paths:

Thanks for replay

2

Are you generating new data in /mnt/analisilog/www.access_log? If there is nothing new, it won't send anything. Notice that's INFO level - nothing is wrong. That is, unless new logs ARE being generated.

3

Hi @rachelkelly, Before run filebeat i use this command:

awslogs get apache www.cur.com.access_log_* --start='7m ago' --end='5m ago' --aws-region eu-west-1 --profile UTAwsLogsRO > /mnt/analisilog/www.access_log

That command populate my file www.access_log. So i just tried run that command and after run filebeat but the situation doesn't change.
This is the full message when start filebeat:

2017/07/13 08:03:47.553189 beat.go:285: INFO Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]
2017/07/13 08:03:47.553238 beat.go:186: INFO Setup Beat: filebeat; Version: 5.4.3
2017/07/13 08:03:47.553350 output.go:258: INFO Loading template enabled. Reading template file: /etc/filebeat/filebeat.template.json
2017/07/13 08:03:47.553457 metrics.go:23: INFO Metrics logging every 30s
2017/07/13 08:03:47.553491 output.go:269: INFO Loading template enabled for Elasticsearch 2.x. Reading template file: /etc/filebeat/filebeat.template-es2x.json
2017/07/13 08:03:47.553999 output.go:281: INFO Loading template enabled for Elasticsearch 6.x. Reading template file: /etc/filebeat/filebeat.template-es6x.json
2017/07/13 08:03:47.554569 client.go:123: INFO Elasticsearch url: https://search-analisilog-k2ymrubrfy.eu-west-1.es.amazonaws.com:443
2017/07/13 08:03:47.554620 outputs.go:108: INFO Activated elasticsearch as output plugin.
2017/07/13 08:03:47.554711 publish.go:295: INFO Publisher name: wind-tools
2017/07/13 08:03:47.554817 async.go:63: INFO Flush Interval set to: 1s
2017/07/13 08:03:47.554836 async.go:64: INFO Max Bulk Size set to: 50
2017/07/13 08:03:47.555018 beat.go:221: INFO filebeat start running.
2017/07/13 08:03:47.555104 registrar.go:85: INFO Registry file set to: /var/lib/filebeat/registry
2017/07/13 08:03:47.555181 registrar.go:106: INFO Loading registrar data from /var/lib/filebeat/registry
2017/07/13 08:03:47.555324 registrar.go:123: INFO States Loaded from registrar: 1
2017/07/13 08:03:47.555355 crawler.go:38: INFO Loading Prospectors: 1
2017/07/13 08:03:47.555367 registrar.go:236: INFO Starting Registrar
2017/07/13 08:03:47.555490 prospector_log.go:65: INFO Prospector with previous states loaded: 1
2017/07/13 08:03:47.555571 prospector.go:124: INFO Starting prospector of type: log; id: 18068218180201798470
2017/07/13 08:03:47.555610 crawler.go:58: INFO Loading and starting Prospectors completed. Enabled prospectors: 1
2017/07/13 08:03:47.555647 spooler.go:63: INFO Starting spooler: spool_size: 2048; idle_timeout: 5s
2017/07/13 08:03:47.555467 sync.go:41: INFO Start sending events to output
2017/07/13 08:04:17.553777 metrics.go:39: INFO Non-zero metrics in the last 30s: publish.events=1 registrar.states.current=1 registrar.states.update=1 registrar.writes=1

Thanks for replay

4

The log is incomplete. Is there a failure message coming up? With incomplete logs my guess: filebeat can not properly connect to your elasticsearch host.

5

Note that it's telling you the metrics it's sending - "non-zero metrics" here means non-empty metrics, as far as I can see. Here, it's not telling you "sending empty," it's telling you it's sending SOMEthing. You may be able to query elasticsearch for the contents of your access_log.

6

metrics don't say anything was shipped.

7

I don't understand where is the problem

8

Do you have a more complete log? The log is somewhat 'incomplete' (have filebeat run for a little longer). In the log provided is no indicator wether data has been shipped or failed.

Also enable debug, to get some more details? Have you checked the log for errors?

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.