Metricbeat is unable to send data to elastic search

Elastic Stack Beats
1

Hello,

I have setup Elasticsearch and Kibana in a VM and then started setting up metricbeat. Elasticsearch & Kibana is working fine. But I am unable to send metricbeat data to elastic search.

Once I start running metricbeat I am getting below Error:
<
2019-01-25T19:13:28.634+0530 INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://10.81.86.26:9200)) with 24 reconnect attempt(s)
2019-01-25T19:13:28.634+0530 INFO [publish] pipeline/retry.go:189 retryer: send unwait-signal to consumer
2019-01-25T19:13:28.634+0530 INFO [publish] pipeline/retry.go:191 done
2019-01-25T19:13:28.634+0530 INFO [publish] pipeline/retry.go:166 retryer: send wait signal to consumer
2019-01-25T19:13:28.634+0530 INFO [publish] pipeline/retry.go:168 done
2019-01-25T19:13:54.119+0530 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":1810,"time":{"ms":4}},"total":{"ticks":4000,"time":{"ms":8},"value":4000},"user":{"ticks":2190,"time":{"ms":4}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":5},"info":{"ephemeral_id":"4780c44a-018b-415b-a8b0-e87aff9f8846","uptime":{"ms":1173056}},"memstats":{"gc_next":46086736,"memory_alloc":23581632,"memory_total":428077984}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":4117,"retry":50}}},"system":{"load":{"1":0,"15":0.05,"5":0.01,"norm":{"1":0,"15":0.0125,"5":0.0025}}}}}}
2019-01-25T19:14:24.121+0530 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":1820,"time":{"ms":2}},"total":{"ticks":4020,"time":{"ms":7},"value":4020},"user":{"ticks":2200,"time":{"ms":5}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":5},"info":{"ephemeral_id":"4780c44a-018b-415b-a8b0-e87aff9f8846","uptime":{"ms":1203056}},"memstats":{"gc_next":46086736,"memory_alloc":23839664,"memory_total":428336016}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":4117}}},"system":{"load":{"1":0,"15":0.05,"5":0.01,"norm":{"1":0,"15":0.0125,"5":0.0025}}}}}}
2019-01-25T19:14:28.635+0530 ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://10.81.86.26:9200)): Get http://10.81.86.26:9200: dial tcp 10.81.86.26:9200: connect: connection refused
2019-01-25T19:14:28.635+0530 INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://10.81.86.26:9200)) with 25 reconnect attempt(s)
2019-01-25T19:14:28.635+0530 INFO [publish] pipeline/retry.go:189 retryer: send unwait-signal to consumer
2019-01-25T19:14:28.635+0530 INFO [publish] pipeline/retry.go:191 done
2019-01-25T19:14:28.635+0530 INFO [publish] pipeline/retry.go:166 retryer: send wait signal to consumer
2019-01-25T19:14:28.635+0530 INFO [publish] pipeline/retry.go:168 done
2019-01-25T19:14:54.121+0530 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s
/>

My metricbeat.yml file look like
<
#========================== Modules configuration ============================
metricbeat.modules:

#------------------------------- System Module -------------------------------

  • module: system
    metricsets:

    • cpu # CPU usage
    • load # CPU load averages
    • memory # Memory usage
    • network # Network IO
    • process # Per process metrics
    • process_summary # Process summary
    • uptime # System Uptime
      #- core # Per CPU core usage
      #- diskio # Disk IO
      #- filesystem # File system usage for each mountpoint
      #- fsstat # File system summary metrics
      #- raid # Raid
      #- socket # Sockets and connection info (linux only)
      enabled: true
      period: 10s
      processes: ['.*']

    Configure the metric types that are included by these metricsets.

    cpu.metrics: ["percentages"] # The other available options are normalized_percentages and ticks.
    core.metrics: ["percentages"] # The other available option is ticks.

output.elasticsearch:

Array of hosts to connect to.

hosts: ["http://10.81.86.26:9200"]

/>

Pls help me on this issue.

Thanks,
Chethan

2

From the same host as the one running Metricbeat, could you please run the following?

curl http://10.81.86.26:9200
3

I am getting below message:

curl http://10.81.86.26:9200
curl: (7) Failed connect to 10.81.86.26:9200; Connection refused

I ran same command on Elasticsearch & Kibana host, I am getting same error message as above.

Thanks,
Chethan

4

That means your Metricbeat host cannot talk to the Elasticsearch host (10.81.86.26) on TCP port 9200. I would check your network configuration and see if there's something blocking access, perhaps a firewall rule or VM networking configuration.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.