Metricbeat problem with Ingest Node

Elastic Stack Beats
1

Hi,
I would like to add ingest node for data from metricbeat. Unfortunately i have errors like that:

metricbeat logs

2019-07-03T10:58:40.182Z INFO elasticsearch/client.go:721 Connected to Elasticsearch version 6.6.1
2019-07-03T10:58:40.186Z INFO template/load.go:130 Template already exists and will not be overwritten.
2019-07-03T10:58:40.186Z INFO instance/beat.go:894 Template successfully loaded.
2019-07-03T10:58:40.186Z INFO pipeline/output.go:105 Connection to backoff(elasticsearch(http://elastic_pat:9200)) established
2019-07-03T10:58:40.187Z INFO [publish] pipeline/retry.go:189 retryer: send unwait-signal to consumer
2019-07-03T10:58:40.187Z INFO [publish] pipeline/retry.go:191 done
2019-07-03T10:58:40.188Z INFO [publish] pipeline/retry.go:166 retryer: send wait signal to consumer
2019-07-03T10:58:40.188Z INFO [publish] pipeline/retry.go:168 done
2019-07-03T10:58:41.627Z ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure
2019-07-03T10:58:41.627Z INFO pipeline/output.go:95 Connecting to backoff(elasticsearch(http://elastic_pat:9200))
2019-07-03T10:58:41.627Z INFO [publish] pipeline/retry.go:189 retryer: send unwait-signal to consumer
2019-07-03T10:58:41.627Z INFO [publish] pipeline/retry.go:191 done
2019-07-03T10:58:41.627Z INFO [publish] pipeline/retry.go:166 retryer: send wait signal to consumer
2019-07-03T10:58:41.627Z INFO [publish] pipeline/retry.go:168 done
2019-07-03T10:58:41.628Z INFO elasticsearch/client.go:721 Connected to Elasticsearch version 6.6.1

Here is

elasticsearch.yml

cluster.name: "docker-cluster"
node.name: "docker-node"
node.master: true
node.data: true
network.host: 0.0.0.0
http.port: 9200
http.enabled: true

path.data: /usr/share/elasticsearch/data

path.logs: /var/log/elasticsearch

xpack.monitoring.collection.enabled: true
xpack.security.enabled: false

plugin.mandatory: ingest-attachment

Path to directory where to store the data (separate multiple locations by comma):

path.data: /var/data/elasticsearch

Path to log files:

path.logs: /var/log/elasticsearch

minimum_master_nodes need to be explicitly set when bound on a public IP

set to 1 to allow single node clusters

Details: https://github.com/elastic/elasticsearch/pull/17288

discovery.zen.minimum_master_nodes: 1

and

meatricbeat.yml

metricbeat.modules:

  • module: system
    metricsets:
    • cpu
    • load
    • core
    • filesystem
    • memory
      cpu.metrics: [percentages, normalized_percentages]
      period: 10s

monitoring:
enabled: true
elasticsearch:
hosts: ["http://elastic_pat:9200"]

output.elasticsearch:
hosts: ["http://elastic_pat:9200"]
pipeline: "metric_pipeline"

I created ingest node like that:

pipeline

PUT /_ingest/pipeline/metric_pipeline
{
"description": "metric",
"processors": [
{
"append": {
"if": "agent.system=='metricbeat'",
"field": "name_system",
"value": "xyz"
}
}]

}

Can anyone help me? i would really appreciate

2

problem solved, i had to add ctx prefix, example:
"if": "ctx.agent.system=='metricbeat'"

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.