Hi,
I would like to add ingest node for data from metricbeat. Unfortunately i have errors like that:
metricbeat logs
2019-07-03T10:58:40.182Z INFO elasticsearch/client.go:721 Connected to Elasticsearch version 6.6.1
2019-07-03T10:58:40.186Z INFO template/load.go:130 Template already exists and will not be overwritten.
2019-07-03T10:58:40.186Z INFO instance/beat.go:894 Template successfully loaded.
2019-07-03T10:58:40.186Z INFO pipeline/output.go:105 Connection to backoff(elasticsearch(http://elastic_pat:9200)) established
2019-07-03T10:58:40.187Z INFO [publish] pipeline/retry.go:189 retryer: send unwait-signal to consumer
2019-07-03T10:58:40.187Z INFO [publish] pipeline/retry.go:191 done
2019-07-03T10:58:40.188Z INFO [publish] pipeline/retry.go:166 retryer: send wait signal to consumer
2019-07-03T10:58:40.188Z INFO [publish] pipeline/retry.go:168 done
2019-07-03T10:58:41.627Z ERROR pipeline/output.go:121 Failed to publish events: temporary bulk send failure
2019-07-03T10:58:41.627Z INFO pipeline/output.go:95 Connecting to backoff(elasticsearch(http://elastic_pat:9200))
2019-07-03T10:58:41.627Z INFO [publish] pipeline/retry.go:189 retryer: send unwait-signal to consumer
2019-07-03T10:58:41.627Z INFO [publish] pipeline/retry.go:191 done
2019-07-03T10:58:41.627Z INFO [publish] pipeline/retry.go:166 retryer: send wait signal to consumer
2019-07-03T10:58:41.627Z INFO [publish] pipeline/retry.go:168 done
2019-07-03T10:58:41.628Z INFO elasticsearch/client.go:721 Connected to Elasticsearch version 6.6.1
Here is
elasticsearch.yml
cluster.name: "docker-cluster"
node.name: "docker-node"
node.master: true
node.data: true
network.host: 0.0.0.0
http.port: 9200
http.enabled: true
path.data: /usr/share/elasticsearch/data
path.logs: /var/log/elasticsearch
xpack.monitoring.collection.enabled: true
xpack.security.enabled: false
plugin.mandatory: ingest-attachment
Path to directory where to store the data (separate multiple locations by comma):
path.data: /var/data/elasticsearch
Path to log files:
path.logs: /var/log/elasticsearch
minimum_master_nodes need to be explicitly set when bound on a public IP
set to 1 to allow single node clusters
Details: https://github.com/elastic/elasticsearch/pull/17288
discovery.zen.minimum_master_nodes: 1
and
meatricbeat.yml
metricbeat.modules:
- module: system
metricsets:- cpu
- load
- core
- filesystem
- memory
cpu.metrics: [percentages, normalized_percentages]
period: 10s
monitoring:
enabled: true
elasticsearch:
hosts: ["http://elastic_pat:9200"]
output.elasticsearch:
hosts: ["http://elastic_pat:9200"]
pipeline: "metric_pipeline"
I created ingest node like that:
pipeline
PUT /_ingest/pipeline/metric_pipeline
{
"description": "metric",
"processors": [
{
"append": {
"if": "agent.system=='metricbeat'",
"field": "name_system",
"value": "xyz"
}
}]
}
Can anyone help me? i would really appreciate