Something is wrong with pipeline

I have Stack without Logstash. But I need to correct some fields in kibana (timestamp for example). I try to use pipeline for this purpose. This is my config file:
filebeat.yml


filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.log

multiline.pattern: '^\['
multiline.negate: true
multiline.match: after


output.elasticsearch:
  hosts: ["localhost:9200"]
  pipeline: "rename_hostname"

setup.template.overwrite: true
setup.dashboards.overwrite: true
output.elasticsearch.index: "docker-one"
setup.template.name: "docker"
setup.template.pattern: "docker-*"
setup.dashboards.index: "docker-*"

setup.kibana:
  host: "localhost:5601"

And I execute this command in devtools in Kibana:

 PUT _ingest/pipeline/rename_hostname
{
  "processors": [
    {
      "rename": {
        "field": "hostname",
        "target_field": "host",
        "ignore_missing": true
      }
    }
  ]
}

... and nothing changes in kibana interface. What I did wrong?

Are there any errors in your Filebeat logs? Such as a mapping error for host? You might need to customize your index template if you intent to make host a keyword because in the latest version host is mapped as an object. It does have a host.name field though.

Filebeat working fine and I dont get any errors. It does not matter for me what type of processors I have used. For example if my pipeline like:

PUT _ingest/pipeline/examplepipe
{
  "description" : "Change smth",
  "processors": [
    ]
}

then how I can understand: are me log files go through this pipeline? Because other processors seems do not work.

Elasticsearch provides stats about each pipeline that you can use to check if any events are passing through.

GET _nodes/stats/ingest

Your config appears correct to me. But how about trying to put pipeline as part of the filebeat input config instead. reference

filebeat.inputs:
- type: log
  paths:
    - /var/log/*.log
  pipeline: my_pipeline

My problems seems to be solved. But I can't describe what exactly I was doing wrong. Thanks for some helpfull advices.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.