Beats Data not enriching when using Logstash + Elastic output plugin

Hi Folks,
This is a fairly basic question so i'm sure i'm missing something straightforward.
We are using out of the box everything in a brand new POC environment.

I have created an enrichment policy and associated pipeline that works great when i run data through the ingest node locally in Elasticsearch (via dev console). *pipeline details below

Example of success with enrichment pipeline

POST metricbeat-7.6.2-2020.04.13/_update_by_query?pipeline=pipeline_appcmdb_host_application&wait_for_completion=false 

Works great! I can see our enrichment data fields populated based on the matching fields

POST metricbeat-7.6.2-2020.04.21/_doc?pipeline=pipeline_appcmdb_host_application 
{doc data here}

This also works just fine.

We have beats 7.6.x running and sending to Logstash. Logstash in turn sends to Elastic via the Elasticsearch Output Plugin (

I'm trying to put our logstash sent data through the same pipeline but we are not getting any enrichment. The tag associated with the pipeline is not being added either suggesting it isn't even trying the pipeline.

here is our Logstash pipeline definition which includes the Elasticsearch plugin with the target pipeline defined.

    # MonitorDevPipe-1 Logstash configuration 
    # Beats -> Logstash -> Elasticsearch pipeline.

    input {
      beats {
        port => 5044

    output {
      elasticsearch {
        hosts => ["http://gtselkdev01:9200"]
        index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    	pipeline => "pipeline_appcmdb_host_application"

Thanks for any help!

pipeline details

    PUT _ingest/pipeline/pipeline_appcmdb_host_application
      "description" : "adds application name to incoming beats data based on beats agent host name",
      "processors" : [
          "enrich": {
              "policy_name": "enrich_appcmdb_host_application",
              "field": "host.hostname",
              "target_field": "appcmdb_application",
              "ignore_missing": true,
              "tag": "enrich_appcmdb_host_application"

What does an event that is not getting enriched by the pipeline look like?

OK - so this was a simple solution. The Logstash config was not pointing to the right pipeline .CONF file. Everything worked when i pointed it to the right file. Amazing how that works

Anyhow thanks a bunch to our local Elastic sales engineer for spending time and identifying this one for me. Shout out at @Michael_Mulcahy.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.