# How to debug elasticsearch logstash ingestion pipelines

**URL:** <https://discuss.elastic.co/t/how-to-debug-elasticsearch-logstash-ingestion-pipelines/237618>\
**Category:** Elasticsearch\
**Created:** [June 18, 2020, 10:47am UTC](https://discuss.elastic.co/t/how-to-debug-elasticsearch-logstash-ingestion-pipelines/237618 "2020-06-18T10:47:38Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![genehunter29009](https://avatars.discourse-cdn.com/v4/letter/g/45deac/32.png) [@genehunter29009](https://discuss.elastic.co/u/genehunter29009)\
**Post date:** [June 18, 2020, 10:47am UTC](https://discuss.elastic.co/t/how-to-debug-elasticsearch-logstash-ingestion-pipelines/237618/1 "2020-06-18T10:47:38Z")

</div>

Trying to use the pipelines in a beats config on logstash. The issue is its not seeing the pipeline so the data is going straight into the index unprocessed. I am sure something is wrong with the code that builds the pipeline name pipeline =\> "%{[@metadata][pipeline]}" but I dont see any documentation on that. My pipelines are named

filebeat-7.7.0-iis-access-pipeline  
filebeat-7.7.1-iis-error-pipeline

Anyone have any suggestions on how I can find out what %{[@metadata][pipeline]} is pointing too or know why it is not working?

input {  
beats {  
port =\> 5100  
}  
}

output {  
if [@metadata][pipeline] {  
elasticsearch {  
hosts =\> ["[https://ipaddresshere:9200](https://ipaddresshere:9200)"]  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.ww}"  
pipeline =\> "%{[@metadata][pipeline]}"  
user =\> "elastic"  
password =\> "changeme"  
cacert =\> "/etc/logstash/certs/cacerts.pem"  
}  
}  
else {  
elasticsearch {  
hosts =\> ["[https://ipaddresshere:9200](https://ipaddresshere:9200)"]  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.ww}"  
user =\> "elastic"  
password =\> "changeme"  
cacert =\> "/etc/logstash/certs/cacerts.pem"  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![genehunter29009](https://avatars.discourse-cdn.com/v4/letter/g/45deac/32.png) [@genehunter29009](https://discuss.elastic.co/u/genehunter29009)\
**Post date:** [June 22, 2020, 10:28am UTC](https://discuss.elastic.co/t/how-to-debug-elasticsearch-logstash-ingestion-pipelines/237618/2 "2020-06-22T10:28:57Z")

</div>

I figured out there was no pipeline parameter being passed. I assume you have to create your own. I did get this code to work with 7.7

input {  
beats {  
port =\> 5100  
}  
}  
output {  
if [agent][type] == "filebeat" {  
if [fileset][name] == "access" {  
elasticsearch {  
hosts =\> ["[https://ipaddresshere:9200](https://ipaddresshere:9200)"]  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.ww}"  
pipeline =\> "%{[@metadata][beat]}-%{[@metadata][version]}-iis-access-pipeline"  
user =\> "elastic"  
password =\> "changeme"  
cacert =\> "/etc/logstash/certs/cacerts.pem"  
}  
}  
}  
if [agent][type] == "filebeat" {  
if [fileset][name] == "error" {  
elasticsearch {  
hosts =\> ["[https://ipaddresshere:9200](https://ipaddresshere:9200)"]  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.ww}"  
pipeline =\> "%{[@metadata][beat]}-%{[@metadata][version]}-iis-error-pipeline"  
user =\> "elastic"  
password =\> "changeme"  
cacert =\> "/etc/logstash/certs/cacerts.pem"  
}  
}  
}  
if [agent][type] != "filebeat" {  
elasticsearch {  
hosts =\> ["[https://10.29.144.38:9200](https://10.29.144.38:9200)"]  
manage\_template =\> false  
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.ww}"  
user =\> "elastic"  
password =\> "changeme"  
cacert =\> "/etc/logstash/certs/cacerts.pem"  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 20, 2020, 10:36am UTC](https://discuss.elastic.co/t/how-to-debug-elasticsearch-logstash-ingestion-pipelines/237618/3 "2020-07-20T10:36:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.