# Cannot configure grok pipeline to processors in the elastic agent

**URL:** https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933
**Category:** Elasticsearch
**Tags:** fleet
**Created:** [August 30, 2023, 1:29am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933 "2023-08-30T01:29:55Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![vanhaiit90](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vanhaiit90/32/119189_2.png) [@vanhaiit90](https://discuss.elastic.co/u/vanhaiit90)
#### Post date: [August 30, 2023, 1:29am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933/1 "2023-08-30T01:29:55Z")

</div>

Hi everyone!

I completed and successful configure grok debuger log format of haproxy.

 ![ảnh](https://us1.discourse-cdn.com/elastic/original/3X/5/2/52437e4a1dc286708ebfa01017380dcc5d222efc.png)

And next I was added code grok pattens to pipeline

 ![ảnh](https://us1.discourse-cdn.com/elastic/original/3X/4/3/43e4df2a0508de9c78ada9b6dc0c789901542624.png)

Finally step I have added pipeline to processors in the elastic agent but seem it is wrong.

 ![ảnh](https://us1.discourse-cdn.com/elastic/original/3X/f/4/f46133d7f8e6baad72d31c39c449447e998a9dd4.png)

I was tried configure again but is is wrong. Everyone help me to solve this problem, thank you very much

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [August 30, 2023, 2:21am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933/2 "2023-08-30T02:21:11Z")

</div>

Hi @vanhaiit90

Exactly which integration did you use? The HAproxy?

I think You are adding the pipeline in the wrong place.... Only processors go there

See [this tutorial](https://www.elastic.co/guide/en/fleet/8.9/data-streams-pipeline-tutorial.html)

If you scroll to the bottom there is a custom pipeline...

Add Custom Pipeline

You need to add you grok to that pipeline.

 ![Screenshot 2023-08-29 at 7.32.31 PM](https://us1.discourse-cdn.com/elastic/original/3X/0/9/0911ae6c82ea4f430765ba9be425dcfe20927954.png)

---

<div class="post-metadata">

### Author: ![vanhaiit90](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vanhaiit90/32/119189_2.png) [@vanhaiit90](https://discuss.elastic.co/u/vanhaiit90)
#### Post date: [August 30, 2023, 3:00am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933/3 "2023-08-30T03:00:51Z")

</div>

no I do not. I mean I finished configuring the pipeline and then I inserted the text pipeline: \*\*

> logs-haproxy.log@custom

\*\* into the proccessors on elastic agent and it was wrong

 ![ảnh](https://us1.discourse-cdn.com/elastic/original/3X/f/4/f46133d7f8e6baad72d31c39c449447e998a9dd4.png)

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [August 30, 2023, 4:54am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933/4 "2023-08-30T04:54:31Z")

</div>

Where you are adding the pipeline in the processor is incorrect...

Follow the tutorial.

When you add the custom pipeline where I showed it will automatically be called.

 ![Screenshot 2023-08-29 at 10.12.21 PM](https://us1.discourse-cdn.com/elastic/original/3X/4/0/401e6d49f8803dbfc8cc91188bd5c5eafd4a72f9.png)

---

<div class="post-metadata">

### Author: ![vanhaiit90](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vanhaiit90/32/119189_2.png) [@vanhaiit90](https://discuss.elastic.co/u/vanhaiit90)
#### Post date: [September 5, 2023, 2:06am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933/5 "2023-09-05T02:06:11Z")

</div>

Hi Stephenb!

I have configured it exactly according to the steps you showed. However I seen it is seem it not true.

 ![ảnh](https://us1.discourse-cdn.com/elastic/original/3X/6/f/6fdbc76ce0414096c91e091ee8992d4981176d5c.png)

The custom pipelines for grok haproxy was configured

 ![ảnh](https://us1.discourse-cdn.com/elastic/original/3X/4/9/49254c82a3b239a2c189705f6d68c92b688d0df3.png)

 ![ảnh](https://us1.discourse-cdn.com/elastic/original/3X/0/2/02f6024a35531a598370942ded3371b66620e2bf.png)

> %{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:hostname} %{WORD:process}[%{NUMBER:pid}]: %{IP:client\_ip}:%{NUMBER:client\_port} [%{NOTSPACE:haproxy.request\_date}] %{WORD:frontend} %{WORD:backend}/%{WORD:server} %{NUMBER:time\_queue}/%{NUMBER:time\_backend\_connect}/%{NUMBER:time\_duration}/%{NUMBER:time\_active} %{NUMBER:requests} %{NUMBER:bytes\_read} %{NUMBER:frontend\_errors} - - --%{NOTSPACE:backend\_queue} %{NOTSPACE:backend\_connect}/%{NOTSPACE:server\_queue}/%{NOTSPACE:server\_response}/%{NOTSPACE:server\_duration} %{NOTSPACE:session\_status} "%{WORD:http\_method} %{URIPATHPARAM:request\_path} HTTP/%{NUMBER:http\_version}"

Please help me check the grok patterns to see if they are correct

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [September 5, 2023, 2:17am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933/6 "2023-09-05T02:17:56Z")

</div>

Hi @vanhaiit90

You will need to provide :

several sample log entries...

And the complete custom ingest pipeline not just the grok pattern.

Also since you are using an Integration it will try to run the default pipeline first so are you sure that error message is not from the default pipeline which run first?

Did you see if the `event.original` is still available in the document?

Are you sure your pipeline is not working are the fields there?

You can add a simple set processor in your custom pipeline to make sure it is run...

Screenshots are very hard to debug... Actual text results are much better.

You can also go Kibana - Stack Management - Ingest Pipeline

Pick your custom pipeline... And edit it and then test with existing document with the pipeline tester.. you will need the index name and \_id which you can get from Discover

 ![Screenshot 2023-09-04 at 7.39.57 PM](https://us1.discourse-cdn.com/elastic/original/3X/1/b/1ba1d2a9bd2ea42ce27ec0e2f0859e4aa2b977e4.png)  
 ![Screenshot 2023-09-04 at 7.41.17 PM](https://us1.discourse-cdn.com/elastic/original/3X/a/1/a10abd38d591b13a36b4dc1af4de6969242a60b7.png)

---

<div class="post-metadata">

### Author: ![vanhaiit90](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vanhaiit90/32/119189_2.png) [@vanhaiit90](https://discuss.elastic.co/u/vanhaiit90)
#### Post date: [September 7, 2023, 4:49am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933/7 "2023-09-07T04:49:56Z")

</div>

thank @stephenb very much I have resolved it

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [September 7, 2023, 5:21am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933/8 "2023-09-07T05:21:25Z")

</div>

Hi @vanhaiit90

What was the issue and resolution? it can help another user if you explain what the issue was and how you solved it ... as a good member of the community 🙂

---

<div class="post-metadata">

### Author: ![vanhaiit90](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vanhaiit90/32/119189_2.png) [@vanhaiit90](https://discuss.elastic.co/u/vanhaiit90)
#### Post date: [September 7, 2023, 6:45am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933/9 "2023-09-07T06:45:03Z")

</div>

Because someone has configured it incorrectly according to the original config file of haproxy.cfg. And I was able to promptly backup and configure the haproxy.cfg file to match the fields corresponding to elasticsearch

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 5, 2023, 6:45am UTC](https://discuss.elastic.co/t/cannot-configure-grok-pipeline-to-processors-in-the-elastic-agent/341933/10 "2023-10-05T06:45:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
