# Double processor in ingest pipeline

**URL:** <https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768>\
**Category:** Elastic Agent\
**Created:** [December 10, 2024, 2:22pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768 "2024-12-10T14:22:05Z")\
**Posts on this page:** 11\
**Page:** 2

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 15, 2024, 2:20pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/21 "2024-12-15T14:20:52Z")

</div>

Ahh good catch...

The second solution with the `grok` does not do that.. pretty sure it is what you asked for ... Did you try that?

You might need to adjust if does not work perfectly... escape characters etc.. I ran it against your sample and it works... But the field `k8.pod.labels` appears more than once I think the way I have it it will ignore the one with the `{"app".... ` etc

---

<div class="post-metadata">

**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\
**Post date:** [December 16, 2024, 8:25am UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/22 "2024-12-16T08:25:13Z")

</div>

Hi,  
It seems to work correctly even if k8.pod.labels appears twice with different formats.  
I have to study the grok processor that seems very powerful!  
Thanks a lot!

---

<div class="post-metadata">

**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\
**Post date:** [December 16, 2024, 3:46pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/23 "2024-12-16T15:46:44Z")

</div>

ehm...  
is it possible that POST /\_ingest/pipeline/logs-trend\_micro\_vision\_one.detection@custom/\_simulate?verbose=true  
does not report an error for the 2 types of expected docs but that the data stream logs-trend\_micro\_vision\_one.detection-default is no longer populated? How can I investigate the error?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 16, 2024, 4:32pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/24 "2024-12-16T16:32:54Z")

</div>

> [@Cristina\_Marletta\_Li](#):
>
> ehm...  
> is it possible that POST /\_ingest/pipeline/logs-trend\_micro\_vision\_one.detection@custom/\_simulate?verbose=true  
> does not report an error for the 2 types of expected docs but that the data stream logs-trend\_micro\_vision\_one.detection-default is no longer populated? How can I investigate the error?

Apologies I am not clear what you are asking...

I am not sure what 2 types of expected documents

`_simulate` makes sure

1. the Ingest Pipeline works  
but
2. does not write the document to an index... There can be mapping errors when the document is being written and thus can be rekected

If you want to test 2) and make sure the document can actually be written to the datastream

You can POST an actual document to the datastream ...  
if there is an error it will show up.

To do this... put the `event.original` content in the `message` field you will also need to change the `@timestamp` I also add a tag `test_document` so I can find it easily.

It will either the document will be written or there will be an error.

```auto
POST logs-trend_micro_vision_one.detection-default/_doc
{
  		"tags": [
			"preserve_original_event",
			"preserve_duplicate_custom_fields",
			"forwarded",
			"trend_micro_vision_one-detection",
			"test_document"
		],
  		"@timestamp": "2024-12-11T17:09:59.000Z",
		"message": "{\"act\":[\"log\"],\"clusterId\":\" *************-2okw77rZ7bD1Qk0q9bLfTRAk2VR\",\"clusterName\":\"************* \",\"clusterType\":\"k8s\",\"containerId\":\"7ad0e152f9b6dd132fdc0d6ec3f4450499dfb476dbaad1dbd492d8f07eae1efe\",\"containerImage\":\" ***********.jfrog.io/*********** /tomcat:9.0.73-***\",\"containerImageDigest\":\"sha256:f9859cbc865035a55922648dbe394a34b36afc238399e561760d3dad86516563\",\"containerName\":\"tomcat-container\",\"customTags\":[\"mitre_command_and_control\",\"process\"],\"detectionType\":\"process\",\"endpointHostName\":\"****.eu-south-1.compute.internal\",\"eventId\":\"100119\",\"eventName\":\"SECURITY_RISK_DETECTION\",\"eventSourceType\":3,\"eventTime\":1733936999000,\"eventTimeDT\":\"2024-12-11T17:09:59+00:00\",\"groupId\":\" *************-2okw77rZ7bD1Qk0q9bLfTRAk2VR\",\"k8sNamespace\":\"sic-iam\",\"k8sPodId\":\"b777d8d9-441e-46ab-b34f-dac23b316636\",\"k8sPodName\":\"pnt-test-*** -0\",\"logKey\":\"e99c5cff-18f2-2e55-1b2b-ac77279777b5\",\"logReceivedTime\":\"1733937090989\",\"parentCmd\":\"bash ./new-entrypoint.sh\",\"parentName\":\"bash\",\"parentPid\":37520,\"pname\":\"Vision One Container Security\",\"policyId\":\"LogOnlyPolicy-2ZQuYzNIRyA8rb5AQo7wyR4ncvq\",\"policyName\":\"LogOnlyPolicy\",\"processCmd\":\"curl --head --location --connect-timeout 5 --max-time 5 --write-out %{http_code} --silent --output /dev/null http://localhost:8080/ ***\",\"processName\":\"curl\",\"processPid\":1534304,\"productCode\":\"scs\",\"pver\":\"2021-12-01T00:00:00.0000000Z\",\"rawDataStr\":\"{\\\"clusterID\\\":\\\"************* -2okw77rZ7bD1Qk0q9bLfTRAk2VR\\\",\\\"clusterName\\\":\\\" *************\\\",\\\"customerID\\\":\\\"7c49c1e2-f12c-4f97-9a67-fb2f44ea87a7\\\",\\\"details\\\":{\\\"container.id\\\":\\\"7ad0e152f9b6dd132fdc0d6ec3f4450499dfb476dbaad1dbd492d8f07eae1efe\\\",\\\"container.image.digest\\\":\\\"sha256:f9859cbc865035a55922648dbe394a34b36afc238399e561760d3dad86516563\\\",\\\"container.image.repository\\\":\\\"**** viva.jfrog.io/ ********/tomcat\\\",\\\"container.image.tag\\\":\\\"9.0.73-*** \\\",\\\"container.name\\\":\\\"tomcat-container\\\",\\\"evt.arg.exe\\\":\\\"curl\\\",\\\"evt.arg.filename\\\":\\\"\\u003cNA\\u003e\\\",\\\"evt.arg.mode\\\":\\\"\\u003cNA\\u003e\\\",\\\"evt.arg.name\\\":\\\"\\u003cNA\\u003e\\\",\\\"evt.category\\\":\\\"process\\\",\\\"evt.dir\\\":\\\"\\u003c\\\",\\\"evt.num\\\":\\\"8267962726\\\",\\\"evt.rawtime\\\":\\\"1733936999010100132\\\",\\\"evt.time\\\":\\\"17:09:59.010100132\\\",\\\"evt.type\\\":\\\"execve\\\",\\\"k8s.ns.name\\\":\\\"sic-iam\\\",\\\"k8s.pod.id\\\":\\\"b777d8d9-441e-46ab-b34f-dac23b316636\\\",\\\"k8s.pod.labels\\\":\\\"app:pnt-test- ***,controller-revision-hash:pnt-test-*** -7668f9d96d,statefulset.kubernetes.io/pod-name:pnt-test- ***-0\\\",\\\"k8s.pod.name\\\":\\\"pnt-test-*** -0\\\",\\\"proc.args\\\":\\\"--head --location --connect-timeout 5 --max-time 5 --write-out %{http_code} --silent --output /dev/null http://localhost:8080/ ***\\\",\\\"proc.cmdline\\\":\\\"curl --head --location --connect-timeout 5 --max-time 5 --write-out %{http_code} --silent --output /dev/null http://localhost:8080/*** \\\",\\\"proc.exe\\\":\\\"curl\\\",\\\"proc.exeline\\\":\\\"curl --head --location --connect-timeout 5 --max-time 5 --write-out %{http_code} --silent --output /dev/null http://localhost:8080/ ***\\\",\\\"proc.exepath\\\":\\\"/usr/bin/curl\\\",\\\"proc.name\\\":\\\"curl\\\",\\\"proc.pcmdline\\\":\\\"bash ./new-entrypoint.sh\\\",\\\"proc.pid\\\":\\\"1534304\\\",\\\"proc.pid.ts\\\":\\\"1733936999008820540\\\",\\\"proc.pname\\\":\\\"bash\\\",\\\"proc.ppid\\\":\\\"37520\\\",\\\"proc.ppid.ts\\\":\\\"1733427545111053154\\\",\\\"user.loginuid\\\":\\\"-1\\\",\\\"user.name\\\":\\\"root\\\"},\\\"hostname\\\":\\\"****.eu-south-1.compute.internal\\\",\\\"id\\\":\\\"2q50NZGekOuxLvx2iLCOQYHouki\\\",\\\"k8s.ns.name\\\":\\\"sic-iam\\\",\\\"k8s.pod.labels\\\":{\\\"app\\\":\\\"pnt-test- ***\\\",\\\"controller-revision-hash\\\":\\\"pnt-test-*** -7668f9d96d\\\",\\\"statefulset.kubernetes.io/pod-name\\\":\\\"pnt-test- ***-0\\\"},\\\"k8s.pod.name\\\":\\\"pnt-test-*** -0\\\",\\\"mitigation\\\":\\\"log\\\",\\\"name\\\":\\\"(T1105)Launch Ingress Remote File Copy Tools in Container\\\",\\\"orchestration\\\":\\\"k8s\\\",\\\"policyID\\\":\\\"LogOnlyPolicy-2ZQuYzNIRyA8rb5AQo7wyR4ncvq\\\",\\\"policyName\\\":\\\"LogOnlyPolicy\\\",\\\"productCode\\\":\\\"scs\\\",\\\"ruleID\\\":\\\"TM-00000049\\\",\\\"rulesets\\\":[{\\\"id\\\":\\\"LogOnlyRuleset-2ZQuYrP1rWUXTWC8ln6HQRni7th\\\",\\\"name\\\":\\\"LogOnlyRuleset\\\"}],\\\"severity\\\":\\\"notice\\\",\\\"tags\\\":[\\\"mitre_command_and_control\\\",\\\"process\\\"],\\\"timestamp\\\":\\\"2024-12-11T17:09:59.010100Z\\\",\\\"type\\\":\\\"syscall\\\",\\\"version\\\":\\\"2021-12-01\\\"}\",\"rt_utc\":\"2024-12-11T17:09:59.0000000Z\",\"ruleIdStr\":\"TM-00000049\",\"ruleName\":\"(T1105)Launch Ingress Remote File Copy Tools in Container\",\"ruleSetId\":\"LogOnlyRuleset-2ZQuYrP1rWUXTWC8ln6HQRni7th\",\"ruleSetName\":\"LogOnlyRuleset\",\"searchDL\":\"DDL\",\"severity\":4,\"sourceType\":\"syscall\",\"uuid\":\"319bf2a2-02c9-4e24-b927-1b78725d7c94\"}"
}

```

---

<div class="post-metadata">

**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\
**Post date:** [December 16, 2024, 5:50pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/25 "2024-12-16T17:50:01Z")

</div>

But no log?  
I don't want to change the index

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 16, 2024, 6:58pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/26 "2024-12-16T18:58:05Z")

</div>

@Cristina_Marletta_Li

Sure you can turn up the agent logging and elasticsearch logging and hunt through them to find the mapping error if that is what is....

or just post to a `test` namespace and that will put in a different data stream that you can then clean up and not change the current index.

```auto
POST logs-trend_micro_vision_one.detection-test/_doc
...........................................^^^^

```

That will put the document in a separate datastream  
`logs-trend_micro_vision_one.detection-test`  
and backing index  
`.ds-logs-trend_micro_vision_one.detection-test-2024.12.16-000001` (or similar)

which you can clean up... and not change the current / `default` datastream and index. It will still run all the ingest pipelines and use the correct mappings.

Otherwise, yes you will need to turn up logging on the Elasticsearch cluster and the Agent logging and dig through them to find the error.

---

<div class="post-metadata">

**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\
**Post date:** [December 17, 2024, 2:15pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/27 "2024-12-17T14:15:35Z")

</div>

Ok Stephenb,  
I have to practice.  
The grok processor is very very interesting.  
Very very interesting the grok processor.  
Where can I find the keywords of the patterns (and their description): DATA, IP, WORD etc.?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 17, 2024, 2:29pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/28 "2024-12-17T14:29:25Z")

</div>

Docs

> **[Grok processor | Elasticsearch Guide \[8.17\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/grok-processor.html)**

Patterns

> **[elasticsearch/libs/grok/src/main/resources/patterns/legacy at 8.17 ·...](https://github.com/elastic/elasticsearch/tree/8.17/libs/grok/src/main/resources/patterns/legacy)**
>
> Free and Open Source, Distributed, RESTful Search Engine - elastic/elasticsearch

Helpfully tool  
[https://grokconstructor.appspot.com/do/construction](https://grokconstructor.appspot.com/do/construction)

It is just Regex under the covers...

---

<div class="post-metadata">

**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\
**Post date:** [December 17, 2024, 3:18pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/29 "2024-12-17T15:18:24Z")

</div>

Thank you @stephenb but the builtin keywords like WORD, IP...?

---

<div class="post-metadata">

**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\
**Post date:** [December 17, 2024, 3:20pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/30 "2024-12-17T15:20:02Z")

</div>

Found it!  
In github!

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 17, 2024, 3:24pm UTC](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768/31 "2024-12-17T15:24:28Z")

</div>

It Takes a little practice... but `grok` is very powerful...

You should also learn `dissect` and `kv` together you can do a lot if needed.

[Previous page](https://discuss.elastic.co/t/double-processor-in-ingest-pipeline/371768.md?page=1)