# Cef log with custom udp integration **URL:** \ **Category:** SIEM\ **Created:** [May 22, 2025, 12:09pm UTC](https://discuss.elastic.co/t/cef-log-with-custom-udp-integration/378421 "2025-05-22T12:09:03Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\ **Post date:** [May 22, 2025, 12:09pm UTC](https://discuss.elastic.co/t/cef-log-with-custom-udp-integration/378421/1 "2025-05-22T12:09:03Z")
Hi all, I am using Custom UDP integration to do CEF log ingestion. I cannot use CEF integration due to limitations of that integration. The decode\_cef processor is not available in the pipelines. What is the best way to do CEF parsing with custom UDP integration without writing a costly and detailed custom pipeline? ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [May 22, 2025, 1:08pm UTC](https://discuss.elastic.co/t/cef-log-with-custom-udp-integration/378421/2 "2025-05-22T13:08:25Z")
> [@Cristina\_Marletta\_Li](#): > > The decode\_cef processor is not available in the pipelines. The `decode_cef` is a Filebeat processor, not an Ingest processor, to use it in the Custom UDP integration you need to add it to the Processor lists in the _Advanced options_ part. Something like this: ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/0/e00bab2e68bb29cb586e8a21efe6e3a96936be09.png) > [@Cristina\_Marletta\_Li](#): > > I cannot use CEF integration due to limitations of that integration. What limitations? The Custom CEF integration is basically a custom tcp, custom udp or custom log integration with some built-in processors, like a `rename` to rename `message` into `event.original` and a `decode_cef` processor. The udp input for the Custom CEF integration is defined [here](https://github.com/elastic/integrations/blob/main/packages/cef/data_stream/log/agent/stream/udp.yml.hbs). ---
**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\ **Post date:** [May 22, 2025, 2:03pm UTC](https://discuss.elastic.co/t/cef-log-with-custom-udp-integration/378421/3 "2025-05-22T14:03:19Z")
In fact, you cannot rename the dataset name in CEF integration. If you rename the dataset, you cannot associate a custom ingestion pipeline. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [May 22, 2025, 2:09pm UTC](https://discuss.elastic.co/t/cef-log-with-custom-udp-integration/378421/4 "2025-05-22T14:09:30Z")
> [@Cristina\_Marletta\_Li](#): > > In fact, you cannot rename the dataset name in CEF integration. I didn't tested, but you can change it here: ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/d/3dc8160e4dada050845f0e6b3f5ee9ba84a8370c.png) > [@Cristina\_Marletta\_Li](#): > > If you rename the dataset, you cannot associate a custom ingestion pipeline. Can you share the agente configuration after you created it? You normally have the option to add a custom ingest pipeline in all integrations, but sometimes this only shows up after the integration is created. Also, if you use a custom dataset name like `logs-custom.dataset-namespace`, you need to have a custom template where you can set a custom ingest pipeline. ---
**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\ **Post date:** [May 22, 2025, 2:19pm UTC](https://discuss.elastic.co/t/cef-log-with-custom-udp-integration/378421/5 "2025-05-22T14:19:19Z")
I opened a case to support about it. The problem seems related to this: > > > \*\*Kibana version:\*\* > > Seen at least in 8.6, 8.7 and 8.8. Update: there are still …issues in 8.16. > > \*\*Describe the bug:\*\* > > When the data stream of an integration package is configured to use a custom dataset, it may end up using the fallback index templates for logs and metrics, instead of the template installed with the package, missing important mappings or metadata. > > This can cause issues like: > \* Unexpected mappings for important fields (see https://github.com/elastic/integrations/issues/6566#issuecomment-1590951433). > \* Data loss if the data cannot be ingested with the used mappings. > \* Problems using Fleet features that rely on metadata. For example if the data stream doesn't include the package name in meta, it cannot be reused and may cause issues when migrating from integrations to input packages starting on 8.8. > \* Apart of the mappings, no \`@custom\` pipeline is referenced, so no custom processing can be added. > > This is confirmed at least with: > \* The logs package (before 2.0), when the dataset is different to \`logs.logs\`. > \* Prometheus package when the dataset is different to \`prometheus.collector\`. > \* Windows Event Logs when the dataset is different to \`winlog.winlog\`. > > \*\*Steps to reproduce:\*\* > 1. Create an integration policy for an integration package that allows the use of custom datasets. > 2. Use a custom dataset that is different to \`\.\\`. > 3. Ingest data. > 4. Check how the created data stream is missing certain metadata such as the package name, and is using the \`logs\` or \`metrics\` built-in data streams. > > \*\*Expected behavior:\*\* > > The data stream uses the index template installed with the package. This is what happens with input packages or with integration packages that don't use custom datasets. > > \*\*Screenshots (if relevant):\*\* > !\[imagen\](https://github.com/elastic/kibana/assets/15763/83593138-080c-4326-80dd-8d0cfd42c530) > > \`\`\` > GET \_data\_stream/metrics-prometheus\* > ... > "\_meta": { > "managed": true, > "description": "default metrics template installed by x-pack" > }, > ... > \`\`\` Basically, whenever we change the dataset name to something that default index template (logs-cef.log), index pattern logs-cef.log-\* doesn't cover, this dataset/indices will never get the expected ingestion pipelines. I tried to hange the dataset name to something that can match the index pattern, for example: logs-cef.log-prod. By doing this, the agent stopped ingesting events. Only by renaming the dataset name back to cef.log did events start coming in again. I had no problems with Custom UDP integration. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 19, 2025, 2:19pm UTC](https://discuss.elastic.co/t/cef-log-with-custom-udp-integration/378421/6 "2025-06-19T14:19:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.