# Extracting and mapping fields from event.original in Trend Micro Vision One Integration

**URL:** <https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206>\
**Category:** Elastic Agent\
**Created:** [November 28, 2024, 4:20pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206 "2024-11-28T16:20:56Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![cyberm](https://avatars.discourse-cdn.com/v4/letter/c/3ab097/32.png) [@cyberm](https://discuss.elastic.co/u/cyberm)\
**Post date:** [November 28, 2024, 4:20pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/1 "2024-11-28T16:20:56Z")

</div>

Hi everyone,

I’m trying to properly configure the Vision One integration in Kibana to collect and manage logs. Here’s what I’ve done so far:

1. I’ve set up the Vision One integration in Kibana.
2. I’ve configured the policy on the agent installed on the same machine, which collects logs via API.

The setup works partially—I’m receiving various logs from Vision One. However, I’ve noticed an issue with some important data found in the `event.original` field. This field contains a full JSON object that isn’t mapped, but it has information that would be very useful for me.

My goal is to extract specific information from `event.original` and map it to other fields so I can manage it better. I’m considering using an ingest pipeline to do this.

Additionally, I’m having trouble handling logs related to _container security_ with this setup.

Has anyone dealt with a similar situation? Could you share advice or examples of how to set up an ingest pipeline to extract and map fields from `event.original`?

Any help or tips would be greatly appreciated!

Thanks a lot!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 28, 2024, 4:29pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/2 "2024-11-28T16:29:12Z")

</div>

> [@cyberm](#):
>
> However, I’ve noticed an issue with some important data found in the `event.original` field. This field contains a full JSON object that isn’t mapped, but it has information that would be very useful for me.

Can you provide any example? Probably this kind of data is not being parsed by the built-in integration, so you would need to build your own parsers by creating a custom ingest pipeline.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 28, 2024, 4:38pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/3 "2024-11-28T16:38:09Z")

</div>

Hi @cyberm Welcome to the community.

What version Elastic and Kibana?  
What are you using to collect the Elastic Agent integrations?  
Are you speaking of the Trend Micro Vision One?  
Which Logs do you want to add the extra parsing to?

Yes if you want to work with `event.original` make sure you set `Preserve original event` looks like you are doing that.

So I would do a quick read of this (you do need to understand it all but good background)

> **[Data streams | Fleet and Elastic Agent Guide \[8.16\] | Elastic](https://www.elastic.co/guide/en/fleet/8.16/data-streams.html)**

Then, here is how you add a custom pipeline to the end of the Vision One Processing

> **[Tutorial: Transform data with custom ingest pipelines | Fleet and Elastic...](https://www.elastic.co/guide/en/fleet/8.16/data-streams-pipeline-tutorial.html)**

Then, in the custom pipeline, you will probably want to use the `JSON` processor which should extract the fields... then you can rename drop, move the fields etc if you like.

> **[JSON processor | Elasticsearch Guide \[8.16\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/json-processor.html)**

Pro tip: put the expanded JSON field under a field like `details` so you can see it first ... if you write them to the root of the document, you may overwrite key /processed fields.

---

<div class="post-metadata">

**Author:** ![cyberm](https://avatars.discourse-cdn.com/v4/letter/c/3ab097/32.png) [@cyberm](https://discuss.elastic.co/u/cyberm)\
**Post date:** [November 29, 2024, 3:16pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/4 "2024-11-29T15:16:29Z")

</div>

Hi everyone,

This image shows the part of the document we are focusing on, extracted from the index populated through the integration with **Trend Micro Vision One** :

 ![image001](https://us1.discourse-cdn.com/elastic/original/3X/0/8/08de514796f9f00f5bb6ca28b7ac857367b0d7b9.png)

The complete document looks like this and includes the `event.original` attribute, which is a complete JSON:

```auto
{
    "_index": "\"***\"",
    "_id": "\"***\"",
    "_version": 1,
    "_score": 0,
    "_source": {
        "agent": {
            "name": "\"***\"",
            "id": "\"***\"",
            "ephemeral_id": "\"***\"",
            "type": "\"***\"",
            "version": "\"***\""
        },
        "trend_micro_vision_one": {
            "alert": {
                "impact_scope": {
                    "email_address_count": 0,
                    "entities": [
                        {
                            "provenance": [
                                "\"***\""
                            ],
                            "managementScopeGroupId": "\"***\"",
                            "related_indicator_id": [
                                1,
                                2,
                                3,
                                4
                            ],
                            "id": "\"***\"",
                            "type": "\"***\"",
                            "value": {
                                "name": "\"***\"",
                                "guid": "\"***\"",
                                "ips": [
                                    "\"***\""
                                ]
                            }
                        },
                        {
                            "provenance": [
                                "\"***\""
                            ],
                            "managementScopeGroupId": "\"***\"",
                            "related_indicator_id": [
                                2,
                                4,
                                5,
                                6,
                                7,
                                8,
                                9,
                                10,
                                11,
                                12,
                                13,
                                14,
                                15,
                                16
                            ],
                            "id": "\"***\"",
                            "type": "\"***\"",
                            "value": {
                                "account_value": "\"***\""
                            }
                        }
                    ],
                    "desktop_count": 1,
                    "server_count": 0,
                    "account_count": 0
                },
                "description": "\"***\"",
                "indicators": [
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\"",
                            "\"***\""
                        ],
                        "id": 1,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\"",
                            "\"***\""
                        ],
                        "id": 2,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\"",
                            "\"***\""
                        ],
                        "id": 3,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\"",
                            "\"***\""
                        ],
                        "id": 4,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 5,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 6,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 7,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 8,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 9,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 10,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 11,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 12,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 13,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 14,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 15,
                        "type": "\"***\"",
                        "value": "\"***\""
                    },
                    {
                        "provenance": [
                            "\"***\""
                        ],
                        "field": "\"***\"",
                        "filter_id": [
                            "\"***\""
                        ],
                        "related_entities": [
                            "\"***\""
                        ],
                        "id": 16,
                        "type": "\"***\"",
                        "value": "\"***\""
                    }
                ],
                "schema_version": "\"***\"",
                "investigation_status": "\"***\"",
                "alert_provider": "\"***\"",
                "model": "\"***\"",
                "matched_rule": [
                    {
                        "filter": [
                            {
                                "date": "\"***\"",
                                "name": "\"***\"",
                                "mitre_technique_id": [
                                    "\"***\""
                                ],
                                "id": "\"***\"",
                                "events": [
                                    {
                                        "date": "\"***\"",
                                        "type": "\"***\"",
                                        "uuid": "\"***\""
                                    }
                                ]
                            },
                            {
                                "date": "\"***\"",
                                "name": "\"***\"",
                                "mitre_technique_id": [
                                    "\"***\""
                                ],
                                "id": "\"***\"",
                                "events": [
                                    {
                                        "date": "\"***\"",
                                        "type": "\"***\"",
                                        "uuid": "\"***\""
                                    }
                                ]
                            }
                        ],
                        "name": "\"***\"",
                        "id": "\"***\""
                    }
                ],
                "created_date": "\"***\"",
                "workbench_link": "\"***\""
            }
        },
        "log": {
            "level": "\"***\""
        },
        "elastic_agent": {
            "id": "\"***\"",
            "version": "\"***\"",
            "snapshot": false
        },
        "url": {
            "path": "\"***\"",
            "fragment": "\"***\"",
            "extension": "\"***\"",
            "original": "\"***\"",
            "scheme": "\"***\"",
            "domain": "\"***\""
        },
        "tags": [
            "\"***\"",
            "\"***\"",
            "\"***\""
        ],
        "cloud": {
            "availability_zone": "\"***\"",
            "instance": {
                "name": "\"***\"",
                "id": "\"***\""
            },
            "provider": "\"***\"",
            "service": {
                "name": "\"***\""
            },
            "machine": {
                "type": "\"***\""
            }
        },
        "input": {
            "type": "\"***\""
        },
        "@timestamp": "\"***\"",
        "ecs": {
            "version": "\"***\""
        },
        "related": {
            "ip": [
                "\"***\""
            ]
        },
        "data_stream": {
            "namespace": "\"***\"",
            "type": "\"***\"",
            "dataset": "\"***\""
        },
        "event": {
            "severity": 30,
            "agent_id_status": "\"***\"",
            "ingested": "\"***\"",
            "original": "\"eventOriginalinJSON\"",
            "created": "\"***\"",
            "kind": "\"***\"",
            "id": "\"***\"",
            "category": [
                "\"***\""
            ],
            "type": [
                "\"***\""
            ],
            "dataset": "\"***\""
        }
    },
    "fields": {
        "elastic_agent.version": [
            "\"***\""
        ],
        "event.category": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.entities.value.account_value": [
            "\"***\""
        ],
        "url.original.text": [
            "\"***\""
        ],
        "cloud.availability_zone": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.schema_version": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.description": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.entities.value.guid": [
            "\"***\""
        ],
        "log.level": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.entities.value.name": [
            "\"***\""
        ],
        "agent.name": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.indicators.id": [
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\""
        ],
        "event.agent_id_status": [
            "\"***\""
        ],
        "event.kind": [
            "\"***\""
        ],
        "url.fragment": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.email_address_count": [
            0
        ],
        "event.severity": [
            30
        ],
        "trend_micro_vision_one.alert.matched_rule.filter.id": [
            "\"***\"",
            "\"***\""
        ],
        "trend_micro_vision_one.alert.matched_rule.filter.mitre_technique_id": [
            "\"***\"",
            "\"***\""
        ],
        "event.original": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.entities.provenance": [
            "\"***\"",
            "\"***\""
        ],
        "trend_micro_vision_one.alert.matched_rule.id": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.matched_rule.name": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.model": [
            "\"***\""
        ],
        "url.extension": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.entities.managementScopeGroupId": [
            "\"***\"",
            "\"***\""
        ],
        "input.type": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.alert_provider": [
            "\"***\""
        ],
        "data_stream.type": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.created_date": [
            "\"***\""
        ],
        "tags": [
            "\"***\"",
            "\"***\"",
            "\"***\""
        ],
        "cloud.machine.type": [
            "\"***\""
        ],
        "cloud.provider": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.investigation_status": [
            "\"***\""
        ],
        "url.path": [
            "\"***\""
        ],
        "agent.id": [
            "\"***\""
        ],
        "cloud.service.name": [
            "\"***\""
        ],
        "ecs.version": [
            "\"***\""
        ],
        "event.created": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.matched_rule.filter.name": [
            "\"***\"",
            "\"***\""
        ],
        "agent.version": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.desktop_count": [
            1
        ],
        "trend_micro_vision_one.alert.indicators.field": [
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.entities.value.ips": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.indicators.related_entities": [
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.account_count": [
            0
        ],
        "url.scheme": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.indicators.provenance": [
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\""
        ],
        "cloud.instance.id": [
            "\"***\""
        ],
        "agent.type": [
            "\"***\""
        ],
        "event.module": [
            "\"***\""
        ],
        "related.ip": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.indicators.filter_id": [
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\""
        ],
        "elastic_agent.snapshot": [
            false
        ],
        "trend_micro_vision_one.alert.impact_scope.server_count": [
            0
        ],
        "trend_micro_vision_one.alert.matched_rule.filter.events.uuid": [
            "\"***\"",
            "\"***\""
        ],
        "trend_micro_vision_one.alert.indicators.value": [
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.entities.related_indicator_id": [
            1,
            2,
            3,
            4,
            2,
            4,
            5,
            6,
            7,
            8,
            9,
            10,
            11,
            12,
            13,
            14,
            15,
            16
        ],
        "trend_micro_vision_one.alert.matched_rule.filter.date": [
            "\"***\"",
            "\"***\""
        ],
        "trend_micro_vision_one.alert.matched_rule.filter.events.date": [
            "\"***\"",
            "\"***\""
        ],
        "elastic_agent.id": [
            "\"***\""
        ],
        "data_stream.namespace": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.matched_rule.filter.events.type": [
            "\"***\"",
            "\"***\""
        ],
        "trend_micro_vision_one.alert.indicators.type": [
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\"",
            "\"***\""
        ],
        "trend_micro_vision_one.alert.workbench_link": [
            "\"***\""
        ],
        "event.ingested": [
            "\"***\""
        ],
        "url.original": [
            "\"***\""
        ],
        "@timestamp": [
            "\"***\""
        ],
        "data_stream.dataset": [
            "\"***\""
        ],
        "event.type": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.entities.id": [
            "\"***\"",
            "\"***\""
        ],
        "url.domain": [
            "\"***\""
        ],
        "agent.ephemeral_id": [
            "\"***\""
        ],
        "trend_micro_vision_one.alert.impact_scope.entities.type": [
            "\"***\"",
            "\"***\""
        ],
        "event.id": [
            "\"***\""
        ],
        "event.dataset": [
            "\"***\""
        ],
        "cloud.instance.name": [
            "\"***\""
        ]
    }
}

```

Within this document, here is the specific content of the `event.original` field:

```auto
{
    "alertProvider": "\"***\"",
    "createdDateTime": "\"***\"",
    "description": "\"***\"",
    "id": "\"***\"",
    "impactScope": {
        "accountCount": 0,
        "cloudIdentityCount": 0,
        "containerCount": 1,
        "desktopCount": 1,
        "emailAddressCount": 0,
        "entities": [
            {
                "entityId": "\"***\"",
                "entityType": "\"***\"",
                "entityValue": {
                    "guid": "\"***\"",
                    "ips": [
                        "\"***\""
                    ],
                    "name": "\"***\""
                },
                "managementScopeGroupId": "\"***\"",
                "provenance": [
                    "\"***\""
                ],
                "relatedEntities": [],
                "relatedIndicatorIds": [
                    1,
                    2,
                    3,
                    4
                ]
            },
            {
                "entityId": "\"***\"",
                "entityType": "\"***\"",
                "entityValue": "\"***\"",
                "managementScopeGroupId": "\"***\"",
                "provenance": [
                    "\"***\""
                ],
                "relatedEntities": [],
                "relatedIndicatorIds": [
                    2,
                    4,
                    5,
                    6,
                    7,
                    8,
                    9,
                    10,
                    11,
                    12,
                    13,
                    14,
                    15,
                    16
                ]
            }
        ],
        "serverCount": 0
    },
    "indicators": [
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 1,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\"",
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 2,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\"",
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 3,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\"",
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 4,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\"",
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 5,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 6,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 7,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 8,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 9,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 10,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 11,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 12,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 13,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 14,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"***\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 15,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"***\""
        },
        {
            "field": "\"valueTobeExtract\"",
            "filterIds": [
                "\"***\""
            ],
            "id": 16,
            "provenance": [
                "\"***\""
            ],
            "relatedEntities": [
                "\"***\""
            ],
            "type": "\"***\"",
            "value": "\"valueTobeExtract\""
        }
    ],
    "investigationResult": "\"***\"",
    "investigationStatus": "\"***\"",
    "matchedRules": [
        {
            "id": "\"***\"",
            "matchedFilters": [
                {
                    "id": "\"***\"",
                    "matchedDateTime": "\"***\"",
                    "matchedEvents": [
                        {
                            "matchedDateTime": "\"***\"",
                            "type": "\"***\"",
                            "uuid": "\"***\""
                        }
                    ],
                    "mitreTechniqueIds": [
                        "\"***\""
                    ],
                    "name": "\"***\""
                },
                {
                    "id": "\"***\"",
                    "matchedDateTime": "\"***\"",
                    "matchedEvents": [
                        {
                            "matchedDateTime": "\"***\"",
                            "type": "\"***\"",
                            "uuid": "\"***\""
                        }
                    ],
                    "mitreTechniqueIds": [
                        "\"***\""
                    ],
                    "name": "\"***\""
                }
            ],
            "name": "\"***\""
        }
    ],
    "model": "\"***\"",
    "modelId": "\"***\"",
    "modelType": "\"***\"",
    "ownerIds": [],
    "schemaVersion": "\"***\"",
    "score": 30,
    "severity": "\"***\"",
    "status": "\"***\"",
    "updatedDateTime": "\"***\"",
    "workbenchLink": "\"***\""
}

```

My goal is to extract the values associated with the key `valueToBeExtract` from this nested JSON (`event.original`).

I also noticed that when I perform a search using KQL on the attribute `trend_micro_vision_one.alert.indicators.value`, it only evaluates the first value in the JSON object, not the subsequent ones. I’m not sure why this happens.

Thanks in advance for any suggestions or help!

---

<div class="post-metadata">

**Author:** ![cyberm](https://avatars.discourse-cdn.com/v4/letter/c/3ab097/32.png) [@cyberm](https://discuss.elastic.co/u/cyberm)\
**Post date:** [November 29, 2024, 3:29pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/5 "2024-11-29T15:29:33Z")

</div>

I’m using Elasticsearch and Kibana version 8.16.0. To collect Elastic Agent integrations, I have configured the **Trend Micro Vision One** integration. Specifically, there is an agent that forwards the requested logs via API and writes them to an Elasticsearch index.

Regarding the logs I want to parse, I need to extract and parse the values of `field` and `value` from the following object:

```auto
{
    "field": "\"valueTobeExtract\"",
    "filterIds": ["\"***\""],
    "id": 16,
    "provenance": ["\"***\""],
    "relatedEntities": ["\"***\""],
    "type": "\"***\"",
    "value": "\"valueTobeExtract\""
}

```

I can also confirm that I enabled the option to preserve the original event (`Preserve original event`) during the integration setup. I hope this clarifies my setup and requirements!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 29, 2024, 4:35pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/6 "2024-11-29T16:35:08Z")

</div>

You redacted so many thing that is pretty hard to compare the parsed event and the original event.

But from what I was able to understand your message is already being parsed.

What you want is inside the field `trend_micro_vision_one.indicators`, the main issue is that field is an array of json objects, the individual fields here will not be parsed.

You still can access them.

If you want to parse some individual fields you will need to use an ingest pipeline to do that, probably will need to use the `script` processor to traverse the array and get the value of the field you want, but I do not have much experience with this.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 29, 2024, 4:48pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/7 "2024-11-29T16:48:41Z")

</div>

What exactly do you want to do with this field? Maybe you can access it in the way it is already parsed.

---

<div class="post-metadata">

**Author:** ![strawgate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/strawgate/32/131008_2.png) [@strawgate](https://discuss.elastic.co/u/strawgate)\
**Post date:** [November 29, 2024, 4:49pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/8 "2024-11-29T16:49:39Z")

</div>

> [@cyberm](#):
>
> also noticed that when I perform a search using KQL on the attribute `trend_micro_vision_one.alert.indicators.value`, it only evaluates the first value in the JSON object, not the subsequent ones. I’m not sure why this happens.

Querying nested objects like this in KQL requires special syntax, see:

> **[Kibana Query Language | Kibana Guide \[8.16\] | Elastic](https://www.elastic.co/guide/en/kibana/current/kuery-query.html#_querying_nested_fields)**

---

<div class="post-metadata">

**Author:** ![cyberm](https://avatars.discourse-cdn.com/v4/letter/c/3ab097/32.png) [@cyberm](https://discuss.elastic.co/u/cyberm)\
**Post date:** [December 3, 2024, 10:22am UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/9 "2024-12-03T10:22:14Z")

</div>

The integration with VisionOne includes a mapping, but it does not cover all the fields we are interested in, only a subset. Some attributes in _event.original_ are relevant to us, but they are not searchable because they are internal objects that cannot be queried.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [December 3, 2024, 12:37pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/10 "2024-12-03T12:37:49Z")

</div>

The example you shared is of a field that it is already parsed.

The `indicators` object is parsed into the field `trend_micro_vision_one.alert.indicators`.

As mentioned, this is originally an array of json objects.

You can search on it by using `trend_micro_vision_one.alert.indicators.field` for example.

> [@cyberm](#):
>
> I also noticed that when I perform a search using KQL on the attribute `trend_micro_vision_one.alert.indicators.value`, it only evaluates the first value in the JSON object, not the subsequent ones. I’m not sure why this happens.

Can you provide some example of what you mean with this?

If you search for `trend_micro_vision_one.alert.indicators.value: "valueTobeExtract"` it will return documents where at least one of the items in the indicators array has the field named `value` equal to `valueTobeExtract`.

---

<div class="post-metadata">

**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\
**Post date:** [December 6, 2024, 10:27am UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/11 "2024-12-06T10:27:22Z")

</div>

Hi Stephenb,  
It would not appear possible to add a custom pipeline to the Trend Micro Vision One integration.  
There is no option to add a custom pipeline in Edit integration.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [December 6, 2024, 11:32am UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/12 "2024-12-06T11:32:58Z")

</div>

> [@Cristina\_Marletta\_Li](#):
>
> There is no option to add a custom pipeline in Edit integration.

It is possible, where did you try to add it?

You need to add the integration into the policy, and then edit it, then in the advanced options for each data set you will have something like this:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/b/3b7a75dd4e00acc95ef7045591aad77683b7d31e.png)

And when you click in _Add custom pipeline_ it will directs you to the pipeline creation page:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/d/6da9c2d1a6f08484a42c83cdaf809f75e74fa133.png)

Also, if you want to create a custom ingest pipeline that will apply to all data sets, you just need to create an ingest pipeline with the following name:

```auto
logs-trend_micro_vision_one.integration@custom

```

This will then be applied to every data set.

---

<div class="post-metadata">

**Author:** ![Cristina\_Marletta\_Li](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cristina_marletta_li/32/137793_2.png) [@Cristina\_Marletta\_Li](https://discuss.elastic.co/u/Cristina_Marletta_Li)\
**Post date:** [December 6, 2024, 3:16pm UTC](https://discuss.elastic.co/t/extracting-and-mapping-fields-from-event-original-in-trend-micro-vision-one-integration/371206/13 "2024-12-06T15:16:06Z")

</div>

Thank you.  
Now the problem is extract a nested json.  
I mean, I have a field (event.original) that contains in a field a stringfy json. If I apply the JSON processor through custom pipeline I get the parsing of all the fields in event.original (in custom field details). Now I would like to recursively apply the processor on the field details.rawDataStr that in turn contains a stringfy json.