# Packetbeat Processors - Dropping fields from nested objects in an array

**URL:** <https://discuss.elastic.co/t/packetbeat-processors-dropping-fields-from-nested-objects-in-an-array/282988>\
**Category:** Beats\
**Tags:** packetbeat\
**Created:** [September 1, 2021, 6:00am UTC](https://discuss.elastic.co/t/packetbeat-processors-dropping-fields-from-nested-objects-in-an-array/282988 "2021-09-01T06:00:21Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![yotamgod](https://avatars.discourse-cdn.com/v4/letter/y/73ab20/32.png) [@yotamgod](https://discuss.elastic.co/u/yotamgod)\
**Post date:** [September 1, 2021, 6:00am UTC](https://discuss.elastic.co/t/packetbeat-processors-dropping-fields-from-nested-objects-in-an-array/282988/1 "2021-09-01T06:00:21Z")

</div>

I recently started working with Packetbeat.

For my use-case, I only need some specific fields (to the point where if I could, I would completely rewrite the mapping, but am leaving that as a last resort).

I tried removing some of the fields from the "dns.answers" array of objects, but what I did doesn't seem to have any effect:

```auto
    - include_fields:
        fields:
          - dns.question.name
          - dns.question.type
          - dns.answers
          - dns.answers_count
          - dns.resolved_ip
    - drop_fields:
        fields:
          - dns.answers.name

```

In addition, I also tried including only the fields I want but that didn't seem to work either, e.g:

```auto
- include_fields:
        fields:
          - dns.question.name
          - dns.question.type
          - dns.answers.data
          - dns.answers_count
          - dns.resolved_ip

```

Any ideas?  
If rewriting the template/mapping of the index is the best choice, or perhaps using the Ingest Node Pipelines is a better approach, I'd love to hear it.

Thanks

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 1, 2021, 10:27am UTC](https://discuss.elastic.co/t/packetbeat-processors-dropping-fields-from-nested-objects-in-an-array/282988/2 "2021-09-01T10:27:35Z")

</div>

Could you please share you whole configuration? Your configuration seems correct, but you might have some indentation error.

---

<div class="post-metadata">

**Author:** ![yotamgod](https://avatars.discourse-cdn.com/v4/letter/y/73ab20/32.png) [@yotamgod](https://discuss.elastic.co/u/yotamgod)\
**Post date:** [September 1, 2021, 10:46am UTC](https://discuss.elastic.co/t/packetbeat-processors-dropping-fields-from-nested-objects-in-an-array/282988/3 "2021-09-01T10:46:08Z")

</div>

Sure. here is my entire processors section of the YAML file.

```auto
processors:
    - drop_event:
        when:
          not:
            equals:
              dns.response_code: "NOERROR"
    - include_fields:
        fields:
          - dns.question.name
          - dns.question.type
          - dns.answers
          - dns.answers_count
          - dns.resolved_ip
          - dns.txt_answers
    - drop_fields:
        fields:
          - dns.answers.name
    - copy_fields:
        fields:
          - from: dns.answers.data
            to: dns.txt_answers
        ignore_missing: true
        fail_on_error: false

```

In addition, if you see I tried copying the "dns.answers.data" field to a separate field (like what is done in the dns.resolved\_ip field), and this doesn't seem to work either.  
The field configuration in the mapping is:

```auto
setup.template.overwrite: true
setup.template.append_fields:
- name: dns.txt_answers
  type: text
  level: extended

```

---

<div class="post-metadata">

**Author:** ![yotamgod](https://avatars.discourse-cdn.com/v4/letter/y/73ab20/32.png) [@yotamgod](https://discuss.elastic.co/u/yotamgod)\
**Post date:** [September 1, 2021, 11:21am UTC](https://discuss.elastic.co/t/packetbeat-processors-dropping-fields-from-nested-objects-in-an-array/282988/4 "2021-09-01T11:21:30Z")

</div>

Also, here is the error I get:  
Failed to copy fields in copy\_fields processor: could not fetch value for key: dns.answers.data, Error: expected map but type is common.MapStr

I think it might have to do with the following issue:

> [@Using a processor in a filebeat module may or may not actually find fields](https://discuss.elastic.co/t/using-a-processor-in-a-filebeat-module-may-or-may-not-actually-find-fields/191461):
>
> (Elasticsearch and filebeat are both v7.2) I was encountering a lot of difficulty using the convert processor to change types, so I simplified things down to using rename. The results of my tests are confusing. Given the following config block, I'd expect 6 fields to get renamed: - module: osquery result: enabled: true input: processors: - rename: fields: - from: "input.type" to: "input.typerenamed" - from: "fileset.name" …

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 29, 2021, 1:21pm UTC](https://discuss.elastic.co/t/packetbeat-processors-dropping-fields-from-nested-objects-in-an-array/282988/5 "2021-09-29T13:21:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.