# Removing a field with dots in an ingest pipeline

**URL:** <https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570>\
**Category:** Elasticsearch\
**Tags:** ingest-pipeline\
**Created:** [May 10, 2021, 11:18am UTC](https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570 "2021-05-10T11:18:36Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Anabella\_Cristaldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anabella_cristaldi/32/23612_2.png) [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\
**Post date:** [May 10, 2021, 11:18am UTC](https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570/1 "2021-05-10T11:18:36Z")

</div>

Hi all,  
I'm using filebeat with an ingest simple ingest pipeline and I want to remove some of the host's field, for example the host.os

I've defined a ingest pipeline

```
PUT _ingest/pipeline/appsalutmw
{
  "description": "APP Salut Ingest Pipeline",
  "processors": [
    {
      "set": {
        "field": "event.module",
        "value": "appsalutmw"
      }
    },
    {
      "remove": {
        "field": "host.os.name"
      }
    }
  ]
}

```

and when I simulate (using the same json I receive from filebeat) I get the error:

```
 {
  "docs" : [
    {
      "error" : {
        "root_cause" : [
          {
            "type" : "illegal_argument_exception",
            "reason" : "field [host] not present as part of path [host.os.name]"
          }
        ],
        "type" : "illegal_argument_exception",
        "reason" : "field [host] not present as part of path [host.os.name]"
      }
    }
  ]
}

```

I've tried to used (before the remove processor) the dot\_expander but with not luck

```
POST _ingest/pipeline/appsalutmw/_simulate
{
  "docs": [
    {
      "_source": {
        "host.os.name": [
          "Red Hat Enterprise Linux Server"
        ]
      }
    }
  ]
}

```

I'm able to remove it with an script processor but I was wondering why I'm not able to do it with the remove processor

Thank you  
Regards  
Ana

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 11, 2021, 3:06am UTC](https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570/2 "2021-05-11T03:06:29Z")

</div>

Hi @Anabella_Cristaldi

In that second example can you show the entire simulate plus the dock doc or at least the host.os.name portion so we can see.

It does seem like the dot expander issue but I know you said you tried it.

Show me the whole simulate pipeline plus the document plus the error.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 11, 2021, 4:05am UTC](https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570/3 "2021-05-11T04:05:11Z")

</div>

Interesting... the `dot_expander` works a little difference that I expected ..

```
PUT _ingest/pipeline/test-discuss
{
  "description": "APP Salut Ingest Pipeline",
  "processors": [
    {
      "dot_expander": {
        "field": "host.os.name"
      }
    },
    {
      "dot_expander": {
        "field": "host.os.rev"
      }
    },
    {
      "set": {
        "field": "event.module",
        "value": "appsalutmw"
      }
    },
    {
      "remove": {
        "field": "host.os.name",
        "ignore_failure": true
      }
    },
    {
      "remove": {
        "field": "my.test.field",
        "ignore_failure": true
      }
    }
  ]
}

```

Then simulate

```
POST /test-discuss/_doc?pipeline=test-discuss
{
  "host.os.name": "Red Hat Enterprise Linux Server",
  "host.os.rev": "32",
  "my": {
    "test": {
      "field": "value",
      "other_field" : "other_value"
    }
  }
}

```

Then results

```
GET /test-discuss/_search

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test-discuss",
        "_type" : "_doc",
        "_id" : "rVeWWXkBl0GsIo33gNji",
        "_score" : 1.0,
        "_source" : {
          "host" : {
            "os" : {
              "rev" : "32"
            }
          },
          "my" : {
            "test" : {
              "other_field" : "other_value"
            }
          },
          "event" : {
            "module" : "appsalutmw"
          }
        }
      }
    ]
  }
}
```

---

<div class="post-metadata">

**Author:** ![Anabella\_Cristaldi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anabella_cristaldi/32/23612_2.png) [@Anabella\_Cristaldi](https://discuss.elastic.co/u/Anabella_Cristaldi)\
**Post date:** [May 11, 2021, 2:50pm UTC](https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570/4 "2021-05-11T14:50:21Z")

</div>

Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 8, 2021, 2:51pm UTC](https://discuss.elastic.co/t/removing-a-field-with-dots-in-an-ingest-pipeline/272570/5 "2021-06-08T14:51:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.