# Drop document in ingest pipeline

**URL:** https://discuss.elastic.co/t/drop-document-in-ingest-pipeline/173394
**Category:** Elasticsearch
**Created:** [March 21, 2019, 8:07pm UTC](https://discuss.elastic.co/t/drop-document-in-ingest-pipeline/173394 "2019-03-21T20:07:44Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![leochavez](https://avatars.discourse-cdn.com/v4/letter/l/f05b48/32.png) [@leochavez](https://discuss.elastic.co/u/leochavez)
#### Post date: [March 21, 2019, 8:07pm UTC](https://discuss.elastic.co/t/drop-document-in-ingest-pipeline/173394/1 "2019-03-21T20:07:44Z")

</div>

Hi

I need to _drop_ a document in an _ingest pipeline_ , ie to not index it at all, I need to do it filtering the messages by the type of severity and discard them

I am testing it in the following way

```
PUT _ingest/pipeline/drop_model
{
  "processors": [
   {
      "drop": {
        "if": "ctx.severity == 'info'"
      }
    }
  ]
}

```

The logs in kibana that I need to discard are of the type

```
{
  "_index": "logstash",
  "_type": "events",
  "_id": "AWmh3Bauu--TaD0PBnfj",
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2019-03-21T20:05:48.966480+00:00",
    "@version": "1",
    "host": "linux-009",
    "severity": "info",
    "facility": "user",
    "tag": "updatemgr",
    "pid": 0,
    "msg": "2019-03-21T20:05:48:946Z 'VcIntegrity' 139793736918784 INFO [vcIntegrity, 1519] Getting IP Address from host name: linux-009"
  },
  "fields": {
    "@timestamp": [
      "2019-03-21T20:05:48.966Z"
    ]
  },
  "sort": [
    1553198748966
  ]
}
```

---

<div class="post-metadata">

### Author: ![Yogesh\_Gaikwad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogesh_gaikwad/32/27025_2.png) [@Yogesh\_Gaikwad](https://discuss.elastic.co/u/Yogesh_Gaikwad)
#### Post date: [March 22, 2019, 3:14am UTC](https://discuss.elastic.co/t/drop-document-in-ingest-pipeline/173394/2 "2019-03-22T03:14:34Z")

</div>

Hi @leochavez,

Looks like you might not have _drop_ processor for your ingest.  
Could you please invoke following API and share the output?  
`GET /_nodes/ingest`

The output should show you all the available processors in your setup.  
Hope this helps.

Regards,  
Yogesh Gaikwad

---

<div class="post-metadata">

### Author: ![leochavez](https://avatars.discourse-cdn.com/v4/letter/l/f05b48/32.png) [@leochavez](https://discuss.elastic.co/u/leochavez)
#### Post date: [March 26, 2019, 4:36pm UTC](https://discuss.elastic.co/t/drop-document-in-ingest-pipeline/173394/3 "2019-03-26T16:36:08Z")

</div>

I can see the drop processor on my ingest, check:

```
 "ingest" : {
        "processors" : [
          {
            "type" : "append"
          },
          {
            "type" : "bytes"
          },
          {
            "type" : "convert"
          },
          {
            "type" : "date"
          },
          {
            "type" : "date_index_name"
          },
          {
            "type" : "dissect"
          },
          {
            "type" : "dot_expander"
          },
          {
            "type" : "drop"
          },
          {
            "type" : "fail"
          },
          {
```

---

<div class="post-metadata">

### Author: ![Yogesh\_Gaikwad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yogesh_gaikwad/32/27025_2.png) [@Yogesh\_Gaikwad](https://discuss.elastic.co/u/Yogesh_Gaikwad)
#### Post date: [March 27, 2019, 6:51am UTC](https://discuss.elastic.co/t/drop-document-in-ingest-pipeline/173394/4 "2019-03-27T06:51:19Z")

</div>

Hi @leochavez,

Do you see any error or it does the indexing? If any error could you please share.  
Just a guess, Did you try with `ctx.severity == \"info\"'` instead of `ctx.severity == 'info'`?

Thank you.

I tried on my setup:-

```auto
PUT /_ingest/pipeline/my-pipe-1 '{ "processors" : [{ "drop" : { "if" : "ctx.sev_string == \"info\"" } }] }'

```

Try indexing doc with "info", verify response `successful` is `0`, result `noop`

```auto
POST '/my-index/_doc/1?pipeline=my-pipe-1&pretty' '{ "sev_string" : "info", "severity": 5 }' -Q

{
  "_index" : "my-index",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : -3,
  "result" : "noop",
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "failed" : 0
  }
}

```

Try indexing doc with "warn", verify response `successful` is `1`, result `created`

```auto
POST '/my-index/_doc/2?pipeline=my-pipe-1&pretty' '{ "sev_string" : "warn", "severity": 5 }' -Q

{
  "_index" : "my-index",
  "_type" : "_doc",
  "_id" : "2",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

```

Verify

```auto
GET /my-index/_search?pretty -Q

{
  "took" : 5,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "my-index",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "severity" : 5,
          "sev_string" : "warn"
        }
      }
    ]
  }
}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 24, 2019, 6:51am UTC](https://discuss.elastic.co/t/drop-document-in-ingest-pipeline/173394/5 "2019-04-24T06:51:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.