# Ingest Pipeline Convert Processor is not converting

**URL:** <https://discuss.elastic.co/t/ingest-pipeline-convert-processor-is-not-converting/267166>\
**Category:** Elasticsearch\
**Created:** [March 14, 2021, 12:23am UTC](https://discuss.elastic.co/t/ingest-pipeline-convert-processor-is-not-converting/267166 "2021-03-14T00:23:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mhare](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhare/32/52213_2.png) [@mhare](https://discuss.elastic.co/u/mhare)\
**Post date:** [March 14, 2021, 12:23am UTC](https://discuss.elastic.co/t/ingest-pipeline-convert-processor-is-not-converting/267166/1 "2021-03-14T00:23:16Z")

</div>

I am using Filebeat, Elasticsearch and Kibana 7.10.2 in docker containers  
I have a working pipeline:

```auto
PUT /_ingest/pipeline/test_grok_pipeline
{
  "description": "Test grok pattern",
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": [
          """%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{WORD:timeoffset}%{SPACE}%{WORD:thread}%{SPACE}%{HOSTNAME:processName}%{SPACE}%{HOSTNAME:sourceName}%{SPACE}%{WORD:logType}%{SPACE}%{GREEDYDATAMULTILINE:message}"""
        ],
        "on_failure": [
          {
            "set": {
              "field": "error.message_grok",
              "value": "error in grok processor"
            }
          }
        ],
        "pattern_definitions": {
          "MESSAGE": "(\r|\n|.)*",
          "GREEDYDATAMULTILINE": "(.|\n)*"
        }
      }
    },
    {
      "date": {
        "field": "timestamp",
        "target_field": "@timestamp",
        "formats": ["ISO8601"], 
        "timezone": "America/Los_Angeles",
        "on_failure": [
          {
            "set": {
              "field": "error.message_date",
              "value": "error in date processor"
            }
          }
        ]
      }
    }
  ]
}

```

except the fields I am parsing with grok are not all typed so I cannot use them as a Term in Kibana. SO I added a Convert Processor:

```auto
{
  "description": "Test grok pattern",
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": [
          """%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{WORD:timeoffset}%{SPACE}%{WORD:thread}%{SPACE}%{HOSTNAME:processName}%{SPACE}%{HOSTNAME:sourceName}%{SPACE}%{WORD:logType}%{SPACE}%{GREEDYDATAMULTILINE:message}"""
        ],
        "on_failure": [
          {
            "set": {
              "field": "error.message_grok",
              "value": "error in grok processor"
            }
          }
        ],
        "pattern_definitions": {
          "MESSAGE": "(\r|\n|.)*",
          "GREEDYDATAMULTILINE": "(.|\n)*"
        }
      }
    },
    {
      "date": {
        "field": "timestamp",
        "target_field": "@timestamp",
        "formats": ["ISO8601"], 
        "timezone": "America/Los_Angeles",
        "on_failure": [
          {
            "set": {
              "field": "error.message_date",
              "value": "error in date processor"
            }
          }
        ]
      }
    },
    {
      "convert": {
        "field": "sourceName",
        "type": "string"
      }
    }
  ]
}

```

But, this does not seem to be working. There is still the small '?' next to the field name in Discover and I still cannot use it as a Term in a Visualization.

What am I missing here?

Thanks!

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 14, 2021, 2:00am UTC](https://discuss.elastic.co/t/ingest-pipeline-convert-processor-is-not-converting/267166/2 "2021-03-14T02:00:34Z")

</div>

Heya @mhare

Good to see your moving forward.

What Version are you on?

Before 7.11 you have to go into

Stack Management / Index Patterns and refresh the index pattern then go back to Discover before it will show the type in Discover and be useable in a visualization.

7.11 forward you do not need to do that.

Also have you created a mapping/ index template for this index so you define the field types ahead of time?

If not each text field will be both `text` and `keyword` (term)

---

<div class="post-metadata">

**Author:** ![mhare](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mhare/32/52213_2.png) [@mhare](https://discuss.elastic.co/u/mhare)\
**Post date:** [March 14, 2021, 2:37pm UTC](https://discuss.elastic.co/t/ingest-pipeline-convert-processor-is-not-converting/267166/3 "2021-03-14T14:37:22Z")

</div>

OK, I am such an idiot. I had completely forgotten to refresh the index 🙄  
That solved it right away.  
So, if I understand, in 7.11 I won't need to do the refresh? Maybe it's time to 'refresh' my containers 😀  
Thanks so much for the continued help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 11, 2021, 2:37pm UTC](https://discuss.elastic.co/t/ingest-pipeline-convert-processor-is-not-converting/267166/4 "2021-04-11T14:37:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.