# How to change the data type values

**URL:** <https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396>\
**Category:** Logstash\
**Created:** [May 3, 2018, 8:07am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396 "2018-05-03T08:07:11Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 8:07am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/1 "2018-05-03T08:07:11Z")

</div>

i have data like this is json file  
`{\"dataType\":\"System.Single\",\"host\":\"ssss\",\"name\":\"123/xxx\",\"123234\":\"333\",\"time\":\"2018/5/3 7:49:35\",\"value\":\"116.3047\"}`

question:  
if the **dataType** is **System.Single** need set the **value** is **number** 。  
if the **dataType** is **System.String** nedd set the **value** is **string**.  
it can add field in the output.  
how configure the .conf file  
thanks.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 3, 2018, 8:17am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/2 "2018-05-03T08:17:38Z")

</div>

When you index data into Elasticsearch, each field must have a single mapping within each index. Are you planning on sending these different types of data to different indices?

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 8:26am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/3 "2018-05-03T08:26:40Z")

</div>

in one index. it can add field about it .  
like in es data:  
"message": "{"dataType":"System.Single","host":"xxx","name":"xx/xxx","123":"222","time":"2018/5/3 8:03:22","value":"-6.5625"}",  
if dataType is System.Single i add a filed as **value\_number** the value is **-6.5625**  
if datatype is System.String i add a filed as **value\_string** the value is **"stringtest"**

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 3, 2018, 8:36am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/4 "2018-05-03T08:36:37Z")

</div>

Yes, if you split it into different fields you can do that.

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 8:38am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/5 "2018-05-03T08:38:45Z")

</div>

how can configure the filter file  
i don't know how to configure the .conf file  
can help me?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 3, 2018, 8:44am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/6 "2018-05-03T08:44:15Z")

</div>

Use a [json filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html) to parse the message. Then use [conditionals](https://www.elastic.co/guide/en/logstash/6.2/config-examples.html#using-conditionals) on the `dataType` field to copy over data from `value` to the appropriate field name. Use the [mutate filter](https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-mutate.html) to [convert](https://www.elastic.co/guide/en/logstash/6.2/plugins-filters-mutate.html#plugins-filters-mutate-convert) the type to `float` where appropriate. You can then remove the `value` field if you want to.

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 8:46am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/7 "2018-05-03T08:46:36Z")

</div>

i can't open use give url can give me a complete example as my ask.  
thanks so much.. @Christian_Dahlqvist  
😂😂😂😂😂😂😂

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 9:02am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/8 "2018-05-03T09:02:00Z")

</div>

the filter is error can't start logstash @Christian_Dahlqvist

```
filter 
{ 
  json {  
    source => "message"
    id => "json_train"
    if [message][dataType] == "System.Single"
     {
    add_field => { "value_number" => "%{value}"
     }
  }
  date {  
    match => ["time", "YYYY/M/dd HH:mm:ss"]
    timezone => "Asia/Shanghai"
    locale => "cn" 
    target => "report_time"
  }

}
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 3, 2018, 9:05am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/9 "2018-05-03T09:05:55Z")

</div>

You can not have conditionals within a filter, so the conditionals need to be outside the json filter.

```auto
filter { 
  json {  
    source => "message"
    id => "json_train"
  }

  if [dataType] == "System.Single" {
    mutate {  
      add_field => { "value_number" => "%{value}"}
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 9:11am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/10 "2018-05-03T09:11:18Z")

</div>

is like this  
also can't start 😂😂😂😂😂 can help me to slove the error?

```
filter
{
    if [message][dataType] == "System.Single"{
  json {
    source => "message"
    id => "json_train"
    add_field => { "value_number" => "%{value}"
     }
     } else {
  json {
    source => "message"
    id => "json_train1"
     add_field => { "value_string" => "%{value}"
     }
   }
  }
  date {
    match => ["time", "YYYY/M/dd HH:mm:ss"]
    timezone => "Asia/Shanghai"
    locale => "cn"
    target => "report_time"
  }

}
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 3, 2018, 9:13am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/11 "2018-05-03T09:13:36Z")

</div>

You will not have access to the dataType field until you have run the json filter, which is why your version does not work. I spotted an error in my example and have updated it. Try that and continue to build out the logic within the conditionals until you have what you are looking for.

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 9:17am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/12 "2018-05-03T09:17:29Z")

</div>

use your new filter is also can't start with

`[2018-05-03T17:16:26,761][ERROR][logstash.agent] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 35, column 17 (byte 545) after filter \n{ \n json { \n source => \"message\"\n id => \"json_train\"\n }\n\n if [dataType] == \"System.Single\" {\n mutate { \n add_field => { \"value_number\" => \"%{value}\"}\n }\n \n date {\n match => [\"time\", \"YYYY/M/dd HH:mm:ss\"]\n timezone => \"Asia/Shanghai\"\n locale => \"cn\"\n target => \"report_time\"\n }\n\n}\n\noutput {\n elasticsearch ", :backtrace=>["/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/compiler.rb:42:in`compile\_imperative'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/compiler.rb:50:in `compile_graph'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/compiler.rb:12:in`block in compile\_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/compiler.rb:11:in`compile\_sources'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/pipeline.rb:51:in `initialize'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/pipeline.rb:169:in`initialize'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/pipeline\_action/create.rb:40:in `execute'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:315:in`block in converge\_state'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:312:in`block in converge\_state'", "org/jruby/RubyArray.java:1734:in `each'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:299:in`converge\_state'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:141:in`with\_pipelines'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:90:in`execute'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/runner.rb:348:in `block in execute'", "/opt/ES/logstash-6.2.3-es/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in`block in initialize'"]}`

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 9:21am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/13 "2018-05-03T09:21:05Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> You can not have conditionals within a filter, so the conditionals need to be outside the json filter.
> 
> filter {  
> json {  
> source =\> "message"  
> id =\> "json\_train"  
> }
> 
> if [dataType] == "System.Single" {  
> mutate {  
> add\_field =\> { "value\_number" =\> "%{value}"}  
> }  
> }

use your give filter  
also can't start  
with

`Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of #, => at line 26, column 17 (byte 410) after filter { \n json { \n source => \"message\"\n id => \"json_train\"\n }\n\n if [dataType] == \"System.Single\" {\n mutate { \n add_field => { \"value_number\" => \"%{value}\"}\n }\n}\n\noutput {\n elasticsearch ", :backtrace=>["/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/compiler.rb:42:in `compile\_imperative'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/compiler.rb:50:in `compile_graph'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/compiler.rb:12:in `block in compile\_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/compiler.rb:11:in `compile\_sources'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/pipeline.rb:51:in `initialize'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/pipeline.rb:169:in `initialize'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/pipeline\_action/create.rb:40:in `execute'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:315:in `block in converge\_state'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:312:in `block in converge\_state'", "org/jruby/RubyArray.java:1734:in `each'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:299:in `converge\_state'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:141:in `with\_pipelines'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/opt/ES/logstash-6.2.3-es/logstash-core/lib/logstash/runner.rb:348:in `block in execute'", "/opt/ES/logstash-6.2.3-es/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}`

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 3, 2018, 9:25am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/14 "2018-05-03T09:25:18Z")

</div>

Think I missed a curly brace. Have added it.

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 9:55am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/15 "2018-05-03T09:55:46Z")

</div>

as my full filter can't start can slove me where is error? 😂😂😂😂😂😂😂  
thanks @Christian_Dahlqvist

```
filter { 
  json {  
    source => "message"
    id => "json_train"
  }

  if [dataType] == "System.Single" {
    mutate {  
      add_field => { "value_number" => "%{value}"}
    }else if [dataType] == "System.Double" {
    mutate {
      add_field => { "value_number" => "%{value}"}
    }else if [dataType] == "System.Int16" {
    mutate {
      add_field => { "value_number" => "%{value}"}
    }else if [dataType] == "System.Int32" {
    mutate {
      add_field => { "value_number" => "%{value}"}
    } else if [dataType] == "System.Int64" {
    mutate {
      add_field => { "value_number" => "%{value}"}
    } else if [dataType] == "System.Boolean" {
    mutate {
      add_field => { "value_Boolean" => "%{value}"}
    } else {
    mutate {
      add_field => { "value_string" => "%{value}"}
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 3, 2018, 9:57am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/16 "2018-05-03T09:57:47Z")

</div>

You are missing a number of curly braces before the `else` statements.

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 10:02am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/17 "2018-05-03T10:02:16Z")

</div>

thanks so much . slove it .  
thanks so.  
:elasticheart::elasticheart::elasticheart::elasticheart::elasticheart::elasticheart::elasticheart:

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 10:11am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/18 "2018-05-03T10:11:29Z")

</div>

the date filed **value\_Boolean** is **True** or **False** , i wan't change to **1** or **0** in es .  
how can do it ?  
@Christian_Dahlqvist

---

<div class="post-metadata">

**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\
**Post date:** [May 3, 2018, 10:22am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/19 "2018-05-03T10:22:24Z")

</div>

```
filter { 
  json {  
    source => "message"
    id => "json_train"
  }

  if [dataType] == "System.Single" {
    mutate {  
      add_field => { "value_number" => "%{value}"}
      } 
    } else if [dataType] == "System.Double" {
    mutate {
      add_field => { "value_number" => "%{value}"}
      }
    } else if [dataType] == "System.Int16" {
    mutate {
      add_field => { "value_number" => "%{value}"}
      }
    } else if [dataType] == "System.Int32" {
    mutate {
      add_field => { "value_number" => "%{value}"}
      }
    } else if [dataType] == "System.Int64" {
    mutate {
      add_field => { "value_number" => "%{value}"}
      }
    } else if [dataType] == "System.Boolean" {
    mutate {
      add_field => { "value_Boolean" => "%{value}"}
      }
    } else {
    mutate {
      add_field => { "value_string" => "%{value}"}
        }
    }

  date {
# match => ["time", "YYYY/M/dd HH:mm:ss +08:00"]
    match => ["time", "YYYY/M/dd HH:mm:ss"]
    timezone => "Asia/Shanghai"
    locale => "cn"
    target => "report_time"
  }

}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 31, 2018, 10:22am UTC](https://discuss.elastic.co/t/how-to-change-the-data-type-values/130396/20 "2018-05-31T10:22:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.