ElasticSearch saving integer field as String

Elastic Stack Elasticsearch
1

Hi,

Could some one help me figure out what is the issue with below Logstash
Configuration /mapping template . ES seems to ignore the mappings specified
in the template and stores everything as string.

I am using Logstash csv plugin to create index. My logstash conf file has
csv filter as below:

filter {
csv {
columns => ["ESpeedRef","ProcessName",TotalMilliSec,AnyFloatNumber]
separator => ","
}

output {
elasticsearch{
protocol => "http"
host => "localhost"
index => "test_index"
}
}

Below json template is used to define field mappings. I use curl to send
template to ES.

{"template": "test_index",
"settings": {
"number_of_shards": 1
},
"mappings": {
"user": {
"properties": {
"ESpeedRef": {"type" : "string"},
"ProcessName": {"type" : "string"},
"TotalMilliSec": {"type" : "integer"},
"AnyFloatNumber": {"type" : "float"}
}
}
}
}

When creating index, Elastic Search does apply this template. I can confirm
this my looking at below console stmts:

[2014-11-24 13:22:51,099][INFO ][cluster.metadata ] [Captain
America] [
test_index] creating index, cause [auto(bulk api)], shards [1]/[1], mappings
[user]
[2014-11-24 13:22:51,146][INFO ][cluster.metadata ] [Captain
America] [
test_index] update_mapping [logs] (dynamic)

In Kibana i see the fields TotalMilliSec and AnyFloatNumber are saved as
string.. why??

Thanks
Rashmi

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/568cb316-5946-4fbc-ad87-f65a413b990d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Hi,

You can define mapping in ES Template.

Thanks,
Sumit

3

Hi Rashmi,

Do not you set “type” in Logstash configuration?

If you don’t set, logstash use “logs” for “index_type” in output elasticsearch.
See : https://github.com/elasticsearch/logstash/blob/1.4/lib/logstash/outputs/elasticsearch.rb#L300

Then you should change “logs” instead of “user” in your mappings in template.

Jun Ohtani
johtani@gmail.com
blog : http://blog.johtani.info
twitter : http://twitter.com/johtani

2014/11/25 3:39、Rashmi Gogia gogia.rashmi@gmail.com のメール：

Hi,

Could some one help me figure out what is the issue with below Logstash Configuration /mapping template . ES seems to ignore the mappings specified in the template and stores everything as string.

I am using Logstash csv plugin to create index. My logstash conf file has csv filter as below:

filter {
csv {
columns => ["ESpeedRef","ProcessName",TotalMilliSec,AnyFloatNumber]
separator => ","
}

output {
elasticsearch{
protocol => "http"
host => "localhost"
index => "test_index"
}
}

Below json template is used to define field mappings. I use curl to send template to ES.

{"template": "test_index",
"settings": {
"number_of_shards": 1
},
"mappings": {
"user": {
"properties": {
"ESpeedRef": {"type" : "string"},
"ProcessName": {"type" : "string"},
"TotalMilliSec": {"type" : "integer"},
"AnyFloatNumber": {"type" : "float"}
}
}
}
}

When creating index, Elastic Search does apply this template. I can confirm this my looking at below console stmts:

[2014-11-24 13:22:51,099][INFO ][cluster.metadata ] [Captain America] [
test_index] creating index, cause [auto(bulk api)], shards [1]/[1], mappings
[user]
[2014-11-24 13:22:51,146][INFO ][cluster.metadata ] [Captain America] [
test_index] update_mapping [logs] (dynamic)

In Kibana i see the fields TotalMilliSec and AnyFloatNumber are saved as string.. why??

Thanks
Rashmi

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/568cb316-5946-4fbc-ad87-f65a413b990d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/EE908F38-2DED-4624-A4CD-2E3DC9774C65%40gmail.com.
For more options, visit https://groups.google.com/d/optout.