Help with mapping update

Elastic Stack Elasticsearch
1

Greetings,

I am trying to update mapping to make certain fields not_analyzed. The
index gets created from default template, now I'm trying to add the
following mapping to it (long) and get an error. The index is empty.

I'm probably missing something very obvious.

Thank you,

David

========

$ curl -XPUT localhost:9200/backend/mytype/_mapping -d mytype_mapping.json
{"error":"RemoteTransportException[[XXXX / Luis
Buñuel][inet[/10.XX.XX.XX:9333]][indices:admin/mapping/put]]; nested:
NullPointerException; ","status":500}(dk)dk-

========

updated mapping:
{
"mappings" : {

{"mytype" : {
"_ttl" : {
"enabled" : true,
"default" : 172800000
},
"properties" : {
"@timestamp" : {
"type" : "date",
"format" : "dateOptionalTime"
},
"@version" : {
"type" : "string"
},
"file" : {
"type" : "string"
},
"host" : {
"type" : "string"
},
"json" : {
"properties" : {
"XXXX.buffer.size" : {
"type" : "long"
},
"action" : {
"type" : "string","index" : "not_analyzed"
},

[ ....]

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6222b24f-7cf1-42ea-a0e6-68b52c4e9e76%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Try deleting the index and recreating using the updated mapping?

Also as a suggestion, avoid using TTL, it's very resource intensive and if
you are using time based data then you should use time based indices.

On 25 March 2015 at 08:28, David Kleiner david.kleiner@gmail.com wrote:

Greetings,

I am trying to update mapping to make certain fields not_analyzed. The
index gets created from default template, now I'm trying to add the
following mapping to it (long) and get an error. The index is empty.

I'm probably missing something very obvious.

Thank you,

David

========

$ curl -XPUT localhost:9200/backend/mytype/_mapping -d mytype_mapping.json
{"error":"RemoteTransportException[[XXXX / Luis
Buñuel][inet[/10.XX.XX.XX:9333]][indices:admin/mapping/put]]; nested:
NullPointerException; ","status":500}(dk)dk-

========

updated mapping:
{
"mappings" : {

{"mytype" : {
"_ttl" : {
"enabled" : true,
"default" : 172800000
},
"properties" : {
"@timestamp" : {
"type" : "date",
"format" : "dateOptionalTime"
},
"@version" : {
"type" : "string"
},
"file" : {
"type" : "string"
},
"host" : {
"type" : "string"
},
"json" : {
"properties" : {
"XXXX.buffer.size" : {
"type" : "long"
},
"action" : {
"type" : "string","index" : "not_analyzed"
},

[ ....]

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6222b24f-7cf1-42ea-a0e6-68b52c4e9e76%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6222b24f-7cf1-42ea-a0e6-68b52c4e9e76%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8rkdzDc-VXV1wx%2BoXBPAiAnoiHf6AkAETXYFc1Rvi8jg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thank you Mark, I'll keep trying, will also try elasticdump to load custom
mapping or play with templates.

Cheers,

David

On Tuesday, March 24, 2015 at 3:44:55 PM UTC-7, Mark Walkom wrote:

Try deleting the index and recreating using the updated mapping?

Also as a suggestion, avoid using TTL, it's very resource intensive and if
you are using time based data then you should use time based indices.

On 25 March 2015 at 08:28, David Kleiner <david....@gmail.com
<javascript:>> wrote:

Greetings,

I am trying to update mapping to make certain fields not_analyzed. The
index gets created from default template, now I'm trying to add the
following mapping to it (long) and get an error. The index is empty.

I'm probably missing something very obvious.

Thank you,

David

========

$ curl -XPUT localhost:9200/backend/mytype/_mapping -d mytype_mapping.json
{"error":"RemoteTransportException[[XXXX / Luis
Buñuel][inet[/10.XX.XX.XX:9333]][indices:admin/mapping/put]]; nested:
NullPointerException; ","status":500}(dk)dk-

========

updated mapping:
{
"mappings" : {

{"mytype" : {
"_ttl" : {
"enabled" : true,
"default" : 172800000
},
"properties" : {
"@timestamp" : {
"type" : "date",
"format" : "dateOptionalTime"
},
"@version" : {
"type" : "string"
},
"file" : {
"type" : "string"
},
"host" : {
"type" : "string"
},
"json" : {
"properties" : {
"XXXX.buffer.size" : {
"type" : "long"
},
"action" : {
"type" : "string","index" : "not_analyzed"
},

[ ....]

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/6222b24f-7cf1-42ea-a0e6-68b52c4e9e76%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/6222b24f-7cf1-42ea-a0e6-68b52c4e9e76%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7ffd5cda-ca1f-443e-b476-27f1d3765c1f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

I got the index template to work, turns out it's really straightforward. I
now do have .raw fields and all the world is green.

I'll add the template to my other index so it gets geoip mapping when I
roll to April.

Cheers,

David

On Tuesday, March 24, 2015 at 2:28:38 PM UTC-7, David Kleiner wrote:

Greetings,

I am trying to update mapping to make certain fields not_analyzed. The
index gets created from default template, now I'm trying to add the
following mapping to it (long) and get an error. The index is empty.

I'm probably missing something very obvious.

Thank you,

David

========

$ curl -XPUT localhost:9200/backend/mytype/_mapping -d mytype_mapping.json
{"error":"RemoteTransportException[[XXXX / Luis
Buñuel][inet[/10.XX.XX.XX:9333]][indices:admin/mapping/put]]; nested:
NullPointerException; ","status":500}(dk)dk-

========

updated mapping:
{
"mappings" : {

{"mytype" : {
"_ttl" : {
"enabled" : true,
"default" : 172800000
},
"properties" : {
"@timestamp" : {
"type" : "date",
"format" : "dateOptionalTime"
},
"@version" : {
"type" : "string"
},
"file" : {
"type" : "string"
},
"host" : {
"type" : "string"
},
"json" : {
"properties" : {
"XXXX.buffer.size" : {
"type" : "long"
},
"action" : {
"type" : "string","index" : "not_analyzed"
},

[ ....]

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4114d717-ae65-493f-8f7f-c2cec8567e2a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

My confusion just leveled-up :slight_smile:

So the mapping is updated, all string fields now have .raw copies but I
don't see them in Kibana. Perhaps my understanding of this mechanism is
incomplete and I need to manually copy fields to their .raw counterparts in
logstash so that ES query returns documents with .raw fields in them..

And I did remove the TTL so my cluster is happy again.

On Tuesday, March 24, 2015 at 4:36:05 PM UTC-7, David Kleiner wrote:

I got the index template to work, turns out it's really straightforward. I
now do have .raw fields and all the world is green.

I'll add the template to my other index so it gets geoip mapping when I
roll to April.

Cheers,

David

[trimmed]

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e09155d6-42fd-4342-a53e-201e3f638038%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

6

Are you using KB4 or 3?
If 4 then you need to refresh the fields under the index settings.

On 25 March 2015 at 11:06, David Kleiner david.kleiner@gmail.com wrote:

My confusion just leveled-up :slight_smile:

So the mapping is updated, all string fields now have .raw copies but I
don't see them in Kibana. Perhaps my understanding of this mechanism is
incomplete and I need to manually copy fields to their .raw counterparts in
logstash so that ES query returns documents with .raw fields in them..

And I did remove the TTL so my cluster is happy again.

On Tuesday, March 24, 2015 at 4:36:05 PM UTC-7, David Kleiner wrote:

I got the index template to work, turns out it's really straightforward.
I now do have .raw fields and all the world is green.

I'll add the template to my other index so it gets geoip mapping when I
roll to April.

Cheers,

David

[trimmed]

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e09155d6-42fd-4342-a53e-201e3f638038%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e09155d6-42fd-4342-a53e-201e3f638038%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X93PAVFmAfjODzhD93Hn3XJCKuuyco4NU5vXczc3aGyhg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

7

Indeed, it just works in 4!

Thank you again,

David

On Tuesday, March 24, 2015 at 5:49:16 PM UTC-7, Mark Walkom wrote:

Are you using KB4 or 3?
If 4 then you need to refresh the fields under the index settings.

On 25 March 2015 at 11:06, David Kleiner <david....@gmail.com
<javascript:>> wrote:

My confusion just leveled-up :slight_smile:

So the mapping is updated, all string fields now have .raw copies but I
don't see them in Kibana. Perhaps my understanding of this mechanism is
incomplete and I need to manually copy fields to their .raw counterparts in
logstash so that ES query returns documents with .raw fields in them..

And I did remove the TTL so my cluster is happy again.

On Tuesday, March 24, 2015 at 4:36:05 PM UTC-7, David Kleiner wrote:

I got the index template to work, turns out it's really straightforward.
I now do have .raw fields and all the world is green.

I'll add the template to my other index so it gets geoip mapping when I
roll to April.

Cheers,

David

[trimmed]

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e09155d6-42fd-4342-a53e-201e3f638038%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e09155d6-42fd-4342-a53e-201e3f638038%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/14929de4-fce5-4a49-aaa3-ebda189632b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.