Set field as not_analyzed in Elastic Search

Elastic Stack Elasticsearch
1

I have a string field in my Elastic search index.When I view it in Kibana it splits it to form multiple words, I want to prevent this from happenening.

This is the warning I get in Kibana:

Careful! The field selected contains analyzed strings. Analyzed strings are highly unique and can use a lot of memory to visualize. Values such as foo-bar will be broken into foo and bar. See Mapping Types for more information on setting this field as not_analyzed.

I use Elastic search for logging and I have a new log index created everyday which has the following mask - "log-YYYY.MM.DD".

When I run http://localhost:9200/log-*/_mapping, i get the followin output:

{"log-2016.08.22":{"mappings":{"logEvent":{"properties":{"className":{"type":"string"},"domain":{"type":"string"},"exception":{"type":"object"},"fileName":{"type":"string"},"fix":{"type":"string"},"fullInfo":{"type":"string"},"hostName":{"type":"string"},"identity":{"type":"string"},"level":{"type":"string"},"lineNumber":{"type":"string"},"loggerName":{"type":"string"},"message":{"type":"string"},"messageObject":{"properties":{"ID":{"type":"long"},"Message":{"type":"string"},"Type":{"type":"string"}}},"methodName":{"type":"string"},"properties":{"properties":{"@timestamp":{"type":"date","format":"strict_date_optional_time||epoch_millis"},"log4net:HostName":{"type":"string"},"log4net:Identity":{"type":"string"},"log4net:UserName":{"type":"string"}}},"threadName":{"type":"string"},"timeStamp":{"type":"date","format":"strict_date_optional_time||epoch_millis"},"userName":{"type":"string"}}}}},"log-2016.08.19":{"mappings":{"logEvent":{"properties":{"className":{"type":"string"},"domain":{"type":"string"},"exception":{"type":"object"},"fileName":{"type":"string"},"fix":{"type":"string"},"fullInfo":{"type":"string"},"hostName":{"type":"string"},"identity":{"type":"string"},"level":{"type":"string"},"lineNumber":{"type":"string"},"loggerName":{"type":"string"},"message":{"type":"string"},"messageObject":{"properties":{"ID":{"type":"long"},"Message":{"type":"string"},"Type":{"type":"string"}}},"methodName":{"type":"string"},"properties":{"properties":{"@timestamp":{"type":"date","format":"strict_date_optional_time||epoch_millis"},"log4net:HostName":{"type":"string"},"log4net:Identity":{"type":"string"},"log4net:UserName":{"type":"string"}}},"threadName":{"type":"string"},"timeStamp":{"type":"date","format":"strict_date_optional_time||epoch_millis"},"userName":{"type":"string"}}}}}}

My question is how can I make "Message":{"type":"string"} not analyzed for all the indexes which start with log-*?What command do I run to do this?Or how can I make ALL strings not_analyzed?Can someone please show me a step-by-step example?

2

no step by step example,you should re-mapping your index to renew your filed into not analyzed

3

Ok and how do I do this?What command do I run to re-map my index?

4

What version are you on?

5

Use an index template to define the desired mappings for new indexes. You might want to use Logstash's index template for logstash-* indexes as a starting point.

6

2.3.5

7

Can you give me an example?

8

Can you give me an example?

Example of what, exactly?

9

Example of a command I can execute in the browser or using CURL to make the Message field not_analyzed

10

curl -XPOST 'localhost:9200/myindex/_close'

curl -XPUT 'localhost:9200/myindex/_settings' -d '{
"analysis" : {
"analyzer":{
"content":{
"type":"custom",
"tokenizer":"whitespace"
}
}
}
}'

11

https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html

12

I can't use myindex it needs to be a wild card.As I've mentioned in my original post I have indexes created dynamically with the following names "log-YYYY.MM.DD" so the not_analyzed needs to be applied to all indexes starting with log- I tried taking your code and changing it to log- and log but when I paste it in the curl console it says > } is not recognized as internal or external command

13

1.how many index do you have now?
2.reindex the data though as it is not possible to change mapping for an existing field.

i suggent you create new index(just alter the properties "not_analyzed") ,and import the old data into the new index.

14

Example of a command I can execute in the browser or using CURL to make the Message field not_analyzed

See Index templates | Elasticsearch Guide [8.11] | Elastic for more about index templates. Here's the template used by Logstash 2.3.4: https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/v2.7.1/lib/logstash/outputs/elasticsearch/elasticsearch-template.json See the definition of the @version for an example of how a string field can be made not_analyzed.

15

1)I have one at the moment and it's called - log-2016.08.22 tomorrow log-2016.08.23 will automatically be created and so on

2)How can I create a new index???My indexes are created dynamically and have have X number of properties.What I do know for a fact is that I will always have a field called Message in my index and I need some way to make this field not_analyzed for any index called log-*

16

Ok so I took that JSON and changed the template field to be "log-*" and I changed @version to be @Message because my field is called Message but when I copy and paste the JSON into the command prompt I get a message saying } is not recognized as an internal or external command what am I doing wrong?

17

How can I create a new index?

With the create index API, for example.

but when I copy and paste the JSON into the command prompt I get a message saying } is not recognized as an internal or external command

Perhaps you're not single-quoting the JSON string? See the previous example from @emperor.

18

I have wrapped my json in single quotes.Here is my batch file:

curl PUT /_template/template_1
'{
"template": "log-*",
"settings": {
"number_of_shards": 1
},
"mappings": {
"type1": {
"_source": {
"enabled": false
},
"properties": {
"Message": {
"type": "string",
"index": "not_analyzed"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z YYYY"
}
}
}
}
}'

pause

19

When I run the batch file it still says '}' is not recognized as an internal or external command

Capture.PNG806×494 4.17 KB

20

Please follow @emperor's example. (It's -XPUT not PUT and you're missing -d. )