Logstash conf file with not_analyzed field

Elastic Stack Logstash
1

Hi ,

I want to load my Json file to the Elasticsearch and add i need that one of the fields will be not_analyzed
This is my conf file:
input {
file{
path => ["/root/scripts/vdm-server_10-4_parasoft_103.json"]
type => "json"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter{
json {
source => "message"
}
}

filter{
date {
match => ["create_time", "MM/dd/yyyy"]
target => "@timestamp"
}
}

output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => localhost
index => index_tmp1
}
}

example for a line from the json file:
{"project":"server" ,"branch":"10" ,"type":"Flow" ,"build_number":103 ,"severity":"Severity 1 = Highest" ,"line":804 ,"tool":"c++test" ,"pkg":"" ,"msg":"prior to initialization" ,"locfile":"/build_scripts/external/minizip/src333/zip.c" ,"rule":"BD-1" ,"auth":"hudson" ,"create_time":"06/05/2016" }

The field I need as not analyzed is "severity".

Thanks :slight_smile:
Chen

2

You need to do that in ES via a mapping or a template, see https://www.elastic.co/guide/en/elasticsearch/guide/current/mapping-analysis.html

3

Is there any way to define not_analyzed field in the conf file of the Logstash ?

4

No, it is an ES concept, not an LS one.

5

So if i want to split a Chart in the Kibana by this field , how can I do it ?

6

This is Logstash forum. You should start a thread in Kibana.

7

So if i want to split a Chart in the Kibana by this field , how can I do it ?

If you don't want the field to be analyzed and split into tokens, modify the indexes' mappings by changing the index template.

8

Thanks :slight_smile:

9

Can we not configure the template directly from logstash config file as indicated by the documentation like:

elasticsearch {
  hosts => localhost
  index => index_tmp1
  template => "/path/to/mytemplate"
  template_overwrite => true
}

Since the log transformation is control from the logstash config, it makes sense to place the mapping definition also on the logstash side.

10

Can we not configure the template directly from logstash config file as indicated by the documentation

Sure you can.

11

Hi @magnusbaeck , I did not find this in Logstash Online Doc. How can you configure the template directly from logstash config file ?
Thanks in advance, good day.
Antonin

12

Look at the template-related options listed in the documentation of the elasticsearch output plugin.