Fields Mapping | not_analyzed

Elastic Stack Elasticsearch
1

Hi Friends,

I'm starting using the ELK Stack to create a sort of Log Archiving and Log Analyses tool to my team.

I have some firewalls and need change some fields to not_analyzed.

I don't know why, but when I load Kibana, all my fields seems as Analyzed. What is the right manner to apply fields mapping templates?

Thanks in advance for the help.

Here is my conf of Logstash:

.... output { if [type] == "netscreen_traffic" { elasticsearch { hosts => ["localhost:9200"] index => "firewall-traffic-%{+YYYY.MM.DD}" manage_template => false template_overwrite => true template_name => "firewall-traffic" #template => "/etc/logstash/templates/firewall-traffic.json" } }

Here is my mapping json:

{ "template" : "firewall-traffic", "mappings": { "netscreen_traffic": { "action": { "type": "string" }, "device": { "type": "string", "index": "not_analyzed" }, "device_id": { "type": "string", "index": "not_analyzed" }, "dst_ip": { "type": "ip" }, "dst_port": { "type": "string" }, "dst_xlated_ip": { "type": "ip" }, "dst_xlated_port": { "type": "string" }, "dst_zone": { "type": "string", "index": "not_analyzed" }, "duration": { "type": "string" }, "host": { "type": "string", "index": "not_analyzed" }, "message": { "type": "string" }, "policy_id": { "type": "string" }, "priority": { "type": "string" }, "proto": { "type": "string" }, "rcvd": { "type": "integer" }, "reason": { "type": "string", "index": "not_analyzed" }, "sent": { "type": "integer" }, "service": { "type": "string" }, "session_id": { "type": "string" }, "src_ip": { "type": "ip" }, "src_port": { "type": "string" }, "src_xlated_ip": { "type": "ip" }, "src_xlated_port": { "type": "string" }, "src_zone": { "type": "string", "index": "not_analyzed" }, "start_time": { "type": "date" }, "syslog5424_pri": { "type": "string" }, "tags": { "type": "string" }, "type": { "type": "string" } } } }

2

Which fields are you trying to set as not analysed?

3

Hi @warkolm, thanks for you awnser, these are the fields what I need don't be analyzed.

These fileds are defined in a custom pattern file, by grok filter.

4

Ok, just making sure :slight_smile:

Have you refreshed the index pattern in KB settings?
Has the template been properly applied to the index it is creating?

5

HI @warkolm,

I wrote everything from scratch. I get the default template to compare with mine and now I have generated this bellow.

I wasn't use properly the Properties before. :blush:

Now I rewrite, submit and applied with success the template to ElasticSearch .

Thanks in advance for you help and sorry my badly english.

{ "template": "firewall-cleanup-*", "settings": { "index.refresh_interval": "10s" }, "mappings": { "netscreen_traffic": { "properties": { "@timestamp": { "type": "date", "format": "strict_date_optional_time||epoch_millis" }, "@version": { "type": "string" }, "action": { "type": "string" }, "device": { "type": "string", "index": "not_analyzed" }, "device_id": { "type": "string", "index": "not_analyzed" }, "dst_ip": { "type": "ip" }, "dst_port": { "type": "string" }, "dst_xlated_ip": { "type": "ip" }, "dst_xlated_port": { "type": "string" }, "dst_zone": { "type": "string", "index": "not_analyzed" }, "duration": { "type": "string" }, "host": { "type": "string", "index": "not_analyzed" }, "message": { "type": "string" }, "policy_id": { "type": "string" }, "priority": { "type": "string" }, "proto": { "type": "string" }, "rcvd": { "type": "string" }, "reason": { "type": "string", "index": "not_analyzed" }, "sent": { "type": "string" }, "service": { "type": "string" }, "session_id": { "type": "string" }, "src_ip": { "type": "ip" }, "src_port": { "type": "string" }, "src_xlated_ip": { "type": "ip" }, "src_xlated_port": { "type": "string" }, "src_zone": { "type": "string", "index": "not_analyzed" }, "start_time": { "type": "string" }, "syslog5424_pri": { "type": "string" }, "tags": { "type": "string" }, "type": { "type": "string" } } } } }

6

And do the _types of the documents you are indexing match the index?

7

Hi @warkolm,

Thank you for you assistance. Now Its working well. :smiley:

Now my challenge is match other firewall model, based in JUNOS OS. I´ll open new topic to keep clean.