Grok will not change field type to Int or IP

Elastic Stack Logstash
1

While trying to match my VPN connections with the match below :

 match => [ "message", "%{WORD:openvpn_user}/%{IP:openvpn_scr_ip}:%{INT:openvpn_scr_port} MULTI_sva: pool returned IPv4=%{IP:openvpn_ip}" ]

I get my logs from my firewall and then add the tag VPN when i find VPN in the message.Then if tag-VPN exists i use grok{} with this match.
Fields get parched and then the index gets created all the fields that i wanted to make IP or INT or w/e come out String.Where should i look to fix that?

Thanks a lot.

2

To set fields to the ip type, modify the index template used by Elasticsearch (see options for Logstash's elasticsearch output). You can set fields to be integers there too, but you should also change %{INT:openvpn_scr_port} to %{INT:openvpn_scr_port:int} to convert the Logstash field to an integer. This will affect ES's autodetection so that should give you an integer field even without modifying the index template, but since you need to modify the index template anyway you might as well fix the integer fields.

1 Like
3

Ok i changed the port to be number(shows up as string but the actual type has changed in the json so i guess in the new index it shall take place).

Since i have not made the whole setup in this system can you tell me where i can find the path to the template the elastic uses for the index containing these mappings?

Thanks in your really fast response.

4

Have a look at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-*-java/lib/logstash/outputs/elasticsearch/elasticsearch-template.json.

5

Ok great so far found the template.

If i get it right from documentation i should add:

"properties" : {
"vpn":{
"type" : "object",
"dynamic" : true,
"properties": {
"openvpn_scr_ip" : { "type" : "ip" , "doc_values" : true },
etc,
etc
}
}
}

sorry if i am asking 2much here but i wouldn't like to break anything.Seriously thanks a lot for all the help so far.

6

You don't need "type": "object" and "dynamic": true both otherwise it looks okay.