KV-Pairs and type conversion

Hello,

I have an input like this

<14>1 2017-09-11T14:00:27.582+02:00 SECRETHOSTNAME RT_FLOW - RT_FLOW_SESSION_CREATE [junos@2636.1.1.1.2.40 source-address="1.1.1.1" source-port="53780" destination-address="2.2.2.2" destination-port="389" service-name="junos-ldap" nat-source-address="3.3.3.3" nat-source-port="53780" nat-destination-address="4.4.4.4" nat-destination-port="389" src-nat-rule-type="N/A" src-nat-rule-name="N/A" dst-nat-rule-type="N/A" dst-nat-rule-name="N/A" protocol-id="6" policy-name="newNetAccess" source-zone-name="BLUB" destination-zone-name="BLAA" session-id-32="28173" username="N/A" roles="N/A" packet-incoming-interface="reth0.999" application="UNKNOWN" nested-application="UNKNOWN" encrypted="UNKNOWN"]

I managed to extract all fields I need with kv-pairs filter.
But... everything seems to be a string, so I cannot draw and things like that as ip-addresses are not ip-addresses.
I searched a lot now, but I can't find a solution.

this is what I am doing now and my question is how can I determine for each pair what type of data it is?

if "RT_FLOW" in [message] {
        grok {
        match => [ "message","%{GREEDYDATA:kvpairs}" ]
  }

kv {
   source => "kvpairs"
}

thank you very much in advance

//desete

But... everything seems to be a string, so I cannot draw and things like that as ip-addresses are not ip-addresses.

Whether a field is mapped as an IP address depends on the mappings of the ES index. You can use index templates to explicitly map fields as e.g. IP addresses.

You can use a mutate filter and its convert option to convert string fields in Logstash to numbers but even though you have fields containing numbers (like ports) they're not numbers that you'd typically process numerically (like compute sums or averages) so it's not terribly important.

thanks Magnus.

You can use index templates to explicitly map fields as e.g. IP addresses.

any recommended reading?

thx and br
//seb

https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/ip.html

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.