Kv filter values is always returned as string

[kgdi]

Hi,

I am using the kv filter to get values from extra_fields and it works fine, except it shows up as string in the index and not number.

Is there any limitations in using mutate on kv generated values?

One entry of the "extra_fields":

"-" "stage.domain.no" sn="" rt=0.054 ua="10.4.44.4:10553" us="302" ut="0.052" ul="0" cs=-"GET /identity/connect/authorize?response_type=id_token%20token&client_id=client&state=K6EacsCotqv28WKl7729EHP&redirect_uri=https%3A%2F%2Fstage.domain.no%2F&scope=openid%20api1&nonce=K6EacsCotqv2Kl7729EHP HTTP/1.1" 302 0 "https://stage.domain.no/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "-" "-" "-" "-" "-"

My config:

kv {
        source => extra_fields
 }
 mutate {
   convert => ["rt", "integer"]
 }

The rt value is still shown as string in the index. I tried to re-create the index without luck.

[magnusbaeck (Magnus Bäck)]

What does an example event stored in ES look like?

Is there any limitations in using mutate on kv generated values?

No.

[kgdi]

Ok, great.

Here is an example of the extra_fields field:

[magnusbaeck (Magnus Bäck)]

Please copy/paste the raw JSON from the JSON tab in Kibana.

[kgdi]

{
"_index": "nginx-stage-2018.09.03",
"_type": "nginx_logs",
"_id": "MnCSn2UB6On70KxsT87W",
"_version": 1,
"_score": null,
"_source": {
"input": {
"type": "log"
},
"sn": """",
"rt": "0.050",
"@timestamp": "2018-09-03T13:14:45.000Z",
"auth": "-",
"source": "/var/log/nginx/access.log",
"ut": "0.052",
"request": "/identity/connect/authorize?response_type=id_token%20token&client_id=client&state=4Tz8eHEkZOxlkLcPigTObZtDR3NqE2tM6hkE&redirect_uri=https%3A%2F%2Fstage.domain.no%2F&scope=openid%20api1&nonce=4Tz8eHEkZOxlkLcPik8LcgTObZtDR3NqE2tM6hkE",
"referrer": ""https://stage.domain.no/\"",
"host": {
"name": "o-sf-lb01"
},
"httpversion": "1.1",
"name": "Chrome",
"response": 302,
"tags": [
"beats_input_codec_plain_applied",
"nginx-geoip"
],
"os": "Windows 8.1",
"beat": {
"name": "o-sf-lb01",
"hostname": "o-sf-lb01",
"version": "6.4.0"
},
"offset": 5717103,
"us": "302",
"ul": "0",
"cs": "-"GET",
"extra_fields": " "-" "stage.domain.no" sn="" rt=0.050 ua="10.33.33.44:13333" us="302" ut="0.052" ul="0" cs=-"GET /identity/connect/authorize?response_type=id_token%20token&client_id=client&state=4Tz8eHEkZOxlkik8LcgTObZtDR3NqE2tM6hkE&redirect_uri=https%3A%2F%2Fstage.domain.no%2F&scope=openid%20api1&nonce=4Tz8eHEkZOxlkLcPik8LcgTObZtDR3NqE2tM6hkE HTTP/1.1" 302 0 "https://stage.domain.no/\" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "-" "-" "-" "-" "-"",
"message": "33.33.22.22 - - [03/Sep/2018:15:14:45 +0200] "GET /identity/connect/authorize?response_type=id_token%20token&client_id=client&state=4Tz8eHEkZOxlkLcPiTObZtDR3NqE2tM6hkE&redirect_uri=https%3A%2F%2Fstage.domain.no%2F&scope=openid%20api1&nonce=4Tz8eHEkZOxlkLcPik8LcgTObZtDR3NqE2tM6hkE HTTP/1.1" 302 0 "https://stage.domain.no/\" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "stage.domain.no" sn="" rt=0.050 ua="10.33.33.44:13333" us="302" ut="0.052" ul="0" cs=-"GET /identity/connect/authorize?response_type=id_token%20token&client_id=client&state=4Tz8eHEkZOxlkLcPik8LcgTObZtDR3NqE2tM6hkE&redirect_uri=https%3A%2F%2Fstage.domain.no%2F&scope=openid%20api1&nonce=4Tz8eHEkZOxlkLcPik8LcgTObZtDR3NqE2tM6hkE HTTP/1.1" 302 0 "https://stage.domain.no/\" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36" "-" "-" "-" "-" "-" "-"",
"prospector": {
"type": "log"
},
"clientip": "33.33.22.22",
"geoip": {
"city_name": "Oslo",
"country_code3": "NO",
"country_code2": "NO",
"timezone": "Europe/Oslo",
"ip": "33.33.22.22",
"country_name": "Norway",
"region_code": "03",
"postal_code": "0001",
"longitude": 10.7487,
"latitude": 59.905,
"region_name": "Oslo County",
"continent_code": "EU",
"location": {
"lon": 10.7487,
"lat": 59.905
}
},
"agent": ""Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36"",
"bytes": 0,
"major": "68",
"patch": "3440",
"build": "",
"/identity/connect/authorize?response_type": [
"id_token%20token&client_id=client&state=4Tz8eHEkZOxlkLcPik8LcgTObZtDR3NqE2tM6hkE&redirect_uri=https%3A%2F%2Fstage.domain.no%2F&scope=openid%20api1&nonce=4Tz8eHEkZOxlkLcPik8LcgTObZtDR3NqE2tM6hkE",
"id_token%20token&client_id=client&state=4Tz8eHEkZOxlkLcPik8LcgTObZtDR3NqE2tM6hkE&redirect_uri=https%3A%2F%2Fstage.domain.no%2F&scope=openid%20api1&nonce=4Tz8eHEkZOxlkLcPik8LcgTObZtDR3NqE2tM6hkE"
],
"ident": "-",
"verb": "GET",
"os_name": "Windows 8.1",
"ua": "10.33.33.44:13333",
"@version": "1",
"device": "Other",
"minor": "0"
},
"fields": {
"@timestamp": [
"2018-09-03T13:14:45.000Z"
]
},
"sort": [
1535980485000
]
}

[kgdi]

I got it to work.

I deleted the index and the index pattern and rt shows up as "number" now

Thank you for your fast replies @magnusbaeck