Newbee with logstash an JSON

Elastic Stack Logstash
1

Hi,
I'm polling a JSON with th following config file:

input {
http_poller {
urls => {
myresource => "http://192.168.32.40/json/daily.json"
}
request_timeout => 60
interval => 60
codec => "json"
}
}

output {
elasticsearch {}
stdout{codec => rubydebug}
}

It works but all data as cast as string: i thinh it's beacause all numeric values are between quote.

This is stdout:

{
"title" => "METEO",
"location" => "France",
"time" => "14-Jun-2016 09:44",
"lat" => "43° 42.06' N",
"lon" => "001° 02.34' E",
"alt" => "213 metres",
"hardware" => "WMRS200",
"uptime" => "12 days, 20 hours, 27 minutes",
"serverUptime" => "10 days, 1 hour, 8 minutes",
"weewxVersion" => "3.4.0",
"stats" => {
"current" => {
"outTemp" => "19.7",
"windchill" => "19.7",
"heatIndex" => "19.7",
"dewpoint" => "10.9",
"humidity" => "57",
"insideHumidity" => "45",
"barometer" => "1008.6",
"barometerTrendDelta" => "3 hours",
"barometerTrendData" => "-1.4 mbar",
"windSpeed" => "7",
"windDir" => "246",
"windDirText" => "OSO",
"windGust" => "15",

So, how can i filter values to convert string to float ??

Thank you

2

Use the mutate filter's convert option.

3

Hi Magnus,

I have already try to cast in my case outTemp :

filter {
mutate {
convert => { "outTemp" => "float" }
}
}

But no effect ... always string in kabana

4

That's probably because the field's mapping in ES is still a string. Mappings can't be changed for an index without reindexing. Assuming you have the default time-series indexes you can just wait until the next day. (But you can already verify that the raw document in ES has outTemp as a number rather than a string.)

5

Hi Magnus,

I have wait next day:
Looking in the metadata with head :disappointed: :

"outTemp": {

"fielddata": {
    "format": "disabled"
},
"norms": {
    "enabled": false
},
"type": "string",
"fields": {
    "raw": {
        "ignore_above": 256,
        "index": "not_analyzed",
        "type": "string"
    }
}
6

just FYI i'm runing logstash et elasticsearch on Windows...

7

That's surprising. But what does the raw document look like? Specifically, what data type does the outTemp field have? Using a stdout { codec => rubydebug } makes it easy to check this.