Need to filter out data which is greater than somehting


(R) #1

Hi Guys,

I need to build visualization our of data where I need to filter out on one field which is greater than 1500. How do I do that can someone please advise?


(Peter Pisljar) #2

Which version of kibana are you using ? In latest versions in should be as simple as:

  • click add new filter on the filter bar
  • select the field you are interested in
  • select greater than in the operation dropdown
  • enter 1500 in the value field

(R) #3

are you talking about 6.x? I am still using 5.x. How is that possible in Kibana 5


(R) #4

to be specific its 5.6


(Peter Pisljar) #5

you could type something like fieldname: >1500 in the query bar


(R) #6

nah that is not happeining :frowning:


(Peter Pisljar) #7

whats the result you are getting vs the result you are expecting ? could you copy paste request and response from spy panel (click the little up arrow at the bottom of your visualization)


(R) #8

here is the json which from Kibana

{

"_index": "logstash-isnhp-2017.12.19",
"_type": "doc",
"_id": "AWBvFsZ3DOVoqfVcKq0O",
"_version": 1,
"_score": null,
"_source": {
"lastseen": "last_seen",
"@timestamp": "2017-12-19T14:04:26.736Z",
"geoip": {
"timezone": "America/Los_Angeles",
"ip": "107.175.49.23",
"latitude": 34.0494,
"continent_code": "NA",
"city_name": "Los Angeles",
"country_name": "United States",
"country_code2": "US",
"dma_code": 803,
"country_code3": "US",
"region_name": "California",
"location": {
"lon": -118.2641,
"lat": 34.0494
},
"postal_code": "90014",
"region_code": "CA",
"longitude": -118.2641
},
"month": "12",
"year": "2017",
"@version": "1",
"host": [
"remote_host"
],
"count": "count",
"message": "{"remote_host":"107.175.49.23","count":"2","last_seen":"2017-12-19"}",
"ipaddr": "107.175.49.23",
"day": "19",
"nooftimes": "2"
},
"fields": {
"@timestamp": [
1513692266736
]
},
"highlight": {
"nooftimes": [
"@kibana-highlighted-field@2@/kibana-highlighted-field@"
]
},
"sort": [
1513692266736
]
}

DO you see "nooftimes": "2" I want to build a dashboard which has "nooftimes": "> 1500"

here are the parsers

filter {
grok {
match => [ "message", '%{WORD:host}":"%{IPV4:ipaddr}","%{WORD:count}":"%{NUMBER:nooftimes}","%{WORD:lastseen}":"%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDA
Y:day}"' ]
}
geoip { source => "ipaddr" }

}


(Peter Pisljar) #9

open spy panel and copy paste Request and Response tabs. also please provide the exact thing you have in your query bar.

thanks


(Peter Pisljar) #10

also, here is the documentation on lucene query syntax (which query bar uses)

https://www.elastic.co/guide/en/elasticsearch/reference/5.6/query-dsl-query-string-query.html#query-string-syntax


(R) #11

Yep I am trying that but somehow dang the query is not working out...or lets say I want to filter out events in discover tab instead of building visualization or dashboard


(Peter Pisljar) #12

from the document you passed it looks like nooftimes is indexed as string, should be indexed as number for this to work

can you check management->index patterns->select your index pattern and locate your field ... what type is it ?


(R) #13

Yeah that is correct its parsed as a string..though if you check in my logstash config this has been parsed as a NUMBER. What it should be then? Or can I edit that in kibana?


(Peter Pisljar) #14

that won't define how the field is indexed in elasticsearch.

elasticsearch does auto recognize the field type based on the first indexed document

so if your first document had a string in that place, that will be string for all the following documents.

there is a way to force the field type using the elastic search mappings

at this point you will probably need to reindex all your documents to get this to work.

you could also use scripted fields to do the conversion on-the-fly, however that will use a lot of resources and might not be performant enough (specially if your dataset is big)


(R) #15

hmm thats right..this may be the issue with the template I used? I Guess mapping was done using that template.


(system) #16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.