Query a range of values in phrase

Elastic Stack Kibana
1

Hi
I already have a log in the message field as below

Message:Info matched and sent Http/1.2 200(345).
Message:Info mismatch http/1.2 404(21)

The value 200 is the status , but I don't have it in a separate field. How can I filter range of values in a text field. I'm trying to filter the logs based on their status.

Please suggest me on this

Thanks in advance.

2

Hi @Sahithi_Muddasani

You can try using wildcard query

{"query":{ "wildcard": { "message.keyword" : "*200*" }}}

You can also check Regexp query, to compare and see what works better for you.

Regards, Dzmitry

3

Hi Dzmitry
I tried using wildcard but it's giving me all the results which have 2 .I wanted to have only results that are followed by receive1.1 and should not also filter out if 2 is present in time i.e (28ms)
I have message in the log as
Info matched 2 , Post v/consumer/receive1.1 200(16ms).
Info matched 2, post v/consumer/receive1.1 404 (28ms)

So when I tried using * it is giving me and ? Is not working here.

4

You can try building a regular expression and use Regexp query to match your criteria, but it will make your search much slower. You can also create a scripted field with response code, but it still slows down your search experience.

Creating a pipeline that adds a field with the response code based on your rules, then reindex documents and use field filtering in Discover. Fixing data design is a better way to make things work fast and reliable.

Regards, Dzmitry

5

I tried this scripted field but the format on some messages (space or no space between response code and "(" is different so this maybe isn't a good solution;

In the index pattern, scripted fields tab, I added this number type scripted field. It finds the "(" char and then gets substring starting 3 chars in front of that and up to that "(".

doc['Message.keyword'].value.substring(doc['Message.keyword'].value.indexOf("(")-3, doc['Message.keyword'].value.indexOf("("))

image
image2048×707 48.1 KB

So it's really better to improve the ingest side to get the different parts of the message into separate fields.

6

Thank you @Dzmitry and @LeeDr.
I can't edit the data design so used scripted field and it worked perfectly for my requirement.

7

I created a scripted field "responsecode" Of number type. I'm unable to use this field to filter. Whenever I try, it says few shards failed.

I'm storing a substring from message field in responsecode field. I assumed the particular substring is still in string format so I'm getting some error and tried parseInt method. Even that didn't work, I got result as . I checked for mapping of this particular field in index from console. Result was
"Index":{
"Mappings:{}
}

I wanted to use the scripted field to filter based on range of response codes.