Search on kibana discover dashboard

[artec (Davide)]

Hi all,
this is my first topic so I would say an "Hello!" at everyone !
I set up an environment with filebeat -> logstash -> es -> kibana to collect our application logs.
The events are like (stored in message field)

http> [29/11/2017 15:08:23] GET /multisession/x-2963163/Login
QRY> (1 row in 1~1 msec) Select * From TABLE1 Where ( IDUSER=? ) [1]
QRY> (0 rows in 0~0 msec) Select * From TABLE2 Where ( IDUSER=? ) [1]
http> Response in 6 msec. ResponseOk="/login/Home.html"

The problem is if I try a search with a regexp like /[0-9]{3}/ msec , it not consider an unique phrase.
Sorry for any mistake but I'm a newbie !

Davide

[jbudz (Jon Budzenski)]

Is your field mapped as text? Kibana's running a query string query which will end up passing this through elasticsearch's analyzer, causing it to tokenize on certain characters like spaces.

If it's possible I'd recommend parsing these logs out to individual fields before they reach elasticsearch. You'll be able to run aggregations and queries like show me all requests that take > 10 ms. You can do this with grok filters in either elasticsearch or logstash

[artec (Davide)]

Hi Jon,
thanks for your reply.
Yes, I set message like text and not_analyzed. As you suggest, I set the total request time (from my example, 6) in another field and works fine.
But if I would search in my events all the query (QRY> (0 rows in 0~0 msec) Select * From TABLE2 Where ( IDUSER=? ) [1]) that spent more than 3 msec, how can I do this ?

Thanks a lot
Davide

[jbudz (Jon Budzenski)]

If you run a match query it'll run both your query and your data through the analyzer - https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html#query-dsl-term-query for further reading. In the search bar in kibana I'd try something like:

{ "match": { "my_field": { "query": "(QRY> (0 rows in 0~0 msec) Select * From TABLE2 Where ( IDUSER=? ) [1])" } } }

or by creating a filter using the add filter button below the query bar.

Alternatively if you map it as keyword, it won't be run through an analyzer. One thing to keep in mind is there is a size limit on keyword fields at 32766 bytes(~15k characters), so if the queries are longer it may not be an option.