Regex query in search field of kibana dashboard

Elastic Stack Kibana
1

Hello Everyone,

I am using a regex query in search field of kibana dashboard.
my query is page: "http://www.somename.com/places/" *

result expected:

All the extension pages of "http://www.somename.com/places/" but i am getting few results with "http://www.somename.com/"

mapping for the index

PUT search_index
{
"mappings":
{
"all":
{
"properties" :
{
"page" :
{
"type" : "string",
"index": "not_analyzed"
}
}
}
}
}

Could you please let me know what query should i use to get pages http://www.somename.com/places/ extended to it .

Thank you

2

Hi Bhavana,

Can you try this: page:http://www.somename.com/places/* and see what happens?

There is some good learning resources here for understanding queries in Kibana:

Thanks,
Bhavya

3

Thanks for the reply,

I tried "page:http://www.somename.com/places/*" it is returning zero results. I could not figure out the solution after reading the article too.

Page field is "not_analyzed" and all the letters in the URL are small letters as the article mentioned about case-sensitive in "not_analyzed" field .

4

Hi Bhavana,

Wild cards should work on not_analyzed fields. Not sure what's really going on here. Can you please give us a sample document here? Will try to load it into my local machine and see whats up.

Thanks,
Bhavya

5

Bhavana,

Try this: page:http://www.somename.com/places/* and let me know. I got help and found out that we might need to escape the colon.

Thanks,
Bhavya

6

page:http://www.somename.com/places/* didn't worked out.

sample document is as below:

        "_source": {
           "date": "2017-02-03 12:00:00",
           "clientID": "http://www.somename.com/,
           "page": "http://www.somename.com/places/xxxx",
           "country": "India"
        }

Mapping

"page": {
"type": "string",
"index": "not_analyzed"
}

Thank you,
Bhavana.

7

Bhavana,

Can you please tell me which Kibana/ES version you are on?

Thanks,
Bhavya

8

Thank you for the reply,

ES version - 2.3.4 , Kibana version - 4.5.4.

9

Hi Bhavana,

Can you also post us what error do you see?

Thanks,
Bhavya

10

I am not getting any error with the regular expression used,

Regex in the search field of kibana dashboard

page:"http://www.somename.com/places/"*

Result is same in the below cases

page:"http://www.somename.com/places/"*
page:"http://www.somename.com/"*

Result expected

All the extension pages of "http://www.somename.com/places/" but i am getting results with "http://www.somename.com/yyy/" too

Thank you,
Bhavana

11

Can you try without using double quotes. And if you see a parser error please paste it in?

page:http://www.somename.com/places/*
page:http://www.somename.com/*

Thanks,
Bhavya

12

case 1)

page:http://www.somename.com/*

Error: RequesttoElasticsearchfailed: {
  "error": {
    "root_cause": [
      {
        "type": "query_parsing_exception",
        "reason": "Failed to parse query [page:http\\://www.somename.com/*]",
        "index": "allfields",
        "line": 1,
        "col": 206
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "allfields",
        "node": "Yoq",
        "reason": {
          "type": "query_parsing_exception",
          "reason": "Failed to parse query [page:http\\://www.somename.com/*]",
          "index": "allfields",
          "line": 1,
          "col": 206,
          "caused_by": {
            "type": "parse_exception",
            "reason": "Cannot parse 'page:http\\://www.somename.com/*': Lexical error at line 1, column 34.  Encountered: <EOF> after : \"/*\"",
            "caused_by": {
              "type": "token_mgr_error",
              "reason": "Lexical error at line 1, column 34.  Encountered: <EOF> after : \"/*\""
            }
          }
        }
      }
    ]
  }
}

case 2)

page:http://www.somename.com/places/*

No error but showing all the pages in the index.

14

Hi Bhavana,

Try searching for page:places and keep us posted?

Thanks
Bhavya

15

OK Bhavana. We got the answer from someone who actually co-wrote elasticsearch the definitive guide :smiley:
Here is what you need to try:

The `/` need to be escaped as well, so `page.keyword:http\\:\\/\\/www.somename.com\\/places\\/*`, otherwise `//` is considered an empty regex

You can use this to debug your queries to understand what is really being searched.
POST search_index/_validate/query?explain
{
"query": {
"query_string": {
"query": "page.keyword:http\://www.somename.com\/places\/*",
"analyze_wildcard": true
}
}
}

Let us know what happens.

Thanks,
Bhavya

16 
case 1) 
page.keyword:"http\\:\\/\\/www.somename.com\\/places\\/"*

showing all the values in the index

case 2) 
page.keyword:"http\\:\\/\\/www.somename.com\\/places\\/*"

No results

case 3)
page:http\\:\\/\\/www.somename.com\\/places\\/*

failed to parse query

case 4)
page.keyword:http\\:\\/\\/www.somename.com\\/places\\/*

failed to parse query

case 5)
"page.keyword:http\\:\\/\\/www.somename.com\\/places\\/*"

No results

case 6)
"page.keyword:http\\:\\/\\/www.somename.com\\/places\\/"*

showing all the values in the index

Could not able to get the required result with the above changes too,

Thank you,
Bhavana

17

Bhavana,

Can you please try this?
page:http://www.somename.com/places/*

Thanks,
Bhavya

18

page:http://www.somename.com/places/* regex worked out perfectly to display all the extension pages of "http://www.somename.com/places/"

Great work.

Thank You,
Bhavana.