Regex query in search field of kibana dashboard

[Bhavana]

Hello Everyone,

I am using a regex query in search field of kibana dashboard.
my query is page: "http://www.somename.com/places/" *

result expected:

All the extension pages of "http://www.somename.com/places/" but i am getting few results with "http://www.somename.com/"

mapping for the index

PUT search_index
{
"mappings":
{
"all":
{
"properties" :
{
"page" :
{
"type" : "string",
"index": "not_analyzed"
}
}
}
}
}

Could you please let me know what query should i use to get pages http://www.somename.com/places/ extended to it .

Thank you

[bhavyarm]

Hi Bhavana,

Can you try this: page:http://www.somename.com/places/* and see what happens?

There is some good learning resources here for understanding queries in Kibana:

Elasticsearch/Kibana Queries - In Depth Tutorial

Thanks,
Bhavya

[Bhavana]

Thanks for the reply,

I tried "page:http://www.somename.com/places/*" it is returning zero results. I could not figure out the solution after reading the article too.

Page field is "not_analyzed" and all the letters in the URL are small letters as the article mentioned about case-sensitive in "not_analyzed" field .

[bhavyarm]

Hi Bhavana,

Wild cards should work on not_analyzed fields. Not sure what's really going on here. Can you please give us a sample document here? Will try to load it into my local machine and see whats up.

Thanks,
Bhavya

[bhavyarm]

Bhavana,

Try this: page:http://www.somename.com/places/* and let me know. I got help and found out that we might need to escape the colon.

Thanks,
Bhavya

[Bhavana]

page:http://www.somename.com/places/* didn't worked out.

sample document is as below:

        "_source": {
           "date": "2017-02-03 12:00:00",
           "clientID": "http://www.somename.com/,
           "page": "http://www.somename.com/places/xxxx",
           "country": "India"
        }

Mapping

"page": {
"type": "string",
"index": "not_analyzed"
}

Thank you,
Bhavana.

[bhavyarm]

Bhavana,

Can you please tell me which Kibana/ES version you are on?

Thanks,
Bhavya

[Bhavana]

Thank you for the reply,

ES version - 2.3.4 , Kibana version - 4.5.4.

[bhavyarm]

Hi Bhavana,

Can you also post us what error do you see?

Thanks,
Bhavya

[Bhavana]

I am not getting any error with the regular expression used,

Regex in the search field of kibana dashboard

page:"http://www.somename.com/places/"*

Result is same in the below cases

page:"http://www.somename.com/places/"*
page:"http://www.somename.com/"*

Result expected

All the extension pages of "http://www.somename.com/places/" but i am getting results with "http://www.somename.com/yyy/" too

Thank you,
Bhavana

[bhavyarm]

Can you try without using double quotes. And if you see a parser error please paste it in?

page:http://www.somename.com/places/*
page:http://www.somename.com/*

Thanks,
Bhavya

[Bhavana]

case 1)

page:http://www.somename.com/*

Error: RequesttoElasticsearchfailed: {
  "error": {
    "root_cause": [
      {
        "type": "query_parsing_exception",
        "reason": "Failed to parse query [page:http\\://www.somename.com/*]",
        "index": "allfields",
        "line": 1,
        "col": 206
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "allfields",
        "node": "Yoq",
        "reason": {
          "type": "query_parsing_exception",
          "reason": "Failed to parse query [page:http\\://www.somename.com/*]",
          "index": "allfields",
          "line": 1,
          "col": 206,
          "caused_by": {
            "type": "parse_exception",
            "reason": "Cannot parse 'page:http\\://www.somename.com/*': Lexical error at line 1, column 34.  Encountered: <EOF> after : \"/*\"",
            "caused_by": {
              "type": "token_mgr_error",
              "reason": "Lexical error at line 1, column 34.  Encountered: <EOF> after : \"/*\""
            }
          }
        }
      }
    ]
  }
}

case 2)

page:http://www.somename.com/places/*

No error but showing all the pages in the index.

[bhavyarm]

Hi Bhavana,

Try searching for page:places and keep us posted?

Thanks
Bhavya

[bhavyarm]

OK Bhavana. We got the answer from someone who actually co-wrote elasticsearch the definitive guide
Here is what you need to try:

The `/` need to be escaped as well, so `page.keyword:http\\:\\/\\/www.somename.com\\/places\\/*`, otherwise `//` is considered an empty regex

You can use this to debug your queries to understand what is really being searched.
POST search_index/_validate/query?explain
{
"query": {
"query_string": {
"query": "page.keyword:http\://www.somename.com\/places\/*",
"analyze_wildcard": true
}
}
}

Let us know what happens.

Thanks,
Bhavya

[Bhavana]

case 1) 
page.keyword:"http\\:\\/\\/www.somename.com\\/places\\/"*

showing all the values in the index

case 2) 
page.keyword:"http\\:\\/\\/www.somename.com\\/places\\/*"

No results

case 3)
page:http\\:\\/\\/www.somename.com\\/places\\/*

failed to parse query

case 4)
page.keyword:http\\:\\/\\/www.somename.com\\/places\\/*

failed to parse query

case 5)
"page.keyword:http\\:\\/\\/www.somename.com\\/places\\/*"

No results

case 6)
"page.keyword:http\\:\\/\\/www.somename.com\\/places\\/"*

showing all the values in the index

Could not able to get the required result with the above changes too,

Thank you,
Bhavana

[bhavyarm]

Bhavana,

Can you please try this?
page:http://www.somename.com/places/*

Thanks,
Bhavya

[Bhavana]

page:http://www.somename.com/places/* regex worked out perfectly to display all the extension pages of "http://www.somename.com/places/"

Great work.

Thank You,
Bhavana.