This simple Kibana query has me flummoxed

Elastic Stack Kibana
1

I'm trying to narrow down a search based on something that should be a very easy thing to ask. I have a substring that will only sometimes show up in my messages with the following format:

12345@foo

Initially I thought to use regex to solve this, but I get no results:
/[0-9]{5}\@foo/

No results. Same with this:
/[0-9]{5}@foo/

So tried just this, which worked:
/[0-9]{5}/

But that isn't specific enough. So then, just to isolate the problem I tried the following queries. None of them return results:
@
\@
/\@/
"@"
"\@"

... and this just returns everything:
/@/

Something about the "at" character is fubar. I don't know what it is or how to fix it. Can I get an assist please?

2

Hi Brandon,

If the field you're searching was processed with the standard analyzer then the @ symbol was probably treated as a separator so you have terms "12345" and "foo", but not "12345@foo". But it would be in a 'raw' or 'unanalyzed' field.
Here's a great guide to understand the issue; https://www.timroes.de/2016/05/29/elasticsearch-kibana-queries-in-depth-tutorial/#using-json-in-the-kibana-search

If I go to the Dev Tools > Console I can do a request like this;

GET makelogs-0/_mappings

and in the response I'll look for a field like @message and see that there it is "type": "test", but there's a field called "raw" with "type":"keyword" (which is not analyzed);

        "properties": {
          "@message": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword"
              }
            }
          },

If I search on @message:"Mozilla/4.0" I do get these hits. But notice that the highlighting in the results doesn't include the / because it's a separator and not part of the analyzed terms it searched.
And I get the same results if I replace / with -.

image
image.png2366×821 73.5 KB

Kibana doesn't let me search the .raw field in Discover so I can't change my query to that.

But in Visualizations I can do a terms aggregation on the .raw field, and I can search on exact strings. For example, this works for my data and it only returns documents that match this exact string;

@message.raw:"0.113.226.80 - - [2017-11-28T15:12:34.663Z] "GET /uploads/gregory-harbaugh.jpg HTTP/1.1" 200 9757 "-" "Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.50 Safari/534.24""

Regards,
Lee

3

Lee,

Thank you for the detailed reply! This is exactly the explanation I was looking for.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.