Search for specific symbols


(Nik) #1

I'm using ELK on AWS. I want to analyze AWS ELB (load balancer) logs, and later on, I would love to see here Nginx logs as well.

I've setup everything and it works! So, thanks for your software guys!

Meanwhile, I have a small issue.

I've configured insertion logs into search cluster, then I went to Kibana UI and choose the default index (which is: elblogs-*, in my case).

Then I executed a mapping update to treat some fields in a way which will allow me to search for specific symbols (like: -, /, \ etc).

Here is the query:
curl -XPUT 'https://awses-domain/elblogs*/_mapping/elb-access-logs' --data "@update_mapper_for_raw.json"

Here is the content of the file:

{
	"properties": {
		"request_uri_path": {
			"type": "text",
			"fields": {
				"raw": {
					"type": "keyword",
					"index": "not_analyzed"
				}
			}
		}
	}
}

Then I've double checked the mapping
curl -XGET 'https://awses-domain/elblogs*/_mapping

and I can see my changes here

{
  "elblogs-2017.08.17": {
    "aliases": {},
    "mappings": {
      "elb-access-logs": {
        "properties": {
          ...
         	"request_uri_path": {
	        "type": "text",
	        "fields": {
        	  "keyword": {
	    	     "type": "keyword",
		          "ignore_above": 256
 		      },
              "raw": {
     	     	"type": "keyword"
	          }
     	    }
          },
          ...
        }
      }
    },
    "settings": {
      "index": {
        "creation_date": "1502960475384",
        "number_of_shards": "5",
        "number_of_replicas": "1",
        "uuid": "TcdTaIooTwqnXBkFDV_qMg",
        "version": {
          "created": "5030299"
        },
        "provided_name": "elblogs-2017.08.17"
      }
    }
  }
}

Even though in Kibana I can see it is as searchable

I can see no result on any type of the search against this field, I can't see this field either in any other search result. How can I fix it?


(Jim Unger) #2

Two questions.

  1. Did you click on the refresh button in your index pattern?

  2. Did you index new documents after you updated your mapping? Updating your mapping will not modify any of the documents that have already been indexed.


(Nik) #3

Thanks for your questions and yes my answer is "Yes" for both of them.

Regarding the search I think I start getting any of the results but I'm confused about - symbol and slash \

Because now if I insert into the search field (Kubana UI) following text

request_ur_path.raw=mysearchword\-*

Kibana shows me results like

  • mysearchword-newthing
  • newthing-mysearchword/bla/bla
  • and even this blabla-mysearchword

then how I can specify that I want to find a word + dash + any other word but not the end of the string, not slash etc.. kinda regex?

Thanks


(Jim Unger) #4

Yes, it is acting as a regular expression (although elasticsearch/kibana use a specific flavor of regex)


(Nik) #5

Oh, Ok I see what is wrong, it is the wrong syntax, the right one is

request_uri_path.raw:mysearchkeyword I mean field_name:search_value and before I've used = and not :

The only bad thing that there is no highlighted result in case search against .raw field

Is it possible to change somehow?

Thanks


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.