# "Best" way for (partial) search in URIs (e.g. requests/referrer)?

**URL:** <https://discuss.elastic.co/t/best-way-for-partial-search-in-uris-e-g-requests-referrer/209633>\
**Category:** Elasticsearch\
**Created:** [November 27, 2019, 8:34am UTC](https://discuss.elastic.co/t/best-way-for-partial-search-in-uris-e-g-requests-referrer/209633 "2019-11-27T08:34:49Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mw\_jko](https://avatars.discourse-cdn.com/v4/letter/m/db5fbb/32.png) [@mw\_jko](https://discuss.elastic.co/u/mw_jko)\
**Post date:** [November 27, 2019, 8:34am UTC](https://discuss.elastic.co/t/best-way-for-partial-search-in-uris-e-g-requests-referrer/209633/1 "2019-11-27T08:34:49Z")

</div>

Hi there,

I'm dumping an nginx access log into elasticsearch (I just use the standard analyzer for all fields atm (this is what I learned afterwards, the defaults in elasticsearch are just too good to bother you with this kind of topics early on 😁 ))

I struggle a bit with looking up data. It _usually_ works but I have some cases where the standard analyzer interferes (I think).

I kinda have some different request patterns. For example:

```auto
/public/a_16432_d0FqR/file/data/36541_536277.png
/access/access/logout?sid=f5a4875f7ee771174f1df1
/file/File/getThumbnail/835/64/64?sid=f5a4875f7ee771174f1df1
/page/setDelete/905?sid=f5a4875f7ee771174f1df1
/dashboard/dashboard/execute?sid=f5a4875f7ee771174f1df1&a_u=16432_79958

```

Standardanalyzer allows it to look up parts (e.g. `setDelete` but not `delete`) which is ok'ish to me (pain is not big enough to justify aditional changes here). But looking up specific GET parameters does not work when just looking for parts. E.g. `a_u` returns results, `a_u=16432_79958` can be looked up as well, but what I need is `a_u=16432` which does not work (probably because of the `_`) .

My template looks like this atm:

```auto
      ...
      "request": {
        "dynamic": true,
        "properties": {
          "keyword": {
            "type": "keyword"
          },
          "raw": {
            "type": "text"
          }
        }
      }
     ...

```

My "requirements" are:

- allow search for fragments (e.g. `page/setDelete`) - just `delete` would be the icing on the cake)
- allow search for complete url
- allow search for (parts) of specific GET parameters (e.g. `a_u=12345` when the complete parameter is `a_u=12345_6789`)

Can someone please give me some advice what to adjust to solve my problem? (Or point me to the relevant part?) Is the analyzer wrong? (If so, what's the best for this kind of data?)

Thanks in advance! 🙂

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [November 28, 2019, 2:42pm UTC](https://discuss.elastic.co/t/best-way-for-partial-search-in-uris-e-g-requests-referrer/209633/2 "2019-11-28T14:42:06Z")

</div>

You're right - the standard analyzer is wrong for your use case. You can see what the standard analyzer does on your URLs by using the `_analyze` API:

```auto
GET my_index/_analyze
{
  "analyzer": "standard",
  "text": [
    "/page/setDelete/905?sid=f5a4875f7ee771174f1df1",
    "/dashboard/dashboard/execute?sid=f5a4875f7ee771174f1df1&a_u=16432_79958"
  ]
}

```

The output shows you that for example `setDelete` and `a_u=12345_6789` do not get broken up into separate tokens by the standard analyzer, preventing you from being able to search for just `a_u=12345`.

You'll need to create a custom analyzer to support your use case. [The documentation has an example of a custom "camelcase" analyzer](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-pattern-analyzer.html#_camelcase_tokenizer) with a tokenizer that uses regular expressions to break up strings. That could be a good starting point for yours. Maybe something like this?

```auto
PUT my_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_analyzer": {
          "type": "pattern",
          "pattern": "([^\\p{L}\\d=]+)|(?<=[\\p{L}&&[^\\p{Lu}]])(?=\\p{Lu})|(?<=\\p{Lu})(?=\\p{Lu}[\\p{L}&&[^\\p{Lu}]])"
        }
      }
    }
  }
}

GET my_index/_analyze
{
  "analyzer": "my_analyzer",
  "text": [
    "/page/setDelete/905?sid=f5a4875f7ee771174f1df1",
    "/dashboard/dashboard/execute?sid=f5a4875f7ee771174f1df1&a_u=16432_79958"
  ]
}

```

Keep in mind that you cannot change the analyzer for existing fields in your mapping. You'll have to add a new multifield to your mapping, or create a new index, with a mapping that uses your new custom analyzer.

---

<div class="post-metadata">

**Author:** ![mw\_jko](https://avatars.discourse-cdn.com/v4/letter/m/db5fbb/32.png) [@mw\_jko](https://discuss.elastic.co/u/mw_jko)\
**Post date:** [November 29, 2019, 8:56am UTC](https://discuss.elastic.co/t/best-way-for-partial-search-in-uris-e-g-requests-referrer/209633/3 "2019-11-29T08:56:10Z")

</div>

Thank you @abdon! I will play around with this 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 27, 2019, 8:56am UTC](https://discuss.elastic.co/t/best-way-for-partial-search-in-uris-e-g-requests-referrer/209633/4 "2019-12-27T08:56:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.