# Best way to search a field for a large set of values **URL:** \ **Category:** Kibana\ **Created:** [February 26, 2017, 4:09am UTC](https://discuss.elastic.co/t/best-way-to-search-a-field-for-a-large-set-of-values/76518 "2017-02-26T04:09:20Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![brandont](https://avatars.discourse-cdn.com/v4/letter/b/a6a055/32.png) [@brandont](https://discuss.elastic.co/u/brandont)\ **Post date:** [February 26, 2017, 4:09am UTC](https://discuss.elastic.co/t/best-way-to-search-a-field-for-a-large-set-of-values/76518/1 "2017-02-26T04:09:20Z")
I'm creating a visualization in Kibana and want to filter a `url.keyword` field so that only a subset of about 50 urls are included in the search. In addition, I want to filter everything on a second field `field2.keyword:true`. I curently have a query attached to the visualization that looks something like this: > `(url.keyword:"http://a.com" OR ... OR url.keyword:"http://z.com") AND field2.keyword:true` This seems to take a long time to execute (although it was much faster on 2.3 with `.raw` fields) and when I put several of these visualizations together on a dashboard it even causes the JVM heap to overflow and crashes ES. Is there a more efficient way to write this query? I feel like I should be using the json input option instead but I'm not sure it actually queries any differently on the backend. I appreciate the assistance. Thanks, Brandon ---
**Author:** ![Stacey\_Gammon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stacey_gammon/32/14025_2.png) [@Stacey\_Gammon](https://discuss.elastic.co/u/Stacey_Gammon)\ **Post date:** [February 27, 2017, 4:27pm UTC](https://discuss.elastic.co/t/best-way-to-search-a-field-for-a-large-set-of-values/76518/2 "2017-02-27T16:27:57Z")
You can try a JSON filter like so (just replace `ip` with `url.keyword` and your own values): ```auto { "query": { "bool": { "filter": { "terms": { "ip": [ "185.231.86.130", "166.24.243.98", "155.171.113.115" ] } } } } } ``` ![](https://us1.discourse-cdn.com/elastic/original/2X/8/84ac5dee292e09d79e0ba8e8add47ddd024f6edc.png) It does actually send it to es differently: With the filter: ```auto "query":{"bool":{"must":[{"query_string":{"query":"*","analyze_wildcard":true}}, {"bool":{"filter":{"terms":{"ip":["185.231.86.130","166.24.243.98","155.171.113.115"]}}}}, ``` Using an OR query `ip:"185.231.86.130" OR ip:"166.24.243.98" OR ip:"155.171.113.115"`: ```auto "query":{"bool":{"must":[{"query_string":{"query":"ip:\"185.231.86.130\" OR ip:\"166.24.243.98\" OR ip:\"155.171.113.115\"","analyze_wildcard":true}}, ``` Let me know if that improves performance. ---
**Author:** ![brandont](https://avatars.discourse-cdn.com/v4/letter/b/a6a055/32.png) [@brandont](https://discuss.elastic.co/u/brandont)\ **Post date:** [February 27, 2017, 8:03pm UTC](https://discuss.elastic.co/t/best-way-to-search-a-field-for-a-large-set-of-values/76518/3 "2017-02-27T20:03:10Z")
Thanks for the response. That's good to know. However, this doesn't seem to work when I add it to the JSON input under the y-axis (count) of a visualization. I want to be able to visualize a date histogram that only displays the count for the specified urls and I think the JSON input in the visualization section may accept different information than the JSON filter in Discover. I'm not sure this is the case, though. ---
**Author:** ![Stacey\_Gammon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stacey_gammon/32/14025_2.png) [@Stacey\_Gammon](https://discuss.elastic.co/u/Stacey_Gammon)\ **Post date:** [February 27, 2017, 10:01pm UTC](https://discuss.elastic.co/t/best-way-to-search-a-field-for-a-large-set-of-values/76518/4 "2017-02-27T22:01:44Z")
You should be able to do this without using the JSON input in visualize by using pinned filters. Go into the discover tab and create a filter with the url. I'll walk through my ip case. Hit the little `+` magnifying glass. ![](https://us1.discourse-cdn.com/elastic/original/2X/0/0ecb65bf89a7df1e909978dd462f8a1913f6b53a.png) Then to the filter that was added at the top, click the edit button and paste your list query. ![](https://us1.discourse-cdn.com/elastic/original/2X/6/6a29760115f491a3e32dfcd4db4fcfd3e2123752.png) Then click the thumbtack to pin this filter. ![](https://us1.discourse-cdn.com/elastic/original/2X/6/64b2f74fe9cf69c043eedc5c97ffea0c920b6990.png) With the filter pinned, you can go over into the visualize tab and create your visualization that will be filtered to that list. ![](https://us1.discourse-cdn.com/elastic/original/2X/3/396f9271d11d76ca5f0151306e6c3e99c62098e6.png) When you save the visualization, the filter will be saved with it, so whether you are viewing it in a dashboard, or opening it back up, it will retain this filter. There might be an easier way to achieve this, but this is the first thing that comes to mind. Let me know if that helps! ---
**Author:** ![brandont](https://avatars.discourse-cdn.com/v4/letter/b/a6a055/32.png) [@brandont](https://discuss.elastic.co/u/brandont)\ **Post date:** [February 27, 2017, 10:28pm UTC](https://discuss.elastic.co/t/best-way-to-search-a-field-for-a-large-set-of-values/76518/5 "2017-02-27T22:28:59Z")
Wow! That's awesome, thank you. The ability to pin filters is a powerful one. I was always upset with the fact that I couldn't create filters from the visualization interface but this at least gives me the ability to access and edit them. I will check in again when I have found out if using filters is more efficient than querying a search. ---
**Author:** ![brandont](https://avatars.discourse-cdn.com/v4/letter/b/a6a055/32.png) [@brandont](https://discuss.elastic.co/u/brandont)\ **Post date:** [March 13, 2017, 5:05pm UTC](https://discuss.elastic.co/t/best-way-to-search-a-field-for-a-large-set-of-values/76518/6 "2017-03-13T17:05:15Z")
In conclusion, using filters seemed to be a more stable method for searching for a large list of values. My searches are more stable and much more easy to construct in this fashion. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 10, 2017, 5:05pm UTC](https://discuss.elastic.co/t/best-way-to-search-a-field-for-a-large-set-of-values/76518/7 "2017-04-10T17:05:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.