# Unique URL and query param patching

**URL:** <https://discuss.elastic.co/t/unique-url-and-query-param-patching/32298>\
**Category:** Elasticsearch\
**Created:** [October 15, 2015, 4:12pm UTC](https://discuss.elastic.co/t/unique-url-and-query-param-patching/32298 "2015-10-15T16:12:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![intenta](https://avatars.discourse-cdn.com/v4/letter/i/b5a626/32.png) [@intenta](https://discuss.elastic.co/u/intenta)\
**Post date:** [October 15, 2015, 4:12pm UTC](https://discuss.elastic.co/t/unique-url-and-query-param-patching/32298/1 "2015-10-15T16:12:09Z")

</div>

I'm trying to solve this problem: Goal

To build a system that supports a highly concurrent & scalable URL matching algorithm.

I'm looking for tools, technologies, patterns & strategies.

Task

Imagine you have an InputURL:

[www.awesomesite.com/product?color=pink&size=small&utm\_campaign=google](http://www.awesomesite.com/product?color=pink&size=small&utm_campaign=google)  
In some data storage / structure / mechanism we might store the PossibleURLs:

record 1: [www.awesomesite.com/product?size=small&color=red](http://www.awesomesite.com/product?size=small&color=red)  
record 2: [www.awesomesite.com/product?size=small](http://www.awesomesite.com/product?size=small)  
record 3: [www.awesomesite.com/product?size=small&color=blue](http://www.awesomesite.com/product?size=small&color=blue)  
record 4: [www.awesomesite.com/product?size=medium&color=blue](http://www.awesomesite.com/product?size=medium&color=blue)  
record 5: [www.awesomesite.com/product?size=large&color=blue](http://www.awesomesite.com/product?size=large&color=blue)  
record 6: [www.awesomesite.com/product?size=small&country=us](http://www.awesomesite.com/product?size=small&country=us)  
record 7: [www.awesomesite.com/product?size=small&color=pink](http://www.awesomesite.com/product?size=small&color=pink)  
... millions more of variations.  
I want to build a system that can take the InputURL and within a few ms, return back all the PossibleURLs where the InputURL contains all of the PossibleURL params.

In the scenario above, because the PossibleURL contain "size=small" & "color=pink" these records would have been matched:

record 2: [www.awesomesite.com/product?size=small](http://www.awesomesite.com/product?size=small)  
record 6: [www.awesomesite.com/product?size=small&country=us](http://www.awesomesite.com/product?size=small&country=us)  
record 7: [www.awesomesite.com/product?size=small&color=pink](http://www.awesomesite.com/product?size=small&color=pink)  
Constraints

High Scalability: Requests will be coming in at rates of 1000's to hundreds of thousands per second.  
Speed: The matching needs to be done in \< 10 ms  
Huge Record Size: The # of PossibleRecords will grow to millions. However they will not always be from the same domain.

A friend suggested that it could be done using elastic search. Can I use ElasticSearch to handle the logic & Elastic.co to support to handle the infrastructure?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 15, 2015, 8:39pm UTC](https://discuss.elastic.co/t/unique-url-and-query-param-patching/32298/2 "2015-10-15T20:39:00Z")

</div>

Is this specific to a found cluster you have or just a general ES question?

---

<div class="post-metadata">

**Author:** ![intenta](https://avatars.discourse-cdn.com/v4/letter/i/b5a626/32.png) [@intenta](https://discuss.elastic.co/u/intenta)\
**Post date:** [October 15, 2015, 11:37pm UTC](https://discuss.elastic.co/t/unique-url-and-query-param-patching/32298/3 "2015-10-15T23:37:45Z")

</div>

I'm looking to Signup for Found, if its the right solution for the Job.I  
believe it could be, but was hoping you guys could give me a little insight  
as to whether or not ElasticSearch could be the tool and your platform  
could facilitate the requirements I have.  
From what i can tell, its possible, to do what i'm thinking with elastic  
search, and we will probably need a decent size cluster for the level of  
traffic and url's we have coming through our platform.

If it would help to jump on a call to answer questions and clarify our  
needs let me know.

Thanks so much for your help,

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 15, 2015, 11:53pm UTC](https://discuss.elastic.co/t/unique-url-and-query-param-patching/32298/4 "2015-10-15T23:53:15Z")

</div>

Ok let me move this to the Elasticsearch category and we can discuss the core question first 🙂

---

<div class="post-metadata">

**Author:** ![intenta](https://avatars.discourse-cdn.com/v4/letter/i/b5a626/32.png) [@intenta](https://discuss.elastic.co/u/intenta)\
**Post date:** [October 16, 2015, 1:32am UTC](https://discuss.elastic.co/t/unique-url-and-query-param-patching/32298/5 "2015-10-16T01:32:46Z")

</div>

Sounds good.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:44pm UTC](https://discuss.elastic.co/t/unique-url-and-query-param-patching/32298/6 "2017-07-05T23:44:28Z")

</div>