# Url normalization with elasticsearch

**URL:** <https://discuss.elastic.co/t/url-normalization-with-elasticsearch/8391>\
**Category:** Elasticsearch\
**Created:** [July 11, 2012, 8:57pm UTC](https://discuss.elastic.co/t/url-normalization-with-elasticsearch/8391 "2012-07-11T20:57:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ahmet\_Akyol](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ahmet_akyol/32/2389_2.png) [@Ahmet\_Akyol](https://discuss.elastic.co/u/Ahmet_Akyol)\
**Post date:** [July 11, 2012, 8:57pm UTC](https://discuss.elastic.co/t/url-normalization-with-elasticsearch/8391/1 "2012-07-11T20:57:35Z")

</div>

Hi,

Has anyone tried url normalization with elasticsearch? Or can you share  
some hints?

" URL normalization [http://en.wikipedia.org/wiki/URL\_normalization](http://en.wikipedia.org/wiki/URL_normalization) (or  
URL canonicalization) is the process by which URLs are modified and  
standardized in a consistent manner. The goal of the normalization process  
is to transform a URL into a normalized or canonical URL so it is possible  
to determine if two syntactically different URLs are equivalent. " from  
wikipedia.

Here is an example of the problem:

( from

> **[Partnering to help solve duplicate content issues](https://blogs.bing.com/webmaster/2009/02/12/partnering-to-help-solve-duplicate-content-issues)**
>
> One of the most common challenges search engines run into when indexing a website is identifying and consolidating duplicate pages. Duplicates can occur when any given webpage has multiple URLs that point to it. For...

  
)

_URL_ _Description_[http://mysite.com](http://mysite.com) A webmaster may consider this their  
authoritative or _canonical_ URL for their homepage.

[http://www.mysite.com](http://www.mysite.com) However, you can add 'www' to most websites and  
still get the same home page.

[http://mysite.com/default.aspx](http://mysite.com/default.aspx) You can also often add the specific  
filename of the homepage and get the same page

[http://mysite.com/default.aspx?promo=ABC](http://mysite.com/default.aspx?promo=ABC) Many times websites use  
parameters to track things like where customers are coming from (in this  
case an offline promotion), or parameters that determine how the content on  
the page is formatted.

---

<div class="post-metadata">

**Author:** ![przkal1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/przkal1/32/2262_2.png) [@przkal1](https://discuss.elastic.co/u/przkal1)\
**Post date:** [June 24, 2013, 8:53am UTC](https://discuss.elastic.co/t/url-normalization-with-elasticsearch/8391/2 "2013-06-24T08:53:31Z")

</div>

Hey,

Have you make any progress in this topic since you had this problem? If  
yes, could you show me your research effects?

Thanks

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:29am UTC](https://discuss.elastic.co/t/url-normalization-with-elasticsearch/8391/3 "2017-07-06T02:29:50Z")

</div>