# Any analyzer / mapping suggestions for Web search?

**URL:** <https://discuss.elastic.co/t/any-analyzer-mapping-suggestions-for-web-search/6960>\
**Category:** Elasticsearch\
**Created:** [March 9, 2012, 2:29pm UTC](https://discuss.elastic.co/t/any-analyzer-mapping-suggestions-for-web-search/6960 "2012-03-09T14:29:23Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Will\_Ezell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/will_ezell/32/2960_2.png) [@Will\_Ezell](https://discuss.elastic.co/u/Will_Ezell)\
**Post date:** [March 9, 2012, 2:29pm UTC](https://discuss.elastic.co/t/any-analyzer-mapping-suggestions-for-web-search/6960/1 "2012-03-09T14:29:23Z")

</div>

We are planning to develop a site search type functionality to replace some  
aging and painful nutch infrastructure. We are obviously not looking to  
rebuild google, but assuming we have a decent spider, does anyone  
have analyzer/mapping suggestions for a site search index?

Off the top of my head, the mapping would include fields like:

host  
site  
content length  
url  
mime/type  
title  
description (from html or tika)  
content  
last modified  
keywords  
language  
author  
geo\_stuff

I was wondering if anyone had any experiences or analyzer / mapping  
definitions they'd be willing to share?

Will

* * *

Will Ezell

> **[Open Source Content Management System and Headless CMS](https://dotcms.com/)**
>
> dotCMS is a open source content management system and headless CMS for managing and delivering content driven web apps and sites.

---

<div class="post-metadata">

**Author:** ![egaumer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/egaumer/32/2365_2.png) [@egaumer](https://discuss.elastic.co/u/egaumer)\
**Post date:** [March 9, 2012, 8:49pm UTC](https://discuss.elastic.co/t/any-analyzer-mapping-suggestions-for-web-search/6960/2 "2012-03-09T20:49:58Z")

</div>

We use nutch to crawl and feed elasticsearch so you might be able to  
continue to use nutch as your crawler.

> **[GitHub - mattweber/elasticsearch-mocksolrplugin: Use Solr clients/tools with...](https://github.com/mattweber/elasticsearch-mocksolrplugin)**
>
> Use Solr clients/tools with ElasticSearch. Contribute to mattweber/elasticsearch-mocksolrplugin development by creating an account on GitHub.

-Eric

On Friday, March 9, 2012 9:29:23 AM UTC-5, Will Ezell wrote:

> We are planning to develop a site search type functionality to replace  
> some aging and painful nutch infrastructure. We are obviously not looking  
> to rebuild google, but assuming we have a decent spider, does anyone  
> have analyzer/mapping suggestions for a site search index?
> 
> Off the top of my head, the mapping would include fields like:
> 
> host  
> site  
> content length  
> url  
> mime/type  
> title  
> description (from html or tika)  
> content  
> last modified  
> keywords  
> language  
> author  
> geo\_stuff
> 
> I was wondering if anyone had any experiences or analyzer / mapping  
> definitions they'd be willing to share?
> 
> Will
> 
> * * *
> 
> Will Ezell  
> [http://dotcms.com](http://dotcms.com)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:36am UTC](https://discuss.elastic.co/t/any-analyzer-mapping-suggestions-for-web-search/6960/3 "2017-07-06T03:36:28Z")

</div>

Closed on July 6, 2017, 3:36am UTC