# Using a sitemap to get elasticSearch content
**URL:** https://discuss.elastic.co/t/using-a-sitemap-to-get-elasticsearch-content/8684
**Category:** Elasticsearch
**Created:** [August 8, 2012, 7:58pm UTC](https://discuss.elastic.co/t/using-a-sitemap-to-get-elasticsearch-content/8684 "2012-08-08T19:58:32Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@muk\_dal](https://discuss.elastic.co/u/muk_dal)
#### Post date: [August 8, 2012, 7:58pm UTC](https://discuss.elastic.co/t/using-a-sitemap-to-get-elasticsearch-content/8684/1 "2012-08-08T19:58:32Z")
Hello,
Has anyone successfully used a sitemap (such as
[http://www.nytimes.com/sitemaps/sitemap\_news/sitemap.xml.gz](http://www.nytimes.com/sitemaps/sitemap_news/sitemap.xml.gz)) to get content for indexing in elasticSearch?
Apart from the html content pages, I'd also like to index metadata from the sitemap entries (such as news:title).
My first attempt was to modify an existing river (wikipedia or fsriver), but it has not yet worked.
thanks,
Muk
---
### Author:  [@Ivan](https://discuss.elastic.co/u/Ivan)
#### Post date: [August 9, 2012, 6:07pm UTC](https://discuss.elastic.co/t/using-a-sitemap-to-get-elasticsearch-content/8684/2 "2012-08-09T18:07:52Z")
The way I see it, rivers are suited for data that is constantly
updating, so real time updates are required: twitter, couchdb updates,
rabbitmq, etc... Sitemaps are fairly static compared to the real time
nature of Twitter. IMHO, an external process that reads and parses a
sitemap and uses this data to index Elasticsearch would be better.
That said, any Java process can be made into a river. Elasticsearch
doesn't work with XML, so a custom XML parser is required. Surely,
someone abstracted sitemap parsing into a library by now.
Cheers,
Ivan
On Wed, Aug 8, 2012 at 12:58 PM, muk dal [mukesh.dalal@gmail.com](mailto:mukesh.dalal@gmail.com) wrote:
> Hello,
>
> Has anyone successfully used a sitemap (such as
>
> [http://www.nytimes.com/sitemaps/sitemap\_news/sitemap.xml.gz](http://www.nytimes.com/sitemaps/sitemap_news/sitemap.xml.gz)) to get content
> for indexing in elasticSearch?
> Apart from the html content pages, I'd also like to index metadata from the
> sitemap entries (such as news:title).
>
> My first attempt was to modify an existing river (wikipedia or fsriver), but
> it has not yet worked.
>
> thanks,
> Muk
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:16am UTC](https://discuss.elastic.co/t/using-a-sitemap-to-get-elasticsearch-content/8684/3 "2017-07-06T03:16:55Z")