ElasticSearch 0.9.0.RC1 / RSS River 0.1.0 - Not working

Elastic Stack Elasticsearch
1

Hi All,

I've installed the latest ES, also tried the stable 0.20.x releases, with
the 0.1.0 river and nothing seems to happen. I cant see any requests
hitting our server which hosts the RSS feeds. There just doesnt seem to be
any activity.

I followed the examples in;

http://www.pilato.fr/rssriver/

Doesnt seem to work. Here are the steps I followed after installing the
river.

Create index

curl -XPUT 'http://localhost:9200/someindexname/' -d '{}'

Create river

curl -XPUT 'localhost:9200/_river/someindexname/_meta' -d '{
"type": "rss",
"rss": {
"feeds" : [ {
"name": "somerssfeed",
"url": "http://someurl/rss.xml",
"update_rate": 120000,
"ignore_ttl": true
}
]
}
}'

Create mappings

curl -XPUT 'http://localhost:9200/someindexname/page/_mapping' -d '{
"page" : {
"properties" : {
"feedname" : {"type" : "string"},
"title" : {"type" : "string"},
"description" : {"type" : "string"},
"author" : {"type" : "string"},
"link" : {"type" : "string"}
}
}
}'

Perform search

curl -XGET 'http://localhost:9200/someindexname/_search?pretty'

In /var/log/elasticsearch/[clustername].log file I can see the following
entries when I restart ES;

[2013-03-27 14:41:32,109][INFO ][river.rss ] [servername]
[rss][someindexname] Starting rss stream
[2013-03-27 14:41:32,401][INFO ][river.rss ] [servername]
[rss][someindexname] creating rss stream river [someindexname] for [
http://someurl/rss.xml] every [120000] ms

But once again, nothing happens. Any help/ideas would be brilliant.

Cheers,

Kris

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

I suppose that you modify the URL with the right one?
Could you change debug level to debug for river.rss?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 mars 2013 à 05:02, Kristof Kowalski kristof@kowalski.ms a écrit :

Hi All,

I've installed the latest ES, also tried the stable 0.20.x releases, with the 0.1.0 river and nothing seems to happen. I cant see any requests hitting our server which hosts the RSS feeds. There just doesnt seem to be any activity.

I followed the examples in;

http://www.pilato.fr/rssriver/

Doesnt seem to work. Here are the steps I followed after installing the river.

Create index

curl -XPUT 'http://localhost:9200/someindexname/' -d '{}'

Create river

curl -XPUT 'localhost:9200/_river/someindexname/_meta' -d '{
"type": "rss",
"rss": {
"feeds" : [ {
"name": "somerssfeed",
"url": "http://someurl/rss.xml",
"update_rate": 120000,
"ignore_ttl": true
}
]
}
}'

Create mappings

curl -XPUT 'http://localhost:9200/someindexname/page/_mapping' -d '{
"page" : {
"properties" : {
"feedname" : {"type" : "string"},
"title" : {"type" : "string"},
"description" : {"type" : "string"},
"author" : {"type" : "string"},
"link" : {"type" : "string"}
}
}
}'

Perform search

curl -XGET 'http://localhost:9200/someindexname/_search?pretty'

In /var/log/elasticsearch/[clustername].log file I can see the following entries when I restart ES;

[2013-03-27 14:41:32,109][INFO ][river.rss ] [servername] [rss][someindexname] Starting rss stream
[2013-03-27 14:41:32,401][INFO ][river.rss ] [servername] [rss][someindexname] creating rss stream river [someindexname] for [http://someurl/rss.xml] every [120000] ms

But once again, nothing happens. Any help/ideas would be brilliant.

Cheers,

Kris

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi David,

Thanks for the reply.

I ensured that I could do a curl -XGET to the url of the RSS feed and it
was fine from the ES server. I could see the feed being transmitted, so the
URL is a-ok.

With regards to the debug setting for river.rss, where is that? I've got
DEBUG enabled in my logging.yml config file.

Regards,

--*
Kristof Kowalski

/t @krzko http://twitter.com/krzko /w kowalski.ms

On Wednesday, 27 March 2013 17:38:18 UTC+11, David Pilato wrote:

I suppose that you modify the URL with the right one?
Could you change debug level to debug for river.rss?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 mars 2013 à 05:02, Kristof Kowalski <kri...@kowalski.ms<javascript:>>
a écrit :

Hi All,

I've installed the latest ES, also tried the stable 0.20.x releases, with
the 0.1.0 river and nothing seems to happen. I cant see any requests
hitting our server which hosts the RSS feeds. There just doesnt seem to be
any activity.

I followed the examples in;

http://www.pilato.fr/rssriver/
GitHub - dadoonet/rssriver: Elasticsearch Rss River (PROJECT STOPPED)

Doesnt seem to work. Here are the steps I followed after installing the
river.

Create index

curl -XPUT 'http://localhost:9200/someindexname/' -d '{}'

Create river

curl -XPUT 'localhost:9200/_river/someindexname/_meta' -d '{
"type": "rss",
"rss": {
"feeds" : [ {
"name": "somerssfeed",
"url": "http://someurl/rss.xml",
"update_rate": 120000,
"ignore_ttl": true
}
]
}
}'

Create mappings

curl -XPUT 'http://localhost:9200/someindexname/page/_mapping' -d '{
"page" : {
"properties" : {
"feedname" : {"type" : "string"},
"title" : {"type" : "string"},
"description" : {"type" : "string"},
"author" : {"type" : "string"},
"link" : {"type" : "string"}
}
}
}'

Perform search

curl -XGET 'http://localhost:9200/someindexname/_search?pretty'

In /var/log/elasticsearch/[clustername].log file I can see the following
entries when I restart ES;

[2013-03-27 14:41:32,109][INFO ][river.rss ] [servername]
[rss][someindexname] Starting rss stream
[2013-03-27 14:41:32,401][INFO ][river.rss ] [servername]
[rss][someindexname] creating rss stream river [someindexname] for [
http://someurl/rss.xml] every [120000] ms

But once again, nothing happens. Any help/ideas would be brilliant.

Cheers,

Kris

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

I think that something like:
rootLogger: DEBUG, console, file
logger:
river.rss: DEBUG

should work.
(in logging.yml file)

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 28 mars 2013 à 00:06, Kristof Kowalski kristof@kowalski.ms a écrit :

Hi David,

Thanks for the reply.

I ensured that I could do a curl -XGET to the url of the RSS feed and it was fine from the ES server. I could see the feed being transmitted, so the URL is a-ok.

With regards to the debug setting for river.rss, where is that? I've got DEBUG enabled in my logging.yml config file.

Regards,

--
Kristof Kowalski

/t @krzko /w kowalski.ms

On Wednesday, 27 March 2013 17:38:18 UTC+11, David Pilato wrote:
I suppose that you modify the URL with the right one?
Could you change debug level to debug for river.rss?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 mars 2013 à 05:02, Kristof Kowalski kri...@kowalski.ms a écrit :

Hi All,

I've installed the latest ES, also tried the stable 0.20.x releases, with the 0.1.0 river and nothing seems to happen. I cant see any requests hitting our server which hosts the RSS feeds. There just doesnt seem to be any activity.

I followed the examples in;

http://www.pilato.fr/rssriver/
GitHub - dadoonet/rssriver: Elasticsearch Rss River (PROJECT STOPPED)

Doesnt seem to work. Here are the steps I followed after installing the river.

Create index

curl -XPUT 'http://localhost:9200/someindexname/' -d '{}'

Create river

curl -XPUT 'localhost:9200/_river/someindexname/_meta' -d '{
"type": "rss",
"rss": {
"feeds" : [ {
"name": "somerssfeed",
"url": "http://someurl/rss.xml",
"update_rate": 120000,
"ignore_ttl": true
}
]
}
}'

Create mappings

curl -XPUT 'http://localhost:9200/someindexname/page/_mapping' -d '{
"page" : {
"properties" : {
"feedname" : {"type" : "string"},
"title" : {"type" : "string"},
"description" : {"type" : "string"},
"author" : {"type" : "string"},
"link" : {"type" : "string"}
}
}
}'

Perform search

curl -XGET 'http://localhost:9200/someindexname/_search?pretty'

In /var/log/elasticsearch/[clustername].log file I can see the following entries when I restart ES;

[2013-03-27 14:41:32,109][INFO ][river.rss ] [servername] [rss][someindexname] Starting rss stream
[2013-03-27 14:41:32,401][INFO ][river.rss ] [servername] [rss][someindexname] creating rss stream river [someindexname] for [http://someurl/rss.xml] every [120000] ms

But once again, nothing happens. Any help/ideas would be brilliant.

Cheers,

Kris

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Thanks for tip.

Have added river.rss to the logger there's not a big change in the logger
at.

logging.yml;
rootLogger: DEBUG, console, file
logger:

log action execution errors for easier debugging

action: DEBUG
river.rss: DEBUG

  • /var/log/elasticsearch/[clustername].log*
    [2013-03-28 10:20:23,919][DEBUG][river ] [cos-poc-es-01]
    creating river [rss][someindexname]
    [2013-03-28 10:20:23,933][INFO ][river.rss ] [cos-poc-es-01]
    [rss][someindexname] Starting rss stream
    [2013-03-28 10:20:23,956][INFO ][river.rss ] [cos-poc-es-01]
    [rss][someindexname] creating rss stream river [someindexname] for [
    http://someurl/rss.xml] every [60000] ms

That's all I get.

I've had a look at the rssriver.java and based on the strings in there I'm
sure it's meant to be doing more. :slight_smile:

Is it meant to retrieve/index the feed straight after a 'service
elasticsearch restart'?

--
Kristof Kowalski

/t @krzko http://twitter.com/krzko /w kowalski.ms

--
Kristof Kowalski

/e kristof@kowalski.ms /t @krzko http://twitter.com/krzko /w kowalski.ms

On 28 March 2013 10:11, David Pilato david@pilato.fr wrote:

I think that something like:
rootLogger: DEBUG, console, file
logger:
river.rss: DEBUG

should work.
(in logging.yml file)

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 28 mars 2013 à 00:06, Kristof Kowalski kristof@kowalski.ms a écrit :

Hi David,

Thanks for the reply.

I ensured that I could do a curl -XGET to the url of the RSS feed and it
was fine from the ES server. I could see the feed being transmitted, so the
URL is a-ok.

With regards to the debug setting for river.rss, where is that? I've got
DEBUG enabled in my logging.yml config file.

Regards,

--*
Kristof Kowalski

/t @krzko http://twitter.com/krzko /w kowalski.ms

On Wednesday, 27 March 2013 17:38:18 UTC+11, David Pilato wrote:

I suppose that you modify the URL with the right one?
Could you change debug level to debug for river.rss?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 mars 2013 à 05:02, Kristof Kowalski kri...@kowalski.ms a écrit :

Hi All,

I've installed the latest ES, also tried the stable 0.20.x releases, with
the 0.1.0 river and nothing seems to happen. I cant see any requests
hitting our server which hosts the RSS feeds. There just doesnt seem to be
any activity.

I followed the examples in;

http://www.pilato.fr/rssriver/
https://github.com/dadoonet/**rssriver/https://github.com/dadoonet/rssriver/

Doesnt seem to work. Here are the steps I followed after installing the
river.

Create index

curl -XPUT 'http://localhost:9200/**someindexname/'<http://localhost:9200/someindexname/'> -d
'{}'

Create river

curl -XPUT 'localhost:9200/_river/**someindexname/_meta' -d '{
"type": "rss",
"rss": {
"feeds" : [ {
"name": "somerssfeed",
"url": "http://someurl/rss.xml",
"update_rate": 120000,
"ignore_ttl": true
}
]
}
}'

Create mappings

curl -XPUT 'http://localhost:9200/**someindexname/page/_mapping'<http://localhost:9200/someindexname/page/_mapping'>
-**d '{
"page" : {
"properties" : {
"feedname" : {"type" : "string"},
"title" : {"type" : "string"},
"description" : {"type" : "string"},
"author" : {"type" : "string"},
"link" : {"type" : "string"}
}
}
}'

Perform search

curl -XGET 'http://localhost:9200/**someindexname/_search?pretty'<http://localhost:9200/someindexname/_search?pretty'>

In /var/log/elasticsearch/[**clustername].log file I can see the
following entries when I restart ES;

[2013-03-27 14:41:32,109][INFO ][river.rss ] [servername]
[rss][someindexname] Starting rss stream
[2013-03-27 14:41:32,401][INFO ][river.rss ] [servername]
[rss][someindexname] creating rss stream river [someindexname] for [
http://someurl/rss.xml] every [120000] ms

But once again, nothing happens. Any help/ideas would be brilliant.

Cheers,

Kris

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@**googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/FaK8LEsfevM/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.