FS River not working in elasticsearch cluster

Elastic Stack Elasticsearch
1

I am using:

  • FS River version: 0.0.3
  • Elasticsearch: 0.20.6

Setup:

  • Two nodes of elasticsearch in a cluster, es-one & es-two
  • A local file system on es-one at /tmp/data
  • A remotely mounted (via fuse-sshfs) file system on es-two at /tmp/data

SCENARIO 1:
Issued the following rest call to create fsriver:
curl -XPUT es-one:9200/_river/data-river/_meta -d '{
"type": "fs",
"fs": {
"url": "/tmp/data",
"update_rate": 120000
},
"index": {
"index": "data",
"type": "doc",
"bulk_size": 50
}
}'

Expectation:

  • River with name [data-river] should be created
  • All files under /tmp/data should be indexed

Actual:

  • [data-river] wasn't created
  • Expected files didn't get indexed

SCENARIO 2:
After SCENARIO-1, I brought down elasticsearch server at es-two.
Then,

  • [data-river] got created
  • all expected files got indexed

Question:
For FS river to get created and start indexing, do we require to have only
one node running in the cluster?

Thanks,
Imdad/Vidhi

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Did you install the FSRiver plugin on both nodes and restart them both?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 23 mai 2013 à 10:58, Imdad Ahmed imdhmd@gmail.com a écrit :

I am using:

  • FS River version: 0.0.3
  • Elasticsearch: 0.20.6

Setup:

  • Two nodes of elasticsearch in a cluster, es-one & es-two
  • A local file system on es-one at /tmp/data
  • A remotely mounted (via fuse-sshfs) file system on es-two at /tmp/data

SCENARIO 1:
Issued the following rest call to create fsriver:
curl -XPUT es-one:9200/_river/data-river/_meta -d '{
"type": "fs",
"fs": {
"url": "/tmp/data",
"update_rate": 120000
},
"index": {
"index": "data",
"type": "doc",
"bulk_size": 50
}
}'

Expectation:

  • River with name [data-river] should be created
  • All files under /tmp/data should be indexed

Actual:

  • [data-river] wasn't created
  • Expected files didn't get indexed

SCENARIO 2:
After SCENARIO-1, I brought down elasticsearch server at es-two.
Then,

  • [data-river] got created
  • all expected files got indexed

Question:
For FS river to get created and start indexing, do we require to have only one node running in the cluster?

Thanks,
Imdad/Vidhi

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

This is a late reply to this thread, to bring it to closure.

Yes, Pilato, we did follow the correct steps. But the problem was something
else.
Elasticsearch user on es-two didnt have proper access to /tmp/data.
Once we corrected accessibility issue, this problem no longer occured.

On Thursday, May 23, 2013 4:38:32 PM UTC+5:30, David Pilato wrote:

Did you install the FSRiver plugin on both nodes and restart them both?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr
| @scrutmydocs https://twitter.com/scrutmydocs

Le 23 mai 2013 à 10:58, Imdad Ahmed <imd...@gmail.com <javascript:>> a
écrit :

I am using:

  • FS River version: 0.0.3
  • Elasticsearch: 0.20.6

Setup:

  • Two nodes of elasticsearch in a cluster, es-one & es-two
  • A local file system on es-one at /tmp/data
  • A remotely mounted (via fuse-sshfs) file system on es-two at /tmp/data

SCENARIO 1:
Issued the following rest call to create fsriver:
curl -XPUT es-one:9200/_river/data-river/_meta -d '{
"type": "fs",
"fs": {
"url": "/tmp/data",
"update_rate": 120000
},
"index": {
"index": "data",
"type": "doc",
"bulk_size": 50
}
}'

Expectation:

  • River with name [data-river] should be created
  • All files under /tmp/data should be indexed

Actual:

  • [data-river] wasn't created
  • Expected files didn't get indexed

SCENARIO 2:
After SCENARIO-1, I brought down elasticsearch server at es-two.
Then,

  • [data-river] got created
  • all expected files got indexed

Question:
For FS river to get created and start indexing, do we require to have only
one node running in the cluster?

Thanks,
Imdad/Vidhi

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Thanks for the feedback.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 23 août 2013 à 10:35, Imdad Ahmed imdhmd@gmail.com a écrit :

This is a late reply to this thread, to bring it to closure.

Yes, Pilato, we did follow the correct steps. But the problem was something else.
Elasticsearch user on es-two didnt have proper access to /tmp/data.
Once we corrected accessibility issue, this problem no longer occured.

On Thursday, May 23, 2013 4:38:32 PM UTC+5:30, David Pilato wrote:
Did you install the FSRiver plugin on both nodes and restart them both?

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 23 mai 2013 à 10:58, Imdad Ahmed imd...@gmail.com a écrit :

I am using:

  • FS River version: 0.0.3
  • Elasticsearch: 0.20.6

Setup:

  • Two nodes of elasticsearch in a cluster, es-one & es-two
  • A local file system on es-one at /tmp/data
  • A remotely mounted (via fuse-sshfs) file system on es-two at /tmp/data

SCENARIO 1:
Issued the following rest call to create fsriver:
curl -XPUT es-one:9200/_river/data-river/_meta -d '{
"type": "fs",
"fs": {
"url": "/tmp/data",
"update_rate": 120000
},
"index": {
"index": "data",
"type": "doc",
"bulk_size": 50
}
}'

Expectation:

  • River with name [data-river] should be created
  • All files under /tmp/data should be indexed

Actual:

  • [data-river] wasn't created
  • Expected files didn't get indexed

SCENARIO 2:
After SCENARIO-1, I brought down elasticsearch server at es-two.
Then,

  • [data-river] got created
  • all expected files got indexed

Question:
For FS river to get created and start indexing, do we require to have only one node running in the cluster?

Thanks,
Imdad/Vidhi

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Stp j utilise fs river pour indexer des journaux sur lesquels j px mener une recherche sur le contenu ultérieurement . Apres une annee je me suis trouver avec un problem que elastic n indec plus leq nouveaux doc ou accuse un grand retard pour le faire je compte sur votre aide

6

Juste une precision un noeud un cluster

7

Please use english here. There is a dedicated French area.
Also don't hijack old threads and create your own.

You can also open an issue in the project.