Real Time Python Update on a River


(Thibaut Lapierre) #1

Hi,
I use the twitter river who use bulk indexing.

I have a Python script who analyse tweets and return some data.
So i want to analyse each tweet and add two fields to the river with the
returned data.

Maybe i can build a second sheme with id and treatment status in order to
run the script separately and update each doc, but i'm pretty sure there is
a cleaner/easier solution.

Thanks for helping

Thibaut

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a480947a-5790-4b84-838e-c0d37c191079%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Honza Král) #2

Hi,

I am afraid the easiest solution here is to not use rivers but instead
do the loading yourself - using a dedicated python process to consume
the twitter stream, enriching the data and loading it into
elasticsearch using the stream_bulk helper in the official python
client (0).

0 - http://elasticsearch-py.readthedocs.org/en/latest/helpers.html#elasticsearch.helpers.streaming_bulk

Hope this helps,
Honza

On Tue, Mar 25, 2014 at 5:38 PM, Thibaut Lapierre
thibaut.lapierre@gmail.com wrote:

Hi,
I use the twitter river who use bulk indexing.

I have a Python script who analyse tweets and return some data.
So i want to analyse each tweet and add two fields to the river with the
returned data.

Maybe i can build a second sheme with id and treatment status in order to
run the script separately and update each doc, but i'm pretty sure there is
a cleaner/easier solution.

Thanks for helping

Thibaut

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a480947a-5790-4b84-838e-c0d37c191079%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABfdDirZK3-bhP4kZU%2B3szYTK%2BJBibdswT%3DuVJrFWqRk%2BB8u4A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3