# Twitter River : stopping after an arror **URL:** \ **Category:** Elasticsearch\ **Created:** [March 22, 2012, 7:28pm UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099 "2012-03-22T19:28:50Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 22, 2012, 7:28pm UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/1 "2012-03-22T19:28:50Z")
Hi there, I started a twitter river on a ec2 instance but it failed after 1 hour and half. It didn't restart automatically even if river displayed "stream failure, restarting stream..." in logs. Any thoughts ? Should I open a bug ? [2012-03-21 21:53:36,025][INFO][river.twitter] [Callisto] [twitter][twitter] creating twitter stream river for [null] [2012-03-21 21:53:36,137][INFO][river.twitter] [Callisto] [twitter][twitter] starting twitter stream [2012-03-21 21:53:36,269][INFO][twitter4j.TwitterStreamImpl] Establishing connection. [2012-03-21 21:53:36,965][INFO][twitter4j.TwitterStreamImpl] Connection established. [2012-03-21 21:53:36,965][INFO][twitter4j.TwitterStreamImpl] Receiving status stream. [2012-03-21 23:17:45,903][INFO][twitter4j.TwitterStreamImpl] Stream closed. [2012-03-21 23:17:45,904][WARN][river.twitter] [Callisto] [twitter][twitter] stream failure, restarting stream... Stream closed.Relevant discussions can be on the Internet at: ``` http://www.google.co.jp/search?q=70971d2e or http://www.google.co.jp/search?q=000687bf ``` TwitterException{exceptionCode=[70971d2e-000687bf 70971d2e-0006876e], statusCode=-1, retryAfter=-1, rateLimitStatus=null, featureSpecificRateLimitStatus=null, version=2.2.5} ``` at ``` twitter4j.AbstractStreamImplementation.handleNextElement(AbstractStreamImple mentation.java:167) ``` at twitter4j.StatusStreamImpl.next(StatusStreamImpl.java:67) at ``` twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java :443) Caused by: java.io.IOException: Premature EOF ``` at ``` sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.jav a:556) ``` at ``` sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:600) ``` at ``` sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:687) ``` at java.io.FilterInputStream.read(FilterInputStream.java:133) at ``` sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConn ection.java:2582) ``` at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:153) at java.io.BufferedReader.readLine(BufferedReader.java:316) at java.io.BufferedReader.readLine(BufferedReader.java:379) at ``` twitter4j.AbstractStreamImplementation.handleNextElement(AbstractStreamImple mentation.java:86) ``` ... 2 more ``` Thanks, David. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 25, 2012, 11:38am UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/2 "2012-03-25T11:38:06Z")
Strange..., maybe you can recreate it with a TRACE log level set on river.twitter? On Thu, Mar 22, 2012 at 9:28 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Hi there,\*\*\*\* > > * * * > > * * * > > I started a twitter river on a ec2 instance but it failed after 1 hour and > half.\*\*\*\* > > It didn’t restart automatically even if river displayed “stream failure, > restarting stream...” in logs.\*\*\*\* > > * * * > > Any thoughts ?\*\*\*\* > > Should I open a bug ?\*\*\*\* > > * * * > > [2012-03-21 21:53:36,025][INFO][river.twitter] [Callisto] > [twitter][twitter] creating twitter stream river for [null]\*\*\*\* > > [2012-03-21 21:53:36,137][INFO][river.twitter] [Callisto] > [twitter][twitter] starting twitter stream\*\*\*\* > > [2012-03-21 21:53:36,269][INFO][twitter4j.TwitterStreamImpl] Establishing > connection.\*\*\*\* > > [2012-03-21 21:53:36,965][INFO][twitter4j.TwitterStreamImpl] Connection > established.\*\*\*\* > > [2012-03-21 21:53:36,965][INFO][twitter4j.TwitterStreamImpl] Receiving > status stream.\*\*\*\* > > [2012-03-21 23:17:45,903][INFO][twitter4j.TwitterStreamImpl] Stream > closed.\*\*\*\* > > [2012-03-21 23:17:45,904][WARN][river.twitter] [Callisto] > [twitter][twitter] stream failure, restarting stream...\*\*\*\* > > Stream closed.Relevant discussions can be on the Internet at:\*\*\*\* > > ``` > http://www.google.co.jp/search?q=70971d2e or **** > > http://www.google.co.jp/search?q=000687bf **** > > ``` > > TwitterException{exceptionCode=[70971d2e-000687bf 70971d2e-0006876e], > statusCode=-1, retryAfter=-1, rateLimitStatus=null, > featureSpecificRateLimitStatus=null, version=2.2.5}\*\*\*\* > > ``` > at > > ``` > > twitter4j.AbstractStreamImplementation.handleNextElement(AbstractStreamImplementation.java:167) > > * * * > > ``` > at twitter4j.StatusStreamImpl.next(StatusStreamImpl.java:67) **** > > at > > ``` > > twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java:443) > > * * * > > Caused by: java.io.IOException: Premature EOF\*\*\*\* > > ``` > at > > ``` > > sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:556) > > * * * > > ``` > at > > ``` > > sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:600) > > * * * > > ``` > at > > ``` > > sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:687)\*\*\*\* > > ``` > at java.io.FilterInputStream.read(FilterInputStream.java:133) **** > > at > > ``` > > sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2582) > > * * * > > ``` > at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282) **** > > at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324) **** > > at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176) **** > > at java.io.InputStreamReader.read(InputStreamReader.java:184) **** > > at java.io.BufferedReader.fill(BufferedReader.java:153) **** > > at java.io.BufferedReader.readLine(BufferedReader.java:316) **** > > at java.io.BufferedReader.readLine(BufferedReader.java:379) **** > > at > > ``` > > twitter4j.AbstractStreamImplementation.handleNextElement(AbstractStreamImplementation.java:86) > > * * * > > ``` > ... 2 more **** > > ``` > > * * * > > Thanks,\*\*\*\* > > David.\*\*\*\* ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 29, 2012, 5:23pm UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/3 "2012-03-29T17:23:38Z")
Hi Shay, I finally got the error again. Traces are here : > You will see that 2012-03-29 05:13:11,994][WARN][river.twitter] [Princess Python] [twitter][devoxx] stream failure, restarting stream... And then every hour : [2012-03-29 09:40:56,945][DEBUG][http.netty] [Princess Python] Caught exception while handling client http traffic, closing connection [id: 0x007b7258, /195.101.137.28:2806 =\> /10.28.31.164:80] java.io.IOException: Connection timed out ``` at sun.nio.ch.FileDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251) at sun.nio.ch.IOUtil.read(IOUtil.java:218) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254) at ``` org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.j ava:315) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioWorker.processSelectedK eys(NioWorker.java:274) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.ja va:194) ``` at ``` org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamin gRunnable.java:102) ``` at ``` org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadL ockProofWorker.java:42) ``` at ``` java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 10) ``` at ``` java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 03) ``` at java.lang.Thread.run(Thread.java:679) ``` What do you think ? Thanks David. De : [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) [[mailto:elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com)] De la part de Shay Banon Envoyé : dimanche 25 mars 2012 13:38 À : [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) Objet : Re: Twitter River : stopping after an arror Strange..., maybe you can recreate it with a TRACE log level set on river.twitter? On Thu, Mar 22, 2012 at 9:28 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: Hi there, I started a twitter river on a ec2 instance but it failed after 1 hour and half. It didn’t restart automatically even if river displayed “stream failure, restarting stream...” in logs. Any thoughts ? Should I open a bug ? [2012-03-21 21:53:36,025][INFO][river.twitter] [Callisto] [twitter][twitter] creating twitter stream river for [null] [2012-03-21 21:53:36,137][INFO][river.twitter] [Callisto] [twitter][twitter] starting twitter stream [2012-03-21 21:53:36,269][INFO][twitter4j.TwitterStreamImpl] Establishing connection. [2012-03-21 21:53:36,965][INFO][twitter4j.TwitterStreamImpl] Connection established. [2012-03-21 21:53:36,965][INFO][twitter4j.TwitterStreamImpl] Receiving status stream. [2012-03-21 23:17:45,903][INFO][twitter4j.TwitterStreamImpl] Stream closed. [2012-03-21 23:17:45,904][WARN][river.twitter] [Callisto] [twitter][twitter] stream failure, restarting stream... Stream closed.Relevant discussions can be on the Internet at: ``` http://www.google.co.jp/search?q=70971d2e or http://www.google.co.jp/search?q=000687bf ``` TwitterException{exceptionCode=[70971d2e-000687bf 70971d2e-0006876e], statusCode=-1, retryAfter=-1, rateLimitStatus=null, featureSpecificRateLimitStatus=null, version=2.2.5} ``` at ``` twitter4j.AbstractStreamImplementation.handleNextElement(AbstractStreamImple mentation.java:167) ``` at twitter4j.StatusStreamImpl.next(StatusStreamImpl.java:67) at ``` twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java :443) Caused by: java.io.IOException: Premature EOF ``` at ``` sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.jav a:556) ``` at ``` sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:600) ``` at ``` sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:687) ``` at java.io.FilterInputStream.read(FilterInputStream.java:133) at ``` sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConn ection.java:2582) ``` at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:153) at java.io.BufferedReader.readLine(BufferedReader.java:316) at java.io.BufferedReader.readLine(BufferedReader.java:379) at ``` twitter4j.AbstractStreamImplementation.handleNextElement(AbstractStreamImple mentation.java:86) ``` ... 2 more ``` Thanks, David. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 4, 2012, 10:12pm UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/4 "2012-04-04T22:12:10Z")
Hi Shay, Sorry to nag. Could you give a look at this log file ? I have to restart my instance (or delete the river and recreate it) about every two or three days. Any idea ? Thanks David. De : [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) [[mailto:elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com)] De la part de David Pilato Envoyé : jeudi 29 mars 2012 19:24 À : [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) Objet : RE: Twitter River : stopping after an arror Hi Shay, I finally got the error again. Traces are here : > You will see that 2012-03-29 05:13:11,994][WARN][river.twitter] [Princess Python] [twitter][devoxx] stream failure, restarting stream... And then every hour : [2012-03-29 09:40:56,945][DEBUG][http.netty] [Princess Python] Caught exception while handling client http traffic, closing connection [id: 0x007b7258, /195.101.137.28:2806 =\> /10.28.31.164:80] java.io.IOException: Connection timed out ``` at sun.nio.ch.FileDispatcher.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251) at sun.nio.ch.IOUtil.read(IOUtil.java:218) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254) at ``` org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.j ava:315) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioWorker.processSelectedK eys(NioWorker.java:274) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.ja va:194) ``` at ``` org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamin gRunnable.java:102) ``` at ``` org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadL ockProofWorker.java:42) ``` at ``` java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 10) ``` at ``` java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 03) ``` at java.lang.Thread.run(Thread.java:679) ``` What do you think ? Thanks David. De : [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) [[mailto:elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com)] De la part de Shay Banon Envoyé : dimanche 25 mars 2012 13:38 À : [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) Objet : Re: Twitter River : stopping after an arror Strange..., maybe you can recreate it with a TRACE log level set on river.twitter? On Thu, Mar 22, 2012 at 9:28 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: Hi there, I started a twitter river on a ec2 instance but it failed after 1 hour and half. It didn’t restart automatically even if river displayed “stream failure, restarting stream...” in logs. Any thoughts ? Should I open a bug ? [2012-03-21 21:53:36,025][INFO][river.twitter] [Callisto] [twitter][twitter] creating twitter stream river for [null] [2012-03-21 21:53:36,137][INFO][river.twitter] [Callisto] [twitter][twitter] starting twitter stream [2012-03-21 21:53:36,269][INFO][twitter4j.TwitterStreamImpl] Establishing connection. [2012-03-21 21:53:36,965][INFO][twitter4j.TwitterStreamImpl] Connection established. [2012-03-21 21:53:36,965][INFO][twitter4j.TwitterStreamImpl] Receiving status stream. [2012-03-21 23:17:45,903][INFO][twitter4j.TwitterStreamImpl] Stream closed. [2012-03-21 23:17:45,904][WARN][river.twitter] [Callisto] [twitter][twitter] stream failure, restarting stream... Stream closed.Relevant discussions can be on the Internet at: ``` http://www.google.co.jp/search?q=70971d2e or http://www.google.co.jp/search?q=000687bf ``` TwitterException{exceptionCode=[70971d2e-000687bf 70971d2e-0006876e], statusCode=-1, retryAfter=-1, rateLimitStatus=null, featureSpecificRateLimitStatus=null, version=2.2.5} ``` at ``` twitter4j.AbstractStreamImplementation.handleNextElement(AbstractStreamImple mentation.java:167) ``` at twitter4j.StatusStreamImpl.next(StatusStreamImpl.java:67) at ``` twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java :443) Caused by: java.io.IOException: Premature EOF ``` at ``` sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.jav a:556) ``` at ``` sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:600) ``` at ``` sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:687) ``` at java.io.FilterInputStream.read(FilterInputStream.java:133) at ``` sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConn ection.java:2582) ``` at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:153) at java.io.BufferedReader.readLine(BufferedReader.java:316) at java.io.BufferedReader.readLine(BufferedReader.java:379) at ``` twitter4j.AbstractStreamImplementation.handleNextElement(AbstractStreamImple mentation.java:86) ``` ... 2 more ``` Thanks, David. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 8, 2012, 5:49pm UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/5 "2012-04-08T17:49:05Z")
I don't see anything interesting there, strange... . Based on teh code, it seems like the twitter4j library is being properly restarted and we try to reconnet properly, I will try and recreate it... On Thu, Apr 5, 2012 at 1:12 AM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote: > Hi Shay,\*\*\*\* > > * * * > > * * * > > Sorry to nag. Could you give a look at this log file ?\*\*\*\* > > I have to restart my instance (or delete the river and recreate it) about > every two or three days.\*\*\*\* > > * * * > > Any idea ?\*\*\*\* > > Thanks\*\*\*\* > > David.\*\*\*\* > > * * * > > _De :_ [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) [mailto: > [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com)] _De la part de_ David Pilato > _Envoyé :_ jeudi 29 mars 2012 19:24 > _À :_ [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) > _Objet :_ RE: Twitter River : stopping after an arror\*\*\*\* > > * * * > > Hi Shay,\*\*\*\* > > * * * > > * * * > > I finally got the error again. Traces are here : > [ES Twitter River log · GitHub](https://gist.github.com/2240225)\*\*\*\* > > * * * > > You will see that\*\*\*\* > > 2012-03-29 05:13:11,994][WARN][river.twitter] [Princess > Python] [twitter][devoxx] stream failure, restarting stream...\*\*\*\* > > * * * > > * * * > > And then every hour :\*\*\*\* > > [2012-03-29 09:40:56,945][DEBUG][http.netty] [Princess > Python] Caught exception while handling client http traffic, closing > connection [id: 0x007b7258, /195.101.137.28:2806 =\> /10.28.31.164:80]\*\*\*\* > > java.io.IOException: Connection timed out\*\*\*\* > > ``` > at sun.nio.ch.FileDispatcher.read0(Native Method) **** > > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) **** > > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251) **** > > at sun.nio.ch.IOUtil.read(IOUtil.java:218) **** > > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254)* > > ``` > > * * * > > ``` > at > > ``` > > org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:315) > > * * * > > ``` > at > > ``` > > org.elasticsearch.common.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:274) > > * * * > > ``` > at > > ``` > > org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:194) > > * * * > > ``` > at > > ``` > > org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102) > > * * * > > ``` > at > > ``` > > org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > > * * * > > ``` > at > > ``` > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > * * * > > ``` > at > > ``` > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > * * * > > ``` > at java.lang.Thread.run(Thread.java:679) **** > > ``` > > * * * > > * * * > > * * * > > What do you think ?\*\*\*\* > > * * * > > Thanks\*\*\*\* > > David.\*\*\*\* > > * * * > > * * * > > _De :_ [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) [ > [mailto:elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com)] _De > la part de_ Shay Banon > _Envoyé :_ dimanche 25 mars 2012 13:38 > _À :_ [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) > _Objet :_ Re: Twitter River : stopping after an arror\*\*\*\* > > * * * > > Strange..., maybe you can recreate it with a TRACE log level set on > river.twitter?\*\*\*\* > > On Thu, Mar 22, 2012 at 9:28 PM, David Pilato [david@pilato.fr](mailto:david@pilato.fr) wrote:\*\*\*\* > > Hi there,\*\*\*\* > > * * * > > * * * > > I started a twitter river on a ec2 instance but it failed after 1 hour and > half.\*\*\*\* > > It didn’t restart automatically even if river displayed “stream failure, > restarting stream...” in logs.\*\*\*\* > > * * * > > Any thoughts ?\*\*\*\* > > Should I open a bug ?\*\*\*\* > > * * * > > [2012-03-21 21:53:36,025][INFO][river.twitter] [Callisto] > [twitter][twitter] creating twitter stream river for [null]\*\*\*\* > > [2012-03-21 21:53:36,137][INFO][river.twitter] [Callisto] > [twitter][twitter] starting twitter stream\*\*\*\* > > [2012-03-21 21:53:36,269][INFO][twitter4j.TwitterStreamImpl] Establishing > connection.\*\*\*\* > > [2012-03-21 21:53:36,965][INFO][twitter4j.TwitterStreamImpl] Connection > established.\*\*\*\* > > [2012-03-21 21:53:36,965][INFO][twitter4j.TwitterStreamImpl] Receiving > status stream.\*\*\*\* > > [2012-03-21 23:17:45,903][INFO][twitter4j.TwitterStreamImpl] Stream > closed.\*\*\*\* > > [2012-03-21 23:17:45,904][WARN][river.twitter] [Callisto] > [twitter][twitter] stream failure, restarting stream...\*\*\*\* > > Stream closed.Relevant discussions can be on the Internet at:\*\*\*\* > > ``` > http://www.google.co.jp/search?q=70971d2e or **** > > http://www.google.co.jp/search?q=000687bf **** > > ``` > > TwitterException{exceptionCode=[70971d2e-000687bf 70971d2e-0006876e], > statusCode=-1, retryAfter=-1, rateLimitStatus=null, > featureSpecificRateLimitStatus=null, version=2.2.5}\*\*\*\* > > ``` > at > > ``` > > twitter4j.AbstractStreamImplementation.handleNextElement(AbstractStreamImplementation.java:167) > > * * * > > ``` > at twitter4j.StatusStreamImpl.next(StatusStreamImpl.java:67) **** > > at > > ``` > > twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java:443) > > * * * > > Caused by: java.io.IOException: Premature EOF\*\*\*\* > > ``` > at > > ``` > > sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:556) > > * * * > > ``` > at > > ``` > > sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:600) > > * * * > > ``` > at > > ``` > > sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:687)\*\*\*\* > > ``` > at java.io.FilterInputStream.read(FilterInputStream.java:133) **** > > at > > ``` > > sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2582) > > * * * > > ``` > at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:282) **** > > at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:324) **** > > at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:176) **** > > at java.io.InputStreamReader.read(InputStreamReader.java:184) **** > > at java.io.BufferedReader.fill(BufferedReader.java:153) **** > > at java.io.BufferedReader.readLine(BufferedReader.java:316) **** > > at java.io.BufferedReader.readLine(BufferedReader.java:379) **** > > at > > ``` > > twitter4j.AbstractStreamImplementation.handleNextElement(AbstractStreamImplementation.java:86) > > * * * > > ``` > ... 2 more **** > > ``` > > * * * > > Thanks,\*\*\*\* > > David.\*\*\*\* > > * * * ---
**Author:** ![Mikola\_Senyk\_2](https://avatars.discourse-cdn.com/v4/letter/m/278dde/32.png) [@Mikola\_Senyk\_2](https://discuss.elastic.co/u/Mikola_Senyk_2)\ **Post date:** [November 5, 2012, 11:32am UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/6 "2012-11-05T11:32:20Z")
I have the same trouble with twitter river. You can easy to repeat this bug if temporary disconnect your network. After error in log: ... stream failure, restarting stream... You can switch your network on, but twitter river can't turn back. Could I restart the river without shutting down cluster? -- ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 5, 2012, 9:01pm UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/7 "2012-11-05T21:01:15Z")
Same for me. Have a look at this old thread: [https://groups.google.com/forum/?fromgroups=&nomobile=true#!search/Twitter$20stream$20pilato/elasticsearch/6OTSds3DSSg/oK9KdCsm6mkJ](https://groups.google.com/forum/?fromgroups=&nomobile=true#!search/Twitter%2420stream%2420pilato/elasticsearch/6OTSds3DSSg/oK9KdCsm6mkJ) I did not find any way to fix it. So, I created a crontab script that look at logs and restart the node each time I see the error. The other node (2 nodes cluster) recreate the river and continue to fetch Twitter content. Here is a gist: [https://gist.github.com/4020312](https://gist.github.com/4020312) ## HTH David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 5 nov. 2012 à 12:32, Mikola Senyk [mikola.senyk@gmail.com](mailto:mikola.senyk@gmail.com) a écrit : I have the same trouble with twitter river. You can easy to repeat this bug if temporary disconnect your network. After error in log: ... stream failure, restarting stream... You can switch your network on, but twitter river can't turn back. ## Could I restart the river without shutting down cluster? -- ---
**Author:** ![Mikola\_Senyk\_2](https://avatars.discourse-cdn.com/v4/letter/m/278dde/32.png) [@Mikola\_Senyk\_2](https://discuss.elastic.co/u/Mikola_Senyk_2)\ **Post date:** [November 6, 2012, 7:07am UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/8 "2012-11-06T07:07:15Z")
Thank you David! Your crontab script is nice temporary solution. -- ---
**Author:** ![sujoysett](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@sujoysett](https://discuss.elastic.co/u/sujoysett)\ **Post date:** [November 8, 2012, 6:01am UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/9 "2012-11-08T06:01:59Z")
Hi, First to mention, I am not much into core elasticsearch code, but I do quite a bit of custom modification on the plugins part to support my business cases. My statement here might sound too novice. My apologies. I had faced the same problem stated here in this thread while I built the plugin directly using command as stated in link > **[GitHub - elastic/elasticsearch-river-twitter: Twitter River Plugin for...](https://github.com/elastic/elasticsearch-river-twitter)** > > Twitter River Plugin for elasticsearch (STOPPED). Contribute to elastic/elasticsearch-river-twitter development by creating an account on GitHub. I needed to modify the code of twitter river several times for custom requirements, and have faced two compilation problems. Resolving those problems and building the jar on my end has incidentally resolved this re-connection problem. (I am using elasticsearch 0.19.9) Following are the changes I made in the code. I am really not very sure whether I made the correct changes or not, the experts can really help better here, but those just worked for me. ``` // TODO, we can update the status of the river to RECONNECT logger.warn("failed to connect after failure, throttling", e); threadPool.schedule(TimeValue.timeValueSeconds(10), ``` ThreadPool.Names.CACHED, new Runnable() { @Override public void run() { reconnect(); } }); changed to ``` // TODO, we can update the status of the river to RECONNECT logger.warn("failed to connect after failure, throttling", e); threadPool.schedule(TimeValue.timeValueSeconds(10), ``` ThreadPool.Names.CACHE, new Runnable() { @Override public void run() { reconnect(); } }); and .. ``` public void onException(Exception ex) { logger.warn("stream failure, restarting stream...", ex); threadPool.cached().execute(new Runnable() { @Override public void run() { reconnect(); } }); } ``` changed to ``` @Override public void onException(Exception ex) { logger.warn("stream failure, restarting stream...", ex); threadPool.executor("CACHED").execute(new Runnable() { @Override public void run() { reconnect(); } }); } ``` And to emphasize, these were creating compilation problem at my end, I didn't just do a random code change. Also when using behind corporate firewall, the following open issue described at [https://github.com/elasticsearch/elasticsearch-river-twitter/issues/15](https://github.com/elasticsearch/elasticsearch-river-twitter/issues/15) needs to be addressed. Regards, -- Sujoy On Tuesday, November 6, 2012 12:37:15 PM UTC+5:30, Mikola Senyk wrote: > Thank you David! > Your crontab script is nice temporary solution. -- ---
**Author:** ![sujoysett](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@sujoysett](https://discuss.elastic.co/u/sujoysett)\ **Post date:** [November 9, 2012, 7:53am UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/10 "2012-11-09T07:53:37Z")
The last change should be - public void onException(Exception ex) { logger.warn("stream failure, restarting stream...", ex); threadPool.executor(ThreadPool.Names.CACHE).execute(new Runnable() { @Override public void run() { logger.debug(" executing reconnection "); reconnect(); } }); } Sorry, that I copied from a wrong development location. ☹ - Sujoy On Thursday, November 8, 2012 11:31:59 AM UTC+5:30, Sujoy Sett wrote: > Hi, > > First to mention, I am not much into core elasticsearch code, but I do > quite a bit of custom modification on the plugins part to support my > business cases. My statement here might sound too novice. My apologies. > > I had faced the same problem stated here in this thread while I built the > plugin directly using command as stated in link > [GitHub - elastic/elasticsearch-river-twitter: Twitter River Plugin for elasticsearch (STOPPED)](https://github.com/elasticsearch/elasticsearch-river-twitter) > > I needed to modify the code of twitter river several times for custom > requirements, and have faced two compilation problems. Resolving those > problems and building the jar on my end has incidentally resolved > this re-connection problem. (I am using elasticsearch 0.19.9) > > Following are the changes I made in the code. I am really not very sure > whether I made the correct changes or not, the experts can really help > better here, but those just worked for me. > > ``` > // TODO, we can update the status of the river to RECONNECT > logger.warn("failed to connect after failure, throttling", e); > threadPool.schedule(TimeValue.timeValueSeconds(10), > > ``` > > ThreadPool.Names.CACHED, new Runnable() { > @Override > public void run() { > reconnect(); > } > }); > > changed to > > ``` > // TODO, we can update the status of the river to RECONNECT > logger.warn("failed to connect after failure, throttling", e); > threadPool.schedule(TimeValue.timeValueSeconds(10), > > ``` > > ThreadPool.Names.CACHE, new Runnable() { > @Override > public void run() { > reconnect(); > } > }); > > and .. > > ``` > public void onException(Exception ex) { > logger.warn("stream failure, restarting stream...", ex); > threadPool.cached().execute(new Runnable() { > @Override > public void run() { > reconnect(); > } > }); > } > > ``` > > changed to > > ``` > @Override > public void onException(Exception ex) { > logger.warn("stream failure, restarting stream...", ex); > threadPool.executor("CACHED").execute(new Runnable() { > @Override > public void run() { > reconnect(); > } > }); > } > > ``` > > And to emphasize, these were creating compilation problem at my end, I > didn't just do a random code change. Also when using behind corporate > firewall, the following open issue described at > [https://github.com/elasticsearch/elasticsearch-river-twitter/issues/15](https://github.com/elasticsearch/elasticsearch-river-twitter/issues/15) needs > to be addressed. > > Regards, > -- Sujoy > > On Tuesday, November 6, 2012 12:37:15 PM UTC+5:30, Mikola Senyk wrote: > > > Thank you David! > > Your crontab script is nice temporary solution. -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:05am UTC](https://discuss.elastic.co/t/twitter-river-stopping-after-an-arror/7099/11 "2017-07-06T03:05:12Z")