What to do if ES document count continuously increases - with zero indexing ongoing


(Swaroop CH) #1

Hello,

We have bulk-indexed a fresh ES 1.0.1 cluster with index.refresh_interval : -1 (disabled), and then set index.refresh_interval: 5s about 20 hours ago, the document count is continuously increasing since then, is this normal or expected? How do I know when it'll be done? Our expected document count was about half of the current document count in the new cluster.

1394593149 02:59:09 167713281
1394593153 02:59:13 167723653
1394593156 02:59:16 167720017
...
1394593220 03:00:20 167800614
1394593224 03:00:24 167812913
1394593228 03:00:28 167812056

( while true; do curl -s http://54.xxx.xxx.xxx:9201/_cat/count; sleep 2; done )

Looking forward to any advice or suggestions on what to look into.

Thanks.

Regards,
Swaroop

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/355871394593302%40web26h.yandex.ru.
For more options, visit https://groups.google.com/d/optout.


(Swaroop CH) #2

The source of the problem is Marvel - is there anyway to disable Marvel indexing?

Trying to set marvel.agent.indices: "-*" says "ignoring transient setting [marvel.agent.indices], not dynamically updateable"

Regards,
Swaroop

12.03.2014, 08:31, "Swaroop CH" swaroopch@yandex.com:

Hello,

We have bulk-indexed a fresh ES 1.0.1 cluster with index.refresh_interval : -1 (disabled), and then set index.refresh_interval: 5s about 20 hours ago, the document count is continuously increasing since then, is this normal or expected? How do I know when it'll be done? Our expected document count was about half of the current document count in the new cluster.

1394593149 02:59:09 167713281
1394593153 02:59:13 167723653
1394593156 02:59:16 167720017
...
1394593220 03:00:20 167800614
1394593224 03:00:24 167812913
1394593228 03:00:28 167812056

( while true; do curl -s http://54.xxx.xxx.xxx:9201/_cat/count; sleep 2; done )

Looking forward to any advice or suggestions on what to look into.

Thanks.

Regards,
Swaroop

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/355871394593302%40web26h.yandex.ru.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/55311394602944%40web7h.yandex.ru.
For more options, visit https://groups.google.com/d/optout.


(Clinton Gormley) #3

Have you looked at the docs?

http://www.elasticsearch.org/guide/en/marvel/current/index.html#configuration

On 12 March 2014 06:42, Swaroop CH swaroopch@yandex.com wrote:

The source of the problem is Marvel - is there anyway to disable Marvel
indexing?

Trying to set marvel.agent.indices: "-*" says "ignoring transient
setting [marvel.agent.indices], not dynamically updateable"

Regards,
Swaroop

12.03.2014, 08:31, "Swaroop CH" swaroopch@yandex.com:

Hello,

We have bulk-indexed a fresh ES 1.0.1 cluster with
index.refresh_interval : -1 (disabled), and then set
index.refresh_interval: 5s about 20 hours ago, the document count is
continuously increasing since then, is this normal or expected? How do I
know when it'll be done? Our expected document count was about half of the
current document count in the new cluster.

1394593149 02:59:09 167713281
1394593153 02:59:13 167723653
1394593156 02:59:16 167720017
...
1394593220 03:00:20 167800614
1394593224 03:00:24 167812913
1394593228 03:00:28 167812056

( while true; do curl -s http://54.xxx.xxx.xxx:9201/_cat/count; sleep
2; done )

Looking forward to any advice or suggestions on what to look into.

Thanks.

Regards,
Swaroop

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/355871394593302%40web26h.yandex.ru
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/55311394602944%40web7h.yandex.ru
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSrxe0qZKX%2BWWDTpQWFN3rBsDcjFpiOzkLA2YZMpz1gew%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Swaroop CH) #4

Hi Clinton, Thanks for the reply. I did look at the docs, I tried setting marvel.agent.enabled: false via the API, but ES logs an error saying it is "not dynamically updateable", and so on. Eventually, I solved it by disabling shard allocation, shutting down all the nodes of the cluster, uninstalling Marvel, and starting master-only nodes followed by data-only nodes. Regards,

Swaroop 12.03.2014, 16:18, "Clinton Gormley" <clint@traveljury.com>:

Have you looked at the docs? http://www.elasticsearch.org/guide/en/marvel/current/index.html#configuration

On 12 March 2014 06:42, Swaroop CH <swaroopch@yandex.com> wrote:

The source of the problem is Marvel - is there anyway to disable Marvel indexing?

Trying to set marvel.agent.indices: "-*" says "ignoring transient setting [marvel.agent.indices], not dynamically updateable"

Regards,

Swaroop

12.03.2014, 08:31, "Swaroop CH" <swaroopch@yandex.com>:

> Hello,

>

> We have bulk-indexed a fresh ES 1.0.1 cluster with index.refresh_interval : -1 (disabled), and then set index.refresh_interval: 5s about 20 hours ago, the document count is continuously increasing since then, is this normal or expected? How do I know when it'll be done? Our expected document count was about half of the current document count in the new cluster.

>

> 1394593149 02:59:09 167713281

> 1394593153 02:59:13 167723653

> 1394593156 02:59:16 167720017

> ...

> 1394593220 03:00:20 167800614

> 1394593224 03:00:24 167812913

> 1394593228 03:00:28 167812056

>

> ( while true; do curl -s http://54.xxx.xxx.xxx:9201/_cat/count; sleep 2; done )

>

> Looking forward to any advice or suggestions on what to look into.

>

> Thanks.

>

> Regards,

> Swaroop

>

> --

> You received this message because you are subscribed to the Google Groups "elasticsearch" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/355871394593302%40web26h.yandex.ru.

> For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/55311394602944%40web7h.yandex.ru.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKSrxe0qZKX%2BWWDTpQWFN3rBsDcjFpiOzkLA2YZMpz1gew%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/401231394645319%40web9m.yandex.ru.

For more options, visit https://groups.google.com/d/optout.


(Clinton Gormley) #5

On 12 March 2014 18:28, Swaroop CH swaroopch@yandex.com wrote:

Thanks for the reply. I did look at the docs, I tried setting
marvel.agent.enabled: false via the API, but ES logs an error saying it
is "not dynamically updateable", and so on.

Yes, you have to set it in the config file (at least for now)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPt3XKRsZ2xLMPFr6-O2W__HuK2iyEhYwU9bnn%3DVfbKgZM1kyQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #6