Index doc, pdf, odt .... => cluster : yellow, why?

Tanguy_Bernard · July 10, 2014, 1:35pm

Hello, this is my code when I index ( I use this because I want to make a
search inside a document (on this content))

PUT test

PUT test/my_type/_mapping
{

"my_type" : {*
```
   "properties" : {*
```
```
       "my_file" : {*
```
```
           "type" : "attachment",*
```
```
           "fields" : {*
```

               "my_file" : { "term_vector":"with_positions_offsets"}*

```
           }*
```
```
       }*
```
```
   }*
```
}*
}

I open my doc, I convert it in base64 and I index it.

PUT test/my_type/1
{

"my_file" : "my file in base64",*
"name": "the name of file"*

}

And I search :

GET test/my_etape/_search?pretty=true
{"size": 50,
"query": {

"query_string": {*
```
  "query": "my keywords"*
```
}*
},"highlight": {"fields": {"my_file":{"term_vector" : "*
with_positions_offsets*"}*
}
}
}

I don't understand why my cluster turn yellow ?
Have you an explanation for me or an other way to do this, please ?

Thank to you in advance.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c7ec2914-0498-414d-96a1-ccffeff93148%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Rafal_Kuc_3 · July 10, 2014, 1:39pm

Hello!

Can you check and paste the cluster health (curl -XGET 'localhost:9200/_cluster/health?pretty')?

You probably have a single Elasticsearch node and you are suing the default configuration? If so than, by default, Elasticsearch created 5 shards and 1 replica of each. This means that while the 5 primary shards were properly initialized, the replicas are left alone (by default ES don't put primary shard and its replicas on the same node).

--

Regards,

Rafał Kuć

Performance Monitoring * Log Analytics * Search Analytics

Solr & Elasticsearch Support * http://sematext.com/

Hello, this is my code when I index ( I use this because I want to make a search inside a document (on this content))

PUT test

PUT test/my_type/_mapping

{

"my_type" : {


    "properties" : {


        "my_file" : {


            "type" : "attachment",


            "fields" : {


                "my_file" : { "term_vector":"with_positions_offsets"}


            }


        }


    }


}

}

I open my doc, I convert it in base64 and I index it.

PUT test/my_type/1

{

"my_file" : "my file in base64",

"name": "the name of file"

}

And I search :

GET test/my_etape/_search?pretty=true

{"size": 50,

"query": {

"query_string": {


   


   "query": "my keywords"


   


}


},"highlight": {"fields": {"my_file":{"term_vector" : "with_positions_offsets"}

}

I don't understand why my cluster turn yellow ?

Have you an explanation for me or an other way to do this, please ?

Thank to you in advance.

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c7ec2914-0498-414d-96a1-ccffeff93148%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Tanguy_Bernard · July 10, 2014, 1:43pm

Health of cluster: yellow (35 50)

curl -XGET 'localhost:9200/_cluster/health?pretty'
{
"cluster_name": "elasticsearch",
"status": "yellow",
"timed_out": false,
"number_of_nodes": 1,
"number_of_data_nodes": 1,
"active_primary_shards": 35,
"active_shards": 35,
"relocating_shards": 0,
"initializing_shards": 0,
"unassigned_shards": 15
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b7d49cbb-687f-4eac-8b54-9b5b24c886db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Rafal_Kuc_3 · July 10, 2014, 1:57pm

Hello!

You can see this is exactly the case - you have a single node and Elasticsearch won't assign primary shard and replicas of it on the same node. If you would run a second node Elasticsearch would allocate that 15 unassigned shards and rebalance the shards between the two node cluster - try that out, even locally.

Of course, if you are using that single node for development you can just remove the replicas by running:

curl -XPUT 'localhost:9200/_settings' -d '{"index" : { "number_of_replicas" : 0 }}'

However remember that in production you usually don't want to have primaries and their replicas on the same node and you want more than a single Elasticsearch node running.

--

Regards,

Rafał Kuć

Performance Monitoring * Log Analytics * Search Analytics

Solr & Elasticsearch Support * http://sematext.com/

Health of cluster: yellow (35 50)

curl -XGET 'localhost:9200/_cluster/health?pretty'

{

"cluster_name": "elasticsearch",

"status": "yellow",

"timed_out": false,

"number_of_nodes": 1,

"number_of_data_nodes": 1,

"active_primary_shards": 35,

"active_shards": 35,

"relocating_shards": 0,

"initializing_shards": 0,

"unassigned_shards": 15

}

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b7d49cbb-687f-4eac-8b54-9b5b24c886db%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Tanguy_Bernard · July 10, 2014, 2:11pm

Thank you.

In my case I have many client, I use one index by client and a single node.
I work with elasticsearch recently and for the moment I don't understand
why to use a second node ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a7920608-37ce-4f17-b6f7-4c9b2806dacb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Rafal_Kuc_3 · July 10, 2014, 2:30pm

Hello!

Multiple nodes for high availability - everything crashes, and you don't want to loose Elasticsearch and the ability to search and analyze your data. Also for better performance - you can have your index built of many shards that are spread across multiple nodes and have replicas of that shards, so you can physical copies that can handle the traffic if one node is not able to handle it.

--

Regards,

Rafał Kuć

Performance Monitoring * Log Analytics * Search Analytics

Solr & Elasticsearch Support * http://sematext.com/

Thank you.

In my case I have many client, I use one index by client and a single node.

I work with elasticsearch recently and for the moment I don't understand why to use a second node ?

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a7920608-37ce-4f17-b6f7-4c9b2806dacb%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Tanguy_Bernard · July 10, 2014, 2:34pm

Thank you Rafał Kuć. Really interesting.

Tanguy

Le jeudi 10 juillet 2014 16:30:25 UTC+2, Rafał Kuć a écrit :

Hello!

Multiple nodes for high availability - everything crashes, and you don't
want to loose Elasticsearch and the ability to search and analyze your
data. Also for better performance - you can have your index built of many
shards that are spread across multiple nodes and have replicas of that
shards, so you can physical copies that can handle the traffic if one node
is not able to handle it.

*-- Regards, Rafał Kuć Performance Monitoring * Log Analytics * Search
Analytics Solr & Elasticsearch Support * *http://sematext.com/

Thank you.

In my case I have many client, I use one index by client and a single node.
I work with elasticsearch recently and for the moment I don't understand
why to use a second node ?

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a7920608-37ce-4f17-b6f7-4c9b2806dacb%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a7920608-37ce-4f17-b6f7-4c9b2806dacb%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/981734e9-1d3f-4a42-ad63-e637fe373e7d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Steven_Williams · March 3, 2015, 9:29am

Hello

I get Yellow status cluster as well.

I have 2 nodes (two different machines) and on occasion the cluster goes
from green to yellow (my cluster health is below, and at the moment all is
good)

"cluster_name" : "elasticsearch",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2,
"active_primary_shards" : 11,
"active_shards" : 22,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0
}

I have read what the different status colours mean, but I cant find
anywhere that says WHY these things happen i.e. network issue, cpu load etc
etc.

Are the any main reasons that would give cause to red / yellow reasons?

ive got some monitoring on my cluster to alert me when its not green, and I
can go and restart the slave server, then it goes back to green.. but still
this does not identify why this happens?

Any comments would be greatly appreciated

Cheers

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8807a6e5-6060-48a1-b052-5aa09ee4434f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

warkolm · March 3, 2015, 10:39am

You should really create a new thread for this rather than resurrecting an
old one.

On 3 March 2015 at 20:29, Steven Williams cybordax@googlemail.com wrote:

Hello

I get Yellow status cluster as well.

I have 2 nodes (two different machines) and on occasion the cluster goes
from green to yellow (my cluster health is below, and at the moment all is
good)

"cluster_name" : "elasticsearch",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2,
"active_primary_shards" : 11,
"active_shards" : 22,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0
}

I have read what the different status colours mean, but I cant find
anywhere that says WHY these things happen i.e. network issue, cpu load etc
etc.

Are the any main reasons that would give cause to red / yellow reasons?

ive got some monitoring on my cluster to alert me when its not green, and
I can go and restart the slave server, then it goes back to green.. but
still this does not identify why this happens?

Any comments would be greatly appreciated

Cheers

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8807a6e5-6060-48a1-b052-5aa09ee4434f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/8807a6e5-6060-48a1-b052-5aa09ee4434f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8Ms9ZLCZg4obVxH0c1OdOcmL1U0wB2C_yP9go3r-aQ-A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Steven_Williams · March 3, 2015, 10:46am

i shall do that, thanks

On Tue, Mar 3, 2015 at 9:29 AM, Steven Williams cybordax@googlemail.com
wrote:

Hello

I get Yellow status cluster as well.

I have 2 nodes (two different machines) and on occasion the cluster goes
from green to yellow (my cluster health is below, and at the moment all is
good)

"cluster_name" : "elasticsearch",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2,
"active_primary_shards" : 11,
"active_shards" : 22,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0
}

I have read what the different status colours mean, but I cant find
anywhere that says WHY these things happen i.e. network issue, cpu load etc
etc.

Are the any main reasons that would give cause to red / yellow reasons?

ive got some monitoring on my cluster to alert me when its not green, and
I can go and restart the slave server, then it goes back to green.. but
still this does not identify why this happens?

Any comments would be greatly appreciated

Cheers

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/-jKGlBBuEn4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8807a6e5-6060-48a1-b052-5aa09ee4434f%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/8807a6e5-6060-48a1-b052-5aa09ee4434f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANRN3tbcQJHqo1c2Pwp0GMmHJZf7eF-LDZ0R%3DqcMjyBCzgokQA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.