Number of replicas automatically reset to 1


(Antoine Garrido) #4

thanks, where i can find that ? Even if i don't launch the script (only made to pass data to ES) the status changes to yellow without reasons ...
Add : to be precise:
i create my index 'coloc', set the number_of_replicas to 0 then fill my index with a script. ES create automatic template to fill an index ? If you need my script tell me. I really need help on this :wink:


(David Pilato) #5

If you are using Kibana, then you probably have one replica for its index.

But there is nothing wrong to have a yellow cluster. What problem you want to solve?


(Antoine Garrido) #6

I am not using Kibana. ElasticSearch is installed on the server and launched as a service.
ES is used to store some datas and displays them on a Google Maps API.
So i have a script to explore my database and fill ElasticSearch.
But when I launch this script, it works for 10min (maybe) and then i get this error message Elasticsearch\Common\Exceptions\ServerErrorResponseException
So i check the _cluster/health it appears to be yellow. Then i check my index localhost:9200/coloc and SURPRISE, the number_of_replicas is set to 1. I noted that ES works perfectly if my index's replicas stay to 0. But it doesnt... I am very lost :confused:


(David Pilato) #7

Why do you mean by "work perfectly"?
What does not work when the cluster is yellow?


(Antoine Garrido) #8

Yesterday, during all the process of my script, the cluster stayed to green state. The script has finished to load my datas in ES without any problems and my markers were set on my Google maps api.

Today, nothing works... If i launch the script i get the error "Elasticsearch\Common\Exceptions\ServerErrorResponseException" and the cluster state becomes yellow.

You talked about template earlier, how this can help me ?

Add : ES receives a part of my data then ServerErrorResponse... I think ES is not stable but don't know why
[root@stock /]# curl localhost:9200/_cluster/health?pretty { "cluster_name" : "elasticsearch", "status" : "yellow", "timed_out" : false, "number_of_nodes" : 1, "number_of_data_nodes" : 1, "active_primary_shards" : 5, "active_shards" : 5, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 5, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 50.0 }

[root@stock /]# curl localhost:9200/coloc

{"coloc":{"aliases":{},"mappings":{"annonces_propose":{"properties":{"location":{"type":"geo_point"},"price":{"type":"float"},"publish_date":{"type":"date","format":"yyyy-MM-dd HH:mm:ss"}}}},"settings":{"index":{"creation_date":"1524914951379","number_of_shards":"5","number_of_replicas":"1","uuid":"Z9UUTzcLRlq_hsrRTw7bBQ","version":{"created":"5000299"},"provided_name":"coloc"}}}}


(David Pilato) #9

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.


(David Pilato) #10

Everything seems ok here.

May be something wrong with your code.

Here is the link for index templates: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html

But this is not going to change anything IMO.

May be share also elasticsearch logs?


(Antoine Garrido) #11

Except the filling data log i get that :

org.elasticsearch.env.ShardLockObtainFailedException: [coloc][4]: obtaining shard lock timed out after 0ms
        at org.elasticsearch.env.NodeEnvironment$InternalShardLock.acquire(NodeEnvironment.java:711) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.env.NodeEnvironment.shardLock(NodeEnvironment.java:630) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.env.NodeEnvironment.lockAllForIndex(NodeEnvironment.java:576) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.env.NodeEnvironment.deleteIndexDirectorySafe(NodeEnvironment.java:527) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.indices.IndicesService.deleteIndexStoreIfDeletionAllowed(IndicesService.java:642) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.indices.IndicesService.deleteIndexStore(IndicesService.java:629) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.indices.IndicesService.removeIndex(IndicesService.java:517) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.indices.IndicesService.deleteIndex(IndicesService.java:572) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.indices.cluster.IndicesClusterStateService.deleteIndices(IndicesClusterStateService.java:258) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:192) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cluster.service.ClusterService.runTasksForExecutor(ClusterService.java:729) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cluster.service.ClusterService$UpdateTask.run(ClusterService.java:915) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:458) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:238) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:201) [elasticsearch-5.0.2.jar:5.0.2]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]

and this from my last test script launching :

[2018-04-28T16:00:01,738][INFO ][o.e.c.m.MetaDataCreateIndexService] [M32YSFl] [coloc] creating index, cause [api], templates [], shards [5]/[1], mappings [annonces_propose]
[2018-04-28T16:02:32,513][INFO ][o.e.c.m.MetaDataCreateIndexService] [M32YSFl] [coloc] creating index, cause [api], templates [], shards [5]/[1], mappings [annonces_propose]
[2018-04-28T16:04:31,332][INFO ][o.e.c.m.MetaDataCreateIndexService] [M32YSFl] [coloc] creating index, cause [api], templates [], shards [5]/[1], mappings [annonces_propose]
[2018-04-28T16:06:31,559][INFO ][o.e.c.m.MetaDataCreateIndexService] [M32YSFl] [coloc] creating index, cause [api], templates [], shards [5]/[1], mappings [annonces_propose]
[2018-04-28T16:20:04,975][INFO ][o.e.c.m.MetaDataCreateIndexService] [M32YSFl] [coloc] creating index, cause [api], templates [], shards [5]/[1], mappings [annonces_propose]
[2018-04-28T16:22:23,157][INFO ][o.e.c.m.MetaDataCreateIndexService] [M32YSFl] [coloc] creating index, cause [api], templates [], shards [5]/[1], mappings [annonces_propose]
[2018-04-28T16:23:16,678][INFO ][o.e.c.m.MetaDataCreateIndexService] [M32YSFl] [coloc] creating index, cause [api], templates [], shards [5]/[0], mappings [annonces_propose]
[2018-04-28T16:23:17,506][INFO ][o.e.c.r.a.AllocationService] [M32YSFl] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[coloc][4]] ...]).
[2018-04-28T16:23:17,823][INFO ][o.e.c.m.MetaDataMappingService] [M32YSFl] [coloc/r2_vcVFxSe-BpzyL3_XhdA] update_mapping [annonces_propose]
[2018-04-28T16:23:20,844][INFO ][o.e.c.m.MetaDataMappingService] [M32YSFl] [coloc/r2_vcVFxSe-BpzyL3_XhdA] update_mapping [annonces_propose]
[2018-04-28T16:32:17,376][INFO ][o.e.c.m.MetaDataCreateIndexService] [M32YSFl] [coloc] creating index, cause [api], templates [], shards [5]/[1], mappings [annonces_propose]

I don't really see an error that can cause the stop of my script. What do you think about it ? What kind of error i must search ? The file log is much too long to put it there.


(David Pilato) #12

You can share the full log on gist.github.com

And share exactly what your script is doing.

Also, please use instead latest 6.2 version if possible.


(Antoine Garrido) #13

Hello,

Here is the link to my elasticsearch logs and the script I use to fill it.


(David Pilato) #14

Any idea why you have so many indices creation call?

Are you removing manually, then testing again?

Sadly I don't really read PHP code. But may be this is because you put in __construct() method the creation itself, which means to me that anytime you create an instance, it creates one index?
BTW number of shards could be set to 1 as well but I don't understand why this is not happening.
It's like you are calling:

PUT coloc

instead of

PUT coloc
{
  "settings": {
    // ...
  },
  "mappings": {
    // ...
  }
}

(Antoine Garrido) #15

I was using an older version of ES before this one. Is it possible it use an old config file ?? I cleaned up all the server's folder before installing new version.


(David Pilato) #16

I don't know.


(Antoine Garrido) #17

J'ai du nouveau! Le script ne plante plus. Toutes mes données sont récupérées mais (nouveauté) ne sont pas affichées à la demande... Je remarque que sous le dossier
/var/lib/elasticsearch/nodes/0/indices
de mon serveur, le dossier généré automatiquement change constamment de nom pendant que je remplis ES avec mes données... Est-ce un comportement normal ?


(David Pilato) #18

J'ai déplacé ta question dans #in-your-native-tongue:discussions-en-francais.

le dossier généré automatiquement change constamment de nom pendant que je remplis ES avec mes données..

Peut-être mais de toute façon tu ne devrais pas aller farfouiller dans ce répertoire.

Utilise les API à la place. Genre:

GET _cat/indices?v

(Antoine Garrido) #19

J'ai déplacé ta question dans #in-your-native-tongue:discussions-en-francais.

Super merci, je ne me suis pas rendu compte de la langue dans laquelle j'écrivais.
Voilà la sortie de la commande

curl localhost:9200/_cat/coloc?v

[root@stock elasticsearch]# curl localhost:9200/_cat/indices?v
health status index uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   coloc chMicsd6QYmWNHsNKEdyaA   5   1          0            0       650b           650b

et 30 secondes plus tard....

[root@stock elasticsearch]# curl localhost:9200/_cat/indices?v
health status index uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   coloc K2dcfFjcQOu5OrDs-65KIw   5   1          0            0       650b           650b

(David Pilato) #20

Ca veut dire que tu détruis l'index à chaque run et que tu le reconstruis.


(Antoine Garrido) #22

Est-ce que quelqu'un qui utilise PHP devant ElasticSearch pourrait m'aiguiller sur le debug de mon script ? Comme le précise dadonnet :

Ca veut dire que tu détruis l'index à chaque run et que tu le reconstruis.

Mais impossible de comprendre comment j'arrive à ce résultat ...

D'avance merci :wink:


(David Pilato) #23

Cette ligne ?

$response = $this->elastic->indices()->delete($params);

(Antoine Garrido) #24

Je l'ai commentée pour essayer. A la base, le but était qu'à chaque initialisation, on repart à zéro et on reconstruis tout. Toujours pas d'affichage cependant.
Voilà où j'utilise les fonctions citées precedemment :

public function elasticAddAction(){
        $annonces = $this->annonceTable->fetchAnnoncesPropose();

        $i = 0;

        $this->searchService->init();
        foreach($annonces as $annonce){
            $city = $this->villeTable->getVille($annonce->id_ville);
            $coordinates = $this->searchService->get_coordinates($annonce->adresse.', '.$annonce->nom_ville,$city->cp);

            if($coordinates !== false){
                $this->searchService->reset_conn();

                $url_provider = $this->urlProvider;
                $url = $url_provider(null, $annonce->id_membre, $annonce, false, true, null, '#form-search-global');

                $result = $this->searchService->add_annonce($annonce, $coordinates['lat'],$coordinates['long'], $url);

                var_dump($result);

                //break;
                //break;
                if($i==15){
                    //break;
                }
                $i++;
            }
            else{
                var_dump($annonces);
                var_dump(urlencode($annonce->adresse.' '.$annonce->nom_ville));
            }
            sleep(1.5);
        }

        echo 'FIn ajout';

    }

Par ailleurs j'ajoute l'erreur obtenue la plus récente :

The application has thrown an exception!
======================================================================
Elasticsearch\Common\Exceptions\Missing404Exception
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","index_uuid":"fsA12HldQsGUgxEBjA--Wg","index":"coloc"}],"type":"index_not_found_exception","reason":"no such index","index_uuid":"fsA12HldQsGUgxEBjA--Wg","index":"coloc"},"status":404}