ElasticSearch WARN after installing kibana

Elastic Stack Elasticsearch
#1

After installing kibana,when i operate(GET/POST/DELETE and so on) Elastic Search with High Level Rest Clinet comes up this warning:

Each request comes up with one warning,i have 500 millions documents to process every months,this problem would flush my logs and i can't locate the new problem.

i wonder what causes this problem and how to fix it.Thanks.

Version:
-Elastic Search/kibana: 7.10.1

#2

Welcome to our community! :smiley:

What URL does it mention?

If this is Kibana, then it won't be displayed every single time a new document is ingested.

#3

Thanks for your reply
Urls:

  • POST https://{hostname}:9200/_bulk?timeout=1m (would post this request like million times)
  • GET https://{hostname}:9200/_cat/indices?index=user_data-*
  • DELETE https://{hostname}:9200/user_data-20210325160624?master_timeout=30s&ignore_unavailable=false&expand_wildcards=open%2Cclosed&allow_no_indices=true&ignore_throttled=false&timeout=30s

Thanks again.

#4

Does anybody have the same question and resolve?

#5

I'm surprised that those queries generate a log which is not related.

High Level Rest Clinet

What is the exact Java code you run when this log is printed?

#6

Thanks for your reply!
Here is the java code:

        Assert.notNull(indexName, "es bulk index with null index name!");
        Assert.notNull(userDatas, "es bulk index with null requests entity!");
        if(CollectionUtils.isEmpty(userDatas)){
            return;
        }

        BulkRequest bulkRequest = new BulkRequest();
        for(UserData userData : userDatas){
            Assert.notNull(userData, "userData is null!");
            log.debug(userData.toString());

            RedisUserLabelData labelData;
            try {
                labelData = RedisUserLabelData.buildFromMysqlUserData(userData);
            } catch (Exception e){
                log.error("RedisUserLabelData build from mysql entity UserData failed!", e);
                continue;
            }

            Map<String, Object> dataMap;
            try {
                dataMap = PropertyUtils.describe(labelData);

                /** 去掉class字段 **/
                dataMap.remove("class");
            } catch (Exception e){
                throw new RuntimeException("beanUtil, bean to map transform failed!", e);
            }

            Assert.notNull(dataMap, "beanUtil, bean to map result null map");
            Assert.notEmpty(dataMap, "beanUtil, bean to map result empty map");

            IndexRequest indexRequest = new IndexRequest("posts").source(dataMap).index(indexName);
            bulkRequest.add(indexRequest);

        }

        if(bulkRequest.numberOfActions() != userDatas.size()){
            throw new RuntimeException("bulk request has requests number not equal to the parameter collection size!");
        }

        if(bulkRequest.numberOfActions() > 0) {
            BulkResponse bulkResponse = null;
            try {
                bulkResponse = restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);
            } catch (IOException e) {
                throw new RuntimeException("Error connect to Es", e);
            }

            if(bulkResponse.hasFailures()){
                BulkItemResponse[] bulkItemResponseArr = bulkResponse.getItems();
                for(BulkItemResponse bulkItemResponse : bulkItemResponseArr){
                    if(bulkItemResponse.isFailed()){
                        log.error("error while indexing : " + bulkItemResponse.getFailureMessage(), bulkItemResponse.getFailure().getCause());
                    }
                }
            }
        }

I find no way to highlight the exact code where the log is printed, so i quote here:

bulkResponse = restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);

Everytime I send request(get/post/delete etc) to ElasticSearch,the log is printed.In order not to delay your time, i do not show the other code.
Thanks again!

#7 
IndexRequest indexRequest = new IndexRequest("posts").source(dataMap).index(indexName);

Why did you add .index(indexName)?
And what is the value of indexName?

#8

how does it know which index I want to update if not add .index(indexName) ?
The indexName is
"UserData" + "-" + new SimpleDateFormat("yyyyMMddHHmmss").format(Calendar.getInstance().getTime()
After the update finishing, the index would be alias as "UserData"

#9

But you set the index name with:

new IndexRequest("posts")

So I'd write instead:

IndexRequest indexRequest = new IndexRequest(indexName).source(dataMap);

And BTW, I think you are missing the JSON content type.

I think it should be:

IndexRequest indexRequest = new IndexRequest(indexName).source(dataMap, XContentType.JSON));

(But unsure as I never use maps to send data to Elasticsearch).

I can't see the correlation with the log message...

this request accesses system indices: [.kibana_1, .kibana_task_manager_1]

I'm wondering if we are really looking at the exact code which is generating this log.
I'd have expect a _search request TBH.

Are you sure you are not running a Search after the bulk or at the same time?

#10

Yes, you're right.Thanks for your correction.

I'm convinced that there no Search running at the same tiem.

Search also would print the log as follow:
2021-04-02 15:59:10 [20210402155858991] [http-nio-8233-exec-1] WARN org.elasticsearch.client.RestClient - request [PUT https://hostname:9200/sync_ads_order_persist-in/_doc/37944?timeout=1m] returned 1 warnings: [299 Elasticsearch-7.10.1-1c34507e66d7db1211f66f3513706fdf548736aa "this request accesses system indices: [.kibana_1, .kibana_task_manager_1], but in a future major version, direct access to system indices will be prevented by default"]

BTW, I am wondering if Search Guard leads to this problem, because I installed Search Guard and Kibana at the same time.

Thank for your help and your patience.

#11

Ha! That's probably it.

We don't support this 3rd party plugin here. We have security available in our default free distribution.