Refresh_interval

Elastic Stack Elasticsearch
1

hello

I have eight elasticsearch data nodes (ver. 6.4.1) (8 shards, 2 replica)

they have following data

indexed disk size : 21.5GB
Document Count : 67.7m
Segment Count : 336

A large amount of data is bached and indexed at once
I know there is a way to adjust the segment size

please advise how it is good to measure and adjust for quick search, index

thanks

2

Welcome!

It's not clear here what is the problem you are facing. Could you describe what you have today which looks wrong to you and what you'd like it to be?

3

sorry
not good at english T T

I'm indexing using multiple batch process
i want to reduce the batch process time without affecting the search
(indexing rate avg : 45,000/s, indexing latency avg: 0.3ms)
i think reducing the segment size will speed up the batch process

Is there a way to tune?

4

What is the output of:

GET /
GET /_cat/nodes?v
GET /_cat/health?v
GET /_cat/indices?v

If some outputs are too big, please share them on gist.github.com and link them here.

5

here is the output
some names have changed

GET /

{
  "name": "name",
  "cluster_name": "name",
  "cluster_uuid": "id",
  "version": {
    "number": "6.4.3",
    "build_flavor": "default",
    "build_type": "tar",
    "build_hash": "",
    "build_date": "",
    "build_snapshot": false,
    "lucene_version": "7.4.0",
    "minimum_wire_compatibility_version": "5.6.0",
    "minimum_index_compatibility_version": "5.0.0"
  },
  "tagline": "You Know, for Search"
}

GET /_cat/nodes?v

ip           heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
1.1.1.1           30          93   3    1.77    1.83     1.66 di        -      d6.xxx.com
1.1.1.2           30          93   6    2.42    2.47     2.44 di        -      d2.xxx.com
1.1.1.3           51           6   0    0.00    0.01     0.00 m         -      m3.xxx.com
1.1.1.4           14          87   4    2.66    2.90     2.70 di        -      d4.xxx.com
1.1.1.5           5           61   0    0.04    0.01     0.00 -         -      c1.xxx.com
1.1.1.6           71          93   3    2.21    2.32     2.08 di        -      d8.xxx.com
1.1.1.7           46          92   4    3.03    3.34     3.05 di        -      d3.xxx.com
1.1.1.8           21          91   9    2.08    1.82     1.63 di        -      d1.xxx.com
1.1.1.9           3           56   0    0.12    0.14     0.14 -         -      c2.xxx.com
1.1.1.10          7           57   0    0.03    0.10     0.06 m         -      m2.xxx.com
1.1.1.11          59          92   2    2.60    2.51     2.10 di        -      d5.xxx.com
1.1.1.12          24          93   1    1.38    2.03     2.05 di        -      d7.xxx.com
1.1.1.13          12          60   0    0.19    0.23     0.12 m         *      m1.xxx.com

GET /_cat/health?v

epoch      timestamp cluster    status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1580285411 17:10:11  filter green          13         8    202 101    0    0        0             0                  -                100.0%

GET /_cat/indices?v

health status index                                  uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .monitoring-es-6-2020.01.25            bO5PdJy0S3e6vFmVHOAxnw   1   1     623106         1442    654.5mb        326.8mb
green  open   .monitoring-es-6-2020.01.29            xRAsBYlxQc2JtSk3NU3IWA   1   1     211303          994    285.9mb        153.2mb
green  open   test_202001291710      d6vl1KXzTUeZsD5CZcn0Eg   8   1      87798            0     27.4mb         13.5mb
green  open   .monitoring-kibana-6-2020.01.26        ZG0t53TJRXSPasxD5ILGbQ   1   1       8638            0      4.3mb          2.1mb
green  open   .monitoring-es-6-2020.01.24            XRwQufivQgWdSLzBXHierw   1   1     613250         2222    647.3mb        327.1mb
green  open   test-snapshot                      wWUi1pUCR52Zds-6_4GsMQ   4   1   35669872     16443279      7.2gb          3.2gb
green  open   .monitoring-kibana-6-2020.01.25        QBz2H-DTQDO5NBfZ3AiyYA   1   1       8639            0      4.1mb            2mb
green  open   test2_202001290600 8QqSd73IRo-eXzk-K_Q49g   8   1  216760922            0     27.1gb         13.4gb
green  open   test5_202001291427                  4tXIOmjjQUCqe4JmU4lCtg   8   1   69447852            0     21.7gb         10.8gb
green  open   .monitoring-kibana-6-2020.01.24        YK20iA0TTyWoxA2n_tyQug   1   1       8639            0      4.2mb          2.1mb
green  open   .monitoring-es-6-2020.01.26            QevGtFv2QgOuDhmeeMaWMw   1   1     618372         2210    651.3mb        327.7mb
green  open   .monitoring-kibana-6-2020.01.23        9gcHaLfvTtCAK7-l1jA9pw   1   1       8639            0      4.2mb            2mb
green  open   .monitoring-es-6-2020.01.23            ZLqMPc4xQ0SGWL883TL3qg   1   1     620700         1608    642.4mb        325.2mb
green  open   .monitoring-kibana-6-2020.01.28        UN3r6PHOSxWLMRC8OF0xnw   1   1       8639            0      4.3mb          2.1mb
green  open   .monitoring-es-6-2020.01.28            pQGnFAYEQaSQPJr4OxmfUw   1   1     621271         1374    645.5mb        322.9mb
green  open   test5_202001291533                  2oHJplhJRmC4vT3oqHWlpw   8   1   69439421            0     21.9gb         10.9gb
green  open   test2_202001291200 LLf026xOQHaOn3K2gnWCpg   8   1  216789310            0     27.1gb         13.4gb
green  open   .monitoring-kibana-6-2020.01.29        rhrhtGIeSW6v8_woMFg1Dg   1   1       2946            0      1.5mb        783.5kb
green  open   .monitoring-es-6-2020.01.27            YOOp1jkTRX6hlHsN-BdGXg   1   1     619541          630    639.2mb        319.4mb
green  open   test1_202001291636                  SwYOh7TARqWD3JrONQWKUQ   8   1   43859694            0     17.4gb            9gb
green  open   .monitoring-kibana-6-2020.01.22        LOQiiG88TPO5j6waSFpuGg   1   1       8639            0      4.4mb          2.2mb
green  open   .monitoring-kibana-6-2020.01.27        JW0NrOzMRV-yKMl1QPTblg   1   1       8639            0      4.3mb          2.1mb
green  open   .kibana                                1QWEu2QxTG2PCRmIAMq1rg   1   1          3            0     35.5kb         17.7kb
green  open   test4_202001291705      _vmWfIqNTqup2H5jDjEXoA   8   1      87775            0     27.4mb         13.6mb
green  open   test3_202001291703                 sSZ5c_zBQTWlxHoLNb2uXA   8   1    1903663           79    475.3mb        236.2mb
green  open   .monitoring-es-6-2020.01.22            qTch0eFsQKK5XfGeTLX9TA   1   1     619190         2068    657.2mb        329.9mb
green  open   test3_202001291708                 IpbMSRHsQOyo4ZSLj8M7MQ   8   1    1903651            6      477mb        237.4mb
6

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:

image
image764×92 16.4 KB

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.

I updated your post but please format your next posts.

What makes you unhappy with the current results?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.