hello
I have eight elasticsearch data nodes (ver. 6.4.1) (8 shards, 2 replica)
they have following data
indexed disk size : 21.5GB
Document Count : 67.7m
Segment Count : 336
A large amount of data is bached and indexed at once
I know there is a way to adjust the segment size
please advise how it is good to measure and adjust for quick search, index
thanks
Welcome!
It's not clear here what is the problem you are facing. Could you describe what you have today which looks wrong to you and what you'd like it to be?
sorry
not good at english T T
I'm indexing using multiple batch process
i want to reduce the batch process time without affecting the search
(indexing rate avg : 45,000/s, indexing latency avg: 0.3ms)
i think reducing the segment size will speed up the batch process
Is there a way to tune?
What is the output of:
GET /
GET /_cat/nodes?v
GET /_cat/health?v
GET /_cat/indices?v
If some outputs are too big, please share them on gist.github.com and link them here.
here is the output
some names have changed
GET /
{
"name": "name",
"cluster_name": "name",
"cluster_uuid": "id",
"version": {
"number": "6.4.3",
"build_flavor": "default",
"build_type": "tar",
"build_hash": "",
"build_date": "",
"build_snapshot": false,
"lucene_version": "7.4.0",
"minimum_wire_compatibility_version": "5.6.0",
"minimum_index_compatibility_version": "5.0.0"
},
"tagline": "You Know, for Search"
}
GET /_cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
1.1.1.1 30 93 3 1.77 1.83 1.66 di - d6.xxx.com
1.1.1.2 30 93 6 2.42 2.47 2.44 di - d2.xxx.com
1.1.1.3 51 6 0 0.00 0.01 0.00 m - m3.xxx.com
1.1.1.4 14 87 4 2.66 2.90 2.70 di - d4.xxx.com
1.1.1.5 5 61 0 0.04 0.01 0.00 - - c1.xxx.com
1.1.1.6 71 93 3 2.21 2.32 2.08 di - d8.xxx.com
1.1.1.7 46 92 4 3.03 3.34 3.05 di - d3.xxx.com
1.1.1.8 21 91 9 2.08 1.82 1.63 di - d1.xxx.com
1.1.1.9 3 56 0 0.12 0.14 0.14 - - c2.xxx.com
1.1.1.10 7 57 0 0.03 0.10 0.06 m - m2.xxx.com
1.1.1.11 59 92 2 2.60 2.51 2.10 di - d5.xxx.com
1.1.1.12 24 93 1 1.38 2.03 2.05 di - d7.xxx.com
1.1.1.13 12 60 0 0.19 0.23 0.12 m * m1.xxx.com
GET /_cat/health?v
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1580285411 17:10:11 filter green 13 8 202 101 0 0 0 0 - 100.0%
GET /_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .monitoring-es-6-2020.01.25 bO5PdJy0S3e6vFmVHOAxnw 1 1 623106 1442 654.5mb 326.8mb
green open .monitoring-es-6-2020.01.29 xRAsBYlxQc2JtSk3NU3IWA 1 1 211303 994 285.9mb 153.2mb
green open test_202001291710 d6vl1KXzTUeZsD5CZcn0Eg 8 1 87798 0 27.4mb 13.5mb
green open .monitoring-kibana-6-2020.01.26 ZG0t53TJRXSPasxD5ILGbQ 1 1 8638 0 4.3mb 2.1mb
green open .monitoring-es-6-2020.01.24 XRwQufivQgWdSLzBXHierw 1 1 613250 2222 647.3mb 327.1mb
green open test-snapshot wWUi1pUCR52Zds-6_4GsMQ 4 1 35669872 16443279 7.2gb 3.2gb
green open .monitoring-kibana-6-2020.01.25 QBz2H-DTQDO5NBfZ3AiyYA 1 1 8639 0 4.1mb 2mb
green open test2_202001290600 8QqSd73IRo-eXzk-K_Q49g 8 1 216760922 0 27.1gb 13.4gb
green open test5_202001291427 4tXIOmjjQUCqe4JmU4lCtg 8 1 69447852 0 21.7gb 10.8gb
green open .monitoring-kibana-6-2020.01.24 YK20iA0TTyWoxA2n_tyQug 1 1 8639 0 4.2mb 2.1mb
green open .monitoring-es-6-2020.01.26 QevGtFv2QgOuDhmeeMaWMw 1 1 618372 2210 651.3mb 327.7mb
green open .monitoring-kibana-6-2020.01.23 9gcHaLfvTtCAK7-l1jA9pw 1 1 8639 0 4.2mb 2mb
green open .monitoring-es-6-2020.01.23 ZLqMPc4xQ0SGWL883TL3qg 1 1 620700 1608 642.4mb 325.2mb
green open .monitoring-kibana-6-2020.01.28 UN3r6PHOSxWLMRC8OF0xnw 1 1 8639 0 4.3mb 2.1mb
green open .monitoring-es-6-2020.01.28 pQGnFAYEQaSQPJr4OxmfUw 1 1 621271 1374 645.5mb 322.9mb
green open test5_202001291533 2oHJplhJRmC4vT3oqHWlpw 8 1 69439421 0 21.9gb 10.9gb
green open test2_202001291200 LLf026xOQHaOn3K2gnWCpg 8 1 216789310 0 27.1gb 13.4gb
green open .monitoring-kibana-6-2020.01.29 rhrhtGIeSW6v8_woMFg1Dg 1 1 2946 0 1.5mb 783.5kb
green open .monitoring-es-6-2020.01.27 YOOp1jkTRX6hlHsN-BdGXg 1 1 619541 630 639.2mb 319.4mb
green open test1_202001291636 SwYOh7TARqWD3JrONQWKUQ 8 1 43859694 0 17.4gb 9gb
green open .monitoring-kibana-6-2020.01.22 LOQiiG88TPO5j6waSFpuGg 1 1 8639 0 4.4mb 2.2mb
green open .monitoring-kibana-6-2020.01.27 JW0NrOzMRV-yKMl1QPTblg 1 1 8639 0 4.3mb 2.1mb
green open .kibana 1QWEu2QxTG2PCRmIAMq1rg 1 1 3 0 35.5kb 17.7kb
green open test4_202001291705 _vmWfIqNTqup2H5jDjEXoA 8 1 87775 0 27.4mb 13.6mb
green open test3_202001291703 sSZ5c_zBQTWlxHoLNb2uXA 8 1 1903663 79 475.3mb 236.2mb
green open .monitoring-es-6-2020.01.22 qTch0eFsQKK5XfGeTLX9TA 1 1 619190 2068 657.2mb 329.9mb
green open test3_202001291708 IpbMSRHsQOyo4ZSLj8M7MQ 8 1 1903651 6 477mb 237.4mb
Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.
Or use markdown style like:
```
CODE
```
This is the icon to use if you are not using markdown format:
There's a live preview panel for exactly this reasons.
Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
I updated your post but please format your next posts.
What makes you unhappy with the current results?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
