# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1718 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1719 --- ## [Index size seems massive to what the data is being sent](https://discuss.elastic.co/t/index-size-seems-massive-to-what-the-data-is-being-sent/126977)
**Author:** [@kmroz](https://discuss.elastic.co/u/kmroz)\ **Replies:** 4\ **Last updated:** [April 16, 2018, 2:57pm UTC](https://discuss.elastic.co/t/index-size-seems-massive-to-what-the-data-is-being-sent/126977 "2018-04-16T14:57:08Z")
Hello, I am sending metrics via collectd on a 10s interval rate. my daily index comes out to a size from 25-27gb. This seems a bit massive? The index settings are setup as follows "number\_of\_shards": "5", "number\_of\_re… --- ## [How to get aggregated data by a field from ES with two values accumulated](https://discuss.elastic.co/t/how-to-get-aggregated-data-by-a-field-from-es-with-two-values-accumulated/128192)
**Author:** [@evgenygrin](https://discuss.elastic.co/u/evgenygrin)\ **Replies:** 0\ **Last updated:** [April 16, 2018, 1:14pm UTC](https://discuss.elastic.co/t/how-to-get-aggregated-data-by-a-field-from-es-with-two-values-accumulated/128192 "2018-04-16T13:14:55Z")
For the last two days my team deals with solving an issue of querying the data from Elasticsearch DB (ES). Our purpose is to get aggregated data by a field from ES with two values accumulated. If I would translate it to… --- ## [Elastic Search max\_num\_segments in forcemerge](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472)
**Author:** [@raj.shah1593](https://discuss.elastic.co/u/raj.shah1593)\ **Replies:** 14\ **Last updated:** [April 16, 2018, 12:54pm UTC](https://discuss.elastic.co/t/elastic-search-max-num-segments-in-forcemerge/127472 "2018-04-16T12:54:45Z")
Hi, I am currently using ES v6.1.2. I have an index of size 1.3 tb consisting of 16 shards (8 primaries). The primary shard size amounts to around 700GB. I am planning to perform a forcemerge operation on the index. … --- ## [Index settings or templates changed back overnight](https://discuss.elastic.co/t/index-settings-or-templates-changed-back-overnight/127793)
**Author:** [@Constantin07](https://discuss.elastic.co/u/Constantin07)\ **Replies:** 2\ **Last updated:** [April 16, 2018, 12:42pm UTC](https://discuss.elastic.co/t/index-settings-or-templates-changed-back-overnight/127793 "2018-04-16T12:42:27Z")
Hi All, I've tried to change the index settings (templates too) for the below special (+X-Pack) indexes from 1 primary 0 replica to 1 primary 2 replicas but overnight the settings have been changed back. Let's say fo… --- ## [Query with two aggregations](https://discuss.elastic.co/t/query-with-two-aggregations/128144)
**Author:** [@KibanaUse](https://discuss.elastic.co/u/KibanaUse)\ **Replies:** 6\ **Last updated:** [April 16, 2018, 12:26pm UTC](https://discuss.elastic.co/t/query-with-two-aggregations/128144 "2018-04-16T12:26:33Z")
Hello, I am tying to write a query to first get the last state value per customer, and then counting those up. So I have logs of customers with states. Each time a state of a customer changes, I write this to a log fil… --- ## [Remove Elasticsearch Timer Daemon Threads](https://discuss.elastic.co/t/remove-elasticsearch-timer-daemon-threads/127998)
**Author:** [@neel\_patel](https://discuss.elastic.co/u/neel_patel)\ **Replies:** 6\ **Last updated:** [April 16, 2018, 12:14pm UTC](https://discuss.elastic.co/t/remove-elasticsearch-timer-daemon-threads/127998 "2018-04-16T12:14:36Z")
Hi While doing bulk request to elasticsearch ,we are observing to many daemon thread. Which are Elasticsearch Timer daemon thread. How we can stop or kill these and free the memory? For every bulk request it generati… --- ## [Elasticsearch can't index](https://discuss.elastic.co/t/elasticsearch-cant-index/128162)
**Author:** [@chrigui94](https://discuss.elastic.co/u/chrigui94)\ **Replies:** 4\ **Last updated:** [April 16, 2018, 11:18am UTC](https://discuss.elastic.co/t/elasticsearch-cant-index/128162 "2018-04-16T11:18:14Z")
ES was working great and suddenly I received this error: \[2018-04-16T10:49:07,818\]\[WARN \]\[o.e.h.n.Netty4HttpServerTransport\] \[SST-P1307000FW\] caught exception while handling client http traffic, closing connection \[id:… --- ## [Node crashes without logs randomly. StubRoutines::jbyte\_disjoint\_arraycopy](https://discuss.elastic.co/t/node-crashes-without-logs-randomly-stubroutines-jbyte-disjoint-arraycopy/127273)
**Author:** [@imehl](https://discuss.elastic.co/u/imehl)\ **Replies:** 3\ **Last updated:** [April 16, 2018, 10:50am UTC](https://discuss.elastic.co/t/node-crashes-without-logs-randomly-stubroutines-jbyte-disjoint-arraycopy/127273 "2018-04-16T10:50:32Z")
One node of my cluster randomly crashes without any log in /var/log/elasticsearch and I don't know how to solve/debug this problem :frowning: Here is the hs\_err\_pid.log https://pastebin.com/tKiGSUPh There is enough fr… --- ## [Add a 2nd and 3rd node to a elasticsearch cluster](https://discuss.elastic.co/t/add-a-2nd-and-3rd-node-to-a-elasticsearch-cluster/128163)
**Author:** [@dao](https://discuss.elastic.co/u/dao)\ **Replies:** 2\ **Last updated:** [April 16, 2018, 10:18am UTC](https://discuss.elastic.co/t/add-a-2nd-and-3rd-node-to-a-elasticsearch-cluster/128163 "2018-04-16T10:18:18Z")
Hello, I have set up a 1 node elasticsearch cluster. I have pushed my data and set up security (xpack) now, that I know it runs OK, I want to add 2 nodes to it to reach a 3 node cluster. How can I proceed? is there a b… --- ## [Monitor Elastic Search Re-Index Progress](https://discuss.elastic.co/t/monitor-elastic-search-re-index-progress/128014)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 3\ **Last updated:** [April 16, 2018, 9:27am UTC](https://discuss.elastic.co/t/monitor-elastic-search-re-index-progress/128014 "2018-04-16T09:27:00Z")
When re-indexing large indexes it can take some time. There is a Reindex API which helps monitor the progress of a re-index and I've written a PowerShell Script which uses this to display a progress bar. In the hopes tha… --- ## [How to layout Filebeats setup](https://discuss.elastic.co/t/how-to-layout-filebeats-setup/128150)
**Author:** [@astropanic](https://discuss.elastic.co/u/astropanic)\ **Replies:** 0\ **Last updated:** [April 16, 2018, 9:26am UTC](https://discuss.elastic.co/t/how-to-layout-filebeats-setup/128150 "2018-04-16T09:26:39Z")
I've read the documentation, but not understand what is the way the Filebeat is supposed to work with geo data, especially the NGINX module. I installed an ES instance, installed the ingest geo ip and ingest user agent … --- ## [elasticsearch.exceptions.TransportError: TransportError 503: Data too large](https://discuss.elastic.co/t/elasticsearch-exceptions-transporterror-transporterror-503-data-too-large/128116)
**Author:** [@Vaibhav\_Kumar](https://discuss.elastic.co/u/Vaibhav_Kumar)\ **Replies:** 6\ **Last updated:** [April 16, 2018, 9:12am UTC](https://discuss.elastic.co/t/elasticsearch-exceptions-transporterror-transporterror-503-data-too-large/128116 "2018-04-16T09:12:48Z")
I'm trying to get the response from ES hitting it from python code but it is showing the below error: elasticsearch.exceptions.TransportError: TransportError(503, u'search\_phase\_execution\_exception', u'\[request\] Data to… --- ## [\[SOLVED\] Filebeats not writing data](https://discuss.elastic.co/t/solved-filebeats-not-writing-data/128143)
**Author:** [@astropanic](https://discuss.elastic.co/u/astropanic)\ **Replies:** 1\ **Last updated:** [April 16, 2018, 9:10am UTC](https://discuss.elastic.co/t/solved-filebeats-not-writing-data/128143 "2018-04-16T09:10:04Z")
Hi, my Nginx Filebeat module stopped to write data to ES. ({"type"=\>"cluster\_block\_exception", "reason"=\>"blocked by: \[FORBIDDEN/12/index read-only / allow delete (api)\] It was solved by: PUT \_settings { "index": { … --- ## [Input Elasticshearch](https://discuss.elastic.co/t/input-elasticshearch/127442)
**Author:** [@Mee](https://discuss.elastic.co/u/Mee)\ **Replies:** 2\ **Last updated:** [April 16, 2018, 8:31am UTC](https://discuss.elastic.co/t/input-elasticshearch/127442 "2018-04-16T08:31:47Z")
Hello there! I would like to index pptx documents. I install Elasticsarch and Ingest Attachment plugin. I've stored my document in a directory on local. Now I don't know how to import my directory as an input of Elast… --- ## [When i use date\_histogram it use utc time](https://discuss.elastic.co/t/when-i-use-date-histogram-it-use-utc-time/128121)
**Author:** [@chenzilong444](https://discuss.elastic.co/u/chenzilong444)\ **Replies:** 3\ **Last updated:** [April 16, 2018, 8:16am UTC](https://discuss.elastic.co/t/when-i-use-date-histogram-it-use-utc-time/128121 "2018-04-16T08:16:52Z")
"date\_histogram" : { "field" : "@timestamp", "interval" : "day" }, "aggregations": { "salestime": { "buckets": \[ { "key\_as\_string": "2018-04-02T00:00:00.000Z"… --- ## [Implement concurrency while inserting data to ES using jdbc conection in logstash config file](https://discuss.elastic.co/t/implement-concurrency-while-inserting-data-to-es-using-jdbc-conection-in-logstash-config-file/128122)
**Author:** [@manjur](https://discuss.elastic.co/u/manjur)\ **Replies:** 0\ **Last updated:** [April 16, 2018, 8:00am UTC](https://discuss.elastic.co/t/implement-concurrency-while-inserting-data-to-es-using-jdbc-conection-in-logstash-config-file/128122 "2018-04-16T08:00:24Z")
2m Hi, I am using below config file for logstash. I have used schedule= 1 min as jdbc scheduler which keep on inserting data to ES in every 1 min. But when number of records are more i am not able to identify weather i… --- ## [Elasticsearch \_cluster/health shows as red](https://discuss.elastic.co/t/elasticsearch-cluster-health-shows-as-red/127756)
**Author:** [@harika.gudumasu](https://discuss.elastic.co/u/harika.gudumasu)\ **Replies:** 3\ **Last updated:** [April 16, 2018, 7:05am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-health-shows-as-red/127756 "2018-04-16T07:05:12Z")
Hello, I am new to elasticsearch and trying to start in one node. It says as below. \[2018-04-12T08:45:55,908\]\[INFO \]\[o.e.c.s.ClusterApplierService\] \[lonrs11800\] new\_master {lonrs11800}{YF-Mz0X1TcWrKsmEHwMrLQ}{a-BLHg\_qT… --- ## [Excessive garbage collection](https://discuss.elastic.co/t/excessive-garbage-collection/127527)
**Author:** [@davidkarlsen](https://discuss.elastic.co/u/davidkarlsen)\ **Replies:** 10\ **Last updated:** [April 16, 2018, 6:36am UTC](https://discuss.elastic.co/t/excessive-garbage-collection/127527 "2018-04-16T06:36:32Z")
I have a very strange situation around GC: x-pack will report around 50% heap use, and based on the graphs in x-pack it seems to do a full gc around every hour, still I am experiencing performance problems and seeing: 2… --- ## [Exclude "in-progress" documents in search](https://discuss.elastic.co/t/exclude-in-progress-documents-in-search/128094)
**Author:** [@mattkallo](https://discuss.elastic.co/u/mattkallo)\ **Replies:** 3\ **Last updated:** [April 16, 2018, 3:36am UTC](https://discuss.elastic.co/t/exclude-in-progress-documents-in-search/128094 "2018-04-16T03:36:53Z")
Hi, Is there a way to mark a document "in-process" or to set it as "non-visible" to the search? Our requirement is do some post-processing after the document is indexed, however we don't want those document to appear in… --- ## [Elasticsearch cluster stats api return different values on different node](https://discuss.elastic.co/t/elasticsearch-cluster-stats-api-return-different-values-on-different-node/127786)
**Author:** [@l0calh0st](https://discuss.elastic.co/u/l0calh0st)\ **Replies:** 5\ **Last updated:** [April 16, 2018, 2:56am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-stats-api-return-different-values-on-different-node/127786 "2018-04-16T02:56:32Z")
\[es\] on master \_cluster/stats: "fs" : { "total\_in\_bytes" : 15874832596992, "free\_in\_bytes" : 12578652061696, "available\_in\_bytes" : 12578652061696, "spins" : "true" }, node analyzer1(hot) \_cluster/stats: "fs" : … --- ## [Internode connection in sIngle primary shard with one replica per node?](https://discuss.elastic.co/t/internode-connection-in-single-primary-shard-with-one-replica-per-node/128095)
**Author:** [@Jason\_Woo](https://discuss.elastic.co/u/Jason_Woo)\ **Replies:** 1\ **Last updated:** [April 16, 2018, 1:52am UTC](https://discuss.elastic.co/t/internode-connection-in-single-primary-shard-with-one-replica-per-node/128095 "2018-04-16T01:52:23Z")
From reading the Definitive Guide, I learned that the coordinating node forwards a read request to a different shard copy (assuming multiple primary shards and nodes). https://www.elastic.co/guide/en/elasticsearch/guid… --- ## [Deployment strategy for synonyms / stopwords file](https://discuss.elastic.co/t/deployment-strategy-for-synonyms-stopwords-file/127402)
**Author:** [@Brett\_Anderson](https://discuss.elastic.co/u/Brett_Anderson)\ **Replies:** 1\ **Last updated:** [April 15, 2018, 10:10pm UTC](https://discuss.elastic.co/t/deployment-strategy-for-synonyms-stopwords-file/127402 "2018-04-15T22:10:44Z")
If you're using a synonym and / or stopwords file as part of your search analyzer definition my understanding is that you can update the analyzer by updating the synonym or stopwords file on every member of the cluster. … --- ## [ES 5.4 synonyms and shingles don't seem to work together](https://discuss.elastic.co/t/es-5-4-synonyms-and-shingles-dont-seem-to-work-together/127552)
**Author:** [@Brett\_Anderson](https://discuss.elastic.co/u/Brett_Anderson)\ **Replies:** 1\ **Last updated:** [April 15, 2018, 9:50pm UTC](https://discuss.elastic.co/t/es-5-4-synonyms-and-shingles-dont-seem-to-work-together/127552 "2018-04-15T21:50:31Z")
I've applied the following synonym filter to an analyzer just on the search side (i.e. applied at search time, not index time): "synonym":{ "type":"synonym", "expand":true, "synonyms": \[ "ubergroun… --- ## [Update Multiple Objects at same time](https://discuss.elastic.co/t/update-multiple-objects-at-same-time/128062)
**Author:** [@Amit\_khanduri](https://discuss.elastic.co/u/Amit_khanduri)\ **Replies:** 8\ **Last updated:** [April 15, 2018, 5:43pm UTC](https://discuss.elastic.co/t/update-multiple-objects-at-same-time/128062 "2018-04-15T17:43:32Z")
What is the best and optimize way to update multiple objects at same time in elasticSearch 6.2 ? I want to update the key : value pair of all the objects except one !! Thanks in Advance !! --- ## [Elastic start but not open port](https://discuss.elastic.co/t/elastic-start-but-not-open-port/128084)
**Author:** [@NickOver](https://discuss.elastic.co/u/NickOver)\ **Replies:** 4\ **Last updated:** [April 15, 2018, 6:17pm UTC](https://discuss.elastic.co/t/elastic-start-but-not-open-port/128084 "2018-04-15T18:17:42Z")
Hi! I instal elastic on raspbian. After instalation i run it by type: serivce elasticsearch start Output from service elasticsearch status ● elasticsearch.service - LSB: Starts elasticsearch Loaded: loaded (/etc/… --- ## [Tshark json into elasticsearch](https://discuss.elastic.co/t/tshark-json-into-elasticsearch/128004)
**Author:** [@nkaenzig](https://discuss.elastic.co/u/nkaenzig)\ **Replies:** 2\ **Last updated:** [April 15, 2018, 12:40pm UTC](https://discuss.elastic.co/t/tshark-json-into-elasticsearch/128004 "2018-04-15T12:40:14Z")
Dear community, I have some .pcap files that I want to index using elastic search. After reading the following article, I wanted to follow the same strategy, namely convert the .pcap to .json using tshark -T ek test.pc… --- ## [Understanding reasons for cluster going to yellow state](https://discuss.elastic.co/t/understanding-reasons-for-cluster-going-to-yellow-state/127817)
**Author:** [@bsarkar](https://discuss.elastic.co/u/bsarkar)\ **Replies:** 1\ **Last updated:** [April 15, 2018, 11:47am UTC](https://discuss.elastic.co/t/understanding-reasons-for-cluster-going-to-yellow-state/127817 "2018-04-15T11:47:02Z")
I am investigating why our production clusters go from GREEN to YELLOW state. Specifically, I am confused between two kinds of logs that are logged by the master node: \[es-m01-rm\] Cluster health status changed from \[GR… --- ## [Social graph distance based scoring using ES](https://discuss.elastic.co/t/social-graph-distance-based-scoring-using-es/128038)
**Author:** [@adambenayoun](https://discuss.elastic.co/u/adambenayoun)\ **Replies:** 3\ **Last updated:** [April 15, 2018, 11:34am UTC](https://discuss.elastic.co/t/social-graph-distance-based-scoring-using-es/128038 "2018-04-15T11:34:31Z")
Hi, We want to implement a personalized user search using ES. The idea is that we have a graph in which nodes represent users and edges represent trust, on top of that we want to let users search for other users such th… --- ## [Can I add new mapping for new type?](https://discuss.elastic.co/t/can-i-add-new-mapping-for-new-type/128067)
**Author:** [@adolfjap](https://discuss.elastic.co/u/adolfjap)\ **Replies:** 1\ **Last updated:** [April 15, 2018, 8:53am UTC](https://discuss.elastic.co/t/can-i-add-new-mapping-for-new-type/128067 "2018-04-15T08:53:46Z")
If i add new type, Can I add mapping for this type? --- ## [ES as primary database](https://discuss.elastic.co/t/es-as-primary-database/128064)
**Author:** [@Abhijit\_Das](https://discuss.elastic.co/u/Abhijit_Das)\ **Replies:** 1\ **Last updated:** [April 15, 2018, 7:53am UTC](https://discuss.elastic.co/t/es-as-primary-database/128064 "2018-04-15T07:53:02Z")
Hello guys , I have a application running currently with sql server as primary data store .app is primarily used as deal searching purpose . So I am planning when saving data ,I will save in sql server and using rive… [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1717) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1719)