# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1697 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1698 --- ## [Curl Output is not complete](https://discuss.elastic.co/t/curl-output-is-not-complete/255048)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 6\ **Last updated:** [November 15, 2020, 5:33pm UTC](https://discuss.elastic.co/t/curl-output-is-not-complete/255048 "2020-11-15T17:33:36Z")
Hello Team I am using Elasticsearch Version 7.8.0. I am executing an query on Elasticsearch index. The output of CURL command is not complete, IT shows incomplete records only. I suppose there is some limit / default … --- ## [Kibana can't connect to elasticsearch in linux](https://discuss.elastic.co/t/kibana-cant-connect-to-elasticsearch-in-linux/255429)
**Author:** [@Ehsan\_sarshar](https://discuss.elastic.co/u/Ehsan_sarshar)\ **Replies:** 2\ **Last updated:** [November 15, 2020, 5:17pm UTC](https://discuss.elastic.co/t/kibana-cant-connect-to-elasticsearch-in-linux/255429 "2020-11-15T17:17:36Z")
I have a docker-compose file with a kibana.yml everything is working correctly in macos but in linux it throw this error "Unable to revive connection: http://es01:9200/" --- ## [Does Filebeat have to use 'beats\_system' or 'elastic' to connect to ElasticSearch?](https://discuss.elastic.co/t/does-filebeat-have-to-use-beats-system-or-elastic-to-connect-to-elasticsearch/255434)
**Author:** [@houmie](https://discuss.elastic.co/u/houmie)\ **Replies:** 1\ **Last updated:** [November 15, 2020, 3:14pm UTC](https://discuss.elastic.co/t/does-filebeat-have-to-use-beats-system-or-elastic-to-connect-to-elasticsearch/255434 "2020-11-15T15:14:19Z")
Sorry, which one is the best practice? Because when we run elasticsearch-setup-passwords auto it generates passwords for beats\_system and elastic for us. output.elasticsearch: hosts: \["localhost:9200"\] protocol: "ht… --- ## [Wrong doc\_count in term\_vectors?](https://discuss.elastic.co/t/wrong-doc-count-in-term-vectors/255440)
**Author:** [@flm](https://discuss.elastic.co/u/flm)\ **Replies:** 0\ **Last updated:** [November 15, 2020, 2:58pm UTC](https://discuss.elastic.co/t/wrong-doc-count-in-term-vectors/255440 "2020-11-15T14:58:15Z")
doc\_count is incremented when adding the same document several times. To reproduce execute twice: PUT /test\_index/\_doc/1 { "somefield": "somedata" } There is 1 document in the index but doc\_count=2: { "\_index"… --- ## [Rollup jobs: Error while inserting same job id, but can't view metadata](https://discuss.elastic.co/t/rollup-jobs-error-while-inserting-same-job-id-but-cant-view-metadata/255321)
**Author:** [@kelk](https://discuss.elastic.co/u/kelk)\ **Replies:** 2\ **Last updated:** [November 15, 2020, 9:27am UTC](https://discuss.elastic.co/t/rollup-jobs-error-while-inserting-same-job-id-but-cant-view-metadata/255321 "2020-11-15T09:27:31Z")
I was creating a job in 7.9.3 Elastic, but below is the error I'm getting. I had created before and deleted similar job. Any idea where to find the metadata and clear it off? {"error":{"root\_cause":\[{"type":"status\_exce… --- ## [Query data from C# program the result not equal to the data filter from kibana](https://discuss.elastic.co/t/query-data-from-c-program-the-result-not-equal-to-the-data-filter-from-kibana/255428)
**Author:** [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Replies:** 0\ **Last updated:** [November 15, 2020, 8:13am UTC](https://discuss.elastic.co/t/query-data-from-c-program-the-result-not-equal-to-the-data-filter-from-kibana/255428 "2020-11-15T08:13:51Z")
I am new to C#, and I want to write a program to query data from elasticsearch. First of all, I load kibana sample log and want to query from my C# program, and below is my C# program. I want to filter clientip is 20.10… --- ## [Question about log rotation and log order](https://discuss.elastic.co/t/question-about-log-rotation-and-log-order/255427)
**Author:** [@Guohao\_Yi](https://discuss.elastic.co/u/Guohao_Yi)\ **Replies:** 0\ **Last updated:** [November 15, 2020, 7:21am UTC](https://discuss.elastic.co/t/question-about-log-rotation-and-log-order/255427 "2020-11-15T07:21:13Z")
I have learned that filebeat can be configured properly to handle log rotation correctly. However, I am not sure if the following situation I am going to mention is possibile to configure or not. We want to collect logs… --- ## [How to remove duplicate fields within doc?](https://discuss.elastic.co/t/how-to-remove-duplicate-fields-within-doc/254316)
**Author:** [@Kyle\_Neal](https://discuss.elastic.co/u/Kyle_Neal)\ **Replies:** 1\ **Last updated:** [November 15, 2020, 7:08am UTC](https://discuss.elastic.co/t/how-to-remove-duplicate-fields-within-doc/254316 "2020-11-15T07:08:02Z")
Hi, Trying to migrate from 5.X to 6.X but have issues when reading ES data that was stored before the migration. The error I'm getting back is: {error,{400, \[{\<\<"error"\>\>, \[{\<\<"root\_cause"\>\>, … --- ## [Filter results in a visualization using JSON (filter for a "greater than" result)](https://discuss.elastic.co/t/filter-results-in-a-visualization-using-json-filter-for-a-greater-than-result/255269)
**Author:** [@forestgump](https://discuss.elastic.co/u/forestgump)\ **Replies:** 1\ **Last updated:** [November 15, 2020, 6:01am UTC](https://discuss.elastic.co/t/filter-results-in-a-visualization-using-json-filter-for-a-greater-than-result/255269 "2020-11-15T06:01:14Z")
Hi, I have a data table which counts a total of certain parameter in each server. I need to filter and show in this table where the "Total" is more than "10,000". Other servers which has less than 10,000 is not signific… --- ## [System has not been booted with systemd as init system (PID 1)](https://discuss.elastic.co/t/system-has-not-been-booted-with-systemd-as-init-system-pid-1/255422)
**Author:** [@Naba01](https://discuss.elastic.co/u/Naba01)\ **Replies:** 0\ **Last updated:** [November 15, 2020, 5:04am UTC](https://discuss.elastic.co/t/system-has-not-been-booted-with-systemd-as-init-system-pid-1/255422 "2020-11-15T05:04:03Z")
I have installed elasticsearch and kibana on my WSL.Able to start services using "sudo /etc/init.d/" command But unable to start services for logstash Below command is not working: root@Flamingo:~# sudo systemctl star… --- ## [Logstash error after installing ssl](https://discuss.elastic.co/t/logstash-error-after-installing-ssl/254807)
**Author:** [@Cosmin\_Ciobanu](https://discuss.elastic.co/u/Cosmin_Ciobanu)\ **Replies:** 4\ **Last updated:** [November 11, 2020, 12:52am UTC](https://discuss.elastic.co/t/logstash-error-after-installing-ssl/254807 "2020-11-11T00:52:09Z")
\[ERROR\] 2020-11-09 10:08:12.337 \[\[main\]-pipeline-manager\] javapipeline - Pipeline error {:pipeline\_id=\>"main", :exception=\>#\Elasticsearch document deduplication efficiency & optimization](https://discuss.elastic.co/t/logstash-elasticsearch-document-deduplication-efficiency-optimization/255336)
**Author:** [@petersedivec](https://discuss.elastic.co/u/petersedivec)\ **Replies:** 2\ **Last updated:** [November 14, 2020, 9:25pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-document-deduplication-efficiency-optimization/255336 "2020-11-14T21:25:23Z")
Quick Summary: Looking for feedback on our data deduplication approach using generated doc IDs for time-based event data vs leveraging ES for generating document IDs. Data Overview: We are working with an event dataset … --- ## [Dashboard kibana in to web page](https://discuss.elastic.co/t/dashboard-kibana-in-to-web-page/254749)
**Author:** [@grian](https://discuss.elastic.co/u/grian)\ **Replies:** 2\ **Last updated:** [November 14, 2020, 9:21pm UTC](https://discuss.elastic.co/t/dashboard-kibana-in-to-web-page/254749 "2020-11-14T21:21:35Z")
Hi guys, I added an iframe with a kibana dashboard in a web page created with google sites, but when I go to enter the login crediants in kibana I don't access the view as if I don't recognize them, why? --- ## [Cant get cluster health](https://discuss.elastic.co/t/cant-get-cluster-health/255405)
**Author:** [@KOTOXJle6](https://discuss.elastic.co/u/KOTOXJle6)\ **Replies:** 2\ **Last updated:** [November 14, 2020, 7:11pm UTC](https://discuss.elastic.co/t/cant-get-cluster-health/255405 "2020-11-14T19:11:38Z")
Hello everyone! A couple of days ago, the monitoring cluster suddenly broke down. It just stopped showing the latest data at some point. Message standard When you try to enable the built-in monitor nothing happens … --- ## [Node.role: \[data\_cold\]](https://discuss.elastic.co/t/node-role-data-cold/255410)
**Author:** [@mutt13y](https://discuss.elastic.co/u/mutt13y)\ **Replies:** 1\ **Last updated:** [November 14, 2020, 6:56pm UTC](https://discuss.elastic.co/t/node-role-data-cold/255410 "2020-11-14T18:56:17Z")
Hi According to I should set my cold nodes to data cold I have node.attr.class: cold node.roles: \["data\_cold"\] set in elasticsearch.yml Yet I get this on startup \[2020-11-14T12:24:27,891\]\[ERROR\]\[o.e.b.Bootstr… --- ## [Newbie: CentOS + Docker + Elastic Stack = Master not discovered yet](https://discuss.elastic.co/t/newbie-centos-docker-elastic-stack-master-not-discovered-yet/255240)
**Author:** [@Elitlogik](https://discuss.elastic.co/u/Elitlogik)\ **Replies:** 16\ **Last updated:** [November 14, 2020, 6:10pm UTC](https://discuss.elastic.co/t/newbie-centos-docker-elastic-stack-master-not-discovered-yet/255240 "2020-11-14T18:10:18Z")
I'm trying to start learning Elastic Stack, but I can't get it to even start. I have: Installed CentOS 8 Minimal Installed Docker, following this: https://docs.docker.com/engine/install/centos/ Managing non-root and … --- ## [How to get multiline JSON parsing configured?](https://discuss.elastic.co/t/how-to-get-multiline-json-parsing-configured/255409)
**Author:** [@houmie](https://discuss.elastic.co/u/houmie)\ **Replies:** 1\ **Last updated:** [November 14, 2020, 5:39pm UTC](https://discuss.elastic.co/t/how-to-get-multiline-json-parsing-configured/255409 "2020-11-14T17:39:45Z")
Hello, When I try to activate multiline JSON parsing directly I get this error message:: Exiting: Failed to start crawler: starting input failed: Error while initializing input: When using the JSON decoder and multili… --- ## [I want to convert my date format yyyymmdd into month name date and year like mmm dd yyy (Apr 30 2012)](https://discuss.elastic.co/t/i-want-to-convert-my-date-format-yyyymmdd-into-month-name-date-and-year-like-mmm-dd-yyy-apr-30-2012/255394)
**Author:** [@Lokesh\_Mayan](https://discuss.elastic.co/u/Lokesh_Mayan)\ **Replies:** 11\ **Last updated:** [November 14, 2020, 5:19pm UTC](https://discuss.elastic.co/t/i-want-to-convert-my-date-format-yyyymmdd-into-month-name-date-and-year-like-mmm-dd-yyy-apr-30-2012/255394 "2020-11-14T17:19:26Z")
Hi I am having a log like the below \<#\> 20200924 15:14:40.918 280018000 EV.INF \[ MVINB-LSL2.T1G1\_ATMTH\_SL main.main TSP1.T1G1\_ATMTH\_SL \] Process T1G1\_ATMTH\_SL starting in progress... in the … --- ## [How to remove all the Host&Cloud fields added to Filebeats/Elasticsearch index?](https://discuss.elastic.co/t/how-to-remove-all-the-host-cloud-fields-added-to-filebeats-elasticsearch-index/255399)
**Author:** [@houmie](https://discuss.elastic.co/u/houmie)\ **Replies:** 2\ **Last updated:** [November 14, 2020, 5:12pm UTC](https://discuss.elastic.co/t/how-to-remove-all-the-host-cloud-fields-added-to-filebeats-elasticsearch-index/255399 "2020-11-14T17:12:50Z")
Hello, I finally got Filebeats/Elasticsearch/Kibana working. However the stack seems to have injected a lot of cloud tags (e.g. cloud.account.id) and host tags (host.os.kernel) that has inflated the data and makes it h… --- ## [Elasticsearch cluster is crashing often](https://discuss.elastic.co/t/elasticsearch-cluster-is-crashing-often/254640)
**Author:** [@alx2020](https://discuss.elastic.co/u/alx2020)\ **Replies:** 6\ **Last updated:** [November 14, 2020, 3:20pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-is-crashing-often/254640 "2020-11-14T15:20:55Z")
We have several large clusters v1.7. Please don't ask why :frowning: - we are on the way to upgrade to v7 but need some hints to make it stable. Recently one of the clusters started crashing several times per day and we… --- ## [How to seperate message field filebeat](https://discuss.elastic.co/t/how-to-seperate-message-field-filebeat/255167)
**Author:** [@Hi\_u\_Nguy\_n\_Dang](https://discuss.elastic.co/u/Hi_u_Nguy_n_Dang)\ **Replies:** 5\ **Last updated:** [November 14, 2020, 1:00pm UTC](https://discuss.elastic.co/t/how-to-seperate-message-field-filebeat/255167 "2020-11-14T13:00:39Z")
Guys, how can I seperate message field with filebeat to more field ? It keep just giving me message field, I am trying to use grok filter but it does not work. Filebeat agent work in 1 machine meanwhile elastic server wo… --- ## [Elastic and Logstash pipeline for Customer history](https://discuss.elastic.co/t/elastic-and-logstash-pipeline-for-customer-history/255181)
**Author:** [@ratheeshkrishnan92](https://discuss.elastic.co/u/ratheeshkrishnan92)\ **Replies:** 2\ **Last updated:** [November 14, 2020, 9:00am UTC](https://discuss.elastic.co/t/elastic-and-logstash-pipeline-for-customer-history/255181 "2020-11-14T09:00:14Z")
Hello Team, I am using Elasticsearch version 6.2.4. I have below scenario and need a work around for this. Planning to have two indices with below properties. EmployeeId, EmployeeName, Designation. index 1 : Employee… --- ## [Create Alerting in Kibana using API](https://discuss.elastic.co/t/create-alerting-in-kibana-using-api/255221)
**Author:** [@Baptiste\_Orsoni](https://discuss.elastic.co/u/Baptiste_Orsoni)\ **Replies:** 5\ **Last updated:** [November 14, 2020, 2:53am UTC](https://discuss.elastic.co/t/create-alerting-in-kibana-using-api/255221 "2020-11-14T02:53:55Z")
Hello, I would like to know if it is possible to create alerts with the Elastic Search API. Let me explain; I have a script below that sends me the traffic of a client during the last 5 minutes: GET /app-sms-smpp-… --- ## [Does logstash-input-s3-sns-sqs support backup\_to\_bucket?](https://discuss.elastic.co/t/does-logstash-input-s3-sns-sqs-support-backup-to-bucket/255389)
**Author:** [@Ying-Yi\_Huang](https://discuss.elastic.co/u/Ying-Yi_Huang)\ **Replies:** 0\ **Last updated:** [November 14, 2020, 12:48am UTC](https://discuss.elastic.co/t/does-logstash-input-s3-sns-sqs-support-backup-to-bucket/255389 "2020-11-14T00:48:05Z")
Hello, I want to use the plugin to read from S3 and then backup the files completed to a backup S3 back. It doesn't seem to work. Here is my config: input { s3snssqs { region =\> "us-e… --- ## [CANVAS Vertical Bar Chart with more than one Y axis](https://discuss.elastic.co/t/canvas-vertical-bar-chart-with-more-than-one-y-axis/255380)
**Author:** [@jose-sonicwebdev.com](https://discuss.elastic.co/u/jose-sonicwebdev.com)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 10:17pm UTC](https://discuss.elastic.co/t/canvas-vertical-bar-chart-with-more-than-one-y-axis/255380 "2020-11-13T22:17:35Z")
Hi I was wondering if CANVAS allowed for more than one Y axis on Bar Charts. I want to use the same input for X Axis but Two Y axis columns . I need to add "Week Total" and "Supplemental Total" to the Y Axis filters … --- ## [Elasticsearch service won't start, nested elasticsearch node state exception](https://discuss.elastic.co/t/elasticsearch-service-wont-start-nested-elasticsearch-node-state-exception/255376)
**Author:** [@penguinairlines](https://discuss.elastic.co/u/penguinairlines)\ **Replies:** 3\ **Last updated:** [November 13, 2020, 10:01pm UTC](https://discuss.elastic.co/t/elasticsearch-service-wont-start-nested-elasticsearch-node-state-exception/255376 "2020-11-13T22:01:48Z")
Hello, I noticed one of my elasticsearch nodes is down and started digging into the issue. I found the service won't start and notice a couple of key errors: ElasticsearchException\[failed to bind service\]; nested: Corr… --- ## [When we're writing to the Elasticsearch cluster, Reads are 10000x slower. (Understanding slow term query)](https://discuss.elastic.co/t/when-were-writing-to-the-elasticsearch-cluster-reads-are-10000x-slower-understanding-slow-term-query/255022)
**Author:** [@rex-remind](https://discuss.elastic.co/u/rex-remind)\ **Replies:** 10\ **Last updated:** [November 13, 2020, 9:31pm UTC](https://discuss.elastic.co/t/when-were-writing-to-the-elasticsearch-cluster-reads-are-10000x-slower-understanding-slow-term-query/255022 "2020-11-13T21:31:46Z")
Hello, I have recently indexed 25M docs into 2 nodes (+1 master) of 5 shards x 3 Availability Zones. I'm running in AWS ES, large boxes with with plenty of EBS storage, resource usage is low, average doc size is \< 1KiB.… --- ## [ML Jobs](https://discuss.elastic.co/t/ml-jobs/255357)
**Author:** [@Elk\_huh](https://discuss.elastic.co/u/Elk_huh)\ **Replies:** 1\ **Last updated:** [November 13, 2020, 9:25pm UTC](https://discuss.elastic.co/t/ml-jobs/255357 "2020-11-13T21:25:38Z")
Is there a way to do ML jobs for a high count of a specific field ? like i want high count of event.code 4626 --- ## [Correct Sizing and compression on Elasticsearch](https://discuss.elastic.co/t/correct-sizing-and-compression-on-elasticsearch/255349)
**Author:** [@xziten](https://discuss.elastic.co/u/xziten)\ **Replies:** 4\ **Last updated:** [November 13, 2020, 8:54pm UTC](https://discuss.elastic.co/t/correct-sizing-and-compression-on-elasticsearch/255349 "2020-11-13T20:54:36Z")
Hi, I am trying to get some ratio about Elasticsearch cluster. I have this base : \*\*For 100 Gb of raw data / day :\*\* \* 110 Gb of data are indexed (~ +10% for datatype) \* 220 Gb with 1 replica \* 6600 Gb per month \* +… --- ## [Index Templates - Differences between \_template & \_index\_template](https://discuss.elastic.co/t/index-templates-differences-between-template-index-template/255367)
**Author:** [@Eduardo\_Iglesias](https://discuss.elastic.co/u/Eduardo_Iglesias)\ **Replies:** 2\ **Last updated:** [November 13, 2020, 8:09pm UTC](https://discuss.elastic.co/t/index-templates-differences-between-template-index-template/255367 "2020-11-13T20:09:24Z")
Hello everyone, Need help to identify the differences between the subject mentioned endpoints usage. \_index\_template endpoint POST \_index\_template/index-template-1 { "index\_patterns": \["logs-2019\*"\], "template": … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1696) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1698)