# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1798 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1799 --- ## [I want to send json formated logs via filebeat to elasticsearch](https://discuss.elastic.co/t/i-want-to-send-json-formated-logs-via-filebeat-to-elasticsearch/249069)
**Author:** [@aakash41](https://discuss.elastic.co/u/aakash41)\ **Replies:** 1\ **Last updated:** [September 18, 2020, 9:43am UTC](https://discuss.elastic.co/t/i-want-to-send-json-formated-logs-via-filebeat-to-elasticsearch/249069 "2020-09-18T09:43:33Z")
Can someone please let me know, if below config is correct. The content of logs in JSON but extension is .log. type: log paths: /opt/Application.log json.keys\_under\_root: true json.add\_error\_key: true Below is t… --- ## [2-node cluster debian](https://discuss.elastic.co/t/2-node-cluster-debian/248834)
**Author:** [@zen.xen](https://discuss.elastic.co/u/zen.xen)\ **Replies:** 4\ **Last updated:** [September 18, 2020, 9:13am UTC](https://discuss.elastic.co/t/2-node-cluster-debian/248834 "2020-09-18T09:13:00Z")
Hello, I try to build 2-node cluster but it doesn't work. I have installed bitnami-elk-7.9.1-0-linux-debian-10-x86\_64-nami.ova , I made two nodes: debian-01: 192.168.11.96, and debian-02: 192.168.11.97 configuration f… --- ## [How does logstash read multiple gunzipped files](https://discuss.elastic.co/t/how-does-logstash-read-multiple-gunzipped-files/249074)
**Author:** [@Praneet\_Khandelwal](https://discuss.elastic.co/u/Praneet_Khandelwal)\ **Replies:** 0\ **Last updated:** [September 18, 2020, 9:07am UTC](https://discuss.elastic.co/t/how-does-logstash-read-multiple-gunzipped-files/249074 "2020-09-18T09:07:48Z")
I have set up a simple logstash pipeline with a single file input (mode= 'read'). I have given a regex pattern in the path attribute. I am trying to read a number of gunzipped files matching the regex in the path. The th… --- ## [Date Long Overflow 7.9](https://discuss.elastic.co/t/date-long-overflow-7-9/248488)
**Author:** [@Escalus](https://discuss.elastic.co/u/Escalus)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 9:00am UTC](https://discuss.elastic.co/t/date-long-overflow-7-9/248488 "2020-09-18T09:00:18Z")
Hello, I get a long overflow in my date fields if the time is pior to 1970. The epoch\_time is a negativ long value and throws an arithmetic\_exception on reindex { "index" : "xxx", "type" : "\_doc", "id" : "pub.10… --- ## [Giop in logstash file](https://discuss.elastic.co/t/giop-in-logstash-file/248971)
**Author:** [@hiba](https://discuss.elastic.co/u/hiba)\ **Replies:** 6\ **Last updated:** [September 18, 2020, 8:52am UTC](https://discuss.elastic.co/t/giop-in-logstash-file/248971 "2020-09-18T08:52:26Z")
I create a giop filed in my logstash config file (location). In my data source the latitude and the longitude fields can be an empty string. so whene i load data into elasticserach i lose all the documents wich have th… --- ## [Sharding hot vs warm Nodes](https://discuss.elastic.co/t/sharding-hot-vs-warm-nodes/245960)
**Author:** [@fvegaucr1](https://discuss.elastic.co/u/fvegaucr1)\ **Replies:** 7\ **Last updated:** [September 18, 2020, 8:22am UTC](https://discuss.elastic.co/t/sharding-hot-vs-warm-nodes/245960 "2020-09-18T08:22:43Z")
Hi, hope you all doing good, I will like to see if someone can help me with this. I have a cluster of 35 nodes running version 7.4.2 3 x of them are master nodes 2 x coordinators 8 x hot 2 x cold 20 x Warm We get … --- ## [Right way to make ingest pipeline work](https://discuss.elastic.co/t/right-way-to-make-ingest-pipeline-work/249055)
**Author:** [@fxx](https://discuss.elastic.co/u/fxx)\ **Replies:** 0\ **Last updated:** [September 18, 2020, 4:53am UTC](https://discuss.elastic.co/t/right-way-to-make-ingest-pipeline-work/249055 "2020-09-18T04:53:27Z")
create a ingest pipeline in kibana test with kibana.dev-tool.grok .... ok test with kibana.ingest.test-pipeline ... ok test with POST index/?pipeline=xxx ... ok call by filebeat.yml filebeat.yml output.elastics… --- ## [Filebeat getting after few hours of starting](https://discuss.elastic.co/t/filebeat-getting-after-few-hours-of-starting/248990)
**Author:** [@Shefali](https://discuss.elastic.co/u/Shefali)\ **Replies:** 4\ **Last updated:** [September 18, 2020, 8:05am UTC](https://discuss.elastic.co/t/filebeat-getting-after-few-hours-of-starting/248990 "2020-09-18T08:05:15Z")
Having this issue on production. We have installed filebeat using tar file. After starting, in few hours it's stopping without any error. On our other lower environment we have installed using rpm and never seen this is… --- ## [Logstash and JSON array split](https://discuss.elastic.co/t/logstash-and-json-array-split/247482)
**Author:** [@ranu\_shekhar](https://discuss.elastic.co/u/ranu_shekhar)\ **Replies:** 4\ **Last updated:** [September 18, 2020, 8:02am UTC](https://discuss.elastic.co/t/logstash-and-json-array-split/247482 "2020-09-18T08:02:29Z")
Hi, I have the following input in JSON file: \[{"id":"9906fce735f8282ef85ccd2e5d49cbd7d3aaef2c","short\_id":"9906fce7","created\_at":"2020-08-21T16:01:20.000+05:30","parent\_ids":"d4e1afec8e2069f514701697c7a6ffd4cda5fc74",… --- ## [High CPU usage on only 1 Data node](https://discuss.elastic.co/t/high-cpu-usage-on-only-1-data-node/248407)
**Author:** [@fgourichon](https://discuss.elastic.co/u/fgourichon)\ **Replies:** 6\ **Last updated:** [September 18, 2020, 7:54am UTC](https://discuss.elastic.co/t/high-cpu-usage-on-only-1-data-node/248407 "2020-09-18T07:54:26Z")
Hi there, I'm trying to understand why we sometimes get high CPU utilisation on only one Data node of our cluster. We mainly use 2 indexes (80GB for one, 500MB for the second) with 5 shards and one replica each which se… --- ## [Search performance loss in Elasticsearch 7](https://discuss.elastic.co/t/search-performance-loss-in-elasticsearch-7/248871)
**Author:** [@agutierrezrodriguez](https://discuss.elastic.co/u/agutierrezrodriguez)\ **Replies:** 8\ **Last updated:** [September 18, 2020, 7:33am UTC](https://discuss.elastic.co/t/search-performance-loss-in-elasticsearch-7/248871 "2020-09-18T07:33:08Z")
Hi! We are upgrading our Elasticsearch 5.6 clusters to Elasticsearch 7.6. We're detecting that the search performance is worst in Elasticsearch 7.6 than 5.6. We are testing empty clusters with the same installation pr… --- ## [Audit logging configuration via elasticsearch.yml](https://discuss.elastic.co/t/audit-logging-configuration-via-elasticsearch-yml/248357)
**Author:** [@itokai](https://discuss.elastic.co/u/itokai)\ **Replies:** 1\ **Last updated:** [September 18, 2020, 7:31am UTC](https://discuss.elastic.co/t/audit-logging-configuration-via-elasticsearch-yml/248357 "2020-09-18T07:31:08Z")
Hi, started to configure amount of logging recently and figured out how to do it in log4j2.properties, like below basic example for testing: ######## Server JSON ############################ appender.rolling.t… --- ## [High Search Rate spikes for AWS elasticsearch](https://discuss.elastic.co/t/high-search-rate-spikes-for-aws-elasticsearch/249023)
**Author:** [@mehak](https://discuss.elastic.co/u/mehak)\ **Replies:** 6\ **Last updated:** [September 18, 2020, 7:14am UTC](https://discuss.elastic.co/t/high-search-rate-spikes-for-aws-elasticsearch/249023 "2020-09-18T07:14:23Z")
Hi Team, We are seeing high SearchRate spike at a certain times in a day along with Index and Search latency. We are having a cluster of 6 nodes with 3 master ones and other 3 are data nodes with 200 GB EBS volume. Not … --- ## [Is there any painless console to use](https://discuss.elastic.co/t/is-there-any-painless-console-to-use/249043)
**Author:** [@dotnet85](https://discuss.elastic.co/u/dotnet85)\ **Replies:** 1\ **Last updated:** [September 18, 2020, 7:01am UTC](https://discuss.elastic.co/t/is-there-any-painless-console-to-use/249043 "2020-09-18T07:01:59Z")
I am using Kibana 7.2.0. and I cannot find Painless Playground in DevTools. I think it is undess X-Pack? Is there a way to activate it? or an online console to get familier with Painless? or standalone tool for it? Than… --- ## [Uptime monitoring of HTTPS URLs using Heartbeat](https://discuss.elastic.co/t/uptime-monitoring-of-https-urls-using-heartbeat/248931)
**Author:** [@ksaha](https://discuss.elastic.co/u/ksaha)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 5:51am UTC](https://discuss.elastic.co/t/uptime-monitoring-of-https-urls-using-heartbeat/248931 "2020-09-18T05:51:43Z")
Hi Team, Can we monitor uptime of HTTPS URLs using heartbeat ? And can heartbeat monitor uptime of URLs with DNS ? Any help will be appreciated. Thanks. --- ## [Question about Snapshot and Restore](https://discuss.elastic.co/t/question-about-snapshot-and-restore/249041)
**Author:** [@Divit\_Sharma](https://discuss.elastic.co/u/Divit_Sharma)\ **Replies:** 4\ **Last updated:** [September 18, 2020, 5:48am UTC](https://discuss.elastic.co/t/question-about-snapshot-and-restore/249041 "2020-09-18T05:48:31Z")
Suppose we have two cluster : A and B Data from A is getting synced to B every 12 hour. So incremental snapshots from A are getting restored on B. Suppose on a day, at first 12 hr, index1 was created with 10 docs. Now … --- ## [Elasticsearch: Aggregations request timezone field not working](https://discuss.elastic.co/t/elasticsearch-aggregations-request-timezone-field-not-working/249061)
**Author:** [@yunut](https://discuss.elastic.co/u/yunut)\ **Replies:** 0\ **Last updated:** [September 18, 2020, 5:46am UTC](https://discuss.elastic.co/t/elasticsearch-aggregations-request-timezone-field-not-working/249061 "2020-09-18T05:46:20Z")
Elasticsearch Version: 7.7.0v Rest client Vesion: 7.7.0v The query created by Postman and the request created by QueryBuilder are the same, but the TimeZone field is not applied to java requests. here is request targe… --- ## [How & When to process/convert data in ELK](https://discuss.elastic.co/t/how-when-to-process-convert-data-in-elk/248816)
**Author:** [@aspat](https://discuss.elastic.co/u/aspat)\ **Replies:** 4\ **Last updated:** [September 18, 2020, 5:22am UTC](https://discuss.elastic.co/t/how-when-to-process-convert-data-in-elk/248816 "2020-09-18T05:22:37Z")
Hello, I am planning to use ELK My current SQL table is like below Time, Field ID, Filed\_name, Field\_Sub\_ID, value, extra value, type To view this data we have provided excel add-in. User can provide basic filter to… --- ## [Daily/hourly aggregation using transform](https://discuss.elastic.co/t/daily-hourly-aggregation-using-transform/248677)
**Author:** [@liorg2](https://discuss.elastic.co/u/liorg2)\ **Replies:** 11\ **Last updated:** [September 18, 2020, 5:15am UTC](https://discuss.elastic.co/t/daily-hourly-aggregation-using-transform/248677 "2020-09-18T05:15:47Z")
Hello I want to have a daily aggregation index with daily avg, percentiles etc . I've created a daily transform with: (more complex than below) "pivot": { "group\_by": { "datetime": { "date\_histogram… --- ## [Filebeat nginx access logs not loading with required fields in kibana dashboard](https://discuss.elastic.co/t/filebeat-nginx-access-logs-not-loading-with-required-fields-in-kibana-dashboard/248944)
**Author:** [@pratap1991](https://discuss.elastic.co/u/pratap1991)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 5:12am UTC](https://discuss.elastic.co/t/filebeat-nginx-access-logs-not-loading-with-required-fields-in-kibana-dashboard/248944 "2020-09-18T05:12:44Z")
filebeat nginx access logs not loading with required fields in kibana dashboard i am using below logstash filter for to get access logs fields but i am not able get required fields ( access.message and other fields) #… --- ## [ELastic upgrade](https://discuss.elastic.co/t/elastic-upgrade/248768)
**Author:** [@chandra\_bera](https://discuss.elastic.co/u/chandra_bera)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 5:07am UTC](https://discuss.elastic.co/t/elastic-upgrade/248768 "2020-09-18T05:07:02Z")
Is there any video or documents to upgrade the Elastic stack 7.5 to 7.9 --- ## [How to change Kibana Notification popup CSS](https://discuss.elastic.co/t/how-to-change-kibana-notification-popup-css/248271)
**Author:** [@chandu5565](https://discuss.elastic.co/u/chandu5565)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 4:25am UTC](https://discuss.elastic.co/t/how-to-change-kibana-notification-popup-css/248271 "2020-09-18T04:25:02Z")
Hi, Objective : I have to change the notification popup from bottom to top. I have upgraded kibana from 7.5.1 to 7.9.0 Earlier we had made change in file /usr/share/kibana/node\_modules/@elastic/eui/dist/eui\_theme\_ligh… --- ## [Not able to start metricbeat](https://discuss.elastic.co/t/not-able-to-start-metricbeat/248963)
**Author:** [@Shriram\_Wasule](https://discuss.elastic.co/u/Shriram_Wasule)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 3:25am UTC](https://discuss.elastic.co/t/not-able-to-start-metricbeat/248963 "2020-09-18T03:25:27Z")
i am getting this error while i am trying to start metricbeat on one of my hosts write error: failed to rotate backups: failed to rotate backups: rename C:\\metricbeat\\logs\\metricbeat C:\\metricbeat\\logs\\metricbeat.1: Acc… --- ## [How to Round up the result to the nearest integer](https://discuss.elastic.co/t/how-to-round-up-the-result-to-the-nearest-integer/248897)
**Author:** [@Karrie\_Koo](https://discuss.elastic.co/u/Karrie_Koo)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 2:01am UTC](https://discuss.elastic.co/t/how-to-round-up-the-result-to-the-nearest-integer/248897 "2020-09-18T02:01:50Z")
For the log text, {{heapUsedPct.0}} \<= This field shows the result with many decimal places, I want to round up to the nearest integer, but how can I do it? { "trigger": { "schedule": { "interval": "15m" } }, "in… --- ## [My first project ... beginner question](https://discuss.elastic.co/t/my-first-project-beginner-question/249000)
**Author:** [@leostereo](https://discuss.elastic.co/u/leostereo)\ **Replies:** 3\ **Last updated:** [September 18, 2020, 1:00am UTC](https://discuss.elastic.co/t/my-first-project-beginner-question/249000 "2020-09-18T01:00:26Z")
Hi guys , here I am learning elk. I managed to: 1 Create an index. PUT /dhcp\_leases { "settings": { "number\_of\_shards": 1 }, "mappings": { "properties": { "router": { "type": "text" }, "curre… --- ## [Prerequisites for ELK and Filebeat](https://discuss.elastic.co/t/prerequisites-for-elk-and-filebeat/248946)
**Author:** [@sumitsahay](https://discuss.elastic.co/u/sumitsahay)\ **Replies:** 5\ **Last updated:** [September 18, 2020, 12:56am UTC](https://discuss.elastic.co/t/prerequisites-for-elk-and-filebeat/248946 "2020-09-18T00:56:10Z")
Hi Team, Just wanted to know the CPU and RAM Prerequisites for ELK and Filebeat, if we deploy this in Infra where we can captures log around 500 client node. Please suggest the configuration for this. Thanks --- ## [Can not create script field by using regex in kibana](https://discuss.elastic.co/t/can-not-create-script-field-by-using-regex-in-kibana/248967)
**Author:** [@le\_ba\_nam](https://discuss.elastic.co/u/le_ba_nam)\ **Replies:** 2\ **Last updated:** [September 18, 2020, 12:53am UTC](https://discuss.elastic.co/t/can-not-create-script-field-by-using-regex-in-kibana/248967 "2020-09-18T00:53:53Z")
I want to create new script field in Kibana. I want to using regex to cut some text from another field ('message') and fill to new script field. I follow instruction link below: ( Match a string and return that matc… --- ## [Visualize benchmark summary report in kibana](https://discuss.elastic.co/t/visualize-benchmark-summary-report-in-kibana/249015)
**Author:** [@sammi1](https://discuss.elastic.co/u/sammi1)\ **Replies:** 1\ **Last updated:** [September 17, 2020, 11:51pm UTC](https://discuss.elastic.co/t/visualize-benchmark-summary-report-in-kibana/249015 "2020-09-17T23:51:28Z")
Can we create a sample to visualize summary report in kibana? --- ## [Increasing index.max\_result\_window using CentOS CLI](https://discuss.elastic.co/t/increasing-index-max-result-window-using-centos-cli/246432)
**Author:** [@hishan42](https://discuss.elastic.co/u/hishan42)\ **Replies:** 11\ **Last updated:** [September 17, 2020, 11:28pm UTC](https://discuss.elastic.co/t/increasing-index-max-result-window-using-centos-cli/246432 "2020-09-17T23:28:13Z")
I am using graylog with mongodb and elasticsearch 6.8 on CentOS 7. Now I have received an error saying "Elasticsearch limits the search result to 10000 messages". How do I increase the index.max\_result\_window? I dont … --- ## [How to modify data table column values in Kibana](https://discuss.elastic.co/t/how-to-modify-data-table-column-values-in-kibana/248989)
**Author:** [@geekganesh](https://discuss.elastic.co/u/geekganesh)\ **Replies:** 2\ **Last updated:** [September 17, 2020, 11:26pm UTC](https://discuss.elastic.co/t/how-to-modify-data-table-column-values-in-kibana/248989 "2020-09-17T23:26:13Z")
In Elastic search we store events, I have built data table which aggregates based on event types. I have filter which checks for event.keyword : "job-completed". I am getting the count as 1 or 0 but i want to display as … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1797) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1799)