# Latest **URL:** https://discuss.elastic.co/latest.md?page=545 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 546 --- ## [Aggregate - Output issues](https://discuss.elastic.co/t/aggregate-output-issues/342536)
**Author:** [@vymk](https://discuss.elastic.co/u/vymk)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 11:34am UTC](https://discuss.elastic.co/t/aggregate-output-issues/342536 "2023-09-11T11:34:44Z")
I asked for some aggregation code a while ago (Help with aggregation code) and now finally had the time to get back at this (and changed the output concept a bit). So I want to aggregate data from multiple documents with… --- ## [Only one record gets created in the Elastic search](https://discuss.elastic.co/t/only-one-record-gets-created-in-the-elastic-search/342603)
**Author:** [@almostepic](https://discuss.elastic.co/u/almostepic)\ **Replies:** 10\ **Last updated:** [September 11, 2023, 12:16pm UTC](https://discuss.elastic.co/t/only-one-record-gets-created-in-the-elastic-search/342603 "2023-09-11T12:16:26Z")
I am using Logstash and created a conf file which allows me to input data into Elasticsearch. The data is related to git statistics from azure as I am creating a dashboard which will help to see information such as tota… --- ## [Transform script via Create Watcher API](https://discuss.elastic.co/t/transform-script-via-create-watcher-api/342608)
**Author:** [@ddoroshenko](https://discuss.elastic.co/u/ddoroshenko)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 12:07pm UTC](https://discuss.elastic.co/t/transform-script-via-create-watcher-api/342608 "2023-09-11T12:07:39Z")
Hi! I'm trying to put new watcher via Create Watcher API curl -X PUT "localhost:9200/\_watcher/watch/my-watch?pretty" -H 'Content-Type: application/json' -d' { ... "actions" { "problem": { "transform": { … --- ## [The primary shard is unassigned](https://discuss.elastic.co/t/the-primary-shard-is-unassigned/342710)
**Author:** [@zytine](https://discuss.elastic.co/u/zytine)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 11:59am UTC](https://discuss.elastic.co/t/the-primary-shard-is-unassigned/342710 "2023-09-11T11:59:32Z")
Hello, My es cluster health status turned to be red because of two unsigined shards.One of them is the primary shard and I got the following error when I executed "/ cluster/allocation/explain",the other is the replicat… --- ## [Configure Kibana With out Enrollment Token](https://discuss.elastic.co/t/configure-kibana-with-out-enrollment-token/342587)
**Author:** [@Debasis\_Mallick](https://discuss.elastic.co/u/Debasis_Mallick)\ **Replies:** 6\ **Last updated:** [September 11, 2023, 11:52am UTC](https://discuss.elastic.co/t/configure-kibana-with-out-enrollment-token/342587 "2023-09-11T11:52:46Z")
Hi Team, Could you please help me to configure Kibana with elastic cluster without an enrollment token? Due to some constraint, I had removed the security settings from elasticsearch.yml file due to which I am not able… --- ## [How to enable email connector in kibana rules?](https://discuss.elastic.co/t/how-to-enable-email-connector-in-kibana-rules/342736)
**Author:** [@jaimika\_kosambia](https://discuss.elastic.co/u/jaimika_kosambia)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 11:49am UTC](https://discuss.elastic.co/t/how-to-enable-email-connector-in-kibana-rules/342736 "2023-09-11T11:49:15Z")
How to enable email connector in kibana rules? --- ## [Elasticsearch system indices](https://discuss.elastic.co/t/elasticsearch-system-indices/342737)
**Author:** [@Swapnadeep\_Mondal](https://discuss.elastic.co/u/Swapnadeep_Mondal)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 11:45am UTC](https://discuss.elastic.co/t/elasticsearch-system-indices/342737 "2023-09-11T11:45:11Z")
Hi team, we are using self self-managed Elasticsearch cluster. And we are using self-monitoring monitoring. So Elasticsearch creates monitoring logs named ".monitoring-es-7-" automatically and these indices are not ass… --- ## [Kibana Table Chart Viz to show "-" for unique count of value on weekends](https://discuss.elastic.co/t/kibana-table-chart-viz-to-show-for-unique-count-of-value-on-weekends/342734)
**Author:** [@ArpithaS](https://discuss.elastic.co/u/ArpithaS)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 10:46am UTC](https://discuss.elastic.co/t/kibana-table-chart-viz-to-show-for-unique-count-of-value-on-weekends/342734 "2023-09-11T10:46:37Z")
Hi Everyone, I am trying to create a table using lens viz in Kibana. I need the unique count of a field\_name to be displayed per day (using date histogram with interval: per day) . I have added a query to exclude the w… --- ## [How to whitelist a hunreds nested field](https://discuss.elastic.co/t/how-to-whitelist-a-hunreds-nested-field/342563)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 10:36am UTC](https://discuss.elastic.co/t/how-to-whitelist-a-hunreds-nested-field/342563 "2023-09-11T10:36:04Z")
I plan to whitelist around 600 fields from 3000 fields in my index pattern but how do I do it? if I use mutate rename and then use mutate remove to delete the rest of it. it will put so much work into it. I wonder if th… --- ## [Merge two indices created by two different Logstash conf file](https://discuss.elastic.co/t/merge-two-indices-created-by-two-different-logstash-conf-file/342705)
**Author:** [@Priyaansh\_Dwivedi](https://discuss.elastic.co/u/Priyaansh_Dwivedi)\ **Replies:** 2\ **Last updated:** [September 11, 2023, 9:46am UTC](https://discuss.elastic.co/t/merge-two-indices-created-by-two-different-logstash-conf-file/342705 "2023-09-11T09:46:07Z")
I have two logs from Jitsi Meet application: jicofo.log and jvb.log. These logs are transferred from Filebeat to Logstash. I have separate Logstash configuration files for both incoming logs, and they are stored in separ… --- ## [Hi, I am trying to deploy the Elser (built-in) model, and all the configurations seems to be fine. But the deployment has started and it takes forever to complete the deployment](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350)
**Author:** [@Manasa4](https://discuss.elastic.co/u/Manasa4)\ **Replies:** 9\ **Last updated:** [September 11, 2023, 9:16am UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350 "2023-09-11T09:16:43Z")
In my case I have run the deployment at 11:27AM EST on 30th of August , but till 11:30 AM EST 31st of August, it still seems to be running. Also, tried increasing the RAM and space for Elastic search and also for the M… --- ## [How to get matched documents from ElasticSearch for a nested array fields matching specified values](https://discuss.elastic.co/t/how-to-get-matched-documents-from-elasticsearch-for-a-nested-array-fields-matching-specified-values/342724)
**Author:** [@Nid](https://discuss.elastic.co/u/Nid)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 9:12am UTC](https://discuss.elastic.co/t/how-to-get-matched-documents-from-elasticsearch-for-a-nested-array-fields-matching-specified-values/342724 "2023-09-11T09:12:04Z")
I am using elasticsearch-8.7.0. . I am a beginner, stuck on one thing. Please help. I have defined a mapping as below: mappings = { "properties": { "change\_id": {"type": "text", "analyzer": "english"}, "changes": { … --- ## [Logstash input json splitted by newline “\\n”](https://discuss.elastic.co/t/logstash-input-json-splitted-by-newline-n/342719)
**Author:** [@tomaxp13](https://discuss.elastic.co/u/tomaxp13)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 8:33am UTC](https://discuss.elastic.co/t/logstash-input-json-splitted-by-newline-n/342719 "2023-09-11T08:33:45Z")
I send the bulk to logstash where the jsons are separated by a newline. it looks like that: {"tenantId":"abcdefg","userSessionId":"zxxxxxxxxxxxxxx","startTime":1234,"endTime":12345,"duration":111,"internalUserId":"1234… --- ## [Logstash Docker to write Logstash internal logs /usr/share/logstash/logs](https://discuss.elastic.co/t/logstash-docker-to-write-logstash-internal-logs-usr-share-logstash-logs/342715)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 7:19am UTC](https://discuss.elastic.co/t/logstash-docker-to-write-logstash-internal-logs-usr-share-logstash-logs/342715 "2023-09-11T07:19:17Z")
Currently docker logs to stdout. I need to write to /usr/share/logstash/logs. cat logstash.yml path.logs: /usr/share/logstash/logs logstash.yml is in /usr/share/logstash/config. I trigger logstash from command line … --- ## [CompressingStoredFieldsReader instances take up a lot of memory](https://discuss.elastic.co/t/compressingstoredfieldsreader-instances-take-up-a-lot-of-memory/342717)
**Author:** [@emmning](https://discuss.elastic.co/u/emmning)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 8:17am UTC](https://discuss.elastic.co/t/compressingstoredfieldsreader-instances-take-up-a-lot-of-memory/342717 "2023-09-11T08:17:16Z")
Hello folks Our clients encountered errors below recently when performing index and query requests: TransportError(429, u'circuit\_breaking\_exception', {u'status': 429, u'error': {u'bytes\_wanted': 32385970160, u'durabil… --- ## [Merge two indexes to one](https://discuss.elastic.co/t/merge-two-indexes-to-one/342585)
**Author:** [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\ **Replies:** 4\ **Last updated:** [September 11, 2023, 6:57am UTC](https://discuss.elastic.co/t/merge-two-indexes-to-one/342585 "2023-09-11T06:57:45Z")
Hi i would like to merge two indexes to one index, with same fields except for one field. i tried the reindex api but it is not working with regular expressions in the indexname. my index has timestamp attached to it. s… --- ## [Logstash Parallelism (pipeline workers) does not work with Persistent Queue](https://discuss.elastic.co/t/logstash-parallelism-pipeline-workers-does-not-work-with-persistent-queue/342673)
**Author:** [@zalseryani](https://discuss.elastic.co/u/zalseryani)\ **Replies:** 2\ **Last updated:** [September 11, 2023, 6:44am UTC](https://discuss.elastic.co/t/logstash-parallelism-pipeline-workers-does-not-work-with-persistent-queue/342673 "2023-09-11T06:44:15Z")
Issue We found that when setting pipeline.workers to 4 and having 4 CPU, filters and output were not running in parallel, while disabling the persistence queue and relying on memory , the parallel workers were working … --- ## [Logstash failed to parse field \[host\] of type \[text\] in document](https://discuss.elastic.co/t/logstash-failed-to-parse-field-host-of-type-text-in-document/342697)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 6:38am UTC](https://discuss.elastic.co/t/logstash-failed-to-parse-field-host-of-type-text-in-document/342697 "2023-09-11T06:38:19Z")
\[2023-09-10T19:02:57,621\]\[WARN \]\[logstash.outputs.amazonelasticsearch\]\[main\]\[58792cc6d6e46359a72de39af72a6b76e760ccb0beb773f15a78ec2ef0b24671\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_… --- ## ["Elasticsearch" integration not displaying data in \[Elasticsearch\] Ingest Pipelines dashboard](https://discuss.elastic.co/t/elasticsearch-integration-not-displaying-data-in-elasticsearch-ingest-pipelines-dashboard/342614)
**Author:** [@Mark\_Duncan](https://discuss.elastic.co/u/Mark_Duncan)\ **Replies:** 3\ **Last updated:** [September 11, 2023, 6:24am UTC](https://discuss.elastic.co/t/elasticsearch-integration-not-displaying-data-in-elasticsearch-ingest-pipelines-dashboard/342614 "2023-09-11T06:24:37Z")
The \[Elasticsearch\] Ingest Pipelines dashboard has very limited visualisations populated with data. Please see the screenshot. Is it that the dashboards aren't setup correctly or is there some other issue? Thanks in ad… --- ## [Combining two fields from tow different documents within the same index based on conditions](https://discuss.elastic.co/t/combining-two-fields-from-tow-different-documents-within-the-same-index-based-on-conditions/342708)
**Author:** [@DivyaDileep](https://discuss.elastic.co/u/DivyaDileep)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 6:12am UTC](https://discuss.elastic.co/t/combining-two-fields-from-tow-different-documents-within-the-same-index-based-on-conditions/342708 "2023-09-11T06:12:56Z")
I am having data as below in one of the index @timestamp instance\_name dskIndex dskPercent dskPath Sep 8, 2023 @ 21:45:27.332 ssc-b 3 - … --- ## [Metricbeat Azure billing metricset](https://discuss.elastic.co/t/metricbeat-azure-billing-metricset/342037)
**Author:** [@Lalita\_Kumari](https://discuss.elastic.co/u/Lalita_Kumari)\ **Replies:** 2\ **Last updated:** [September 11, 2023, 5:43am UTC](https://discuss.elastic.co/t/metricbeat-azure-billing-metricset/342037 "2023-09-11T05:43:53Z")
I have Metricbeat installed in my system and have enabled Azure module and right now fetching data subscription wise. But i want to fetch the data Tag wise, more specifically under Subscriptions we have resources and und… --- ## [Issue with Filebeat Kubernetes Configuration for Ingress Logs](https://discuss.elastic.co/t/issue-with-filebeat-kubernetes-configuration-for-ingress-logs/342397)
**Author:** [@Ankur\_Mahajan](https://discuss.elastic.co/u/Ankur_Mahajan)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 5:38am UTC](https://discuss.elastic.co/t/issue-with-filebeat-kubernetes-configuration-for-ingress-logs/342397 "2023-09-11T05:38:43Z")
I'm attempting to use Filebeat on Kubernetes to ship my ingress logs and take advantage of the Nginx module dashboard. I've followed the configurations provided in the documentation, but I'm encountering an unusual issue… --- ## ["Error" Icon displayed in random times when opening kibana dashboard](https://discuss.elastic.co/t/error-icon-displayed-in-random-times-when-opening-kibana-dashboard/342646)
**Author:** [@mostafaelsayed](https://discuss.elastic.co/u/mostafaelsayed)\ **Replies:** 3\ **Last updated:** [September 11, 2023, 4:36am UTC](https://discuss.elastic.co/t/error-icon-displayed-in-random-times-when-opening-kibana-dashboard/342646 "2023-09-11T04:36:36Z")
Hello, I have a couple of dashboards in kibana, where each one can have more than 10 views (mostly data tables and charts). Most of the time, I encounter this Error Icon (shown in random views, no specifc ones) as show… --- ## [How to Read Real time data from url (.php real time file) and send data to elasticsearch? is there any method in logstash or filebeat to read real time data from url.?](https://discuss.elastic.co/t/how-to-read-real-time-data-from-url-php-real-time-file-and-send-data-to-elasticsearch-is-there-any-method-in-logstash-or-filebeat-to-read-real-time-data-from-url/342572)
**Author:** [@bbkunbi](https://discuss.elastic.co/u/bbkunbi)\ **Replies:** 2\ **Last updated:** [September 11, 2023, 4:31am UTC](https://discuss.elastic.co/t/how-to-read-real-time-data-from-url-php-real-time-file-and-send-data-to-elasticsearch-is-there-any-method-in-logstash-or-filebeat-to-read-real-time-data-from-url/342572 "2023-09-11T04:31:31Z")
TOPIC: Fetch data from url and send to Elasticsearch. In php file (Ex: https://temeprature.co/temp.log) my real time data is written. i want to read this latest data log and send it to Elasticsearch how to do this task? … --- ## [Grok pattern matching both the logs](https://discuss.elastic.co/t/grok-pattern-matching-both-the-logs/342586)
**Author:** [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\ **Replies:** 4\ **Last updated:** [September 11, 2023, 4:26am UTC](https://discuss.elastic.co/t/grok-pattern-matching-both-the-logs/342586 "2023-09-11T04:26:19Z")
Hi i have two types of logs, which differ just by one value at the end. can i get a single grok pattern to match both the files \[08/Sep/2023:13:28:50 +0530\] | 404 | 1 ms | 773 B | 127.0.0.1 | - | - | - | "GET /… --- ## [Kibana Graph Plugin - Error](https://discuss.elastic.co/t/kibana-graph-plugin-error/342683)
**Author:** [@drggfish](https://discuss.elastic.co/u/drggfish)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 4:06am UTC](https://discuss.elastic.co/t/kibana-graph-plugin-error/342683 "2023-09-11T04:06:53Z")
I am getting errors on the Graph Plugin when I try to use fields of type "double." There are no errors with fields of type "long." Does the Graph Plugin support doubles? Here is the Error: Error : unsupported\_operatio… --- ## [Help me - How can I configure firebeat to read log files from log folder](https://discuss.elastic.co/t/help-me-how-can-i-configure-firebeat-to-read-log-files-from-log-folder/342599)
**Author:** [@Huy\_Hoang\_Le](https://discuss.elastic.co/u/Huy_Hoang_Le)\ **Replies:** 3\ **Last updated:** [September 11, 2023, 3:49am UTC](https://discuss.elastic.co/t/help-me-how-can-i-configure-firebeat-to-read-log-files-from-log-folder/342599 "2023-09-11T03:49:04Z")
I have an app that write log to .log file to a log folder that mounted from container to host. I'm very new to ELK overall so I want filebeat to read .log files generated in that log folder. Here is my filebeat config f… --- ## [Generate monthly report from elastic index](https://discuss.elastic.co/t/generate-monthly-report-from-elastic-index/342691)
**Author:** [@JasonREC](https://discuss.elastic.co/u/JasonREC)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 2:56am UTC](https://discuss.elastic.co/t/generate-monthly-report-from-elastic-index/342691 "2023-09-11T02:56:54Z")
Hi, I want to generate a monthly report from my elastic index Currently, I have an index which contains time series data from July to now(Sep) after September, I want to generate the summary report of July, Aug And Se… --- ## [Kql search bar not applying](https://discuss.elastic.co/t/kql-search-bar-not-applying/342687)
**Author:** [@Farah\_Bhr](https://discuss.elastic.co/u/Farah_Bhr)\ **Replies:** 0\ **Last updated:** [September 10, 2023, 10:13pm UTC](https://discuss.elastic.co/t/kql-search-bar-not-applying/342687 "2023-09-10T22:13:24Z")
Hello , I am developing a cutomized kibana plugin and I added the kql search bar on the top , but my problem is that it is not applying the search bar on the visualized hits Here is my try const doAsyncSearch = async … --- ## [How to Insert 50Million documents per 30sec in elasticsearch cluster?](https://discuss.elastic.co/t/how-to-insert-50million-documents-per-30sec-in-elasticsearch-cluster/342669)
**Author:** [@coldcoder8502](https://discuss.elastic.co/u/coldcoder8502)\ **Replies:** 6\ **Last updated:** [September 10, 2023, 3:52pm UTC](https://discuss.elastic.co/t/how-to-insert-50million-documents-per-30sec-in-elasticsearch-cluster/342669 "2023-09-10T15:52:11Z")
Hi all, Iam facing difficulty to insert 50million documents per 30 sec from source to Elasticsearch cluster. I have 7 sources so total = 350million documents per 30 sec. I have one machine with 500GB Ram, 500Tb storage … [Previous page](https://discuss.elastic.co/latest.md?page=544) [Next page](https://discuss.elastic.co/latest.md?page=546)