# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=486 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 487 --- ## [Filebeat K8s deployment - Duplicated Filestream ID](https://discuss.elastic.co/t/filebeat-k8s-deployment-duplicated-filestream-id/338008)
**Author:** [@msanft](https://discuss.elastic.co/u/msanft)\ **Replies:** 0\ **Last updated:** [July 10, 2023, 9:40am UTC](https://discuss.elastic.co/t/filebeat-k8s-deployment-duplicated-filestream-id/338008 "2023-07-10T09:40:27Z")
Hey all, I'm deploying Filebeat in a Kubernetes cluster as a daemonset. I see the following error message in the Filebeat Pod logs: { "log.level":"error", "@timestamp":"2023-07-10T09:18:38.720Z", "log.logger":… --- ## [Mapping an object field in order to show in the Kibana discover](https://discuss.elastic.co/t/mapping-an-object-field-in-order-to-show-in-the-kibana-discover/336843)
**Author:** [@JasonREC](https://discuss.elastic.co/u/JasonREC)\ **Replies:** 2\ **Last updated:** [July 10, 2023, 9:37am UTC](https://discuss.elastic.co/t/mapping-an-object-field-in-order-to-show-in-the-kibana-discover/336843 "2023-07-10T09:37:11Z")
HI, I have a field called "unqiue\_db\_tag" in my document. somehow it does not show on the table but only on JSON I looked at other discussion says it could be due to this field is not yet dont the mapping since I add … --- ## [How to take the backup of 3months data of elasticsearch?](https://discuss.elastic.co/t/how-to-take-the-backup-of-3months-data-of-elasticsearch/337653)
**Author:** [@merson](https://discuss.elastic.co/u/merson)\ **Replies:** 10\ **Last updated:** [July 10, 2023, 9:28am UTC](https://discuss.elastic.co/t/how-to-take-the-backup-of-3months-data-of-elasticsearch/337653 "2023-07-10T09:28:41Z")
In elasticsearch, it is runned three months and the size of elk is 40gb then I want to backup the elasticsearch datas. so what to do the backup of elasticsearch for 3months without using snapshot and restore. --- ## [Filebeat TCP input with SSL - Logs not received in correct format](https://discuss.elastic.co/t/filebeat-tcp-input-with-ssl-logs-not-received-in-correct-format/337994)
**Author:** [@wasimasif](https://discuss.elastic.co/u/wasimasif)\ **Replies:** 0\ **Last updated:** [July 10, 2023, 6:47am UTC](https://discuss.elastic.co/t/filebeat-tcp-input-with-ssl-logs-not-received-in-correct-format/337994 "2023-07-10T06:47:36Z")
I have created a TCP input but i have to secure communication using SSL. Following is my filebeat input configuration. This input starts and don't have any errors. Clients is also able to connect (verified via openssl ). … --- ## [Which Elasticsearch node should I send my query to?](https://discuss.elastic.co/t/which-elasticsearch-node-should-i-send-my-query-to/337937)
**Author:** [@ANUBHAV\_GUPTA](https://discuss.elastic.co/u/ANUBHAV_GUPTA)\ **Replies:** 3\ **Last updated:** [July 10, 2023, 6:45am UTC](https://discuss.elastic.co/t/which-elasticsearch-node-should-i-send-my-query-to/337937 "2023-07-10T06:45:51Z")
Let's say I have a cluster with total of 10 nodes where 4 are master eligible and rest are data nodes. So, how can I decide to which endpoint I should ping to have the best availability? --- ## [Getting Error "Exiting: /usr/share/filebeat/data/filebeat.lock: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data)"](https://discuss.elastic.co/t/getting-error-exiting-usr-share-filebeat-data-filebeat-lock-data-path-already-locked-by-another-beat-please-make-sure-that-multiple-beats-are-not-sharing-the-same-data-path-path-data/337991)
**Author:** [@Sharad\_Nautiyal](https://discuss.elastic.co/u/Sharad_Nautiyal)\ **Replies:** 0\ **Last updated:** [July 10, 2023, 6:12am UTC](https://discuss.elastic.co/t/getting-error-exiting-usr-share-filebeat-data-filebeat-lock-data-path-already-locked-by-another-beat-please-make-sure-that-multiple-beats-are-not-sharing-the-same-data-path-path-data/337991 "2023-07-10T06:12:39Z")
Hi Everyone, We have a central system to monitor logs for all the K8s clusters pods including specific label. There are specific pods for which we want to setup a different pre-processing system but when we are deployi… --- ## [Filebeat cloudwatch input not reading dynamic log groups/ log streams](https://discuss.elastic.co/t/filebeat-cloudwatch-input-not-reading-dynamic-log-groups-log-streams/337982)
**Author:** [@arungiyer](https://discuss.elastic.co/u/arungiyer)\ **Replies:** 0\ **Last updated:** [July 10, 2023, 3:45am UTC](https://discuss.elastic.co/t/filebeat-cloudwatch-input-not-reading-dynamic-log-groups-log-streams/337982 "2023-07-10T03:45:51Z")
I am using filebeat docker image (8.7.1) to run an ecs service that reads cloudwatch logs and send to an index. My cloudwatch log groups gets created dynamically so i am using "log\_group\_name\_prefix" to identify all log … --- ## [How to provide source Field in \_msearch query in ElasticSearch java client version 8](https://discuss.elastic.co/t/how-to-provide-source-field-in-msearch-query-in-elasticsearch-java-client-version-8/337924)
**Author:** [@maulik\_trapasiya](https://discuss.elastic.co/u/maulik_trapasiya)\ **Replies:** 2\ **Last updated:** [July 10, 2023, 3:19am UTC](https://discuss.elastic.co/t/how-to-provide-source-field-in-msearch-query-in-elasticsearch-java-client-version-8/337924 "2023-07-10T03:19:14Z")
My Elasticsearch's documents are of high size. My service is Java application and its using Elasticsearch java client version 8. Need to run \_msearch query on ES. MultisearchBody don't have field of \_source. in ES native… --- ## [Ingest Pipeline for parsing multiline fields giving provided Grok expressions do not match field value error error](https://discuss.elastic.co/t/ingest-pipeline-for-parsing-multiline-fields-giving-provided-grok-expressions-do-not-match-field-value-error-error/337699)
**Author:** [@SecretAsianMan](https://discuss.elastic.co/u/SecretAsianMan)\ **Replies:** 1\ **Last updated:** [July 9, 2023, 9:40pm UTC](https://discuss.elastic.co/t/ingest-pipeline-for-parsing-multiline-fields-giving-provided-grok-expressions-do-not-match-field-value-error-error/337699 "2023-07-09T21:40:24Z")
I am trying to parse a multiline log file as shown below. This is the processor that I have currently configured for the multiline log file. \[ { "grok": { "field": "message", "patterns": \[ "… --- ## [Reindexing an index which had document added by ingest pipeline](https://discuss.elastic.co/t/reindexing-an-index-which-had-document-added-by-ingest-pipeline/337951)
**Author:** [@Aditya\_Teltia](https://discuss.elastic.co/u/Aditya_Teltia)\ **Replies:** 12\ **Last updated:** [July 9, 2023, 8:02pm UTC](https://discuss.elastic.co/t/reindexing-an-index-which-had-document-added-by-ingest-pipeline/337951 "2023-07-09T20:02:52Z")
I have an index my-idx-09-2022. I made a ingest pipeline so that all the updates from now of my-idx-09-2022 will go to a new index i.e my-idx-new-09-2023. Python code: def create\_write\_redirect\_pipeline(source\_client, … --- ## [Gork regex](https://discuss.elastic.co/t/gork-regex/337623)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 8\ **Last updated:** [July 9, 2023, 6:29pm UTC](https://discuss.elastic.co/t/gork-regex/337623 "2023-07-09T18:29:28Z")
Hi I use this logstash gork: %{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:loglevel} %{DATA:id} \[%{DATA}\] %{DATA:jboss\_errors}(?=:|$) here is the log: 2023-06-30 09:09:55,941 ERROR CUS.InEP-AAAA-123194144 \[invocation\] WF… --- ## [How does Allocation of shards happens, when a node leaves cluster?](https://discuss.elastic.co/t/how-does-allocation-of-shards-happens-when-a-node-leaves-cluster/337960)
**Author:** [@Shashank\_Agrawal](https://discuss.elastic.co/u/Shashank_Agrawal)\ **Replies:** 3\ **Last updated:** [July 9, 2023, 5:09pm UTC](https://discuss.elastic.co/t/how-does-allocation-of-shards-happens-when-a-node-leaves-cluster/337960 "2023-07-09T17:09:33Z")
I want to know the exact procedure followed, for the allocation of shards on a node when the node leaves the cluster. Facts I know - 1.) ES waits for sometime before the reassigning the shards. 2.) For primary shards, … --- ## [Elasticsearch service not starting](https://discuss.elastic.co/t/elasticsearch-service-not-starting/337954)
**Author:** [@Jefferson\_Lourthusam](https://discuss.elastic.co/u/Jefferson_Lourthusam)\ **Replies:** 5\ **Last updated:** [July 9, 2023, 3:17pm UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/337954 "2023-07-09T15:17:49Z")
Elasticsearch service not starting , we can see below in Elasticsearch-STG logs low disk watermark \[85%\] exceeded on free: 37.4gb\[14.9%\], replicas will not be assigned to this node --- ## [Ingest pipeline routing documents to appropriate target index requires permissions on target index](https://discuss.elastic.co/t/ingest-pipeline-routing-documents-to-appropriate-target-index-requires-permissions-on-target-index/337923)
**Author:** [@Jurgen\_Wagner\_DVT](https://discuss.elastic.co/u/Jurgen_Wagner_DVT)\ **Replies:** 4\ **Last updated:** [July 8, 2023, 9:20pm UTC](https://discuss.elastic.co/t/ingest-pipeline-routing-documents-to-appropriate-target-index-requires-permissions-on-target-index/337923 "2023-07-08T21:20:46Z")
Suppose you don't trust data-feeding users to place documents into the right index, so you create a virtual index with an ingestion pipeline that determines the proper target index alias based on a few fields in each doc… --- ## [Missing authentication credential for REST request](https://discuss.elastic.co/t/missing-authentication-credential-for-rest-request/337339)
**Author:** [@kbfifi](https://discuss.elastic.co/u/kbfifi)\ **Replies:** 2\ **Last updated:** [July 8, 2023, 1:24pm UTC](https://discuss.elastic.co/t/missing-authentication-credential-for-rest-request/337339 "2023-07-08T13:24:06Z")
I'm trying to finish my single node setup with elasticsearch(ES) and kibana. I'm using docker-compose and ES version 7.17. I wanted to go without security however it is my understanding that adding adapters like Mongo DB… --- ## [Extract value from path in logstash](https://discuss.elastic.co/t/extract-value-from-path-in-logstash/337936)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 0\ **Last updated:** [July 8, 2023, 12:20pm UTC](https://discuss.elastic.co/t/extract-value-from-path-in-logstash/337936 "2023-07-08T12:20:18Z")
Hi need to extract value from path in logstash, here is my logpath: /data/app/20230707/\*/\* /data/app1/20230707/host1/\*.log /data/app2/20230707/host2/\*.log need to extract these field from path (FYI: hostname must be … --- ## [Filtering logic in elastic search output](https://discuss.elastic.co/t/filtering-logic-in-elastic-search-output/337922)
**Author:** [@Minika](https://discuss.elastic.co/u/Minika)\ **Replies:** 0\ **Last updated:** [July 8, 2023, 12:29am UTC](https://discuss.elastic.co/t/filtering-logic-in-elastic-search-output/337922 "2023-07-08T00:29:07Z")
Hi, I am trying to apply a filter logic in OCP Logstash pipeline. My pipeline receive logs from filebeat which contain a fields tag named logtype(that states the type of log) My motive is to use the logtype value and sen… --- ## [Mocking Search Results in new Java API](https://discuss.elastic.co/t/mocking-search-results-in-new-java-api/337012)
**Author:** [@silentfilm](https://discuss.elastic.co/u/silentfilm)\ **Replies:** 3\ **Last updated:** [July 7, 2023, 9:30pm UTC](https://discuss.elastic.co/t/mocking-search-results-in-new-java-api/337012 "2023-07-07T21:30:45Z")
Are there any examples of how to mock an Elasticsearch search result for the Java API for unit tests with Mockito? Do you mock the entire search result or individual hits? If I search the Internet for examples I only see… --- ## [Logstash using codec line is not working](https://discuss.elastic.co/t/logstash-using-codec-line-is-not-working/337816)
**Author:** [@cressprm](https://discuss.elastic.co/u/cressprm)\ **Replies:** 1\ **Last updated:** [July 7, 2023, 9:29pm UTC](https://discuss.elastic.co/t/logstash-using-codec-line-is-not-working/337816 "2023-07-07T21:29:17Z")
I am new to ELK and having trouble configuring a simple Logstash pipeline. Despite enabling debug logging(--log.level=debug), I can only find a message that says 'Received line' in the logs, and nothing else. Not sure wh… --- ## [Elasticsearch.service: Main process exited, code=killed, status=9/KILL](https://discuss.elastic.co/t/elasticsearch-service-main-process-exited-code-killed-status-9-kill/337796)
**Author:** [@bbkunbi](https://discuss.elastic.co/u/bbkunbi)\ **Replies:** 3\ **Last updated:** [July 7, 2023, 9:23pm UTC](https://discuss.elastic.co/t/elasticsearch-service-main-process-exited-code-killed-status-9-kill/337796 "2023-07-07T21:23:04Z")
Errror: elasticsearch.service: Main process exited, code=killed, status=9/KILL ul 06 17:32:05 linux systemd\[1\]: elasticsearch.service: Main process exited, code=killed, status=9/KILL Jul 06 17:32:05 linux systemd\[1\]: e… --- ## [Add new field to index based on maths calculation from other fields in the same index](https://discuss.elastic.co/t/add-new-field-to-index-based-on-maths-calculation-from-other-fields-in-the-same-index/337571)
**Author:** [@patcan](https://discuss.elastic.co/u/patcan)\ **Replies:** 12\ **Last updated:** [July 7, 2023, 9:15pm UTC](https://discuss.elastic.co/t/add-new-field-to-index-based-on-maths-calculation-from-other-fields-in-the-same-index/337571 "2023-07-07T21:15:07Z")
Hi, I use elastic-agent on EKS with kubernetes integration. One of the field such as kubernetes.volume.fs.used.pct in the index provides incorrect values I was able to get the correct value using the following formula… --- ## [Filter by Date in URL](https://discuss.elastic.co/t/filter-by-date-in-url/337083)
**Author:** [@Lehmer](https://discuss.elastic.co/u/Lehmer)\ **Replies:** 1\ **Last updated:** [July 7, 2023, 7:40pm UTC](https://discuss.elastic.co/t/filter-by-date-in-url/337083 "2023-07-07T19:40:13Z")
Hi, I need to access a kibana web filtering by date, but I need to put the filter in the URL. I know how to filter Namespaces and Jobs with the URL using queries: https:// kibana-host/s/desa/app/dashboards#/view/9908… --- ## [Canvas : Grouping of Elements](https://discuss.elastic.co/t/canvas-grouping-of-elements/337881)
**Author:** [@ksaimohan2k](https://discuss.elastic.co/u/ksaimohan2k)\ **Replies:** 2\ **Last updated:** [July 7, 2023, 6:26pm UTC](https://discuss.elastic.co/t/canvas-grouping-of-elements/337881 "2023-07-07T18:26:36Z")
Is it possible to group different elements in Canvas? Just attach them. --- ## [Building Custom Elastic Synthetics Dashboards](https://discuss.elastic.co/t/building-custom-elastic-synthetics-dashboards/337892)
**Author:** [@DougR](https://discuss.elastic.co/u/DougR)\ **Replies:** 2\ **Last updated:** [July 7, 2023, 6:00pm UTC](https://discuss.elastic.co/t/building-custom-elastic-synthetics-dashboards/337892 "2023-07-07T18:00:04Z")
I'm using Elastic Cloud, v8.8.2. I'm receiving a request to create an executive dashboard based on Elastic Uptime and Elastic Synthetics data. This dashboard would group related applications into an easy-to view format,… --- ## [Codec Avro Plugin Forward Compatibility](https://discuss.elastic.co/t/codec-avro-plugin-forward-compatibility/337911)
**Author:** [@fine\_porcupine](https://discuss.elastic.co/u/fine_porcupine)\ **Replies:** 0\ **Last updated:** [July 7, 2023, 5:59pm UTC](https://discuss.elastic.co/t/codec-avro-plugin-forward-compatibility/337911 "2023-07-07T17:59:55Z")
I am processing serialized messages from SQS using Logstash. The messages have been serialized using a FORWARD TRANSITIVE schema. The schema may change in the future. To deserialize these messages, I'd like to use the Co… --- ## [Can't install plugin "usageCollection" - invalid url](https://discuss.elastic.co/t/cant-install-plugin-usagecollection-invalid-url/337830)
**Author:** [@lenny1](https://discuss.elastic.co/u/lenny1)\ **Replies:** 3\ **Last updated:** [July 7, 2023, 5:45pm UTC](https://discuss.elastic.co/t/cant-install-plugin-usagecollection-invalid-url/337830 "2023-07-07T17:45:41Z")
Hello, when I try to install a Kibana Plugin using kibana-plugin install usageCollection it results in error invalid url, as it tries to fetch the file from URL https://artifacts.elastic.co/downloads/kibana-plugins/usag… --- ## [How to disable "usageCollection" plugin in kibana.yml?](https://discuss.elastic.co/t/how-to-disable-usagecollection-plugin-in-kibana-yml/337838)
**Author:** [@lenny1](https://discuss.elastic.co/u/lenny1)\ **Replies:** 1\ **Last updated:** [July 7, 2023, 5:25pm UTC](https://discuss.elastic.co/t/how-to-disable-usagecollection-plugin-in-kibana-yml/337838 "2023-07-07T17:25:36Z")
Hello, how do you disable "usageCollection" plugin in kibana/conf/kibana.yml? Best regards, Martin --- ## [Passing Filters to Kibana Dashboard Rendered In iFrame](https://discuss.elastic.co/t/passing-filters-to-kibana-dashboard-rendered-in-iframe/337710)
**Author:** [@kevfar](https://discuss.elastic.co/u/kevfar)\ **Replies:** 1\ **Last updated:** [July 7, 2023, 5:18pm UTC](https://discuss.elastic.co/t/passing-filters-to-kibana-dashboard-rendered-in-iframe/337710 "2023-07-07T17:18:54Z")
Hello! I am working with a variety of Kibana dashboards in a React application that renders the user's selected dashboard inside an iFrame. When the dashboard page opens, context from the user is passed in through the da… --- ## [Enhanced table plugin-8.7.1 availability](https://discuss.elastic.co/t/enhanced-table-plugin-8-7-1-availability/335248)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 12\ **Last updated:** [July 7, 2023, 5:14pm UTC](https://discuss.elastic.co/t/enhanced-table-plugin-8-7-1-availability/335248 "2023-07-07T17:14:09Z")
Hello @fbaligand , Can you please provide any info about how long will it take to get latest version of enhanced data table version-8.7.1.Checked github Not available yet. If this is not available then ,will it be fine… --- ## [Getting 1 of 19 shards failed on kibana dashboard](https://discuss.elastic.co/t/getting-1-of-19-shards-failed-on-kibana-dashboard/337703)
**Author:** [@Nishant\_Chauhan](https://discuss.elastic.co/u/Nishant_Chauhan)\ **Replies:** 0\ **Last updated:** [July 5, 2023, 5:20pm UTC](https://discuss.elastic.co/t/getting-1-of-19-shards-failed-on-kibana-dashboard/337703 "2023-07-05T17:20:31Z")
Hi, I have installed elastic agent version 7.17, on fleet page it shows healthy. However when i go to its dashboard, it is showing below error on all dashboards. ## illegal\_argument\_exception at \`shard\` 0\`index\` metri… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=485) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=487)