# Latest **URL:** https://discuss.elastic.co/latest.md?page=1044 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1045 --- ## [Automation of Data Analysis](https://discuss.elastic.co/t/automation-of-data-analysis/309879)
**Author:** [@elkstack2](https://discuss.elastic.co/u/elkstack2)\ **Replies:** 1\ **Last updated:** [July 18, 2022, 12:17pm UTC](https://discuss.elastic.co/t/automation-of-data-analysis/309879 "2022-07-18T12:17:46Z")
Hi all! I am interning at an organisation and have been asked to research automation in the ELK stack - I don't really know what I'm doing so any help would be appreciated! One of the increasingly large challenges my or… --- ## [Filebeat: Autodiscover with Nomad + Docker Metadata](https://discuss.elastic.co/t/filebeat-autodiscover-with-nomad-docker-metadata/309404)
**Author:** [@adsr](https://discuss.elastic.co/u/adsr)\ **Replies:** 2\ **Last updated:** [July 18, 2022, 12:10pm UTC](https://discuss.elastic.co/t/filebeat-autodiscover-with-nomad-docker-metadata/309404 "2022-07-18T12:10:26Z")
Hi everyone, I am currently struggling with adding the docker metadata to my events which are fetched via the filebeat autodiscover nomad provider. filebeat.yml tyfilebeat.autodiscover: providers: - type: nomad … --- ## [Elastic v8.3 Install Errors MacOS](https://discuss.elastic.co/t/elastic-v8-3-install-errors-macos/308690)
**Author:** [@Sameer\_Malik](https://discuss.elastic.co/u/Sameer_Malik)\ **Replies:** 3\ **Last updated:** [July 18, 2022, 11:59am UTC](https://discuss.elastic.co/t/elastic-v8-3-install-errors-macos/308690 "2022-07-18T11:59:24Z")
Elasticsearch Version 8.3.1 OS Version macOS 12.3.1 Problem Description I'm very new to Elasticsearch. I'm trying to install ES v8.3.1 on my Mac by following the instructions on the download website. However, once I ru… --- ## [Group by date with aggr and apply sub aggr with group by date in elastic search](https://discuss.elastic.co/t/group-by-date-with-aggr-and-apply-sub-aggr-with-group-by-date-in-elastic-search/309894)
**Author:** [@surendrakandira](https://discuss.elastic.co/u/surendrakandira)\ **Replies:** 0\ **Last updated:** [July 18, 2022, 11:49am UTC](https://discuss.elastic.co/t/group-by-date-with-aggr-and-apply-sub-aggr-with-group-by-date-in-elastic-search/309894 "2022-07-18T11:49:37Z")
How to implement below C# entity logic in Elasticsearch var queryable1 = queryable.GroupBy(g =\> g.Date).Select(x =\> new GroupChartDto() { DailyPnL = x.Sum(s =\> s.DailyPnL), Date = x.Key, AggregateSum = queryable.Whe… --- ## [In ES 7.8 parent breaker is tripping a lot and causing unallocation of shards](https://discuss.elastic.co/t/in-es-7-8-parent-breaker-is-tripping-a-lot-and-causing-unallocation-of-shards/309635)
**Author:** [@sreekanth](https://discuss.elastic.co/u/sreekanth)\ **Replies:** 2\ **Last updated:** [July 18, 2022, 10:58am UTC](https://discuss.elastic.co/t/in-es-7-8-parent-breaker-is-tripping-a-lot-and-causing-unallocation-of-shards/309635 "2022-07-18T10:58:08Z")
We have an 8 node cluster and our load (mainly bulk ingest) is pretty high. Earlier the same load was handled well by 6 nodes in ES6.8. Now after moving to 7.8, we see many replica shards get unallocated during load. al… --- ## [Overriding logstash access "remote\_host" with client IP - Spring Boot](https://discuss.elastic.co/t/overriding-logstash-access-remote-host-with-client-ip-spring-boot/309887)
**Author:** [@Wes1](https://discuss.elastic.co/u/Wes1)\ **Replies:** 0\ **Last updated:** [July 18, 2022, 10:57am UTC](https://discuss.elastic.co/t/overriding-logstash-access-remote-host-with-client-ip-spring-boot/309887 "2022-07-18T10:57:59Z")
Good day, Does anyone know how to override "remote\_host" field within logstash access. Currently we are logging the load balancer's IP. I would like to use the "X-Forwarded-For" header. Thank you. --- ## [Group by date with aggr and apply sub aggr with group by date in elastic search](https://discuss.elastic.co/t/group-by-date-with-aggr-and-apply-sub-aggr-with-group-by-date-in-elastic-search/309877)
**Author:** [@surendrakandira](https://discuss.elastic.co/u/surendrakandira)\ **Replies:** 0\ **Last updated:** [July 18, 2022, 10:23am UTC](https://discuss.elastic.co/t/group-by-date-with-aggr-and-apply-sub-aggr-with-group-by-date-in-elastic-search/309877 "2022-07-18T10:23:20Z")
I am trying to apply filter with aggregations. c# code , I want to apply this logic in elastic search queryable = queryable.OrderBy(o =\> o.Date).ToList(); var queryable1 = queryable.GroupBy(g =\> g… --- ## [Date fields not being returned in results](https://discuss.elastic.co/t/date-fields-not-being-returned-in-results/309416)
**Author:** [@Rav\_Panchalingam](https://discuss.elastic.co/u/Rav_Panchalingam)\ **Replies:** 2\ **Last updated:** [July 18, 2022, 9:32am UTC](https://discuss.elastic.co/t/date-fields-not-being-returned-in-results/309416 "2022-07-18T09:32:30Z")
Hello, we have Elastic App Search via Elastic Cloud. Version 8.3.2 When we index documents with a Date field, that field does not appear in results and we cannot filter using that field. Here is an example document ind… --- ## [Succeeded to take snapshot of closed Elasticsearch index with ignore\_unavailable=false](https://discuss.elastic.co/t/succeeded-to-take-snapshot-of-closed-elasticsearch-index-with-ignore-unavailable-false/309853)
**Author:** [@amirmono](https://discuss.elastic.co/u/amirmono)\ **Replies:** 2\ **Last updated:** [July 18, 2022, 9:19am UTC](https://discuss.elastic.co/t/succeeded-to-take-snapshot-of-closed-elasticsearch-index-with-ignore-unavailable-false/309853 "2022-07-18T09:19:48Z")
According to the Elasticsearch Create snapshot API documentation , when creating a manual snapshot of a closed index with ignore\_unavailable = false - the snapshot should fail: ignore\_unavailable (Optional, Boolean) If… --- ## [Connection problem at the second login](https://discuss.elastic.co/t/connection-problem-at-the-second-login/309851)
**Author:** [@tamir\_raviv](https://discuss.elastic.co/u/tamir_raviv)\ **Replies:** 3\ **Last updated:** [July 18, 2022, 8:51am UTC](https://discuss.elastic.co/t/connection-problem-at-the-second-login/309851 "2022-07-18T08:51:22Z")
Hey. Running elasticsearch locally. Using elasticsearch-8.3.2 and kibana-8.3.2. I'm able to run everything smoothly the first time. After some timeout, when i'm not running anything, Kibana asking me to login again when … --- ## [How do I extract partial fields from log.file.path](https://discuss.elastic.co/t/how-do-i-extract-partial-fields-from-log-file-path/309854)
**Author:** [@regyhuang](https://discuss.elastic.co/u/regyhuang)\ **Replies:** 0\ **Last updated:** [July 18, 2022, 7:36am UTC](https://discuss.elastic.co/t/how-do-i-extract-partial-fields-from-log-file-path/309854 "2022-07-18T07:36:11Z")
for example log.file.path:/data/logs/service-name/service-name.log I want to extract the service-name of log.file.path as a tags ,How to configure filebeat.yml --- ## [App search - synonyms](https://discuss.elastic.co/t/app-search-synonyms/308590)
**Author:** [@kanavsha](https://discuss.elastic.co/u/kanavsha)\ **Replies:** 2\ **Last updated:** [July 18, 2022, 7:46am UTC](https://discuss.elastic.co/t/app-search-synonyms/308590 "2022-07-18T07:46:52Z")
Hello, Just a quick thing : Given - I have an understanding that we need to add synonyms to the engines where we want to search using similar meaning words. Question - Is synonyms for basic or obvious words are taken … --- ## [Cannot connect to elastic cluster on VPN machine](https://discuss.elastic.co/t/cannot-connect-to-elastic-cluster-on-vpn-machine/309848)
**Author:** [@Michal\_Bogusz](https://discuss.elastic.co/u/Michal_Bogusz)\ **Replies:** 2\ **Last updated:** [July 18, 2022, 7:43am UTC](https://discuss.elastic.co/t/cannot-connect-to-elastic-cluster-on-vpn-machine/309848 "2022-07-18T07:43:47Z")
Hi, I am new to Elasticsearch. What I managed to do so far is to bring up ElasticSearch container and successfully connect to it using both http connection(via Postman) and using python client. However, now I want to br… --- ## [Logstash http filter with JWT](https://discuss.elastic.co/t/logstash-http-filter-with-jwt/309856)
**Author:** [@pszemesy](https://discuss.elastic.co/u/pszemesy)\ **Replies:** 0\ **Last updated:** [July 18, 2022, 7:42am UTC](https://discuss.elastic.co/t/logstash-http-filter-with-jwt/309856 "2022-07-18T07:42:10Z")
Hi, I have to creat a Logstash pipeline where I have to enrich the data from Remedy. But Remedy requires JWT based authentication. Are there any way to use JWT token based authentication in http filter? Thanks. --- ## [Duplicate document](https://discuss.elastic.co/t/duplicate-document/309827)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 3\ **Last updated:** [July 18, 2022, 7:24am UTC](https://discuss.elastic.co/t/duplicate-document/309827 "2022-07-18T07:24:17Z")
I have few servers running metricbeats and all of them sends data to proxy server on port xyz then proxy redirects them to three logstash server in load balance manner. all three logstash runs same configuration via pi… --- ## [Elasticsearch.Net.ElasticsearchClientException: BulkAll halted after failed product check](https://discuss.elastic.co/t/elasticsearch-net-elasticsearchclientexception-bulkall-halted-after-failed-product-check/309076)
**Author:** [@dawood\_ahmad](https://discuss.elastic.co/u/dawood_ahmad)\ **Replies:** 2\ **Last updated:** [July 18, 2022, 5:09am UTC](https://discuss.elastic.co/t/elasticsearch-net-elasticsearchclientexception-bulkall-halted-after-failed-product-check/309076 "2022-07-18T05:09:23Z")
Unhandled exception. Elasticsearch.Net.ElasticsearchClientException: BulkAll halted after failed product check at Nest.BlockingSubscribeExtensions.WaitOnObservable\[TObservable,TObserve,TObserver\](TObservable observabl… --- ## [Create user in elastic with only read permssion](https://discuss.elastic.co/t/create-user-in-elastic-with-only-read-permssion/309829)
**Author:** [@superuser](https://discuss.elastic.co/u/superuser)\ **Replies:** 1\ **Last updated:** [July 18, 2022, 3:58am UTC](https://discuss.elastic.co/t/create-user-in-elastic-with-only-read-permssion/309829 "2022-07-18T03:58:24Z")
Hi Team, I am trying to create a user in Kibana, who will have access to only Dashboard and Discover tabs. an it possible to give only one index to the space where one will be able to just view (in discover ) the visua… --- ## [How to add an additional ingest pipeline to filebeat when using a module](https://discuss.elastic.co/t/how-to-add-an-additional-ingest-pipeline-to-filebeat-when-using-a-module/309834)
**Author:** [@Zack1](https://discuss.elastic.co/u/Zack1)\ **Replies:** 5\ **Last updated:** [July 18, 2022, 3:06am UTC](https://discuss.elastic.co/t/how-to-add-an-additional-ingest-pipeline-to-filebeat-when-using-a-module/309834 "2022-07-18T03:06:59Z")
I have apache logs, so I use the apache module in filebeat. However, I also want to apply an ingest pipeline I have made that adds an email field, but apparently the output.elasticsearch.pipeline option doesn't work when… --- ## [What indexes will be used during elastic search?](https://discuss.elastic.co/t/what-indexes-will-be-used-during-elastic-search/309819)
**Author:** [@davecomputertips](https://discuss.elastic.co/u/davecomputertips)\ **Replies:** 7\ **Last updated:** [July 18, 2022, 3:06am UTC](https://discuss.elastic.co/t/what-indexes-will-be-used-during-elastic-search/309819 "2022-07-18T03:06:28Z")
I am confused because this is ambiguous question. Is it asking about inverted index? Or this? My teacher died last week and we haven't got a new teacher yet :frowning: RIP --- ## [Single JSON document parsed as individual key pair in logstash](https://discuss.elastic.co/t/single-json-document-parsed-as-individual-key-pair-in-logstash/309787)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 1\ **Last updated:** [July 18, 2022, 3:00am UTC](https://discuss.elastic.co/t/single-json-document-parsed-as-individual-key-pair-in-logstash/309787 "2022-07-18T03:00:27Z")
Hello, I hope my message find the community members and their loved ones safe and healthy. I am trying to ingest ~30,000 documents which are API responses with selective keys stored as a JSON document. \[ { … --- ## [Why artifacts.elastic.co/GPG-KEY-elasticsearch returns 403 for russian IPs, but not always?](https://discuss.elastic.co/t/why-artifacts-elastic-co-gpg-key-elasticsearch-returns-403-for-russian-ips-but-not-always/300969)
**Author:** [@h\_Kleiser](https://discuss.elastic.co/u/h_Kleiser)\ **Replies:** 26\ **Last updated:** [July 17, 2022, 10:57pm UTC](https://discuss.elastic.co/t/why-artifacts-elastic-co-gpg-key-elasticsearch-returns-403-for-russian-ips-but-not-always/300969 "2022-07-17T22:57:50Z")
I'm making rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch and it return 403, but not always, like 30% of all request passes normally. What's the point of such behaviour? --- ## [Failed to start Filebeat sends log files to Logstash or directly to Elasticsearch](https://discuss.elastic.co/t/failed-to-start-filebeat-sends-log-files-to-logstash-or-directly-to-elasticsearch/309477)
**Author:** [@Nishi\_Shah](https://discuss.elastic.co/u/Nishi_Shah)\ **Replies:** 2\ **Last updated:** [July 17, 2022, 10:53pm UTC](https://discuss.elastic.co/t/failed-to-start-filebeat-sends-log-files-to-logstash-or-directly-to-elasticsearch/309477 "2022-07-17T22:53:44Z")
I am trying to configure filebeat(7.17.5). when i restart filebeat only status is showing active \[root@ELK-LAB-VM filebeat\]# systemctl status filebeat ● filebeat.service - Filebeat sends log files to Logstash or dir… --- ## [How to parse CRI logs](https://discuss.elastic.co/t/how-to-parse-cri-logs/309814)
**Author:** [@redoran](https://discuss.elastic.co/u/redoran)\ **Replies:** 0\ **Last updated:** [July 17, 2022, 10:35am UTC](https://discuss.elastic.co/t/how-to-parse-cri-logs/309814 "2022-07-17T10:35:33Z")
Hello, (sry for my english) I'm trying to parse CRI logs. Logs not in JSON format are not parsed. An example of logs that are parsed correctly 2022-07-15T14:50:51.994932036Z stdout F {"@timestamp":"timestamp","@versio… --- ## [How to determine number of master nodes](https://discuss.elastic.co/t/how-to-determine-number-of-master-nodes/309780)
**Author:** [@wisam\_9mol](https://discuss.elastic.co/u/wisam_9mol)\ **Replies:** 10\ **Last updated:** [July 17, 2022, 9:37am UTC](https://discuss.elastic.co/t/how-to-determine-number-of-master-nodes/309780 "2022-07-17T09:37:58Z")
Is there a way to find out how many master nodes I need for a number of data nodes For example If I have 100 data nodes, how many masters nodes do I need? In the case of 12 data nodes Or 18 data nodes --- ## [Is there a way to place one or more elasticsearch shards on a separate disk?](https://discuss.elastic.co/t/is-there-a-way-to-place-one-or-more-elasticsearch-shards-on-a-separate-disk/309797)
**Author:** [@jalustig](https://discuss.elastic.co/u/jalustig)\ **Replies:** 5\ **Last updated:** [July 17, 2022, 8:21am UTC](https://discuss.elastic.co/t/is-there-a-way-to-place-one-or-more-elasticsearch-shards-on-a-separate-disk/309797 "2022-07-17T08:21:52Z")
I have a large elasticsearch index which is stored on an external blob storage device attached to my server, but I also have a large amount of free space on the main hard disk. Id like to put that space to use, if possib… --- ## [Duplicates between rolled over indices](https://discuss.elastic.co/t/duplicates-between-rolled-over-indices/309807)
**Author:** [@wisemandev](https://discuss.elastic.co/u/wisemandev)\ **Replies:** 1\ **Last updated:** [July 17, 2022, 8:04am UTC](https://discuss.elastic.co/t/duplicates-between-rolled-over-indices/309807 "2022-07-17T08:04:59Z")
Hi everyone, I realise this is a fairly common part of the learning curve with implementing ES, I just cant find quite the right info to ease my mind about our indexing strategy. Hoping to get some guidance here. We ha… --- ## [Pipelines don't working](https://discuss.elastic.co/t/pipelines-dont-working/309793)
**Author:** [@White\_Hat](https://discuss.elastic.co/u/White_Hat)\ **Replies:** 3\ **Last updated:** [July 17, 2022, 6:35am UTC](https://discuss.elastic.co/t/pipelines-dont-working/309793 "2022-07-17T06:35:30Z")
I want to log two separate servers that each has filebeat installed. this is my pipeline: - pipeline.id: beats-server config.string: | input { beats { port =\> 5400 } } output { if \[source\] == 'src' { … --- ## [ES and Kibana are working but I can't access it](https://discuss.elastic.co/t/es-and-kibana-are-working-but-i-cant-access-it/309795)
**Author:** [@RusseL](https://discuss.elastic.co/u/RusseL)\ **Replies:** 1\ **Last updated:** [July 17, 2022, 3:57am UTC](https://discuss.elastic.co/t/es-and-kibana-are-working-but-i-cant-access-it/309795 "2022-07-17T03:57:05Z")
Hi there. I installed the latest versions of elasticsearch and kibana. sudo systemctl status elasticsearcrh.service and sudo systemctl status kibana.service When I run them both appear to be working. I am able to m… --- ## [Log timestamp is not getting through date filter getting date\_time\_parse\_exception](https://discuss.elastic.co/t/log-timestamp-is-not-getting-through-date-filter-getting-date-time-parse-exception/309705)
**Author:** [@tahseen\_fatima](https://discuss.elastic.co/u/tahseen_fatima)\ **Replies:** 17\ **Last updated:** [July 16, 2022, 9:23pm UTC](https://discuss.elastic.co/t/log-timestamp-is-not-getting-through-date-filter-getting-date-time-parse-exception/309705 "2022-07-16T21:23:19Z")
Hi Team, I have following logstash configuration for where I have written the date pattern to parse my timestamp, but time is not working. I'm getting \_dateparsefailure. here is the configuration. input{.............}… --- ## [How to use dsl](https://discuss.elastic.co/t/how-to-use-dsl/309718)
**Author:** [@Aniket\_Pant](https://discuss.elastic.co/u/Aniket_Pant)\ **Replies:** 2\ **Last updated:** [July 16, 2022, 6:29pm UTC](https://discuss.elastic.co/t/how-to-use-dsl/309718 "2022-07-16T18:29:43Z")
Hi all, I want to learn elasticsearch DSL but i dont know what is the use of query , filter. I want practical questions which teach me actual use of particular function. Sorry for this silly question but i want to learn… [Previous page](https://discuss.elastic.co/latest.md?page=1043) [Next page](https://discuss.elastic.co/latest.md?page=1045)