# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1016 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1017 --- ## [Elasticsearch 8.1 fail to start](https://discuss.elastic.co/t/elasticsearch-8-1-fail-to-start/299524)
**Author:** [@mayayisim](https://discuss.elastic.co/u/mayayisim)\ **Replies:** 1\ **Last updated:** [March 13, 2022, 12:37pm UTC](https://discuss.elastic.co/t/elasticsearch-8-1-fail-to-start/299524 "2022-03-13T12:37:27Z")
Good day Please assist with Elasticsearch failing to start, I checked the logs and its complaining about access denied on /var/run/Elasticsearch. I then created the path and now it's complaining about the Elasticsearch.… --- ## [Output format](https://discuss.elastic.co/t/output-format/299558)
**Author:** [@avnere](https://discuss.elastic.co/u/avnere)\ **Replies:** 0\ **Last updated:** [March 13, 2022, 8:20am UTC](https://discuss.elastic.co/t/output-format/299558 "2022-03-13T08:20:47Z")
Hi, Is there a way to get csv output format when querying by DSL? I show that when querying by SQL, you have to option to do it. Thanks... --- ## [Elastic agent failing to enroll on a fresh install](https://discuss.elastic.co/t/elastic-agent-failing-to-enroll-on-a-fresh-install/298895)
**Author:** [@epheria](https://discuss.elastic.co/u/epheria)\ **Replies:** 3\ **Last updated:** [March 13, 2022, 6:50am UTC](https://discuss.elastic.co/t/elastic-agent-failing-to-enroll-on-a-fresh-install/298895 "2022-03-13T06:50:03Z")
Hello! I am having trouble getting an elastic agent to enroll on a host for monitoring (more specific details to come below). As an overview, here's what I'm working with: Freshly installed Elasticsearch and kibana (… --- ## [How about collect "request.body.bytes" in Nginx ingest-pipeline provided by Filebeat?](https://discuss.elastic.co/t/how-about-collect-request-body-bytes-in-nginx-ingest-pipeline-provided-by-filebeat/299554)
**Author:** [@Lee\_Sun\_Ro](https://discuss.elastic.co/u/Lee_Sun_Ro)\ **Replies:** 0\ **Last updated:** [March 13, 2022, 5:11am UTC](https://discuss.elastic.co/t/how-about-collect-request-body-bytes-in-nginx-ingest-pipeline-provided-by-filebeat/299554 "2022-03-13T05:11:35Z")
Hello, I'm a light user of Filebeat. According to the documentation, the nginx.ingress\_controller collects the body size of the http request, but the nginx access module does not. Why is there such a difference? Or am… --- ## [Null Value Replacement in Mapping does not work using NEST](https://discuss.elastic.co/t/null-value-replacement-in-mapping-does-not-work-using-nest/299553)
**Author:** [@xef](https://discuss.elastic.co/u/xef)\ **Replies:** 0\ **Last updated:** [March 13, 2022, 4:09am UTC](https://discuss.elastic.co/t/null-value-replacement-in-mapping-does-not-work-using-nest/299553 "2022-03-13T04:09:42Z")
We have ES version 17.14 and have a mapping for our Index: .Keyword(s =\> s.Name(c =\> c.Website).Index(false).NullValue("")) But if we try to use the update API : var result = await client.UpdateAsync\(post.Id,… --- ## [Remove backslash from Json logs in Kibana](https://discuss.elastic.co/t/remove-backslash-from-json-logs-in-kibana/299550)
**Author:** [@gooner\_holmes](https://discuss.elastic.co/u/gooner_holmes)\ **Replies:** 0\ **Last updated:** [March 13, 2022, 3:27am UTC](https://discuss.elastic.co/t/remove-backslash-from-json-logs-in-kibana/299550 "2022-03-13T03:27:25Z")
filebeat.inputs: - type: log paths: - '/var/lib/docker/containers/\*/\*.log' multiline.type: pattern multiline.pattern: '^\\\\' multiline.negate: true multiline.match: after processors: - add\_docker\_metadat… --- ## [I regret that I couldn't find an OCR parser to handle image/ocr-png.Please set the OCR\_STRATEGY to NO\_OCR or configure yourOCR parser correctly](https://discuss.elastic.co/t/i-regret-that-i-couldnt-find-an-ocr-parser-to-handle-image-ocr-png-please-set-the-ocr-strategy-to-no-ocr-or-configure-yourocr-parser-correctly/299512)
**Author:** [@Morphus1](https://discuss.elastic.co/u/Morphus1)\ **Replies:** 8\ **Last updated:** [March 13, 2022, 3:18am UTC](https://discuss.elastic.co/t/i-regret-that-i-couldnt-find-an-ocr-parser-to-handle-image-ocr-png-please-set-the-ocr-strategy-to-no-ocr-or-configure-yourocr-parser-correctly/299512 "2022-03-13T03:18:59Z")
Hi Everyone, I'm trying to use OCR when indexing but not getting any results. I've tried using tesseract for images and Tika for the pdfs but not a single one will get indexed with "content". The error message is: 09… --- ## [Elastic Query Bool Logic/Subquery Structure Help](https://discuss.elastic.co/t/elastic-query-bool-logic-subquery-structure-help/299491)
**Author:** [@bsmith](https://discuss.elastic.co/u/bsmith)\ **Replies:** 1\ **Last updated:** [March 12, 2022, 5:09pm UTC](https://discuss.elastic.co/t/elastic-query-bool-logic-subquery-structure-help/299491 "2022-03-12T17:09:51Z")
Need help converting some SQL into elastic and stuck on some of the bool logic and probably isn't structured properly. Goal is to get a count of users that are active at a specified start date and are still active at the… --- ## [Creating a new column from existing columns](https://discuss.elastic.co/t/creating-a-new-column-from-existing-columns/299528)
**Author:** [@carl25en](https://discuss.elastic.co/u/carl25en)\ **Replies:** 1\ **Last updated:** [March 12, 2022, 5:00pm UTC](https://discuss.elastic.co/t/creating-a-new-column-from-existing-columns/299528 "2022-03-12T17:00:40Z")
Dear Elasticsearch community, I was wondering how I can create a DateTime column from 5 columns (forecasted\_year,month,day,hour,minute) as it is shown in the image (after creating a new index). I am not sure if this sh… --- ## [Records are getting dropped while creating map using logstash](https://discuss.elastic.co/t/records-are-getting-dropped-while-creating-map-using-logstash/299509)
**Author:** [@Deepak\_Dandotiya](https://discuss.elastic.co/u/Deepak_Dandotiya)\ **Replies:** 1\ **Last updated:** [March 12, 2022, 3:26pm UTC](https://discuss.elastic.co/t/records-are-getting-dropped-while-creating-map-using-logstash/299509 "2022-03-12T15:26:42Z")
Hi Experts, I need your help here. Context is, I am trying to read the data from DB after multiple joins and then forming a Map in Elasticsearch using ID column as agreegate and trying to handle the duplicate event from… --- ## [Elasticsearch Index creation time](https://discuss.elastic.co/t/elasticsearch-index-creation-time/299529)
**Author:** [@nunex\_17](https://discuss.elastic.co/u/nunex_17)\ **Replies:** 0\ **Last updated:** [March 12, 2022, 3:26pm UTC](https://discuss.elastic.co/t/elasticsearch-index-creation-time/299529 "2022-03-12T15:26:17Z")
Hi, Somehow my indexes are created each day at around 17pm. Indexes are being rolled over every day. What is happening is, for example, at 17pm of 11/03 the index-name-2022-03-11-00001 is created. Then, at 17pm of 12/0… --- ## [Filebeat nginx dashboard kubernetes](https://discuss.elastic.co/t/filebeat-nginx-dashboard-kubernetes/299520)
**Author:** [@Mohsin\_Ijaz](https://discuss.elastic.co/u/Mohsin_Ijaz)\ **Replies:** 0\ **Last updated:** [March 12, 2022, 1:16pm UTC](https://discuss.elastic.co/t/filebeat-nginx-dashboard-kubernetes/299520 "2022-03-12T13:16:04Z")
Hi, I am new to ELK and IT field started as a devops recently so you can consider me being a complete noob. I am having trouble setting up the filebeat nginx dashboard. deployed the ELK on Kubernetes cluster, using th… --- ## [Errors getting started with Kibana dev](https://discuss.elastic.co/t/errors-getting-started-with-kibana-dev/299335)
**Author:** [@drnw](https://discuss.elastic.co/u/drnw)\ **Replies:** 3\ **Last updated:** [March 12, 2022, 10:02am UTC](https://discuss.elastic.co/t/errors-getting-started-with-kibana-dev/299335 "2022-03-12T10:02:42Z")
I'm unsuccessful when following the guidance at Getting started | Kibana Guide \[master\] | Elastic I have tried with my MacBook and with a VM on Azure both end up with the same result. Kibana starts and visiting in the b… --- ## [Connect filebeat to ELK server from 2 different machines](https://discuss.elastic.co/t/connect-filebeat-to-elk-server-from-2-different-machines/298435)
**Author:** [@Dhia\_Saibi](https://discuss.elastic.co/u/Dhia_Saibi)\ **Replies:** 2\ **Last updated:** [March 12, 2022, 9:55am UTC](https://discuss.elastic.co/t/connect-filebeat-to-elk-server-from-2-different-machines/298435 "2022-03-12T09:55:34Z")
hi i'm asking how can i connect filebeat installed on one machine to ELK server on another machine..who can help me with this process? --- ## [Logstash filter loglevel info warn and error](https://discuss.elastic.co/t/logstash-filter-loglevel-info-warn-and-error/299306)
**Author:** [@waelboss](https://discuss.elastic.co/u/waelboss)\ **Replies:** 7\ **Last updated:** [March 12, 2022, 9:23am UTC](https://discuss.elastic.co/t/logstash-filter-loglevel-info-warn-and-error/299306 "2022-03-12T09:23:05Z")
i have this JSON data: "result": \[ { "payload": { "context": "default", "level": "DEBUG", "logger": "org.forgerock.opendj.ldap.CachedConnectionPool", "me… --- ## [Problems caused by creating replica shards too quickly](https://discuss.elastic.co/t/problems-caused-by-creating-replica-shards-too-quickly/299513)
**Author:** [@slowup](https://discuss.elastic.co/u/slowup)\ **Replies:** 0\ **Last updated:** [March 12, 2022, 7:07am UTC](https://discuss.elastic.co/t/problems-caused-by-creating-replica-shards-too-quickly/299513 "2022-03-12T07:07:29Z")
When we execute batch execution, if the primary shard is indexed, we are increasing the replica shard at that time, But at that time, the search engine creates replica shards, and as you can see from the metrix, a large … --- ## [Kibana error after upgrade to version 8.0 \[config validation of \[logging\].dest\]: definition for this key is missing\]](https://discuss.elastic.co/t/kibana-error-after-upgrade-to-version-8-0-config-validation-of-logging-dest-definition-for-this-key-is-missing/297948)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 3\ **Last updated:** [March 11, 2022, 11:03pm UTC](https://discuss.elastic.co/t/kibana-error-after-upgrade-to-version-8-0-config-validation-of-logging-dest-definition-for-this-key-is-missing/297948 "2022-03-11T23:03:26Z")
Hi, I had successfully upgraded to version 8.0 on ELK , while the rest of the stack works , kibana has failed to start citing dev-elk-app01 kibana\[19018\]: FATAL Error: \[config validation of \[logging\].dest\]: definitio… --- ## [Not able to create role mapping in kibana](https://discuss.elastic.co/t/not-able-to-create-role-mapping-in-kibana/299472)
**Author:** [@Amisha\_Thakkar1](https://discuss.elastic.co/u/Amisha_Thakkar1)\ **Replies:** 3\ **Last updated:** [March 11, 2022, 8:44pm UTC](https://discuss.elastic.co/t/not-able-to-create-role-mapping-in-kibana/299472 "2022-03-11T20:44:37Z")
I have assigned one user as superuser and I am trying to search for role mapping under stack management in Kibana. But I am not able to see that. Even I tried to assign role using role mapping api. Still not able to log… --- ## [Cleaning up datastreams' indexes](https://discuss.elastic.co/t/cleaning-up-datastreams-indexes/299499)
**Author:** [@hebph001](https://discuss.elastic.co/u/hebph001)\ **Replies:** 0\ **Last updated:** [March 11, 2022, 8:24pm UTC](https://discuss.elastic.co/t/cleaning-up-datastreams-indexes/299499 "2022-03-11T20:24:27Z")
Hi there Elastic community, I'm really new to Elastic and run into an architectural problem with my cloud cluster. I need to delete unneeded old indices to free up some memory. I would like to know what would be the be… --- ## [Elastic Search Failing on multi tier environment](https://discuss.elastic.co/t/elastic-search-failing-on-multi-tier-environment/299495)
**Author:** [@Ish\_Singh](https://discuss.elastic.co/u/Ish_Singh)\ **Replies:** 0\ **Last updated:** [March 11, 2022, 7:32pm UTC](https://discuss.elastic.co/t/elastic-search-failing-on-multi-tier-environment/299495 "2022-03-11T19:32:47Z")
Hi All , we currently have 2 servers for TFS and 1 server hosts the Elasticsearch but somehow the search keeps on failing while accessing and i see the error in developer tools.. i believe its trying to cluster between 2… --- ## [Search based on runtime boolean calculated from now](https://discuss.elastic.co/t/search-based-on-runtime-boolean-calculated-from-now/299488)
**Author:** [@MarkAllan](https://discuss.elastic.co/u/MarkAllan)\ **Replies:** 0\ **Last updated:** [March 11, 2022, 7:14pm UTC](https://discuss.elastic.co/t/search-based-on-runtime-boolean-calculated-from-now/299488 "2022-03-11T19:14:07Z")
I need to be able to search on a field that is not part of the stored document but is derived from a field on that document. The document field is a timestamp called expiry\_utc\_timestamp. I need to be able to filter on … --- ## [Need help with a NOT in if statement, is it "not" or "!"?](https://discuss.elastic.co/t/need-help-with-a-not-in-if-statement-is-it-not-or/299486)
**Author:** [@teebu](https://discuss.elastic.co/u/teebu)\ **Replies:** 1\ **Last updated:** [March 11, 2022, 6:58pm UTC](https://discuss.elastic.co/t/need-help-with-a-not-in-if-statement-is-it-not-or/299486 "2022-03-11T18:58:41Z")
\# filter out local ips if !(\[http\]\[request\]\[headers\]\[CF-Connecting-IP\] =~ "^10.0.\*" or \[http\]\[request\]\[headers\]\[CF-Connecting-IP\] =~ "^127.0.\*" or \[http\]\[request\]\[headers\]\[CF-Connecting-IP\] == "0.0.0.0") … --- ## [Metricbeat user privileges for data ingest?](https://discuss.elastic.co/t/metricbeat-user-privileges-for-data-ingest/299481)
**Author:** [@kmiklas](https://discuss.elastic.co/u/kmiklas)\ **Replies:** 1\ **Last updated:** [March 11, 2022, 6:08pm UTC](https://discuss.elastic.co/t/metricbeat-user-privileges-for-data-ingest/299481 "2022-03-11T18:08:05Z")
What are the privileges required for a metricbeat user to receive data? Under (Hamburger) -\> Stack Management -\> Security -\> Users, I've set up a metricbeat user. Its purpose is to receive metrics. What are its require… --- ## [Saving documents in multiple indexes/Retention policy based on documents fields (Looking for ideas)](https://discuss.elastic.co/t/saving-documents-in-multiple-indexes-retention-policy-based-on-documents-fields-looking-for-ideas/299482)
**Author:** [@scott\_stash](https://discuss.elastic.co/u/scott_stash)\ **Replies:** 0\ **Last updated:** [March 11, 2022, 5:31pm UTC](https://discuss.elastic.co/t/saving-documents-in-multiple-indexes-retention-policy-based-on-documents-fields-looking-for-ideas/299482 "2022-03-11T17:31:23Z")
Using 7.17.. currently I read log files in with filebeat or filebeat/logstash saving in ES. Log lines are dissected and we have a field called loglevel, for ERROR, WARN and INFO. If the message is an ERROR a field call… --- ## [Elasticsearch 8.1 normalizer in strings error](https://discuss.elastic.co/t/elasticsearch-8-1-normalizer-in-strings-error/299476)
**Author:** [@willypol](https://discuss.elastic.co/u/willypol)\ **Replies:** 7\ **Last updated:** [March 11, 2022, 5:30pm UTC](https://discuss.elastic.co/t/elasticsearch-8-1-normalizer-in-strings-error/299476 "2022-03-11T17:30:17Z")
Hello, I've been using the next component template in elastic 7. I recently upgraded to elastic 8.1 and it won't let me create the template like before. PUT \_component\_template/keywords { "template": { "mappings":… --- ## [Snapshot - Register repository for Snapshot](https://discuss.elastic.co/t/snapshot-register-repository-for-snapshot/299388)
**Author:** [@Prabakar](https://discuss.elastic.co/u/Prabakar)\ **Replies:** 3\ **Last updated:** [March 11, 2022, 5:00pm UTC](https://discuss.elastic.co/t/snapshot-register-repository-for-snapshot/299388 "2022-03-11T17:00:51Z")
Hello Team, We are using Elastic on Kubernetes (ECK) with elastic Enterprise license. We have created on- premise elastic clusters. We are trying to create repository to take snapshots, but we are getting below error wh… --- ## [Installed logstash rpm, service keeps stopping with no clear reason](https://discuss.elastic.co/t/installed-logstash-rpm-service-keeps-stopping-with-no-clear-reason/299446)
**Author:** [@jelmew](https://discuss.elastic.co/u/jelmew)\ **Replies:** 2\ **Last updated:** [March 11, 2022, 4:56pm UTC](https://discuss.elastic.co/t/installed-logstash-rpm-service-keeps-stopping-with-no-clear-reason/299446 "2022-03-11T16:56:28Z")
Hi all, We are trying to setup logstash to connect to elastic cloud. We downloaded the rpm and installed it. But even without any configuration pointing to our elastic cloud instance, the service just stops with the mes… --- ## [Grok pattern for stashing Ansible logs](https://discuss.elastic.co/t/grok-pattern-for-stashing-ansible-logs/299227)
**Author:** [@santy1](https://discuss.elastic.co/u/santy1)\ **Replies:** 6\ **Last updated:** [March 11, 2022, 4:06pm UTC](https://discuss.elastic.co/t/grok-pattern-for-stashing-ansible-logs/299227 "2022-03-11T16:06:57Z")
Hello, I am new to Elasticsearch. I am on Elasticsearch 7.17.1. Below are the logs that I am trying to parse. 2022-03-08 14:52:50,672 p=1654827 u=ansible n=ansible | TASK \[lvm : Formatting xfs filesystem\] \*\* 2022-03-… --- ## [How to read group by buckets key values using "ctx" from below query](https://discuss.elastic.co/t/how-to-read-group-by-buckets-key-values-using-ctx-from-below-query/299383)
**Author:** [@srinivasa\_adaboina](https://discuss.elastic.co/u/srinivasa_adaboina)\ **Replies:** 2\ **Last updated:** [March 11, 2022, 4:23pm UTC](https://discuss.elastic.co/t/how-to-read-group-by-buckets-key-values-using-ctx-from-below-query/299383 "2022-03-11T16:23:48Z")
{ "\_shards": { "total": 155, "failed": 0, "successful": 155, "skipped": 0 }, "hits": { "hits": , "total": { "value": 5535, "relation": "eq" }, "max\_score": null }, "took": 44, "timed\_out": false, "aggreg… --- ## [Logstash crashes when kafka producer sends less fields than excepted](https://discuss.elastic.co/t/logstash-crashes-when-kafka-producer-sends-less-fields-than-excepted/299475)
**Author:** [@guriv](https://discuss.elastic.co/u/guriv)\ **Replies:** 0\ **Last updated:** [March 11, 2022, 3:48pm UTC](https://discuss.elastic.co/t/logstash-crashes-when-kafka-producer-sends-less-fields-than-excepted/299475 "2022-03-11T15:48:15Z")
Hello we are using logstash-kafka-integration plugin to read messages from a kafka topic using avro encoding If my logstash consumer refers to a schema registry file that defines 2 optional fields but receive a message… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1015) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1017)