# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=691 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 692 --- ## [Logstash problem with disabling ecs\_compatibility](https://discuss.elastic.co/t/logstash-problem-with-disabling-ecs-compatibility/321740)
**Author:** [@Anonym123](https://discuss.elastic.co/u/Anonym123)\ **Replies:** 2\ **Last updated:** [December 21, 2022, 12:18pm UTC](https://discuss.elastic.co/t/logstash-problem-with-disabling-ecs-compatibility/321740 "2022-12-21T12:18:06Z")
logstash image is: 7.17.3 with the prune plugin This is my logstash.conf pipeline.ecs\_compatibility: disabled input { beats { port =\> "XXXXXX" ecs\_compatibility =\> disabled } } … --- ## [Scrape metrics from elastic search 8](https://discuss.elastic.co/t/scrape-metrics-from-elastic-search-8/321758)
**Author:** [@smiley\_tamy](https://discuss.elastic.co/u/smiley_tamy)\ **Replies:** 0\ **Last updated:** [December 21, 2022, 12:17pm UTC](https://discuss.elastic.co/t/scrape-metrics-from-elastic-search-8/321758 "2022-12-21T12:17:42Z")
elasticsearch-prometheus-exporter is used to collect all relevant metrics and makes them available to Prometheus via the Elasticsearch REST API. http://:9200/\_prometheus/metrics This plugin supports till 7.17.7 version … --- ## [How to connect local kibana with elasticsearch ssh port forward?](https://discuss.elastic.co/t/how-to-connect-local-kibana-with-elasticsearch-ssh-port-forward/321757)
**Author:** [@tariqsajid](https://discuss.elastic.co/u/tariqsajid)\ **Replies:** 0\ **Last updated:** [December 21, 2022, 12:11pm UTC](https://discuss.elastic.co/t/how-to-connect-local-kibana-with-elasticsearch-ssh-port-forward/321757 "2022-12-21T12:11:34Z")
Hello, I would want to know how to connect my laptop's running kibana to elasticsearch on AWS while port forwarding it to my local system. chrome extension ## Multi Elasticsearch Head I'm not sure how to link Kibana to… --- ## [Logstash-output-syslog never writes anything to local syslog on REL8 in podman with podman-compose](https://discuss.elastic.co/t/logstash-output-syslog-never-writes-anything-to-local-syslog-on-rel8-in-podman-with-podman-compose/321697)
**Author:** [@Dale\_Bingham](https://discuss.elastic.co/u/Dale_Bingham)\ **Replies:** 5\ **Last updated:** [December 21, 2022, 12:07pm UTC](https://discuss.elastic.co/t/logstash-output-syslog-never-writes-anything-to-local-syslog-on-rel8-in-podman-with-podman-compose/321697 "2022-12-21T12:07:40Z")
I have ELK in my podman-compose setup and I can push any logs I get to Logstash through to Elasticsearch just fine. It is when I am trying to also send to syslog for those that already have a process to pull all syslog d… --- ## [Logstash Scheduled Run of conf file and error alert](https://discuss.elastic.co/t/logstash-scheduled-run-of-conf-file-and-error-alert/321743)
**Author:** [@Sandeep\_Raju](https://discuss.elastic.co/u/Sandeep_Raju)\ **Replies:** 1\ **Last updated:** [December 21, 2022, 11:41am UTC](https://discuss.elastic.co/t/logstash-scheduled-run-of-conf-file-and-error-alert/321743 "2022-12-21T11:41:32Z")
Hi , I'm using logstash to push my CSV data from a location to elasticsearch database. In my project I want a scheduled CSV data push at a particular time. Also want to know if any error is generated while pushing dat… --- ## [Internal Server Error after trying to save a Query](https://discuss.elastic.co/t/internal-server-error-after-trying-to-save-a-query/318416)
**Author:** [@LightX](https://discuss.elastic.co/u/LightX)\ **Replies:** 2\ **Last updated:** [November 23, 2022, 11:43am UTC](https://discuss.elastic.co/t/internal-server-error-after-trying-to-save-a-query/318416 "2022-11-23T11:43:12Z")
Hello together, we are using Kibana with version 7.8.1, unfortunately I am completely noob on Kibana so would really appricate any help. Basically, as soon as I try to "save search" I get internal server errors.... If … --- ## [Cluster yellow after upgrade from 7.16.2 to 7.17.7](https://discuss.elastic.co/t/cluster-yellow-after-upgrade-from-7-16-2-to-7-17-7/321412)
**Author:** [@duckasylum](https://discuss.elastic.co/u/duckasylum)\ **Replies:** 2\ **Last updated:** [December 21, 2022, 10:04am UTC](https://discuss.elastic.co/t/cluster-yellow-after-upgrade-from-7-16-2-to-7-17-7/321412 "2022-12-21T10:04:24Z")
Hi, I started a rolling upgrade as per directions from the guide. I have 6 data nodes (n1 and n2 are also ingest nodes) and after successfully upgrading n4 I upgraded n6 and the cluster is unable to get from yellow to gr… --- ## [Average size of data indices](https://discuss.elastic.co/t/average-size-of-data-indices/321625)
**Author:** [@m3bgwad](https://discuss.elastic.co/u/m3bgwad)\ **Replies:** 5\ **Last updated:** [December 21, 2022, 10:03am UTC](https://discuss.elastic.co/t/average-size-of-data-indices/321625 "2022-12-21T10:03:36Z")
I have more of questions regarding size of data. what is average of data can Elasticsearch store par second? if there is 1M document of data in log file and volume 1G after storing in elastic database what is the new … --- ## [How to index text files (pdf, doc, txt...) in Java?](https://discuss.elastic.co/t/how-to-index-text-files-pdf-doc-txt-in-java/321678)
**Author:** [@gagidza](https://discuss.elastic.co/u/gagidza)\ **Replies:** 5\ **Last updated:** [December 21, 2022, 9:48am UTC](https://discuss.elastic.co/t/how-to-index-text-files-pdf-doc-txt-in-java/321678 "2022-12-21T09:48:53Z")
Hello everyone, some time ago I started to deal with Elasticsearch. Now I am faced with the task of indexing various text files from a certain directory for subsequent search by their contents. Since Elasticsearch uses … --- ## [Elastic search transform not working on nested fields](https://discuss.elastic.co/t/elastic-search-transform-not-working-on-nested-fields/321649)
**Author:** [@pranay\_jain](https://discuss.elastic.co/u/pranay_jain)\ **Replies:** 2\ **Last updated:** [December 21, 2022, 9:45am UTC](https://discuss.elastic.co/t/elastic-search-transform-not-working-on-nested-fields/321649 "2022-12-21T09:45:31Z")
Hi, I have an index which looks like below "properties": { "employee\_id": { "type": "long" }, "employee\_details": { "type": "nested", "dynamic": "false", "properties": { "city": { "t… --- ## [Java BulkProcessor](https://discuss.elastic.co/t/java-bulkprocessor/321733)
**Author:** [@Atefeh](https://discuss.elastic.co/u/Atefeh)\ **Replies:** 1\ **Last updated:** [December 21, 2022, 9:43am UTC](https://discuss.elastic.co/t/java-bulkprocessor/321733 "2022-12-21T09:43:18Z")
Bulk Java API supported adding a payload to each request in ES version 6.8 : public BulkProcessor add(DocWriteRequest request, @Nullable Object payload) { internalAdd(request, payload); return this; … --- ## [Logstash not able to connect to elastic search hosted on asw](https://discuss.elastic.co/t/logstash-not-able-to-connect-to-elastic-search-hosted-on-asw/321737)
**Author:** [@shivam585](https://discuss.elastic.co/u/shivam585)\ **Replies:** 8\ **Last updated:** [December 21, 2022, 9:17am UTC](https://discuss.elastic.co/t/logstash-not-able-to-connect-to-elastic-search-hosted-on-asw/321737 "2022-12-21T09:17:30Z")
\[2022-12-21T07:09:39,189\]\[INFO \]\[logstash.runner \] Log4j configuration path used is: /etc/logstash/log4j2.properties \[2022-12-21T07:09:39,205\]\[INFO \]\[logstash.runner \] Starting Logstash {"logstash.versi… --- ## [Fleet server loading slowly](https://discuss.elastic.co/t/fleet-server-loading-slowly/321108)
**Author:** [@krzychohoho](https://discuss.elastic.co/u/krzychohoho)\ **Replies:** 1\ **Last updated:** [December 21, 2022, 9:15am UTC](https://discuss.elastic.co/t/fleet-server-loading-slowly/321108 "2022-12-21T09:15:57Z")
Hi, I came across a strange issue. My Fleet server is permamently loading. After ten minutes it pops up but there are other issues. This is what I see for 4 - 10 minutes every time: My Elastick stack is v 8.5.1. The… --- ## [How to connect es cluster](https://discuss.elastic.co/t/how-to-connect-es-cluster/321712)
**Author:** [@duocoding](https://discuss.elastic.co/u/duocoding)\ **Replies:** 1\ **Last updated:** [December 21, 2022, 8:39am UTC](https://discuss.elastic.co/t/how-to-connect-es-cluster/321712 "2022-12-21T08:39:27Z")
// Create the low-level client RestClient restClient = RestClient.builder( new HttpHost("localhost", 9200)).build(); // Create the transport with a Jackson mapper ElasticsearchTransport transport = new RestClientTra… --- ## [Elastic Agent 8.5.3 Logging](https://discuss.elastic.co/t/elastic-agent-8-5-3-logging/321732)
**Author:** [@zirota](https://discuss.elastic.co/u/zirota)\ **Replies:** 0\ **Last updated:** [December 21, 2022, 8:24am UTC](https://discuss.elastic.co/t/elastic-agent-8-5-3-logging/321732 "2022-12-21T08:24:42Z")
Hello, I am facing some difficulties in configuring the agent log path when running in standalone mode. Agent Version: 8.5.3 Distribution: linux\_x86-64 Config: Startup command: ./elastic-agent -run -c \ **Author:** [@fim01](https://discuss.elastic.co/u/fim01)\ **Replies:** 3\ **Last updated:** [December 21, 2022, 8:14am UTC](https://discuss.elastic.co/t/kibana-transforms/321141 "2022-12-21T08:14:41Z") I use Pivot to group data within the Transforms section in Kibana. I'm wondering if its possible to group by a certain field within a defined time intervall? In my case I would like to group by a field only within a da… --- ## [javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user](https://discuss.elastic.co/t/javax-security-auth-login-loginexception-could-not-login-the-client-is-being-asked-for-a-password-but-the-kafka-client-code-does-not-currently-support-obtaining-a-password-from-the-user-not-available-to-garner-authentication-information-from-the-user/321723)
**Author:** [@Balaganesh\_G](https://discuss.elastic.co/u/Balaganesh_G)\ **Replies:** 0\ **Last updated:** [December 21, 2022, 7:19am UTC](https://discuss.elastic.co/t/javax-security-auth-login-loginexception-could-not-login-the-client-is-being-asked-for-a-password-but-the-kafka-client-code-does-not-currently-support-obtaining-a-password-from-the-user-not-available-to-garner-authentication-information-from-the-user/321723 "2022-12-21T07:19:49Z")
Hi Team, I have using logstash 8.3.3 and migirated to new Logstash 8.5.3 facing below error. javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client cod… --- ## [Override logging.level in a Fleet managed Filebeat?](https://discuss.elastic.co/t/override-logging-level-in-a-fleet-managed-filebeat/321688)
**Author:** [@Akash\_Deep](https://discuss.elastic.co/u/Akash_Deep)\ **Replies:** 4\ **Last updated:** [December 21, 2022, 6:50am UTC](https://discuss.elastic.co/t/override-logging-level-in-a-fleet-managed-filebeat/321688 "2022-12-21T06:50:56Z")
I have a Fleet managed Elastic Agent running as a service which on systemctl start elastic-agent starts-up filebeat with some flags(as seen in the picture). HOW DO I OVERRIDE THE LOGGING.LEVEL IN A MANAGED SETUP? I'… --- ## [Storage upgrade | new disk added | how to extend storage for ELK](https://discuss.elastic.co/t/storage-upgrade-new-disk-added-how-to-extend-storage-for-elk/321715)
**Author:** [@theacodes](https://discuss.elastic.co/u/theacodes)\ **Replies:** 1\ **Last updated:** [December 21, 2022, 6:42am UTC](https://discuss.elastic.co/t/storage-upgrade-new-disk-added-how-to-extend-storage-for-elk/321715 "2022-12-21T06:42:55Z")
Hi! I've added a new disk for storage upgrade and I want to retain data for 45 day then move it to new disk but that data should still be searchable after 45 days. can someone brief me on how to get this done. I've al… --- ## [Upgrade from Transport Client to HLRC / JAVA API Client? Which to choose?](https://discuss.elastic.co/t/upgrade-from-transport-client-to-hlrc-java-api-client-which-to-choose/320869)
**Author:** [@Umang\_Pachaury](https://discuss.elastic.co/u/Umang_Pachaury)\ **Replies:** 6\ **Last updated:** [December 21, 2022, 6:30am UTC](https://discuss.elastic.co/t/upgrade-from-transport-client-to-hlrc-java-api-client-which-to-choose/320869 "2022-12-21T06:30:08Z")
Hello Team, I am fairly new to Elasticsearch and I have project in which we use Java Client provided by Elasticsearch to fetch logs from Elasticsearch and use those logs in our Application. The thing is that the project … --- ## [Using runtime field in another runtime field in Kibana index template](https://discuss.elastic.co/t/using-runtime-field-in-another-runtime-field-in-kibana-index-template/320734)
**Author:** [@mikhatanu](https://discuss.elastic.co/u/mikhatanu)\ **Replies:** 2\ **Last updated:** [December 21, 2022, 6:13am UTC](https://discuss.elastic.co/t/using-runtime-field-in-another-runtime-field-in-kibana-index-template/320734 "2022-12-21T06:13:33Z")
I have a runtime field called "throughput\_in" with type long and the following script: throughput\_in: emit(doc\['volume\_uplink'\].value / doc\['duration'\].value) and a field called "throughput\_out" with type long: throug… --- ## [Logstash filter code that sends a query to related indices by wild card \[indexname-\*\]](https://discuss.elastic.co/t/logstash-filter-code-that-sends-a-query-to-related-indices-by-wild-card-indexname/321718)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 0\ **Last updated:** [December 21, 2022, 5:54am UTC](https://discuss.elastic.co/t/logstash-filter-code-that-sends-a-query-to-related-indices-by-wild-card-indexname/321718 "2022-12-21T05:54:50Z")
Hello All, Can someone please help me out how can I write custom filter in logstash to achieve the following: Making a custom logstash filter that sends a query to related indices by wild card (indexname-\*) for records… --- ## [Kibana 8.4: How do disable ML , APM, or entire observability app?](https://discuss.elastic.co/t/kibana-8-4-how-do-disable-ml-apm-or-entire-observability-app/316447)
**Author:** [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Replies:** 1\ **Last updated:** [October 26, 2022, 12:54pm UTC](https://discuss.elastic.co/t/kibana-8-4-how-do-disable-ml-apm-or-entire-observability-app/316447 "2022-10-26T12:54:42Z")
We run elastic 8.4.3, elasticsearch + kibana. We dont run any "ml", but kibana tries to call elasticsearch ML api: \[2022-10-12T01:19:52.716+00:00\]\[ERROR\]\[plugins.taskManager\] Task ML:saved-objects-sync "ML:saved-object… --- ## [Delete by query error (version conflict)](https://discuss.elastic.co/t/delete-by-query-error-version-conflict/321365)
**Author:** [@theacodes](https://discuss.elastic.co/u/theacodes)\ **Replies:** 1\ **Last updated:** [December 21, 2022, 4:57am UTC](https://discuss.elastic.co/t/delete-by-query-error-version-conflict/321365 "2022-12-21T04:57:17Z")
I'm trying to delete some specific documents from an index with a query POST /.ds-logs-system.syslog-default-2022.10.30-000004/\_delete\_by\_query?pretty { "query": { "match": { "process.name": "install.sh" … --- ## [Keytool asking for password for keystore , none applied ever](https://discuss.elastic.co/t/keytool-asking-for-password-for-keystore-none-applied-ever/321713)
**Author:** [@Atul\_Chadha](https://discuss.elastic.co/u/Atul_Chadha)\ **Replies:** 0\ **Last updated:** [December 21, 2022, 3:57am UTC](https://discuss.elastic.co/t/keytool-asking-for-password-for-keystore-none-applied-ever/321713 "2022-12-21T03:57:27Z")
We are on ES version 7.17 and trying to setting up cross cluster ( remote ). I am trying to add cert from one node to remote in the elastic-certificates.p12 file using below command keytool -importcert -noprompt -keysto… --- ## [How to replace "and" operator inside "must\_not" to group clauses. Query migration to ES 7.6.1](https://discuss.elastic.co/t/how-to-replace-and-operator-inside-must-not-to-group-clauses-query-migration-to-es-7-6-1/321706)
**Author:** [@aanaya](https://discuss.elastic.co/u/aanaya)\ **Replies:** 2\ **Last updated:** [December 21, 2022, 12:51am UTC](https://discuss.elastic.co/t/how-to-replace-and-operator-inside-must-not-to-group-clauses-query-migration-to-es-7-6-1/321706 "2022-12-21T00:51:31Z")
Hi guys, I'm trying to migrate following query to ES 7.6.1, but not sure how to replace "and" functionality cause it looks like it's not longer supported in ES 7.6.1. ... "must\_not": \[ { "and": \[ { … --- ## [Error 403 Forbidden when trying to install a plugin](https://discuss.elastic.co/t/error-403-forbidden-when-trying-to-install-a-plugin/321707)
**Author:** [@vadd98](https://discuss.elastic.co/u/vadd98)\ **Replies:** 1\ **Last updated:** [December 21, 2022, 12:50am UTC](https://discuss.elastic.co/t/error-403-forbidden-when-trying-to-install-a-plugin/321707 "2022-12-21T00:50:48Z")
Hi, I'm running a Dockerized Elasticsearch cluster on multiple Italian servers, but one of those is always getting a 403 http response from Elasticsearch servers. This is the exception I'm getting: Exception in thread … --- ## [Unable to connect to kibana with a secure cluster](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444)
**Author:** [@diegz](https://discuss.elastic.co/u/diegz)\ **Replies:** 19\ **Last updated:** [October 26, 2022, 12:15pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444 "2022-10-26T12:15:22Z")
Hello everyone, I have configured a cluster of 5 secure nodes with this configuration (adapted on each node :slight\_smile: xpack.security.http.ssl: enabled: true keystore.path: /etc/elasticsearch/new/http.p12 tru… --- ## [Internal monitoring indices shards in UNASSIGNED state](https://discuss.elastic.co/t/internal-monitoring-indices-shards-in-unassigned-state/321247)
**Author:** [@ppine7](https://discuss.elastic.co/u/ppine7)\ **Replies:** 8\ **Last updated:** [December 20, 2022, 11:14pm UTC](https://discuss.elastic.co/t/internal-monitoring-indices-shards-in-unassigned-state/321247 "2022-12-20T23:14:52Z")
Hi, We have an 8.x ES cluster running on Elastic cloud, All seems to be working ok except that a few internal monitoring indices have one shard marked as UNASSIGNED and this causes the cluster state to be YELLOW, all th… --- ## [Fleet server and agents unable to write data to Elasticsearch - Bad Certificate](https://discuss.elastic.co/t/fleet-server-and-agents-unable-to-write-data-to-elasticsearch-bad-certificate/321675)
**Author:** [@rude](https://discuss.elastic.co/u/rude)\ **Replies:** 1\ **Last updated:** [December 20, 2022, 10:59pm UTC](https://discuss.elastic.co/t/fleet-server-and-agents-unable-to-write-data-to-elasticsearch-bad-certificate/321675 "2022-12-20T22:59:30Z")
I'm getting an error when Fleet agents are sending data to Elastic. On Kibana, at the fleet tab I can see Server and Agent are reporting correctly. If I change any of the policies they get redeployed. However, no data is… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=690) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=692)