# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1291 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1292 --- ## [How to give hidden file path in input.conf for file input plugin](https://discuss.elastic.co/t/how-to-give-hidden-file-path-in-input-conf-for-file-input-plugin/280617)
**Author:** [@priyankab](https://discuss.elastic.co/u/priyankab)\ **Replies:** 2\ **Last updated:** [August 7, 2021, 3:02am UTC](https://discuss.elastic.co/t/how-to-give-hidden-file-path-in-input-conf-for-file-input-plugin/280617 "2021-08-07T03:02:00Z")
input{ file { exclude =\> "\*.gz" path =\> "/home/abc/work/.productLogs/.auditLogs/.auditLog" type =\> "audit" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" stat\_interval =\> 5 ignore\_older =\> 0 } } --- ## [How do I relate 2 objects with no common field?](https://discuss.elastic.co/t/how-do-i-relate-2-objects-with-no-common-field/280688)
**Author:** [@creativeguitar](https://discuss.elastic.co/u/creativeguitar)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 10:13pm UTC](https://discuss.elastic.co/t/how-do-i-relate-2-objects-with-no-common-field/280688 "2021-08-06T22:13:51Z")
Input Data: \* Operating Systems CSVs \* Fields: OperatingSystem, general\_availability, end\_of\_life \* Examples: \* OperatingSystem------ general\_availability------end\_of\_life \* RHEL 7.9----------------------… --- ## [Mapping: How to do your own calculations](https://discuss.elastic.co/t/mapping-how-to-do-your-own-calculations/280641)
**Author:** [@NoobUser404](https://discuss.elastic.co/u/NoobUser404)\ **Replies:** 2\ **Last updated:** [August 6, 2021, 9:59pm UTC](https://discuss.elastic.co/t/mapping-how-to-do-your-own-calculations/280641 "2021-08-06T21:59:45Z")
How can we do our own calculations through mappings? I've try this configuration but I get an error: PUT nameIndex/\_mapping { "runtime": { "cpuIndex": { "type": "long", "script": { "source": "e… --- ## [Compress elasticsearch bulkrequest using java high level rest client and bulk processor](https://discuss.elastic.co/t/compress-elasticsearch-bulkrequest-using-java-high-level-rest-client-and-bulk-processor/280425)
**Author:** [@Praveen\_Atluri](https://discuss.elastic.co/u/Praveen_Atluri)\ **Replies:** 7\ **Last updated:** [August 6, 2021, 9:32pm UTC](https://discuss.elastic.co/t/compress-elasticsearch-bulkrequest-using-java-high-level-rest-client-and-bulk-processor/280425 "2021-08-06T21:32:36Z")
Is there anyway to compress the elasticsearch bulk request payload using java high level rest client and bulk processor? I have tried below requestConfigBuilder.setContentCompressionEnabled(true); But it do… --- ## [Attach a label to ES query so that to distinguish between the multiple responses](https://discuss.elastic.co/t/attach-a-label-to-es-query-so-that-to-distinguish-between-the-multiple-responses/280683)
**Author:** [@NLSVTN](https://discuss.elastic.co/u/NLSVTN)\ **Replies:** 2\ **Last updated:** [August 6, 2021, 8:42pm UTC](https://discuss.elastic.co/t/attach-a-label-to-es-query-so-that-to-distinguish-between-the-multiple-responses/280683 "2021-08-06T20:42:51Z")
Hi, I am issuing multiple queries (MultiSearch in python Elasticsearch DSL) for ES index and I need a way to set a label 'type' (which could be one of several values) for each query, so that it would be present in the r… --- ## [Kibana detections doesn't generate signals ( Query Result available at Discover )](https://discuss.elastic.co/t/kibana-detections-doesnt-generate-signals-query-result-available-at-discover/280682)
**Author:** [@soufiane\_adn](https://discuss.elastic.co/u/soufiane_adn)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 7:44pm UTC](https://discuss.elastic.co/t/kibana-detections-doesnt-generate-signals-query-result-available-at-discover/280682 "2021-08-06T19:44:27Z")
Hello Team, any help concerning detection signals. we have result available on discover and rule query created on KIBANA but without any alert generated. on rule creation, we tried Quick query preview and it give us r… --- ## [How to enrich a set of linked documents with a field value from one document?](https://discuss.elastic.co/t/how-to-enrich-a-set-of-linked-documents-with-a-field-value-from-one-document/280673)
**Author:** [@Micah\_Hunsberger](https://discuss.elastic.co/u/Micah_Hunsberger)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 5:12pm UTC](https://discuss.elastic.co/t/how-to-enrich-a-set-of-linked-documents-with-a-field-value-from-one-document/280673 "2021-08-06T17:12:06Z")
I have several log sources (files) that are being ingested by filebeat. There will be several (say, 5-20) log lines that are all linked by a single field (I'll call it the transaction id). In just one of the many logs th… --- ## [Can I pass an array into a plugin through a field](https://discuss.elastic.co/t/can-i-pass-an-array-into-a-plugin-through-a-field/280672)
**Author:** [@R.L](https://discuss.elastic.co/u/R.L)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 4:50pm UTC](https://discuss.elastic.co/t/can-i-pass-an-array-into-a-plugin-through-a-field/280672 "2021-08-06T16:50:48Z")
I am currently writing a custom java plugin for Logstash. One of the input settings is an array. This array might be set in another field within the logstash event. so basically I want to be able to do this: #array\_f… --- ## [Trying to Get Grok Pattern for Error.log](https://discuss.elastic.co/t/trying-to-get-grok-pattern-for-error-log/280670)
**Author:** [@shailesh](https://discuss.elastic.co/u/shailesh)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 4:37pm UTC](https://discuss.elastic.co/t/trying-to-get-grok-pattern-for-error-log/280670 "2021-08-06T16:37:33Z")
Hello Guts - i am trying to write a Grok Pattern for the below log file 20210621.00h02m29s RESPONSE: sent 127.0.0.1 status 502 (Tunnel Connection Failed) for 'xxxx.com:443/ for the above log line i am trying to parse … --- ## [Filebeat stops publishing logs to Kafka without any error message](https://discuss.elastic.co/t/filebeat-stops-publishing-logs-to-kafka-without-any-error-message/280654)
**Author:** [@rribeiro](https://discuss.elastic.co/u/rribeiro)\ **Replies:** 1\ **Last updated:** [August 6, 2021, 4:15pm UTC](https://discuss.elastic.co/t/filebeat-stops-publishing-logs-to-kafka-without-any-error-message/280654 "2021-08-06T16:15:29Z")
Hi, An instance of Filebeat 7.13.0 (running on Openshift) stops publishing logs to Kafka after some time. The source logs are read from a NFS volume and are rotated every 5 minutes (a timestamp is added to the filename… --- ## [Elasticsearch NodeJS client shows inaccessible node as alive](https://discuss.elastic.co/t/elasticsearch-nodejs-client-shows-inaccessible-node-as-alive/280664)
**Author:** [@anishsana](https://discuss.elastic.co/u/anishsana)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 3:21pm UTC](https://discuss.elastic.co/t/elasticsearch-nodejs-client-shows-inaccessible-node-as-alive/280664 "2021-08-06T15:21:20Z")
I have a simple function in NodeJS which creates an Elasticsearch client with a cluster of nodes and returns one of the node URLs that is active: // Elastic cluster nodes const ELASTIC\_CLUSTER\_NODES = \[ 'http://node… --- ## [Performance of Elasticsearch for number of documents vs number of properties in a document](https://discuss.elastic.co/t/performance-of-elasticsearch-for-number-of-documents-vs-number-of-properties-in-a-document/280497)
**Author:** [@firabby](https://discuss.elastic.co/u/firabby)\ **Replies:** 3\ **Last updated:** [August 6, 2021, 3:55pm UTC](https://discuss.elastic.co/t/performance-of-elasticsearch-for-number-of-documents-vs-number-of-properties-in-a-document/280497 "2021-08-06T15:55:50Z")
I want to store timeseries data in elasticsearch. But I am not sure how I should model the document. Let's say, the document looks like this. { "timestamp": "iso 8601 date time", "common\_property1": "value", "common\_pr… --- ## [Kibana not starting up after enabling https (Single node) , seems to be missing credentials?](https://discuss.elastic.co/t/kibana-not-starting-up-after-enabling-https-single-node-seems-to-be-missing-credentials/280659)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 4\ **Last updated:** [August 6, 2021, 3:48pm UTC](https://discuss.elastic.co/t/kibana-not-starting-up-after-enabling-https-single-node-seems-to-be-missing-credentials/280659 "2021-08-06T15:48:32Z")
Hello, I'm using Kibana 7.14 and had enabled https recently . All the services logstash and elasticsearch start up successfully while kibana fails . I had been looking at the kibana logs and they show this {"type":"log… --- ## [Elastic rejected execution of coordinating operation](https://discuss.elastic.co/t/elastic-rejected-execution-of-coordinating-operation/280656)
**Author:** [@mug3n451](https://discuss.elastic.co/u/mug3n451)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 2:39pm UTC](https://discuss.elastic.co/t/elastic-rejected-execution-of-coordinating-operation/280656 "2021-08-06T14:39:29Z")
Hi, i have a problem with elasticsearch on kubernetes. here the problem: "current\_state" : "unassigned", "unassigned\_info" : { "reason" : "ALLOCATION\_FAILED", "at" : "2021-08-06T14:28:31.995Z", "failed\_al… --- ## [Questions Regarding Deleting Source Data After Transform](https://discuss.elastic.co/t/questions-regarding-deleting-source-data-after-transform/279353)
**Author:** [@Ashfur](https://discuss.elastic.co/u/Ashfur)\ **Replies:** 5\ **Last updated:** [August 6, 2021, 2:23pm UTC](https://discuss.elastic.co/t/questions-regarding-deleting-source-data-after-transform/279353 "2021-08-06T14:23:42Z")
From what I read from the other posts, if we are running a continuous transform, we should not delete any data from the source index as that will trigger the transform to re-compute and the deleted records will no longer… --- ## [Logstash cannot compile the .conf file](https://discuss.elastic.co/t/logstash-cannot-compile-the-conf-file/279735)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 7\ **Last updated:** [August 6, 2021, 2:21pm UTC](https://discuss.elastic.co/t/logstash-cannot-compile-the-conf-file/279735 "2021-08-06T14:21:52Z")
Hello, so I have this weird situation that all my logstash config files just stopped working. I have noticed it started probably like a few weeks ago after something happened. Only one thing I recognize I have done that… --- ## [Wazuh manager failed to start](https://discuss.elastic.co/t/wazuh-manager-failed-to-start/280626)
**Author:** [@Jedrick](https://discuss.elastic.co/u/Jedrick)\ **Replies:** 2\ **Last updated:** [August 6, 2021, 1:57pm UTC](https://discuss.elastic.co/t/wazuh-manager-failed-to-start/280626 "2021-08-06T13:57:26Z")
For your kind assistance regarding my kibana that is not working. I already tried to restart all services. kibana, filebeat, elasticsearch, wazuh-manager. There status are all ok and active running. But once a access … --- ## [Update by query for each field with value by sql query](https://discuss.elastic.co/t/update-by-query-for-each-field-with-value-by-sql-query/280192)
**Author:** [@Rammi22](https://discuss.elastic.co/u/Rammi22)\ **Replies:** 5\ **Last updated:** [August 6, 2021, 11:14am UTC](https://discuss.elastic.co/t/update-by-query-for-each-field-with-value-by-sql-query/280192 "2021-08-06T11:14:17Z")
Hello, following task: get each document one by one from elasticserach server add field to document with value stored in a database (MariaDB) The \_id of the document is the reference to its record in the database: I… --- ## [Index vs Update](https://discuss.elastic.co/t/index-vs-update/280637)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 2\ **Last updated:** [August 6, 2021, 10:27am UTC](https://discuss.elastic.co/t/index-vs-update/280637 "2021-08-06T10:27:08Z")
Hello I'd like to ask the same question again raised here: What's the benefit of the Update API (POST /\_update) vs Index (PUT /\_doc). As far as I can see Update (with doc\_as\_upsert: true) is identical in effect to Inde… --- ## [Finding Duplicates from an index with nested field](https://discuss.elastic.co/t/finding-duplicates-from-an-index-with-nested-field/280634)
**Author:** [@fastkt](https://discuss.elastic.co/u/fastkt)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 9:37am UTC](https://discuss.elastic.co/t/finding-duplicates-from-an-index-with-nested-field/280634 "2021-08-06T09:37:33Z")
Hi I am fairly new to ES and need some help. I have an index { "shop\_index": { "properties": { "shop\_key": { "type": "nested", "properties": { "shop\_id": "keyword", "shop… --- ## [Unused visualizations](https://discuss.elastic.co/t/unused-visualizations/280624)
**Author:** [@artobstrel](https://discuss.elastic.co/u/artobstrel)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 8:37am UTC](https://discuss.elastic.co/t/unused-visualizations/280624 "2021-08-06T08:37:20Z")
I want to remove all visualizations that are not used on dashboards. How can I get a list of all such visualizations through api or in some other way? Example Used visualization has dashboard parent: Unused visual… --- ## [JDBC streaming plugin, variable statement](https://discuss.elastic.co/t/jdbc-streaming-plugin-variable-statement/280630)
**Author:** [@JFO\_90](https://discuss.elastic.co/u/JFO_90)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 9:06am UTC](https://discuss.elastic.co/t/jdbc-streaming-plugin-variable-statement/280630 "2021-08-06T09:06:18Z")
Hi, Im using elasticsearch input plugin to obtain info from elastic, with this extracted info i use the JDBC streaming plugin to get information from BD with some fields that i extracted from elastic. The problem that … --- ## [Kibana on Arch Linux: "NetworkError when attempting to fetch resource."](https://discuss.elastic.co/t/kibana-on-arch-linux-networkerror-when-attempting-to-fetch-resource/280413)
**Author:** [@brian.rene](https://discuss.elastic.co/u/brian.rene)\ **Replies:** 4\ **Last updated:** [August 6, 2021, 8:07am UTC](https://discuss.elastic.co/t/kibana-on-arch-linux-networkerror-when-attempting-to-fetch-resource/280413 "2021-08-06T08:07:26Z")
Hello everyone, I have been tasked with setting up a self-hosted instance of ELK in a KVM environment using Arch Linux. The root filesystem is write-protected (at runtime) and any directories or files that services need… --- ## [Kubernetes autodiscover with annotation not working](https://discuss.elastic.co/t/kubernetes-autodiscover-with-annotation-not-working/280621)
**Author:** [@edwardlol](https://discuss.elastic.co/u/edwardlol)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 8:00am UTC](https://discuss.elastic.co/t/kubernetes-autodiscover-with-annotation-not-working/280621 "2021-08-06T08:00:23Z")
I want to enable log collecting on specific pods (or namespace, but I'm trying to enable it on pods first). I disabled hints/default\_config filebeat.autodiscover: providers: - type: kubernetes node: ${NODE\_… --- ## [How to query for a documents based on the type of a field?](https://discuss.elastic.co/t/how-to-query-for-a-documents-based-on-the-type-of-a-field/280620)
**Author:** [@k1b](https://discuss.elastic.co/u/k1b)\ **Replies:** 0\ **Last updated:** [August 6, 2021, 7:59am UTC](https://discuss.elastic.co/t/how-to-query-for-a-documents-based-on-the-type-of-a-field/280620 "2021-08-06T07:59:02Z")
I have a geoip.loc field, of type geo\_point. A few documents accidentally got added into the index with this field as (a pair of) float, which is now causing a mapping conflict. I'd like to query for these few document… --- ## [2 node elasticsearch cluster for production like environment](https://discuss.elastic.co/t/2-node-elasticsearch-cluster-for-production-like-environment/280595)
**Author:** [@cool999](https://discuss.elastic.co/u/cool999)\ **Replies:** 4\ **Last updated:** [August 6, 2021, 7:26am UTC](https://discuss.elastic.co/t/2-node-elasticsearch-cluster-for-production-like-environment/280595 "2021-08-06T07:26:18Z")
Hi Team, From below document, I am trying to find if I can use 2 node cluster for production like environment. I know 3 node cluster is always better choice. link above says, if you create 2 node es cluster, then mak… --- ## [Setting up 2-Node cluster on local network](https://discuss.elastic.co/t/setting-up-2-node-cluster-on-local-network/280608)
**Author:** [@nyinyithann](https://discuss.elastic.co/u/nyinyithann)\ **Replies:** 1\ **Last updated:** [August 6, 2021, 5:48am UTC](https://discuss.elastic.co/t/setting-up-2-node-cluster-on-local-network/280608 "2021-08-06T05:48:39Z")
Hi, I am trying to setup 2-node cluster on local network - one node on mac and the other on ubuntu. I am able to connect the node on mac from ubuntu and vice versa. But I see only one node in cluster at all time. I w… --- ## [Logstash and mongodb using JDBC is not working](https://discuss.elastic.co/t/logstash-and-mongodb-using-jdbc-is-not-working/280487)
**Author:** [@TranQuangHa307](https://discuss.elastic.co/u/TranQuangHa307)\ **Replies:** 0\ **Last updated:** [August 5, 2021, 3:16am UTC](https://discuss.elastic.co/t/logstash-and-mongodb-using-jdbc-is-not-working/280487 "2021-08-05T03:16:27Z")
This is my configuration file gives output as \[2021-08-05T10:13:30,445\]\[ERROR\]\[logstash.inputs.jdbc \]\[main\]\[c5849bc82bc4f5a1e20e8bbd042d4783536556e6cc82bbb47534d1a4e0939319\] Unable to connect to database. Tried 1… --- ## [Elastic Agent Prometheus integration stopped working in 7.14](https://discuss.elastic.co/t/elastic-agent-prometheus-integration-stopped-working-in-7-14/280602)
**Author:** [@hilt86](https://discuss.elastic.co/u/hilt86)\ **Replies:** 1\ **Last updated:** [August 6, 2021, 12:33am UTC](https://discuss.elastic.co/t/elastic-agent-prometheus-integration-stopped-working-in-7-14/280602 "2021-08-06T00:33:24Z")
Since v 7.14 my prometheus integration seems to have stopped working. I'm using the api/v1/query endpoint of Prometheus and can poll it using curl but the integration seems to have stopped working... This is in the logs… --- ## [Adding fields to a filter context](https://discuss.elastic.co/t/adding-fields-to-a-filter-context/280356)
**Author:** [@jjammin92](https://discuss.elastic.co/u/jjammin92)\ **Replies:** 4\ **Last updated:** [August 5, 2021, 10:04pm UTC](https://discuss.elastic.co/t/adding-fields-to-a-filter-context/280356 "2021-08-05T22:04:29Z")
Hello, I'm working on a filter query to speed up some searches but I only want to return certain fields. I've been running into some issues adding the fields I want to display to my query. This is what I have working: … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1290) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1292)