# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=855 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 856 --- ## [How to recreate shard failure](https://discuss.elastic.co/t/how-to-recreate-shard-failure/310114)
**Author:** [@vel](https://discuss.elastic.co/u/vel)\ **Replies:** 3\ **Last updated:** [July 22, 2022, 6:03am UTC](https://discuss.elastic.co/t/how-to-recreate-shard-failure/310114 "2022-07-22T06:03:50Z")
I am trying to find a way for recreating the shard failure in Elasticsearch When querying ES can return a 200 response , even if one of the shards had failed to respond. how do I recreate error, for testing. { "too… --- ## [Logstash filter not working, when all the filters are applied at once, but working when applied only one filter](https://discuss.elastic.co/t/logstash-filter-not-working-when-all-the-filters-are-applied-at-once-but-working-when-applied-only-one-filter/310186)
**Author:** [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\ **Replies:** 7\ **Last updated:** [July 22, 2022, 5:15am UTC](https://discuss.elastic.co/t/logstash-filter-not-working-when-all-the-filters-are-applied-at-once-but-working-when-applied-only-one-filter/310186 "2022-07-22T05:15:23Z")
I have added 3 filters in Logstash but at a time only 2 are working, but all the three are not working at the same time. One of the filter is throwing error. When I applied single filter that is working fine, when 2 that… --- ## [Query for end date greater than now or doesn't exist not working](https://discuss.elastic.co/t/query-for-end-date-greater-than-now-or-doesnt-exist-not-working/310361)
**Author:** [@Ellie\_Y](https://discuss.elastic.co/u/Ellie_Y)\ **Replies:** 0\ **Last updated:** [July 22, 2022, 3:43am UTC](https://discuss.elastic.co/t/query-for-end-date-greater-than-now-or-doesnt-exist-not-working/310361 "2022-07-22T03:43:40Z")
My 'must' query works fine but the should returns the following error: "No query registered for \[exists\]" I want to find documents that either do not have a stopPublishDate or if they do have one the date must be in th… --- ## [Esrally benchmarking elastic cluster running on kubernetes on AWS](https://discuss.elastic.co/t/esrally-benchmarking-elastic-cluster-running-on-kubernetes-on-aws/309641)
**Author:** [@amitsa](https://discuss.elastic.co/u/amitsa)\ **Replies:** 7\ **Last updated:** [July 22, 2022, 3:10am UTC](https://discuss.elastic.co/t/esrally-benchmarking-elastic-cluster-running-on-kubernetes-on-aws/309641 "2022-07-22T03:10:38Z")
Hi All, 1 esrally pods 1 master es pod 1 data es pod All these are running on separate ec2 instances. I am using i4i.16xLarge instance type. i4i.16xLarge 64 vCPUs 512 gb ram BULK SIZE 1000 BULK INDEXING CLIENT … --- ## [Does the new elasticsearch Java API client BulkOperation, BulkRequest, BulkResponse works similarly with HLRC BulkProcessor?](https://discuss.elastic.co/t/does-the-new-elasticsearch-java-api-client-bulkoperation-bulkrequest-bulkresponse-works-similarly-with-hlrc-bulkprocessor/310356)
**Author:** [@ALX\_DM](https://discuss.elastic.co/u/ALX_DM)\ **Replies:** 2\ **Last updated:** [July 22, 2022, 2:16am UTC](https://discuss.elastic.co/t/does-the-new-elasticsearch-java-api-client-bulkoperation-bulkrequest-bulkresponse-works-similarly-with-hlrc-bulkprocessor/310356 "2022-07-22T02:16:58Z")
It was said that there is no equivalent to HLRC BulkProcessor in the new elasticsearch Java API client BUT there are these BulkOperation, BulkRequest, BulkResponse...are they similar in a way? --- ## [Search template: Only aggregate with param condition](https://discuss.elastic.co/t/search-template-only-aggregate-with-param-condition/310350)
**Author:** [@Emporea](https://discuss.elastic.co/u/Emporea)\ **Replies:** 0\ **Last updated:** [July 21, 2022, 11:00pm UTC](https://discuss.elastic.co/t/search-template-only-aggregate-with-param-condition/310350 "2022-07-21T23:00:59Z")
Maybe I am misunderstanding the obvious. But how am i able to use a search template (mustache) and only aggregate certain aggs if a param is set to true? For example I have this search template { id: "test\_template",… --- ## [How can I 'start over' from scratch with a clean installation of ES?](https://discuss.elastic.co/t/how-can-i-start-over-from-scratch-with-a-clean-installation-of-es/310349)
**Author:** [@KKora](https://discuss.elastic.co/u/KKora)\ **Replies:** 2\ **Last updated:** [July 21, 2022, 10:55pm UTC](https://discuss.elastic.co/t/how-can-i-start-over-from-scratch-with-a-clean-installation-of-es/310349 "2022-07-21T22:55:18Z")
I installed ES and deleted the Elasticsearch folder. When I 'start over' from scratch with a clean installation, It does not creating certificate files? How to clean ES installation from my machine ? --- ## [elasticsearch.AuthenticationException: AuthenticationException(401, 'security\_exception', 'unable to authenticate user \[elastic\] for REST request \[/\]')](https://discuss.elastic.co/t/elasticsearch-authenticationexception-authenticationexception-401-security-exception-unable-to-authenticate-user-elastic-for-rest-request/310353)
**Author:** [@pivotallifesciences](https://discuss.elastic.co/u/pivotallifesciences)\ **Replies:** 0\ **Last updated:** [July 21, 2022, 11:38pm UTC](https://discuss.elastic.co/t/elasticsearch-authenticationexception-authenticationexception-401-security-exception-unable-to-authenticate-user-elastic-for-rest-request/310353 "2022-07-21T23:38:52Z")
from query\_patents.patents\_query\_tools import query\_big\_query from elasticsearch import Elasticsearch, helpers import configparser config = configparser.ConfigParser() config.read('tools/patent\_scraper/google\_patents/ex… --- ## [Rolling up different status types](https://discuss.elastic.co/t/rolling-up-different-status-types/309762)
**Author:** [@Chicken\_Bake](https://discuss.elastic.co/u/Chicken_Bake)\ **Replies:** 5\ **Last updated:** [July 21, 2022, 10:49pm UTC](https://discuss.elastic.co/t/rolling-up-different-status-types/309762 "2022-07-21T22:49:53Z")
So i have a large amount of data being written to an index everyday. the main index is over written as the amount of data is quite large. The data consist of 3 fields per record and its a timestamp, status, extstatus. An… --- ## [Error fortinet module - 7.17.3](https://discuss.elastic.co/t/error-fortinet-module-7-17-3/310348)
**Author:** [@jdxuul](https://discuss.elastic.co/u/jdxuul)\ **Replies:** 1\ **Last updated:** [July 21, 2022, 10:48pm UTC](https://discuss.elastic.co/t/error-fortinet-module-7-17-3/310348 "2022-07-21T22:48:53Z")
Hi guys, Actually i tried to parse fortigate firewall logs with the filebeat module however I'm getting a strange input values into the documents. Looking the pipeline is someting related to timezone, i've tried to cha… --- ## [ELK bin modules don't work - x.pack,security issues?](https://discuss.elastic.co/t/elk-bin-modules-dont-work-x-pack-security-issues/310344)
**Author:** [@Nate\_F](https://discuss.elastic.co/u/Nate_F)\ **Replies:** 1\ **Last updated:** [July 21, 2022, 10:48pm UTC](https://discuss.elastic.co/t/elk-bin-modules-dont-work-x-pack-security-issues/310344 "2022-07-21T22:48:14Z")
I could be very wrong here as I have not deployed ELK in years. I opted to use the bitnami image to speed things up but I'm not sure if I should just scrap what I have and do the install manually. I'm trying to set perm… --- ## [Strategy for keeping multiple indices in the same cluster synchronized](https://discuss.elastic.co/t/strategy-for-keeping-multiple-indices-in-the-same-cluster-synchronized/310329)
**Author:** [@mbklein](https://discuss.elastic.co/u/mbklein)\ **Replies:** 1\ **Last updated:** [July 21, 2022, 10:44pm UTC](https://discuss.elastic.co/t/strategy-for-keeping-multiple-indices-in-the-same-cluster-synchronized/310329 "2022-07-21T22:44:44Z")
I have four indices in the same cluster (running ES v7.17, in case that matters) with different mappings and settings. For reasons that will soon become clear, we can call them v1 and v2-a, v2-b, and v2-c. v1 contains t… --- ## [Do you want live notifications when people reply to your posts? Enable Notifications Delays in updates in Opensearch](https://discuss.elastic.co/t/do-you-want-live-notifications-when-people-reply-to-your-posts-enable-notifications-delays-in-updates-in-opensearch/310342)
**Author:** [@Abdullah\_Jaffer](https://discuss.elastic.co/u/Abdullah_Jaffer)\ **Replies:** 1\ **Last updated:** [July 21, 2022, 9:43pm UTC](https://discuss.elastic.co/t/do-you-want-live-notifications-when-people-reply-to-your-posts-enable-notifications-delays-in-updates-in-opensearch/310342 "2022-07-21T21:43:41Z")
We are using AWS Opensearch 1.0 as a document database to serve our retail users. We use Apache Kafka to sync Opensearch. This use-case requires Opensearch documents to be refreshed near real time for searching. We have … --- ## [Convert/Ingest Docker container Environment Variables into Filebeat/Logstash fields](https://discuss.elastic.co/t/convert-ingest-docker-container-environment-variables-into-filebeat-logstash-fields/310340)
**Author:** [@Julian\_Barnett](https://discuss.elastic.co/u/Julian_Barnett)\ **Replies:** 0\ **Last updated:** [July 21, 2022, 9:17pm UTC](https://discuss.elastic.co/t/convert-ingest-docker-container-environment-variables-into-filebeat-logstash-fields/310340 "2022-07-21T21:17:04Z")
I'm currently using filebeat 8.3.0 and we're copying a few container LABELS to fields via something like this: - copy\_fields: fields: - from: container.labels.custom\_label to: custom\_label We ha… --- ## [Logstash Elapsed filter not working](https://discuss.elastic.co/t/logstash-elapsed-filter-not-working/310220)
**Author:** [@nakula](https://discuss.elastic.co/u/nakula)\ **Replies:** 8\ **Last updated:** [July 21, 2022, 8:02pm UTC](https://discuss.elastic.co/t/logstash-elapsed-filter-not-working/310220 "2022-07-21T20:02:37Z")
I am trying to find the different between the start time and end time of a transaction. I am using the sample log file as below.. 2022-07-17T12:36:30.3081415Z Info 38 \[xxx.frame.logging.serviceCols\] \["Level3"\] : Wires-… --- ## [Elastic Agent Error](https://discuss.elastic.co/t/elastic-agent-error/310248)
**Author:** [@suchandinfo](https://discuss.elastic.co/u/suchandinfo)\ **Replies:** 1\ **Last updated:** [July 21, 2022, 7:18pm UTC](https://discuss.elastic.co/t/elastic-agent-error/310248 "2022-07-21T19:18:41Z")
Hi, Please help me to resolve the below error. I have installed elastic agent in my linux server and trying to send logs to elastic.cloud. I am getting below error from linux machine. 2022-07-21T08:24:07.506Z ER… --- ## [Kibana alert](https://discuss.elastic.co/t/kibana-alert/309415)
**Author:** [@Jathurshan\_Sumandira](https://discuss.elastic.co/u/Jathurshan_Sumandira)\ **Replies:** 0\ **Last updated:** [July 12, 2022, 12:19pm UTC](https://discuss.elastic.co/t/kibana-alert/309415 "2022-07-12T12:19:54Z")
Hi Team, I am using Kibana alert rules and I used Elasticsearch query type. { "query": { "bool": { "filter": { "script": { "script": " String alert = 'No alert'; if(doc.containsKey('startTime') && doc\['startTime'… --- ## [Winlogbeat Service Still Running but Security Log Events No Longer Published After Windows Auto-Archives Event Log- Sometimes](https://discuss.elastic.co/t/winlogbeat-service-still-running-but-security-log-events-no-longer-published-after-windows-auto-archives-event-log-sometimes/310325)
**Author:** [@eafrost.cissp](https://discuss.elastic.co/u/eafrost.cissp)\ **Replies:** 1\ **Last updated:** [July 21, 2022, 6:44pm UTC](https://discuss.elastic.co/t/winlogbeat-service-still-running-but-security-log-events-no-longer-published-after-windows-auto-archives-event-log-sometimes/310325 "2022-07-21T18:44:03Z")
I'm running Elastic Stack v8.2.0 on Windows. I've noticed that sometimes/randomly winlogbeat stops publishing events when the Windows event log fills up and auto-archives itself. The winlogbeat service is still running b… --- ## [Install Fleet server](https://discuss.elastic.co/t/install-fleet-server/310074)
**Author:** [@Hani\_Atalla](https://discuss.elastic.co/u/Hani_Atalla)\ **Replies:** 3\ **Last updated:** [July 21, 2022, 6:19pm UTC](https://discuss.elastic.co/t/install-fleet-server/310074 "2022-07-21T18:19:44Z")
I am trying to install a Fleet server v 8.3.3 but not having much luck. I tried both the "quick" and "advanced" setups and same error related to certificate. Some notes: Elasticsearch 8.3. 3 is running on host X and o… --- ## [Setting Default Time Window Dynamically](https://discuss.elastic.co/t/setting-default-time-window-dynamically/310203)
**Author:** [@badmire](https://discuss.elastic.co/u/badmire)\ **Replies:** 2\ **Last updated:** [July 21, 2022, 6:17pm UTC](https://discuss.elastic.co/t/setting-default-time-window-dynamically/310203 "2022-07-21T18:17:50Z")
Hello! I am trying to build a dashboard that will act a lot like the discover tab, but offer visualizations and some shortcuts for our less tech savvy users. I have the setting for "save time with dashboard" set to tru… --- ## [Aggregation with script](https://discuss.elastic.co/t/aggregation-with-script/310216)
**Author:** [@Jason\_Yu1](https://discuss.elastic.co/u/Jason_Yu1)\ **Replies:** 4\ **Last updated:** [July 21, 2022, 5:53pm UTC](https://discuss.elastic.co/t/aggregation-with-script/310216 "2022-07-21T17:53:56Z")
I have some containers like those "\_source": { "testId": "test001", "containers": \[ { "name": "containers1", … --- ## [Parsing custom log timestamps, how?](https://discuss.elastic.co/t/parsing-custom-log-timestamps-how/310205)
**Author:** [@jerrac](https://discuss.elastic.co/u/jerrac)\ **Replies:** 3\ **Last updated:** [July 21, 2022, 5:41pm UTC](https://discuss.elastic.co/t/parsing-custom-log-timestamps-how/310205 "2022-07-21T17:41:58Z")
I'm having issues figuring out how to get a timestamp out of a custom log. I've been trying to use the dissect and timestamp processors via the Custom configurations field in the fleet policy -\> custom log screen. Here… --- ## [Trying to get the Force Direct Graph from Vega examples to work in Cabana](https://discuss.elastic.co/t/trying-to-get-the-force-direct-graph-from-vega-examples-to-work-in-cabana/310320)
**Author:** [@petersm05](https://discuss.elastic.co/u/petersm05)\ **Replies:** 2\ **Last updated:** [July 21, 2022, 5:25pm UTC](https://discuss.elastic.co/t/trying-to-get-the-force-direct-graph-from-vega-examples-to-work-in-cabana/310320 "2022-07-21T17:25:10Z")
Hi, I'm trying to get the Force Direct Graph example to work in Kibana, but I get "node not found: 27955" From what I can see the node is present in the node-data. Spec: { "$schema": "https://vega.github.io/schema/ve… --- ## [Jaccard similarities](https://discuss.elastic.co/t/jaccard-similarities/310324)
**Author:** [@vsaraph](https://discuss.elastic.co/u/vsaraph)\ **Replies:** 0\ **Last updated:** [July 21, 2022, 4:58pm UTC](https://discuss.elastic.co/t/jaccard-similarities/310324 "2022-07-21T16:58:20Z")
I have an index, say attributes, whose documents all have a field, say items, which is an array of strings. I want to be able to take an array of strings, and write an elasticsearch query that gives me back all documents… --- ## [Index is empty using Transform](https://discuss.elastic.co/t/index-is-empty-using-transform/310181)
**Author:** [@Ruben\_Marinho](https://discuss.elastic.co/u/Ruben_Marinho)\ **Replies:** 4\ **Last updated:** [July 21, 2022, 4:33pm UTC](https://discuss.elastic.co/t/index-is-empty-using-transform/310181 "2022-07-21T16:33:16Z")
Hi all, I'm trying to create a transform on ElasticSearch. I started by creating this using console and got the expected results. In the console I use: POST \_transform/\_preview { "source": { "index": "test", … --- ## [Discover UI slow after upgrade](https://discuss.elastic.co/t/discover-ui-slow-after-upgrade/310211)
**Author:** [@Brian\_Turnbull](https://discuss.elastic.co/u/Brian_Turnbull)\ **Replies:** 5\ **Last updated:** [July 21, 2022, 4:19pm UTC](https://discuss.elastic.co/t/discover-ui-slow-after-upgrade/310211 "2022-07-21T16:19:53Z")
We have a ~100TB logs cluster with 26 nodes that currently has about 20k shards in total. We had previously been running 7.6 and upgraded to 7.17.4. We're running into problems with very slow initial load and refresh. … --- ## [How to add new column in the datatable](https://discuss.elastic.co/t/how-to-add-new-column-in-the-datatable/310172)
**Author:** [@malik123](https://discuss.elastic.co/u/malik123)\ **Replies:** 5\ **Last updated:** [July 21, 2022, 4:13pm UTC](https://discuss.elastic.co/t/how-to-add-new-column-in-the-datatable/310172 "2022-07-21T16:13:03Z")
Hi all, Currently I am working on the sample data that is provided by the kibana. I want to display a new column in the datatable named "total\_Hours". I want to add "Unique Visits (Total)" and "Unique Visits (Last Hour… --- ## [CSP blocking Kibana](https://discuss.elastic.co/t/csp-blocking-kibana/310314)
**Author:** [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\ **Replies:** 3\ **Last updated:** [July 21, 2022, 4:04pm UTC](https://discuss.elastic.co/t/csp-blocking-kibana/310314 "2022-07-21T16:04:30Z")
Kibana now has experimental support for a more restrictive Content Security Policy(CSP). Hi, how can I disable it? Since update to 8.3 I have problems with view Kibana because of CSP blocking content in new Chrome. Ple… --- ## [Manipulating data in logstash](https://discuss.elastic.co/t/manipulating-data-in-logstash/310304)
**Author:** [@KeithL](https://discuss.elastic.co/u/KeithL)\ **Replies:** 1\ **Last updated:** [July 21, 2022, 2:47pm UTC](https://discuss.elastic.co/t/manipulating-data-in-logstash/310304 "2022-07-21T14:47:17Z")
Beginner's question; pardon my potentially poor terminology. I'd like to manipulate ingested data to include geo\_point data to facilitate visualisation in elastic/kibana e.g given this: { "Entities" =\> \[ \[… --- ## [Input beats - output syslog, what in the middle](https://discuss.elastic.co/t/input-beats-output-syslog-what-in-the-middle/310310)
**Author:** [@Kakos](https://discuss.elastic.co/u/Kakos)\ **Replies:** 0\ **Last updated:** [July 21, 2022, 2:40pm UTC](https://discuss.elastic.co/t/input-beats-output-syslog-what-in-the-middle/310310 "2022-07-21T14:40:43Z")
Hi, The messages i get from beats are completely unstructured. The most important of all they include characters like \\t and \\n and actually don't recognize the new lines and tabs which exist in the raw data. So they lo… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=854) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=856)