# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=201 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 202 --- ## [\[Kibana API\] Cannot list rules using Kibana API, successfull response, empty data](https://discuss.elastic.co/t/kibana-api-cannot-list-rules-using-kibana-api-successfull-response-empty-data/363414)
**Author:** [@wocek](https://discuss.elastic.co/u/wocek)\ **Replies:** 0\ **Last updated:** [July 19, 2024, 10:36am UTC](https://discuss.elastic.co/t/kibana-api-cannot-list-rules-using-kibana-api-successfull-response-empty-data/363414 "2024-07-19T10:36:44Z")
Hi all! I'm having a peculiar problem with Kibana API. I'm writing a powershell script to manage my rules/alerts. Here's the problem: I want to hit this endpoint /api/alerting/rules/\_find to retrieve list of rules tha… --- ## [Merging the data column wise from two different indexes](https://discuss.elastic.co/t/merging-the-data-column-wise-from-two-different-indexes/363296)
**Author:** [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)\ **Replies:** 6\ **Last updated:** [July 19, 2024, 10:07am UTC](https://discuss.elastic.co/t/merging-the-data-column-wise-from-two-different-indexes/363296 "2024-07-19T10:07:00Z")
Hi, I am trying to merge two indexes column wise based on a common key from both the indices. I have tried logstash ruby code, transforms but nothing seems working to me. I am getting individual records having the colum… --- ## [Java API Client Conflict? java.lang.NoSuchMethodError: org.elasticsearch.client.RequestOptions$Builder.setHttpAsyncResponseConsumerFactory](https://discuss.elastic.co/t/java-api-client-conflict-java-lang-nosuchmethoderror-org-elasticsearch-client-requestoptions-builder-sethttpasyncresponseconsumerfactory/363367)
**Author:** [@linusField](https://discuss.elastic.co/u/linusField)\ **Replies:** 1\ **Last updated:** [July 19, 2024, 9:22am UTC](https://discuss.elastic.co/t/java-api-client-conflict-java-lang-nosuchmethoderror-org-elasticsearch-client-requestoptions-builder-sethttpasyncresponseconsumerfactory/363367 "2024-07-19T09:22:08Z")
Hi! We are trying to upgrade from Elastic Search 7 to 8. We have been using the Rest High Level Client 7.17.22 and want to upgrade it to 8.14.3. As suggested by the migration guide, we should be able to have both vers… --- ## [Configuration Does not Work (xpack.reporting.csv.useByteOrderMarkEncoding)](https://discuss.elastic.co/t/configuration-does-not-work-xpack-reporting-csv-usebyteordermarkencoding/363407)
**Author:** [@Faker](https://discuss.elastic.co/u/Faker)\ **Replies:** 0\ **Last updated:** [July 19, 2024, 8:17am UTC](https://discuss.elastic.co/t/configuration-does-not-work-xpack-reporting-csv-usebyteordermarkencoding/363407 "2024-07-19T08:17:22Z")
ES and Kibana version : 8.11.1 my kibana.yml x-pack.reporting xpack.reporting.enabled: true xpack.reporting.encryptionKey: "something\_secret\_i\_set" xpack.reporting.csv.checkForFormulas: true xpack.reporting.csv.esca… --- ## [Is it possible to forward logs from elasticsearch to qradar](https://discuss.elastic.co/t/is-it-possible-to-forward-logs-from-elasticsearch-to-qradar/362769)
**Author:** [@yar](https://discuss.elastic.co/u/yar)\ **Replies:** 3\ **Last updated:** [July 19, 2024, 7:51am UTC](https://discuss.elastic.co/t/is-it-possible-to-forward-logs-from-elasticsearch-to-qradar/362769 "2024-07-19T07:51:40Z")
Hi, I have deployed ELK stack into a couple of VMs. The logs are gathered using fleet agents, and they're being sent into elastic. Is there a way to forward the logs from Elasticsearch to Qradar on-prem deployment? Or s… --- ## [Extract nested documents to new index(non-nestify)](https://discuss.elastic.co/t/extract-nested-documents-to-new-index-non-nestify/363401)
**Author:** [@HyebinHong](https://discuss.elastic.co/u/HyebinHong)\ **Replies:** 0\ **Last updated:** [July 19, 2024, 7:46am UTC](https://discuss.elastic.co/t/extract-nested-documents-to-new-index-non-nestify/363401 "2024-07-19T07:46:39Z")
Hello! I'm wondering I can extract nested documents for the new index. For example, if there is an index of company which also containing products data as a nested field, i want to make a new product index from the fo… --- ## [Need Help with Merging Data from Two Elasticsearch Indices](https://discuss.elastic.co/t/need-help-with-merging-data-from-two-elasticsearch-indices/363343)
**Author:** [@prashant1](https://discuss.elastic.co/u/prashant1)\ **Replies:** 4\ **Last updated:** [July 19, 2024, 7:32am UTC](https://discuss.elastic.co/t/need-help-with-merging-data-from-two-elasticsearch-indices/363343 "2024-07-19T07:32:15Z")
Hi, I need to merge data from two indices (test1 and test2) based on a common key (information). Here’s a simplified example of my indices: Index test1: sample\_id information price --------- ------- ---… --- ## [Filebeat output using SAS token to azure eventhub](https://discuss.elastic.co/t/filebeat-output-using-sas-token-to-azure-eventhub/363400)
**Author:** [@Bhrugu\_Sharma](https://discuss.elastic.co/u/Bhrugu_Sharma)\ **Replies:** 0\ **Last updated:** [July 19, 2024, 7:20am UTC](https://discuss.elastic.co/t/filebeat-output-using-sas-token-to-azure-eventhub/363400 "2024-07-19T07:20:33Z")
I want to send my filebeat logs to eventhubs using SAS token, earlier we wanted to use Kafka output but we came to know Kafka doesn't support SAS tokens, so now is there some way we can send data using SAS token, for exa… --- ## [Deprecation logging -- logger level](https://discuss.elastic.co/t/deprecation-logging-logger-level/363387)
**Author:** [@Edmund\_Cheng](https://discuss.elastic.co/u/Edmund_Cheng)\ **Replies:** 2\ **Last updated:** [July 19, 2024, 6:52am UTC](https://discuss.elastic.co/t/deprecation-logging-logger-level/363387 "2024-07-19T06:52:10Z")
Hi, Just wanted to understand if there is any difference in terms of the message content if the level is set to WARN or INFO for deprecation logging? For e.g { "persistent": { "logger.org.elasticsearch.deprecati… --- ## [How to Remove Prefixes and Ensure Parsing Integrity When Forwarding Logs to QRadar Using Logstash](https://discuss.elastic.co/t/how-to-remove-prefixes-and-ensure-parsing-integrity-when-forwarding-logs-to-qradar-using-logstash/363385)
**Author:** [@wangsubo](https://discuss.elastic.co/u/wangsubo)\ **Replies:** 2\ **Last updated:** [July 19, 2024, 6:34am UTC](https://discuss.elastic.co/t/how-to-remove-prefixes-and-ensure-parsing-integrity-when-forwarding-logs-to-qradar-using-logstash/363385 "2024-07-19T06:34:41Z")
I have two questions that need answers: When forwarding logs from Logstash to QRadar, the logs come with a Logstash prefix. How can we remove this prefix? Using Elastic-Agent to send logs to Elasticsearch allows au… --- ## [Logstash related nested json help](https://discuss.elastic.co/t/logstash-related-nested-json-help/363382)
**Author:** [@jgisler](https://discuss.elastic.co/u/jgisler)\ **Replies:** 6\ **Last updated:** [July 19, 2024, 5:27am UTC](https://discuss.elastic.co/t/logstash-related-nested-json-help/363382 "2024-07-19T05:27:06Z")
Hello Guru's, I have never ingested nested json before and I'm stuck. Log examples: {"system":"aa","logLevel":"\[INFO\]","log":{"0":"TT000000 joining provisioningEvent and will publish topics: provisioningEvent"}} {"sys… --- ## [Stream log4j2 logs to logstash hosted on an azure VM via tcp](https://discuss.elastic.co/t/stream-log4j2-logs-to-logstash-hosted-on-an-azure-vm-via-tcp/363377)
**Author:** [@asandhu](https://discuss.elastic.co/u/asandhu)\ **Replies:** 3\ **Last updated:** [July 18, 2024, 11:38pm UTC](https://discuss.elastic.co/t/stream-log4j2-logs-to-logstash-hosted-on-an-azure-vm-via-tcp/363377 "2024-07-18T23:38:56Z")
Hi All I'm a bit new to the ELK stack so greatly appreciate any help I have a mulesoft application on my local machine (looking to host the mule application on anypoint platform later) whose logs I am trying to send to… --- ## [Using custom ingest pipeline for AWS integration](https://discuss.elastic.co/t/using-custom-ingest-pipeline-for-aws-integration/363381)
**Author:** [@shiftey](https://discuss.elastic.co/u/shiftey)\ **Replies:** 0\ **Last updated:** [July 18, 2024, 11:06pm UTC](https://discuss.elastic.co/t/using-custom-ingest-pipeline-for-aws-integration/363381 "2024-07-18T23:06:07Z")
Hi there, Im fairly new to Elastic/Kibana, please bear with me. We use Elastic v7.17. I've setup an AWS Cloudwatch integration to collect logs from a custom app which logs to Cloudwatch. The log is unstructured, so I h… --- ## [Image Panel in Kibana](https://discuss.elastic.co/t/image-panel-in-kibana/363371)
**Author:** [@erikg](https://discuss.elastic.co/u/erikg)\ **Replies:** 0\ **Last updated:** [July 18, 2024, 6:44pm UTC](https://discuss.elastic.co/t/image-panel-in-kibana/363371 "2024-07-18T18:44:54Z")
Hello I uploaded an image with the background so it can match the space color, How do I fix this? --- ## [Can you please help me to move indexes from test VPS to server?](https://discuss.elastic.co/t/can-you-please-help-me-to-move-indexes-from-test-vps-to-server/363370)
**Author:** [@filmovtv](https://discuss.elastic.co/u/filmovtv)\ **Replies:** 0\ **Last updated:** [July 18, 2024, 6:09pm UTC](https://discuss.elastic.co/t/can-you-please-help-me-to-move-indexes-from-test-vps-to-server/363370 "2024-07-18T18:09:21Z")
Hello. Don't judge me harshly, I'm new to this business, for 5 days I can't transfer indexes from the test VPS to the server. I prepared and registered a repository and create a snapshot of all indexes on VPS here: /mo… --- ## [Splitting Records With Start and End Times Into Separate Documents With A Timestamp Every X Minutes](https://discuss.elastic.co/t/splitting-records-with-start-and-end-times-into-separate-documents-with-a-timestamp-every-x-minutes/360997)
**Author:** [@nickbarry](https://discuss.elastic.co/u/nickbarry)\ **Replies:** 6\ **Last updated:** [July 18, 2024, 3:59pm UTC](https://discuss.elastic.co/t/splitting-records-with-start-and-end-times-into-separate-documents-with-a-timestamp-every-x-minutes/360997 "2024-07-18T15:59:26Z")
I'm currently attempting to split a single activity SQL record with a start and end time into multiple documents in an index. Each document will have a single timestamp (and duplicated data fields) and each timestamp is… --- ## [Access field that was excluded from \_source](https://discuss.elastic.co/t/access-field-that-was-excluded-from-source/363329)
**Author:** [@Chenko](https://discuss.elastic.co/u/Chenko)\ **Replies:** 6\ **Last updated:** [July 18, 2024, 2:17pm UTC](https://discuss.elastic.co/t/access-field-that-was-excluded-from-source/363329 "2024-07-18T14:17:50Z")
Hi, As mentioned in the title, I have an index with a few fields that are not included in the \_source field. I was wondering, since we can still search on these fields, is there a way to access their values? One solut… --- ## [Recommended shard quantity](https://discuss.elastic.co/t/recommended-shard-quantity/363298)
**Author:** [@Fernando\_Oliveira](https://discuss.elastic.co/u/Fernando_Oliveira)\ **Replies:** 2\ **Last updated:** [July 18, 2024, 2:02pm UTC](https://discuss.elastic.co/t/recommended-shard-quantity/363298 "2024-07-18T14:02:07Z")
Hello. I created a cluster following some practices for redundancy : 3 - node master 2 - node hot 2 - node warm What would be the recommended amount of shard for my index? --- ## [Setup small elasticsearch cluster](https://discuss.elastic.co/t/setup-small-elasticsearch-cluster/363310)
**Author:** [@vijay\_kaali](https://discuss.elastic.co/u/vijay_kaali)\ **Replies:** 2\ **Last updated:** [July 18, 2024, 1:39pm UTC](https://discuss.elastic.co/t/setup-small-elasticsearch-cluster/363310 "2024-07-18T13:39:35Z")
Hi i would like to install a small elasticsearch for monitoring 30 servers . have few questions how many elasticsearch nodes needed \[2 is fine?\] if availability not an issue ? how much memory/core do we need to start … --- ## [DataStream creates hidden indicas](https://discuss.elastic.co/t/datastream-creates-hidden-indicas/363347)
**Author:** [@Jeyhun\_Rahimov](https://discuss.elastic.co/u/Jeyhun_Rahimov)\ **Replies:** 3\ **Last updated:** [July 18, 2024, 1:00pm UTC](https://discuss.elastic.co/t/datastream-creates-hidden-indicas/363347 "2024-07-18T13:00:13Z")
I use Elastic.Serilog.Sinks lib to send logs to Elastic with Serilog. This code sends logs to ES, but the created indexes are hidden. Why? How can I set the index unhidden? How can I set the index format? public sta… --- ## [Elasticsearch failed to start when running with PVC in OKE](https://discuss.elastic.co/t/elasticsearch-failed-to-start-when-running-with-pvc-in-oke/363340)
**Author:** [@Praveen\_Reddy](https://discuss.elastic.co/u/Praveen_Reddy)\ **Replies:** 2\ **Last updated:** [July 18, 2024, 1:00pm UTC](https://discuss.elastic.co/t/elasticsearch-failed-to-start-when-running-with-pvc-in-oke/363340 "2024-07-18T13:00:00Z")
Elastic failing to start with below error. When i don't link any mount points then it is working. But when i link with PVC it fails to start. Looks like it is related to permission issue. "failed to obtain node locks, t… --- ## [Delete Old Log elastic search](https://discuss.elastic.co/t/delete-old-log-elastic-search/363314)
**Author:** [@Agaaam](https://discuss.elastic.co/u/Agaaam)\ **Replies:** 4\ **Last updated:** [July 18, 2024, 12:56pm UTC](https://discuss.elastic.co/t/delete-old-log-elastic-search/363314 "2024-07-18T12:56:16Z")
Hallo, Currently im using Elastic Kibana Filebeat and logstash to colellect log accross server and docker , and suddenly the drive got too bigs and take so much disk space, what can i do to delete 3 old month logs ? t… --- ## [Copy routing info from old index to new index](https://discuss.elastic.co/t/copy-routing-info-from-old-index-to-new-index/363320)
**Author:** [@abhadauria](https://discuss.elastic.co/u/abhadauria)\ **Replies:** 3\ **Last updated:** [July 18, 2024, 11:33am UTC](https://discuss.elastic.co/t/copy-routing-info-from-old-index-to-new-index/363320 "2024-07-18T11:33:43Z")
Hi Team, We want to reindex the data from older version of index(6.x) to new version(7.x) We have created new index using the same settings and mapping in ES cluster 7.17, tried to initiate re-indexing using below requ… --- ## [Drop\_event processor in Powershell Channels](https://discuss.elastic.co/t/drop-event-processor-in-powershell-channels/362377)
**Author:** [@fsch](https://discuss.elastic.co/u/fsch)\ **Replies:** 3\ **Last updated:** [July 18, 2024, 11:29am UTC](https://discuss.elastic.co/t/drop-event-processor-in-powershell-channels/362377 "2024-07-18T11:29:10Z")
Hello We’ve recently rolled out the Elastic Agent on our Windows servers and since our main focus lies on building rules based on powershell events we would like to filter out as much of those events as possible before … --- ## [AI Assistant not being set up](https://discuss.elastic.co/t/ai-assistant-not-being-set-up/363325)
**Author:** [@mansibora](https://discuss.elastic.co/u/mansibora)\ **Replies:** 1\ **Last updated:** [July 18, 2024, 9:24am UTC](https://discuss.elastic.co/t/ai-assistant-not-being-set-up/363325 "2024-07-18T09:24:36Z")
The knowledge base is not getting installed, gives error and even the AI Assistant is giving error. I have even added the secret key for my GPT-4 OpenAI in the Connector. --- ## [Split chart in lens](https://discuss.elastic.co/t/split-chart-in-lens/363336)
**Author:** [@intellion](https://discuss.elastic.co/u/intellion)\ **Replies:** 1\ **Last updated:** [July 18, 2024, 9:20am UTC](https://discuss.elastic.co/t/split-chart-in-lens/363336 "2024-07-18T09:20:04Z")
The already existing topic to this request was closed 4 years ago. Any update on whether splitting a chart in lens is now possible? --- ## [Kibana Codebase Local Setup](https://discuss.elastic.co/t/kibana-codebase-local-setup/363192)
**Author:** [@Priyanka\_Narne](https://discuss.elastic.co/u/Priyanka_Narne)\ **Replies:** 3\ **Last updated:** [July 18, 2024, 9:14am UTC](https://discuss.elastic.co/t/kibana-codebase-local-setup/363192 "2024-07-18T09:14:57Z")
I am setting the Kibana codebase locally. I followed the steps given in https://www.elastic.co/guide/en/kibana/current/development-getting-started.html When I run the command yarn kbn bootstrap, I am getting errors. … --- ## [Filebeat multiline how to parse both structured and unstructured logs in a file?](https://discuss.elastic.co/t/filebeat-multiline-how-to-parse-both-structured-and-unstructured-logs-in-a-file/363335)
**Author:** [@Tom\_N](https://discuss.elastic.co/u/Tom_N)\ **Replies:** 0\ **Last updated:** [July 18, 2024, 9:03am UTC](https://discuss.elastic.co/t/filebeat-multiline-how-to-parse-both-structured-and-unstructured-logs-in-a-file/363335 "2024-07-18T09:03:45Z")
I am currently trying to use multiline to parse my files. They are inputted into filebeat using filestream. The logs in the file follow a structured format, except for the first log and last log \[2015-08-24 11:49:14,38… --- ## [Kibana from 7.17 to 8.11and from 8.11 to 8.13 version upgraded but kibana system indices are not created ilm policy and rollover\_alias properly so facing below error](https://discuss.elastic.co/t/kibana-from-7-17-to-8-11and-from-8-11-to-8-13-version-upgraded-but-kibana-system-indices-are-not-created-ilm-policy-and-rollover-alias-properly-so-facing-below-error/363332)
**Author:** [@priyanka\_sn](https://discuss.elastic.co/u/priyanka_sn)\ **Replies:** 0\ **Last updated:** [July 18, 2024, 8:29am UTC](https://discuss.elastic.co/t/kibana-from-7-17-to-8-11and-from-8-11-to-8-13-version-upgraded-but-kibana-system-indices-are-not-created-ilm-policy-and-rollover-alias-properly-so-facing-below-error/363332 "2024-07-18T08:29:32Z")
Dear All, I am facing below error in 8.11 and 8.13 version of elasticsearch Elasticsearch.log file continuously. Please help me to solve this issue. I think this is system index restored\_kibana-event-log-ds. what is th… --- ## [Update the json input file in Logstash](https://discuss.elastic.co/t/update-the-json-input-file-in-logstash/363331)
**Author:** [@mario\_kazela](https://discuss.elastic.co/u/mario_kazela)\ **Replies:** 0\ **Last updated:** [July 18, 2024, 8:18am UTC](https://discuss.elastic.co/t/update-the-json-input-file-in-logstash/363331 "2024-07-18T08:18:34Z")
Hi, I just want to asking, I has a json file which contains of several fields likes: { "ID": 12345 "title": "Example Post", "views\_count": 881, "author": "John Doe", "content": "This is an example post." } … [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=200) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=202)