# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=366 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 367 --- ## [Logstash is processing old documents](https://discuss.elastic.co/t/logstash-is-processing-old-documents/347678)
**Author:** [@Cruz](https://discuss.elastic.co/u/Cruz)\ **Replies:** 3\ **Last updated:** [November 24, 2023, 9:12am UTC](https://discuss.elastic.co/t/logstash-is-processing-old-documents/347678 "2023-11-24T09:12:38Z")
When I restart the logstash service, the old documents are coming out. I tried to stopping the filebeat service where the logs are coming from and I deleted the old documents. But when I restart the logstash service the… --- ## [What does "\_ignored" tag mean in hits](https://discuss.elastic.co/t/what-does-ignored-tag-mean-in-hits/344300)
**Author:** [@Aiswarya\_S](https://discuss.elastic.co/u/Aiswarya_S)\ **Replies:** 2\ **Last updated:** [November 24, 2023, 7:19am UTC](https://discuss.elastic.co/t/what-does-ignored-tag-mean-in-hits/344300 "2023-11-24T07:19:49Z")
In my Elastic search pulled data, I am getting an ignored tag in the hits but yet the data is coming correctly... so what does that ignored tag mean? { "took": 9, "timed\_out": false, "\_shards": { "total": 1, … --- ## [Vault Logging using Elasticsearch](https://discuss.elastic.co/t/vault-logging-using-elasticsearch/347897)
**Author:** [@VijayIQA](https://discuss.elastic.co/u/VijayIQA)\ **Replies:** 0\ **Last updated:** [November 24, 2023, 5:17am UTC](https://discuss.elastic.co/t/vault-logging-using-elasticsearch/347897 "2023-11-24T05:17:42Z")
Hi team, As per elastic docs at vault audit enable socket address=${ELASTIC\_AGENT\_IP}:9007 socket\_type=tcp In the place of ELASTIC\_AGENT\_IP I placed Elasticsearch IP and port as 9200 in this case getting an error as … --- ## [Elasticsearch CPU usage](https://discuss.elastic.co/t/elasticsearch-cpu-usage/347689)
**Author:** [@VijayIQA](https://discuss.elastic.co/u/VijayIQA)\ **Replies:** 4\ **Last updated:** [November 24, 2023, 3:10am UTC](https://discuss.elastic.co/t/elasticsearch-cpu-usage/347689 "2023-11-24T03:10:17Z")
Hi Team, Cluster monitoring by Kibana stack monitoring in that able to get all parameters but not getting CPU usages of the nodes. --- ## [How to use LruRedux cache in ruby filter](https://discuss.elastic.co/t/how-to-use-lruredux-cache-in-ruby-filter/347893)
**Author:** [@Chen\_Wei](https://discuss.elastic.co/u/Chen_Wei)\ **Replies:** 0\ **Last updated:** [November 24, 2023, 2:44am UTC](https://discuss.elastic.co/t/how-to-use-lruredux-cache-in-ruby-filter/347893 "2023-11-24T02:44:03Z")
Somehow we have some logs having duplicated events, we want to dedup the events using fingerprint and LRU cache in the logstash pipeline, So I write a ruby file require "lru\_redux" def register(params) limit = para… --- ## [Index has disappeared](https://discuss.elastic.co/t/index-has-disappeared/347889)
**Author:** [@juancamiloll](https://discuss.elastic.co/u/juancamiloll)\ **Replies:** 5\ **Last updated:** [November 23, 2023, 10:47pm UTC](https://discuss.elastic.co/t/index-has-disappeared/347889 "2023-11-23T22:47:53Z")
Hello I have several sources that ELK processes, as you know from /etc/logstash/conf.d a .conf file is created for each of the sources to be processed either by GROK or CSV, I don't know if there is another way. One of… --- ## [Time fields show different time](https://discuss.elastic.co/t/time-fields-show-different-time/346651)
**Author:** [@juancamiloll](https://discuss.elastic.co/u/juancamiloll)\ **Replies:** 8\ **Last updated:** [November 23, 2023, 10:25pm UTC](https://discuss.elastic.co/t/time-fields-show-different-time/346651 "2023-11-23T22:25:01Z")
Hello again, I find a new problem where in the logs of a Paloalto I see that the "ReceivedTime" field and the "column103" field show a different time. I would appreciate your help input { file { path =\> "… --- ## [Logstash Multiline and line codec differences](https://discuss.elastic.co/t/logstash-multiline-and-line-codec-differences/347466)
**Author:** [@randomnamegenerator](https://discuss.elastic.co/u/randomnamegenerator)\ **Replies:** 6\ **Last updated:** [November 23, 2023, 7:42pm UTC](https://discuss.elastic.co/t/logstash-multiline-and-line-codec-differences/347466 "2023-11-23T19:42:09Z")
Hello All, We have application logs coming in from a number of different hosts (shipped with filebeat) and have obvserved a mixing of datastreams for one of the log types. We changed the logstash input.config from vers… --- ## [How to namespace indexes - Automatic not Manual](https://discuss.elastic.co/t/how-to-namespace-indexes-automatic-not-manual/347886)
**Author:** [@Alexander\_Mills](https://discuss.elastic.co/u/Alexander_Mills)\ **Replies:** 1\ **Last updated:** [November 23, 2023, 7:33pm UTC](https://discuss.elastic.co/t/how-to-namespace-indexes-automatic-not-manual/347886 "2023-11-23T19:33:31Z")
Mongo has namespacing via different databases on the same db server RabbitMQ has namespacing via different exhanges How can I automatically namespace indices with Elastic without manually namespacing keys with prod-x… --- ## [Setting Up Logstash In Docker-Compose For Bulk Ingest Of CSV Files In Local Machine](https://discuss.elastic.co/t/setting-up-logstash-in-docker-compose-for-bulk-ingest-of-csv-files-in-local-machine/346916)
**Author:** [@Ethan777100](https://discuss.elastic.co/u/Ethan777100)\ **Replies:** 112\ **Last updated:** [November 23, 2023, 5:55pm UTC](https://discuss.elastic.co/t/setting-up-logstash-in-docker-compose-for-bulk-ingest-of-csv-files-in-local-machine/346916 "2023-11-23T17:55:14Z")
CONTINUATION FROM kibana-8-11-0-failed-to-start-exit-code-1 My use case is to bulk ingest csv files into Elasticsearch. Understand i need Logstash to do it. Not sure how to start. Should I be using a default or cus… --- ## [Kubernetes Heartbeat autodiscover not working](https://discuss.elastic.co/t/kubernetes-heartbeat-autodiscover-not-working/347867)
**Author:** [@Paul\_B](https://discuss.elastic.co/u/Paul_B)\ **Replies:** 5\ **Last updated:** [November 23, 2023, 6:41pm UTC](https://discuss.elastic.co/t/kubernetes-heartbeat-autodiscover-not-working/347867 "2023-11-23T18:41:49Z")
Hi all, I've used the Kubernetes manifest file from this part of the documentation - Running Heartbeat on Kubernetes | Heartbeat Reference \[8.11\] | Elastic The deployment is working and the standalone monitors I've cre… --- ## [Duplicate logs in Logstash](https://discuss.elastic.co/t/duplicate-logs-in-logstash/347630)
**Author:** [@marcowiskhy](https://discuss.elastic.co/u/marcowiskhy)\ **Replies:** 8\ **Last updated:** [November 23, 2023, 6:15pm UTC](https://discuss.elastic.co/t/duplicate-logs-in-logstash/347630 "2023-11-23T18:15:47Z")
I collect VPN logs through Logstash and index them in Elasticsearch, but I'm having the following problem: For each unique VPN connection (represented by TunnelID), there should be only one tunnel-up event and one tunne… --- ## [The Output Isolator Pattern: Inquiry regarding downstream pipeline failures](https://discuss.elastic.co/t/the-output-isolator-pattern-inquiry-regarding-downstream-pipeline-failures/347878)
**Author:** [@Kihyun\_Hwang](https://discuss.elastic.co/u/Kihyun_Hwang)\ **Replies:** 2\ **Last updated:** [November 23, 2023, 5:14pm UTC](https://discuss.elastic.co/t/the-output-isolator-pattern-inquiry-regarding-downstream-pipeline-failures/347878 "2023-11-23T17:14:12Z")
I have applied the Output Isolator pattern to send logs to two ES clusters. However, as mentioned in the reference: "If any of the persistent queues of the downstream pipelines (in the example above, buffered-es and bu… --- ## [Cannot login to kibana afer activating SE-LINUX](https://discuss.elastic.co/t/cannot-login-to-kibana-afer-activating-se-linux/347882)
**Author:** [@uli67](https://discuss.elastic.co/u/uli67)\ **Replies:** 0\ **Last updated:** [November 23, 2023, 4:49pm UTC](https://discuss.elastic.co/t/cannot-login-to-kibana-afer-activating-se-linux/347882 "2023-11-23T16:49:10Z")
Hello, I am running docker.elastic.co/elasticsearch/elasticsearch:8.11.0 eswrapper docker.elastic.co/beats/elastic-agent:8.11.0 7 days ago Up 3 minutes 0.0.0.0:8220-\>8220/tcp… --- ## [Enabling fingerprint file\_identiy](https://discuss.elastic.co/t/enabling-fingerprint-file-identiy/347780)
**Author:** [@MajorNickle](https://discuss.elastic.co/u/MajorNickle)\ **Replies:** 4\ **Last updated:** [November 23, 2023, 3:55pm UTC](https://discuss.elastic.co/t/enabling-fingerprint-file-identiy/347780 "2023-11-23T15:55:09Z")
Hey all, I'm stuck with getting the fingerprint file\_identity working. I have a few nfs mounts from which I'm reading log files. I tried enabling the fingerprint option in the scanner but it seems like it's not really b… --- ## [Delay of one hour in the events to ELK](https://discuss.elastic.co/t/delay-of-one-hour-in-the-events-to-elk/347875)
**Author:** [@billy.castillo.73](https://discuss.elastic.co/u/billy.castillo.73)\ **Replies:** 0\ **Last updated:** [November 23, 2023, 3:23pm UTC](https://discuss.elastic.co/t/delay-of-one-hour-in-the-events-to-elk/347875 "2023-11-23T15:23:40Z")
The events of a system that are being parsed from filebeat to our ELK are arriving an hour late. I have already configured the netscout.yml module of filebeat with the grok processors, with var.tz\_offset and with date -… --- ## [Translate kibana bar chart to TSVB](https://discuss.elastic.co/t/translate-kibana-bar-chart-to-tsvb/347421)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 3\ **Last updated:** [November 23, 2023, 2:12pm UTC](https://discuss.elastic.co/t/translate-kibana-bar-chart-to-tsvb/347421 "2023-11-23T14:12:26Z")
Hello All, I have visual made using Vertical Bar chart and over there I can't split x axis twice .I need to show max of duration for given startTime (x-axis),but same time x-axis also show release to compare. Given rel… --- ## [Cluster currently has \[1000\]/\[1000\] maximum normal shards open](https://discuss.elastic.co/t/cluster-currently-has-1000-1000-maximum-normal-shards-open/347719)
**Author:** [@DaddyYusk](https://discuss.elastic.co/u/DaddyYusk)\ **Replies:** 4\ **Last updated:** [November 23, 2023, 1:32pm UTC](https://discuss.elastic.co/t/cluster-currently-has-1000-1000-maximum-normal-shards-open/347719 "2023-11-23T13:32:03Z")
Hi, From the title, this is an error I usually encounter with my Elastic Proof Of Concept. The workaround is easy, I simply close and delete some indices from time to time... But now I'm currently deploying Elastic in… --- ## [Backing Up ES Indices](https://discuss.elastic.co/t/backing-up-es-indices/347815)
**Author:** [@Nijal](https://discuss.elastic.co/u/Nijal)\ **Replies:** 2\ **Last updated:** [November 23, 2023, 1:30pm UTC](https://discuss.elastic.co/t/backing-up-es-indices/347815 "2023-11-23T13:30:54Z")
I have a few questions about Snapshots, What is the correct approach to save snapshots to long term data store such as tape storage What is the correct proceedure to restore the snapshots stored in the tape storage. Ho… --- ## [Syncing Huge DataSet From MySQL to Elasticsearch](https://discuss.elastic.co/t/syncing-huge-dataset-from-mysql-to-elasticsearch/347853)
**Author:** [@Aswini\_Kumar\_Rout](https://discuss.elastic.co/u/Aswini_Kumar_Rout)\ **Replies:** 0\ **Last updated:** [November 23, 2023, 1:04pm UTC](https://discuss.elastic.co/t/syncing-huge-dataset-from-mysql-to-elasticsearch/347853 "2023-11-23T13:04:53Z")
Hi Team, We have one requirement to use Elasticsearch in our legacy application which has MySQL datbase and the size of the DB is around 300 GB and with 200 or more tables. So, here I am bit confused that - which would… --- ## [Editing a managed policy can break Kibana](https://discuss.elastic.co/t/editing-a-managed-policy-can-break-kibana/347828)
**Author:** [@DaddyYusk](https://discuss.elastic.co/u/DaddyYusk)\ **Replies:** 2\ **Last updated:** [November 23, 2023, 1:19pm UTC](https://discuss.elastic.co/t/editing-a-managed-policy-can-break-kibana/347828 "2023-11-23T13:19:57Z")
Hi, After a successful Fleet Server and Elastic Agent deployment, I wanted to tweak the ILM called "logs" and "metrics" which are both "Managed". But when trying to do so, I encounter this well known warning : So af… --- ## [Search template based on list](https://discuss.elastic.co/t/search-template-based-on-list/347852)
**Author:** [@pszemesy](https://discuss.elastic.co/u/pszemesy)\ **Replies:** 0\ **Last updated:** [November 23, 2023, 12:37pm UTC](https://discuss.elastic.co/t/search-template-based-on-list/347852 "2023-11-23T12:37:30Z")
Hi All, I have an index (contains translations) with the following mappings: document\_name: keyword, ... EN: { content: text, stored\_by: keyword, stored\_at: date, ... } \<\\>: { content: text, store… --- ## [Destination of Audit Logs After Enabling Audit Logging on Kibana](https://discuss.elastic.co/t/destination-of-audit-logs-after-enabling-audit-logging-on-kibana/347846)
**Author:** [@yago82](https://discuss.elastic.co/u/yago82)\ **Replies:** 1\ **Last updated:** [November 23, 2023, 12:16pm UTC](https://discuss.elastic.co/t/destination-of-audit-logs-after-enabling-audit-logging-on-kibana/347846 "2023-11-23T12:16:19Z")
Hi, I am currently exploring the functionalities related to audit logging on Kibana and have a technical query regarding the destination of these logs once audit logging is enabled. Specifically, upon enabling audit lo… --- ## [Rollup then backup indices](https://discuss.elastic.co/t/rollup-then-backup-indices/346036)
**Author:** [@lstoneir](https://discuss.elastic.co/u/lstoneir)\ **Replies:** 1\ **Last updated:** [November 23, 2023, 12:09pm UTC](https://discuss.elastic.co/t/rollup-then-backup-indices/346036 "2023-11-23T12:09:57Z")
Hi there, I have a elastic cluster with 5 nodes (each node 1TB) I want to backup my indices, but I dont have enough resources to backup all indices. I want to rollup indices for example my main indices are hourly, I w… --- ## [Sync 2 indices diffrenet remote clusters](https://discuss.elastic.co/t/sync-2-indices-diffrenet-remote-clusters/347851)
**Author:** [@lstoneir](https://discuss.elastic.co/u/lstoneir)\ **Replies:** 0\ **Last updated:** [November 23, 2023, 11:58am UTC](https://discuss.elastic.co/t/sync-2-indices-diffrenet-remote-clusters/347851 "2023-11-23T11:58:50Z")
Hi I have cluser A with index e.g. User\_info I have another cluster named B I want to sync User\_info (B) with User\_info (A) all time!! Can i do this with logstash? how? --- ## [Auditbeat: system/socket dataset setup failed - guess\_inet\_sock failed: timeout while waiting](https://discuss.elastic.co/t/auditbeat-system-socket-dataset-setup-failed-guess-inet-sock-failed-timeout-while-waiting/347850)
**Author:** [@Stefan\_Bauer](https://discuss.elastic.co/u/Stefan_Bauer)\ **Replies:** 0\ **Last updated:** [November 23, 2023, 11:57am UTC](https://discuss.elastic.co/t/auditbeat-system-socket-dataset-setup-failed-guess-inet-sock-failed-timeout-while-waiting/347850 "2023-11-23T11:57:52Z")
Hi folks, auditbeat fails on 2 Ubuntu 20 systems with the following errors several times a day: auditbeat\[1929153\]: {"log.level":"error","@timestamp":"2023-11-23T08:29:49.984+0100","log.origin":{"file.name":"instance/b… --- ## [Elasticsearch.yml configuration file is missing in linux](https://discuss.elastic.co/t/elasticsearch-yml-configuration-file-is-missing-in-linux/347839)
**Author:** [@krishnapro](https://discuss.elastic.co/u/krishnapro)\ **Replies:** 1\ **Last updated:** [November 23, 2023, 11:48am UTC](https://discuss.elastic.co/t/elasticsearch-yml-configuration-file-is-missing-in-linux/347839 "2023-11-23T11:48:47Z")
I have installed elasticsearch in linux mint but elasticsearch.yml file is missing. I have uninstall and reinstall it but same problem. I don't know what do please help me to fix it. --- ## [Correct user permission / role when using kibana](https://discuss.elastic.co/t/correct-user-permission-role-when-using-kibana/347845)
**Author:** [@yabetsu93](https://discuss.elastic.co/u/yabetsu93)\ **Replies:** 0\ **Last updated:** [November 23, 2023, 11:39am UTC](https://discuss.elastic.co/t/correct-user-permission-role-when-using-kibana/347845 "2023-11-23T11:39:06Z")
Good Day, I was able to deploy kibana helm-charts but setting up kibana\_system and password generated everytime i logged in as kibana\_admin role or elastic after logged out i got 403 forbidden it is kinda annoying and w… --- ## [CSV::MalformedCSVError: Missing or stray quote in line 1](https://discuss.elastic.co/t/csv-missing-or-stray-quote-in-line-1/346726)
**Author:** [@parosio](https://discuss.elastic.co/u/parosio)\ **Replies:** 1\ **Last updated:** [November 23, 2023, 11:05am UTC](https://discuss.elastic.co/t/csv-missing-or-stray-quote-in-line-1/346726 "2023-11-23T11:05:25Z")
Hello, I've read the previuos posts on this topic (and related), but still have problems with csv files containing windows command lines... For example: 98792634295,https://falcon.eu-1.crowdstrike.com/activity/detecti… --- ## [Getting Timeout after sometime](https://discuss.elastic.co/t/getting-timeout-after-sometime/347494)
**Author:** [@deepak\_Bahuguna](https://discuss.elastic.co/u/deepak_Bahuguna)\ **Replies:** 4\ **Last updated:** [November 23, 2023, 10:55am UTC](https://discuss.elastic.co/t/getting-timeout-after-sometime/347494 "2023-11-23T10:55:53Z")
HI Guys, I am new to Elastic, Kibana. I have downloaded the Elastic and Kibana then I run both and it worked fine. What is problem is after installing I have kept it opened in evening and when I see it morning it has sh… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=365) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=367)