# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=924 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 925 --- ## [How do I make my LOGSTASH pipeline of input type ELASTICSEARCH to be running in certain periods automatically?](https://discuss.elastic.co/t/how-do-i-make-my-logstash-pipeline-of-input-type-elasticsearch-to-be-running-in-certain-periods-automatically/305799)
**Author:** [@Daniel\_Oliveira](https://discuss.elastic.co/u/Daniel_Oliveira)\ **Replies:** 1\ **Last updated:** [May 27, 2022, 12:56pm UTC](https://discuss.elastic.co/t/how-do-i-make-my-logstash-pipeline-of-input-type-elasticsearch-to-be-running-in-certain-periods-automatically/305799 "2022-05-27T12:56:45Z")
Hello gentlemen. I have 2 pipelines running by my LOGSTASH. the first input FILE pipeline is running normally and processing .csv data the second pipeline of input type Elasticsearch it runs only the first time, then … --- ## [Find phrase ending with or containing](https://discuss.elastic.co/t/find-phrase-ending-with-or-containing/305801)
**Author:** [@dlazure](https://discuss.elastic.co/u/dlazure)\ **Replies:** 0\ **Last updated:** [May 27, 2022, 12:52pm UTC](https://discuss.elastic.co/t/find-phrase-ending-with-or-containing/305801 "2022-05-27T12:52:47Z")
this is what i am getting from a router log message : script,info CLIENT-ABC: 6.42.11 (long-term) the portion CLIENT-ABC: 6.42.11 will be dynamic and will change from devices to devices. I wou… --- ## [Create Index with settings](https://discuss.elastic.co/t/create-index-with-settings/305771)
**Author:** [@liushida](https://discuss.elastic.co/u/liushida)\ **Replies:** 1\ **Last updated:** [May 27, 2022, 12:31pm UTC](https://discuss.elastic.co/t/create-index-with-settings/305771 "2022-05-27T12:31:13Z")
When I create the index with settings, the response is \[es/indices.create\] failed: \[illegal\_argument\_exception\] unknown setting \[index.settings.analysis.analyzer.custom.type\] so how I should resolve this problem? the E… --- ## [Logstash cpu usage is very high, and there are a large number of thread GC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666)
**Author:** [@1065916870](https://discuss.elastic.co/u/1065916870)\ **Replies:** 6\ **Last updated:** [May 27, 2022, 12:16pm UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666 "2022-05-27T12:16:39Z")
Using 3-node logstash to consume kafka data, logstash cpu usage is very high, and there are a large number of thread GC logstash configuration: 3 node , 16C 32G jvm -Xms16g -Xmx16g two pipeline: pipeline.workers: 16 … --- ## [Dense vector upload slow down](https://discuss.elastic.co/t/dense-vector-upload-slow-down/305609)
**Author:** [@KacperLukawski](https://discuss.elastic.co/u/KacperLukawski)\ **Replies:** 2\ **Last updated:** [May 27, 2022, 11:55am UTC](https://discuss.elastic.co/t/dense-vector-upload-slow-down/305609 "2022-05-27T11:55:16Z")
Hi there! I've been playing a bit with the dense\_vector field recently. I have a collection of thousands of vectors, 100 dimensions each. I created the index with the following config: { "properties": { "vector":… --- ## [Fetch system date and time for remote serverd](https://discuss.elastic.co/t/fetch-system-date-and-time-for-remote-serverd/305739)
**Author:** [@Anagha\_nambiar](https://discuss.elastic.co/u/Anagha_nambiar)\ **Replies:** 6\ **Last updated:** [May 27, 2022, 11:54am UTC](https://discuss.elastic.co/t/fetch-system-date-and-time-for-remote-serverd/305739 "2022-05-27T11:54:47Z")
In Kibana dashboard, we would like to specifically get the system date and time of a remote server. We have installed metricbeat agent in that server. Could you please help to know how to use query in Kibana dashboard … --- ## [Elastic Search and Kibana is not working](https://discuss.elastic.co/t/elastic-search-and-kibana-is-not-working/305606)
**Author:** [@John\_Mari](https://discuss.elastic.co/u/John_Mari)\ **Replies:** 13\ **Last updated:** [May 27, 2022, 11:53am UTC](https://discuss.elastic.co/t/elastic-search-and-kibana-is-not-working/305606 "2022-05-27T11:53:08Z")
Hi Everyone, We installed Elasticsearch and Kibana in our Machine. Previously it was working fine. But, when we tried to access Elasticsearch and Kibana. Its showing following error. Can anyone guide us to solve the… --- ## [ILM policy not working](https://discuss.elastic.co/t/ilm-policy-not-working/302604)
**Author:** [@jisha](https://discuss.elastic.co/u/jisha)\ **Replies:** 5\ **Last updated:** [May 27, 2022, 11:22am UTC](https://discuss.elastic.co/t/ilm-policy-not-working/302604 "2022-05-27T11:22:42Z")
Hi, I am using elk version 8.0.0 in windows OS. There is totally 3 elk nodes 2 logstash and 1 kibana nodes . The ILM is not working based on the duration i set (ex:5mints,the data is not getting deleted).I have create… --- ## [Model\_bytes\_memory\_limit](https://discuss.elastic.co/t/model-bytes-memory-limit/303741)
**Author:** [@alaine](https://discuss.elastic.co/u/alaine)\ **Replies:** 12\ **Last updated:** [May 27, 2022, 10:17am UTC](https://discuss.elastic.co/t/model-bytes-memory-limit/303741 "2022-05-27T10:17:27Z")
A couple of the ML jobs we are running in our environment have hit their "hard limit". I have expanded the "model\_memory\_limit" to allow them more resources in order to run, but they continued to hit the hard limit. When… --- ## [Kafka + Metricbeat error:Failed to fetch offset for (topic, partition)](https://discuss.elastic.co/t/kafka-metricbeat-error-failed-to-fetch-offset-for-topic-partition/305777)
**Author:** [@ekeras](https://discuss.elastic.co/u/ekeras)\ **Replies:** 0\ **Last updated:** [May 27, 2022, 8:15am UTC](https://discuss.elastic.co/t/kafka-metricbeat-error-failed-to-fetch-offset-for-topic-partition/305777 "2022-05-27T08:15:08Z")
I am running elastic stack by this repository. I just changed the network elastic stack is being hosted on, so it would be on the same network as Kafka is. To set up monitoring kafka metrics with Elastic, I have followe… --- ## [Logstash not running own username](https://discuss.elastic.co/t/logstash-not-running-own-username/305703)
**Author:** [@bekk777](https://discuss.elastic.co/u/bekk777)\ **Replies:** 3\ **Last updated:** [May 27, 2022, 7:58am UTC](https://discuss.elastic.co/t/logstash-not-running-own-username/305703 "2022-05-27T07:58:20Z")
Hello, I am running this command: /usr/share/logstash/bin/logstash --debug --config.test\_and\_exit -f /etc/logstash/filebeat-test.conf I know many times published this kind of issue, but I'm getting errors yet. I tried… --- ## [Map Alert Rules to Kibana Widget](https://discuss.elastic.co/t/map-alert-rules-to-kibana-widget/305544)
**Author:** [@jsoule6](https://discuss.elastic.co/u/jsoule6)\ **Replies:** 1\ **Last updated:** [May 27, 2022, 6:40am UTC](https://discuss.elastic.co/t/map-alert-rules-to-kibana-widget/305544 "2022-05-27T06:40:25Z")
Hello, We are looking to use Kibana dashboard capability to provide monitoring capability for our customer. We are going to be create alert rules based on numerous custom conditions and would like to map the rules to wi… --- ## [Logstash Pipeline for Nested data from sql](https://discuss.elastic.co/t/logstash-pipeline-for-nested-data-from-sql/305688)
**Author:** [@vikram\_singh](https://discuss.elastic.co/u/vikram_singh)\ **Replies:** 2\ **Last updated:** [May 27, 2022, 4:57am UTC](https://discuss.elastic.co/t/logstash-pipeline-for-nested-data-from-sql/305688 "2022-05-27T04:57:05Z")
Hi, I am trying to insert nested data from sqldb to Elasticsearch index. I create a query with joins and insert data into Elasticsearch index. I used aggregate, but it is not giving correct result. Mapping PUT testi… --- ## [Query explanation](https://discuss.elastic.co/t/query-explanation/305765)
**Author:** [@sean1223](https://discuss.elastic.co/u/sean1223)\ **Replies:** 0\ **Last updated:** [May 27, 2022, 4:04am UTC](https://discuss.elastic.co/t/query-explanation/305765 "2022-05-27T04:04:33Z")
Is there documentation explaining how to read the equation and the meaning of the symbols? Also, --- ## [Extract a field from a text field](https://discuss.elastic.co/t/extract-a-field-from-a-text-field/305756)
**Author:** [@RaonyO](https://discuss.elastic.co/u/RaonyO)\ **Replies:** 1\ **Last updated:** [May 27, 2022, 3:33am UTC](https://discuss.elastic.co/t/extract-a-field-from-a-text-field/305756 "2022-05-27T03:33:39Z")
I'm trying to extract the fields that come within the "indicator" and "agentDetectionInfo" field, could someone help me? here's the log in json: { "\_index": "xxx", "\_type": "\_doc", "\_id": "xxxxxhvfUu\_M2V8K", "\_… --- ## [Adding new relic support to elasticsearch as a Windows service](https://discuss.elastic.co/t/adding-new-relic-support-to-elasticsearch-as-a-windows-service/305604)
**Author:** [@mmr11408](https://discuss.elastic.co/u/mmr11408)\ **Replies:** 7\ **Last updated:** [May 27, 2022, 2:01am UTC](https://discuss.elastic.co/t/adding-new-relic-support-to-elasticsearch-as-a-windows-service/305604 "2022-05-27T02:01:05Z")
I am adding new relic support to Elasticsearch on Windows. After adding -javaagent, ES starts up fine when invoked from CMD. But it will not start as a service. The error in system log viewer is "Incorrect Function". The… --- ## [Error 401 in kibana.log](https://discuss.elastic.co/t/error-401-in-kibana-log/305762)
**Author:** [@RaonyO](https://discuss.elastic.co/u/RaonyO)\ **Replies:** 0\ **Last updated:** [May 27, 2022, 1:37am UTC](https://discuss.elastic.co/t/error-401-in-kibana-log/305762 "2022-05-27T01:37:40Z")
hello, I have this error appearing all the time in my kibana log. the web interface still works but i would like to resolve this error. can anyone suggest a solution? ps: i use a nginx reverse proxy in front of kibana. {… --- ## [Cannot invoke \\"String.length()\\" because \\"str\\" is null](https://discuss.elastic.co/t/cannot-invoke-string-length-because-str-is-null/305752)
**Author:** [@austin0918](https://discuss.elastic.co/u/austin0918)\ **Replies:** 3\ **Last updated:** [May 27, 2022, 12:54am UTC](https://discuss.elastic.co/t/cannot-invoke-string-length-because-str-is-null/305752 "2022-05-27T00:54:38Z")
Working on migrating Windows DNS logs to data stream and seeing a lot of errors. \[2022-05-26T13:43:37,457\]\[INFO \]\[logstash.outputs.elasticsearch\]\[filebeat-dns-dhcp\]\[7fe0474f97756847637aa742a5ac38f62e0e1460d0c70da893dabe… --- ## [Clean up unnecessary indices after upgrade Elastic Stack version](https://discuss.elastic.co/t/clean-up-unnecessary-indices-after-upgrade-elastic-stack-version/305586)
**Author:** [@norgro2601](https://discuss.elastic.co/u/norgro2601)\ **Replies:** 4\ **Last updated:** [May 27, 2022, 12:13am UTC](https://discuss.elastic.co/t/clean-up-unnecessary-indices-after-upgrade-elastic-stack-version/305586 "2022-05-27T00:13:17Z")
Hi, after I've upgraded our Stack from version 7 to version 8, I see a lot of indices, that seem to be related to version 7 and therefore no longer in use. Unfortunately, I can't find any document, that shows a list of … --- ## [Understanding Elasticsearch Source Code](https://discuss.elastic.co/t/understanding-elasticsearch-source-code/305759)
**Author:** [@nikhilmehta](https://discuss.elastic.co/u/nikhilmehta)\ **Replies:** 0\ **Last updated:** [May 26, 2022, 11:47pm UTC](https://discuss.elastic.co/t/understanding-elasticsearch-source-code/305759 "2022-05-26T23:47:03Z")
Hello Everyone I am new to open source and quite new to reading a large Code Base, is their any particular guidelines, documentation or tutorials which helps one to do so for ES. I am a particularly interested how a ap… --- ## [Security in 8.x upgraded from 7.x](https://discuss.elastic.co/t/security-in-8-x-upgraded-from-7-x/305510)
**Author:** [@seePyou](https://discuss.elastic.co/u/seePyou)\ **Replies:** 3\ **Last updated:** [May 26, 2022, 10:42pm UTC](https://discuss.elastic.co/t/security-in-8-x-upgraded-from-7-x/305510 "2022-05-26T22:42:26Z")
We are running 7.11 of Elastic and we will be moving to 8.x (by first upgrading to 7.17). It would have been good to have security already, but we do not. We want to add it now, but I'm looking at the documentation and… --- ## [Metricbeat 8.2.1 monitoring Elasticsearch 8.2.1 - Template issue](https://discuss.elastic.co/t/metricbeat-8-2-1-monitoring-elasticsearch-8-2-1-template-issue/305649)
**Author:** [@bar0n36](https://discuss.elastic.co/u/bar0n36)\ **Replies:** 5\ **Last updated:** [May 26, 2022, 10:07pm UTC](https://discuss.elastic.co/t/metricbeat-8-2-1-monitoring-elasticsearch-8-2-1-template-issue/305649 "2022-05-26T22:07:40Z")
There appears to be a couple of issues with the Metricbeat 8.2.1 template for the Elasticsearch metric monitoring. I am running a newly built 8.2.1 cluster (in Docker) and attempting to use Metricbeat to gather Stack Mon… --- ## [TLS Logstash](https://discuss.elastic.co/t/tls-logstash/305735)
**Author:** [@Thuunder7](https://discuss.elastic.co/u/Thuunder7)\ **Replies:** 5\ **Last updated:** [May 26, 2022, 7:06pm UTC](https://discuss.elastic.co/t/tls-logstash/305735 "2022-05-26T19:06:23Z")
Hello guys, I have a beats input on a logstash pipeline using SSL /TLS: input { beats { port =\> 5044 ssl =\> true ssl\_certificate =\> "/etc/../mycert.pem" ssl\_key =\> "/etc/.../mycert.p… --- ## [Concatenation between two documents message value](https://discuss.elastic.co/t/concatenation-between-two-documents-message-value/305572)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 2\ **Last updated:** [May 26, 2022, 6:33pm UTC](https://discuss.elastic.co/t/concatenation-between-two-documents-message-value/305572 "2022-05-26T18:33:30Z")
Hi, we assume that i have 3 documents in Elasticsearch and here a example of messages that contains document 1 message : 2022-01-13 17:16:05 - request in formatHexDump is =\> 000: 12 15 46 77 77 55 | ak55s2 .... 000… --- ## [Too many logs for filebeat?](https://discuss.elastic.co/t/too-many-logs-for-filebeat/305486)
**Author:** [@seePyou](https://discuss.elastic.co/u/seePyou)\ **Replies:** 3\ **Last updated:** [May 26, 2022, 5:59pm UTC](https://discuss.elastic.co/t/too-many-logs-for-filebeat/305486 "2022-05-26T17:59:27Z")
I'm trying to run a POC for Filebeats on capturing logs from Signiant file delivery. We get files in and we send files out, and there are processing steps in Signiant. For each event, for every process, there is a log fi… --- ## [Connecting logstash on elastic with security enabled](https://discuss.elastic.co/t/connecting-logstash-on-elastic-with-security-enabled/305636)
**Author:** [@dlazure](https://discuss.elastic.co/u/dlazure)\ **Replies:** 6\ **Last updated:** [May 26, 2022, 5:32pm UTC](https://discuss.elastic.co/t/connecting-logstash-on-elastic-with-security-enabled/305636 "2022-05-26T17:32:46Z")
Hi I am trying to connect a plugin on elasticsearch with security enabled. The plugin is having an unrecoverable error and is restarting portion of logs of logstash \[2022-05-25T18:53:15,071\]\[INFO \]\[logstash.inputs.tc… --- ## [Different Timezones in syslog for some hosts](https://discuss.elastic.co/t/different-timezones-in-syslog-for-some-hosts/305737)
**Author:** [@syedabdullah](https://discuss.elastic.co/u/syedabdullah)\ **Replies:** 0\ **Last updated:** [May 26, 2022, 5:18pm UTC](https://discuss.elastic.co/t/different-timezones-in-syslog-for-some-hosts/305737 "2022-05-26T17:18:47Z")
Hi Everyone, I am relatively new to ELK stack I added a new source of syslog but that source is in UTC, When I forward the syslogs, I have its sources (hosts) in UTC and everything else in EST and once they go into Elas… --- ## [Stats requests to data nodes timing out while ingestion is happening](https://discuss.elastic.co/t/stats-requests-to-data-nodes-timing-out-while-ingestion-is-happening/305100)
**Author:** [@ethompson](https://discuss.elastic.co/u/ethompson)\ **Replies:** 7\ **Last updated:** [May 26, 2022, 5:07pm UTC](https://discuss.elastic.co/t/stats-requests-to-data-nodes-timing-out-while-ingestion-is-happening/305100 "2022-05-26T17:07:04Z")
I've got an on-prem hosted ElasticStack set up that looks like \[apps that send logs via http\]-\>Logstash Shipper-\>Kafka-\>3 Logstash Ingesters-\>6 ElasticSearches (3 master, 3 data). Everything is on 7.16.2, and has been h… --- ## [Influx Plugin: Data loss during Output server down-time](https://discuss.elastic.co/t/influx-plugin-data-loss-during-output-server-down-time/305624)
**Author:** [@AurelioAranzana](https://discuss.elastic.co/u/AurelioAranzana)\ **Replies:** 5\ **Last updated:** [May 26, 2022, 5:06pm UTC](https://discuss.elastic.co/t/influx-plugin-data-loss-during-output-server-down-time/305624 "2022-05-26T17:06:17Z")
Hi everyone! I would like to know how Logstash handles when your output server is down. In my case, I'm using the Influxdb output plugin. From what I'm seeing it will keep trying according to the max\_retries parameter… --- ## [HTTP filter logstash error](https://discuss.elastic.co/t/http-filter-logstash-error/305714)
**Author:** [@vishnuhngama](https://discuss.elastic.co/u/vishnuhngama)\ **Replies:** 1\ **Last updated:** [May 26, 2022, 4:50pm UTC](https://discuss.elastic.co/t/http-filter-logstash-error/305714 "2022-05-26T16:50:07Z")
Hi Team, We are facing issue while using http filter in logstash . We are using file input plugin to read from a file and then use the message field in http filter please see the code. Please help u actually its a urg… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=923) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=925)