# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=51 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 52 --- ## [Impossible to create a second index](https://discuss.elastic.co/t/impossible-to-create-a-second-index/347627)
**Author:** [@Christian\_1974](https://discuss.elastic.co/u/Christian_1974)\ **Replies:** 6\ **Last updated:** [November 21, 2023, 2:13pm UTC](https://discuss.elastic.co/t/impossible-to-create-a-second-index/347627 "2023-11-21T14:13:32Z")
Hi, (Sorry, I am not english. I am french, so, please, be patient with me, in english :)). I created a configuration to test logstash. The configuration log my local syslog in Kibana. That, that works. But if I try to … --- ## [SSL Error when running logstash avro schema registry in ubuntu](https://discuss.elastic.co/t/ssl-error-when-running-logstash-avro-schema-registry-in-ubuntu/347605)
**Author:** [@amaleswar](https://discuss.elastic.co/u/amaleswar)\ **Replies:** 0\ **Last updated:** [November 21, 2023, 9:19am UTC](https://discuss.elastic.co/t/ssl-error-when-running-logstash-avro-schema-registry-in-ubuntu/347605 "2023-11-21T09:19:59Z")
I\`m trying to migrate from CentOS to Ubuntu. Able to install the Logstash and added registry info and running the config locally to make sure, it is running as expected. But it is throwing below error. \[ERROR\] 2023-11-2… --- ## [Logstash buffer for Sentinel - Architecture questions](https://discuss.elastic.co/t/logstash-buffer-for-sentinel-architecture-questions/347596)
**Author:** [@zatury](https://discuss.elastic.co/u/zatury)\ **Replies:** 1\ **Last updated:** [November 21, 2023, 8:11am UTC](https://discuss.elastic.co/t/logstash-buffer-for-sentinel-architecture-questions/347596 "2023-11-21T08:11:23Z")
Hello, My company is currently transitioning from Splunk to Sentinel, despite my preference for Elastic. Sentinel utilizes AMA agents to gather logs from various sources, listening on port 514. However, a significant ch… --- ## [Send logs from filebeat to logstash via NGINX reverse proxy](https://discuss.elastic.co/t/send-logs-from-filebeat-to-logstash-via-nginx-reverse-proxy/347590)
**Author:** [@R\_H\_O\_M\_B\_I\_X](https://discuss.elastic.co/u/R_H_O_M_B_I_X)\ **Replies:** 0\ **Last updated:** [November 21, 2023, 7:22am UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-logstash-via-nginx-reverse-proxy/347590 "2023-11-21T07:22:34Z")
Hi I'm looking for a way to forward my logs from filebeat where filebeat is reading logs from my local machine file and sending it to my private server in which logstash is installed via nginx reverse proxy where nginx … --- ## [Docker Hard Disk Image File Being Too Large](https://discuss.elastic.co/t/docker-hard-disk-image-file-being-too-large/347443)
**Author:** [@Ethan777100](https://discuss.elastic.co/u/Ethan777100)\ **Replies:** 2\ **Last updated:** [November 20, 2023, 6:19pm UTC](https://discuss.elastic.co/t/docker-hard-disk-image-file-being-too-large/347443 "2023-11-20T18:19:30Z")
Its now killing my computer space. I'm in the midst of ingesting 537 csv files (total 10.7GB) into elasticsearch. C:\\Users\\ethan\\AppData\\Local\\Docker\\wsl\\data I'm not sure why it became so big. --- ## [How can create datastream automatically](https://discuss.elastic.co/t/how-can-create-datastream-automatically/347531)
**Author:** [@baber1223](https://discuss.elastic.co/u/baber1223)\ **Replies:** 1\ **Last updated:** [November 20, 2023, 2:04pm UTC](https://discuss.elastic.co/t/how-can-create-datastream-automatically/347531 "2023-11-20T14:04:53Z")
This is my logstash output part : after run , It will create index with the name of "TXT" but I want to create automatically datastream with all index templates and .. . Is that possible? output{ stdout{} elast… --- ## [Why my query does not work correctly after create new filed in logstash](https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504)
**Author:** [@baber1223](https://discuss.elastic.co/u/baber1223)\ **Replies:** 5\ **Last updated:** [November 20, 2023, 12:16pm UTC](https://discuss.elastic.co/t/why-my-query-does-not-work-correctly-after-create-new-filed-in-logstash/347504 "2023-11-20T12:16:23Z")
This is my sample data: 134.255.248.30 - - \[20/Nov/2023:09:04:57 +0330\] "GET /serve/finnotech/validateDest?key=3f94393b5eaab29a167e5edc8a99860cba121550053bd113d291d71f146a7fa0¶meters=%7B%22dest%22:%22IR5201900000002… --- ## [Track changes in Logstash](https://discuss.elastic.co/t/track-changes-in-logstash/347452)
**Author:** [@Felicien\_Ihirwe](https://discuss.elastic.co/u/Felicien_Ihirwe)\ **Replies:** 1\ **Last updated:** [November 20, 2023, 12:05pm UTC](https://discuss.elastic.co/t/track-changes-in-logstash/347452 "2023-11-20T12:05:14Z")
How to come from data in 1 to data in 2? Note that the col1 is continuous. col1 2.3 2.3 2.3 5.7 5.7 6.1 6.1 .... .. . I want to achieve this: col1 | col2 2.3 | 1 2.3 |1 2.3 |1 5.7 |2 5.7 |2 6.1 |3 6.1… --- ## [Logstash pipeline does not work](https://discuss.elastic.co/t/logstash-pipeline-does-not-work/347487)
**Author:** [@baber1223](https://discuss.elastic.co/u/baber1223)\ **Replies:** 2\ **Last updated:** [November 20, 2023, 10:56am UTC](https://discuss.elastic.co/t/logstash-pipeline-does-not-work/347487 "2023-11-20T10:56:27Z")
This is my pattern log : 80.253.157.26 - - \[19/Nov/2023:15:17:50 +0330\] "POST /followup/danesh/5b81bc62-d82d-4f98-aacd-eab80474faca HTTP/1.1" 200 852 This is apache log . As I know if I want to use this log in logstash… --- ## [Logstash The order of synchronized data fields is inconsistent with the source end](https://discuss.elastic.co/t/logstash-the-order-of-synchronized-data-fields-is-inconsistent-with-the-source-end/347509)
**Author:** [@haimaren](https://discuss.elastic.co/u/haimaren)\ **Replies:** 1\ **Last updated:** [November 20, 2023, 10:29am UTC](https://discuss.elastic.co/t/logstash-the-order-of-synchronized-data-fields-is-inconsistent-with-the-source-end/347509 "2023-11-20T10:29:58Z")
My Logstash Configuration input { elasticsearch { hosts =\> "http://172.19.23.12:9200" index =\> "\*" size =\> 1000 scroll =\> "5m" docinfo =\> true } } filter { mutate { … --- ## [Why logstash cannot start after define new pipeline](https://discuss.elastic.co/t/why-logstash-cannot-start-after-define-new-pipeline/347483)
**Author:** [@baber1223](https://discuss.elastic.co/u/baber1223)\ **Replies:** 3\ **Last updated:** [November 19, 2023, 8:39pm UTC](https://discuss.elastic.co/t/why-logstash-cannot-start-after-define-new-pipeline/347483 "2023-11-19T20:39:20Z")
I was created pipeline1 in logstash and it was working excellent but I added a new pipeline now when I want to start logstash it shows follow error and cannot start icsearch is unreachable or down?) {:message=\>"No Avail… --- ## [Why does translate not work for me?](https://discuss.elastic.co/t/why-does-translate-not-work-for-me/347478)
**Author:** [@andre22](https://discuss.elastic.co/u/andre22)\ **Replies:** 2\ **Last updated:** [November 19, 2023, 5:45pm UTC](https://discuss.elastic.co/t/why-does-translate-not-work-for-me/347478 "2023-11-19T17:45:08Z")
Hi, I can't wrap my head around why I don't get this translate filter to work. I have a bunch of IoT logfiles (csv) that I want to import. One of the fields (KO-ID) does contain an internal ID of the old log engine, … --- ## [Logstash is shutting down after connecting to Elastic Search due to One or more required cgroup files or directories not found](https://discuss.elastic.co/t/logstash-is-shutting-down-after-connecting-to-elastic-search-due-to-one-or-more-required-cgroup-files-or-directories-not-found/347448)
**Author:** [@sathishkumarD](https://discuss.elastic.co/u/sathishkumarD)\ **Replies:** 6\ **Last updated:** [November 19, 2023, 1:56pm UTC](https://discuss.elastic.co/t/logstash-is-shutting-down-after-connecting-to-elastic-search-due-to-one-or-more-required-cgroup-files-or-directories-not-found/347448 "2023-11-19T13:56:15Z")
\[2023-11-18T14:35:04,262\]\[DEBUG\]\[org.logstash.execution.PeriodicFlush\]\[main\] Pushing flush onto pipeline. \[2023-11-18T14:35:04,906\]\[DEBUG\]\[logstash.instrument.periodicpoller.cgroup\] One or more required cgroup files or … --- ## [Not able to parse completely with grok](https://discuss.elastic.co/t/not-able-to-parse-completely-with-grok/346870)
**Author:** [@sudhir\_singh](https://discuss.elastic.co/u/sudhir_singh)\ **Replies:** 2\ **Last updated:** [November 18, 2023, 10:56pm UTC](https://discuss.elastic.co/t/not-able-to-parse-completely-with-grok/346870 "2023-11-18T22:56:46Z")
"10/Nov/2023:12:59:05 +0530" 192.54.23.32 GET "GET /healthcheck HTTP/1.1" 178 200 453 2 "nginx/1.23.4 (health check server\_192.87.23.870\_Pool-1\_http\_ok)" 127.0.0.1:3000 200 0.001 0.002 0.000 0.002 apimumbcms.hydtimes… --- ## [How to create a script statically in Logstash 7.17](https://discuss.elastic.co/t/how-to-create-a-script-statically-in-logstash-7-17/347446)
**Author:** [@getsolaris](https://discuss.elastic.co/u/getsolaris)\ **Replies:** 18\ **Last updated:** [November 18, 2023, 5:42pm UTC](https://discuss.elastic.co/t/how-to-create-a-script-statically-in-logstash-7-17/347446 "2023-11-18T17:42:50Z")
How to statically generate a script in Logstash 7.17 Hi, I would like to add elements to nested in elasticsearch via script via Logstash. Currently, compilations, cache\_evisions are increasing. I also increased max\_co… --- ## [Different values in new runs of logstash with Aggregate filter](https://discuss.elastic.co/t/different-values-in-new-runs-of-logstash-with-aggregate-filter/347451)
**Author:** [@Ilia](https://discuss.elastic.co/u/Ilia)\ **Replies:** 1\ **Last updated:** [November 18, 2023, 5:40pm UTC](https://discuss.elastic.co/t/different-values-in-new-runs-of-logstash-with-aggregate-filter/347451 "2023-11-18T17:40:10Z")
I've used jdbc input plugin to extract financial transactions from database. But each logical transaction is composed from multiple transactions so I used aggregate filter to merge them in one event. Here is the code of … --- ## [Not able to find logstash plain log in the var/log/logstash directory](https://discuss.elastic.co/t/not-able-to-find-logstash-plain-log-in-the-var-log-logstash-directory/347296)
**Author:** [@Domnic\_Raj\_D](https://discuss.elastic.co/u/Domnic_Raj_D)\ **Replies:** 5\ **Last updated:** [November 17, 2023, 6:44pm UTC](https://discuss.elastic.co/t/not-able-to-find-logstash-plain-log-in-the-var-log-logstash-directory/347296 "2023-11-17T18:44:50Z")
I am not able to find logstash-plain.log in the var/log/logstash directory. Configurations are set to write logs in /var/log/logstash directory. please advise. --- ## [Parsing multiline java execption](https://discuss.elastic.co/t/parsing-multiline-java-execption/347262)
**Author:** [@apsh](https://discuss.elastic.co/u/apsh)\ **Replies:** 3\ **Last updated:** [November 16, 2023, 7:20pm UTC](https://discuss.elastic.co/t/parsing-multiline-java-execption/347262 "2023-11-16T19:20:29Z")
Hello, I am trying to add multiline to handle javaexception in our logs but still having issue : This is my pattern : paths: - /var/log/tomcat10/\* multiline.type: pattern multiline.pattern: '^\\d{2}-\\w{3}-\\d{4… --- ## [Logstash re-ingests files](https://discuss.elastic.co/t/logstash-re-ingests-files/347358)
**Author:** [@hjsroldan](https://discuss.elastic.co/u/hjsroldan)\ **Replies:** 0\ **Last updated:** [November 16, 2023, 6:21pm UTC](https://discuss.elastic.co/t/logstash-re-ingests-files/347358 "2023-11-16T18:21:11Z")
Hi, I have a Logstash to Elasticsearch project where logstash collects all the logs from the server and pushes it to elasticsearch. However, seems like the Logstash ingests my logs multiple times. Ingested logs from yes… --- ## [Adding incremental column based on values of another column](https://discuss.elastic.co/t/adding-incremental-column-based-on-values-of-another-column/346443)
**Author:** [@Felicien\_Ihirwe](https://discuss.elastic.co/u/Felicien_Ihirwe)\ **Replies:** 2\ **Last updated:** [November 16, 2023, 3:13pm UTC](https://discuss.elastic.co/t/adding-incremental-column-based-on-values-of-another-column/346443 "2023-11-16T15:13:58Z")
I want to create a logstash filter to come from table 1 to table 2: col1 in in out in out .. . I want to add a new column that will contain incremental values and the data will look like col1 | col 2 in | … --- ## [Logstash pod is not coming up once the pipeline status is running](https://discuss.elastic.co/t/logstash-pod-is-not-coming-up-once-the-pipeline-status-is-running/347333)
**Author:** [@sathishkumarD](https://discuss.elastic.co/u/sathishkumarD)\ **Replies:** 0\ **Last updated:** [November 16, 2023, 1:04pm UTC](https://discuss.elastic.co/t/logstash-pod-is-not-coming-up-once-the-pipeline-status-is-running/347333 "2023-11-16T13:04:14Z")
Logstash is deployed in a kubernetes cluster and Elastic Search is deployed in another cluster. For transferring the log files from logstash to Elasticsearch using outputs in the logstash.conf file. // output { // … --- ## [Cannot determine timezone from nil logstash](https://discuss.elastic.co/t/cannot-determine-timezone-from-nil-logstash/347217)
**Author:** [@SamehSaeed](https://discuss.elastic.co/u/SamehSaeed)\ **Replies:** 9\ **Last updated:** [November 16, 2023, 12:27pm UTC](https://discuss.elastic.co/t/cannot-determine-timezone-from-nil-logstash/347217 "2023-11-16T12:27:06Z")
I'm getting an error while running logstash " (ArgumentError) Cannot determine timezone from nil\\n(secs:1700041898.446,utc~:"2023-11-15 09:51:38.4460000991821289",ltz~:nil)" I have tried solutions from other threads (a… --- ## [Connection reset between LogStash and ES](https://discuss.elastic.co/t/connection-reset-between-logstash-and-es/347315)
**Author:** [@Giuliano\_Dessimone](https://discuss.elastic.co/u/Giuliano_Dessimone)\ **Replies:** 0\ **Last updated:** [November 16, 2023, 10:06am UTC](https://discuss.elastic.co/t/connection-reset-between-logstash-and-es/347315 "2023-11-16T10:06:50Z")
Hello, has anyone ever had random "connection reset" errors between logstash and elastic using http\_poller? In a reliable and well-tested solution that implements ingestion to an ES cluster via http\_poller, we continuous… --- ## [Why logstash service not work correctly but it is running in the foreground](https://discuss.elastic.co/t/why-logstash-service-not-work-correctly-but-it-is-running-in-the-foreground/347211)
**Author:** [@baber1223](https://discuss.elastic.co/u/baber1223)\ **Replies:** 17\ **Last updated:** [November 16, 2023, 5:38am UTC](https://discuss.elastic.co/t/why-logstash-service-not-work-correctly-but-it-is-running-in-the-foreground/347211 "2023-11-16T05:38:59Z")
Hi. When I am running logstash in the foreground it is working excellent with follow command /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/pipeline1.conf --path.settings /etc/logstash/ but when I am running … --- ## [Logstash deletes source files and not creating index - date parsing issue](https://discuss.elastic.co/t/logstash-deletes-source-files-and-not-creating-index-date-parsing-issue/347243)
**Author:** [@derekmizak](https://discuss.elastic.co/u/derekmizak)\ **Replies:** 10\ **Last updated:** [November 15, 2023, 8:54pm UTC](https://discuss.elastic.co/t/logstash-deletes-source-files-and-not-creating-index-date-parsing-issue/347243 "2023-11-15T20:54:21Z")
I am using Elastic and logstash 8.11 running in docker. When logstash starts it deletes log files from the source directory but nothing is passed to elastic - no index is created. I am not sure why logstash is deleting… --- ## [How to handle replay of eventhub data](https://discuss.elastic.co/t/how-to-handle-replay-of-eventhub-data/347230)
**Author:** [@favetelinguis](https://discuss.elastic.co/u/favetelinguis)\ **Replies:** 0\ **Last updated:** [November 15, 2023, 2:02pm UTC](https://discuss.elastic.co/t/how-to-handle-replay-of-eventhub-data/347230 "2023-11-15T14:02:15Z")
I am currently running some disaster recovery tests on out logstash which uses the Azure Eventhub input plugin and elastic output. My test includes changing the URL to elastic so that sending will fail. However once I re… --- ## [Cannot create datastream under new index template](https://discuss.elastic.co/t/cannot-create-datastream-under-new-index-template/347126)
**Author:** [@Casper\_Thrane](https://discuss.elastic.co/u/Casper_Thrane)\ **Replies:** 5\ **Last updated:** [November 14, 2023, 3:17pm UTC](https://discuss.elastic.co/t/cannot-create-datastream-under-new-index-template/347126 "2023-11-14T15:17:49Z")
Hi I am using logstash running Kubernetes under ECK. I am ingesting both logs (filebeat) and metrics (metricbeat). Now i want ingest data from heartbeat. I get the following error: \[2023-11-14T13:22:23,598\]\[INFO \]\[logs… --- ## [Restart the Logstash 8.8 configuration without restarting it](https://discuss.elastic.co/t/restart-the-logstash-8-8-configuration-without-restarting-it/347095)
**Author:** [@Manal\_A](https://discuss.elastic.co/u/Manal_A)\ **Replies:** 2\ **Last updated:** [November 14, 2023, 2:34pm UTC](https://discuss.elastic.co/t/restart-the-logstash-8-8-configuration-without-restarting-it/347095 "2023-11-14T14:34:07Z")
Hello, Is there a way to reload the Logstash 8.8 configuration without restarting it? Thank you --- ## [Logstash Multiple Output Atomicity](https://discuss.elastic.co/t/logstash-multiple-output-atomicity/347100)
**Author:** [@mile3880](https://discuss.elastic.co/u/mile3880)\ **Replies:** 1\ **Last updated:** [November 14, 2023, 1:14pm UTC](https://discuss.elastic.co/t/logstash-multiple-output-atomicity/347100 "2023-11-14T13:14:48Z")
Does Logstash guarantee atomicity of multiple output options? For Example, if I set 2 different outputs to Elasticsearch A and B, and when connection to Elasticsearch A is temporarily unstable, can Logstash stop sending… --- ## [Field and Value missmatch Paolo Alto OS11 paring Logstash \> Elastic \> Kibana](https://discuss.elastic.co/t/field-and-value-missmatch-paolo-alto-os11-paring-logstash-elastic-kibana/346969)
**Author:** [@Trung\_Nguyen](https://discuss.elastic.co/u/Trung_Nguyen)\ **Replies:** 5\ **Last updated:** [November 14, 2023, 8:30am UTC](https://discuss.elastic.co/t/field-and-value-missmatch-paolo-alto-os11-paring-logstash-elastic-kibana/346969 "2023-11-14T08:30:59Z")
Hi, i'm a new logstash and trying to parsing log from my firewall PAN\_OS 11 but the field and value dose not match, such as field "NAT Destination IP" get value from the "Rule Name" Thanks a lot for any hlep Trung [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=50) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=52)