# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=15 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 16 --- ## [Forward logstash logs to another SIEM](https://discuss.elastic.co/t/forward-logstash-logs-to-another-siem/370959)
**Author:** [@Bantou0](https://discuss.elastic.co/u/Bantou0)\ **Replies:** 2\ **Last updated:** [November 22, 2024, 2:52pm UTC](https://discuss.elastic.co/t/forward-logstash-logs-to-another-siem/370959 "2024-11-22T14:52:57Z")
Hi everyone, I am facing issues with our new servers which can only have one syslog server configured. We are using two SIEM to send logs (logstash and FSIEM). So I would like to configure the logstash virtual IP as the… --- ## [Questions about DNS filter and multiple pipelines](https://discuss.elastic.co/t/questions-about-dns-filter-and-multiple-pipelines/370952)
**Author:** [@Po-temkin](https://discuss.elastic.co/u/Po-temkin)\ **Replies:** 2\ **Last updated:** [November 22, 2024, 2:42pm UTC](https://discuss.elastic.co/t/questions-about-dns-filter-and-multiple-pipelines/370952 "2024-11-22T14:42:35Z")
Hello to everyone! I plan to use Logstash to transform syslog messages into info files To achieve, it I decided to use multi-pipeline configuration: pipelines.yml - pipeline.id: ans\_file\_syslog path.config: "C:/Pro… --- ## [Use host IP (or dns) in the output file path](https://discuss.elastic.co/t/use-host-ip-or-dns-in-the-output-file-path/370944)
**Author:** [@Po-temkin](https://discuss.elastic.co/u/Po-temkin)\ **Replies:** 2\ **Last updated:** [November 22, 2024, 11:21am UTC](https://discuss.elastic.co/t/use-host-ip-or-dns-in-the-output-file-path/370944 "2024-11-22T11:21:57Z")
Hello to everyone! My question looks like obvious, but I missed something basic and stuck =) I have a UDP input that I want to output as a file My desired output file path is: E:/logstash/log/$host\_ip$/$date$.log It… --- ## [Logstask parse failure all the sudden](https://discuss.elastic.co/t/logstask-parse-failure-all-the-sudden/370907)
**Author:** [@devops\_training](https://discuss.elastic.co/u/devops_training)\ **Replies:** 2\ **Last updated:** [November 21, 2024, 6:34pm UTC](https://discuss.elastic.co/t/logstask-parse-failure-all-the-sudden/370907 "2024-11-21T18:34:38Z")
log event "2024-11-21 17:12:48|INFO|endpoint|ar-endpoint|AWS||null|null||Tenant GUID \[ujwujqllkll-232-11919191\], event GUID \[\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\]: Consumed record: thread\[0\], offset\[615049\], key\[null\]" Grok pattern in… --- ## [Ingest multiline auditd events from syslog sources](https://discuss.elastic.co/t/ingest-multiline-auditd-events-from-syslog-sources/370904)
**Author:** [@gamb](https://discuss.elastic.co/u/gamb)\ **Replies:** 0\ **Last updated:** [November 21, 2024, 4:36pm UTC](https://discuss.elastic.co/t/ingest-multiline-auditd-events-from-syslog-sources/370904 "2024-11-21T16:36:32Z")
Hello, I would like to merge (or aggregate) auditd logs with logstash. I have see lot of solutions with an agent but it seem tricky to do with a syslog source :frowning: I received auditd messages form multiple sources… --- ## [Using syslog input plugin not working](https://discuss.elastic.co/t/using-syslog-input-plugin-not-working/370856)
**Author:** [@yquirion](https://discuss.elastic.co/u/yquirion)\ **Replies:** 2\ **Last updated:** [November 20, 2024, 8:43pm UTC](https://discuss.elastic.co/t/using-syslog-input-plugin-not-working/370856 "2024-11-20T20:43:27Z")
Dear all, I'm having a strange issue. I have a pipeline listening for syslog data. When the data comes in, I just see there is a new connection coming from the client (the syslog server) and the connection if close. No… --- ## [How to read UTF-16LE encoded CSV files using file input plugin](https://discuss.elastic.co/t/how-to-read-utf-16le-encoded-csv-files-using-file-input-plugin/370765)
**Author:** [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)\ **Replies:** 1\ **Last updated:** [November 19, 2024, 3:17pm UTC](https://discuss.elastic.co/t/how-to-read-utf-16le-encoded-csv-files-using-file-input-plugin/370765 "2024-11-19T15:17:40Z")
Hi Team, I wanted to read the CSV files which are encoded in UTF-16LE or UTF-8 in logstash. Could you please help me how i can achieve it. I have tried like this which is failing. input { file { id =\> "nasapmena… --- ## [What could be the problem here trying to connect logstash to ElasticSearch 7.5 for the first time?](https://discuss.elastic.co/t/what-could-be-the-problem-here-trying-to-connect-logstash-to-elasticsearch-7-5-for-the-first-time/370714)
**Author:** [@mpniel](https://discuss.elastic.co/u/mpniel)\ **Replies:** 9\ **Last updated:** [November 19, 2024, 12:41pm UTC](https://discuss.elastic.co/t/what-could-be-the-problem-here-trying-to-connect-logstash-to-elasticsearch-7-5-for-the-first-time/370714 "2024-11-19T12:41:22Z")
What could be the problem here trying to connect logstash to Elasticsearch 7.5 for the first time? Nov 18 11:55:34 srLogStash001 run\_tviLogStash.sh\[98904\]: \[2024-11-18T11:55:34,362\]\[WARN \]\[logstash.outputs.elasticsearch… --- ## [Logstash RabbitMQ Output Plugin](https://discuss.elastic.co/t/logstash-rabbitmq-output-plugin/370757)
**Author:** [@Kris9](https://discuss.elastic.co/u/Kris9)\ **Replies:** 0\ **Last updated:** [November 19, 2024, 9:30am UTC](https://discuss.elastic.co/t/logstash-rabbitmq-output-plugin/370757 "2024-11-19T09:30:58Z")
Hello, I am using Logstash with the JDBC input plugin to query for the top entries in a database table and then push them into RabbitMQ using the RabbitMQ output plugin. I have noticed that when the queue is down, Logst… --- ## [Logstash rereading the files even after since\_db has that file entry which was already read](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675)
**Author:** [@venkatkumar229](https://discuss.elastic.co/u/venkatkumar229)\ **Replies:** 7\ **Last updated:** [November 18, 2024, 5:48pm UTC](https://discuss.elastic.co/t/logstash-rereading-the-files-even-after-since-db-has-that-file-entry-which-was-already-read/370675 "2024-11-18T17:48:36Z")
Hi Team, I am trying to read the files from a particular path using logstash file input plugin and i am using file\_completed\_path and since\_db path to track the list of files which are processed. It was working fine and… --- ## [Encrypting Logstash pipeline output](https://discuss.elastic.co/t/encrypting-logstash-pipeline-output/370613)
**Author:** [@Richard\_LaRoche](https://discuss.elastic.co/u/Richard_LaRoche)\ **Replies:** 5\ **Last updated:** [November 18, 2024, 4:10pm UTC](https://discuss.elastic.co/t/encrypting-logstash-pipeline-output/370613 "2024-11-18T16:10:02Z")
I am doing something new with my logstash collectors that I recently built and are working quite well. I need to start encrypting the data I am sending to another part of my organization. I have been looking at pretty … --- ## [Permission problem when starting logstash with systemd on suse](https://discuss.elastic.co/t/permission-problem-when-starting-logstash-with-systemd-on-suse/370219)
**Author:** [@STOV](https://discuss.elastic.co/u/STOV)\ **Replies:** 8\ **Last updated:** [November 18, 2024, 8:32am UTC](https://discuss.elastic.co/t/permission-problem-when-starting-logstash-with-systemd-on-suse/370219 "2024-11-18T08:32:37Z")
Hello all, I'm having trouble getting a logstash instance to work sending events to Elasticsearch. I've troubleshooted quite far and narrowed it down to a problem somehow related to permissions but let me get into the s… --- ## [Logstash filter for slow query logs](https://discuss.elastic.co/t/logstash-filter-for-slow-query-logs/370586)
**Author:** [@FJT](https://discuss.elastic.co/u/FJT)\ **Replies:** 2\ **Last updated:** [November 18, 2024, 5:06am UTC](https://discuss.elastic.co/t/logstash-filter-for-slow-query-logs/370586 "2024-11-18T05:06:18Z")
Hello can you help me and I am new to ELK. I am using a filebeat to send to logstash. and i am trying to parse my slow query log can you help me in creating a grok pattern. So what I want to achieve was to have a separa… --- ## [Pipeline error reader unacceptable code point ' ' (0x0) special characters are not allowed](https://discuss.elastic.co/t/pipeline-error-reader-unacceptable-code-point-0x0-special-characters-are-not-allowed/370546)
**Author:** [@SamehSaeed](https://discuss.elastic.co/u/SamehSaeed)\ **Replies:** 3\ **Last updated:** [November 17, 2024, 1:58pm UTC](https://discuss.elastic.co/t/pipeline-error-reader-unacceptable-code-point-0x0-special-characters-are-not-allowed/370546 "2024-11-17T13:58:48Z")
Hello, I have a pipeline that was working for almost a year, then 4 days ago we restarted elasticsearch as it was taking so much RAM (exceeding jvm.heap size) and the pipeline stopped working and the below error popped … --- ## [I want to recreate the java heap memory issue which is happening in production](https://discuss.elastic.co/t/i-want-to-recreate-the-java-heap-memory-issue-which-is-happening-in-production/370616)
**Author:** [@devops\_training](https://discuss.elastic.co/u/devops_training)\ **Replies:** 2\ **Last updated:** [November 17, 2024, 11:14am UTC](https://discuss.elastic.co/t/i-want-to-recreate-the-java-heap-memory-issue-which-is-happening-in-production/370616 "2024-11-17T11:14:10Z")
Testing in windows instance C:\\monitoring\\logstash\\logstash-8.15.1\\config\\jvm.options Change the below parameters JVM configuration Xms represents the initial size of total heap space Xmx represents the maximum size o… --- ## [Logstash doesn't send the logs to Elastic Cloud](https://discuss.elastic.co/t/logstash-doesnt-send-the-logs-to-elastic-cloud/370426)
**Author:** [@sevbans](https://discuss.elastic.co/u/sevbans)\ **Replies:** 1\ **Last updated:** [November 16, 2024, 5:04pm UTC](https://discuss.elastic.co/t/logstash-doesnt-send-the-logs-to-elastic-cloud/370426 "2024-11-16T17:04:52Z")
I'm trying to send some index dedicated elasticsearch cluster to elasticsearch cloud. sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash-sample.conf Using bundled JDK: /usr/share/logstash/jdk WARNI… --- ## [Logstash file input periodically skips over files](https://discuss.elastic.co/t/logstash-file-input-periodically-skips-over-files/370583)
**Author:** [@ddiguru](https://discuss.elastic.co/u/ddiguru)\ **Replies:** 7\ **Last updated:** [November 15, 2024, 5:02pm UTC](https://discuss.elastic.co/t/logstash-file-input-periodically-skips-over-files/370583 "2024-11-15T17:02:54Z")
I am attempting to use the file input module to "watch" a directory for inbound files. Gzipped log files are shipped to this directory from various remote hosts, and the file input picks them up, decompresses or processe… --- ## [Logstash sizing for a large environment](https://discuss.elastic.co/t/logstash-sizing-for-a-large-environment/370558)
**Author:** [@liuke](https://discuss.elastic.co/u/liuke)\ **Replies:** 3\ **Last updated:** [November 15, 2024, 10:25am UTC](https://discuss.elastic.co/t/logstash-sizing-for-a-large-environment/370558 "2024-11-15T10:25:07Z")
Hello everyone, i'm trying to setup an ELK stack that has to receive approximately 4.5TB of events per day, with peaks of 100k EPS during working hours. The collection must be done using logstash and all the events will… --- ## [How to update Index using "script" in logstash output](https://discuss.elastic.co/t/how-to-update-index-using-script-in-logstash-output/370535)
**Author:** [@vijay117](https://discuss.elastic.co/u/vijay117)\ **Replies:** 1\ **Last updated:** [November 14, 2024, 11:20am UTC](https://discuss.elastic.co/t/how-to-update-index-using-script-in-logstash-output/370535 "2024-11-14T11:20:11Z")
I want to update an Index based on session id present in the data i'm reading so in the current data i would be reading transaction number, createdate and also session id, if this session id matches with the session id … --- ## [Line Break Not Working](https://discuss.elastic.co/t/line-break-not-working/370505)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 3\ **Last updated:** [November 13, 2024, 11:50pm UTC](https://discuss.elastic.co/t/line-break-not-working/370505 "2024-11-13T23:50:58Z")
Good Day, How do you output new line with a break within Logstash. I am currently trying to do this with the following line from my filter: event.set('sender\_domain', sender\_domain.join('\\n')) I want to the output of… --- ## [Output to elasticsearch hitting error 409](https://discuss.elastic.co/t/output-to-elasticsearch-hitting-error-409/370483)
**Author:** [@tbrettinger](https://discuss.elastic.co/u/tbrettinger)\ **Replies:** 4\ **Last updated:** [November 13, 2024, 11:17pm UTC](https://discuss.elastic.co/t/output-to-elasticsearch-hitting-error-409/370483 "2024-11-13T23:17:55Z")
Hi, i'm rather new to elasticsearch, so I guess I am missing something obvious. I am trying to migrate data from opensearch to elasticsearch, following allong this article: After some fiddling, I got logstash to read… --- ## [Logstash 8.15.2 Error :exception=\>#\ **Author:** [@carellevbt](https://discuss.elastic.co/u/carellevbt)\ **Replies:** 2\ **Last updated:** [November 13, 2024, 3:01pm UTC](https://discuss.elastic.co/t/logstash-8-15-2-error-exception-errno-no-such-file-or-directory-no-such-file-or-directory-tmp-logstash-applog-iq-42dac6d2-4c55-4e75-93d8-044b9f04e1ab/370146 "2024-11-13T15:01:50Z") Hello, We recently split up one big pipeline we had into 7 pipelines. Ever since we did that, we get the below error when doing a deployment. Pipeline error {:pipeline\_id=\>"applog-iq", :exception=\>#\ **Author:** [@zadkiel\_meta](https://discuss.elastic.co/u/zadkiel_meta)\ **Replies:** 1\ **Last updated:** [November 12, 2024, 3:39pm UTC](https://discuss.elastic.co/t/how-to-convert-timestamp-value-to-yyyymm-in-logstash-pipeline/370408 "2024-11-12T15:39:17Z") I am very new joiner in logstash pipeline: I want to indexed @timestamp value format as yyyyMM, example "202411" Incomming BillingMonth value ="202411" Logstash Pipeline: filter { event.set("@timestamp", event.get("… --- ## [Logstash errors with writing plugins](https://discuss.elastic.co/t/logstash-errors-with-writing-plugins/370414)
**Author:** [@DOkuwa](https://discuss.elastic.co/u/DOkuwa)\ **Replies:** 0\ **Last updated:** [November 12, 2024, 3:25pm UTC](https://discuss.elastic.co/t/logstash-errors-with-writing-plugins/370414 "2024-11-12T15:25:53Z")
I am creating a logstash plugins while running bundle exec rspec command Failure/Error: raise LogStash::ConfigurationError, I18n.t("logstash.runner.configuration.invalid\_plugin\_settings") LogS… --- ## [Logstash : Invalid FieldReference: \`${http.headers\[\\"User-Agent\\"\]}\`"](https://discuss.elastic.co/t/logstash-invalid-fieldreference-http-headers-user-agent/370403)
**Author:** [@rehanpfmr](https://discuss.elastic.co/u/rehanpfmr)\ **Replies:** 1\ **Last updated:** [November 12, 2024, 3:23pm UTC](https://discuss.elastic.co/t/logstash-invalid-fieldreference-http-headers-user-agent/370403 "2024-11-12T15:23:21Z")
I'm having difficulty in parsing json object, the json field name is having special characters in it and logstash is showing error as JSON parse error, original data now in message field. ERROR message: JSON parse erro… --- ## [Control Sequence of logstash pipelines](https://discuss.elastic.co/t/control-sequence-of-logstash-pipelines/370318)
**Author:** [@vvavad](https://discuss.elastic.co/u/vvavad)\ **Replies:** 3\ **Last updated:** [November 12, 2024, 12:56pm UTC](https://discuss.elastic.co/t/control-sequence-of-logstash-pipelines/370318 "2024-11-12T12:56:53Z")
I want to write logstash pipelines to process csv files and intermediate Elasticsearch index. Below I have explained exact use case: pipeline 1 starts automatically when a file is detected in specified folder. It shoul… --- ## [Filter ssh events](https://discuss.elastic.co/t/filter-ssh-events/370340)
**Author:** [@mariano\_kabakian](https://discuss.elastic.co/u/mariano_kabakian)\ **Replies:** 2\ **Last updated:** [November 12, 2024, 11:02am UTC](https://discuss.elastic.co/t/filter-ssh-events/370340 "2024-11-12T11:02:49Z")
hi ELK gurus, hope you are having a good day. I'm running ELK inside docker. It's working fine, but I'm having an issue. I have a server that send events using filebeat. The thing is I cannot auth events to create a dash… --- ## [Removing New Line Characters in http output](https://discuss.elastic.co/t/removing-new-line-characters-in-http-output/370336)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 1\ **Last updated:** [November 11, 2024, 5:43pm UTC](https://discuss.elastic.co/t/removing-new-line-characters-in-http-output/370336 "2024-11-11T17:43:21Z")
Good Day, Is there a way to remove a New Line Characters from the http\_output command when sending to a server via API. The system receiving the data has to accept the data as a single string of text and it has to be p… --- ## [Running 3 queries in logstash](https://discuss.elastic.co/t/running-3-queries-in-logstash/370304)
**Author:** [@Santanu112](https://discuss.elastic.co/u/Santanu112)\ **Replies:** 1\ **Last updated:** [November 11, 2024, 1:26pm UTC](https://discuss.elastic.co/t/running-3-queries-in-logstash/370304 "2024-11-11T13:26:08Z")
HI , I want to use below sql queries :slight\_smile: select id form table 1 (it will give millions id) select id, name from table 2 select id , location from table 3 My doc should look like below : { id :1 name : a… --- ## [Logstash input plugin development - no examples found](https://discuss.elastic.co/t/logstash-input-plugin-development-no-examples-found/370312)
**Author:** [@DOkuwa](https://discuss.elastic.co/u/DOkuwa)\ **Replies:** 0\ **Last updated:** [November 11, 2024, 11:48am UTC](https://discuss.elastic.co/t/logstash-input-plugin-development-no-examples-found/370312 "2024-11-11T11:48:06Z")
Hello, I am writing my own plugin and when testing it is not showing any example in the code i will this text as BOLD as i am sending the .rb in the spec and lib directory \[root@Daley-DB-second inputs\]# bundle exec r… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=14) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=16)