# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=54 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 55 --- ## [Split message in Logstash does not display complete message](https://discuss.elastic.co/t/split-message-in-logstash-does-not-display-complete-message/345936)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 4\ **Last updated:** [October 27, 2023, 5:59pm UTC](https://discuss.elastic.co/t/split-message-in-logstash-does-not-display-complete-message/345936 "2023-10-27T17:59:42Z")
Hi All, The stdout log of an application is tokenized with a delimiter ~|~. There are a total of 5 delimiters. An excerpt from log is as below: 2023-10-27 11:03:41,294~|~INFO~|~host.com~|~com.controller.LoginControll… --- ## [Logstash http\_poller pulling data recursively](https://discuss.elastic.co/t/logstash-http-poller-pulling-data-recursively/345742)
**Author:** [@Ankita\_Pachauri](https://discuss.elastic.co/u/Ankita_Pachauri)\ **Replies:** 3\ **Last updated:** [October 27, 2023, 5:28pm UTC](https://discuss.elastic.co/t/logstash-http-poller-pulling-data-recursively/345742 "2023-10-27T17:28:33Z")
Hi All, I have to use logstash to pull metric data using ManageEngine AppManager's REST API. The task has to be done in two phases, the first phase would be to pull all objects/resource ids under a specific group using … --- ## [Logstash-salesforce-input plugin interval refresh](https://discuss.elastic.co/t/logstash-salesforce-input-plugin-interval-refresh/345470)
**Author:** [@Samuele\_Lolli](https://discuss.elastic.co/u/Samuele_Lolli)\ **Replies:** 4\ **Last updated:** [October 27, 2023, 1:15pm UTC](https://discuss.elastic.co/t/logstash-salesforce-input-plugin-interval-refresh/345470 "2023-10-27T13:15:45Z")
Hi i have the same issue than @mPanasiewicz. Is there any way to refresh salesforce with a cron exp Continuing the discussion from Logstash-salesforce-input plugin interval refresh: Thanks in advance --- ## [Logstash JMX plugin offline installation - why the creator dont create a zip with all dependency for install offline?](https://discuss.elastic.co/t/logstash-jmx-plugin-offline-installation-why-the-creator-dont-create-a-zip-with-all-dependency-for-install-offline/345884)
**Author:** [@martel](https://discuss.elastic.co/u/martel)\ **Replies:** 0\ **Last updated:** [October 27, 2023, 6:58am UTC](https://discuss.elastic.co/t/logstash-jmx-plugin-offline-installation-why-the-creator-dont-create-a-zip-with-all-dependency-for-install-offline/345884 "2023-10-27T06:58:36Z")
Hey, Im very bored, I read the post about installing a plugin in offline mode, but why not force the creators to make a zip with everything needed to do it offline?? seriously.. only have one gem file and this one must… --- ## [JSON timestamp coming as text](https://discuss.elastic.co/t/json-timestamp-coming-as-text/345633)
**Author:** [@ataylor](https://discuss.elastic.co/u/ataylor)\ **Replies:** 6\ **Last updated:** [October 27, 2023, 12:45am UTC](https://discuss.elastic.co/t/json-timestamp-coming-as-text/345633 "2023-10-27T00:45:25Z")
Basically I am not getting a @timestamp field that is a date - its coming through as text. #this is my .conf file (with some stuff excluded for security) input{ beats{ port =\> 5048 } } filter { json { … --- ## [Extract hostname from log file name](https://discuss.elastic.co/t/extract-hostname-from-log-file-name/345761)
**Author:** [@Indeed2000](https://discuss.elastic.co/u/Indeed2000)\ **Replies:** 9\ **Last updated:** [October 26, 2023, 8:44pm UTC](https://discuss.elastic.co/t/extract-hostname-from-log-file-name/345761 "2023-10-26T20:44:26Z")
Hi on logstash need to use file as input, output as http. now question is how can i extract hostname from log filename, here is file name: /tmp/log.hostname1.20230720 /tmp/log.hostname2.20230720 Any idea Thanks --- ## [Logstash Service With Plugin that close after finish](https://discuss.elastic.co/t/logstash-service-with-plugin-that-close-after-finish/345820)
**Author:** [@Samuele\_Lolli](https://discuss.elastic.co/u/Samuele_Lolli)\ **Replies:** 0\ **Last updated:** [October 26, 2023, 1:13pm UTC](https://discuss.elastic.co/t/logstash-service-with-plugin-that-close-after-finish/345820 "2023-10-26T13:13:58Z")
Hi everyone, i have a quick question. I created a pipeline that after completition end by closing the prompt and my current configuration is logstash as a service in a linux server. What happens if i add the pipeline t… --- ## [Anonymize part of string](https://discuss.elastic.co/t/anonymize-part-of-string/345631)
**Author:** [@ddoroshenko](https://discuss.elastic.co/u/ddoroshenko)\ **Replies:** 3\ **Last updated:** [October 26, 2023, 12:30pm UTC](https://discuss.elastic.co/t/anonymize-part-of-string/345631 "2023-10-26T12:30:56Z")
Hi, I have access logs which contains sensitive data \[2023-00-00T00:00:00.000\] ... "GET /example.com/foo/bar?password=SecretPassword&user=UserName" ... Is it possible to anonymize password value in that string? --- ## [Logstash stopped processing because of an error: (LoadError) failure to load file: java.io.FileNotFoundException: /usr/share/logstash/logstash-core/lib/logstash/build.rb (Permission denied)](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-loaderror-failure-to-load-file-java-io-filenotfoundexception-usr-share-logstash-logstash-core-lib-logstash-build-rb-permission-denied/345801)
**Author:** [@jrajasek](https://discuss.elastic.co/u/jrajasek)\ **Replies:** 0\ **Last updated:** [October 26, 2023, 11:34am UTC](https://discuss.elastic.co/t/logstash-stopped-processing-because-of-an-error-loaderror-failure-to-load-file-java-io-filenotfoundexception-usr-share-logstash-logstash-core-lib-logstash-build-rb-permission-denied/345801 "2023-10-26T11:34:38Z")
Building the custom docker image with these below commands. FROM docker.elastic.co/logstash/logstash:8.10.4 RUN rm -f /usr/share/logstash/pipeline/logstash.conf COPY pipeline/ /usr/share/logstash/pipeline/ COPY confi… --- ## [Redirecting postgresql tables to Elasticsearch](https://discuss.elastic.co/t/redirecting-postgresql-tables-to-elasticsearch/345458)
**Author:** [@krzychohoho](https://discuss.elastic.co/u/krzychohoho)\ **Replies:** 1\ **Last updated:** [October 25, 2023, 5:21pm UTC](https://discuss.elastic.co/t/redirecting-postgresql-tables-to-elasticsearch/345458 "2023-10-25T17:21:22Z")
Hi, I am trying to redirect my postgresql tables to elasticsearch using JDBC and Logstash. I was able to do it but i came across a problem of ingesting the same data over and over again. I know i need to use tracking co… --- ## [Remove HTTP encondings](https://discuss.elastic.co/t/remove-http-encondings/345720)
**Author:** [@lemospt](https://discuss.elastic.co/u/lemospt)\ **Replies:** 3\ **Last updated:** [October 25, 2023, 5:03pm UTC](https://discuss.elastic.co/t/remove-http-encondings/345720 "2023-10-25T17:03:12Z")
Hi guys, i'm integrating log from proxy squid, there is a field called 'Original Received Request Header' that has data like below, User-Agent:%20git/2.30.2%0D%0AProxy-Connection:%20Keep-Alive%0D%0AHost:%20github.priva… --- ## [Logstash fails with "FFI not available" message when starting logstash on Centos 7.9](https://discuss.elastic.co/t/logstash-fails-with-ffi-not-available-message-when-starting-logstash-on-centos-7-9/345387)
**Author:** [@shaigbdb](https://discuss.elastic.co/u/shaigbdb)\ **Replies:** 2\ **Last updated:** [October 25, 2023, 3:37pm UTC](https://discuss.elastic.co/t/logstash-fails-with-ffi-not-available-message-when-starting-logstash-on-centos-7-9/345387 "2023-10-25T15:37:59Z")
1. Logstash version (e.g. bin/logstash --version) - 8.10.2 \*\*2. Logstash installation source \*\* - RPM \*\*3. How is Logstash being run \*\* - systemd JVM - tried both the bundled JVM (openjdk version "17.0.8" 2023-07-18) … --- ## [Logstash 8.6 low performance](https://discuss.elastic.co/t/logstash-8-6-low-performance/344661)
**Author:** [@RobertC1](https://discuss.elastic.co/u/RobertC1)\ **Replies:** 8\ **Last updated:** [October 25, 2023, 2:24pm UTC](https://discuss.elastic.co/t/logstash-8-6-low-performance/344661 "2023-10-25T14:24:09Z")
Hi there I ha a server with Linux Ubuntu 20.04 and ELK 8.6 I noticed that the ingestion proccess became slow and I have not change any parameters. This is the conf file for theindex. input { file { … --- ## [Logstash refusing connection error](https://discuss.elastic.co/t/logstash-refusing-connection-error/345708)
**Author:** [@vaishalik03](https://discuss.elastic.co/u/vaishalik03)\ **Replies:** 2\ **Last updated:** [October 25, 2023, 12:43pm UTC](https://discuss.elastic.co/t/logstash-refusing-connection-error/345708 "2023-10-25T12:43:08Z")
Hi All, I'm trying to connect to Kibana after running the logstash but could see that the connection is getting refused after running the file. Following are the responses on batch. Could you please let me know when a… --- ## [Install logstash-integration-jdbc" exit code: 137 - Dockerfile](https://discuss.elastic.co/t/install-logstash-integration-jdbc-exit-code-137-dockerfile/345719)
**Author:** [@Animesh\_Pathak](https://discuss.elastic.co/u/Animesh_Pathak)\ **Replies:** 0\ **Last updated:** [October 25, 2023, 10:45am UTC](https://discuss.elastic.co/t/install-logstash-integration-jdbc-exit-code-137-dockerfile/345719 "2023-10-25T10:45:21Z")
This is my dockerfile, i have my docker-compose.yml file where i'm building this file using docker-compose up --build. But each time it fails showing exit code 137 Dockerfile FROM docker.elastic.co/logstash/logstash:8.1… --- ## [Logstash does not support to read logs from multiple Docker containers](https://discuss.elastic.co/t/logstash-does-not-support-to-read-logs-from-multiple-docker-containers/345542)
**Author:** [@talbehat](https://discuss.elastic.co/u/talbehat)\ **Replies:** 2\ **Last updated:** [October 25, 2023, 10:13am UTC](https://discuss.elastic.co/t/logstash-does-not-support-to-read-logs-from-multiple-docker-containers/345542 "2023-10-25T10:13:46Z")
I have multiple docker containers in host. For example :- tomcat process docker container, elasticsearch process docker container, postgresql process docker container. And Logstash are running in seperate docker cont… --- ## [@timestamp long vs. string format](https://discuss.elastic.co/t/timestamp-long-vs-string-format/345663)
**Author:** [@tlacuache](https://discuss.elastic.co/u/tlacuache)\ **Replies:** 7\ **Last updated:** [October 24, 2023, 9:37pm UTC](https://discuss.elastic.co/t/timestamp-long-vs-string-format/345663 "2023-10-24T21:37:43Z")
I'm working on a project that has a few different components that both write documents to the same indices. One of these components is Arkime, which writes its documents' @timestamp date as UNIX milliseconds value. This … --- ## [Splitting Logstash message](https://discuss.elastic.co/t/splitting-logstash-message/345597)
**Author:** [@joecarter](https://discuss.elastic.co/u/joecarter)\ **Replies:** 5\ **Last updated:** [October 24, 2023, 3:37pm UTC](https://discuss.elastic.co/t/splitting-logstash-message/345597 "2023-10-24T15:37:38Z")
I am pulling events from an Azure Event Hub, but some of the events are being grouped into a single message containing an array of "records", which I want to be processed as individual messages. The format is: { timest… --- ## [Exception when executing JDBC query exception=\>Sequel::DatabaseError, :message=\>"Java::ComMicrosoftSqlserverJdbc::SQLServerException: Connection reset", :cause=\>"# \ **Author:** [@Vishweshwar](https://discuss.elastic.co/u/Vishweshwar)\ **Replies:** 1\ **Last updated:** [October 24, 2023, 7:57am UTC](https://discuss.elastic.co/t/exception-when-executing-jdbc-query-exception-sequel-databaseerror-message-java-connection-reset-cause-java-connection-reset/345562 "2023-10-24T07:57:36Z") For some time i am able to connect DB but after few seconds i get the error "Exception when executing JDBC query " My logstash config format: input { jdbc { tags =\> "index.conf" jdbc\_connection\_string =\> "jdbc:sqlse… --- ## [Help with Logstash file input](https://discuss.elastic.co/t/help-with-logstash-file-input/345475)
**Author:** [@roman-tasi](https://discuss.elastic.co/u/roman-tasi)\ **Replies:** 11\ **Last updated:** [October 24, 2023, 5:01am UTC](https://discuss.elastic.co/t/help-with-logstash-file-input/345475 "2023-10-24T05:01:02Z")
I am not receiving the contents of fortune.txt for my ELK implementation. This is the input section: file { path =\> "/etc/elasticsearch/scripts/otherScripts/fortune.txt" sincedb\_path =\> "/dev/null" sta… --- ## [Logstash / Docker / Root (RW) access](https://discuss.elastic.co/t/logstash-docker-root-rw-access/345600)
**Author:** [@Phildefer](https://discuss.elastic.co/u/Phildefer)\ **Replies:** 0\ **Last updated:** [October 23, 2023, 10:12pm UTC](https://discuss.elastic.co/t/logstash-docker-root-rw-access/345600 "2023-10-23T22:12:59Z")
Hi, I'd like to have a root access to my container logstash. I would like to do things that the logstash user doesn't allow (updating packages with apt update, adding packages like nano with apt install nano, etc). Ho… --- ## [Fails to receive any log events，when two piplines using the same input port 5044](https://discuss.elastic.co/t/fails-to-receive-any-log-events-when-two-piplines-using-the-same-input-port-5044/345564)
**Author:** [@zhsongbj](https://discuss.elastic.co/u/zhsongbj)\ **Replies:** 2\ **Last updated:** [October 23, 2023, 2:50pm UTC](https://discuss.elastic.co/t/fails-to-receive-any-log-events-when-two-piplines-using-the-same-input-port-5044/345564 "2023-10-23T14:50:04Z")
I encountered a troubling issue for which I'd like to express my gratitude to anyone who can help. One pipeline consistently failed to receive log events. The problem occurred when two pipelines used the same input port,… --- ## [Unable to segregate messages from two Input files](https://discuss.elastic.co/t/unable-to-segregate-messages-from-two-input-files/345492)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 3\ **Last updated:** [October 21, 2023, 1:26pm UTC](https://discuss.elastic.co/t/unable-to-segregate-messages-from-two-input-files/345492 "2023-10-21T13:26:44Z")
Hi Team, I posted this message on stack but not getting any replies. Can someone please help? I need help in seggregrating messages from my two different conf files. I am bit confused about ingestion Here is my first f… --- ## [How do I stringify entire event object in logstash and put it in one field](https://discuss.elastic.co/t/how-do-i-stringify-entire-event-object-in-logstash-and-put-it-in-one-field/345496)
**Author:** [@ghanshyam\_baviskar](https://discuss.elastic.co/u/ghanshyam_baviskar)\ **Replies:** 4\ **Last updated:** [October 21, 2023, 1:24pm UTC](https://discuss.elastic.co/t/how-do-i-stringify-entire-event-object-in-logstash-and-put-it-in-one-field/345496 "2023-10-21T13:24:30Z")
I am trying to implement a dead letter queue pipeline, I want to take entire event , stringify it and put it into a field "strigified\_event". so that it can be monitored for elasticsearch mapper errors input { dead\_le… --- ## [Logstash failing to starting due to the error related to the "i18n" gem](https://discuss.elastic.co/t/logstash-failing-to-starting-due-to-the-error-related-to-the-i18n-gem/345341)
**Author:** [@akhilatham](https://discuss.elastic.co/u/akhilatham)\ **Replies:** 3\ **Last updated:** [October 21, 2023, 12:41am UTC](https://discuss.elastic.co/t/logstash-failing-to-starting-due-to-the-error-related-to-the-i18n-gem/345341 "2023-10-21T00:41:35Z")
I am getting the below error: \[2023-10-18T17:37:02,573\]\[FATAL\]\[logstash.runner\] An unexpected error occurred! {:error=\>#\, :backtrace=\>\["/usr/share/logst… --- ## [Output syslog plugin \[Unable to load plugin\]](https://discuss.elastic.co/t/output-syslog-plugin-unable-to-load-plugin/344676)
**Author:** [@ans\_k](https://discuss.elastic.co/u/ans_k)\ **Replies:** 5\ **Last updated:** [October 20, 2023, 6:17pm UTC](https://discuss.elastic.co/t/output-syslog-plugin-unable-to-load-plugin/344676 "2023-10-20T18:17:05Z")
Hello, I followed this documentation : to install offline output syslog plugin in my machine, the plugin is successfully installed, but when i try to call syslog as output in my config file (logstash), i got "Unable … --- ## [Installing Logstash plugin while service is running](https://discuss.elastic.co/t/installing-logstash-plugin-while-service-is-running/345469)
**Author:** [@Samuele\_Lolli](https://discuss.elastic.co/u/Samuele_Lolli)\ **Replies:** 2\ **Last updated:** [October 20, 2023, 2:34pm UTC](https://discuss.elastic.co/t/installing-logstash-plugin-while-service-is-running/345469 "2023-10-20T14:34:14Z")
Hi everyone, just one quick question. I have on a linux server logstash running with systemctl. I need to try some new pipelines but i need to add a plugin. I can add a new plugin while the service is running? The plug… --- ## [Elasticsearch is processing incoming events/messages from Logstash in a non-sequential order](https://discuss.elastic.co/t/elasticsearch-is-processing-incoming-events-messages-from-logstash-in-a-non-sequential-order/345295)
**Author:** [@Aman\_Yadav1](https://discuss.elastic.co/u/Aman_Yadav1)\ **Replies:** 3\ **Last updated:** [October 20, 2023, 6:44am UTC](https://discuss.elastic.co/t/elasticsearch-is-processing-incoming-events-messages-from-logstash-in-a-non-sequential-order/345295 "2023-10-20T06:44:27Z")
We have a system that synchronises data from MongoDB to Elasticsearch . Here are the key components: MongoDB Source Connector: This component reads events from the MongoDB oplog and produces messages on a Kafka topic. L… --- ## [Error json parsing opensearch logs with logstash](https://discuss.elastic.co/t/error-json-parsing-opensearch-logs-with-logstash/345398)
**Author:** [@Xhar](https://discuss.elastic.co/u/Xhar)\ **Replies:** 3\ **Last updated:** [October 20, 2023, 6:44am UTC](https://discuss.elastic.co/t/error-json-parsing-opensearch-logs-with-logstash/345398 "2023-10-20T06:44:21Z")
Hello, i'm trying to parse suricata, logstash and opensearch logs with dictionary filter, here's part of my config input { file { path =\> "/opt/logs/opensearchTest/opensearch\_server.json" codec =\> "json" t… --- ## [Dissect - Pipeline in Logstash](https://discuss.elastic.co/t/dissect-pipeline-in-logstash/345315)
**Author:** [@lucasyuki](https://discuss.elastic.co/u/lucasyuki)\ **Replies:** 3\ **Last updated:** [October 19, 2023, 2:06pm UTC](https://discuss.elastic.co/t/dissect-pipeline-in-logstash/345315 "2023-10-19T14:06:26Z")
Hi team , im trying to dissect the message log I'm trying to parse the "Message" column, I tested it in Elastic cloud and the command in the ingest pipelines tab was working My config in logstash conf.d input { syslo… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=53) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=55)