# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=73 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 74 --- ## [Logstash unable to parse specific format of log](https://discuss.elastic.co/t/logstash-unable-to-parse-specific-format-of-log/334815)
**Author:** [@SmoZyNS](https://discuss.elastic.co/u/SmoZyNS)\ **Replies:** 11\ **Last updated:** [June 2, 2023, 1:35pm UTC](https://discuss.elastic.co/t/logstash-unable-to-parse-specific-format-of-log/334815 "2023-06-02T13:35:33Z")
Hello I am looking for some help since getting some headaches when trying to parse some logs Raw logs cs1Label=username cs1=/test@test.com cn1Label=actionSuccess cn1=1 deviceCustomDate1Label=userActionTime deviceCusto… --- ## [Help with creating a Logstash configuration file for Postfix log analysis](https://discuss.elastic.co/t/help-with-creating-a-logstash-configuration-file-for-postfix-log-analysis/335078)
**Author:** [@Aleksandr\_Terekhov](https://discuss.elastic.co/u/Aleksandr_Terekhov)\ **Replies:** 0\ **Last updated:** [June 2, 2023, 12:38pm UTC](https://discuss.elastic.co/t/help-with-creating-a-logstash-configuration-file-for-postfix-log-analysis/335078 "2023-06-02T12:38:59Z")
Hello everybody Can someone help to correctly create a configuration file that will display the fields from the postfix log that from status Message-id in Kiban in one line and not as in the screenshot I will be g… --- ## [Invalid version of beats protocol: 69 and 70](https://discuss.elastic.co/t/invalid-version-of-beats-protocol-69-and-70/334823)
**Author:** [@Aleksandr\_Terekhov](https://discuss.elastic.co/u/Aleksandr_Terekhov)\ **Replies:** 4\ **Last updated:** [June 2, 2023, 11:04am UTC](https://discuss.elastic.co/t/invalid-version-of-beats-protocol-69-and-70/334823 "2023-06-02T11:04:38Z")
Hello everybody Help to understand the problem There is an Oracle Linux 8 server on which Postfix and Filebeat 8.7.1 are installed Filebeat configuration # ============================== Filebeat inputs =============… --- ## [LogStash::Json::ParserError: Unexpected end-of-input: expected close marker for Array](https://discuss.elastic.co/t/logstash-unexpected-end-of-input-expected-close-marker-for-array/335071)
**Author:** [@niveditakathal](https://discuss.elastic.co/u/niveditakathal)\ **Replies:** 0\ **Last updated:** [June 2, 2023, 10:54am UTC](https://discuss.elastic.co/t/logstash-unexpected-end-of-input-expected-close-marker-for-array/335071 "2023-06-02T10:54:46Z")
Hi Experts, I want to ingest data from a text file (refer data.txt) to elastic using logstash and in order to achieve it, I have created the logstash.conf file as mentioned below - logstash.conf - input { file { … --- ## [There is a problem with elastic agent pushing logstash](https://discuss.elastic.co/t/there-is-a-problem-with-elastic-agent-pushing-logstash/335063)
**Author:** [@xqaiviwjxzw](https://discuss.elastic.co/u/xqaiviwjxzw)\ **Replies:** 0\ **Last updated:** [June 2, 2023, 9:13am UTC](https://discuss.elastic.co/t/there-is-a-problem-with-elastic-agent-pushing-logstash/335063 "2023-06-02T09:13:29Z")
By changing the original strategy of the elastic agent to push the log to Elasticsearch to push to the new strategy to push to logstash, why the log is still in the original Elasticsearch, but not pushed to the new lo… --- ## [Help with Grok (syntax issue as well as question regarding double quotes)](https://discuss.elastic.co/t/help-with-grok-syntax-issue-as-well-as-question-regarding-double-quotes/333891)
**Author:** [@roman-tasi](https://discuss.elastic.co/u/roman-tasi)\ **Replies:** 7\ **Last updated:** [June 2, 2023, 7:19am UTC](https://discuss.elastic.co/t/help-with-grok-syntax-issue-as-well-as-question-regarding-double-quotes/333891 "2023-06-02T07:19:40Z")
This is a sample log that I want to parse: type=EXECVE msg=audit(1684525987.999:148345): argc=2 a0="vim" a1="logstash-syslog.conf" This is the grok filter I am trying: type=%{WORD:type} msg=audit\\(%{NUMBER:audit}\\): a… --- ## [Output HTTP: Problem to send @metadata from one pipeline into another](https://discuss.elastic.co/t/output-http-problem-to-send-metadata-from-one-pipeline-into-another/335043)
**Author:** [@junchao](https://discuss.elastic.co/u/junchao)\ **Replies:** 0\ **Last updated:** [June 2, 2023, 3:38am UTC](https://discuss.elastic.co/t/output-http-problem-to-send-metadata-from-one-pipeline-into-another/335043 "2023-06-02T03:38:38Z")
I am trying to send the value of \[@metadata\]\[usertag\] from one pipeline 1 to pipeline 2. I tried to parse the value using "headers" setting but the value parsed is the string: "%{\[@metadata\]\[usertag\]}" and not the vari… --- ## [Create Apache Response Code Field](https://discuss.elastic.co/t/create-apache-response-code-field/334913)
**Author:** [@sanjeev1895](https://discuss.elastic.co/u/sanjeev1895)\ **Replies:** 1\ **Last updated:** [June 1, 2023, 6:08pm UTC](https://discuss.elastic.co/t/create-apache-response-code-field/334913 "2023-06-01T18:08:14Z")
Hi Guys, Can anyone help me to do the following configuration to work as expected. I'm trying to create the separate field for apache response code status using grok filter but it print IP address first two octect. Gr… --- ## [Can we have multiple destinations in one jms plugin in logstash cofiguration?](https://discuss.elastic.co/t/can-we-have-multiple-destinations-in-one-jms-plugin-in-logstash-cofiguration/334910)
**Author:** [@Pranjal\_Sett](https://discuss.elastic.co/u/Pranjal_Sett)\ **Replies:** 1\ **Last updated:** [June 1, 2023, 6:02pm UTC](https://discuss.elastic.co/t/can-we-have-multiple-destinations-in-one-jms-plugin-in-logstash-cofiguration/334910 "2023-06-01T18:02:47Z")
So my requirement is want to insert multiple destination name in one JMS plugin. Writing multiple JMS input plugin for more than 1 destination is bit hectic. So, how can we achieve this with one single jms input plugin. … --- ## [How to Setting single table or specific table output to BigQuery?](https://discuss.elastic.co/t/how-to-setting-single-table-or-specific-table-output-to-bigquery/335022)
**Author:** [@aidensV](https://discuss.elastic.co/u/aidensV)\ **Replies:** 0\ **Last updated:** [June 1, 2023, 5:08pm UTC](https://discuss.elastic.co/t/how-to-setting-single-table-or-specific-table-output-to-bigquery/335022 "2023-06-01T17:08:40Z")
BigQuery table ID prefix to be used when creating new tables for log data. Table name will be \\\ --- ## [My logstash conf file doesn't show me the output I don't what's the problem with that](https://discuss.elastic.co/t/my-logstash-conf-file-doesnt-show-me-the-output-i-dont-whats-the-problem-with-that/334999)
**Author:** [@Viknesh.S](https://discuss.elastic.co/u/Viknesh.S)\ **Replies:** 2\ **Last updated:** [June 1, 2023, 3:19pm UTC](https://discuss.elastic.co/t/my-logstash-conf-file-doesnt-show-me-the-output-i-dont-whats-the-problem-with-that/334999 "2023-06-01T15:19:26Z")
--- ## [Issue with multiple pipelines of Logstash](https://discuss.elastic.co/t/issue-with-multiple-pipelines-of-logstash/334908)
**Author:** [@Wang\_Yin](https://discuss.elastic.co/u/Wang_Yin)\ **Replies:** 0\ **Last updated:** [June 1, 2023, 6:58am UTC](https://discuss.elastic.co/t/issue-with-multiple-pipelines-of-logstash/334908 "2023-06-01T06:58:04Z")
I'm using Logstash version 8.8.0. I have two Logstash conf files under /etc/logstash/conf.d folder, one is called "syslog\_cisco.conf", another one is called "test.conf" as below: syslog\_cisco.conf input { udp { … --- ## [Error Attempted to send a bulk request but Elasticsearch appears to be unreachable or down by Lgostash](https://discuss.elastic.co/t/error-attempted-to-send-a-bulk-request-but-elasticsearch-appears-to-be-unreachable-or-down-by-lgostash/334894)
**Author:** [@Hoang\_Vu](https://discuss.elastic.co/u/Hoang_Vu)\ **Replies:** 0\ **Last updated:** [June 1, 2023, 4:03am UTC](https://discuss.elastic.co/t/error-attempted-to-send-a-bulk-request-but-elasticsearch-appears-to-be-unreachable-or-down-by-lgostash/334894 "2023-06-01T04:03:04Z")
I am getting an error like below: System status is Logstash receiving logs and pushing logs to Haproxy then Haproxy Forward back to Elasticsearch the system is running Docker Swarm except Logstash is running building.I … --- ## [Log Forwarding Capabilities](https://discuss.elastic.co/t/log-forwarding-capabilities/334627)
**Author:** [@ddawil](https://discuss.elastic.co/u/ddawil)\ **Replies:** 4\ **Last updated:** [June 1, 2023, 2:20am UTC](https://discuss.elastic.co/t/log-forwarding-capabilities/334627 "2023-06-01T02:20:32Z")
Network devices logs, system logs and Cloud services logs are sent to Elastic for log storage. Logs are processed are stored JSON format. Does Elastic able to do forwarding of logs simultaneously to a SIEM with its orig… --- ## [Log files to Logstash](https://discuss.elastic.co/t/log-files-to-logstash/333063)
**Author:** [@hjsroldan](https://discuss.elastic.co/u/hjsroldan)\ **Replies:** 1\ **Last updated:** [May 31, 2023, 8:43pm UTC](https://discuss.elastic.co/t/log-files-to-logstash/333063 "2023-05-31T20:43:19Z")
Hi, Good day! I have this scenario where I’m trying to collect log files and ship or ingest it to Logstash. Below is my logstash.conf Below is my input file (my-topics-1.txt) which contains 1-25 as shown below. … --- ## [Split Value into different document](https://discuss.elastic.co/t/split-value-into-different-document/332799)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 1\ **Last updated:** [May 31, 2023, 5:12pm UTC](https://discuss.elastic.co/t/split-value-into-different-document/332799 "2023-05-31T17:12:51Z")
Hi there, if i have data like this \[{...},{...},{...}\] how can i split them into different documents like document 1 =\> {...} document 2 =\> {...} document 3 =\> {...} so in that way, I can use the json filter to spre… --- ## [Logstash Enrich and translate plugin use](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897)
**Author:** [@gbandasha](https://discuss.elastic.co/u/gbandasha)\ **Replies:** 6\ **Last updated:** [May 31, 2023, 4:08pm UTC](https://discuss.elastic.co/t/logstash-enrich-and-translate-plugin-use/332897 "2023-05-31T16:08:27Z")
Hello Team, I am trying to enrich the data before it makes its way too elastic, I have tried the below methods but both are currently not working Using the elasticsearch plugin in filter input { kafka { … --- ## [Logstash SWAP OOM](https://discuss.elastic.co/t/logstash-swap-oom/334675)
**Author:** [@nilsen](https://discuss.elastic.co/u/nilsen)\ **Replies:** 2\ **Last updated:** [May 31, 2023, 2:27pm UTC](https://discuss.elastic.co/t/logstash-swap-oom/334675 "2023-05-31T14:27:39Z")
We have the past months installed the ELK stack trying to follow the elastic documentation. Currently using logstash to push approx. 15 logs into our elastic indexes. Hoping to push all of our approx. 100 logs into diffe… --- ## [How to change the date structure to YYYY:MM:DD](https://discuss.elastic.co/t/how-to-change-the-date-structure-to-yyyydd/334789)
**Author:** [@subash\_k](https://discuss.elastic.co/u/subash_k)\ **Replies:** 5\ **Last updated:** [May 31, 2023, 1:18pm UTC](https://discuss.elastic.co/t/how-to-change-the-date-structure-to-yyyydd/334789 "2023-05-31T13:18:44Z")
Hi, I tried multiple way to change the date event into YYYY:MMM:DD as log\_date. below format is actual date event (2023-05-31 10:30:50,244). I tried manual string concatenation even though am getting type as timestamp … --- ## ["The incoming YAML document exceeds the limit: 3145728 code points" in Logstash/ElastiFLOW](https://discuss.elastic.co/t/the-incoming-yaml-document-exceeds-the-limit-3145728-code-points-in-logstash-elastiflow/334803)
**Author:** [@numpty-boy](https://discuss.elastic.co/u/numpty-boy)\ **Replies:** 0\ **Last updated:** [May 31, 2023, 12:48pm UTC](https://discuss.elastic.co/t/the-incoming-yaml-document-exceeds-the-limit-3145728-code-points-in-logstash-elastiflow/334803 "2023-05-31T12:48:41Z")
Since upgrading to logstash 7.17.10 on Centos 7, I've been seeing the above error when starting. I see some other folks have had similar problems 8.7, and there are similar problems reported in RUBY forums. I had no su… --- ## [NameError, missing class name com.ibm.mq.jms.MQQueueConnectionFactory](https://discuss.elastic.co/t/nameerror-missing-class-name-com-ibm-mq-jms-mqqueueconnectionfactory/334784)
**Author:** [@paulov](https://discuss.elastic.co/u/paulov)\ **Replies:** 0\ **Last updated:** [May 31, 2023, 10:17am UTC](https://discuss.elastic.co/t/nameerror-missing-class-name-com-ibm-mq-jms-mqqueueconnectionfactory/334784 "2023-05-31T10:17:52Z")
Hello, I have a JMS plugin configuration with purpose of connecting to IBM MQ. After starting the pipeline I get: \> \> \[WARN \] 2023-05-31 10:38:14.348 \[\[main\]\"NameError", \> :… --- ## [Attempted to resurrect connection to dead ES instance, but got an error](https://discuss.elastic.co/t/attempted-to-resurrect-connection-to-dead-es-instance-but-got-an-error/333650)
**Author:** [@snalaband](https://discuss.elastic.co/u/snalaband)\ **Replies:** 4\ **Last updated:** [May 31, 2023, 9:03am UTC](https://discuss.elastic.co/t/attempted-to-resurrect-connection-to-dead-es-instance-but-got-an-error/333650 "2023-05-31T09:03:02Z")
Attempted to resurrect connection to dead ES instance, but got an error. {:error\_type=\>LogStash::Outputs::Elasticsearch::HttpClient::Pool::BadResponseCodeError, :error=\>"Got response code '401' contacting Elasticsearch a… --- ## [How to get CPU, Memory and Storage from VCenter not VM's using Logstash](https://discuss.elastic.co/t/how-to-get-cpu-memory-and-storage-from-vcenter-not-vms-using-logstash/334747)
**Author:** [@gaetano](https://discuss.elastic.co/u/gaetano)\ **Replies:** 0\ **Last updated:** [May 31, 2023, 6:32am UTC](https://discuss.elastic.co/t/how-to-get-cpu-memory-and-storage-from-vcenter-not-vms-using-logstash/334747 "2023-05-31T06:32:05Z")
I'm trying to send CPU, Memory and Storage parameters of VCenter Server (VMWare) to Elasticsearch node (8.6.1 verson) passing by Logstash 8.6.1. What MIB file should I use to get those specific parameters? This is my Lo… --- ## [Split json array into multiple documents](https://discuss.elastic.co/t/split-json-array-into-multiple-documents/332789)
**Author:** [@manramu22](https://discuss.elastic.co/u/manramu22)\ **Replies:** 1\ **Last updated:** [May 31, 2023, 12:35am UTC](https://discuss.elastic.co/t/split-json-array-into-multiple-documents/332789 "2023-05-31T00:35:04Z")
Hi Logstash community, I have the following json coming from http\_poller. I want to split that into multiple json documents and feed into Elasticsearch. Pls suggest json filter or split. Thanks in advance Input JSON {… --- ## [Getting error in logstash](https://discuss.elastic.co/t/getting-error-in-logstash/334646)
**Author:** [@Sachchan](https://discuss.elastic.co/u/Sachchan)\ **Replies:** 3\ **Last updated:** [May 30, 2023, 11:54am UTC](https://discuss.elastic.co/t/getting-error-in-logstash/334646 "2023-05-30T11:54:15Z")
Hi Team, Getting below error in logstash very frequently. please help in resolving this. \[2023-05-30T13:13:06,480\]\[ERROR\]\[logstash.outputs.elasticsearch\]\[CBC-only-4\]\[72644810f48942e01d7ac6fc35e066a073591417c6633c6aa1fb4… --- ## [Add multiline codec issue on logstash](https://discuss.elastic.co/t/add-multiline-codec-issue-on-logstash/334217)
**Author:** [@raymond0516](https://discuss.elastic.co/u/raymond0516)\ **Replies:** 5\ **Last updated:** [May 30, 2023, 9:06am UTC](https://discuss.elastic.co/t/add-multiline-codec-issue-on-logstash/334217 "2023-05-30T09:06:30Z")
Errors came out if I added multiline under "filter", but it works if I have added under "input". Anyone can give me hints? input { file { path =\> "/tmp/input.log" #codec =\> multiline { # pattern =\> "^%{TIME… --- ## [Configuration Logstash 8.6.1 for monitoring VMWare server](https://discuss.elastic.co/t/configuration-logstash-8-6-1-for-monitoring-vmware-server/334593)
**Author:** [@gaetano](https://discuss.elastic.co/u/gaetano)\ **Replies:** 2\ **Last updated:** [May 30, 2023, 8:22am UTC](https://discuss.elastic.co/t/configuration-logstash-8-6-1-for-monitoring-vmware-server/334593 "2023-05-30T08:22:53Z")
I'm trying to send VMWare server parameters to Elastic Search node 8.6.1 passing by Logstash 8.6.1. I obtained string in get option using Paessler MIB Importer. I downloaded MIB file in official site of VMWare. Thi… --- ## [Logstash pipeline configuration - extract metrics from message field](https://discuss.elastic.co/t/logstash-pipeline-configuration-extract-metrics-from-message-field/334597)
**Author:** [@Piotr\_Maciejek](https://discuss.elastic.co/u/Piotr_Maciejek)\ **Replies:** 0\ **Last updated:** [May 29, 2023, 2:56pm UTC](https://discuss.elastic.co/t/logstash-pipeline-configuration-extract-metrics-from-message-field/334597 "2023-05-29T14:56:29Z")
Hi! I want to confgure logstash pipeline. I got many logs in bulk format: ex of one log entry: {"index":{"\_index":"orchestrator-index","\_id":"xxx"}} {"message":"@metrics Exception count: 10","level":"Information","lo… --- ## [\[ERROR\] \[logstash.agent\] Failed to execute action](https://discuss.elastic.co/t/error-logstash-agent-failed-to-execute-action/334587)
**Author:** [@nml1988](https://discuss.elastic.co/u/nml1988)\ **Replies:** 5\ **Last updated:** [May 29, 2023, 5:33pm UTC](https://discuss.elastic.co/t/error-logstash-agent-failed-to-execute-action/334587 "2023-05-29T17:33:47Z")
Hello! I need help from the community! I am having a failure during the execution of the logstash ingest pipes. The service starts normally, but when executing the different pipelines I see this error for which I ca… --- ## [Logstash - Creating new field by taking first word from an other field](https://discuss.elastic.co/t/logstash-creating-new-field-by-taking-first-word-from-an-other-field/334536)
**Author:** [@Carlos\_T](https://discuss.elastic.co/u/Carlos_T)\ **Replies:** 4\ **Last updated:** [May 29, 2023, 4:38pm UTC](https://discuss.elastic.co/t/logstash-creating-new-field-by-taking-first-word-from-an-other-field/334536 "2023-05-29T16:38:32Z")
Hi all. It must be something plenty of people has answered but I can´t find it :slight\_smile: I've got a pipeline reading a log with the following structure: \[12/May/2022:19:04:50 +0200\] 192.168.0.2 server2 "DROP: Est… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=72) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=74)