# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=225 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 226 --- ## [Parse logstash file format](https://discuss.elastic.co/t/parse-logstash-file-format/274237)
**Author:** [@sulsulatoff](https://discuss.elastic.co/u/sulsulatoff)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 5:00pm UTC](https://discuss.elastic.co/t/parse-logstash-file-format/274237 "2021-05-27T17:00:47Z")
we have an input file with a format like below. How to process it with logstash? { "message" =\> "test", "@timestamp" =\> "2013-12-23T22:30:01.000Z", "@version" =\> "1", "type" =\> "syslog", "host" =\> "0:0:0:0:0:0:0:1:… --- ## [Logstash monitoring shows all metrics except pipeline details](https://discuss.elastic.co/t/logstash-monitoring-shows-all-metrics-except-pipeline-details/274223)
**Author:** [@BradVido](https://discuss.elastic.co/u/BradVido)\ **Replies:** 1\ **Last updated:** [May 27, 2021, 2:42pm UTC](https://discuss.elastic.co/t/logstash-monitoring-shows-all-metrics-except-pipeline-details/274223 "2021-05-27T14:42:46Z")
I'm using Elastic stack 7.8.1, and have configured X-pack monitoring for Logstash in logstash.yml by setting xpack.monitoring.enabled: true. In the Kibana UI, I see metrics, as expected here: And here: But the da… --- ## [Config file for multiple multi-line patterns?](https://discuss.elastic.co/t/config-file-for-multiple-multi-line-patterns/273877)
**Author:** [@1steve](https://discuss.elastic.co/u/1steve)\ **Replies:** 8\ **Last updated:** [May 27, 2021, 2:30pm UTC](https://discuss.elastic.co/t/config-file-for-multiple-multi-line-patterns/273877 "2021-05-27T14:30:52Z")
It looks like the configs described here no longer work; Config file for multiple multiline patterns There is now a codec for multiline inputs; Multiline codec plugin | Logstash Reference \[7.12\] | Elastic input { std… --- ## [Logstash fails to startup after update after update to 6.8.16](https://discuss.elastic.co/t/logstash-fails-to-startup-after-update-after-update-to-6-8-16/274184)
**Author:** [@bobbyphilip](https://discuss.elastic.co/u/bobbyphilip)\ **Replies:** 2\ **Last updated:** [May 27, 2021, 1:08pm UTC](https://discuss.elastic.co/t/logstash-fails-to-startup-after-update-after-update-to-6-8-16/274184 "2021-05-27T13:08:45Z")
I am running elk on debian. Today there was an update installed, which bumped the logstash version to 6.8.16 from 6.8.15 After this logstash fails to start. I see the following in the logs: \[ 2021-05-27T12:06:38,9… --- ## [Fetch only new entries From mssql](https://discuss.elastic.co/t/fetch-only-new-entries-from-mssql/274195)
**Author:** [@akash01](https://discuss.elastic.co/u/akash01)\ **Replies:** 0\ **Last updated:** [May 27, 2021, 11:27am UTC](https://discuss.elastic.co/t/fetch-only-new-entries-from-mssql/274195 "2021-05-27T11:27:08Z")
Hi, I am fetching data from mssql database table which is keep on updating So I want to fetch only the new entries from the table but each time It is fetching all the data. like if there are total 5 records then after… --- ## [Xml filter Xpath nested fields](https://discuss.elastic.co/t/xml-filter-xpath-nested-fields/274190)
**Author:** [@roua.B](https://discuss.elastic.co/u/roua.B)\ **Replies:** 0\ **Last updated:** [May 27, 2021, 11:12am UTC](https://discuss.elastic.co/t/xml-filter-xpath-nested-fields/274190 "2021-05-27T11:12:49Z")
Hello, I have an xml file witch is a collection of tags like this: \ \ \ \ **Author:** [@ima](https://discuss.elastic.co/u/ima)\ **Replies:** 0\ **Last updated:** [May 27, 2021, 11:02am UTC](https://discuss.elastic.co/t/logstash-healthcheck/274185 "2021-05-27T11:02:05Z") Hello does anybody have an idea about healthchecks wa can do for logstash --- ## [Logtash |7.12.1 | Multiple pipelines](https://discuss.elastic.co/t/logtash-7-12-1-multiple-pipelines/273607)
**Author:** [@Tarun\_N](https://discuss.elastic.co/u/Tarun_N)\ **Replies:** 5\ **Last updated:** [May 27, 2021, 8:12am UTC](https://discuss.elastic.co/t/logtash-7-12-1-multiple-pipelines/273607 "2021-05-27T08:12:34Z")
Hello All I have to configure multiple pipelines for which I carried out the changes in pipelines.yml with simple test configuration, after removing all the code to debug to make it work. Below is the sample configurati… --- ## [Same Condition Logs Parse on Logstash](https://discuss.elastic.co/t/same-condition-logs-parse-on-logstash/274139)
**Author:** [@Nazakat](https://discuss.elastic.co/u/Nazakat)\ **Replies:** 0\ **Last updated:** [May 27, 2021, 6:13am UTC](https://discuss.elastic.co/t/same-condition-logs-parse-on-logstash/274139 "2021-05-27T06:13:30Z")
how to parse logs in same condition? --- ## [Help/Advice needed setting up geo-ip filters in an on-prem Logstash to SIEM in Elastic Cloud instance](https://discuss.elastic.co/t/help-advice-needed-setting-up-geo-ip-filters-in-an-on-prem-logstash-to-siem-in-elastic-cloud-instance/273593)
**Author:** [@ronmer](https://discuss.elastic.co/u/ronmer)\ **Replies:** 11\ **Last updated:** [May 27, 2021, 5:26am UTC](https://discuss.elastic.co/t/help-advice-needed-setting-up-geo-ip-filters-in-an-on-prem-logstash-to-siem-in-elastic-cloud-instance/273593 "2021-05-27T05:26:54Z")
I need help confirming the exact steps required to get geo-ip information to map to the SIEM network map in Kabana My environment: Windows and Linux hosts running either Auditbeat, Packetbeat or Winlogbeat, Packetbeat … --- ## [Logstash stdout output Module](https://discuss.elastic.co/t/logstash-stdout-output-module/274127)
**Author:** [@vishakh](https://discuss.elastic.co/u/vishakh)\ **Replies:** 0\ **Last updated:** [May 26, 2021, 11:45pm UTC](https://discuss.elastic.co/t/logstash-stdout-output-module/274127 "2021-05-26T23:45:54Z")
ELK Stack-7.10.0 Filebeat-7.10.0 I've configured the filebeat output to logstash. From the Kibana-UI, I've configured the logstash pipeline as below. Objective: Print the log from filebeat on the stdout input { b… --- ## [Making Logstash configurations dynamic](https://discuss.elastic.co/t/making-logstash-configurations-dynamic/274095)
**Author:** [@usman1](https://discuss.elastic.co/u/usman1)\ **Replies:** 1\ **Last updated:** [May 26, 2021, 5:31pm UTC](https://discuss.elastic.co/t/making-logstash-configurations-dynamic/274095 "2021-05-26T17:31:34Z")
Is there a way to make Logstash configuration file dynamic by reading attribute names and types from a different text file (or any other input mechanism). To explain, if my logstash configuration file looks something li… --- ## [Logtash pid keeps changing](https://discuss.elastic.co/t/logtash-pid-keeps-changing/274073)
**Author:** [@cyberfence](https://discuss.elastic.co/u/cyberfence)\ **Replies:** 4\ **Last updated:** [May 26, 2021, 3:17pm UTC](https://discuss.elastic.co/t/logtash-pid-keeps-changing/274073 "2021-05-26T15:17:54Z")
I am currently running ubuntu 18 server edition and the following is java / logstash version root@monitor:/usr/share/logstash/bin# ./logstash --version Using JAVA\_HOME defined java: /usr/lib/jvm/java-1.8.0-openjdk-amd6… --- ## [Why number is not recognized as integer](https://discuss.elastic.co/t/why-number-is-not-recognized-as-integer/273917)
**Author:** [@dinesh5](https://discuss.elastic.co/u/dinesh5)\ **Replies:** 2\ **Last updated:** [May 26, 2021, 12:23pm UTC](https://discuss.elastic.co/t/why-number-is-not-recognized-as-integer/273917 "2021-05-26T12:23:58Z")
Hi Here is my sample data Madhya Pradesh;22.9734;78.6569;57766;764338;7558 Maharashtra;19.7515;75.7139;351005;5579897;88620 Manipur;24.6637;93.9063;6534;44089;688 Logstash considered the text and float but it doesn'… --- ## [Logstash in free version?](https://discuss.elastic.co/t/logstash-in-free-version/274050)
**Author:** [@Peter\_Boos](https://discuss.elastic.co/u/Peter_Boos)\ **Replies:** 1\ **Last updated:** [May 26, 2021, 12:13pm UTC](https://discuss.elastic.co/t/logstash-in-free-version/274050 "2021-05-26T12:13:57Z")
I got filebeat working with elastic search and kibana. Now i would like to get it working with logstash as an endpoint for filebeat. I'm using using a non clustered free version of ELK using no payed license. In such … --- ## [How to import data from 2 urls and with different index using http\_roller in logstash conf file](https://discuss.elastic.co/t/how-to-import-data-from-2-urls-and-with-different-index-using-http-roller-in-logstash-conf-file/274051)
**Author:** [@Tayfun\_UNAL](https://discuss.elastic.co/u/Tayfun_UNAL)\ **Replies:** 0\ **Last updated:** [May 26, 2021, 11:57am UTC](https://discuss.elastic.co/t/how-to-import-data-from-2-urls-and-with-different-index-using-http-roller-in-logstash-conf-file/274051 "2021-05-26T11:57:45Z")
input { http\_poller { urls =\> { "1" =\> { type =\> "article" method =\> GET url =\> "https://personel.klu.edu.tr/elasticsearch/datasets/makale\_bilgisi.json" headers =\> { Accept =\> "application/… --- ## [Rubydocs for Logstash Event API](https://discuss.elastic.co/t/rubydocs-for-logstash-event-api/273996)
**Author:** [@Robert\_Labrie](https://discuss.elastic.co/u/Robert_Labrie)\ **Replies:** 3\ **Last updated:** [May 26, 2021, 11:14am UTC](https://discuss.elastic.co/t/rubydocs-for-logstash-event-api/273996 "2021-05-26T11:14:11Z")
I'm trying to find the actual RubyDocs for the Event class and I can't find them. Seems they moved at one point from logstash-lib to logstash-core but the references I can find don't even mention the get and set methods … --- ## [CyberArk filebeat xsl translator file](https://discuss.elastic.co/t/cyberark-filebeat-xsl-translator-file/260664)
**Author:** [@PersonaZ](https://discuss.elastic.co/u/PersonaZ)\ **Replies:** 1\ **Last updated:** [May 26, 2021, 11:06am UTC](https://discuss.elastic.co/t/cyberark-filebeat-xsl-translator-file/260664 "2021-05-26T11:06:27Z")
We are trying to send from CyberArk vault to filebeat. Has anyone been able to use a xsl translator file for the vault to send to appropriate information to ELK filebeat? We seen it communicating but I believe the xsl fi… --- ## [Send logs to multiples spaces](https://discuss.elastic.co/t/send-logs-to-multiples-spaces/273912)
**Author:** [@mar-ro](https://discuss.elastic.co/u/mar-ro)\ **Replies:** 3\ **Last updated:** [May 26, 2021, 6:15am UTC](https://discuss.elastic.co/t/send-logs-to-multiples-spaces/273912 "2021-05-26T06:15:25Z")
Good morning. I would like to know if it's possible to send logs from a logstash server to multiples spaces in a same elastic cloud (kibana). If this is possible, what have I to add in logstash.yml file to send the info… --- ## [Unable to read logs files](https://discuss.elastic.co/t/unable-to-read-logs-files/273891)
**Author:** [@legolas\_bilbao](https://discuss.elastic.co/u/legolas_bilbao)\ **Replies:** 5\ **Last updated:** [May 26, 2021, 5:48am UTC](https://discuss.elastic.co/t/unable-to-read-logs-files/273891 "2021-05-26T05:48:37Z")
Good morning, I'm trying to read log files from a concrete path. My configuration is the following input { file { path =\> "/path/proxy/access2.log" start\_position =\> "beginning" … --- ## [Messages are not al same, how to grok?](https://discuss.elastic.co/t/messages-are-not-al-same-how-to-grok/273993)
**Author:** [@edvrfn](https://discuss.elastic.co/u/edvrfn)\ **Replies:** 1\ **Last updated:** [May 25, 2021, 10:26pm UTC](https://discuss.elastic.co/t/messages-are-not-al-same-how-to-grok/273993 "2021-05-25T22:26:47Z")
Logs sent by my gateway are not exactly same, how to handle missing fields in the logs with grok. Below Works TDM,50c9676c6d24,udm-1.9.3.3438 kernel: \[1393127.673482\] IN=br20 \*\*\*OUT=eth4\*\*\* MAC=74:ac:b9:1e:f3:00:e8:b1:… --- ## [LogStash seems to mask jdbc sql exception and exits with code 0](https://discuss.elastic.co/t/logstash-seems-to-mask-jdbc-sql-exception-and-exits-with-code-0/273992)
**Author:** [@trickett\_space](https://discuss.elastic.co/u/trickett_space)\ **Replies:** 1\ **Last updated:** [May 25, 2021, 9:48pm UTC](https://discuss.elastic.co/t/logstash-seems-to-mask-jdbc-sql-exception-and-exits-with-code-0/273992 "2021-05-25T21:48:07Z")
/usr/share/logstash/bin/logstash --path.data /usr/share/logstash/data\_supplier -f ./supplier.conf ... Starting Logstash {"logstash.version"=\>"7.10.2", "jruby.version"=\>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc Open… --- ## [Apply Grok to a log file](https://discuss.elastic.co/t/apply-grok-to-a-log-file/273979)
**Author:** [@Denilson-Semedo](https://discuss.elastic.co/u/Denilson-Semedo)\ **Replies:** 3\ **Last updated:** [May 25, 2021, 8:49pm UTC](https://discuss.elastic.co/t/apply-grok-to-a-log-file/273979 "2021-05-25T20:49:51Z")
I have thease following logs: {"type":"audit\_entry","created":"5/20/2021, 11:12:42 PM","colaborador\_id":"cf7dc62b-dde9-4980-89d8-96eb5707876e","ip":"192.168.112.6","request\_method":"PUT","ajax":false,"route":"/stock/art… --- ## [Jdbc logstash module pulling IP in decimal format](https://discuss.elastic.co/t/jdbc-logstash-module-pulling-ip-in-decimal-format/273948)
**Author:** [@tushar.bansal](https://discuss.elastic.co/u/tushar.bansal)\ **Replies:** 4\ **Last updated:** [May 25, 2021, 6:44pm UTC](https://discuss.elastic.co/t/jdbc-logstash-module-pulling-ip-in-decimal-format/273948 "2021-05-25T18:44:22Z")
Hi, I am pulling events from sqlserver using jdbc module in logstash. I get all events I queried for, but IP is coming in different format. I think it is decimal format. I want them in regular IP notation a.b.c.d forma… --- ## [Extract field's from a message field](https://discuss.elastic.co/t/extract-fields-from-a-message-field/273978)
**Author:** [@Denilson-Semedo](https://discuss.elastic.co/u/Denilson-Semedo)\ **Replies:** 0\ **Last updated:** [May 25, 2021, 5:41pm UTC](https://discuss.elastic.co/t/extract-fields-from-a-message-field/273978 "2021-05-25T17:41:10Z")
I'm colleting logs from a file .log: {"type":"audit\_entry","created":"5/20/2021, 11:12:42 PM","colaborador\_id":"cf7dc62b-dde9-4980-89d8-96eb5707876e","request\_method":"PUT","ajax":false,"route":"/stock/artigos/8c443bfe-… --- ## [Automatically start Logstash when Windows reboot](https://discuss.elastic.co/t/automatically-start-logstash-when-windows-reboot/273839)
**Author:** [@usman1](https://discuss.elastic.co/u/usman1)\ **Replies:** 3\ **Last updated:** [May 25, 2021, 4:12pm UTC](https://discuss.elastic.co/t/automatically-start-logstash-when-windows-reboot/273839 "2021-05-25T16:12:37Z")
I am using Windows OS as a source in which Logstash is installed and run Logstash using the following command on PowerShell: logstash -f logstash.conf Now as I am working in an industrial setting, manually running this… --- ## [Data ingestion got stuck though the Logstash pipelines are running](https://discuss.elastic.co/t/data-ingestion-got-stuck-though-the-logstash-pipelines-are-running/273970)
**Author:** [@Souvik\_Das](https://discuss.elastic.co/u/Souvik_Das)\ **Replies:** 0\ **Last updated:** [May 25, 2021, 3:41pm UTC](https://discuss.elastic.co/t/data-ingestion-got-stuck-though-the-logstash-pipelines-are-running/273970 "2021-05-25T15:41:53Z")
We have deployed 10 logstash config files last year. We started all the config files at once by using the folder where we have kept all the config files(as service). Yesterday we found that, few of them are not pushing d… --- ## [Reading from specific Kafka partitions](https://discuss.elastic.co/t/reading-from-specific-kafka-partitions/273549)
**Author:** [@wkuijsters](https://discuss.elastic.co/u/wkuijsters)\ **Replies:** 2\ **Last updated:** [May 25, 2021, 3:38pm UTC](https://discuss.elastic.co/t/reading-from-specific-kafka-partitions/273549 "2021-05-25T15:38:03Z")
Hi, I'm trying to consume Kafka data hosted on an on-prem Kafka bus using Logstash in a cloud-hosted environment. There is a single massive topic with about 100 partitions, the messages I need are all located in a singl… --- ## [Unable to write grok filter for access.log](https://discuss.elastic.co/t/unable-to-write-grok-filter-for-access-log/273730)
**Author:** [@Ipsidash](https://discuss.elastic.co/u/Ipsidash)\ **Replies:** 7\ **Last updated:** [May 25, 2021, 2:46pm UTC](https://discuss.elastic.co/t/unable-to-write-grok-filter-for-access-log/273730 "2021-05-25T14:46:38Z")
Hi, I have an access.log for which i need grok filter to ingest logs to logstash. I have passing the grok filter in logstash.conf. Access.log: 144.70.113.111 - - \[16/May/2021:23:40:32 -0400\] \[https-jsse-nio-8443-exe-4… --- ## [MariaDB audit](https://discuss.elastic.co/t/mariadb-audit/273934)
**Author:** [@Juju-fr34](https://discuss.elastic.co/u/Juju-fr34)\ **Replies:** 1\ **Last updated:** [May 25, 2021, 2:26pm UTC](https://discuss.elastic.co/t/mariadb-audit/273934 "2021-05-25T14:26:42Z")
Hi everyone. I'm trying to index my mariadb servers audit logs but I'm facing a problem. 90% off my logs are correctly indexed but some of them don't. It is like the produced csv is incorrect and message filed is empty. … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=224) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=226)