# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=584 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 585 --- ## [Resend logs in filebeat to logstash](https://discuss.elastic.co/t/resend-logs-in-filebeat-to-logstash/172400)
**Author:** [@Arthur19](https://discuss.elastic.co/u/Arthur19)\ **Replies:** 0\ **Last updated:** [March 14, 2019, 5:59pm UTC](https://discuss.elastic.co/t/resend-logs-in-filebeat-to-logstash/172400 "2019-03-14T17:59:30Z")
Hi, Thanks in advance for your help. I would like to reload some logs to customize additional fields. I have noticed that registry file in filebeat configuration keeps track of the files already picked. However, if I re… --- ## [Elasticsearch as Logstash output](https://discuss.elastic.co/t/elasticsearch-as-logstash-output/172256)
**Author:** [@Dv\_Thiyanesh](https://discuss.elastic.co/u/Dv_Thiyanesh)\ **Replies:** 13\ **Last updated:** [March 14, 2019, 5:26pm UTC](https://discuss.elastic.co/t/elasticsearch-as-logstash-output/172256 "2019-03-14T17:26:14Z")
Instead of having winlogbeat produces daily index, we can create separate index for separate winlogbeat clientlogs? Or any other idea? --- ## [Aggregate not working on certain events (solved)](https://discuss.elastic.co/t/aggregate-not-working-on-certain-events-solved/172319)
**Author:** [@Lorenzo\_Lapotre](https://discuss.elastic.co/u/Lorenzo_Lapotre)\ **Replies:** 8\ **Last updated:** [March 14, 2019, 4:23pm UTC](https://discuss.elastic.co/t/aggregate-not-working-on-certain-events-solved/172319 "2019-03-14T16:23:01Z")
Hello, I'm trying to aggregate multiple events that are created upon printing a document, events 842,805,307. However it seems that it is not picking up anything from 842 and 805 events. 842 and 805 carry informations l… --- ## [Keyword filter](https://discuss.elastic.co/t/keyword-filter/172344)
**Author:** [@Alexas](https://discuss.elastic.co/u/Alexas)\ **Replies:** 1\ **Last updated:** [March 14, 2019, 3:55pm UTC](https://discuss.elastic.co/t/keyword-filter/172344 "2019-03-14T15:55:25Z")
Configured in which there is an input and output file input { file { path =\> "/ home / sonic / log / vs01 / test\_new.log" start\_position =\> "beginning" ignore\_older =\> 0 sincedb\_path =\> "/ dev / null" }} output … --- ## [Help with filebeat](https://discuss.elastic.co/t/help-with-filebeat/171575)
**Author:** [@joseantoniotg26](https://discuss.elastic.co/u/joseantoniotg26)\ **Replies:** 7\ **Last updated:** [March 14, 2019, 3:51pm UTC](https://discuss.elastic.co/t/help-with-filebeat/171575 "2019-03-14T15:51:18Z")
I have configured the Kibana on a working server and 3 machines with filebeat sending logs. Everything works fine but I need to change the names that appear in the \_index fields of the Kibana so that I get the names of … --- ## [How to replace @timestamp to the log files mentioned date?](https://discuss.elastic.co/t/how-to-replace-timestamp-to-the-log-files-mentioned-date/172306)
**Author:** [@Kharishma](https://discuss.elastic.co/u/Kharishma)\ **Replies:** 4\ **Last updated:** [March 14, 2019, 12:53pm UTC](https://discuss.elastic.co/t/how-to-replace-timestamp-to-the-log-files-mentioned-date/172306 "2019-03-14T12:53:28Z")
Hi, i have attached the configuration file. Can please help me out to copy the LOGTIME to @timestamp. input { beats { port =\> 5044 } } filter { grok{ match =\> {"message" =\> "%{TIMESTAMP\_ISO8601:LOGTIME}|%{WORD:L… --- ## [Error: org.postgresql.Driver not loaded. Any specific solution for this?](https://discuss.elastic.co/t/error-org-postgresql-driver-not-loaded-any-specific-solution-for-this/172308)
**Author:** [@Danyal\_Tarik](https://discuss.elastic.co/u/Danyal_Tarik)\ **Replies:** 2\ **Last updated:** [March 14, 2019, 12:50pm UTC](https://discuss.elastic.co/t/error-org-postgresql-driver-not-loaded-any-specific-solution-for-this/172308 "2019-03-14T12:50:15Z")
This error occurs, when I run the command logstash -f file.conf: input { jdbc { jdbc\_connection\_string =\> 'jdbc:postgresql://localhost:5432/postgres' jdbc\_user =\> 'postgres' jdbc\_validate\_connection =\> true jdbc\_… --- ## [Logstash 6.6.2 on Ubuntu 18.04 loops indefinitely and doing nothing](https://discuss.elastic.co/t/logstash-6-6-2-on-ubuntu-18-04-loops-indefinitely-and-doing-nothing/172075)
**Author:** [@darren\_christopher](https://discuss.elastic.co/u/darren_christopher)\ **Replies:** 3\ **Last updated:** [March 14, 2019, 12:30pm UTC](https://discuss.elastic.co/t/logstash-6-6-2-on-ubuntu-18-04-loops-indefinitely-and-doing-nothing/172075 "2019-03-14T12:30:10Z")
Hi all, I am working with Logstash version 6.6.2 on Ubuntu 18.04 (ElementaryOS that based on Ubuntu 18.04). I tried to upload my file into Elasticsearch and it seems to repeating the loops and doing nothing based on the… --- ## [Multiline codec in Logstash 6.6 not working](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156)
**Author:** [@svm89](https://discuss.elastic.co/u/svm89)\ **Replies:** 6\ **Last updated:** [March 14, 2019, 11:16am UTC](https://discuss.elastic.co/t/multiline-codec-in-logstash-6-6-not-working/172156 "2019-03-14T11:16:57Z")
Hi guys, I am new with the elk stack, but am trying to implement it to get nicer logs for my application. Here is my config: input { file { path =\> "/var/log/file.log" codec =\> multiline { #pattern =\> "^\\d\\… --- ## [Logstash conf file for azure market place elastic search stack](https://discuss.elastic.co/t/logstash-conf-file-for-azure-market-place-elastic-search-stack/172300)
**Author:** [@subhash.parise](https://discuss.elastic.co/u/subhash.parise)\ **Replies:** 0\ **Last updated:** [March 14, 2019, 9:59am UTC](https://discuss.elastic.co/t/logstash-conf-file-for-azure-market-place-elastic-search-stack/172300 "2019-03-14T09:59:53Z")
Dear community members, i am trying to deploy elk cluster by using azure market place template. while selct logstash yes option in UI it's asking A Base-64 encoded string form of Logstash configuration file? can any on… --- ## [Logstash doesn't read changes in file and need to start from bin every time](https://discuss.elastic.co/t/logstash-doesnt-read-changes-in-file-and-need-to-start-from-bin-every-time/172247)
**Author:** [@sukarn001](https://discuss.elastic.co/u/sukarn001)\ **Replies:** 5\ **Last updated:** [March 14, 2019, 9:33am UTC](https://discuss.elastic.co/t/logstash-doesnt-read-changes-in-file-and-need-to-start-from-bin-every-time/172247 "2019-03-14T09:33:42Z")
Hello, I have installed Logstash 6.6.1 in rhel 7, from RPM. Now I have created a file under /etc/logstash/conf.d/apache.conf . This file keeps updating itself. I ran following command to send logs to elasticsearch: .… --- ## [Kibana not showing the correct data while choosing with timestamp & received\_at](https://discuss.elastic.co/t/kibana-not-showing-the-correct-data-while-choosing-with-timestamp-received-at/172181)
**Author:** [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\ **Replies:** 5\ **Last updated:** [March 14, 2019, 5:45am UTC](https://discuss.elastic.co/t/kibana-not-showing-the-correct-data-while-choosing-with-timestamp-received-at/172181 "2019-03-14T05:45:51Z")
I have Below logstash.conf file, i see the data is processing correctly but today i see very weird issues where indices for apic\_logs are not showing the correct syslog\_timestamp . input { file { path =\> \[ "/scrat… --- ## [How to Split the one field into multiple fields using kv plugins](https://discuss.elastic.co/t/how-to-split-the-one-field-into-multiple-fields-using-kv-plugins/172067)
**Author:** [@muralasandeep](https://discuss.elastic.co/u/muralasandeep)\ **Replies:** 4\ **Last updated:** [March 14, 2019, 5:17am UTC](https://discuss.elastic.co/t/how-to-split-the-one-field-into-multiple-fields-using-kv-plugins/172067 "2019-03-14T05:17:55Z")
Hi , This is Sandeep, I am new at logstash, here i am trying to fatch data from .csv file to elasticsearch using logstash and i have 6 fields total , in that 6 fields one of the field(log\_message) have actually data i … --- ## [Issue with Logstash & JDBC input (postgreSQL)](https://discuss.elastic.co/t/issue-with-logstash-jdbc-input-postgresql/171582)
**Author:** [@chaire](https://discuss.elastic.co/u/chaire)\ **Replies:** 5\ **Last updated:** [March 14, 2019, 4:00am UTC](https://discuss.elastic.co/t/issue-with-logstash-jdbc-input-postgresql/171582 "2019-03-14T04:00:10Z")
Hi, I use Logstash to get data from PostgreSQL and then send it to Elasticsearch. I use JDBC Driver. The issue is no index created in ES after I run Logstash. Here is my Logstash config. input { jdbc { jdbc\_dr… --- ## [How can I pass a filebeat \[host\]\[ip\] array to the logstash CIDR filter plugin?](https://discuss.elastic.co/t/how-can-i-pass-a-filebeat-host-ip-array-to-the-logstash-cidr-filter-plugin/171816)
**Author:** [@PSM](https://discuss.elastic.co/u/PSM)\ **Replies:** 2\ **Last updated:** [March 12, 2019, 11:48am UTC](https://discuss.elastic.co/t/how-can-i-pass-a-filebeat-host-ip-array-to-the-logstash-cidr-filter-plugin/171816 "2019-03-12T11:48:38Z")
I'm using the filebeat add\_host\_metadata processor to enrich events with an array of local IP addresses for a host but I can't pass that to the logstash CIDR filter plugin because it sees it as a string, not as an array … --- ## [Grok filter - access log](https://discuss.elastic.co/t/grok-filter-access-log/172202)
**Author:** [@kentatoi](https://discuss.elastic.co/u/kentatoi)\ **Replies:** 3\ **Last updated:** [March 13, 2019, 7:40pm UTC](https://discuss.elastic.co/t/grok-filter-access-log/172202 "2019-03-13T19:40:27Z")
Hi there, I am trying to apply my grok filter on my access log : 10.00.000.00 - - \[08/Feb/2019:09:06:54 -0500\] "GET /bea\_wls\_test\_intern/wl\_test HTTP/1.1" 404 1164 I am trying to apply that : input { stdin {} } … --- ## [How to map a date field in Logstash config file?](https://discuss.elastic.co/t/how-to-map-a-date-field-in-logstash-config-file/170249)
**Author:** [@samnayak](https://discuss.elastic.co/u/samnayak)\ **Replies:** 1\ **Last updated:** [March 13, 2019, 6:09pm UTC](https://discuss.elastic.co/t/how-to-map-a-date-field-in-logstash-config-file/170249 "2019-03-13T18:09:55Z")
Hi All, I am trying to map a date field into Elasticsearch to filter the data using "Time picker" in Kibana Dashboard. My CSV file date format is mm/dd/yy with field name "Service Begin Date". So in my config file, I am… --- ## [ELK to resolve IIS username to show fullname of the user](https://discuss.elastic.co/t/elk-to-resolve-iis-username-to-show-fullname-of-the-user/172030)
**Author:** [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\ **Replies:** 2\ **Last updated:** [March 13, 2019, 4:49pm UTC](https://discuss.elastic.co/t/elk-to-resolve-iis-username-to-show-fullname-of-the-user/172030 "2019-03-13T16:49:28Z")
Currently I have a setup Filebeats \>\> Logstash \>\> Elastic Search \>\> Kibana. I am using this to parse the IIS access logs. In access logs we get the username. But my requirement is to show the full name of the user. … --- ## [Logstash could not be started because there is already another instance](https://discuss.elastic.co/t/logstash-could-not-be-started-because-there-is-already-another-instance/171306)
**Author:** [@muralasandeep](https://discuss.elastic.co/u/muralasandeep)\ **Replies:** 10\ **Last updated:** [March 13, 2019, 3:44pm UTC](https://discuss.elastic.co/t/logstash-could-not-be-started-because-there-is-already-another-instance/171306 "2019-03-13T15:44:09Z")
Hi every one , This is Sandeep, i am new at Logstash and i got one error when i run the Logstash confi file that is \[2019-03-07T18:09:12,677\]\[FATAL\]\[logstash.runner \] Logstash could not be started because the… --- ## [Gelfd Parsing failed](https://discuss.elastic.co/t/gelfd-parsing-failed/172027)
**Author:** [@Sashi](https://discuss.elastic.co/u/Sashi)\ **Replies:** 2\ **Last updated:** [March 13, 2019, 3:38pm UTC](https://discuss.elastic.co/t/gelfd-parsing-failed/172027 "2019-03-13T15:38:48Z")
Hello, I am getting this error on Logstash \[2019-03-12T15:32:41,450\]\[WARN \]\[logstash.inputs.gelf \] Gelfd failed to parse a message skipping {:exception=\>#\. it starte… --- ## [GROK pattern not working in Logstash](https://discuss.elastic.co/t/grok-pattern-not-working-in-logstash/172174)
**Author:** [@kd5dbl](https://discuss.elastic.co/u/kd5dbl)\ **Replies:** 1\ **Last updated:** [March 13, 2019, 3:25pm UTC](https://discuss.elastic.co/t/grok-pattern-not-working-in-logstash/172174 "2019-03-13T15:25:48Z")
Is there a definitive list of characters that need to be escaped in GROK patterns in logstash? I'm trying to bring up logstash with this grok filter: grok { "message" =\> \[ "%{IPORHOST:client\_ip}\\,%{DATA}\\… --- ## [Debug logstash pipeline before save changes](https://discuss.elastic.co/t/debug-logstash-pipeline-before-save-changes/172168)
**Author:** [@javila](https://discuss.elastic.co/u/javila)\ **Replies:** 1\ **Last updated:** [March 13, 2019, 2:52pm UTC](https://discuss.elastic.co/t/debug-logstash-pipeline-before-save-changes/172168 "2019-03-13T14:52:44Z")
Hi there, I'm wondering how can I validate/verify that the changes I've added to my Logstash pipeline are good? Is it possible to have some sort of debug option in this section so when new lines are added we can see wh… --- ## [Logstash date extraction in logs](https://discuss.elastic.co/t/logstash-date-extraction-in-logs/171563)
**Author:** [@JeanSec](https://discuss.elastic.co/u/JeanSec)\ **Replies:** 16\ **Last updated:** [March 13, 2019, 2:47pm UTC](https://discuss.elastic.co/t/logstash-date-extraction-in-logs/171563 "2019-03-13T14:47:04Z")
Hello, I'm new to Elastic stack and i'm currently running an ELK stack on windows with Filebeat. I can see data in Kibana but the displayed timestamp does not correspond to the date written in the logs which is annoying… --- ## [Kibana shows incorrect syslog\_timestamp version 6.5.4](https://discuss.elastic.co/t/kibana-shows-incorrect-syslog-timestamp-version-6-5-4/171966)
**Author:** [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\ **Replies:** 6\ **Last updated:** [March 12, 2019, 5:34pm UTC](https://discuss.elastic.co/t/kibana-shows-incorrect-syslog-timestamp-version-6-5-4/171966 "2019-03-12T17:34:17Z")
I have Below logstash.conf file, i see the data is processing correctly but today i see very weird issues where indices for apic\_logs are not showing the correct syslog\_timestamp . input { file { path =\> \[ "/scrat… --- ## [Parsing multiline, netsted XML using Logstash](https://discuss.elastic.co/t/parsing-multiline-netsted-xml-using-logstash/172142)
**Author:** [@saif3r](https://discuss.elastic.co/u/saif3r)\ **Replies:** 7\ **Last updated:** [March 13, 2019, 1:32pm UTC](https://discuss.elastic.co/t/parsing-multiline-netsted-xml-using-logstash/172142 "2019-03-13T13:32:47Z")
Hello Guys, I'm currently learning Logstash using one of Udemy's courses. While the course is currently focusing on file/http/stdin inputs, I would like to know whether it is possible to parse an xml presented below. Th… --- ## [Logstash jdbc input for multiple queries](https://discuss.elastic.co/t/logstash-jdbc-input-for-multiple-queries/171804)
**Author:** [@rorixrebel](https://discuss.elastic.co/u/rorixrebel)\ **Replies:** 4\ **Last updated:** [March 13, 2019, 1:26pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-for-multiple-queries/171804 "2019-03-13T13:26:12Z")
Got 5+ queries that i need to run every X minutes so instead of creating multiple inputs i was thinking of using statement\_filepath to read all of them but seems like it doesn't quite accept wildcards (/my/queries/path/\*… --- ## [Topics\_pattern of kafka input plugin is not working](https://discuss.elastic.co/t/topics-pattern-of-kafka-input-plugin-is-not-working/169405)
**Author:** [@srineela](https://discuss.elastic.co/u/srineela)\ **Replies:** 1\ **Last updated:** [March 13, 2019, 1:17pm UTC](https://discuss.elastic.co/t/topics-pattern-of-kafka-input-plugin-is-not-working/169405 "2019-03-13T13:17:13Z")
I am having topics in kafka with prefix "abc-". When I use topics\_pattern =\> "abc-\*" in kafka input plugin I am not getting any results in elasticerach. It is not throwing any errors in logstash also. Currently I am usi… --- ## [Parsing different lines in same log](https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998)
**Author:** [@mdlapla](https://discuss.elastic.co/u/mdlapla)\ **Replies:** 5\ **Last updated:** [March 13, 2019, 12:55pm UTC](https://discuss.elastic.co/t/parsing-different-lines-in-same-log/171998 "2019-03-13T12:55:32Z")
Hi, I've been trying to parse a log for a couple of days now using Grok. I made it work and then, all of a sudden, it stopped working and I don't know why. I've been using the grok debugger but I haven't been able to pa… --- ## [Logstash Redis Output - Expire Keys?](https://discuss.elastic.co/t/logstash-redis-output-expire-keys/172020)
**Author:** [@titan1978](https://discuss.elastic.co/u/titan1978)\ **Replies:** 8\ **Last updated:** [March 13, 2019, 12:50pm UTC](https://discuss.elastic.co/t/logstash-redis-output-expire-keys/172020 "2019-03-13T12:50:37Z")
We are building a Redis output configuration that pushes a large stack of data to Redis. The volume is so large that we want to set an EXPIRY for each key that is pushed to Redis. This is very important from both costs a… --- ## [TCP input without delimiter](https://discuss.elastic.co/t/tcp-input-without-delimiter/169429)
**Author:** [@srilumpa](https://discuss.elastic.co/u/srilumpa)\ **Replies:** 3\ **Last updated:** [March 13, 2019, 12:15pm UTC](https://discuss.elastic.co/t/tcp-input-without-delimiter/169429 "2019-03-13T12:15:10Z")
Hi, I am trying to set up a logstash instance collecting data from a TCP Input. My pipeline is set and I am able to send data with a telnet. For now my pipeline looks like this: input { tcp { port =\> 1234 } }… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=583) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=585)