# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=967 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 968 --- ## [Mutate or reformat Winlogbeat timestamp](https://discuss.elastic.co/t/mutate-or-reformat-winlogbeat-timestamp/88412)
**Author:** [@jalcock](https://discuss.elastic.co/u/jalcock)\ **Replies:** 2\ **Last updated:** [June 12, 2017, 1:23pm UTC](https://discuss.elastic.co/t/mutate-or-reformat-winlogbeat-timestamp/88412 "2017-06-12T13:23:34Z")
Hello, I am ingesting winlogbeat to Logstash. For specific events, I am outputting those events to a third party SIEM. The @timestamp field is set to YYYY-MM-DDTHH:MM:SS.milliZ Is it possible to reformat/mutate the… --- ## [Logstash output to Azure storage](https://discuss.elastic.co/t/logstash-output-to-azure-storage/89046)
**Author:** [@Andreas\_Marshall](https://discuss.elastic.co/u/Andreas_Marshall)\ **Replies:** 0\ **Last updated:** [June 12, 2017, 1:26pm UTC](https://discuss.elastic.co/t/logstash-output-to-azure-storage/89046 "2017-06-12T13:26:01Z")
I am currently importing IIS-logs into Logstash using Filebeat, and the Logstash is configured to output the documents into Elasticsearch. Now I also want to output my IIS logs to Azure blobstorage for longtime-backup pu… --- ## [Insert json string (logtastsh ) - SOLVED](https://discuss.elastic.co/t/insert-json-string-logtastsh-solved/88926)
**Author:** [@rocavalcante](https://discuss.elastic.co/u/rocavalcante)\ **Replies:** 4\ **Last updated:** [June 12, 2017, 12:40pm UTC](https://discuss.elastic.co/t/insert-json-string-logtastsh-solved/88926 "2017-06-12T12:40:49Z")
Hello, I would like to add a json string in the logstash output, and in the future save the output "call output\_db" in a postgreSQL database, however, I still can not get the initial step of inserting the whole string in… --- ## [Parse substring in a line in grok](https://discuss.elastic.co/t/parse-substring-in-a-line-in-grok/89024)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 2\ **Last updated:** [June 12, 2017, 11:46am UTC](https://discuss.elastic.co/t/parse-substring-in-a-line-in-grok/89024 "2017-06-12T11:46:51Z")
Hello, I've a sample line 'stock\_status': 'IN\_STOCK', 'title\_ar': u"World's Greatest Mum Card" I want to parse the stock\_status value, so I created this grok filter grok{ match =\> \[ "message", "'stock\_status': '%{GR… --- ## [DISSECT FILTER, field not always specified](https://discuss.elastic.co/t/dissect-filter-field-not-always-specified/88109)
**Author:** [@Vinuja1](https://discuss.elastic.co/u/Vinuja1)\ **Replies:** 4\ **Last updated:** [June 12, 2017, 11:22am UTC](https://discuss.elastic.co/t/dissect-filter-field-not-always-specified/88109 "2017-06-12T11:22:16Z")
I was using a grok filter until i had the " Timeout executing grok : Value too large to output" message. In order to solve this issue, I would like to try a dissect filter. I have big files and sometimes, some field ar… --- ## [Filtering of unusual multiline json](https://discuss.elastic.co/t/filtering-of-unusual-multiline-json/89022)
**Author:** [@Aleksandr\_Matveyev](https://discuss.elastic.co/u/Aleksandr_Matveyev)\ **Replies:** 0\ **Last updated:** [June 12, 2017, 11:06am UTC](https://discuss.elastic.co/t/filtering-of-unusual-multiline-json/89022 "2017-06-12T11:06:51Z")
Hi, everyone. I have unusual log with json INFO | jvm 1 | main | 2017/06/11 10:29:22.699 | Matched response definition: INFO | jvm 1 | main | 2017/06/11 10:29:22.699 | { INFO | jvm 1 | main | 2017… --- ## [Logstash security\_exception, can't write to ES after installing X-pack](https://discuss.elastic.co/t/logstash-security-exception-cant-write-to-es-after-installing-x-pack/88973)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 2\ **Last updated:** [June 12, 2017, 10:30am UTC](https://discuss.elastic.co/t/logstash-security-exception-cant-write-to-es-after-installing-x-pack/88973 "2017-06-12T10:30:15Z")
As the title says, after installing X-pack logstash can't seem to authenticate with ES. \[2017-06-12T13:08:28,307\]\[ERROR\]\[logstash.outputs.elasticsearch\] Got a bad response code from server, but this code is not consid… --- ## [Logstash-output-kafka downgrade offline to 4.x.x](https://discuss.elastic.co/t/logstash-output-kafka-downgrade-offline-to-4-x-x/89012)
**Author:** [@Lieven\_Merckx](https://discuss.elastic.co/u/Lieven_Merckx)\ **Replies:** 0\ **Last updated:** [June 12, 2017, 10:22am UTC](https://discuss.elastic.co/t/logstash-output-kafka-downgrade-offline-to-4-x-x/89012 "2017-06-12T10:22:11Z")
We are working with a kafka broker version 0.9, meaning that logstash 5.4 is by default not compatible for the logstash-output-kafka. 1/ Which version do I need ? I saw on rubygems.org that version 4.0.6 is the most rece… --- ## [Logstash 5.3 Version](https://discuss.elastic.co/t/logstash-5-3-version/88703)
**Author:** [@amruthapbhat](https://discuss.elastic.co/u/amruthapbhat)\ **Replies:** 2\ **Last updated:** [June 12, 2017, 10:22am UTC](https://discuss.elastic.co/t/logstash-5-3-version/88703 "2017-06-12T10:22:04Z")
Hi, I have installed 5.3.3 elasticsearch and kibana. I was looking out for 5.3.3 v of logstash to install it in ubuntu machine. I was not finding url to download the logstash 5.3.3 deb package maunally. Could you p… --- ## [Logstash architecture](https://discuss.elastic.co/t/logstash-architecture/88983)
**Author:** [@Fly](https://discuss.elastic.co/u/Fly)\ **Replies:** 7\ **Last updated:** [June 12, 2017, 9:42am UTC](https://discuss.elastic.co/t/logstash-architecture/88983 "2017-06-12T09:42:10Z")
hi admin I want to ask, Logstash architecture is similar to ELK architecture??? --- ## [Add date field from IIS-log](https://discuss.elastic.co/t/add-date-field-from-iis-log/88596)
**Author:** [@Robert\_Andersson](https://discuss.elastic.co/u/Robert_Andersson)\ **Replies:** 4\ **Last updated:** [June 12, 2017, 9:16am UTC](https://discuss.elastic.co/t/add-date-field-from-iis-log/88596 "2017-06-12T09:16:37Z")
Hi, i'm currently trying to add an additional date field for when the log was created called log\_timestamp. But the Date filter does not seem to convert log\_timestamp into a date type. My config is below. filter { … --- ## [Using Logstash to process syslog](https://discuss.elastic.co/t/using-logstash-to-process-syslog/88636)
**Author:** [@Adyne](https://discuss.elastic.co/u/Adyne)\ **Replies:** 12\ **Last updated:** [June 12, 2017, 8:46am UTC](https://discuss.elastic.co/t/using-logstash-to-process-syslog/88636 "2017-06-12T08:46:57Z")
I read an article https://anderikistan.com/2016/03/26/elk-palo-alto-networks/ and I am now trying to setup Logstash to process Palo Alto logs. However, I am running into 1 issue. I am using a plain Ubuntu 16.04.2 LTS ins… --- ## [Logs generated while logstash stopped weren't collected after restart](https://discuss.elastic.co/t/logs-generated-while-logstash-stopped-werent-collected-after-restart/88493)
**Author:** [@rkj](https://discuss.elastic.co/u/rkj)\ **Replies:** 5\ **Last updated:** [June 12, 2017, 8:02am UTC](https://discuss.elastic.co/t/logs-generated-while-logstash-stopped-werent-collected-after-restart/88493 "2017-06-12T08:02:46Z")
Hi, Im using ELK and filebeat to collect and analise logs on a server. The problem i had was that I stopped logstash for a few hours and started it again but it seems that the logs during that period were not collected b… --- ## [Logstash not reading message from Kafka](https://discuss.elastic.co/t/logstash-not-reading-message-from-kafka/88980)
**Author:** [@Chittaranjan\_Sardar](https://discuss.elastic.co/u/Chittaranjan_Sardar)\ **Replies:** 0\ **Last updated:** [June 12, 2017, 7:07am UTC](https://discuss.elastic.co/t/logstash-not-reading-message-from-kafka/88980 "2017-06-12T07:07:58Z")
I am testing a simple pipeline - Filebeat \> Fafka \> Logstash \> File. Logstash is not reading from Kafka, but I see Kafka has messages when i use this command - bin/kafka-console-consumer.sh --bootstrap-server localhos… --- ## [Want to extract the logs based on keyword](https://discuss.elastic.co/t/want-to-extract-the-logs-based-on-keyword/88810)
**Author:** [@sanupam](https://discuss.elastic.co/u/sanupam)\ **Replies:** 3\ **Last updated:** [June 12, 2017, 5:57am UTC](https://discuss.elastic.co/t/want-to-extract-the-logs-based-on-keyword/88810 "2017-06-12T05:57:33Z")
Hello, I have a log below: Jun 05, 2017 6:45:07 AM sai.pollers.SaiDbPollerServlet handleNotification INFO: msg ======== MapMessage\[null\] Jun 05, 2017 6:45:07 AM sai.mdb.framework.SaiMessageDrivenBean onMessage INFO: … --- ## [Append new information to a field and turn it an array](https://discuss.elastic.co/t/append-new-information-to-a-field-and-turn-it-an-array/87856)
**Author:** [@gringo](https://discuss.elastic.co/u/gringo)\ **Replies:** 2\ **Last updated:** [June 12, 2017, 3:00am UTC](https://discuss.elastic.co/t/append-new-information-to-a-field-and-turn-it-an-array/87856 "2017-06-12T03:00:09Z")
Is it possible to use logstash to send a list of information into a document's fields and turn it into an array? Example Index name: People Type: Eastern Document: Country1 Field: BadPeople Values: scammer can I use … --- ## [Parsing XML file](https://discuss.elastic.co/t/parsing-xml-file/88854)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 2\ **Last updated:** [June 11, 2017, 8:19pm UTC](https://discuss.elastic.co/t/parsing-xml-file/88854 "2017-06-11T20:19:56Z")
Hi I am trying to parse this XML file to fetch all the \ \ tag and the property inside them under \ \ . I also need to count no of \ **Author:** [@smlbiobot](https://discuss.elastic.co/u/smlbiobot)\ **Replies:** 1\ **Last updated:** [June 11, 2017, 7:57pm UTC](https://discuss.elastic.co/t/multiple-logs-of-same-event/88961 "2017-06-11T19:57:39Z") I just started using Logstash yesterday and am loving it, but I don’t understand why sometimes the same event is recorded multiple times even though only one is supposed to be sent. I even have the code print to prompt s… --- ## [How to code grok filter](https://discuss.elastic.co/t/how-to-code-grok-filter/88948)
**Author:** [@ibrahimsharaf](https://discuss.elastic.co/u/ibrahimsharaf)\ **Replies:** 1\ **Last updated:** [June 11, 2017, 2:52pm UTC](https://discuss.elastic.co/t/how-to-code-grok-filter/88948 "2017-06-11T14:52:06Z")
Hello, I have the following two lines (they can occur multiple times) in a logfile 2017-05-01 09:21:35 \[item\_utils.py\] INFO: stock\_status value after cleanup is OUT\_OF\_STOCK 2017-05-01 09:21:35 \[item\_utils.py\] INFO: s… --- ## [Rsyslog not sending data to logstash](https://discuss.elastic.co/t/rsyslog-not-sending-data-to-logstash/88536)
**Author:** [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)\ **Replies:** 1\ **Last updated:** [June 11, 2017, 2:35pm UTC](https://discuss.elastic.co/t/rsyslog-not-sending-data-to-logstash/88536 "2017-06-11T14:35:43Z")
I have followed this Digital Ocean tutorial to send logs to logstash via rsyslog. But I am not getting logs in logstash. I am sending logs to a centralized rsyslog-server, and then to logstash. This rsyslog-server and … --- ## [I cannot start logstash on my machine. Error message inside](https://discuss.elastic.co/t/i-cannot-start-logstash-on-my-machine-error-message-inside/88900)
**Author:** [@dona](https://discuss.elastic.co/u/dona)\ **Replies:** 2\ **Last updated:** [June 10, 2017, 12:43pm UTC](https://discuss.elastic.co/t/i-cannot-start-logstash-on-my-machine-error-message-inside/88900 "2017-06-10T12:43:59Z")
The following is the test command from the tutorial: ./logstash -e 'input { stdin { } } output { stdout {} }' The following is the error. WARNING: Could not find logstash.yml which is typically located in $LS\_HOME… --- ## [ELK 5.4 - LS reverts to automatic mapping when 'type' field is removed or renamed](https://discuss.elastic.co/t/elk-5-4-ls-reverts-to-automatic-mapping-when-type-field-is-removed-or-renamed/88704)
**Author:** [@smyttie](https://discuss.elastic.co/u/smyttie)\ **Replies:** 1\ **Last updated:** [June 10, 2017, 6:34am UTC](https://discuss.elastic.co/t/elk-5-4-ls-reverts-to-automatic-mapping-when-type-field-is-removed-or-renamed/88704 "2017-06-10T06:34:35Z")
Hi there, this post spawns from this one : \[https://discuss.elastic.co/t/elk-stack-5-4-removing-field-named-type-creates-new-mapping-named-logs/88534). I felt the title was not correct and I didn't explain the issue clea… --- ## [Logstash doesn't stop outputting duplicates](https://discuss.elastic.co/t/logstash-doesnt-stop-outputting-duplicates/88890)
**Author:** [@jacoblemperle](https://discuss.elastic.co/u/jacoblemperle)\ **Replies:** 0\ **Last updated:** [June 9, 2017, 8:30pm UTC](https://discuss.elastic.co/t/logstash-doesnt-stop-outputting-duplicates/88890 "2017-06-09T20:30:56Z")
I am receiving input from elasticsearch and outputting to RabbitMQ, there are about 1300 items that should be output to Rabbit but logstash keeps running it i had to cancel at 50000 items posted. I was hoping someone kne… --- ## [Logstash ScrollID error Elasticsearch 1.7.6](https://discuss.elastic.co/t/logstash-scrollid-error-elasticsearch-1-7-6/88861)
**Author:** [@jacoblemperle](https://discuss.elastic.co/u/jacoblemperle)\ **Replies:** 0\ **Last updated:** [June 9, 2017, 3:44pm UTC](https://discuss.elastic.co/t/logstash-scrollid-error-elasticsearch-1-7-6/88861 "2017-06-09T15:44:04Z")
I using logstash 5.4 to do a query off of elasticsearch running on version 1.7.6, I am not asking for anything regarding a scrollId, what would be the reason this is coming out as an error, it is still outputting basical… --- ## [Issue with filter and timestamp](https://discuss.elastic.co/t/issue-with-filter-and-timestamp/88842)
**Author:** [@desai.amogh](https://discuss.elastic.co/u/desai.amogh)\ **Replies:** 3\ **Last updated:** [June 9, 2017, 2:41pm UTC](https://discuss.elastic.co/t/issue-with-filter-and-timestamp/88842 "2017-06-09T14:41:03Z")
I am getting errors, I guess due to the grok filters. check the Logstash parsed log below: { "message" =\> "2017-06-09 05:02:16,751 \[30\] DEBUG - WS206 - ApplicationName - someapp.Logging.someappLogger - REQ\_END P… --- ## [Multiline codec help for SYSLOG files](https://discuss.elastic.co/t/multiline-codec-help-for-syslog-files/88844)
**Author:** [@rgacek](https://discuss.elastic.co/u/rgacek)\ **Replies:** 0\ **Last updated:** [June 9, 2017, 1:07pm UTC](https://discuss.elastic.co/t/multiline-codec-help-for-syslog-files/88844 "2017-06-09T13:07:34Z")
Hello, I am trying to combine the following multiline output into single event using Logstash: 01,03,APR 10 16:25:45:453530,timer,ERROR ,"GFI Timer Error ==\>" 01,03,APR 10 16:25:45:453530,timer,ERROR ,"No 14: Dat… --- ## [Parsing nested json logs iin logstash](https://discuss.elastic.co/t/parsing-nested-json-logs-iin-logstash/88718)
**Author:** [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)\ **Replies:** 6\ **Last updated:** [June 9, 2017, 12:28pm UTC](https://discuss.elastic.co/t/parsing-nested-json-logs-iin-logstash/88718 "2017-06-09T12:28:17Z")
I have a json which I need to store in elasticsearch, Suppose this is the json, {"name":"luvpreet","tasks":{"task1":"logstash","task2":"elastic"},"time":{"time1":"1week","time2":"2week"}} And I want fields, name … --- ## [How to filter only error from log file](https://discuss.elastic.co/t/how-to-filter-only-error-from-log-file/88591)
**Author:** [@baha1](https://discuss.elastic.co/u/baha1)\ **Replies:** 9\ **Last updated:** [June 9, 2017, 11:54am UTC](https://discuss.elastic.co/t/how-to-filter-only-error-from-log-file/88591 "2017-06-09T11:54:27Z")
Hi community, i have a log file contains INFO,WARN and ERROR like : 17:37:17,103 ERROR \[org.apache.catalina.core.ContainerBase.\[jboss.web\].\[default-host\].....rest of text. what is the convenient grok to filter only A… --- ## [Can logstash-input-tcp handle windows certificates](https://discuss.elastic.co/t/can-logstash-input-tcp-handle-windows-certificates/88836)
**Author:** [@Tonni\_Hult](https://discuss.elastic.co/u/Tonni_Hult)\ **Replies:** 0\ **Last updated:** [June 9, 2017, 11:12am UTC](https://discuss.elastic.co/t/can-logstash-input-tcp-handle-windows-certificates/88836 "2017-06-09T11:12:24Z")
Hi When using SSL with the tcp input, can you use a windows certificate (pfx) file or even work against the windows cert stores? Or is a crt & pem file a must? Kind Regards Tonni --- ## [How to convert a field string to list in logstash](https://discuss.elastic.co/t/how-to-convert-a-field-string-to-list-in-logstash/88795)
**Author:** [@Tianrui](https://discuss.elastic.co/u/Tianrui)\ **Replies:** 4\ **Last updated:** [June 9, 2017, 9:05am UTC](https://discuss.elastic.co/t/how-to-convert-a-field-string-to-list-in-logstash/88795 "2017-06-09T09:05:09Z")
HI, How to convert string field to list in logstash. This is my field data. "http\_urls" =\> "\['http://11', 'http://22'\]" How should I write in my logstash's config? How modify it on my code? mutate { … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=966) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=968)