# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=686 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 687 --- ## [CEF codec does not resolve extended fields](https://discuss.elastic.co/t/cef-codec-does-not-resolve-extended-fields/148354)
**Author:** [@jamn03](https://discuss.elastic.co/u/jamn03)\ **Replies:** 1\ **Last updated:** [September 12, 2018, 8:51pm UTC](https://discuss.elastic.co/t/cef-codec-does-not-resolve-extended-fields/148354 "2018-09-12T20:51:18Z")
I have an event flow that is processing Windows logs forwarded via syslog, parsing them, and sending them to ArcSight via the CEF codec. The list of extended fields to be included in the CEF message is variable dependin… --- ## [Parse 4 digit millisecond timestamp in correct format and then replace @timestamp with log\_timestamp](https://discuss.elastic.co/t/parse-4-digit-millisecond-timestamp-in-correct-format-and-then-replace-timestamp-with-log-timestamp/146906)
**Author:** [@simmi\_a](https://discuss.elastic.co/u/simmi_a)\ **Replies:** 5\ **Last updated:** [September 12, 2018, 8:04pm UTC](https://discuss.elastic.co/t/parse-4-digit-millisecond-timestamp-in-correct-format-and-then-replace-timestamp-with-log-timestamp/146906 "2018-09-12T20:04:35Z")
I have log time in 4 digit milliseconds 2018-08-30 22:44:34.1524 13273641624629889532273 Error Error saving Api Log data Tried this to parse it in correct time format but it doesn't work. I have to replace the @timesta… --- ## [Output to file getting deleted](https://discuss.elastic.co/t/output-to-file-getting-deleted/148363)
**Author:** [@dirtyballer3](https://discuss.elastic.co/u/dirtyballer3)\ **Replies:** 1\ **Last updated:** [September 12, 2018, 7:07pm UTC](https://discuss.elastic.co/t/output-to-file-getting-deleted/148363 "2018-09-12T19:07:57Z")
Hi there, Very new to this setup as im using Docker to implement Logstash and i cannot tell if i have something wrong with my Logstash setup or if its a Docker issue so wanted to check here. I have a simple input/outpu… --- ## [Not able to launch logstash from command line](https://discuss.elastic.co/t/not-able-to-launch-logstash-from-command-line/148359)
**Author:** [@Ajay\_Sharma1](https://discuss.elastic.co/u/Ajay_Sharma1)\ **Replies:** 0\ **Last updated:** [September 12, 2018, 4:56pm UTC](https://discuss.elastic.co/t/not-able-to-launch-logstash-from-command-line/148359 "2018-09-12T16:56:21Z")
Hi There I am not able to launch logstash from the command line using below command. C:\\logstash\\logstash-6.4.0\\bin\\logstash -e 'input { stdin { } } output { stdout {} }' When I execute the command, I get the followin… --- ## [Pipeline.yml is getting ingnore](https://discuss.elastic.co/t/pipeline-yml-is-getting-ingnore/148142)
**Author:** [@hariharan053](https://discuss.elastic.co/u/hariharan053)\ **Replies:** 1\ **Last updated:** [September 12, 2018, 5:09pm UTC](https://discuss.elastic.co/t/pipeline-yml-is-getting-ingnore/148142 "2018-09-12T17:09:04Z")
I am getting the below warning when I am trying to the config test because of which my Logstash is getting stuck when pipeline starts. Need help to resolve this . #/usr/share/logstash/bin/logstash --config.test\_and\_exit… --- ## [Syntax for index redirection (in module 'output')?](https://discuss.elastic.co/t/syntax-for-index-redirection-in-module-output/148303)
**Author:** [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\ **Replies:** 1\ **Last updated:** [September 12, 2018, 5:01pm UTC](https://discuss.elastic.co/t/syntax-for-index-redirection-in-module-output/148303 "2018-09-12T17:01:01Z")
In the 'output' section of my beats.conf I want to send different types of input to different indexes. The 'filter' section lets me pick and choose like so: "if \[fileset\]\[module\] =~ " --\> is there a way to do this in th… --- ## [Json parsing in Logstash - is this possible?](https://discuss.elastic.co/t/json-parsing-in-logstash-is-this-possible/148176)
**Author:** [@dmccuk](https://discuss.elastic.co/u/dmccuk)\ **Replies:** 2\ **Last updated:** [September 12, 2018, 3:15pm UTC](https://discuss.elastic.co/t/json-parsing-in-logstash-is-this-possible/148176 "2018-09-12T15:15:52Z")
Hi, I've been attempting to parse this json output with logstash and I'm not having any success. Can anyone tell me if it's actually possible: { "threadName":"threadX", "processName":"processX", "timestamp":… --- ## [Want to extract username from log and add it into TAG](https://discuss.elastic.co/t/want-to-extract-username-from-log-and-add-it-into-tag/147414)
**Author:** [@cleverrocks](https://discuss.elastic.co/u/cleverrocks)\ **Replies:** 2\ **Last updated:** [September 12, 2018, 2:34pm UTC](https://discuss.elastic.co/t/want-to-extract-username-from-log-and-add-it-into-tag/147414 "2018-09-12T14:34:41Z")
Hi, I have a log with bellow pattern: 2018-09-05 11:50:56,152 - INFO - 103.8.115.242 - username1 - com.example.test.service.testService - \[ajp-nio-8009-exec-9\] - /testServiceCall/default/testPage \[page=default, compone… --- ## [Xpath xml parsing problem](https://discuss.elastic.co/t/xpath-xml-parsing-problem/148237)
**Author:** [@mrk](https://discuss.elastic.co/u/mrk)\ **Replies:** 1\ **Last updated:** [September 12, 2018, 1:07pm UTC](https://discuss.elastic.co/t/xpath-xml-parsing-problem/148237 "2018-09-12T13:07:36Z")
Hi, I try to parse simple xml data by xml filter. This is my multiline message read by filebeat: \[2017-06-28 09:26:31\] Received message from localhost \[2017-06-28 09:26:31\] \ \ **Author:** [@artem33region](https://discuss.elastic.co/u/artem33region)\ **Replies:** 2\ **Last updated:** [September 12, 2018, 10:32am UTC](https://discuss.elastic.co/t/filter-section-in-logstash/148273 "2018-09-12T10:32:11Z") Hello! I have logstash 6.2.4 and Winlogbeat 6.4.0 Logstash config: input { beats { port =\> 5044 } } filter { if \[event\_data.LogonType\] == 3 { mutate { replace =\> \[ "event\_data.LogonType", "interactive… --- ## [Logstash \_grokparsefailure](https://discuss.elastic.co/t/logstash-grokparsefailure/148302)
**Author:** [@mdyr](https://discuss.elastic.co/u/mdyr)\ **Replies:** 0\ **Last updated:** [September 12, 2018, 10:58am UTC](https://discuss.elastic.co/t/logstash-grokparsefailure/148302 "2018-09-12T10:58:22Z")
Hi, this is my first post and I am new user of ELK. I have succesfully configured ELK on CentOS and working fine and now I am sending data so that i can create dashboards. However I am stuck at one point and though i ha… --- ## [Does logstash support snmpv3?](https://discuss.elastic.co/t/does-logstash-support-snmpv3/148300)
**Author:** [@PraveenKT](https://discuss.elastic.co/u/PraveenKT)\ **Replies:** 0\ **Last updated:** [September 12, 2018, 10:49am UTC](https://discuss.elastic.co/t/does-logstash-support-snmpv3/148300 "2018-09-12T10:49:20Z")
We want to use SNMPv3 in our cisco routers and switches. Does Logstash support snmpv3? --- ## [Noob index management - naming / cleanup](https://discuss.elastic.co/t/noob-index-management-naming-cleanup/148032)
**Author:** [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\ **Replies:** 7\ **Last updated:** [September 12, 2018, 10:10am UTC](https://discuss.elastic.co/t/noob-index-management-naming-cleanup/148032 "2018-09-12T10:10:52Z")
Ive setup a new beat-\>logstash-\>elastic system and have started collecting data from a number of sources (files, metrics, etc). Looking through the indexes available via Kibana Im seeing nearly a hundred entries marked '… --- ## [Parse key value pair only from the parenthesis](https://discuss.elastic.co/t/parse-key-value-pair-only-from-the-parenthesis/148286)
**Author:** [@Mohit\_Sakarkar](https://discuss.elastic.co/u/Mohit_Sakarkar)\ **Replies:** 0\ **Last updated:** [September 12, 2018, 9:19am UTC](https://discuss.elastic.co/t/parse-key-value-pair-only-from-the-parenthesis/148286 "2018-09-12T09:19:36Z")
Hi, I am new in logstash, I have large file of tomcat application log and I need to parse only request and response which is available in key value pair form in square brackets, please see request and response below fro… --- ## [Logstash error while testing the config file](https://discuss.elastic.co/t/logstash-error-while-testing-the-config-file/147856)
**Author:** [@gopisa](https://discuss.elastic.co/u/gopisa)\ **Replies:** 3\ **Last updated:** [September 12, 2018, 8:39am UTC](https://discuss.elastic.co/t/logstash-error-while-testing-the-config-file/147856 "2018-09-12T08:39:56Z")
hI All, my All servers are in azure cloud .. All Applications are running on java 1.7.0 I have running logstash version is 2.1.2 if i ran the bin\\logstash -f conf\\some.conf --config.reload.automatic getting below m… --- ## [Logstash Failing after setting a custom document ID](https://discuss.elastic.co/t/logstash-failing-after-setting-a-custom-document-id/148068)
**Author:** [@ganessen23](https://discuss.elastic.co/u/ganessen23)\ **Replies:** 4\ **Last updated:** [September 12, 2018, 6:35am UTC](https://discuss.elastic.co/t/logstash-failing-after-setting-a-custom-document-id/148068 "2018-09-12T06:35:59Z")
Hello, I am using ELK version 6.4.0 on a Linux machine. My logstash is failing when I'm trying to set a custom document id, but I am able to import data from the database to Elasticsearch if I remove the document\_id pa… --- ## [Support of iTunes RSS in logstash-input-rss plugin](https://discuss.elastic.co/t/support-of-itunes-rss-in-logstash-input-rss-plugin/148238)
**Author:** [@pk.241011](https://discuss.elastic.co/u/pk.241011)\ **Replies:** 2\ **Last updated:** [September 12, 2018, 12:25am UTC](https://discuss.elastic.co/t/support-of-itunes-rss-in-logstash-input-rss-plugin/148238 "2018-09-12T00:25:27Z")
Is this supported by logstash-input-rss? In other words where I can find which of the below are supported by this plugin? RSSFeedBurner GoogleDocsAtom AtomFeedBurner Atom ITunesRSS RSS Was not able to find relevant d… --- ## [Logstash combining logs from multiple sources](https://discuss.elastic.co/t/logstash-combining-logs-from-multiple-sources/147944)
**Author:** [@Telstra\_Aus](https://discuss.elastic.co/u/Telstra_Aus)\ **Replies:** 1\ **Last updated:** [September 11, 2018, 8:49pm UTC](https://discuss.elastic.co/t/logstash-combining-logs-from-multiple-sources/147944 "2018-09-11T20:49:22Z")
Hi, Scenario: We are trying to scan post execution logs from filebeat and feeding it to logstash to get them in elasticsearch. Problem: Logstash is combining logs from different location/sources in one document. Artef… --- ## [BUG: JSON Filter Plugin needs target?](https://discuss.elastic.co/t/bug-json-filter-plugin-needs-target/148171)
**Author:** [@isaacsanders](https://discuss.elastic.co/u/isaacsanders)\ **Replies:** 3\ **Last updated:** [September 11, 2018, 8:40pm UTC](https://discuss.elastic.co/t/bug-json-filter-plugin-needs-target/148171 "2018-09-11T20:40:41Z")
This log line from the most recent version of this plugin: Parsed JSON object/hash requires a target configuration option {:source=\>"unparsed", :raw=\>" "} Contradicts the documentation here: https://www… --- ## [SystemCallError: Unknown error (SystemCallError) - No message available](https://discuss.elastic.co/t/systemcallerror-unknown-error-systemcallerror-no-message-available/148185)
**Author:** [@isaacsanders](https://discuss.elastic.co/u/isaacsanders)\ **Replies:** 1\ **Last updated:** [September 11, 2018, 7:56pm UTC](https://discuss.elastic.co/t/systemcallerror-unknown-error-systemcallerror-no-message-available/148185 "2018-09-11T19:56:28Z")
I am getting an error: "SystemCallError: Unknown error (SystemCallError) - No message available" This is from the TCP Output: https://github.com/logstash-plugins/logstash-output-tcp/blob/master/lib/logstash/outputs/tcp.… --- ## [How json event is consider while logstash processing](https://discuss.elastic.co/t/how-json-event-is-consider-while-logstash-processing/148151)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 4\ **Last updated:** [September 11, 2018, 4:15pm UTC](https://discuss.elastic.co/t/how-json-event-is-consider-while-logstash-processing/148151 "2018-09-11T16:15:51Z")
HI All, I want to process some json file and i'm sending event to logstash using kafka. My json file contains information like this when i process the file how the logstash will consider the event like it will consider … --- ## [How to make my index default index pattern](https://discuss.elastic.co/t/how-to-make-my-index-default-index-pattern/148020)
**Author:** [@Jas\_Ahluwalia](https://discuss.elastic.co/u/Jas_Ahluwalia)\ **Replies:** 2\ **Last updated:** [September 11, 2018, 4:15pm UTC](https://discuss.elastic.co/t/how-to-make-my-index-default-index-pattern/148020 "2018-09-11T16:15:36Z")
Hi, I have the following in my logstash.conf file: output { elasticsearch { hosts =\> "elasticsearch:9200" index =\> "myIndex-%{+YYYY.MM.dd}" } } How can I get myIndex to be the default index in kibana… --- ## [GROK filtering problem](https://discuss.elastic.co/t/grok-filtering-problem/148160)
**Author:** [@robegome](https://discuss.elastic.co/u/robegome)\ **Replies:** 3\ **Last updated:** [September 11, 2018, 3:08pm UTC](https://discuss.elastic.co/t/grok-filtering-problem/148160 "2018-09-11T15:08:29Z")
Hello everyone! I am creating this topic because I am not sure why I have a different behavior between a GROK debugger web site VS Kibana (Dev Tools / Grok Debugger section), in fact I am not sure if this thread should … --- ## [Aggregate Filter does not work with OLD log files](https://discuss.elastic.co/t/aggregate-filter-does-not-work-with-old-log-files/148162)
**Author:** [@kjgarza](https://discuss.elastic.co/u/kjgarza)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 3:05pm UTC](https://discuss.elastic.co/t/aggregate-filter-does-not-work-with-old-log-files/148162 "2018-09-11T15:05:08Z")
I am trying to aggregate events over a DAY period but discarding any event after one hour of inactivity. the events in ElasticSearch look like this: { "clicks": 6, "unique\_usage": "2018-05-01\_04\_10.5438/0000-00SS\_5… --- ## [Kafka + Logstash - which versions work atm?](https://discuss.elastic.co/t/kafka-logstash-which-versions-work-atm/148141)
**Author:** [@MarcusCaepio](https://discuss.elastic.co/u/MarcusCaepio)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 1:34pm UTC](https://discuss.elastic.co/t/kafka-logstash-which-versions-work-atm/148141 "2018-09-11T13:34:36Z")
Hi All, I am right now testing pushing logs in kafka und pulling them via logstash input kafka. Versions: kafka\_2.11-2.0.0.tgz logstash 6.4.0 I don't get any message via the logstash input, finding messages like thi… --- ## [TLD logstash filter removes other subfields](https://discuss.elastic.co/t/tld-logstash-filter-removes-other-subfields/148139)
**Author:** [@panaman](https://discuss.elastic.co/u/panaman)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 1:31pm UTC](https://discuss.elastic.co/t/tld-logstash-filter-removes-other-subfields/148139 "2018-09-11T13:31:33Z")
I have noticed some odd behavior with the "TLD filter" I have the following filter if \[dns\]\[query\] !~ /in-addr.arpa$/ { tld { source =\> "\[dns\]\[query\]" target =\> "\[dns\]" } } When the above runs it will populate al… --- ## [How to determine bottleneck?](https://discuss.elastic.co/t/how-to-determine-bottleneck/148135)
**Author:** [@ancdaromidi](https://discuss.elastic.co/u/ancdaromidi)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 1:26pm UTC](https://discuss.elastic.co/t/how-to-determine-bottleneck/148135 "2018-09-11T13:26:37Z")
I have current configuration for logstash input { azureblob { storage\_account\_name =\> "" storage\_access\_key =\> "" container =\> "" registry\_path =\> "" codec =\> "json" interval =\> 20 } } filter { split { field … --- ## [V 6.4. - Invalid Frame Type, received 1](https://discuss.elastic.co/t/v-6-4-invalid-frame-type-received-1/148122)
**Author:** [@Sirocco](https://discuss.elastic.co/u/Sirocco)\ **Replies:** 0\ **Last updated:** [September 11, 2018, 12:38pm UTC](https://discuss.elastic.co/t/v-6-4-invalid-frame-type-received-1/148122 "2018-09-11T12:38:18Z")
Hello, I have an issue with logstash: Versions: elasticsearch-6.4.0-1.noarch, kibana-6.4.0-1.x86\_64, logstash-6.4.0-1.noarch The log: \` \< \[2018-09-11T11:22:42,976\]\[INFO \]\[logstash.pipeline \] Pipeline has termi… --- ## [How to Remove the header of CSV files with Logstash?](https://discuss.elastic.co/t/how-to-remove-the-header-of-csv-files-with-logstash/148059)
**Author:** [@akml\_kk](https://discuss.elastic.co/u/akml_kk)\ **Replies:** 3\ **Last updated:** [September 11, 2018, 12:23pm UTC](https://discuss.elastic.co/t/how-to-remove-the-header-of-csv-files-with-logstash/148059 "2018-09-11T12:23:19Z")
Hello, I am asking if is there an option to remove the header row of the CSV files with logstash or any other undesired rows ? --- ## [Namespaces and index clean-up](https://discuss.elastic.co/t/namespaces-and-index-clean-up/147971)
**Author:** [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\ **Replies:** 4\ **Last updated:** [September 11, 2018, 11:24am UTC](https://discuss.elastic.co/t/namespaces-and-index-clean-up/147971 "2018-09-11T11:24:58Z")
I recently did some experimentation with getting data from 'auditd' into logstash. Along the way it created lots and lots of unwanted entries in the elastic index. EG: These are items like '#033.31;1mbuilds#033.0;m.keyw… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=685) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=687)