# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=794 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 795 --- ## [Ignore cron?](https://discuss.elastic.co/t/ignore-cron/125467)
**Author:** [@aclowkey](https://discuss.elastic.co/u/aclowkey)\ **Replies:** 1\ **Last updated:** [March 25, 2018, 4:28pm UTC](https://discuss.elastic.co/t/ignore-cron/125467 "2018-03-25T16:28:37Z")
Hi, How can I make LogStash ignore that cron and just run it once and quit? I want to reuse a pipeline for testing purpose. The issue is this pipeline has a CRON expression to execute every hour. --- ## [Grok filter throwing error for valid pattern](https://discuss.elastic.co/t/grok-filter-throwing-error-for-valid-pattern/125437)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 4\ **Last updated:** [March 25, 2018, 2:54pm UTC](https://discuss.elastic.co/t/grok-filter-throwing-error-for-valid-pattern/125437 "2018-03-25T14:54:43Z")
Hi, I am using ELK GA 5.0.0. I am reading from Kafka using Logstash. I have a message like below; 192.196.0.1\\t-\\t-\\t\[05/Feb/2018:10:39:35 +0000\]\\t\\"GET /ghRme\_Neldtjhqoxy/TlvehaNjshv \\"\\t404\\t-\\t1\\t\\"-\\"\\t\\"-\\"\\t\\"-\\"… --- ## [How to match more formats by dissect](https://discuss.elastic.co/t/how-to-match-more-formats-by-dissect/125463)
**Author:** [@Sunil\_Villuri](https://discuss.elastic.co/u/Sunil_Villuri)\ **Replies:** 3\ **Last updated:** [March 25, 2018, 10:28am UTC](https://discuss.elastic.co/t/how-to-match-more-formats-by-dissect/125463 "2018-03-25T10:28:37Z")
Hi All, i am having different log formats and i matched one log format by using the dissect combination. dissect { mapping =\> { "message" =\> "%{timestamp}|%{component}|%{LogUtils}|%{javaclass}|%{method}|%{delestlog}|… --- ## [Logstash: error in processing events from DLQ](https://discuss.elastic.co/t/logstash-error-in-processing-events-from-dlq/125406)
**Author:** [@pandeesh](https://discuss.elastic.co/u/pandeesh)\ **Replies:** 2\ **Last updated:** [March 25, 2018, 12:14am UTC](https://discuss.elastic.co/t/logstash-error-in-processing-events-from-dlq/125406 "2018-03-25T00:14:46Z")
All, I have configured my primary pipeline to process events from beats plugin and write it to Dead Letter Queue whenever there're any issues. Then I have a secondary pipeline to read from the DLQ and send it to Elasti… --- ## [Logstash Log - \[ERROR\]\[logstash.inputs.file \] each: ignored?](https://discuss.elastic.co/t/logstash-log-error-logstash-inputs-file-each-ignored/124605)
**Author:** [@joaoantoniopereira](https://discuss.elastic.co/u/joaoantoniopereira)\ **Replies:** 1\ **Last updated:** [March 24, 2018, 10:35pm UTC](https://discuss.elastic.co/t/logstash-log-error-logstash-inputs-file-each-ignored/124605 "2018-03-24T22:35:42Z")
Hi! I am running Logstash in Windows and, in my input I have the following lines: file { path =\> "E:/logs/firewall/\*/\*firewall.txt" start\_position =\> "beginning" sincedb\_path =\> "E:/logs/firewall/firewall\_s… --- ## [Using custom target for geoip filter, how to update Elasticsearch template?](https://discuss.elastic.co/t/using-custom-target-for-geoip-filter-how-to-update-elasticsearch-template/125442)
**Author:** [@omeugdoj](https://discuss.elastic.co/u/omeugdoj)\ **Replies:** 6\ **Last updated:** [March 24, 2018, 6:46pm UTC](https://discuss.elastic.co/t/using-custom-target-for-geoip-filter-how-to-update-elasticsearch-template/125442 "2018-03-24T18:46:59Z")
I am enriching my firewall logs with GeoIP information for the source and destination IPs. Because of that, I have to use custom target field names: src\_geoip and dst\_geoip. This is the relevant part for my filter: … --- ## [Error running logstash](https://discuss.elastic.co/t/error-running-logstash/124678)
**Author:** [@Yonder\_Rosales](https://discuss.elastic.co/u/Yonder_Rosales)\ **Replies:** 4\ **Last updated:** [March 24, 2018, 4:50pm UTC](https://discuss.elastic.co/t/error-running-logstash/124678 "2018-03-24T16:50:42Z")
When i execute in terminal ./logstash -f logstash.conf it give me the next error NameError: cannot load Java class org.apache.http.client.methods.HttpRequestBase for\_name at org/jruby/javasupport/JavaClass.java:204 g… --- ## [Why is my Logstash date plugin return PM as AM?](https://discuss.elastic.co/t/why-is-my-logstash-date-plugin-return-pm-as-am/125434)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [March 24, 2018, 9:46am UTC](https://discuss.elastic.co/t/why-is-my-logstash-date-plugin-return-pm-as-am/125434 "2018-03-24T09:46:07Z")
Hi, I am using ELK GA 5.0.0. I am trying to parse my date 2018-03-18 01:44:25 PM. I am expecting 01 PM as 13 in 24 hr format in @timestamp field. Below is the config. input { generator { message =\> "2018-03-18 01:44… --- ## [How can I parse a date containing character T](https://discuss.elastic.co/t/how-can-i-parse-a-date-containing-character-t/125432)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [March 24, 2018, 9:31am UTC](https://discuss.elastic.co/t/how-can-i-parse-a-date-containing-character-t/125432 "2018-03-24T09:31:19Z")
Hi, I am using Logstash GA 5.0.0. I have a date field like 2018-03-23T13:14:33.419Z. I am using data filter pattern yyyy-MM-ddTHH:mm:ss.SSSZ. My logstash is throwing error. How can I fix this? Thanks in advance. --- ## [Is Z in input time valid for Z in Logstash date filter?](https://discuss.elastic.co/t/is-z-in-input-time-valid-for-z-in-logstash-date-filter/125430)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [March 24, 2018, 9:29am UTC](https://discuss.elastic.co/t/is-z-in-input-time-valid-for-z-in-logstash-date-filter/125430 "2018-03-24T09:29:51Z")
Hi, I am using ELK GA 5.0.0. I am taking input from Kafka and I have a date field like 2018-03-23T13:14:33.419Z. In my date filter, I am using the format; date { match =\> \[ "log\_time", "yyyy-MM-ddTHH:mm:ss.SSSZ" \] ti… --- ## [How to process a text file separated by spaces using logstash?](https://discuss.elastic.co/t/how-to-process-a-text-file-separated-by-spaces-using-logstash/125311)
**Author:** [@mak705](https://discuss.elastic.co/u/mak705)\ **Replies:** 1\ **Last updated:** [March 24, 2018, 8:40am UTC](https://discuss.elastic.co/t/how-to-process-a-text-file-separated-by-spaces-using-logstash/125311 "2018-03-24T08:40:04Z")
My log file maker model mileage manufacture\_year engine\_displacement engine\_power body\_type color\_slug stk\_year transmission door\_count seat\_count fuel\_type date\_created date\_last\_seen price\_eur ford galaxy 151000 2011 … --- ## [Logstash grok not working](https://discuss.elastic.co/t/logstash-grok-not-working/125416)
**Author:** [@Benoit\_Martin](https://discuss.elastic.co/u/Benoit_Martin)\ **Replies:** 1\ **Last updated:** [March 24, 2018, 7:44am UTC](https://discuss.elastic.co/t/logstash-grok-not-working/125416 "2018-03-24T07:44:45Z")
Hi, I try to use regex syntax with grok/match. I don't have any error and the match not working. Anything wrong here ??? grok { match =\> { "\[event\_data\]\[param2\]" =\> "(?\(?\<=\\\)(.\*)(?=\\\<\\/UserId\\\>)… --- ## [Logstash Grok filter code](https://discuss.elastic.co/t/logstash-grok-filter-code/125418)
**Author:** [@VidyaKumar](https://discuss.elastic.co/u/VidyaKumar)\ **Replies:** 1\ **Last updated:** [March 24, 2018, 7:13am UTC](https://discuss.elastic.co/t/logstash-grok-filter-code/125418 "2018-03-24T07:13:36Z")
Hi all, Need your help with grok: How do I code for the following line? 6a 00001020 m 20420 ltel2drv 18/03/14 13:12:17.422057 CC0: DlschInfo: UE=16, SFNSF=7616, cell=PCC, msgType,nRB=000004, HID=0, HARQ\_12=10, nRTX\_12=… --- ## [Logstash Pipeline from 6.1 not working 6.2.1](https://discuss.elastic.co/t/logstash-pipeline-from-6-1-not-working-6-2-1/120548)
**Author:** [@wwalker](https://discuss.elastic.co/u/wwalker)\ **Replies:** 37\ **Last updated:** [March 24, 2018, 3:50am UTC](https://discuss.elastic.co/t/logstash-pipeline-from-6-1-not-working-6-2-1/120548 "2018-03-24T03:50:18Z")
This is exhausting.... Creating a guide and started a new ElasticStack setup from scratch, new VMs and all. Installed ElasticStack 6.2.1 with each application on it's own VM. Started Logstash with my pipeline with no … --- ## [How to parse json messages in elasticstack](https://discuss.elastic.co/t/how-to-parse-json-messages-in-elasticstack/125157)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 7\ **Last updated:** [March 24, 2018, 3:00am UTC](https://discuss.elastic.co/t/how-to-parse-json-messages-in-elasticstack/125157 "2018-03-24T03:00:20Z")
Hi Team, I am receiving messages in JSON format and wanted to know how do I directly send those in elastic? Can I directly store them in elastic without sending them through logstash or has to route it through logstash? … --- ## [Cleaning data](https://discuss.elastic.co/t/cleaning-data/125419)
**Author:** [@DC64](https://discuss.elastic.co/u/DC64)\ **Replies:** 1\ **Last updated:** [March 24, 2018, 1:43am UTC](https://discuss.elastic.co/t/cleaning-data/125419 "2018-03-24T01:43:02Z")
After making some changes to my Logstash configuration file, I still see the results of previous inputs of data in Kibana. I'm looking to delete my old data. On windows, where is the data saved? I'm using data that only… --- ## [Security Concerns - Logstash echoes credentials to stdout](https://discuss.elastic.co/t/security-concerns-logstash-echoes-credentials-to-stdout/124475)
**Author:** [@jameslamb](https://discuss.elastic.co/u/jameslamb)\ **Replies:** 4\ **Last updated:** [March 23, 2018, 7:56pm UTC](https://discuss.elastic.co/t/security-concerns-logstash-echoes-credentials-to-stdout/124475 "2018-03-23T19:56:08Z")
Hello, I was going to open an issue on GitHub, but the issue template directed me here. I'm very new to Logstash, but noticed something concerning tonight and wanted to ask if the community is aware of this. On failed … --- ## [Samples of grok filter](https://discuss.elastic.co/t/samples-of-grok-filter/124427)
**Author:** [@jason\_smith](https://discuss.elastic.co/u/jason_smith)\ **Replies:** 5\ **Last updated:** [March 23, 2018, 7:52pm UTC](https://discuss.elastic.co/t/samples-of-grok-filter/124427 "2018-03-23T19:52:44Z")
Hello folks, Being new to ELK, I struggled to understand from the following sample grok filter script which reads Apache log file as shown in the picture. I want to find out from this forum if there are any good sample… --- ## [Duplicate record exatly twice](https://discuss.elastic.co/t/duplicate-record-exatly-twice/125235)
**Author:** [@david.derosa](https://discuss.elastic.co/u/david.derosa)\ **Replies:** 7\ **Last updated:** [March 23, 2018, 7:06pm UTC](https://discuss.elastic.co/t/duplicate-record-exatly-twice/125235 "2018-03-23T19:06:00Z")
Hi all, i use elastisearch 6.2.2 in a really basic test environment (no cluster, single node). I've just realized that all records are duplicated exactly twice. The two records have different \_id but all other fields … --- ## [Can i use filebeat dashboards?](https://discuss.elastic.co/t/can-i-use-filebeat-dashboards/125402)
**Author:** [@yodog](https://discuss.elastic.co/u/yodog)\ **Replies:** 0\ **Last updated:** [March 23, 2018, 6:25pm UTC](https://discuss.elastic.co/t/can-i-use-filebeat-dashboards/125402 "2018-03-23T18:25:52Z")
due to the fact that i need the real @timestamp from the log lines, not when the line was received, i'll have to move from filebeat \> elasticsearch to filebeat \> logstash \> elasticsearch. shame, because i like the si… --- ## [If no ls output for a specific data stream does data get discarded?](https://discuss.elastic.co/t/if-no-ls-output-for-a-specific-data-stream-does-data-get-discarded/125093)
**Author:** [@ELK\_GB](https://discuss.elastic.co/u/ELK_GB)\ **Replies:** 4\ **Last updated:** [March 23, 2018, 6:06pm UTC](https://discuss.elastic.co/t/if-no-ls-output-for-a-specific-data-stream-does-data-get-discarded/125093 "2018-03-23T18:06:43Z")
Hi, I have a question in rergards to Logstash handling data if no output specified. For example, I send logs to ES based on some metadata that I specify on the source. lets say we have the following scenario: output … --- ## [Best way to parse a log folder](https://discuss.elastic.co/t/best-way-to-parse-a-log-folder/124992)
**Author:** [@sana](https://discuss.elastic.co/u/sana)\ **Replies:** 2\ **Last updated:** [March 23, 2018, 5:01pm UTC](https://discuss.elastic.co/t/best-way-to-parse-a-log-folder/124992 "2018-03-23T17:01:48Z")
i have 6 folders,, at the end of the day a new log file is added to each folder, i used logstash file input to parse the old files to logstash file { path =\> "/home/sana/pfe/docker/logs/\*\*/\*.log" type =\> "syslog" star… --- ## [Netflow module and \*.conf file](https://discuss.elastic.co/t/netflow-module-and-conf-file/125246)
**Author:** [@ggdagg](https://discuss.elastic.co/u/ggdagg)\ **Replies:** 2\ **Last updated:** [March 23, 2018, 4:57pm UTC](https://discuss.elastic.co/t/netflow-module-and-conf-file/125246 "2018-03-23T16:57:58Z")
Hi, I set up the Netflow module for Logstash and it works well. In my Logstash config (in /etc/logstash/logstash.yml), I added: modules: - name: netflow var.input.udp.port: 2055 var.kibana.host: "mykibanahost.com:56… --- ## [Filter for Kafka and Zookeeper System Logs](https://discuss.elastic.co/t/filter-for-kafka-and-zookeeper-system-logs/124276)
**Author:** [@jake\_smith](https://discuss.elastic.co/u/jake_smith)\ **Replies:** 3\ **Last updated:** [March 23, 2018, 11:35am UTC](https://discuss.elastic.co/t/filter-for-kafka-and-zookeeper-system-logs/124276 "2018-03-23T11:35:40Z")
Hi all - I'm new to elastic and grok. I'm trying to get Kafka/Zookeeper system logs filtered and parsed by Logstash (shipped by filebeat), then stored in Elasticsearch, so that they may be viewable on Kibana all on CentO… --- ## [Created timestamp on elasticsearch output](https://discuss.elastic.co/t/created-timestamp-on-elasticsearch-output/125384)
**Author:** [@JamesB](https://discuss.elastic.co/u/JamesB)\ **Replies:** 0\ **Last updated:** [March 23, 2018, 3:24pm UTC](https://discuss.elastic.co/t/created-timestamp-on-elasticsearch-output/125384 "2018-03-23T15:24:42Z")
On ES 6.2 and Logstash 6.1, currently I'm using doc\_as\_upsert with a document id defined by incoming fields. This is to allow updates. However I would also like to maintain a 'created at' date which will be set the first… --- ## [\_grokparsefailure but it works fine in dubugger](https://discuss.elastic.co/t/grokparsefailure-but-it-works-fine-in-dubugger/125233)
**Author:** [@miovinelli](https://discuss.elastic.co/u/miovinelli)\ **Replies:** 19\ **Last updated:** [March 23, 2018, 2:53pm UTC](https://discuss.elastic.co/t/grokparsefailure-but-it-works-fine-in-dubugger/125233 "2018-03-23T14:53:19Z")
Hi all, there is a strange situation cause my pattern works fine in grokdebug.herokuapp.com and also in a debugger offered by X-Pack, but at runtime ingestion events are tagged with "\_grokparsefailure" and I don't under… --- ## [Sending Windows Event Viewer logs w/o using Winlogbeat](https://discuss.elastic.co/t/sending-windows-event-viewer-logs-w-o-using-winlogbeat/125037)
**Author:** [@Feedy](https://discuss.elastic.co/u/Feedy)\ **Replies:** 1\ **Last updated:** [March 23, 2018, 2:24pm UTC](https://discuss.elastic.co/t/sending-windows-event-viewer-logs-w-o-using-winlogbeat/125037 "2018-03-23T14:24:58Z")
Is it possible to ship window event viewer logs from syslog-ng server to ELK? Normally I would install winlogbeat on windows server and ship directly to logstash but for this scenario, I won't be able to install an agent… --- ## [Logstash configuration file error](https://discuss.elastic.co/t/logstash-configuration-file-error/125373)
**Author:** [@CDR](https://discuss.elastic.co/u/CDR)\ **Replies:** 2\ **Last updated:** [March 23, 2018, 1:55pm UTC](https://discuss.elastic.co/t/logstash-configuration-file-error/125373 "2018-03-23T13:55:31Z")
For the life of me I cannot figure out why my logstash.conf file is wrong. input{ tcp{ host =\> "127.0.0.1" port =\> 514 } udp{ host =\> "127.0.0.1" port =\> 514 } } filter{ grok{ match =\> { “message” =\> … --- ## [Logstash: ruby filter: how to create empty event?](https://discuss.elastic.co/t/logstash-ruby-filter-how-to-create-empty-event/123271)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 3\ **Last updated:** [March 23, 2018, 1:53pm UTC](https://discuss.elastic.co/t/logstash-ruby-filter-how-to-create-empty-event/123271 "2018-03-23T13:53:15Z")
Hi, Is it possible to create additional empty events in ruby filter plugin? Only option I found in documentation https://www.elastic.co/guide/en/logstash/current/plugins-filters-ruby.html is to clone the current event. … --- ## [Configure SSL for each Elastic node](https://discuss.elastic.co/t/configure-ssl-for-each-elastic-node/125057)
**Author:** [@elasticprof](https://discuss.elastic.co/u/elasticprof)\ **Replies:** 4\ **Last updated:** [March 23, 2018, 1:30pm UTC](https://discuss.elastic.co/t/configure-ssl-for-each-elastic-node/125057 "2018-03-23T13:30:06Z")
Hi, I have 5 nodes and each node is having different ssl certificate, in such case how do configure ssl for each one of them input { stdin {} beats { port =\> 5044 } } #filter { # #} output { elas… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=793) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=795)