# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1012 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1013 --- ## [Timestamp format in logstash 1.5.6](https://discuss.elastic.co/t/timestamp-format-in-logstash-1-5-6/79755)
**Author:** [@manopmk](https://discuss.elastic.co/u/manopmk)\ **Replies:** 8\ **Last updated:** [March 24, 2017, 6:02am UTC](https://discuss.elastic.co/t/timestamp-format-in-logstash-1-5-6/79755 "2017-03-24T06:02:30Z")
My log have this format 11-30-2016 12:13:57 INFO - Mapped "{\[/transaction/get-account-history\],methods=\[POST\],params=\[\],headers=\[\],consumes=\[\],produces=\[\],custom=\[\]}" then how do i include timestamp format in filter … --- ## [Logstash not reading remote log files](https://discuss.elastic.co/t/logstash-not-reading-remote-log-files/54919)
**Author:** [@Suren92](https://discuss.elastic.co/u/Suren92)\ **Replies:** 12\ **Last updated:** [March 24, 2017, 6:00am UTC](https://discuss.elastic.co/t/logstash-not-reading-remote-log-files/54919 "2017-03-24T06:00:36Z")
Hi, I have some log files in a remote system, I need to send those to elasticsearch. But logstash is not reading the remote log files. I have specified the following configuration for logstash. input { file{ path =\> … --- ## [Parsing log with partial json](https://discuss.elastic.co/t/parsing-log-with-partial-json/79674)
**Author:** [@mikemi](https://discuss.elastic.co/u/mikemi)\ **Replies:** 3\ **Last updated:** [March 24, 2017, 1:19am UTC](https://discuss.elastic.co/t/parsing-log-with-partial-json/79674 "2017-03-24T01:19:22Z")
2017-03-21T23:53:10-04:00 INFO login {"input":"user123","success":false} {"host":"www.example.com","path":"user","clientip":"127.0.0.1"} This log file starts with a timestmap, loglevel, and event type. Next are 2 sec… --- ## [Logstash -Could not find any executable java binary. Please install java](https://discuss.elastic.co/t/logstash-could-not-find-any-executable-java-binary-please-install-java/79774)
**Author:** [@Bob\_Metelsky](https://discuss.elastic.co/u/Bob_Metelsky)\ **Replies:** 4\ **Last updated:** [March 23, 2017, 9:45pm UTC](https://discuss.elastic.co/t/logstash-could-not-find-any-executable-java-binary-please-install-java/79774 "2017-03-23T21:45:23Z")
Folks - somehow the 5.2 logstash cant find JAVA 5.2 ES & kibana run fine nn775 ~ :( # rpm -q logstash logstash-5.2.2-1.noarch nn775 ~ # /usr/share/logstash/bin/logstash Could not find any executable java binary. Pleas… --- ## [How to specify multiple filters in cloudwatch plugin?](https://discuss.elastic.co/t/how-to-specify-multiple-filters-in-cloudwatch-plugin/79811)
**Author:** [@tri](https://discuss.elastic.co/u/tri)\ **Replies:** 0\ **Last updated:** [March 23, 2017, 8:49pm UTC](https://discuss.elastic.co/t/how-to-specify-multiple-filters-in-cloudwatch-plugin/79811 "2017-03-23T20:49:16Z")
Hi all, Just wonder if anybody have a working example of using more than one values in the cloudwatch plugin filters parameter? I have a situation where I need to specify a number of tags in the filters parameter in ord… --- ## [Any plans for a visual Logstash pipeline editor?](https://discuss.elastic.co/t/any-plans-for-a-visual-logstash-pipeline-editor/54574)
**Author:** [@val](https://discuss.elastic.co/u/val)\ **Replies:** 4\ **Last updated:** [March 23, 2017, 7:26pm UTC](https://discuss.elastic.co/t/any-plans-for-a-visual-logstash-pipeline-editor/54574 "2017-03-23T19:26:24Z")
I was wondering if there were any plans of providing a visual pipeline editor for Logstash at some point? After writing tons of Logstash configuration files in many different projects over the past few years, I started f… --- ## [Smtp authentication error](https://discuss.elastic.co/t/smtp-authentication-error/79647)
**Author:** [@August\_rush](https://discuss.elastic.co/u/August_rush)\ **Replies:** 6\ **Last updated:** [March 23, 2017, 6:05pm UTC](https://discuss.elastic.co/t/smtp-authentication-error/79647 "2017-03-23T18:05:47Z")
I am trying to send emails email { port =\> 587 address =\> "smtp.gmail.com" username =\> "@gmail.com" password =\> "" authentication =\> "plain" use\_tls =\> true from =\> "@gmail.com" subject =\> "logstash alert" to =\>… --- ## [Need help configuring my grok filters](https://discuss.elastic.co/t/need-help-configuring-my-grok-filters/79500)
**Author:** [@Rich\_Johnson](https://discuss.elastic.co/u/Rich_Johnson)\ **Replies:** 5\ **Last updated:** [March 23, 2017, 6:02pm UTC](https://discuss.elastic.co/t/need-help-configuring-my-grok-filters/79500 "2017-03-23T18:02:09Z")
As I am only starting to test with ElasticStack, I have only configured 3 hosts with filebeat installed to forward apache logs to logstash. My grok filter looks like this: input { beats { port =\> "5044" … --- ## [Best Input to use for Bulk Loop API requests](https://discuss.elastic.co/t/best-input-to-use-for-bulk-loop-api-requests/79785)
**Author:** [@Dallas\_Toth](https://discuss.elastic.co/u/Dallas_Toth)\ **Replies:** 0\ **Last updated:** [March 23, 2017, 5:19pm UTC](https://discuss.elastic.co/t/best-input-to-use-for-bulk-loop-api-requests/79785 "2017-03-23T17:19:49Z")
I've been battling with http\_poller and exec and pipe and can not seem to find the best method to accomplish this below input. I have a API that only returns 100 results for each call and I need to bring in records for… --- ## [Unable to copy data from one ES instance to another using LS](https://discuss.elastic.co/t/unable-to-copy-data-from-one-es-instance-to-another-using-ls/79772)
**Author:** [@crags](https://discuss.elastic.co/u/crags)\ **Replies:** 0\ **Last updated:** [March 23, 2017, 4:05pm UTC](https://discuss.elastic.co/t/unable-to-copy-data-from-one-es-instance-to-another-using-ls/79772 "2017-03-23T16:05:10Z")
Hi. I'm trying to copy some data from our production ES stack to our stage stack for testing using logstash, and for some reason it's just not working. I think it may be to do with the query, but I'm not sure for defin… --- ## [Logstash throughput drops to 0 with no apparent memory problems and 0 cpu usage](https://discuss.elastic.co/t/logstash-throughput-drops-to-0-with-no-apparent-memory-problems-and-0-cpu-usage/79596)
**Author:** [@David\_McClain](https://discuss.elastic.co/u/David_McClain)\ **Replies:** 8\ **Last updated:** [March 23, 2017, 3:05pm UTC](https://discuss.elastic.co/t/logstash-throughput-drops-to-0-with-no-apparent-memory-problems-and-0-cpu-usage/79596 "2017-03-23T15:05:37Z")
Logstash Version: 2.4 Elasticsearch Version: 2.3.4 Redis version: 2.19 I have an indexing setup where Logstash reads from Redis, parses the data, and indexes into Elasticsearch (fairly standard). I monitor the list si… --- ## [Trouble parsing date with date filter](https://discuss.elastic.co/t/trouble-parsing-date-with-date-filter/79646)
**Author:** [@jclose](https://discuss.elastic.co/u/jclose)\ **Replies:** 4\ **Last updated:** [March 23, 2017, 2:51pm UTC](https://discuss.elastic.co/t/trouble-parsing-date-with-date-filter/79646 "2017-03-23T14:51:33Z")
I'm having trouble with some date parsing. The entry I'm getting doesn't have a year in the datestamp, so I'm trying to pull the year out of @timestamp, and then put the proper timestamp back into @timestamp. So my da… --- ## [Logstash timestamp/time/date problems](https://discuss.elastic.co/t/logstash-timestamp-time-date-problems/79753)
**Author:** [@hologram](https://discuss.elastic.co/u/hologram)\ **Replies:** 3\ **Last updated:** [March 23, 2017, 1:58pm UTC](https://discuss.elastic.co/t/logstash-timestamp-time-date-problems/79753 "2017-03-23T13:58:47Z")
Hi all I need some help. I set my first elk this morning . I used this tutorial. https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-7 At the end I wa… --- ## [Skipping older files](https://discuss.elastic.co/t/skipping-older-files/79653)
**Author:** [@blacklobo](https://discuss.elastic.co/u/blacklobo)\ **Replies:** 4\ **Last updated:** [March 23, 2017, 1:46pm UTC](https://discuss.elastic.co/t/skipping-older-files/79653 "2017-03-23T13:46:01Z")
How can I get Logstash to open and process files modified more than 24 hours ago? Logstash is skipping files that were written over 24 hours ago. I found a topic that references the Logstash forwarder accepting a "dea… --- ## [Sending post request via nodejs to logstash](https://discuss.elastic.co/t/sending-post-request-via-nodejs-to-logstash/79269)
**Author:** [@nabiliii](https://discuss.elastic.co/u/nabiliii)\ **Replies:** 4\ **Last updated:** [March 23, 2017, 1:34pm UTC](https://discuss.elastic.co/t/sending-post-request-via-nodejs-to-logstash/79269 "2017-03-23T13:34:04Z")
Hi everyone! I want to sent http post requests via nodejs code to a logstash with a basic configuration file : input { http { } } output { elasticsearch { hosts =\> \["localhost:9200"\] } stdout { codec =\> rubydebug… --- ## [Date filter in logstash.conf not parsing the date while grok parses it by timestamp\_iso8601](https://discuss.elastic.co/t/date-filter-in-logstash-conf-not-parsing-the-date-while-grok-parses-it-by-timestamp-iso8601/30104)
**Author:** [@bubu.senapati](https://discuss.elastic.co/u/bubu.senapati)\ **Replies:** 20\ **Last updated:** [March 23, 2017, 1:31pm UTC](https://discuss.elastic.co/t/date-filter-in-logstash-conf-not-parsing-the-date-while-grok-parses-it-by-timestamp-iso8601/30104 "2017-03-23T13:31:23Z")
I have a date field in a logfile that follows as 2015-09-26 03:45:12:051-0400. I want to parse it which i am successfully able to do by using TIMESTAMP\_ISO8601 grok pattern. But the result is a string field which is succ… --- ## [Plugin Installation behind firewall](https://discuss.elastic.co/t/plugin-installation-behind-firewall/79744)
**Author:** [@Tim\_James](https://discuss.elastic.co/u/Tim_James)\ **Replies:** 0\ **Last updated:** [March 23, 2017, 12:36pm UTC](https://discuss.elastic.co/t/plugin-installation-behind-firewall/79744 "2017-03-23T12:36:58Z")
Hi, I'm trying to install the beats plugin for logstash. I get the following error which I presume is because we can't access the internet: ERROR: Something went wrong when installing logstash-input-beats, message: N… --- ## [String manipulation and format them to timestamp](https://discuss.elastic.co/t/string-manipulation-and-format-them-to-timestamp/79727)
**Author:** [@Juergen\_Kerner](https://discuss.elastic.co/u/Juergen_Kerner)\ **Replies:** 2\ **Last updated:** [March 23, 2017, 11:05am UTC](https://discuss.elastic.co/t/string-manipulation-and-format-them-to-timestamp/79727 "2017-03-23T11:05:54Z")
Hi guys, i am new to logstash and i am stuck with following problem: I have something like a timestamp: 1703220054 First 2 digits indicates year YY 17 Next 2 digits indicates MM 03 Next 2 digits indicates dd 22 Nex… --- ## [Default @timestamp not overridden](https://discuss.elastic.co/t/default-timestamp-not-overridden/79725)
**Author:** [@Fabrizio\_Micheloni](https://discuss.elastic.co/u/Fabrizio_Micheloni)\ **Replies:** 1\ **Last updated:** [March 23, 2017, 10:50am UTC](https://discuss.elastic.co/t/default-timestamp-not-overridden/79725 "2017-03-23T10:50:20Z")
Hi all! I have the following scenario. Example of my log 2017-03-22 16:24:49.710 | abc | def | ghi | 127.0.0.1 | | My message I'm able to parse it properly with a grok filter and trying to override the default t… --- ## [Use multiple codec in one file input](https://discuss.elastic.co/t/use-multiple-codec-in-one-file-input/79722)
**Author:** [@stef35340](https://discuss.elastic.co/u/stef35340)\ **Replies:** 2\ **Last updated:** [March 23, 2017, 10:10am UTC](https://discuss.elastic.co/t/use-multiple-codec-in-one-file-input/79722 "2017-03-23T10:10:09Z")
Hello, Do you know if there is a possibility to use multiple codecs in the same file input? I need to manage "multiline" and "plain" codecs for the same file. Currently, here is my configuration (I have anonymized da… --- ## [Parsing word and pdf file in elasticsearch using logstash](https://discuss.elastic.co/t/parsing-word-and-pdf-file-in-elasticsearch-using-logstash/79212)
**Author:** [@Nikparab](https://discuss.elastic.co/u/Nikparab)\ **Replies:** 13\ **Last updated:** [March 23, 2017, 9:12am UTC](https://discuss.elastic.co/t/parsing-word-and-pdf-file-in-elasticsearch-using-logstash/79212 "2017-03-23T09:12:09Z")
I m trying to read pdf file using logstash i use following config file input { file { path =\> "D:\\ELKstack\\elastic\\page\_ELK.pdf" start\_position =\> "beginning" } } filter { if \[type\] == ".pdf… --- ## [Help with Grok Pattern for SAS log](https://discuss.elastic.co/t/help-with-grok-pattern-for-sas-log/79584)
**Author:** [@DamonWolmarans](https://discuss.elastic.co/u/DamonWolmarans)\ **Replies:** 2\ **Last updated:** [March 23, 2017, 7:59am UTC](https://discuss.elastic.co/t/help-with-grok-pattern-for-sas-log/79584 "2017-03-23T07:59:21Z")
Hi! I have just started playing with the Elastic Stack and I am a complete noob, so please treat this as such. I need help in constructing a grok filter for some logs i am parsing to Logstash, more specifically I nee… --- ## [Logstash Error : Expected one of #, ", ', -, \[, {, \] at line 4, column 14 (byte 33) after input { file { path =\> \[](https://discuss.elastic.co/t/logstash-error-expected-one-of-at-line-4-column-14-byte-33-after-input-file-path/79532)
**Author:** [@manopmk](https://discuss.elastic.co/u/manopmk)\ **Replies:** 13\ **Last updated:** [March 23, 2017, 7:00am UTC](https://discuss.elastic.co/t/logstash-error-expected-one-of-at-line-4-column-14-byte-33-after-input-file-path/79532 "2017-03-23T07:00:39Z")
Im Using mac im unable to load apache log to logstash. Im getting error " Error: Expected one of #, ", ', -, \[, {, \] at line 4, column 14 (byte 33) after input { file { path =\> \[ I tried path =\> "/Users/tcstsb3/Down… --- ## [Logstash does not start main pipeline and produces no output on Ubuntu 16.04](https://discuss.elastic.co/t/logstash-does-not-start-main-pipeline-and-produces-no-output-on-ubuntu-16-04/79650)
**Author:** [@esbenp](https://discuss.elastic.co/u/esbenp)\ **Replies:** 3\ **Last updated:** [March 23, 2017, 5:41am UTC](https://discuss.elastic.co/t/logstash-does-not-start-main-pipeline-and-produces-no-output-on-ubuntu-16-04/79650 "2017-03-23T05:41:42Z")
I have installed Logstash and Elasticsearch on an Ubuntu 16.04 machine with 4 gb ram using apt-get install logstash. When I boot the service with systemctl start logstash it does start the process (I can see it using p… --- ## [Default elasticsearch template for logstash](https://discuss.elastic.co/t/default-elasticsearch-template-for-logstash/79685)
**Author:** [@no\_jihun](https://discuss.elastic.co/u/no_jihun)\ **Replies:** 1\ **Last updated:** [March 23, 2017, 5:20am UTC](https://discuss.elastic.co/t/default-elasticsearch-template-for-logstash/79685 "2017-03-23T05:20:59Z")
Where can I find the default index template which used by logstash to create elasticsearch index? (of version 5) --- ## [Is Codec =\> gzip\_lines supported on 5.1.2?](https://discuss.elastic.co/t/is-codec-gzip-lines-supported-on-5-1-2/79684)
**Author:** [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\ **Replies:** 0\ **Last updated:** [March 23, 2017, 4:51am UTC](https://discuss.elastic.co/t/is-codec-gzip-lines-supported-on-5-1-2/79684 "2017-03-23T04:51:52Z")
I am trying to read a local gzipped file with below, but i can't even install gzip. Error also below. Is it supported yet? file { path =\> "/Users/alliey/Desktop/log\_file/generic20161017.log.gz" start\_position =\> "begin… --- ## [Logstash pipe line block utf8 ascii-8bit](https://discuss.elastic.co/t/logstash-pipe-line-block-utf8-ascii-8bit/55107)
**Author:** [@Sadegh\_Solati](https://discuss.elastic.co/u/Sadegh_Solati)\ **Replies:** 1\ **Last updated:** [March 23, 2017, 2:51am UTC](https://discuss.elastic.co/t/logstash-pipe-line-block-utf8-ascii-8bit/55107 "2017-03-23T02:51:51Z")
Hi It's several days I'm trying to figure out what is causing my elk setup crashing. I found the log which causes this issue. Jul 2 05:45:58 web1gr nginx: 2016/07/02 05:45:58 \[error\] 63#63: \*14012402 FastCGI sent in s… --- ## [Grok to parse SAP NetWeaver access logs](https://discuss.elastic.co/t/grok-to-parse-sap-netweaver-access-logs/79664)
**Author:** [@zhmq0124v](https://discuss.elastic.co/u/zhmq0124v)\ **Replies:** 1\ **Last updated:** [March 23, 2017, 2:02am UTC](https://discuss.elastic.co/t/grok-to-parse-sap-netweaver-access-logs/79664 "2017-03-23T02:02:06Z")
Hi, I am trying to parse the following log but failed, as you can see first 12 lines are not needed and from line 12 it's CATALINA\_DATESTAMP, client ip, http verb, URL, http version, http code, data size and response ti… --- ## [Logstash into another server](https://discuss.elastic.co/t/logstash-into-another-server/79364)
**Author:** [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\ **Replies:** 5\ **Last updated:** [March 22, 2017, 9:07pm UTC](https://discuss.elastic.co/t/logstash-into-another-server/79364 "2017-03-22T21:07:09Z")
Just want to confirm I am doing the right thing: if i want to logstash to another ES server at https://example.com, should i just do below in logstash config output section? I do need the port there, right? output { … --- ## [Input UDP/ netflow no listenning](https://discuss.elastic.co/t/input-udp-netflow-no-listenning/79600)
**Author:** [@pierre1](https://discuss.elastic.co/u/pierre1)\ **Replies:** 1\ **Last updated:** [March 22, 2017, 10:09pm UTC](https://discuss.elastic.co/t/input-udp-netflow-no-listenning/79600 "2017-03-22T22:09:45Z")
hey, i have a problem with UDP this is my configuration : input { udp { port =\> 2055 type =\> "Netflow" codec =\> netflow { versions =\> \[9\] } } } output { if ( \[type\] == "Netflow" ) … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1011) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1013)