# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=777 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 778 --- ## [Logstash-output-file-as time based](https://discuss.elastic.co/t/logstash-output-file-as-time-based/128370)
**Author:** [@jawad846](https://discuss.elastic.co/u/jawad846)\ **Replies:** 3\ **Last updated:** [April 18, 2018, 7:50am UTC](https://discuss.elastic.co/t/logstash-output-file-as-time-based/128370 "2018-04-18T07:50:39Z")
Hi Folks, My output of logstash directed to the file called apache.log. This file needs to be generated in every hour. For Example: apache-2018-04-16-10:00.log or something similar to this. Here my configuration file … --- ## [Getting error: Error registering plugin](https://discuss.elastic.co/t/getting-error-error-registering-plugin/128456)
**Author:** [@sarang](https://discuss.elastic.co/u/sarang)\ **Replies:** 0\ **Last updated:** [April 18, 2018, 4:56am UTC](https://discuss.elastic.co/t/getting-error-error-registering-plugin/128456 "2018-04-18T04:56:53Z")
Hi , I am getting following error while running ls\_http.conf using command logstash -f D:\\ELK\\Input\\ls\_http.conf: \[ERROR\]\[logstash.pipeline \] Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"\ **Author:** [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\ **Replies:** 1\ **Last updated:** [April 18, 2018, 4:36am UTC](https://discuss.elastic.co/t/how-to-access-input-json-message-field/128451 "2018-04-18T04:36:56Z") Hi, i want to access type and logs field from input json here is input json { "type": "dmesg", "hostName": "ramesh-nagargoje", "os": "windo", "bseid": 1234, "logs": "\[ 0.000000\] Initializing cgroup subsys cpuse… --- ## [How to find error from logs using regex pattern?](https://discuss.elastic.co/t/how-to-find-error-from-logs-using-regex-pattern/128099)
**Author:** [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\ **Replies:** 8\ **Last updated:** [April 18, 2018, 4:25am UTC](https://discuss.elastic.co/t/how-to-find-error-from-logs-using-regex-pattern/128099 "2018-04-18T04:25:08Z")
regex Pattern which is saved in the local drive-in abc.json file and input of logs comes from Kafka server and send only error log snippet to elasticsearch if errors are not matched in abc.json then skip dumping of json … --- ## [Logstash - X-pack error conencting to ES](https://discuss.elastic.co/t/logstash-x-pack-error-conencting-to-es/128398)
**Author:** [@Robert\_Bardo](https://discuss.elastic.co/u/Robert_Bardo)\ **Replies:** 2\ **Last updated:** [April 17, 2018, 8:16pm UTC](https://discuss.elastic.co/t/logstash-x-pack-error-conencting-to-es/128398 "2018-04-17T20:16:19Z")
Hello, Does anyone know the cause of this error? Could not fetch all the sources {:exception=\>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=\>"Got response code '403' contacting Elas… --- ## [SNMP Trap Input](https://discuss.elastic.co/t/snmp-trap-input/128427)
**Author:** [@jasonyates](https://discuss.elastic.co/u/jasonyates)\ **Replies:** 0\ **Last updated:** [April 17, 2018, 7:52pm UTC](https://discuss.elastic.co/t/snmp-trap-input/128427 "2018-04-17T19:52:11Z")
I’m having some issues with the SNMP Trap Input. The aim is to parse Traps from Cisco Wireless LAN controllers, I have the device successfully trapping and Logstash is dumping to a file however I’m struggling to get anyt… --- ## [Logstash netflow codec v9 First/EndSwitched timestamps](https://discuss.elastic.co/t/logstash-netflow-codec-v9-first-endswitched-timestamps/128415)
**Author:** [@fc4min0s](https://discuss.elastic.co/u/fc4min0s)\ **Replies:** 0\ **Last updated:** [April 17, 2018, 6:42pm UTC](https://discuss.elastic.co/t/logstash-netflow-codec-v9-first-endswitched-timestamps/128415 "2018-04-17T18:42:50Z")
Hello all !! I am testing the netflow logstash plugin and there seems to me there is a problem with the timestamps as the documents are not showing millisecond precision. A wireshark capture shows StartTime/EndTime fie… --- ## [Logstash Crashing when running via systemctl vs running it on command line](https://discuss.elastic.co/t/logstash-crashing-when-running-via-systemctl-vs-running-it-on-command-line/128263)
**Author:** [@kseth](https://discuss.elastic.co/u/kseth)\ **Replies:** 2\ **Last updated:** [April 17, 2018, 5:30pm UTC](https://discuss.elastic.co/t/logstash-crashing-when-running-via-systemctl-vs-running-it-on-command-line/128263 "2018-04-17T17:30:17Z")
I have a Logstash process that's running to delete documents from a Elasticsearch index when they are actually updated in the underlying db. It runs fine when I run it via command line like below: sudo /usr/share/logsta… --- ## [Json filter with a problem](https://discuss.elastic.co/t/json-filter-with-a-problem/128269)
**Author:** [@juandanielpujols](https://discuss.elastic.co/u/juandanielpujols)\ **Replies:** 1\ **Last updated:** [April 17, 2018, 4:52pm UTC](https://discuss.elastic.co/t/json-filter-with-a-problem/128269 "2018-04-17T16:52:29Z")
Hi, i´m new at this and i haven't been able to separate a json response from an api from the word "Ok" at the beginning. Please help. Below the response. OK: \[ { "EPOLeafNode.LastUpdate" : null, "EPOLeafNode.NodeName" … --- ## [Logstash 6.1.1 : jar-dependencies conflicts](https://discuss.elastic.co/t/logstash-6-1-1-jar-dependencies-conflicts/126131)
**Author:** [@samia](https://discuss.elastic.co/u/samia)\ **Replies:** 8\ **Last updated:** [April 17, 2018, 3:25pm UTC](https://discuss.elastic.co/t/logstash-6-1-1-jar-dependencies-conflicts/126131 "2018-04-17T15:25:04Z")
Hello ! I'm running Logstash 6.1.1 I've updated a logstash plugin : logstash-input-udp. Since this update I'm having apparently a conflict between different versions of jar-dependencies. Here's what I find in the lo… --- ## [Connecting Logstash to Elastic Cloud](https://discuss.elastic.co/t/connecting-logstash-to-elastic-cloud/128362)
**Author:** [@BezVeze](https://discuss.elastic.co/u/BezVeze)\ **Replies:** 4\ **Last updated:** [April 17, 2018, 3:05pm UTC](https://discuss.elastic.co/t/connecting-logstash-to-elastic-cloud/128362 "2018-04-17T15:05:58Z")
Hi there, Can anyone help me around with connecting logstash to elastic cloud ive been struggling a little for a while... let me tell you how my environment is: I have google cloud compute engine machine that have fil… --- ## [ConfigurationError in Logstash](https://discuss.elastic.co/t/configurationerror-in-logstash/128377)
**Author:** [@SRIpandu1729](https://discuss.elastic.co/u/SRIpandu1729)\ **Replies:** 1\ **Last updated:** [April 17, 2018, 2:33pm UTC](https://discuss.elastic.co/t/configurationerror-in-logstash/128377 "2018-04-17T14:33:00Z")
I have 2 inputs, 1 coming from file and another from filebeat from a remote server. I need to index these into elasticsearch. The output should send them to 2 different indes patterns. So I configured my logstash as foll… --- ## [Testing Grok / Mutations on existing data?](https://discuss.elastic.co/t/testing-grok-mutations-on-existing-data/128351)
**Author:** [@devployment](https://discuss.elastic.co/u/devployment)\ **Replies:** 3\ **Last updated:** [April 17, 2018, 2:31pm UTC](https://discuss.elastic.co/t/testing-grok-mutations-on-existing-data/128351 "2018-04-17T14:31:01Z")
Hello, I'm new to ELK. So I'm not 100% where this message fits exactly. It's partly Logstash and partly Elastic I guess. What I currently have. I'm running the ELK stack locally for testing purposes. A Logstash plugin … --- ## [Timezone offset parse issue](https://discuss.elastic.co/t/timezone-offset-parse-issue/128310)
**Author:** [@Ragi](https://discuss.elastic.co/u/Ragi)\ **Replies:** 4\ **Last updated:** [April 17, 2018, 2:02pm UTC](https://discuss.elastic.co/t/timezone-offset-parse-issue/128310 "2018-04-17T14:02:36Z")
When i try to parse the following line 2018-04-17T10:04:15.693+0200 INFO \[dmt\_scheduler\_Worker-4\] \[SampleClass:60\] testMethod executed I found the following error in the log file {:timestamp=\>"2018-04-17T10:04:15.6930… --- ## [Logstash nested hashmap to top level](https://discuss.elastic.co/t/logstash-nested-hashmap-to-top-level/128346)
**Author:** [@pgprb](https://discuss.elastic.co/u/pgprb)\ **Replies:** 2\ **Last updated:** [April 17, 2018, 1:53pm UTC](https://discuss.elastic.co/t/logstash-nested-hashmap-to-top-level/128346 "2018-04-17T13:53:40Z")
Hi there, So basically I have a very complex object (input is avro) that has nested lists, hashmaps and objects inside. Simplifying, something like this: ObjectsA { #list of object A A1 { UUID; HashMap1 { Key: Obj… --- ## [Nested Json with ruby filter](https://discuss.elastic.co/t/nested-json-with-ruby-filter/128193)
**Author:** [@chrigui94](https://discuss.elastic.co/u/chrigui94)\ **Replies:** 11\ **Last updated:** [April 17, 2018, 12:57pm UTC](https://discuss.elastic.co/t/nested-json-with-ruby-filter/128193 "2018-04-17T12:57:15Z")
I want to index this JSON file : {"environment":"urban","envType":"outdoor","positionNumber":1,"frameIndex":15,"fcntUp":22324,"numberOfGateways":3,"numberOfTimestamps":0,"serverTimestamp":"23:44:36","lnsAppEui":"4883C7D… --- ## [Log file per pipeline](https://discuss.elastic.co/t/log-file-per-pipeline/128366)
**Author:** [@CasMeiron](https://discuss.elastic.co/u/CasMeiron)\ **Replies:** 0\ **Last updated:** [April 17, 2018, 1:45pm UTC](https://discuss.elastic.co/t/log-file-per-pipeline/128366 "2018-04-17T13:45:30Z")
Hi, I'm using 4 pipelines with a single Logstash instance. The logs are all mixed up in a single Logstash output log file (/var/log/logstash/logstash-plain.log). Is that possible to say like "pipeline.id=A directs to th… --- ## [JDBC Input plugin : Port number is invalid](https://discuss.elastic.co/t/jdbc-input-plugin-port-number-is-invalid/128340)
**Author:** [@pvderivco](https://discuss.elastic.co/u/pvderivco)\ **Replies:** 4\ **Last updated:** [April 17, 2018, 1:04pm UTC](https://discuss.elastic.co/t/jdbc-input-plugin-port-number-is-invalid/128340 "2018-04-17T13:04:19Z")
I have a configuration for connecting to a SQL Server on my local machine. While trying to connect from docker container, I get below error: Error: Java::ComMicrosoftSqlserverJdbc::SQLServerException: The port number 1… --- ## [Logstash for java server logs](https://discuss.elastic.co/t/logstash-for-java-server-logs/128266)
**Author:** [@adg](https://discuss.elastic.co/u/adg)\ **Replies:** 4\ **Last updated:** [April 17, 2018, 12:09pm UTC](https://discuss.elastic.co/t/logstash-for-java-server-logs/128266 "2018-04-17T12:09:51Z")
Hi i m having issue while parsing Java Exception stack trace, here we customize our logs which include some custom parameter just after time stamp Ex 10:10:03,463#l-21778964801742026### ERROR \[PushNotificationServic… --- ## [How I can read this nginx format with logstash?](https://discuss.elastic.co/t/how-i-can-read-this-nginx-format-with-logstash/128308)
**Author:** [@astropanic](https://discuss.elastic.co/u/astropanic)\ **Replies:** 1\ **Last updated:** [April 17, 2018, 11:52am UTC](https://discuss.elastic.co/t/how-i-can-read-this-nginx-format-with-logstash/128308 "2018-04-17T11:52:29Z")
I can't get my NGINX logs to be parsable by logstash. NGINX config: log\_format main '"$http\_x\_forwarded\_for - $remote\_user \[$time\_local\]" "$host" "$request" "$status" "$bytes\_sent" "$http\_referer" "$http\_user\_agent" … --- ## [Logstash combining multiple entries](https://discuss.elastic.co/t/logstash-combining-multiple-entries/128314)
**Author:** [@prateek\_shrivastava](https://discuss.elastic.co/u/prateek_shrivastava)\ **Replies:** 5\ **Last updated:** [April 17, 2018, 11:48am UTC](https://discuss.elastic.co/t/logstash-combining-multiple-entries/128314 "2018-04-17T11:48:54Z")
My config file looks like: input { file { path =\> "path" start\_position =\> "beginning" codec =\> multiline { pattern =\> "^\\s" what =\> "previous" } } } filter { grok { match =\> \[ "message", "(?… --- ## [Indexing email to elastic search](https://discuss.elastic.co/t/indexing-email-to-elastic-search/128321)
**Author:** [@rouchad\_rouchad](https://discuss.elastic.co/u/rouchad_rouchad)\ **Replies:** 2\ **Last updated:** [April 17, 2018, 11:34am UTC](https://discuss.elastic.co/t/indexing-email-to-elastic-search/128321 "2018-04-17T11:34:56Z")
i have a problem with the file email.conf is not working i got this error : Stack: org/jruby/ext/openssl/SSLSocket.java:218:in connect' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/imap.rb:1492:instart\_tls\_… --- ## [Couldn't convert date\_time type](https://discuss.elastic.co/t/couldnt-convert-date-time-type/128276)
**Author:** [@hskang](https://discuss.elastic.co/u/hskang)\ **Replies:** 2\ **Last updated:** [April 17, 2018, 11:20am UTC](https://discuss.elastic.co/t/couldnt-convert-date-time-type/128276 "2018-04-17T11:20:24Z")
I want to convert the column from string to date\_time but nothing changed. log 2018-04-13T05:37:52.874Z|8|200|0|USR|1510|\[Fire...|104|2018-04-12T11:05:15.352Z|201|Pacific|1510|\[Fire...|R110|2018-04-12T11:05:59.352Z co… --- ## [Logstatsh issue related to stdout](https://discuss.elastic.co/t/logstatsh-issue-related-to-stdout/128212)
**Author:** [@amritesh\_mishra](https://discuss.elastic.co/u/amritesh_mishra)\ **Replies:** 4\ **Last updated:** [April 17, 2018, 11:01am UTC](https://discuss.elastic.co/t/logstatsh-issue-related-to-stdout/128212 "2018-04-17T11:01:59Z")
\[root@ukmr0xtmd01 logstash\]# /usr/share/logstash/bin/logstash -f siae.conf WARNING: Could not find logstash.yml which is typically located in $LS\_HOME/config or /etc/logstash. You can specify the path using --path.setti… --- ## [How to send event to other elasticsearch](https://discuss.elastic.co/t/how-to-send-event-to-other-elasticsearch/128332)
**Author:** [@chenzilong444](https://discuss.elastic.co/u/chenzilong444)\ **Replies:** 2\ **Last updated:** [April 17, 2018, 10:19am UTC](https://discuss.elastic.co/t/how-to-send-event-to-other-elasticsearch/128332 "2018-04-17T10:19:45Z")
output { elasticsearch { hosts =\> \["114.113.198.147:9200"\] index =\> "log" document\_type =\>"logstash" } stdout { codec =\> rubydebug } } \[2018-04-17T18:10:22,805\]\[IN… --- ## [Seeing Grok regexp exception "incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string)"](https://discuss.elastic.co/t/seeing-grok-regexp-exception-incompatible-encoding-regexp-match-utf-8-regexp-with-ascii-8bit-string/120084)
**Author:** [@cb2015](https://discuss.elastic.co/u/cb2015)\ **Replies:** 10\ **Last updated:** [April 17, 2018, 9:07am UTC](https://discuss.elastic.co/t/seeing-grok-regexp-exception-incompatible-encoding-regexp-match-utf-8-regexp-with-ascii-8bit-string/120084 "2018-04-17T09:07:11Z")
Hi everyone, I am seeing this error in my logstash logs \[2018-02-15T10:00:48,009\]\[WARN \]\[logstash.filters.grok \] Grok regexp threw exception {:exception=\>"incompatible encoding regexp match (UTF-8 regexp with ASCII-… --- ## [Yet another date parse problem (noob)](https://discuss.elastic.co/t/yet-another-date-parse-problem-noob/127986)
**Author:** [@robs](https://discuss.elastic.co/u/robs)\ **Replies:** 6\ **Last updated:** [April 17, 2018, 8:06am UTC](https://discuss.elastic.co/t/yet-another-date-parse-problem-noob/127986 "2018-04-17T08:06:32Z")
Hi, I want to send log lines to elastic with logstash. I'm burning hours and can't get it right. My log lines all look like: yyyymmdd hh:mm:ss ev:1 rn:3 (like : 20170312 14:03:55 ev:1 rn:5 etc. etc.) In patterns\_di… --- ## [Logstash Filter CSV last row not reading](https://discuss.elastic.co/t/logstash-filter-csv-last-row-not-reading/128005)
**Author:** [@sabdoul](https://discuss.elastic.co/u/sabdoul)\ **Replies:** 3\ **Last updated:** [April 17, 2018, 7:59am UTC](https://discuss.elastic.co/t/logstash-filter-csv-last-row-not-reading/128005 "2018-04-17T07:59:48Z")
Hello, I have my last line of csv data that I get with http\_poller plugin that is not imported. Some people have solved the problem by adding a blank newline at the end of the csv file. But me in my case my file is reco… --- ## [CSV filter, one line of data is missing](https://discuss.elastic.co/t/csv-filter-one-line-of-data-is-missing/127387)
**Author:** [@resuhp](https://discuss.elastic.co/u/resuhp)\ **Replies:** 4\ **Last updated:** [April 17, 2018, 7:10am UTC](https://discuss.elastic.co/t/csv-filter-one-line-of-data-is-missing/127387 "2018-04-17T07:10:36Z")
Hi, first of all, I would like to say that I'm very new to elk stack and have some problems with understanding what I'm doing wrong. I'm having a problem after Logstash indexing. Let's say that I have one log file with… --- ## [Filebeat vs Logstash](https://discuss.elastic.co/t/filebeat-vs-logstash/127485)
**Author:** [@jawad846](https://discuss.elastic.co/u/jawad846)\ **Replies:** 83\ **Last updated:** [April 16, 2018, 9:33am UTC](https://discuss.elastic.co/t/filebeat-vs-logstash/127485 "2018-04-16T09:33:22Z")
Please give me the solution for forwarding the log to logstash from filebeat, were the filebeat and logstash are in different VM am getting the error of : 2018-04-10T11:39:05.750Z ERROR pipeline/output.go:74 Failed to … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=776) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=778)