# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=989 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 990 --- ## [Elk stucked when input data from local file](https://discuss.elastic.co/t/elk-stucked-when-input-data-from-local-file/84658)
**Author:** [@JIELOGAN](https://discuss.elastic.co/u/JIELOGAN)\ **Replies:** 4\ **Last updated:** [May 5, 2017, 9:08am UTC](https://discuss.elastic.co/t/elk-stucked-when-input-data-from-local-file/84658 "2017-05-05T09:08:14Z")
hi i wanted import my data to elasticsearch through logstash ,the logstash.conf is \`input { file { path =\> "/home/data/accounts.json" } } output { elasticsearch { hosts =\> \["172.16.0.10:9200"\] index =\> "accounts" } } … --- ## [Extract field from json to the top-level](https://discuss.elastic.co/t/extract-field-from-json-to-the-top-level/84667)
**Author:** [@babayotakun](https://discuss.elastic.co/u/babayotakun)\ **Replies:** 2\ **Last updated:** [May 5, 2017, 8:30am UTC](https://discuss.elastic.co/t/extract-field-from-json-to-the-top-level/84667 "2017-05-05T08:30:19Z")
I have a json with the dynamic field, something like that: { field1: "value1", dynamicField: { innerField1: "...", innerField2: "...", ... } } How can I extract all contents from dynamicField to the top level of mes… --- ## [Need 4 different Indexes in ElasticSearch for 4 different logs file format using single Logstash instance](https://discuss.elastic.co/t/need-4-different-indexes-in-elasticsearch-for-4-different-logs-file-format-using-single-logstash-instance/84670)
**Author:** [@hasan\_sharekhan](https://discuss.elastic.co/u/hasan_sharekhan)\ **Replies:** 3\ **Last updated:** [May 5, 2017, 8:26am UTC](https://discuss.elastic.co/t/need-4-different-indexes-in-elasticsearch-for-4-different-logs-file-format-using-single-logstash-instance/84670 "2017-05-05T08:26:19Z")
I have 4 log files from 4 completely different applications For e.g. Log A, B, C, D The log format of A AND B is same and The log format of A, C, and D is different Source of reading log A, B and C is filebeat =\> … --- ## [Logstash is not sending logs in ElasticSearch](https://discuss.elastic.co/t/logstash-is-not-sending-logs-in-elasticsearch/84654)
**Author:** [@honey](https://discuss.elastic.co/u/honey)\ **Replies:** 3\ **Last updated:** [May 5, 2017, 7:21am UTC](https://discuss.elastic.co/t/logstash-is-not-sending-logs-in-elasticsearch/84654 "2017-05-05T07:21:02Z")
Hey, here logstash is creating logs but not displaying anything on Elasticsearch. This is my log\_config file. input { tcp { port =\> 5022 type =\> "syslog" } udp { port =\> 5022 type =\> "syslog" } }… --- ## [Compare multiple types in if in logstash](https://discuss.elastic.co/t/compare-multiple-types-in-if-in-logstash/84668)
**Author:** [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)\ **Replies:** 1\ **Last updated:** [May 5, 2017, 7:17am UTC](https://discuss.elastic.co/t/compare-multiple-types-in-if-in-logstash/84668 "2017-05-05T07:17:22Z")
What is the proper syntax of comparing multiple types in logstash ? like, if\[type\] == grafana OR netdata { # is this correct ? } else if\[type\] == { } else { } --- ## [Logstash is not listening on port 5044](https://discuss.elastic.co/t/logstash-is-not-listening-on-port-5044/84495)
**Author:** [@mahesh\_cs](https://discuss.elastic.co/u/mahesh_cs)\ **Replies:** 12\ **Last updated:** [May 5, 2017, 6:11am UTC](https://discuss.elastic.co/t/logstash-is-not-listening-on-port-5044/84495 "2017-05-05T06:11:31Z")
I am using logstash version 5.3.0 . I have checked using netstat there is no port open of number 5044. I have checked log of logstash, found following error An unexpected error occurred! {:error=\># ArgumentError: Setti… --- ## [Verification failed for logstash-filter-multiline](https://discuss.elastic.co/t/verification-failed-for-logstash-filter-multiline/83991)
**Author:** [@manopmk](https://discuss.elastic.co/u/manopmk)\ **Replies:** 4\ **Last updated:** [May 5, 2017, 6:18am UTC](https://discuss.elastic.co/t/verification-failed-for-logstash-filter-multiline/83991 "2017-05-05T06:18:33Z")
hi Getting this error when i execute the cmd : logstash-plugin install logstash-filter-multiline "Unable to download data from https://rubygems.org/ - SocketError: initialize: name or service not known (ht… --- ## [Possible bug with grok pattern\_definitions](https://discuss.elastic.co/t/possible-bug-with-grok-pattern-definitions/84433)
**Author:** [@jclose](https://discuss.elastic.co/u/jclose)\ **Replies:** 1\ **Last updated:** [May 5, 2017, 5:52am UTC](https://discuss.elastic.co/t/possible-bug-with-grok-pattern-definitions/84433 "2017-05-05T05:52:27Z")
The following configuration is giving me an error. Can someone test if this is giving them an error as well? filter { if \[type\] == "test" { grok { pattern\_definitions =\> { "YMD… --- ## [Fingerprinting stack traces](https://discuss.elastic.co/t/fingerprinting-stack-traces/84193)
**Author:** [@Peter\_Becker](https://discuss.elastic.co/u/Peter_Becker)\ **Replies:** 4\ **Last updated:** [May 5, 2017, 5:46am UTC](https://discuss.elastic.co/t/fingerprinting-stack-traces/84193 "2017-05-05T05:46:32Z")
Hi all, could I get some advice as to how I can efficiently fingerprint stack traces using Logstash? I am aware of the fingerprint filter, but it's not quite enough. I also want to ignore everything in the message th… --- ## [Logstash not sending to elasticsearch](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357)
**Author:** [@Sergio\_Pavez](https://discuss.elastic.co/u/Sergio_Pavez)\ **Replies:** 13\ **Last updated:** [May 5, 2017, 5:23am UTC](https://discuss.elastic.co/t/logstash-not-sending-to-elasticsearch/55357 "2017-05-05T05:23:40Z")
Hello. I have a logstash configuration with a custom template. The template appears when I ask for it: "GET /\_template/\*". I followed this example: Logstash mapping If I send the data to a file with json format this is … --- ## [Pipeline main started and gets stuck and here is my config codr](https://discuss.elastic.co/t/pipeline-main-started-and-gets-stuck-and-here-is-my-config-codr/84637)
**Author:** [@gowtham](https://discuss.elastic.co/u/gowtham)\ **Replies:** 1\ **Last updated:** [May 5, 2017, 5:19am UTC](https://discuss.elastic.co/t/pipeline-main-started-and-gets-stuck-and-here-is-my-config-codr/84637 "2017-05-05T05:19:38Z")
Input{ File{ Path=\>"c:/today. text" } } Output{ Stdout{ Codec=\>rubydebug} File{ Path=\>"c:/today. txt"}} --- ## [Less hits from ElastAlert than the count found in ES](https://discuss.elastic.co/t/less-hits-from-elastalert-than-the-count-found-in-es/84628)
**Author:** [@mr\_chang7](https://discuss.elastic.co/u/mr_chang7)\ **Replies:** 0\ **Last updated:** [May 4, 2017, 8:40pm UTC](https://discuss.elastic.co/t/less-hits-from-elastalert-than-the-count-found-in-es/84628 "2017-05-04T20:40:36Z")
This is more likely real-time monitoring system, sending data by collectd and storing it into ES using LS. Given the same query string and window (in milliseconds) elastalert sometimes report less hits than what I can fi… --- ## [Any advantages using geoip plugin filter with GeoLite2 Dat File?](https://discuss.elastic.co/t/any-advantages-using-geoip-plugin-filter-with-geolite2-dat-file/84594)
**Author:** [@ritchierich](https://discuss.elastic.co/u/ritchierich)\ **Replies:** 0\ **Last updated:** [May 4, 2017, 4:29pm UTC](https://discuss.elastic.co/t/any-advantages-using-geoip-plugin-filter-with-geolite2-dat-file/84594 "2017-05-04T16:29:47Z")
Are there any advantage configuring geoip file with GeoLite Dat File or just using geoip? I haven't seen any write-ups on the use case for when to use GeoLite Dat File or in the geoip documentation I've successful… --- ## [Logstash holding events](https://discuss.elastic.co/t/logstash-holding-events/83539)
**Author:** [@Conall\_Prendergast](https://discuss.elastic.co/u/Conall_Prendergast)\ **Replies:** 4\ **Last updated:** [May 4, 2017, 4:20pm UTC](https://discuss.elastic.co/t/logstash-holding-events/83539 "2017-05-04T16:20:31Z")
Hi, I am trying to make a simple test plugin that will hold every second event and release it with the next event, on the next event. So, given an input sequence of events like so: 1..2..3..4..5..6..7..8 I want an… --- ## [Logstash pipeline not ingesting entire file](https://discuss.elastic.co/t/logstash-pipeline-not-ingesting-entire-file/84035)
**Author:** [@dolanmk](https://discuss.elastic.co/u/dolanmk)\ **Replies:** 3\ **Last updated:** [May 4, 2017, 3:46pm UTC](https://discuss.elastic.co/t/logstash-pipeline-not-ingesting-entire-file/84035 "2017-05-04T15:46:36Z")
Hi, I'm having an issue with my pipeline not reading all of the lines in my file. I am sure the pattern matches all of the lines. I am using a sample log file with 5 lines of standard syntax for my grok pattern. How… --- ## [Logstash - Filter-Date-Match](https://discuss.elastic.co/t/logstash-filter-date-match/84329)
**Author:** [@senthilyes](https://discuss.elastic.co/u/senthilyes)\ **Replies:** 2\ **Last updated:** [May 4, 2017, 3:29pm UTC](https://discuss.elastic.co/t/logstash-filter-date-match/84329 "2017-05-04T15:29:09Z")
Hi Trying to replace the timestamp (interval\_start\_date\_time) retrieved from DB2 to the logstash time stamp. Referred other similar threads but didn't help and finally here. Output without using Date filter: { "@… --- ## [Multiple config files as logstash service](https://discuss.elastic.co/t/multiple-config-files-as-logstash-service/84556)
**Author:** [@Kishore](https://discuss.elastic.co/u/Kishore)\ **Replies:** 7\ **Last updated:** [May 4, 2017, 3:22pm UTC](https://discuss.elastic.co/t/multiple-config-files-as-logstash-service/84556 "2017-05-04T15:22:06Z")
I am using logstash 2.1 version, my /etc/logstash/conf.d/ directory consist of multiple configuration like below. unix-shipper.conf, firewall-shipper.conf, windows-shipper.conf, When i enabled logstash as service, c… --- ## [Unix domain socket as output plugin](https://discuss.elastic.co/t/unix-domain-socket-as-output-plugin/37633)
**Author:** [@mparthas](https://discuss.elastic.co/u/mparthas)\ **Replies:** 2\ **Last updated:** [May 4, 2017, 2:57pm UTC](https://discuss.elastic.co/t/unix-domain-socket-as-output-plugin/37633 "2017-05-04T14:57:38Z")
Hi, Is there any reason why unix domain socket is restricted as input plugin ? I would want to do some more processing and avoiding a disk write. TCP sockets require ports where as unix domain socket just require fils… --- ## [Failed to parse date field yyyy/MM/dd HH:mm:ss \[SOLVED\]](https://discuss.elastic.co/t/failed-to-parse-date-field-yyyy-mm-dd-hh-mm-ss-solved/64836)
**Author:** [@Clement\_Ros](https://discuss.elastic.co/u/Clement_Ros)\ **Replies:** 8\ **Last updated:** [May 4, 2017, 2:06pm UTC](https://discuss.elastic.co/t/failed-to-parse-date-field-yyyy-mm-dd-hh-mm-ss-solved/64836 "2017-05-04T14:06:53Z")
Hi, I'm trying to make a date filter for my servers. And i have an error. Here is the configuration of the shipper : input { file { path =\> \["/var/log/squid/cache.log"\] type =\> … --- ## [Elasticsearch \> logstash \> Elasticsearch failing - illegal latitude value](https://discuss.elastic.co/t/elasticsearch-logstash-elasticsearch-failing-illegal-latitude-value/83432)
**Author:** [@JeremyinNC](https://discuss.elastic.co/u/JeremyinNC)\ **Replies:** 2\ **Last updated:** [May 4, 2017, 1:48pm UTC](https://discuss.elastic.co/t/elasticsearch-logstash-elasticsearch-failing-illegal-latitude-value/83432 "2017-05-04T13:48:26Z")
Dumping from an ES 1.7.6 server to 2.4.4 with the logstash elasticsearch input I see the following message for the majority of my documents: "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"faile… --- ## [How to parse single log file with multiple grok pattern](https://discuss.elastic.co/t/how-to-parse-single-log-file-with-multiple-grok-pattern/83053)
**Author:** [@dbElastic](https://discuss.elastic.co/u/dbElastic)\ **Replies:** 3\ **Last updated:** [May 4, 2017, 11:49am UTC](https://discuss.elastic.co/t/how-to-parse-single-log-file-with-multiple-grok-pattern/83053 "2017-05-04T11:49:38Z")
Hello, I am trying to parse a single log file with multiple grok pattern as below: grok { break\_on\_match =\> false match =\> {"message" =\> \["%{TIMESTAMP\_ISO8601:timestamp\_match}%{SPACE}\[%{SPACE}%{WORD:num… --- ## [Decode error with input plugin pipe](https://discuss.elastic.co/t/decode-error-with-input-plugin-pipe/84525)
**Author:** [@LmYjQ](https://discuss.elastic.co/u/LmYjQ)\ **Replies:** 5\ **Last updated:** [May 4, 2017, 11:48am UTC](https://discuss.elastic.co/t/decode-error-with-input-plugin-pipe/84525 "2017-05-04T11:48:16Z")
I tried the following conf input { file { type =\> "gz\_log" path =\> \["/data1/logs/logs/20170501/\*.log.gz"\] # path =\> \["/var/log/lastlog"\] start\_position =\> "beginning" } pipe { com… --- ## [Plugin version conflict with Logstash 2.4 (logstash-input-kafka and logstash-output-kafka)](https://discuss.elastic.co/t/plugin-version-conflict-with-logstash-2-4-logstash-input-kafka-and-logstash-output-kafka/59562)
**Author:** [@ryac](https://discuss.elastic.co/u/ryac)\ **Replies:** 4\ **Last updated:** [May 4, 2017, 9:38am UTC](https://discuss.elastic.co/t/plugin-version-conflict-with-logstash-2-4-logstash-input-kafka-and-logstash-output-kafka/59562 "2017-05-04T09:38:54Z")
Hi all, I want to use Kafka with Logstash. I've installed Logstash 2.4 and Kafka 0.10.0.1. According to the docs, I need to use version 5.0.0 of the logstash-input-kafka and logstash-output-kafka plugins. When I try … --- ## [Ruby exception occurred: undefined method \`\*' for nil:NilClass](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-for-nil-nilclass/84477)
**Author:** [@jiyu3268891](https://discuss.elastic.co/u/jiyu3268891)\ **Replies:** 2\ **Last updated:** [May 4, 2017, 9:25am UTC](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method-for-nil-nilclass/84477 "2017-05-04T09:25:03Z")
filter { grok { match =\> \["message", "%{TIMESTAMP\_ISO8601:a} %{WORD} %{USER}\] %{WORD} %{WORD} %{WORD:keyvalue}"\] add\_tag =\> "warn" } metrics { meter =\> "events.%{keyvalue}" add\_tag =\> "metric" ignore\_older\_than =\> 10 }… --- ## [Cannot load an invalid configuration in logstash.conf](https://discuss.elastic.co/t/cannot-load-an-invalid-configuration-in-logstash-conf/84491)
**Author:** [@aarishramesh](https://discuss.elastic.co/u/aarishramesh)\ **Replies:** 2\ **Last updated:** [May 4, 2017, 7:14am UTC](https://discuss.elastic.co/t/cannot-load-an-invalid-configuration-in-logstash-conf/84491 "2017-05-04T07:14:39Z")
I am trying to configure logstash for the following inputs - nginx access, nginx error & logs following a custom pattern & below is the logstash.conf input { beats { port =\> 5044 codec =\> multiline { # Grok … --- ## [How can I send my custom Windows AD server event logs to Logstash?](https://discuss.elastic.co/t/how-can-i-send-my-custom-windows-ad-server-event-logs-to-logstash/84485)
**Author:** [@flyfish](https://discuss.elastic.co/u/flyfish)\ **Replies:** 0\ **Last updated:** [May 4, 2017, 5:55am UTC](https://discuss.elastic.co/t/how-can-i-send-my-custom-windows-ad-server-event-logs-to-logstash/84485 "2017-05-04T05:55:08Z")
Hello, I'd like to send my Windows server 2008 AD server logs to ELK via Nxlog, but I only see "Security" log send to KIbana, but I want "Active directory domain services,DHCP server, DNS Server, Application, Security,S… --- ## [Logstash input plugin log4j not working in unix](https://discuss.elastic.co/t/logstash-input-plugin-log4j-not-working-in-unix/84049)
**Author:** [@mdafrozhcl](https://discuss.elastic.co/u/mdafrozhcl)\ **Replies:** 5\ **Last updated:** [May 4, 2017, 5:18am UTC](https://discuss.elastic.co/t/logstash-input-plugin-log4j-not-working-in-unix/84049 "2017-05-04T05:18:42Z")
I am facing Serialization errors with logstash 5.2.1 while trying to use log4j and socket appender. I have a custom log4j layouts/appenders which writes json objects as string while emmiting json string with socket appe… --- ## [Elasticsearch input plugin (from logstash 5.3.2) doesn't work with ES 1.7](https://discuss.elastic.co/t/elasticsearch-input-plugin-from-logstash-5-3-2-doesnt-work-with-es-1-7/84464)
**Author:** [@toffel](https://discuss.elastic.co/u/toffel)\ **Replies:** 0\ **Last updated:** [May 3, 2017, 9:21pm UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-from-logstash-5-3-2-doesnt-work-with-es-1-7/84464 "2017-05-03T21:21:08Z")
According to the release notes of LS 5.3.2 https://www.elastic.co/guide/en/logstash/current/logstash-5-3-2.html#\_input\_plugins\_2 the input plugin is now trying to use scroll in json body of the request, however as I unde… --- ## [Twitter client error - connection refused](https://discuss.elastic.co/t/twitter-client-error-connection-refused/84454)
**Author:** [@Rahul\_Lao](https://discuss.elastic.co/u/Rahul_Lao)\ **Replies:** 2\ **Last updated:** [May 3, 2017, 8:50pm UTC](https://discuss.elastic.co/t/twitter-client-error-connection-refused/84454 "2017-05-03T20:50:03Z")
This is some what same as my logstash.conf is input { twitter { # add your data consumer\_key =\> "x" consumer\_secret =\> "x" oauth\_token =\> "x" oauth\_token\_secret =\> "x" keywords =\> \["GameSpot"\] full\_tweet =\> true # be… --- ## [Getting "Mixing up field types" for keyword](https://discuss.elastic.co/t/getting-mixing-up-field-types-for-keyword/84304)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 1\ **Last updated:** [May 3, 2017, 6:59pm UTC](https://discuss.elastic.co/t/getting-mixing-up-field-types-for-keyword/84304 "2017-05-03T18:59:24Z")
In Logstash I am parsing a field out of a line using %{NOTSPACE:xxhttpversion} . That field is going to be HTTP/1.1 (or maybe HTTP/1.0), so I do not need it analyzed. So in my template (applied after a copy of the defaul… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=988) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=990)