# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=923 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 924 --- ## [Grok keyword to read enter key in log](https://discuss.elastic.co/t/grok-keyword-to-read-enter-key-in-log/97562)
**Author:** [@Vivek\_Sharma](https://discuss.elastic.co/u/Vivek_Sharma)\ **Replies:** 0\ **Last updated:** [August 18, 2017, 10:25am UTC](https://discuss.elastic.co/t/grok-keyword-to-read-enter-key-in-log/97562 "2017-08-18T10:25:03Z")
Hi Just wanted to know if we have any grok keyword that can be used to read enter key from the logs. I have configured filebeat for multiline logs so the logs captured by the logstash have enter key pressed for each ne… --- ## [Beat input plugin and logs which are NOT sent via a beat publisher](https://discuss.elastic.co/t/beat-input-plugin-and-logs-which-are-not-sent-via-a-beat-publisher/97478)
**Author:** [@Geez](https://discuss.elastic.co/u/Geez)\ **Replies:** 2\ **Last updated:** [August 18, 2017, 9:57am UTC](https://discuss.elastic.co/t/beat-input-plugin-and-logs-which-are-not-sent-via-a-beat-publisher/97478 "2017-08-18T09:57:34Z")
Hi all, First post here. I have a question regarding the beat input plugin used in the logstash pipeline. Scenario follows: Most of our (systemd) services are logging to stdout which is then picked up by journald. W… --- ## [Winlogbeat to same index as logstash](https://discuss.elastic.co/t/winlogbeat-to-same-index-as-logstash/96150)
**Author:** [@lschoonaert](https://discuss.elastic.co/u/lschoonaert)\ **Replies:** 8\ **Last updated:** [August 18, 2017, 9:47am UTC](https://discuss.elastic.co/t/winlogbeat-to-same-index-as-logstash/96150 "2017-08-18T09:47:28Z")
Hi there, I’m trying to get all my logs into 1 index - I have DNS logs currently going into logstash (lumberjack), now when I want to sen windows eventlogs, I’d like them to be index in the logstash-\* index. I this poss… --- ## [Error Netflow Plugin on Logstash](https://discuss.elastic.co/t/error-netflow-plugin-on-logstash/97158)
**Author:** [@sreejiths](https://discuss.elastic.co/u/sreejiths)\ **Replies:** 4\ **Last updated:** [August 18, 2017, 9:06am UTC](https://discuss.elastic.co/t/error-netflow-plugin-on-logstash/97158 "2017-08-18T09:06:26Z")
Can help on the error ? Neflow Plugin in Logstash \[root@XXXX logstash\]# bin/logstash-plugin list | grep netflow logstash-codec-netflow \[root@XXXX logstash\]# Netflow Config input { udp { port =\> "2055" host =\> "X.X… --- ## [Field reference in ruby filter](https://discuss.elastic.co/t/field-reference-in-ruby-filter/97527)
**Author:** [@Robert\_Kovacs](https://discuss.elastic.co/u/Robert_Kovacs)\ **Replies:** 1\ **Last updated:** [August 18, 2017, 8:58am UTC](https://discuss.elastic.co/t/field-reference-in-ruby-filter/97527 "2017-08-18T08:58:02Z")
Is it possible to reference a field inside the ruby filters code? I have a field with an ID and I would like to create a variable for each of it’s values if it has not been created yet, or do something with it, if I alr… --- ## [Logstash can't handle multiple pipline](https://discuss.elastic.co/t/logstash-cant-handle-multiple-pipline/97520)
**Author:** [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\ **Replies:** 3\ **Last updated:** [August 18, 2017, 8:57am UTC](https://discuss.elastic.co/t/logstash-cant-handle-multiple-pipline/97520 "2017-08-18T08:57:25Z")
Specs: logstash Version:5.5.1 OS:centos 7.3.1611-x86\_64 logstash pipline conf @/etc/logstash/conf.d/metricbeat.conf #logstash for metricbeat input { beats { port =\> 5044 } } filter { mutate { add\_field… --- ## [Logstash-filter-elasticsearch query](https://discuss.elastic.co/t/logstash-filter-elasticsearch-query/97538)
**Author:** [@user1](https://discuss.elastic.co/u/user1)\ **Replies:** 0\ **Last updated:** [August 18, 2017, 8:42am UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-query/97538 "2017-08-18T08:42:27Z")
Hi, Am having problem in logstash-filter-elaticsearch query : We already have an index stored under elasticsearch. on run time we like to enrich the data under logstash by elasticsearch look up. GET citylist/\_search … --- ## [External list](https://discuss.elastic.co/t/external-list/97429)
**Author:** [@ilya](https://discuss.elastic.co/u/ilya)\ **Replies:** 0\ **Last updated:** [August 17, 2017, 1:43pm UTC](https://discuss.elastic.co/t/external-list/97429 "2017-08-17T13:43:23Z")
Hi, Can we populate network field in CIDR plugin with list of networks stored in external file? filter { cidr { add\_tag =\> \[ "testnet" \] address =\> \[ "%{src\_ip}", "%{dst\_ip}" \] network =\> \[ "/var/netwo… --- ## [Index parent child relation using Logstash (one to many) Configuration issue](https://discuss.elastic.co/t/index-parent-child-relation-using-logstash-one-to-many-configuration-issue/93517)
**Author:** [@malhotras](https://discuss.elastic.co/u/malhotras)\ **Replies:** 13\ **Last updated:** [August 18, 2017, 7:43am UTC](https://discuss.elastic.co/t/index-parent-child-relation-using-logstash-one-to-many-configuration-issue/93517 "2017-08-18T07:43:51Z")
I am using parent-child relation between 2 entities. They are fetched from the database where they come from 2 different tables. I use the SQL join statement for them in logstash config file. Details of the config file: … --- ## [Dropping stacktrace Lines after greedy data](https://discuss.elastic.co/t/dropping-stacktrace-lines-after-greedy-data/97303)
**Author:** [@sentinel](https://discuss.elastic.co/u/sentinel)\ **Replies:** 1\ **Last updated:** [August 18, 2017, 7:22am UTC](https://discuss.elastic.co/t/dropping-stacktrace-lines-after-greedy-data/97303 "2017-08-18T07:22:28Z")
I’ve got these lines that are after my greedy data similar to these: at au.com.xxx.etrading.xxxxx.messaging.tum.AbstractTumProtoSubscriber$TumProtoEventListener.go and logstash attempts to process them thereby generati… --- ## [Translate Filter : Lookup to csv file, with multiple Keys](https://discuss.elastic.co/t/translate-filter-lookup-to-csv-file-with-multiple-keys/97215)
**Author:** [@inhinyera16](https://discuss.elastic.co/u/inhinyera16)\ **Replies:** 2\ **Last updated:** [August 18, 2017, 6:36am UTC](https://discuss.elastic.co/t/translate-filter-lookup-to-csv-file-with-multiple-keys/97215 "2017-08-18T06:36:06Z")
I can use translate filter using single key but how to implement it if i have 2 keys My csv file has 3 columns (key1, key2, value) Is this possible? --- ## [Existing mapping for \[myfield\] must be of type object but found \[text\]](https://discuss.elastic.co/t/existing-mapping-for-myfield-must-be-of-type-object-but-found-text/97250)
**Author:** [@sentinel](https://discuss.elastic.co/u/sentinel)\ **Replies:** 1\ **Last updated:** [August 18, 2017, 5:33am UTC](https://discuss.elastic.co/t/existing-mapping-for-myfield-must-be-of-type-object-but-found-text/97250 "2017-08-18T05:33:54Z")
I am encountering lots of error messages in the elastic logs as I send messages from logstash. What do I have to do to myfield in order to make it an object? “message” =\> “^%{TIMESTAMP\_ISO8601:timestamp},%{NUMBER:tz} \[… --- ## [From several jdbs connection work only one](https://discuss.elastic.co/t/from-several-jdbs-connection-work-only-one/97097)
**Author:** [@Michael\_Limarev](https://discuss.elastic.co/u/Michael_Limarev)\ **Replies:** 1\ **Last updated:** [August 18, 2017, 5:25am UTC](https://discuss.elastic.co/t/from-several-jdbs-connection-work-only-one/97097 "2017-08-18T05:25:00Z")
Hi. I try to make logstash (V 5.5) to read from 2 tables, and it actually does this at first. Logstash reads data from the first table, then from the second one. But then it begins to read only from one of them and I … --- ## [How to install 2 logstash in one machine?](https://discuss.elastic.co/t/how-to-install-2-logstash-in-one-machine/97502)
**Author:** [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\ **Replies:** 3\ **Last updated:** [August 18, 2017, 5:17am UTC](https://discuss.elastic.co/t/how-to-install-2-logstash-in-one-machine/97502 "2017-08-18T05:17:12Z")
How can i run 2 logstash in one machne so that if the output in the first logstash instance in not available, 2nd logstash instance will still continue it’s work. --- ## [How to change a message into multiple key and value in logstash filter](https://discuss.elastic.co/t/how-to-change-a-message-into-multiple-key-and-value-in-logstash-filter/97199)
**Author:** [@Robin\_Guo](https://discuss.elastic.co/u/Robin_Guo)\ **Replies:** 7\ **Last updated:** [August 18, 2017, 4:32am UTC](https://discuss.elastic.co/t/how-to-change-a-message-into-multiple-key-and-value-in-logstash-filter/97199 "2017-08-18T04:32:05Z")
original data : except field in kibana : (I also want to remove message field) agent： metricbeat env: production dc: fr4 pgpool\_activite: 20 pgpool\_idle:8 pgpool\_waiting:100 ... ... .... --- ## [Logstash JVM heap filling up (dynamic field names)](https://discuss.elastic.co/t/logstash-jvm-heap-filling-up-dynamic-field-names/97334)
**Author:** [@Exocomp](https://discuss.elastic.co/u/Exocomp)\ **Replies:** 0\ **Last updated:** [August 17, 2017, 4:49am UTC](https://discuss.elastic.co/t/logstash-jvm-heap-filling-up-dynamic-field-names/97334 "2017-08-17T04:49:23Z")
Here is a summary of what I ran into, hopefully saves someone else some time. Basically, there is a cache mechanism that logstash uses for string interpolation, there is a bug where random keys (or so it seems at the mo… --- ## [Change log level for plugins](https://discuss.elastic.co/t/change-log-level-for-plugins/97486)
**Author:** [@damaren](https://discuss.elastic.co/u/damaren)\ **Replies:** 0\ **Last updated:** [August 17, 2017, 9:03pm UTC](https://discuss.elastic.co/t/change-log-level-for-plugins/97486 "2017-08-17T21:03:03Z")
Is it possible to change the log level in logstash for some specific plugin? I’d like for my “puts” inside the ruby plugin to output in the log level info, but everything else to stay the same. --- ## [Sending cisco wlc syslog to logstash](https://discuss.elastic.co/t/sending-cisco-wlc-syslog-to-logstash/97482)
**Author:** [@Benoit\_Martin](https://discuss.elastic.co/u/Benoit_Martin)\ **Replies:** 0\ **Last updated:** [August 17, 2017, 8:49pm UTC](https://discuss.elastic.co/t/sending-cisco-wlc-syslog-to-logstash/97482 "2017-08-17T20:49:03Z")
Hi, I try to find the best way to handle cisco log with logstash. I find couple example on github but each time I have to add more patterns in my patterns folder. I don’t find those patterns and I don’t know how to crea… --- ## [Cisco\_reason pattern not working](https://discuss.elastic.co/t/cisco-reason-pattern-not-working/95366)
**Author:** [@sreejiths](https://discuss.elastic.co/u/sreejiths)\ **Replies:** 1\ **Last updated:** [August 17, 2017, 8:44pm UTC](https://discuss.elastic.co/t/cisco-reason-pattern-not-working/95366 "2017-08-17T20:44:59Z")
I am quite new to ELk , we are testing ELK V5.2 in out testing environment for Network syslog analytics . We face an issue while creating logstash conf that pattern “CISCO\_RESON” is not been identifed even when its defin… --- ## [Create multiple indexes with same logstash config file](https://discuss.elastic.co/t/create-multiple-indexes-with-same-logstash-config-file/97456)
**Author:** [@elk12](https://discuss.elastic.co/u/elk12)\ **Replies:** 1\ **Last updated:** [August 17, 2017, 7:42pm UTC](https://discuss.elastic.co/t/create-multiple-indexes-with-same-logstash-config-file/97456 "2017-08-17T19:42:41Z")
Hello, I am looking for some help in creating a muliple indexes in ES with just single logstash config file. All dev logs should go to dev index , all sys logs should to sys index and uat logs to uat index. The logstas… --- ## [Elasticsearch index based on source field](https://discuss.elastic.co/t/elasticsearch-index-based-on-source-field/97442)
**Author:** [@Robert\_Kovacs](https://discuss.elastic.co/u/Robert_Kovacs)\ **Replies:** 1\ **Last updated:** [August 17, 2017, 7:41pm UTC](https://discuss.elastic.co/t/elasticsearch-index-based-on-source-field/97442 "2017-08-17T19:41:34Z")
Hi! I would like to know if I could output to a different elasticsearch index based on my logs source field? In the sources I have unique identifiers and I would like to make indexes with these identifiers. --- ## [Drop all grok matches](https://discuss.elastic.co/t/drop-all-grok-matches/97391)
**Author:** [@sentinel](https://discuss.elastic.co/u/sentinel)\ **Replies:** 9\ **Last updated:** [August 17, 2017, 7:38pm UTC](https://discuss.elastic.co/t/drop-all-grok-matches/97391 "2017-08-17T19:38:44Z")
I want to obliterate everything this grok filter matches unconditionally is there a simple way? There are a lot of combinations so it’s better if mated by grok otherwise I will be declaring a lot of conditions. Unless … --- ## [Find and replace string in \[message\] field](https://discuss.elastic.co/t/find-and-replace-string-in-message-field/96319)
**Author:** [@ash007](https://discuss.elastic.co/u/ash007)\ **Replies:** 8\ **Last updated:** [August 17, 2017, 7:34pm UTC](https://discuss.elastic.co/t/find-and-replace-string-in-message-field/96319 "2017-08-17T19:34:11Z")
Hi, I want to replace a string in my message field. How should i proceed in logstash filter. Below is a sample \[message\] log: input to logstash:(before parsing) some text in the beginning Auth: Sec slghltrgj;sjtg;St… --- ## [Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult http://click.pocoo.org/python3/for mitigation steps](https://discuss.elastic.co/t/click-will-abort-further-execution-because-python-3-was-configured-to-use-ascii-as-encoding-for-the-environment-consult-http-click-pocoo-org-python3-for-mitigation-steps/95264)
**Author:** [@jclemons7](https://discuss.elastic.co/u/jclemons7)\ **Replies:** 1\ **Last updated:** [August 17, 2017, 5:05pm UTC](https://discuss.elastic.co/t/click-will-abort-further-execution-because-python-3-was-configured-to-use-ascii-as-encoding-for-the-environment-consult-http-click-pocoo-org-python3-for-mitigation-steps/95264 "2017-08-17T17:05:23Z")
Hello, I’m trying to write a script the invokes Curator to clean up my indexes after 14 days. If I run the following cli script manually: ./opt/elasticsearch-curator/curator\_cli delete\_indices --filter\_list ‘\[{“filtert… --- ## [DNS slower with cache enabled?](https://discuss.elastic.co/t/dns-slower-with-cache-enabled/96744)
**Author:** [@Tom\_Mortimer](https://discuss.elastic.co/u/Tom_Mortimer)\ **Replies:** 2\ **Last updated:** [August 17, 2017, 3:50pm UTC](https://discuss.elastic.co/t/dns-slower-with-cache-enabled/96744 "2017-08-17T15:50:46Z")
Hi, I’m working on a logstash installation which feeds log messages from various sources (syslog, Windows, Checkpoint firewalls etc.) into Elasticsearch for analysis and monitoring. Everything was working nicely until I… --- ## [Timeout executing grok "Value too large to output (347 bytes)!"](https://discuss.elastic.co/t/timeout-executing-grok-value-too-large-to-output-347-bytes/97123)
**Author:** [@nfplatzke](https://discuss.elastic.co/u/nfplatzke)\ **Replies:** 2\ **Last updated:** [August 17, 2017, 3:15pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-value-too-large-to-output-347-bytes/97123 "2017-08-17T15:15:09Z")
OK I have been googling this and I can’t figure it out at all. Every other person who is having this issue is dealing with large documents. The log messages I am indexing are pretty small and I’m still getting this error… --- ## [Two records written by Logstash for each log line](https://discuss.elastic.co/t/two-records-written-by-logstash-for-each-log-line/97413)
**Author:** [@Vivek\_Sharma](https://discuss.elastic.co/u/Vivek_Sharma)\ **Replies:** 1\ **Last updated:** [August 17, 2017, 2:09pm UTC](https://discuss.elastic.co/t/two-records-written-by-logstash-for-each-log-line/97413 "2017-08-17T14:09:01Z")
Hi The ELK setup which I am using is: Filebeat -\> Logstash -\> Elasticsearch -\> Kibana Filebeat configuration file is as below: filebeat.prospectors: -- input\_type: log paths: - C:\\ELK\\log\*.log fields: tech\_sta… --- ## [S3 output plugin doesn't flush the data on shutdown](https://discuss.elastic.co/t/s3-output-plugin-doesnt-flush-the-data-on-shutdown/97152)
**Author:** [@kkirov](https://discuss.elastic.co/u/kkirov)\ **Replies:** 0\ **Last updated:** [August 15, 2017, 10:07pm UTC](https://discuss.elastic.co/t/s3-output-plugin-doesnt-flush-the-data-on-shutdown/97152 "2017-08-15T22:07:43Z")
Version - Logstash 6.0.0-beta1 logstash-output-s3 version 4.09 OS - Windows Server 2012 R2 Standard JAVA\_VERSION="1.8.0\_92" Logstash is running as a service by using NSSM. NSSM calls C:\\logstash-6.0.0-beta1\\bin\\Logst… --- ## [How to re-index a jdbc entity based on timestamp](https://discuss.elastic.co/t/how-to-re-index-a-jdbc-entity-based-on-timestamp/97431)
**Author:** [@Thomas\_Parayil](https://discuss.elastic.co/u/Thomas_Parayil)\ **Replies:** 0\ **Last updated:** [August 17, 2017, 1:47pm UTC](https://discuss.elastic.co/t/how-to-re-index-a-jdbc-entity-based-on-timestamp/97431 "2017-08-17T13:47:23Z")
Hi, I’m trying to do an incremental update of the index after every scheduled index. How to generate a metadata file which will contain the lastexecutiondate (timestamp) and then re-use that value for incremental updat… --- ## [Logstash-\* index is not getting created after installing x-pack5.2.2](https://discuss.elastic.co/t/logstash-index-is-not-getting-created-after-installing-x-pack5-2-2/96957)
**Author:** [@shraddha1](https://discuss.elastic.co/u/shraddha1)\ **Replies:** 0\ **Last updated:** [August 14, 2017, 11:21am UTC](https://discuss.elastic.co/t/logstash-index-is-not-getting-created-after-installing-x-pack5-2-2/96957 "2017-08-14T11:21:38Z")
Hi, I have installed ELK 5.2.2 and x-pack 5.2.2 After installing x-pack, logstash-\* index is not getting created. logstash.conf file is as below input { lumberjack { port =\> 5044 type =\> "logs" ssl\_certificate =\> … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=922) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=924)