# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=966 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 967 --- ## [Everchanging Pattern - Logstash](https://discuss.elastic.co/t/everchanging-pattern-logstash/89209)
**Author:** [@adic26](https://discuss.elastic.co/u/adic26)\ **Replies:** 2\ **Last updated:** [June 13, 2017, 2:35pm UTC](https://discuss.elastic.co/t/everchanging-pattern-logstash/89209 "2017-06-13T14:35:29Z")
Hello all, I am completely a newbie on this. I have started to use the ELK on my system, and it is great to grab the known patterns. So far I have gotten my system to grab logfiles via Filebeat to parse the logfiles a… --- ## [My gsub doesn't recognize predefined patterns. What am I missing?](https://discuss.elastic.co/t/my-gsub-doesnt-recognize-predefined-patterns-what-am-i-missing/89082)
**Author:** [@brandondash](https://discuss.elastic.co/u/brandondash)\ **Replies:** 2\ **Last updated:** [June 13, 2017, 2:03pm UTC](https://discuss.elastic.co/t/my-gsub-doesnt-recognize-predefined-patterns-what-am-i-missing/89082 "2017-06-13T14:03:07Z")
I'm trying to do a string replace on every MONTH in my message, but when I run it nothing gets replaced: if "foo" in \[tags\] { mutate { gsub =\> \[ … --- ## [Create multiple indices grouped by a field](https://discuss.elastic.co/t/create-multiple-indices-grouped-by-a-field/88462)
**Author:** [@munotshubham](https://discuss.elastic.co/u/munotshubham)\ **Replies:** 2\ **Last updated:** [June 13, 2017, 1:49pm UTC](https://discuss.elastic.co/t/create-multiple-indices-grouped-by-a-field/88462 "2017-06-13T13:49:01Z")
Hi, I have a .csv file with 2 million records and I am using logstash to index the .csv file. I do not want to index all records in one index. So I want them to index them based on say states. So basically I'll have 50… --- ## [Logstash-5.4-Executing multiple conf files throws errors](https://discuss.elastic.co/t/logstash-5-4-executing-multiple-conf-files-throws-errors/89179)
**Author:** [@riddhijit\_roy](https://discuss.elastic.co/u/riddhijit_roy)\ **Replies:** 1\ **Last updated:** [June 13, 2017, 1:09pm UTC](https://discuss.elastic.co/t/logstash-5-4-executing-multiple-conf-files-throws-errors/89179 "2017-06-13T13:09:26Z")
In the previous versions of Logstash one could run multiple conf files and it would run without any errors. For instance , "abc-test.conf" and "abc-dev.conf" would run simultaneously . However in logstash-5.4 running t… --- ## [Logstash multiline codec functional question](https://discuss.elastic.co/t/logstash-multiline-codec-functional-question/88845)
**Author:** [@bterhorst](https://discuss.elastic.co/u/bterhorst)\ **Replies:** 7\ **Last updated:** [June 13, 2017, 11:35am UTC](https://discuss.elastic.co/t/logstash-multiline-codec-functional-question/88845 "2017-06-13T11:35:20Z")
Hello, I am kind of new to logstash and was wondering about the functionality of the multiline codec. If I have a multiline codec that has the following definition: input { beats { port =\> "5045" cod… --- ## [Exchange Message Tracking](https://discuss.elastic.co/t/exchange-message-tracking/89164)
**Author:** [@teejayuu](https://discuss.elastic.co/u/teejayuu)\ **Replies:** 3\ **Last updated:** [June 13, 2017, 11:04am UTC](https://discuss.elastic.co/t/exchange-message-tracking/89164 "2017-06-13T11:04:02Z")
Hi, I've been following this tutorial to get Exchange Message Tracking logs into an ELK stack on Windows (https://elijahpaul.co.uk/analysing-exchange-2013-message-tracking-logs-using-elk-elasticsearch-logstash-kibana/). … --- ## [ELK 5.4 - LS reverts to automatic mapping when 'type' field is removed or renamed - ultimate :-)](https://discuss.elastic.co/t/elk-5-4-ls-reverts-to-automatic-mapping-when-type-field-is-removed-or-renamed-ultimate/88534)
**Author:** [@smyttie](https://discuss.elastic.co/u/smyttie)\ **Replies:** 16\ **Last updated:** [June 13, 2017, 8:26am UTC](https://discuss.elastic.co/t/elk-5-4-ls-reverts-to-automatic-mapping-when-type-field-is-removed-or-renamed-ultimate/88534 "2017-06-13T08:26:59Z")
Hi, I'm having an issue with removing the field named "type". I have created an index template for my data and declare a type "MyType". Ingesting data with logstash works very well, but I get an extra field named "… --- ## [Multiple instances on one config file](https://discuss.elastic.co/t/multiple-instances-on-one-config-file/85245)
**Author:** [@bertvervaele](https://discuss.elastic.co/u/bertvervaele)\ **Replies:** 10\ **Last updated:** [June 13, 2017, 7:56am UTC](https://discuss.elastic.co/t/multiple-instances-on-one-config-file/85245 "2017-06-13T07:56:36Z")
I was using logstash-5.2.2 but did an upgrade today to logstash-5.4.0 . In 5.2.2 I could just do bin/logstash -f oracle.conf and bin/logstash -f linux.conf without a problem. But now on 5.4.0 when I try the second… --- ## [Filebeat + Logstash Persisted Queues + ElasticSearch](https://discuss.elastic.co/t/filebeat-logstash-persisted-queues-elasticsearch/89102)
**Author:** [@ryveradam](https://discuss.elastic.co/u/ryveradam)\ **Replies:** 1\ **Last updated:** [June 13, 2017, 7:53am UTC](https://discuss.elastic.co/t/filebeat-logstash-persisted-queues-elasticsearch/89102 "2017-06-13T07:53:59Z")
Greetings fellow Logstashers. I am fairly new to Logstash. I am trying to implement Persisted Queues but I am having trouble understanding some stuff. If my understand is accurate, I would have Logstash running on eac… --- ## [Logstash jms input plugin: Has anybody been able to configure the plugin to use a IBM MQ Topic?](https://discuss.elastic.co/t/logstash-jms-input-plugin-has-anybody-been-able-to-configure-the-plugin-to-use-a-ibm-mq-topic/89151)
**Author:** [@raisinpit](https://discuss.elastic.co/u/raisinpit)\ **Replies:** 0\ **Last updated:** [June 13, 2017, 7:34am UTC](https://discuss.elastic.co/t/logstash-jms-input-plugin-has-anybody-been-able-to-configure-the-plugin-to-use-a-ibm-mq-topic/89151 "2017-06-13T07:34:55Z")
I've been trying to get the plugin to work with a topic, however I keep getting a wrong number of arguments (2 for 1) exception and the JMS Consumer dies . I have gotten the plugin to work with queue's before and using t… --- ## [Best practices to separate two netwflow inputs](https://discuss.elastic.co/t/best-practices-to-separate-two-netwflow-inputs/89142)
**Author:** [@VamPikmin](https://discuss.elastic.co/u/VamPikmin)\ **Replies:** 0\ **Last updated:** [June 13, 2017, 6:22am UTC](https://discuss.elastic.co/t/best-practices-to-separate-two-netwflow-inputs/89142 "2017-06-13T06:22:06Z")
Hey all, I have a logstash input ingesting router flows and I'd like to have another to get Cisco ASA netflows. What's the best way to achieve this? Should I create separate indexes logstash-firewall and logstash-route… --- ## [Receiving from different inputs on same logstash instance](https://discuss.elastic.co/t/receiving-from-different-inputs-on-same-logstash-instance/89121)
**Author:** [@jacoblemperle](https://discuss.elastic.co/u/jacoblemperle)\ **Replies:** 1\ **Last updated:** [June 13, 2017, 6:07am UTC](https://discuss.elastic.co/t/receiving-from-different-inputs-on-same-logstash-instance/89121 "2017-06-13T06:07:22Z")
I am looking to get an ID from RabbitMQ and use those IDs to query an Elasticsearch Input. Is it possible to do this all in one single logstash config? --- ## [Logstash to EalsticSearch config error](https://discuss.elastic.co/t/logstash-to-ealsticsearch-config-error/89135)
**Author:** [@mvelez83](https://discuss.elastic.co/u/mvelez83)\ **Replies:** 2\ **Last updated:** [June 13, 2017, 5:49am UTC](https://discuss.elastic.co/t/logstash-to-ealsticsearch-config-error/89135 "2017-06-13T05:49:01Z")
Hello, I'm very new to ELK stack and I'm trying to connect Logstash to Elastic Search and I keep getting the following error: \[2017-06-13T03:17:53,910\]\[ERROR\]\[logstash.agent \] Cannot create pipeline {:reason… --- ## [Loading dumped non-parsed ES data to Logstash](https://discuss.elastic.co/t/loading-dumped-non-parsed-es-data-to-logstash/88950)
**Author:** [@bandanna](https://discuss.elastic.co/u/bandanna)\ **Replies:** 1\ **Last updated:** [June 13, 2017, 5:32am UTC](https://discuss.elastic.co/t/loading-dumped-non-parsed-es-data-to-logstash/88950 "2017-06-13T05:32:31Z")
I have data in Elasticsearch that was sent by Filebeat to Elasticsearch without being parsed in Logstash, also I have a dump of this data using Elasticdump (indexes and mappings as jsons). I need to parse these logs in… --- ## [Cassandra Plugin for Logstash](https://discuss.elastic.co/t/cassandra-plugin-for-logstash/88929)
**Author:** [@dona](https://discuss.elastic.co/u/dona)\ **Replies:** 1\ **Last updated:** [June 13, 2017, 5:32am UTC](https://discuss.elastic.co/t/cassandra-plugin-for-logstash/88929 "2017-06-13T05:32:05Z")
I would like to use Cassandra data as input for Logstash -\> Elasticsearch, but officially there is no input plugin. Is there some other method or does anyone here have a source for how to configure a cassandra plugin for… --- ## [Logstash date filtering and event @timestamp](https://discuss.elastic.co/t/logstash-date-filtering-and-event-timestamp/88513)
**Author:** [@mulgar](https://discuss.elastic.co/u/mulgar)\ **Replies:** 3\ **Last updated:** [June 13, 2017, 3:08am UTC](https://discuss.elastic.co/t/logstash-date-filtering-and-event-timestamp/88513 "2017-06-13T03:08:49Z")
I'm pulling data in from an ERD database, in short ticket records which can be updated so I'm using sql\_last\_date on an "updated\_date" field to check for new and updated data, and assigning a PK identifier to the ID to u… --- ## [Delete default tag](https://discuss.elastic.co/t/delete-default-tag/44988)
**Author:** [@petar.kozic](https://discuss.elastic.co/u/petar.kozic)\ **Replies:** 4\ **Last updated:** [June 13, 2017, 2:27am UTC](https://discuss.elastic.co/t/delete-default-tag/44988 "2017-06-13T02:27:21Z")
I was searching for this porblem but I don\`t find resolved. On my every log in Logstash I have tag "beats\_input\_codec\_plain\_applied". I\`m add some my tags and this tag is very confusing. Tag is my very important field … --- ## [Passing environment variable from init script to log4j2](https://discuss.elastic.co/t/passing-environment-variable-from-init-script-to-log4j2/89126)
**Author:** [@stecino](https://discuss.elastic.co/u/stecino)\ **Replies:** 0\ **Last updated:** [June 13, 2017, 12:57am UTC](https://discuss.elastic.co/t/passing-environment-variable-from-init-script-to-log4j2/89126 "2017-06-13T00:57:05Z")
Hello, I am running logstash-5.4.1. I am running multiple instances using --path.data option But logs from these instances go to logstash-plain.log At the moment I have export LS\_OPTS="-DInstance\_Name=${2}"$LS\_OPTS … --- ## [Dateparse failure errors when using filters depending on file content](https://discuss.elastic.co/t/dateparse-failure-errors-when-using-filters-depending-on-file-content/88960)
**Author:** [@abd.wsu](https://discuss.elastic.co/u/abd.wsu)\ **Replies:** 10\ **Last updated:** [June 12, 2017, 8:18pm UTC](https://discuss.elastic.co/t/dateparse-failure-errors-when-using-filters-depending-on-file-content/88960 "2017-06-12T20:18:50Z")
Hello, I have a several hundred files which are basically named in three different ways. \`Eg: CUSTOMEREG\_threadStats.csv, CUSTOMEREG\_nodeStats.csv, CUSTOMEREG\_flowStats.csv lines in CUSTOMEREG\_threadStats.csv "… --- ## [HTTP Input - Recording Full JSON Message](https://discuss.elastic.co/t/http-input-recording-full-json-message/88483)
**Author:** [@seth.yes](https://discuss.elastic.co/u/seth.yes)\ **Replies:** 3\ **Last updated:** [June 12, 2017, 8:12pm UTC](https://discuss.elastic.co/t/http-input-recording-full-json-message/88483 "2017-06-12T20:12:10Z")
I've got an HTTP input on Logstash v5.4.0. I'm trying to record the full JSON message that is posted along with all the fields. I've tried something similar to the following input with no luck. Any ideas on what I'm d… --- ## [Multiple Indexes in logstash from multiple filebeats -hostname](https://discuss.elastic.co/t/multiple-indexes-in-logstash-from-multiple-filebeats-hostname/88976)
**Author:** [@Vijayant\_Panda](https://discuss.elastic.co/u/Vijayant_Panda)\ **Replies:** 1\ **Last updated:** [June 12, 2017, 8:06pm UTC](https://discuss.elastic.co/t/multiple-indexes-in-logstash-from-multiple-filebeats-hostname/88976 "2017-06-12T20:06:22Z")
Hi I am a complete newbie to ELK. I have downloaded set up ELK in local.Now i am able to search one log file(from filebeats) in Kibana UI with logstash-\* index. In my json logs I have a filed called hostname:xxxx . i ne… --- ## [How to keep track on elasticsearch events](https://discuss.elastic.co/t/how-to-keep-track-on-elasticsearch-events/88856)
**Author:** [@kabali12345](https://discuss.elastic.co/u/kabali12345)\ **Replies:** 1\ **Last updated:** [June 12, 2017, 8:02pm UTC](https://discuss.elastic.co/t/how-to-keep-track-on-elasticsearch-events/88856 "2017-06-12T20:02:25Z")
Hi.... here i am trying to push ES data into rabbitMq but problem is every time i am reading total data which is present in that index how can i keep option like since\_db so that i can read each event once here is my … --- ## [Logstash regexp if not working](https://discuss.elastic.co/t/logstash-regexp-if-not-working/88488)
**Author:** [@rpuserh](https://discuss.elastic.co/u/rpuserh)\ **Replies:** 2\ **Last updated:** [June 12, 2017, 6:08pm UTC](https://discuss.elastic.co/t/logstash-regexp-if-not-working/88488 "2017-06-12T18:08:26Z")
hi, I have JSON which has key body.message and content of this key can be JSON\[OBJECT\] or text (I cant insert like this to ES as it giving mapping error) I want to move it to another field if it look like text and exists… --- ## [Input/filter/output metrics still exposed after setting enable\_metric: false](https://discuss.elastic.co/t/input-filter-output-metrics-still-exposed-after-setting-enable-metric-false/89088)
**Author:** [@shog](https://discuss.elastic.co/u/shog)\ **Replies:** 0\ **Last updated:** [June 12, 2017, 6:00pm UTC](https://discuss.elastic.co/t/input-filter-output-metrics-still-exposed-after-setting-enable-metric-false/89088 "2017-06-12T18:00:35Z")
Hi, It looks like enable\_metric setting is being ignored and metrics are always enabled for all input/filter/output plugins. How to repro: Config (any tested input/filter/outplut plugin behaves the same way): i… --- ## [Setting a field on Input and Referencing on output](https://discuss.elastic.co/t/setting-a-field-on-input-and-referencing-on-output/88482)
**Author:** [@seth.yes](https://discuss.elastic.co/u/seth.yes)\ **Replies:** 3\ **Last updated:** [June 12, 2017, 5:38pm UTC](https://discuss.elastic.co/t/setting-a-field-on-input-and-referencing-on-output/88482 "2017-06-12T17:38:40Z")
I'm using the HTTP Logstash (5.4.0) input plugin to receive json input data. I'm having users send data to a different port depending on whether it's test, staging or production data. I then need to reference that field … --- ## [Changing @timestamp makes data unqueryable on grafana](https://discuss.elastic.co/t/changing-timestamp-makes-data-unqueryable-on-grafana/89074)
**Author:** [@gfitzy123](https://discuss.elastic.co/u/gfitzy123)\ **Replies:** 1\ **Last updated:** [June 12, 2017, 4:02pm UTC](https://discuss.elastic.co/t/changing-timestamp-makes-data-unqueryable-on-grafana/89074 "2017-06-12T16:02:03Z")
Hi there. Heres the conf file. input { file { path =\> \[ "/Users/garrett/Desktop/failures-march-legacy.csv", "/Users/garrett/Desktop/failures-march-cms.csv" \] start\_position =\> "… --- ## [Logstash configuration file unit testing](https://discuss.elastic.co/t/logstash-configuration-file-unit-testing/88450)
**Author:** [@nileshgattani](https://discuss.elastic.co/u/nileshgattani)\ **Replies:** 5\ **Last updated:** [June 12, 2017, 3:52pm UTC](https://discuss.elastic.co/t/logstash-configuration-file-unit-testing/88450 "2017-06-12T15:52:07Z")
Hi there, Are there any general guidelines on how to go about unit testing logstash config files? I did some research and picked on a few examples, but they all are very old and don't seem to be compatible with logsta… --- ## [Logstash plugin elasticsearch performance issue](https://discuss.elastic.co/t/logstash-plugin-elasticsearch-performance-issue/87775)
**Author:** [@sepibe](https://discuss.elastic.co/u/sepibe)\ **Replies:** 7\ **Last updated:** [June 12, 2017, 2:23pm UTC](https://discuss.elastic.co/t/logstash-plugin-elasticsearch-performance-issue/87775 "2017-06-12T14:23:45Z")
Hello, I'm using a Filebeat-\>Logstash-\>Elasticsearch configuration to monitor and keep trace of different logs in my system. Everythin works correctly and CPU usage is around 20%. There are 4 different types of log f… --- ## [Logstash 2.4.0 running on cmd but not as windows service](https://discuss.elastic.co/t/logstash-2-4-0-running-on-cmd-but-not-as-windows-service/89056)
**Author:** [@aviral\_srivastava](https://discuss.elastic.co/u/aviral_srivastava)\ **Replies:** 0\ **Last updated:** [June 12, 2017, 2:04pm UTC](https://discuss.elastic.co/t/logstash-2-4-0-running-on-cmd-but-not-as-windows-service/89056 "2017-06-12T14:04:33Z")
OS: windows server 2012 R2 Standard Logstash 2.4.0 elasticsearch 2.1 I have sample log file Timestamp Process TID Area Category … --- ## [Stdin LogStash](https://discuss.elastic.co/t/stdin-logstash/89040)
**Author:** [@baha1](https://discuss.elastic.co/u/baha1)\ **Replies:** 5\ **Last updated:** [June 12, 2017, 1:46pm UTC](https://discuss.elastic.co/t/stdin-logstash/89040 "2017-06-12T13:46:12Z")
Hi there, i am working to parse a log file that contains message with one line and multiline like below : Exception in thread "main" java.lang.NullPointerException at com.example.myproject.Book.getTitle(Book.ja… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=965) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=967)