# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=665 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 666 --- ## [Logstash configuration with several sources](https://discuss.elastic.co/t/logstash-configuration-with-several-sources/152848)
**Author:** [@Ludovic9](https://discuss.elastic.co/u/Ludovic9)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 1:24pm UTC](https://discuss.elastic.co/t/logstash-configuration-with-several-sources/152848 "2018-10-17T13:24:45Z")
Hello I have some questions about my future configuration of logstash. At this time, I have One input : input { beats { port =\> 5044 } } one grok filter to parse a log of a ftp software … --- ## [File input and avro codec](https://discuss.elastic.co/t/file-input-and-avro-codec/153055)
**Author:** [@gordonwho](https://discuss.elastic.co/u/gordonwho)\ **Replies:** 0\ **Last updated:** [October 18, 2018, 6:14pm UTC](https://discuss.elastic.co/t/file-input-and-avro-codec/153055 "2018-10-18T18:14:01Z")
All, I am having some issue with getting file input and avro codec to work together. The idea is to read an avro file into a json object and then insert it into Elasticsearch. Below is my config file. input { file… --- ## [JMX fields through LogStash](https://discuss.elastic.co/t/jmx-fields-through-logstash/152545)
**Author:** [@Vishnu59](https://discuss.elastic.co/u/Vishnu59)\ **Replies:** 5\ **Last updated:** [October 18, 2018, 6:03pm UTC](https://discuss.elastic.co/t/jmx-fields-through-logstash/152545 "2018-10-18T18:03:52Z")
Hi There, I have setup JMX monitoring for one of our Java applications, using Logstash JMX plugin. Everything works as expected but one thing am hoping for is the way key value pair is being created. Right now after get… --- ## [Updating Elasticsearch index with SQL data](https://discuss.elastic.co/t/updating-elasticsearch-index-with-sql-data/153053)
**Author:** [@donElk](https://discuss.elastic.co/u/donElk)\ **Replies:** 0\ **Last updated:** [October 18, 2018, 6:02pm UTC](https://discuss.elastic.co/t/updating-elasticsearch-index-with-sql-data/153053 "2018-10-18T18:02:39Z")
Hello, I am trying to find the best way to update an Elasticsearch index that contains data previously ingested using Logstash/jdbc. I would like to have Logstash run as a service every 10 minutes to keep the data in th… --- ## [Aggregate Filter Timeout issue](https://discuss.elastic.co/t/aggregate-filter-timeout-issue/152847)
**Author:** [@dan-at-mimas](https://discuss.elastic.co/u/dan-at-mimas)\ **Replies:** 3\ **Last updated:** [October 18, 2018, 3:18pm UTC](https://discuss.elastic.co/t/aggregate-filter-timeout-issue/152847 "2018-10-18T15:18:33Z")
Hello, I'm processing some historical apache logs using logstash and attempting to use the aggregate filter to identify double-clicks. In this case, a double-click is identified as the same request, from the same client… --- ## [Logstash stuck on close\_wait with Lumberjack](https://discuss.elastic.co/t/logstash-stuck-on-close-wait-with-lumberjack/153023)
**Author:** [@Kazesu](https://discuss.elastic.co/u/Kazesu)\ **Replies:** 0\ **Last updated:** [October 18, 2018, 2:21pm UTC](https://discuss.elastic.co/t/logstash-stuck-on-close-wait-with-lumberjack/153023 "2018-10-18T14:21:36Z")
Hello there, I am using the latest releases of the ELK stack, but I have an issues that I had with Logstash version 2.4.2 and that I also have with my current one. This is a standalone solution that receives logs from s… --- ## [Set default value for a field in Logstash if the field is null or if it doesn't exist](https://discuss.elastic.co/t/set-default-value-for-a-field-in-logstash-if-the-field-is-null-or-if-it-doesnt-exist/152770)
**Author:** [@adrianolimit](https://discuss.elastic.co/u/adrianolimit)\ **Replies:** 2\ **Last updated:** [October 18, 2018, 2:12pm UTC](https://discuss.elastic.co/t/set-default-value-for-a-field-in-logstash-if-the-field-is-null-or-if-it-doesnt-exist/152770 "2018-10-18T14:12:47Z")
Hello, I would like to know if it would be possible to set default value for a field in Logstash if the field is null or if it doesn't exist. In my example, I have a "dead\_letter\_queue" input where I can have one, two … --- ## [Zimbra Mail Server logstach filter](https://discuss.elastic.co/t/zimbra-mail-server-logstach-filter/153020)
**Author:** [@VivekanandMall](https://discuss.elastic.co/u/VivekanandMall)\ **Replies:** 0\ **Last updated:** [October 18, 2018, 2:06pm UTC](https://discuss.elastic.co/t/zimbra-mail-server-logstach-filter/153020 "2018-10-18T14:06:55Z")
I have try to configuration logstash for mail server log filter , But unable to get the result, Request you Kindly help me to share the configuration file for same , Or suggest how can we make the filter , if any tool p… --- ## [Best Method to Manipulate Date & TimeStamp](https://discuss.elastic.co/t/best-method-to-manipulate-date-timestamp/152992)
**Author:** [@londonx](https://discuss.elastic.co/u/londonx)\ **Replies:** 2\ **Last updated:** [October 18, 2018, 1:31pm UTC](https://discuss.elastic.co/t/best-method-to-manipulate-date-timestamp/152992 "2018-10-18T13:31:00Z")
Hi all, looking for a way to efficiently convert an ISO8601 date/time stamp into the full ISO8601 format containing the "T" and "Z" The log contains a date timestamp in the following format: 2018-10-16 00:10:01.7764 … --- ## [Logstash 6.4.x not processing lines added when logstash is down](https://discuss.elastic.co/t/logstash-6-4-x-not-processing-lines-added-when-logstash-is-down/153006)
**Author:** [@Martin\_Ferreyra](https://discuss.elastic.co/u/Martin_Ferreyra)\ **Replies:** 0\ **Last updated:** [October 18, 2018, 1:18pm UTC](https://discuss.elastic.co/t/logstash-6-4-x-not-processing-lines-added-when-logstash-is-down/153006 "2018-10-18T13:18:50Z")
Hello everybody..! Do you know if there is any issue with sincedb files and logstash 6.4.x? I had a 5.6.x cluster and when start the logstash service, the parser successfully added the lines who didn't added while logst… --- ## [Pipeline-to-pipeline & "path.config"](https://discuss.elastic.co/t/pipeline-to-pipeline-path-config/151917)
**Author:** [@jetnet](https://discuss.elastic.co/u/jetnet)\ **Replies:** 2\ **Last updated:** [October 18, 2018, 11:57am UTC](https://discuss.elastic.co/t/pipeline-to-pipeline-path-config/151917 "2018-10-18T11:57:25Z")
hello all! just a quick question: it seems that, the inter-pipeline communication does not support the path.config parameter. I could only make it work with config.string. Any plans to support both config parameters? … --- ## [Same request processed by 2 config file](https://discuss.elastic.co/t/same-request-processed-by-2-config-file/152731)
**Author:** [@Emmanuel\_CHANSON](https://discuss.elastic.co/u/Emmanuel_CHANSON)\ **Replies:** 4\ **Last updated:** [October 18, 2018, 11:35am UTC](https://discuss.elastic.co/t/same-request-processed-by-2-config-file/152731 "2018-10-18T11:35:14Z")
I try to understand why every request I sent is process by my 2 logstash config file although I am using 2 differents port in my input http plugin: Config file 1: input { http { host =\> "0.0.0.0" # default: 0.0.0.… --- ## [Split, get the order](https://discuss.elastic.co/t/split-get-the-order/152816)
**Author:** [@TallGuru](https://discuss.elastic.co/u/TallGuru)\ **Replies:** 4\ **Last updated:** [October 18, 2018, 11:15am UTC](https://discuss.elastic.co/t/split-get-the-order/152816 "2018-10-18T11:15:44Z")
Hi I'm successfully splitting a json with array into several events using "split". What I'm looking for is a way to enumerate the events from the order in the array. Simplified example of the problem: Original message… --- ## [Logstash fail to start - Error: Mainclass logstash-core\\lib\\jars\\animal-sniffer-annotations-1.14.jar](https://discuss.elastic.co/t/logstash-fail-to-start-error-mainclass-logstash-core-lib-jars-animal-sniffer-annotations-1-14-jar/149298)
**Author:** [@MrElastic](https://discuss.elastic.co/u/MrElastic)\ **Replies:** 3\ **Last updated:** [October 18, 2018, 11:13am UTC](https://discuss.elastic.co/t/logstash-fail-to-start-error-mainclass-logstash-core-lib-jars-animal-sniffer-annotations-1-14-jar/149298 "2018-10-18T11:13:54Z")
Hi Everybody! I Downloaded Logstash and tried to run a test "logststash.conf" to gather data from my sql server. Oh, I am using Windows 10. When i try to run logstash -f logststash.conf i get the following error: c… --- ## [Data filter not working properly](https://discuss.elastic.co/t/data-filter-not-working-properly/152802)
**Author:** [@idrees](https://discuss.elastic.co/u/idrees)\ **Replies:** 4\ **Last updated:** [October 18, 2018, 10:47am UTC](https://discuss.elastic.co/t/data-filter-not-working-properly/152802 "2018-10-18T10:47:19Z")
Dear All, i have the following date format 6/1/2016 0:00 the date filter of my logstash file is below date { match =\> \[ "TMSTAMP", "MM/dd/yyyy hh:mm" \] target =\> "@timestamp" but when i query it in the dev to… --- ## [Stitching multiple logstash inputs](https://discuss.elastic.co/t/stitching-multiple-logstash-inputs/152880)
**Author:** [@Anirban\_mandal](https://discuss.elastic.co/u/Anirban_mandal)\ **Replies:** 1\ **Last updated:** [October 18, 2018, 10:28am UTC](https://discuss.elastic.co/t/stitching-multiple-logstash-inputs/152880 "2018-10-18T10:28:45Z")
I have a situation where my first http\_poller logstash is getting a session id as output. Now in the second http\_poller call i need to carry that session id in the request header. I am thinking of separating the two cal… --- ## [Logstash rolling log file](https://discuss.elastic.co/t/logstash-rolling-log-file/152957)
**Author:** [@manikandanid](https://discuss.elastic.co/u/manikandanid)\ **Replies:** 0\ **Last updated:** [October 18, 2018, 8:09am UTC](https://discuss.elastic.co/t/logstash-rolling-log-file/152957 "2018-10-18T08:09:33Z")
Hi, I am new to ES and in the process of learning it. Currently, I need to roll out the log file on size based (100MB each), not more than 2 files to be created in a day and need to preserve the log file only for 4 days… --- ## [How to configure Management and Monitoring？](https://discuss.elastic.co/t/how-to-configure-management-and-monitoring/152943)
**Author:** [@YuYu](https://discuss.elastic.co/u/YuYu)\ **Replies:** 4\ **Last updated:** [October 18, 2018, 7:41am UTC](https://discuss.elastic.co/t/how-to-configure-management-and-monitoring/152943 "2018-10-18T07:41:54Z")
Version: logstash 6.2.4 ,elasticsearch 6.2.4, kibana 6.2.4 and all of them with the xpack. I want to configure centralized pipeline Management and Monitoring for logstash node. The following is my configuration. logsta… --- ## [Multiple Logstash Config Files for different log types](https://discuss.elastic.co/t/multiple-logstash-config-files-for-different-log-types/152939)
**Author:** [@Priyaranjan\_Mudliar](https://discuss.elastic.co/u/Priyaranjan_Mudliar)\ **Replies:** 1\ **Last updated:** [October 18, 2018, 7:21am UTC](https://discuss.elastic.co/t/multiple-logstash-config-files-for-different-log-types/152939 "2018-10-18T07:21:41Z")
Can we have multiple logstash config files for different types of logs to handle? If yes, How will logstash read all the config files? I want to create multiple logstash config files because i may end up getting a huge … --- ## [Multiple Grok filter Plugin extracted from a database or a file](https://discuss.elastic.co/t/multiple-grok-filter-plugin-extracted-from-a-database-or-a-file/152803)
**Author:** [@Priyaranjan\_Mudliar](https://discuss.elastic.co/u/Priyaranjan_Mudliar)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 10:23am UTC](https://discuss.elastic.co/t/multiple-grok-filter-plugin-extracted-from-a-database-or-a-file/152803 "2018-10-17T10:23:21Z")
Is it possible to have multiple grok filter in the .conf file from another file (ex: JSON file) or from a database? For example i have a log file to which i want to change the grok plugin not directly from the .conf con… --- ## [Refresh field list added a lot of new unwanted fields](https://discuss.elastic.co/t/refresh-field-list-added-a-lot-of-new-unwanted-fields/152227)
**Author:** [@undelete](https://discuss.elastic.co/u/undelete)\ **Replies:** 1\ **Last updated:** [October 18, 2018, 5:20am UTC](https://discuss.elastic.co/t/refresh-field-list-added-a-lot-of-new-unwanted-fields/152227 "2018-10-18T05:20:39Z")
Hi, I'm still very new at Elasticsearch. I am explicitly using the ArcSight Module with Logstash to populate my database and today I noticied that after update to 6.4.1 a field was missing in the index. I looked in mana… --- ## [Dashboards for windows authentication (probably mainly AD). ssh authentication for linux](https://discuss.elastic.co/t/dashboards-for-windows-authentication-probably-mainly-ad-ssh-authentication-for-linux/152933)
**Author:** [@ELK123](https://discuss.elastic.co/u/ELK123)\ **Replies:** 0\ **Last updated:** [October 18, 2018, 4:15am UTC](https://discuss.elastic.co/t/dashboards-for-windows-authentication-probably-mainly-ad-ssh-authentication-for-linux/152933 "2018-10-18T04:15:20Z")
Can i know how can i approach this ? Any ideas? --- ## [Could not index event to Elasticsearch status 400](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-status-400/150648)
**Author:** [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Replies:** 1\ **Last updated:** [October 2, 2018, 4:30pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-status-400/150648 "2018-10-02T16:30:44Z")
Hi All, I am getting the following error while index a document: \[2018-10-02T08:00:11,897\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_id=\>nil, :… --- ## [Logstsash Multiline Codec with Filebeat is a must](https://discuss.elastic.co/t/logstsash-multiline-codec-with-filebeat-is-a-must/152887)
**Author:** [@fewknow](https://discuss.elastic.co/u/fewknow)\ **Replies:** 6\ **Last updated:** [October 17, 2018, 8:27pm UTC](https://discuss.elastic.co/t/logstsash-multiline-codec-with-filebeat-is-a-must/152887 "2018-10-17T20:27:04Z")
I have to pump may logs to logstash across out stack. Some examples are container logs coming out off in gelf. Binary logs coming from \*.log files. Application logs that don't have the log4j for logstash any longer so … --- ## [How to set locale for add\_field inside a date filter?](https://discuss.elastic.co/t/how-to-set-locale-for-add-field-inside-a-date-filter/152559)
**Author:** [@ffknob](https://discuss.elastic.co/u/ffknob)\ **Replies:** 1\ **Last updated:** [October 17, 2018, 8:20pm UTC](https://discuss.elastic.co/t/how-to-set-locale-for-add-field-inside-a-date-filter/152559 "2018-10-17T20:20:19Z")
Hi, I'm trying to create a new field with the weekday based on the @timestamp field, which I generate with a date {} filter. The date {} seems to be working just fine, putting the correct value in the @timestamp. But th… --- ## [Logstash filter Cellery JSON log with array entries](https://discuss.elastic.co/t/logstash-filter-cellery-json-log-with-array-entries/152719)
**Author:** [@Bogdan\_Stoica](https://discuss.elastic.co/u/Bogdan_Stoica)\ **Replies:** 0\ **Last updated:** [October 16, 2018, 8:46pm UTC](https://discuss.elastic.co/t/logstash-filter-cellery-json-log-with-array-entries/152719 "2018-10-16T20:46:54Z")
I have a cellery app, let's call it generic myapp. The logs are configured to be in JSON format. We receive some entries from some operators we work with and unfortunatelly, those entries, in fact a specific entry is not… --- ## [Never possible to ingest more than 2.5k with cef codec enabled](https://discuss.elastic.co/t/never-possible-to-ingest-more-than-2-5k-with-cef-codec-enabled/152496)
**Author:** [@P1llus](https://discuss.elastic.co/u/P1llus)\ **Replies:** 2\ **Last updated:** [October 17, 2018, 7:33pm UTC](https://discuss.elastic.co/t/never-possible-to-ingest-more-than-2-5k-with-cef-codec-enabled/152496 "2018-10-17T19:33:12Z")
There has been a few debugging processes around ingesting CEF formated logs using logstash. While everything works just fine for us, it is impossible to push logstash to ingest more than 2.5k EPS. The logstash itself is… --- ## [You can delete](https://discuss.elastic.co/t/you-can-delete/152861)
**Author:** [@kprojects](https://discuss.elastic.co/u/kprojects)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 2:23pm UTC](https://discuss.elastic.co/t/you-can-delete/152861 "2018-10-17T14:23:32Z")
This can be deleted - i had an issue but was headed the wrong direction with it :slight\_smile: --- ## [Logstash plugin not recognized](https://discuss.elastic.co/t/logstash-plugin-not-recognized/152521)
**Author:** [@paul1243](https://discuss.elastic.co/u/paul1243)\ **Replies:** 1\ **Last updated:** [October 17, 2018, 7:06pm UTC](https://discuss.elastic.co/t/logstash-plugin-not-recognized/152521 "2018-10-17T19:06:58Z")
Hi, I have installed the beta version of a plugin logstash-input-snmp, when I try to run the config file, I get this below error message: \[2018-10-15T17:18:18,820\]\[ERROR\]\[logstash.plugins.registry\] Problems loading a p… --- ## [RabbitMQ input: Incomplete info message in logstash startup log ("Connected to RabbitMQ at ")](https://discuss.elastic.co/t/rabbitmq-input-incomplete-info-message-in-logstash-startup-log-connected-to-rabbitmq-at/152879)
**Author:** [@bluecoffee](https://discuss.elastic.co/u/bluecoffee)\ **Replies:** 0\ **Last updated:** [October 17, 2018, 4:42pm UTC](https://discuss.elastic.co/t/rabbitmq-input-incomplete-info-message-in-logstash-startup-log-connected-to-rabbitmq-at/152879 "2018-10-17T16:42:52Z")
Hi, when Logstash is starting and connecting to RabbitMQ to fetch messages from a queue, it prints a message into its log saying "Connected to RabbitMQ at ", but then it does not tell to which host it is connected. See… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=664) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=666)