# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1114 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1115 --- ## [Logstash 2.3.4 not starting up due to uninitialized constant Gem](https://discuss.elastic.co/t/logstash-2-3-4-not-starting-up-due-to-uninitialized-constant-gem/58191)
**Author:** [@biswajit86](https://discuss.elastic.co/u/biswajit86)\ **Replies:** 1\ **Last updated:** [August 18, 2016, 6:50am UTC](https://discuss.elastic.co/t/logstash-2-3-4-not-starting-up-due-to-uninitialized-constant-gem/58191 "2016-08-18T06:50:39Z")
Hi, I was trying to upgrade to logstassh 2.3.4 but after installing the tar.gz file from the website, when I try to start logstash , I am geting a uninitialized constant Gem error. Can someone please help? Command bi… --- ## [Add new field based on request field](https://discuss.elastic.co/t/add-new-field-based-on-request-field/58243)
**Author:** [@Viswanath\_S](https://discuss.elastic.co/u/Viswanath_S)\ **Replies:** 2\ **Last updated:** [August 18, 2016, 6:32am UTC](https://discuss.elastic.co/t/add-new-field-based-on-request-field/58243 "2016-08-18T06:32:24Z")
Hi, I am trying to add a new field based on request field value. if the request field contains "fetchAccountDetails" add a new field called "requestCatagory" with value as "FETCH\_ACCOUNT\_DATA" if the request field con… --- ## [Messages with html code from rsyslog corrupted](https://discuss.elastic.co/t/messages-with-html-code-from-rsyslog-corrupted/58308)
**Author:** [@Binbin\_He](https://discuss.elastic.co/u/Binbin_He)\ **Replies:** 0\ **Last updated:** [August 18, 2016, 4:42am UTC](https://discuss.elastic.co/t/messages-with-html-code-from-rsyslog-corrupted/58308 "2016-08-18T04:42:43Z")
I use rsyslog to collect all server logs to logstash. But these days I found some log with html code will be corrupted by logstash. Here are the details below. Logstash version: 2.3.4 Rsyslog version: 8.20.2 Message… --- ## [Logstash 2.3.4 memory leak](https://discuss.elastic.co/t/logstash-2-3-4-memory-leak/57969)
**Author:** [@strike00](https://discuss.elastic.co/u/strike00)\ **Replies:** 15\ **Last updated:** [August 18, 2016, 1:37am UTC](https://discuss.elastic.co/t/logstash-2-3-4-memory-leak/57969 "2016-08-18T01:37:11Z")
Hi! I've been working on setting up ELK in AWS and been having this issue with Constantly increasing consumption of memory. I started out with 4GB then up to 10GB! But yes, it still consumed causing the logstash server… --- ## [Logstash for SOAP Response](https://discuss.elastic.co/t/logstash-for-soap-response/58303)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 0\ **Last updated:** [August 18, 2016, 12:54am UTC](https://discuss.elastic.co/t/logstash-for-soap-response/58303 "2016-08-18T00:54:43Z")
Hi All Any logstash input plugin is available to store the SOAP Request response to Elasticsearch. I have a WSDL file , i want to stream the response to elasticsearch Regards Ritesh --- ## [Logstash shipping results](https://discuss.elastic.co/t/logstash-shipping-results/58168)
**Author:** [@joshuaclark](https://discuss.elastic.co/u/joshuaclark)\ **Replies:** 11\ **Last updated:** [August 17, 2016, 7:49pm UTC](https://discuss.elastic.co/t/logstash-shipping-results/58168 "2016-08-17T19:49:09Z")
Hi, I'm trying to ship LocustIO results with logstash which look like the following: 10:09:31 Name # reqs # fails Avg Min Max | Median req/… --- ## [Can we use values from sql statement to be inserted in an array in a document](https://discuss.elastic.co/t/can-we-use-values-from-sql-statement-to-be-inserted-in-an-array-in-a-document/58289)
**Author:** [@vtangutoori](https://discuss.elastic.co/u/vtangutoori)\ **Replies:** 0\ **Last updated:** [August 17, 2016, 7:45pm UTC](https://discuss.elastic.co/t/can-we-use-values-from-sql-statement-to-be-inserted-in-an-array-in-a-document/58289 "2016-08-17T19:45:59Z")
Hi, I am trying to import data from SQL Server and i need to create one field in the document as an array and add all the values related to the document from child table. please see the example below. Table 1: Manufact… --- ## [How can i give twitter output to mysql database](https://discuss.elastic.co/t/how-can-i-give-twitter-output-to-mysql-database/58250)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 1\ **Last updated:** [August 17, 2016, 7:32pm UTC](https://discuss.elastic.co/t/how-can-i-give-twitter-output-to-mysql-database/58250 "2016-08-17T19:32:49Z")
i am using twitter input plugin for logstash to get data from twitter. I want to load this data into mysql DB. How can i load that.Is their any way to load data directly into mysql db. Or first load data into elast… --- ## [Logs being sent to LS Server. Nothing showing up in Kibana Dashboard](https://discuss.elastic.co/t/logs-being-sent-to-ls-server-nothing-showing-up-in-kibana-dashboard/57148)
**Author:** [@Sully1](https://discuss.elastic.co/u/Sully1)\ **Replies:** 4\ **Last updated:** [August 17, 2016, 7:29pm UTC](https://discuss.elastic.co/t/logs-being-sent-to-ls-server-nothing-showing-up-in-kibana-dashboard/57148 "2016-08-17T19:29:31Z")
I'm having trouble viewing the logs that are being sent to my logstash server. I'm using filebeat to ship HTTP Logs and IIS logs. Additionally, i'm using winlogbeat to ship my event logs. I receive the following message(… --- ## [Unable to install logstash-output-zabbix](https://discuss.elastic.co/t/unable-to-install-logstash-output-zabbix/58195)
**Author:** [@vardhasu](https://discuss.elastic.co/u/vardhasu)\ **Replies:** 3\ **Last updated:** [August 17, 2016, 2:38pm UTC](https://discuss.elastic.co/t/unable-to-install-logstash-output-zabbix/58195 "2016-08-17T14:38:06Z")
Hi All Downloaded logstash-output-zabix-master as a zip file. Used unpack to unpack this. When trying to install I keep getting the below error bin/logstash-plugin install --local logstash-output-zabbix-master Inst… --- ## [Multiline issue with 2 different patterns for a single event](https://discuss.elastic.co/t/multiline-issue-with-2-different-patterns-for-a-single-event/57896)
**Author:** [@jinal.shah](https://discuss.elastic.co/u/jinal.shah)\ **Replies:** 12\ **Last updated:** [August 17, 2016, 1:13pm UTC](https://discuss.elastic.co/t/multiline-issue-with-2-different-patterns-for-a-single-event/57896 "2016-08-17T13:13:16Z")
Hello I am trying to parse the log below through Logstash, however I can't seem to get the pattern to work. Essentially, I need to have the line straight after the dated line begining with {0=Failed to be part of the s… --- ## [\[ SOLVED\] Remove IIS log](https://discuss.elastic.co/t/solved-remove-iis-log/58240)
**Author:** [@Alexis.d](https://discuss.elastic.co/u/Alexis.d)\ **Replies:** 5\ **Last updated:** [August 17, 2016, 12:46pm UTC](https://discuss.elastic.co/t/solved-remove-iis-log/58240 "2016-08-17T12:46:27Z")
Hey there, I need some help :). I work with ELK and Nxlog for my IIS log and it's work fine :slight\_smile: but i want to log only errors on my web sites and remove all log with sc-status 200 sc-win32-status 0 . He… --- ## [SinceDB rollover](https://discuss.elastic.co/t/sincedb-rollover/58251)
**Author:** [@chhavi\_bahekar](https://discuss.elastic.co/u/chhavi_bahekar)\ **Replies:** 0\ **Last updated:** [August 17, 2016, 12:04pm UTC](https://discuss.elastic.co/t/sincedb-rollover/58251 "2016-08-17T12:04:59Z")
Hi, Can any once explain,when sincedb file will rollover or will delete old entry from since db file ?. Because after some day when since db file is growing to MB,Logstash is not reading file properly. Logstash is rea… --- ## [Logstash crashes when folder/file it is watching gets deleted and rcreated](https://discuss.elastic.co/t/logstash-crashes-when-folder-file-it-is-watching-gets-deleted-and-rcreated/58215)
**Author:** [@sameer\_srivastava](https://discuss.elastic.co/u/sameer_srivastava)\ **Replies:** 3\ **Last updated:** [August 17, 2016, 11:00am UTC](https://discuss.elastic.co/t/logstash-crashes-when-folder-file-it-is-watching-gets-deleted-and-rcreated/58215 "2016-08-17T11:00:19Z")
We have been using logstash 2.3.2 on windows 7 machines to index logs into elastic search. Logstash is configured to watch a folder which has only one file at the time. Using logstash's file input plugin for this purpose… --- ## [Multiple regex matches in a single line](https://discuss.elastic.co/t/multiple-regex-matches-in-a-single-line/58126)
**Author:** [@Ayush\_Rastogi](https://discuss.elastic.co/u/Ayush_Rastogi)\ **Replies:** 4\ **Last updated:** [August 17, 2016, 9:44am UTC](https://discuss.elastic.co/t/multiple-regex-matches-in-a-single-line/58126 "2016-08-17T09:44:09Z")
Hi All, I want to do multiple regex searches in a single line via logstash. My log file DEBUG - PerfLogging input\_count:1 ,className:value,date:16-Aug-2016 07:13:40 ,TimeTaken:0.428 ,output\_count:1 I want to get the … --- ## [ConcurrencyError: interrupted waiting for mutex: null in Logstash 2.3.1](https://discuss.elastic.co/t/concurrencyerror-interrupted-waiting-for-mutex-null-in-logstash-2-3-1/47617)
**Author:** [@Fedele\_Mantuano](https://discuss.elastic.co/u/Fedele_Mantuano)\ **Replies:** 15\ **Last updated:** [August 17, 2016, 9:04am UTC](https://discuss.elastic.co/t/concurrencyerror-interrupted-waiting-for-mutex-null-in-logstash-2-3-1/47617 "2016-08-17T09:04:54Z")
Sometimes I have the following issue: {:timestamp=\>"2016-04-16T04:35:38.758000+0000", :message=\>"Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and resta… --- ## [Can't do division between two floats](https://discuss.elastic.co/t/cant-do-division-between-two-floats/58224)
**Author:** [@Fred\_Dionisio](https://discuss.elastic.co/u/Fred_Dionisio)\ **Replies:** 6\ **Last updated:** [August 17, 2016, 8:59am UTC](https://discuss.elastic.co/t/cant-do-division-between-two-floats/58224 "2016-08-17T08:59:37Z")
Hello. This is the code I have. mutate {convert =\> \["Average Time of Execution", "float"\]} mutate {convert =\> \["Average Time of Execution - Last Hour", "float"\]} ruby { code =\> "event\['last Hour Perc'\] = event\['A… --- ## [Logstash does not output to elasticsearch, only "Flushing buffer at interval" in log](https://discuss.elastic.co/t/logstash-does-not-output-to-elasticsearch-only-flushing-buffer-at-interval-in-log/41345)
**Author:** [@s0mb00n](https://discuss.elastic.co/u/s0mb00n)\ **Replies:** 6\ **Last updated:** [August 17, 2016, 8:49am UTC](https://discuss.elastic.co/t/logstash-does-not-output-to-elasticsearch-only-flushing-buffer-at-interval-in-log/41345 "2016-08-17T08:49:10Z")
My logstash setup stopped outputting to Elasticsearch earlier today. When I run with --debug option, I don't see any error but only mostly the lines like the following: {:timestamp=\>"2016-02-10T03:29:43.431000+0000", … --- ## [Parsing with hyphens](https://discuss.elastic.co/t/parsing-with-hyphens/57421)
**Author:** [@bastianhoss](https://discuss.elastic.co/u/bastianhoss)\ **Replies:** 10\ **Last updated:** [August 17, 2016, 7:29am UTC](https://discuss.elastic.co/t/parsing-with-hyphens/57421 "2016-08-17T07:29:26Z")
Hey there, I'm looking for an opportunity to parse this expression: TEST\_test\_123-NONE-testTEST123-Test\_123 into TEST\_test\_123, NONE, testTEST123, Test\_123 I've tried to use custom patterns like : \[a-zA-Z0-9\_\] , \[/… --- ## [Single syslog input to multiple indices output](https://discuss.elastic.co/t/single-syslog-input-to-multiple-indices-output/57433)
**Author:** [@bastianhoss](https://discuss.elastic.co/u/bastianhoss)\ **Replies:** 2\ **Last updated:** [August 17, 2016, 7:27am UTC](https://discuss.elastic.co/t/single-syslog-input-to-multiple-indices-output/57433 "2016-08-17T07:27:53Z")
Hi Community, searching a way for multiple indice output. In this case with a lot of different programs. Cisco ASA... My inputs is syslog, forwarding rsyslog massages in type "syslog" input { syslog { type… --- ## [Message field within message](https://discuss.elastic.co/t/message-field-within-message/58051)
**Author:** [@jaykumar](https://discuss.elastic.co/u/jaykumar)\ **Replies:** 20\ **Last updated:** [August 17, 2016, 7:14am UTC](https://discuss.elastic.co/t/message-field-within-message/58051 "2016-08-17T07:14:02Z")
Hi All, Raw SYSLOG event: Jul 21 01:19:57.58 172.20.20.100 date=2016-07-20 time=20:12:25 timezone="UTC" device\_name="CRiV-1C" device\_id=1234-567 log\_id=062009517502 log\_type="Event" log\_component="GUI" log\_subtype="A… --- ## [Need to create index on weekly basis](https://discuss.elastic.co/t/need-to-create-index-on-weekly-basis/57893)
**Author:** [@gayathri](https://discuss.elastic.co/u/gayathri)\ **Replies:** 6\ **Last updated:** [August 17, 2016, 6:09am UTC](https://discuss.elastic.co/t/need-to-create-index-on-weekly-basis/57893 "2016-08-17T06:09:02Z")
Hi Team, I want to create my index on weekly basis, Can You suggest any method --- ## [Parsing problems on logstash + filebeat](https://discuss.elastic.co/t/parsing-problems-on-logstash-filebeat/57967)
**Author:** [@dfaropennetwork](https://discuss.elastic.co/u/dfaropennetwork)\ **Replies:** 2\ **Last updated:** [August 17, 2016, 5:35am UTC](https://discuss.elastic.co/t/parsing-problems-on-logstash-filebeat/57967 "2016-08-17T05:35:39Z")
Hi all, I've problems for parsing multiline logs with logstash and filebeat: There my error on logstash: {:timestamp=\>"2016-08-13T05:41:22.216000+0000", :message=\>"fetched an invalid config", :config=\>" filter {\\n if… --- ## [Error Bundler::HTTPError, retrying 1/10](https://discuss.elastic.co/t/error-bundler-httperror-retrying-1-10/58192)
**Author:** [@vardhasu](https://discuss.elastic.co/u/vardhasu)\ **Replies:** 0\ **Last updated:** [August 16, 2016, 9:12pm UTC](https://discuss.elastic.co/t/error-bundler-httperror-retrying-1-10/58192 "2016-08-16T21:12:39Z")
Hi All Tried to install bin/logstash-plugin install --local --no-verify /tmp/logstash-core-2.3.4-java.gem but I keep getting the HTTPError. I tried setting the HTTP\_PROXY=http://127.0.0.1:3128 still same issue. I w… --- ## [Logstash filter duration calculation](https://discuss.elastic.co/t/logstash-filter-duration-calculation/58035)
**Author:** [@yahoo](https://discuss.elastic.co/u/yahoo)\ **Replies:** 2\ **Last updated:** [August 16, 2016, 8:15pm UTC](https://discuss.elastic.co/t/logstash-filter-duration-calculation/58035 "2016-08-16T20:15:14Z")
in my log, each log line has action start time and action end time formatted as HHmmss. I used the following code in the logstash filter. It works well if the action lasts during the same day. However, it runs into probl… --- ## [Percentage on E-mail plugin](https://discuss.elastic.co/t/percentage-on-e-mail-plugin/58026)
**Author:** [@rafaellamuniz](https://discuss.elastic.co/u/rafaellamuniz)\ **Replies:** 5\ **Last updated:** [August 16, 2016, 6:40pm UTC](https://discuss.elastic.co/t/percentage-on-e-mail-plugin/58026 "2016-08-16T18:40:36Z")
Hello, I would like to receive the variable "mem.used\_p" from Topbeat as a percentage (something like 99%), like is showed on Kibana. However, on the e-mail I receive on decimal format: 0.99 Somebody knows how to con… --- ## [S3 input plugin with multiline codec. Last one is always lost](https://discuss.elastic.co/t/s3-input-plugin-with-multiline-codec-last-one-is-always-lost/37727)
**Author:** [@Den3](https://discuss.elastic.co/u/Den3)\ **Replies:** 1\ **Last updated:** [August 16, 2016, 6:34pm UTC](https://discuss.elastic.co/t/s3-input-plugin-with-multiline-codec-last-one-is-always-lost/37727 "2016-08-16T18:34:13Z")
I have log format like this \<\<\<\\n Accept:application/json, text/javascript, \*/\*; q=0.01\\r\\n Content-Type:application/x-www-form-urlencoded; charset=UTF-8\\r\\n Origin:https://discuss.elastic.co\\r\\n Referer:https://disc… --- ## [Add field for source - JSON file via filebeats](https://discuss.elastic.co/t/add-field-for-source-json-file-via-filebeats/58162)
**Author:** [@norandom](https://discuss.elastic.co/u/norandom)\ **Replies:** 1\ **Last updated:** [August 16, 2016, 5:06pm UTC](https://discuss.elastic.co/t/add-field-for-source-json-file-via-filebeats/58162 "2016-08-16T17:06:45Z")
Hi, I am receiving a JSON file with Logstash via the filebeats agents from multiple remote systems. hostabc, hostdef, and so on. I have had the problem that the received JSON file is escaped once Logstash has writte… --- ## [Logstash Input Plugin for kafka 0.9 is not working](https://discuss.elastic.co/t/logstash-input-plugin-for-kafka-0-9-is-not-working/58146)
**Author:** [@hearvishwas](https://discuss.elastic.co/u/hearvishwas)\ **Replies:** 1\ **Last updated:** [August 16, 2016, 4:24pm UTC](https://discuss.elastic.co/t/logstash-input-plugin-for-kafka-0-9-is-not-working/58146 "2016-08-16T16:24:51Z")
logstash input plugin for kafka is not working for me: Here is my conf file: input { kafka { topic\_id =\> 'logstashlogs' } } output { stdout{} } Nothing comes up to the screen, just lostash starts, no error, nothing… --- ## [Logstash cannot load config file](https://discuss.elastic.co/t/logstash-cannot-load-config-file/58113)
**Author:** [@Nizar\_Khalifa](https://discuss.elastic.co/u/Nizar_Khalifa)\ **Replies:** 16\ **Last updated:** [August 16, 2016, 3:46pm UTC](https://discuss.elastic.co/t/logstash-cannot-load-config-file/58113 "2016-08-16T15:46:43Z")
Hello, I have a problem with Logstash configuration. Its running with 500Xmx. When i tried to increase it in /etc/default/logstash to 1g it doesnt take it into account after retasrting. But when i set LS\_HEAP\_SIZE to 1… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1113) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1115)