# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1071 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1072 --- ## [Parsing a log with colon separated fields](https://discuss.elastic.co/t/parsing-a-log-with-colon-separated-fields/67433)
**Author:** [@verno\_s](https://discuss.elastic.co/u/verno_s)\ **Replies:** 3\ **Last updated:** [November 29, 2016, 11:53pm UTC](https://discuss.elastic.co/t/parsing-a-log-with-colon-separated-fields/67433 "2016-11-29T23:53:04Z")
Hi, I've tried to read the Logstash docs on grok parsing but i can't find an answer to my issue. I want to parse a : separated logfile e.g. 29/11/2016 11:49:17 AM : Response: 125 Data connection already open; Tra… --- ## [Java exception when using cap\_net\_bind\_service+ep with Logstash 5.x](https://discuss.elastic.co/t/java-exception-when-using-cap-net-bind-service-ep-with-logstash-5-x/63947)
**Author:** [@nick-george](https://discuss.elastic.co/u/nick-george)\ **Replies:** 6\ **Last updated:** [November 29, 2016, 11:09pm UTC](https://discuss.elastic.co/t/java-exception-when-using-cap-net-bind-service-ep-with-logstash-5-x/63947 "2016-11-29T23:09:16Z")
I require Logstash to listen on TCP and UDP port 514. To do this, I use setcap to allow java to bind to this low port number. After doing this (and rerunning ldconfig), Java throws an exception. logstash: Sending Logst… --- ## [Ruby code filter error](https://discuss.elastic.co/t/ruby-code-filter-error/67391)
**Author:** [@zahodi](https://discuss.elastic.co/u/zahodi)\ **Replies:** 2\ **Last updated:** [November 29, 2016, 10:37pm UTC](https://discuss.elastic.co/t/ruby-code-filter-error/67391 "2016-11-29T22:37:25Z")
Having a hard time pinpointing why I'm getting this error: My filter: filter { ruby { code =\> "event.set('archdate', Time.now.utc.strftime('%Y-%m-%dT%H:%M:%S%z')" } } --- ## [How do I run the core rspec unit tests for logstash from rake without this error?](https://discuss.elastic.co/t/how-do-i-run-the-core-rspec-unit-tests-for-logstash-from-rake-without-this-error/67554)
**Author:** [@PaulChernoch](https://discuss.elastic.co/u/PaulChernoch)\ **Replies:** 1\ **Last updated:** [November 29, 2016, 9:59pm UTC](https://discuss.elastic.co/t/how-do-i-run-the-core-rspec-unit-tests-for-logstash-from-rake-without-this-error/67554 "2016-11-29T21:59:59Z")
I am building logstash 1.5.3-1 from source. When I run the core unit tests using rspec, they all pass, but when I run them using rake, a bunch fail. My environment is an ubuntu trusty docker container on Docker for Mac… --- ## [Best practise method to split events and perform filtering to those events](https://discuss.elastic.co/t/best-practise-method-to-split-events-and-perform-filtering-to-those-events/67115)
**Author:** [@jdmac](https://discuss.elastic.co/u/jdmac)\ **Replies:** 1\ **Last updated:** [November 29, 2016, 9:22pm UTC](https://discuss.elastic.co/t/best-practise-method-to-split-events-and-perform-filtering-to-those-events/67115 "2016-11-29T21:22:00Z")
Hi All, I'm trying to do something that I'm not seeing an easy way to do and I'm not all that familiar with Ruby, so I'd prefer to use standard filters when/where possible. Here's the scope of what I'm trying to achi… --- ## [Tags filed added by logstash](https://discuss.elastic.co/t/tags-filed-added-by-logstash/67531)
**Author:** [@agrm](https://discuss.elastic.co/u/agrm)\ **Replies:** 0\ **Last updated:** [November 29, 2016, 6:08pm UTC](https://discuss.elastic.co/t/tags-filed-added-by-logstash/67531 "2016-11-29T18:08:35Z")
I set a UDP listener in logstash pipe to get packet from a specific port. The packets are in JSON format. When I perform formatting on received packets, Logstash adds additional field (e.g. @timestamp, @version, tags,… --- ## [Jdbc input, no filters, csv output =\> CompatibilityError: incompatible encodings: UTF-8 and ASCII-8BIT](https://discuss.elastic.co/t/jdbc-input-no-filters-csv-output-compatibilityerror-incompatible-encodings-utf-8-and-ascii-8bit/67484)
**Author:** [@reuschling](https://discuss.elastic.co/u/reuschling)\ **Replies:** 1\ **Last updated:** [November 29, 2016, 2:23pm UTC](https://discuss.elastic.co/t/jdbc-input-no-filters-csv-output-compatibilityerror-incompatible-encodings-utf-8-and-ascii-8bit/67484 "2016-11-29T14:23:29Z")
I read out sql server table data with jdbc input plugin, generating a csv file with csv output plugin. At some time (reproducable after a view minutes) it seems that an event has the content that yields to a Compatibilit… --- ## [I cant parse with grok](https://discuss.elastic.co/t/i-cant-parse-with-grok/66875)
**Author:** [@jjgil](https://discuss.elastic.co/u/jjgil)\ **Replies:** 2\ **Last updated:** [November 29, 2016, 1:10pm UTC](https://discuss.elastic.co/t/i-cant-parse-with-grok/66875 "2016-11-29T13:10:37Z")
Hi! I'am new in logstash. My example log: 30116102503C01 016405404007100 I want parse the next way: 30116102503C 01 0164054 04007 100 But, I've only managed to do this: { "tipo": \[ "30116102503C" \], … --- ## [Logstash-input-mongodb for logtash 5](https://discuss.elastic.co/t/logstash-input-mongodb-for-logtash-5/67482)
**Author:** [@sonpv1](https://discuss.elastic.co/u/sonpv1)\ **Replies:** 0\ **Last updated:** [November 29, 2016, 12:43pm UTC](https://discuss.elastic.co/t/logstash-input-mongodb-for-logtash-5/67482 "2016-11-29T12:43:37Z")
Hi Experts, Could you please help me to provide me with logstash-input-mongodb for Logstash 5. I can not resolve this problem. Thank you so much --- ## [Logstash pipeline error](https://discuss.elastic.co/t/logstash-pipeline-error/67461)
**Author:** [@TeePee](https://discuss.elastic.co/u/TeePee)\ **Replies:** 0\ **Last updated:** [November 29, 2016, 10:09am UTC](https://discuss.elastic.co/t/logstash-pipeline-error/67461 "2016-11-29T10:09:14Z")
Hi, When processing input files from S3, sometimes, during that operation, logstash returns the error below and stops. When checking service logstash status is says active but logstash stays dead until the service gets… --- ## [Can check data type before output?](https://discuss.elastic.co/t/can-check-data-type-before-output/67452)
**Author:** [@tao\_hiko](https://discuss.elastic.co/u/tao_hiko)\ **Replies:** 2\ **Last updated:** [November 29, 2016, 10:17am UTC](https://discuss.elastic.co/t/can-check-data-type-before-output/67452 "2016-11-29T10:17:49Z")
I'm using Logstash 5.0 and migrate data from ES 1.7 to ES 5.0 but I have a problem about conflicted data type on the same filed name but different type. How to check data type of each field? Because I cannot change … --- ## [Disk spooling with logstash?](https://discuss.elastic.co/t/disk-spooling-with-logstash/67458)
**Author:** [@tomx1](https://discuss.elastic.co/u/tomx1)\ **Replies:** 2\ **Last updated:** [November 29, 2016, 10:09am UTC](https://discuss.elastic.co/t/disk-spooling-with-logstash/67458 "2016-11-29T10:09:07Z")
We are using the Elastic Stack for our centralized logging. It works great but there is still a lack of knowledge regarding the spool capabilities of logstash shipping those events to elasticsearch. We are running our El… --- ## [Error: java.util.concurrent.RejectedExecutionException: event executor terminated](https://discuss.elastic.co/t/error-java-util-concurrent-rejectedexecutionexception-event-executor-terminated/67444)
**Author:** [@Mahebub\_Sayyed](https://discuss.elastic.co/u/Mahebub_Sayyed)\ **Replies:** 0\ **Last updated:** [November 29, 2016, 8:08am UTC](https://discuss.elastic.co/t/error-java-util-concurrent-rejectedexecutionexception-event-executor-terminated/67444 "2016-11-29T08:08:54Z")
Hi, I am following Logstash tutorial "Parsing Logs with Logstash". Ref: https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html#\_configuring\_logstash\_for\_filebeat\_input I am getting following error. \[20… --- ## [Elasticsearch connection refused](https://discuss.elastic.co/t/elasticsearch-connection-refused/67435)
**Author:** [@roiyax](https://discuss.elastic.co/u/roiyax)\ **Replies:** 6\ **Last updated:** [November 29, 2016, 7:18am UTC](https://discuss.elastic.co/t/elasticsearch-connection-refused/67435 "2016-11-29T07:18:49Z")
Hi, I have a problem in my ELK setup with below log messages. {:timestamp=\>"2016-11-28T21:10:22.958000+1100", :message=\>"Attempted to send a bulk request to Elasticsearch configured at '\[\\"http://localhost:9200\\"\]', bu… --- ## [Creating multiple fields from single field](https://discuss.elastic.co/t/creating-multiple-fields-from-single-field/67382)
**Author:** [@marinoc](https://discuss.elastic.co/u/marinoc)\ **Replies:** 2\ **Last updated:** [November 29, 2016, 12:59am UTC](https://discuss.elastic.co/t/creating-multiple-fields-from-single-field/67382 "2016-11-29T00:59:08Z")
I'm using the csv input filter to ingest csv files into Elasticsearch. One of the fields, "CustomFields" contains key/value pairs like this: CustomFields """Author=Jules J. Toner, S.J."";Binding=Paper;""Publication Dat… --- ## [Can not remove "tags" field?](https://discuss.elastic.co/t/can-not-remove-tags-field/67036)
**Author:** [@agrm](https://discuss.elastic.co/u/agrm)\ **Replies:** 0\ **Last updated:** [November 23, 2016, 9:54pm UTC](https://discuss.elastic.co/t/can-not-remove-tags-field/67036 "2016-11-23T21:54:01Z")
I have the following pipe configuration (See bellow). It removes all additional fields except "tags". I am using the latest elasticsearch and logstash software. What could be wrong that it does not remove "tage"? input … --- ## [GROK Filter for Message to Pull Requested URL Path](https://discuss.elastic.co/t/grok-filter-for-message-to-pull-requested-url-path/67390)
**Author:** [@Mike\_Clarke](https://discuss.elastic.co/u/Mike_Clarke)\ **Replies:** 0\ **Last updated:** [November 28, 2016, 7:05pm UTC](https://discuss.elastic.co/t/grok-filter-for-message-to-pull-requested-url-path/67390 "2016-11-28T19:05:44Z")
Below is a sample of 2 messages that I receive and am trying to run through a GROK filter to capture the requested URL. I would like to discard everything starting with the ? until the next element which is the response… --- ## [Http\_Polling and App-Keys](https://discuss.elastic.co/t/http-polling-and-app-keys/67362)
**Author:** [@AdamOlsson](https://discuss.elastic.co/u/AdamOlsson)\ **Replies:** 2\ **Last updated:** [November 28, 2016, 6:55pm UTC](https://discuss.elastic.co/t/http-polling-and-app-keys/67362 "2016-11-28T18:55:46Z")
Hi! I'm trying to setup Logstash to retrieve data from Pingdom. I have managed to connect to the Pingdom server but i need to add the application key that Pingdom provided me to the header. I can't figure out how or w… --- ## [HTTP\_POLLER and form data](https://discuss.elastic.co/t/http-poller-and-form-data/67384)
**Author:** [@TonyEnnis](https://discuss.elastic.co/u/TonyEnnis)\ **Replies:** 0\ **Last updated:** [November 28, 2016, 5:30pm UTC](https://discuss.elastic.co/t/http-poller-and-form-data/67384 "2016-11-28T17:30:35Z")
We're using the http\_poller input plug-in to communicate with Zabbix. Zabbix requires POST and form data. How do I specify form data when using HTTP\_POLLER? --- ## [Beats Input Exception](https://discuss.elastic.co/t/beats-input-exception/55537)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 4\ **Last updated:** [November 28, 2016, 4:40pm UTC](https://discuss.elastic.co/t/beats-input-exception/55537 "2016-11-28T16:40:10Z")
Hi, Would appreciate your input into this exception I'm seeing with the beats input: {: timestamp =\> "2016-07-14T16:18:11.269000+0100", : message =\> "Beats input: unhandled exception", : exception =\> # \< NoMethodError:… --- ## [New installed Logstash not work as service](https://discuss.elastic.co/t/new-installed-logstash-not-work-as-service/67352)
**Author:** [@breezzz](https://discuss.elastic.co/u/breezzz)\ **Replies:** 2\ **Last updated:** [November 28, 2016, 1:51pm UTC](https://discuss.elastic.co/t/new-installed-logstash-not-work-as-service/67352 "2016-11-28T13:51:21Z")
Hello! I have installed Logstash 5 "yum install logstash" to same machine with elastic client node. Then create simple config: cat /etc/logstash/conf.d/nginx\_input.conf input { file { path =\> "/var/log/… --- ## [File input for gz files](https://discuss.elastic.co/t/file-input-for-gz-files/35822)
**Author:** [@mvenkat\_in](https://discuss.elastic.co/u/mvenkat_in)\ **Replies:** 11\ **Last updated:** [November 28, 2016, 1:36pm UTC](https://discuss.elastic.co/t/file-input-for-gz-files/35822 "2016-11-28T13:36:30Z")
HI I have seen few topics on logstash supporting reading from the gz files. Would like to know if this feature is supported !! I am using logstash 2.1.0. If supported please share me the sample input config. Below … --- ## [Problems with installation of the OUI filter plugin and logstash 5.0.1](https://discuss.elastic.co/t/problems-with-installation-of-the-oui-filter-plugin-and-logstash-5-0-1/67087)
**Author:** [@okazymyrov](https://discuss.elastic.co/u/okazymyrov)\ **Replies:** 3\ **Last updated:** [November 28, 2016, 12:47pm UTC](https://discuss.elastic.co/t/problems-with-installation-of-the-oui-filter-plugin-and-logstash-5-0-1/67087 "2016-11-28T12:47:40Z")
I'm using logstash 5.0.1 on RHEL 7.3 and trying to install the OUI file plugin. The command sudo -u logstash bin/logstash-plugin install logstash-filter-oui returns: ... Bundler could not find compatible versions fo… --- ## [How can I debug what influence the write-in speed from logstash:input:file to es?](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173)
**Author:** [@Morriaty](https://discuss.elastic.co/u/Morriaty)\ **Replies:** 7\ **Last updated:** [November 28, 2016, 10:56am UTC](https://discuss.elastic.co/t/how-can-i-debug-what-influence-the-write-in-speed-from-logstash-input-file-to-es/67173 "2016-11-28T10:56:37Z")
Hi I found it very slow when writing to es. And I've made a test input { file { path =\> "access.log" } } output { elasticsearch { index =\> "ur\_test\_%{+YYYY.MM.dd}" document… --- ## [Negative look behind not working](https://discuss.elastic.co/t/negative-look-behind-not-working/67329)
**Author:** [@Arthur\_Francis](https://discuss.elastic.co/u/Arthur_Francis)\ **Replies:** 0\ **Last updated:** [November 28, 2016, 10:52am UTC](https://discuss.elastic.co/t/negative-look-behind-not-working/67329 "2016-11-28T10:52:44Z")
I am trying to match a line in the logfile which does not start with "Caused by" but the negative lookbehind I am using is not working, even though it works in the grok debugger and also in rubular (http://rubular.com/r/… --- ## [ArgumentError: short\_message is missing. Options version, short\_message and host must be set](https://discuss.elastic.co/t/argumenterror-short-message-is-missing-options-version-short-message-and-host-must-be-set/59033)
**Author:** [@bujar\_metaj](https://discuss.elastic.co/u/bujar_metaj)\ **Replies:** 2\ **Last updated:** [November 28, 2016, 10:22am UTC](https://discuss.elastic.co/t/argumenterror-short-message-is-missing-options-version-short-message-and-host-must-be-set/59033 "2016-11-28T10:22:32Z")
Hi, I need help with this issue my config is as follows input { beats { port =\> 5044 ssl =\> true ssl\_certificate =\> "/etc/pki/tls/certs/logstash-forwarder.crt" ssl\_key =\> "/etc/pki/tls/private/logs… --- ## [Using @metadata in logstash custom pattern for grok?](https://discuss.elastic.co/t/using-metadata-in-logstash-custom-pattern-for-grok/67177)
**Author:** [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\ **Replies:** 2\ **Last updated:** [November 28, 2016, 10:03am UTC](https://discuss.elastic.co/t/using-metadata-in-logstash-custom-pattern-for-grok/67177 "2016-11-28T10:03:30Z")
Hi. I am trying to use @metadata in logstash custom pattern for grok. But looks like @ is not allowed when using in the fieldname. Is there any way to escape @ in the custom pattern? \[2016-11-25T19:51:21,685\]\[ERROR\]\[… --- ## [Trouble sending GELF event](https://discuss.elastic.co/t/trouble-sending-gelf-event/24868)
**Author:** [@nzHillNet](https://discuss.elastic.co/u/nzHillNet)\ **Replies:** 4\ **Last updated:** [November 28, 2016, 10:00am UTC](https://discuss.elastic.co/t/trouble-sending-gelf-event/24868 "2016-11-28T10:00:04Z")
Hi all, I see a few "Google" entries with this issue, but haven't located a fix. In short I have Windows event logs being shipped to Logstash with nxlog in JSON format. I don't currently have a filter, and the output… --- ## [Grok filter only uses last match, can this be changed?](https://discuss.elastic.co/t/grok-filter-only-uses-last-match-can-this-be-changed/67208)
**Author:** [@Arthur\_Francis](https://discuss.elastic.co/u/Arthur_Francis)\ **Replies:** 0\ **Last updated:** [November 25, 2016, 4:57pm UTC](https://discuss.elastic.co/t/grok-filter-only-uses-last-match-can-this-be-changed/67208 "2016-11-25T16:57:52Z")
If a pattern is matched several times within an event, is there a way to take into account only the first match. After doing some testing after my issue in this post here https://discuss.elastic.co/t/grok-filter-behaviou… --- ## [Mutate - replace field string using another pattern](https://discuss.elastic.co/t/mutate-replace-field-string-using-another-pattern/67081)
**Author:** [@Arthur\_Francis](https://discuss.elastic.co/u/Arthur_Francis)\ **Replies:** 2\ **Last updated:** [November 28, 2016, 8:41am UTC](https://discuss.elastic.co/t/mutate-replace-field-string-using-another-pattern/67081 "2016-11-28T08:41:25Z")
I have been reading through the logstash documentation for a way to chnage the value of a field by running it through a different pattern. Ref: replace Value type is hash There is no default value for this… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1070) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1072)