# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=864 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 865 --- ## [Using LIMIT on JDBC input as opposed to jdbc\_page\_size](https://discuss.elastic.co/t/using-limit-on-jdbc-input-as-opposed-to-jdbc-page-size/109970)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 0\ **Last updated:** [December 1, 2017, 5:50pm UTC](https://discuss.elastic.co/t/using-limit-on-jdbc-input-as-opposed-to-jdbc-page-size/109970 "2017-12-01T17:50:51Z")
I am trying to import a large database from a remote server using the JDBC input plugin and have been having some issues. After testing some various configs, I am seeing that if I set LIMIT 10 in the statement itself, I … --- ## [Redis connect to local socket file in Ruby filter?](https://discuss.elastic.co/t/redis-connect-to-local-socket-file-in-ruby-filter/109655)
**Author:** [@jeffkirk1](https://discuss.elastic.co/u/jeffkirk1)\ **Replies:** 10\ **Last updated:** [December 1, 2017, 5:32pm UTC](https://discuss.elastic.co/t/redis-connect-to-local-socket-file-in-ruby-filter/109655 "2017-12-01T17:32:35Z")
Hey folks, I am trying to perform some key-value pair lookups using Redis in a Ruby filter in Elastic 6.0.0. I have the lookup working (filter shown below): filter { ruby { init =\> "require 'redis'; require 'time… --- ## [Grokking Voip logs - grokdebug to logstash.conf](https://discuss.elastic.co/t/grokking-voip-logs-grokdebug-to-logstash-conf/109817)
**Author:** [@Winfield\_Henry](https://discuss.elastic.co/u/Winfield_Henry)\ **Replies:** 3\ **Last updated:** [December 1, 2017, 2:51pm UTC](https://discuss.elastic.co/t/grokking-voip-logs-grokdebug-to-logstash-conf/109817 "2017-12-01T14:51:40Z")
Hello, We are trying to make some sense of our voip logs using ELK. We are currently running ELK 6.0, feeding logstash via filebeat running on syslog server. Voip logs are being deposited in separate .log file in /var/l… --- ## [Problem with logstash output to elastic via ssl](https://discuss.elastic.co/t/problem-with-logstash-output-to-elastic-via-ssl/109248)
**Author:** [@111126](https://discuss.elastic.co/u/111126)\ **Replies:** 4\ **Last updated:** [December 1, 2017, 3:27pm UTC](https://discuss.elastic.co/t/problem-with-logstash-output-to-elastic-via-ssl/109248 "2017-12-01T15:27:37Z")
Hello. I have the next output config: output { file { path =\> "/var/log/logstash/output" } elasticsearch { hosts =\> \["https://127.0.0.1:9200"\] … --- ## [Logstash cuts events with length longer then 2048 symbols](https://discuss.elastic.co/t/logstash-cuts-events-with-length-longer-then-2048-symbols/109940)
**Author:** [@vladf](https://discuss.elastic.co/u/vladf)\ **Replies:** 0\ **Last updated:** [December 1, 2017, 2:00pm UTC](https://discuss.elastic.co/t/logstash-cuts-events-with-length-longer-then-2048-symbols/109940 "2017-12-01T14:00:46Z")
Dear community, I am trying to get some logs from rsyslog to logstash. Logstash recieves a correct json. However when event length is longer then 2048 symbols it cuts it. I am using tcp input to recieve the messages. B… --- ## [500 error when CURLing to Logstash with nmap codec enabled](https://discuss.elastic.co/t/500-error-when-curling-to-logstash-with-nmap-codec-enabled/109914)
**Author:** [@leoponton](https://discuss.elastic.co/u/leoponton)\ **Replies:** 0\ **Last updated:** [December 1, 2017, 11:07am UTC](https://discuss.elastic.co/t/500-error-when-curling-to-logstash-with-nmap-codec-enabled/109914 "2017-12-01T11:07:25Z")
I have the problem described in this linked (and closed) issue: I have changed the CURL statement to --data-binary, but the error remains. command: curl -H "Content-Type:text/xml;charset=UTF-8" 'http://192.168.6.6:80… --- ## [Sincedb data logic](https://discuss.elastic.co/t/sincedb-data-logic/109569)
**Author:** [@Bostjan](https://discuss.elastic.co/u/Bostjan)\ **Replies:** 2\ **Last updated:** [December 1, 2017, 10:58am UTC](https://discuss.elastic.co/t/sincedb-data-logic/109569 "2017-12-01T10:58:32Z")
Hello! I have a question regarding how file information is stored in sincedb file. As I understand, structure of file is file inode/major device name/minor device name/byte offset. What I can see is that all of this in… --- ## [How to conditionaly set type of event](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906)
**Author:** [@rinmalavi](https://discuss.elastic.co/u/rinmalavi)\ **Replies:** 6\ **Last updated:** [December 1, 2017, 10:52am UTC](https://discuss.elastic.co/t/how-to-conditionaly-set-type-of-event/109906 "2017-12-01T10:52:47Z")
I have more than one type of document per file. I want to set them apart with different types. This topic lead me to writing : replace =\> { "type" =\> "%{log\_type}"} "log\_type" was added in grok and separates names … --- ## [Dots filter](https://discuss.elastic.co/t/dots-filter/109754)
**Author:** [@rinmalavi](https://discuss.elastic.co/u/rinmalavi)\ **Replies:** 1\ **Last updated:** [December 1, 2017, 10:51am UTC](https://discuss.elastic.co/t/dots-filter/109754 "2017-12-01T10:51:43Z")
Is what i am trying to do. So i added dots {} to filter clause. And so I get: Couldn't find any filter plugin named 'dots'. Are you sure this is correct? Trying to load the dots filter plugin resulted in this error: … --- ## [Encountered a retryable error,Will Retry with exponential backoff](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff/109883)
**Author:** [@isummersing](https://discuss.elastic.co/u/isummersing)\ **Replies:** 1\ **Last updated:** [December 1, 2017, 10:49am UTC](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff/109883 "2017-12-01T10:49:38Z")
this error occured when i use logstash to collect logs and output it to es ; i use logstash on my linux server and logstash version is 6.0.0 with es and kibana ; also my es shows that "unable to load JNA native support… --- ## [Fail to use logstash-output-gelf](https://discuss.elastic.co/t/fail-to-use-logstash-output-gelf/109730)
**Author:** [@jlbassereau](https://discuss.elastic.co/u/jlbassereau)\ **Replies:** 2\ **Last updated:** [December 1, 2017, 10:31am UTC](https://discuss.elastic.co/t/fail-to-use-logstash-output-gelf/109730 "2017-12-01T10:31:53Z")
Hello, I'm trying to use logstash-output-gelf to send logs to a Graylog instance. I've installed a fresh logstash-6.0.0 and installed the plugin with : logstash-plugin install logstash-output-gelf No error at this … --- ## [ELK+FileBeat](https://discuss.elastic.co/t/elk-filebeat/109873)
**Author:** [@vijaygn1990](https://discuss.elastic.co/u/vijaygn1990)\ **Replies:** 3\ **Last updated:** [December 1, 2017, 9:50am UTC](https://discuss.elastic.co/t/elk-filebeat/109873 "2017-12-01T09:50:25Z")
I have been trying to set a ELK stack with file beat installed on all my cluster. Here my config is I am using my PC(host1) as ELK server with file beat agent installed to monitor logs. I have other PC which has install… --- ## [How to iterate for loop in logstash](https://discuss.elastic.co/t/how-to-iterate-for-loop-in-logstash/109892)
**Author:** [@Shekhar\_Pandey](https://discuss.elastic.co/u/Shekhar_Pandey)\ **Replies:** 0\ **Last updated:** [December 1, 2017, 8:23am UTC](https://discuss.elastic.co/t/how-to-iterate-for-loop-in-logstash/109892 "2017-12-01T08:23:32Z")
I have created two array in logstash..which code are given below: filter{ ruby { code =\> ' event\['ids'\] = \[1100,1101,1102,1103,1104\] event\['value'\] = \["apple","dell","mi","lenovo","nokia"\] if event\[ids\]\[i\] == 1100 … --- ## [Grok filter to retrieve simple words or character from text/string](https://discuss.elastic.co/t/grok-filter-to-retrieve-simple-words-or-character-from-text-string/109177)
**Author:** [@scch](https://discuss.elastic.co/u/scch)\ **Replies:** 7\ **Last updated:** [December 1, 2017, 8:06am UTC](https://discuss.elastic.co/t/grok-filter-to-retrieve-simple-words-or-character-from-text-string/109177 "2017-12-01T08:06:53Z")
Hi , I have used CSV filter to import data . Can someone share example code to use grok filter to retrieve simple words from fields imported previously through csv filter and add them to new field. up to last dot --- ## [Partial match in dictionary of translate - Logstash](https://discuss.elastic.co/t/partial-match-in-dictionary-of-translate-logstash/109861)
**Author:** [@jdinocco](https://discuss.elastic.co/u/jdinocco)\ **Replies:** 1\ **Last updated:** [December 1, 2017, 7:25am UTC](https://discuss.elastic.co/t/partial-match-in-dictionary-of-translate-logstash/109861 "2017-12-01T07:25:43Z")
I hope you can help me, I am trying to add a field based on partial words in a certain field. I'm trying to add the level of severity of the Cisco logs, some examples of logs are: BGP\_SESSION-5-ADJCHANGE SECURITY-SSHD… --- ## [Add current timestamp to a logstash\_processed\_at field](https://discuss.elastic.co/t/add-current-timestamp-to-a-logstash-processed-at-field/109745)
**Author:** [@Claude\_Sanderson](https://discuss.elastic.co/u/Claude_Sanderson)\ **Replies:** 5\ **Last updated:** [December 1, 2017, 6:47am UTC](https://discuss.elastic.co/t/add-current-timestamp-to-a-logstash-processed-at-field/109745 "2017-12-01T06:47:27Z")
Currently there are a lot of FIlebeat instances in our infrastructure. All of them are sending tons of logs to a single Logstash endpoint. Some logs are not in json and come as a text. For example, Postfix logs. Each … --- ## [How to get part of a field of parsed json message in logstash](https://discuss.elastic.co/t/how-to-get-part-of-a-field-of-parsed-json-message-in-logstash/109737)
**Author:** [@elk\_chaser](https://discuss.elastic.co/u/elk_chaser)\ **Replies:** 5\ **Last updated:** [December 1, 2017, 6:46am UTC](https://discuss.elastic.co/t/how-to-get-part-of-a-field-of-parsed-json-message-in-logstash/109737 "2017-12-01T06:46:08Z")
I am trying to get just the date part from @report\_timestamp field of a parsed json message in logstash configuration so that I can name the output log file name based on date, say request\_2017\_08\_01.log my message look… --- ## [Logstash index template](https://discuss.elastic.co/t/logstash-index-template/109732)
**Author:** [@pavuk](https://discuss.elastic.co/u/pavuk)\ **Replies:** 5\ **Last updated:** [December 1, 2017, 5:59am UTC](https://discuss.elastic.co/t/logstash-index-template/109732 "2017-12-01T05:59:26Z")
Sorry for stupid question, it's my first Logstash experience, but I would like to ask about following issue. I have two LS configs: # cat /etc/logstash/conf.d/nginx.conf input { file { type =\> "nginx" start\_… --- ## [\[logstash.outputs.elasticsearch\] UNEXPECTED POOL ERROR](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-unexpected-pool-error/109513)
**Author:** [@Krunal\_kalaria](https://discuss.elastic.co/u/Krunal_kalaria)\ **Replies:** 2\ **Last updated:** [December 1, 2017, 5:07am UTC](https://discuss.elastic.co/t/logstash-outputs-elasticsearch-unexpected-pool-error/109513 "2017-12-01T05:07:24Z")
Hello, i have an issue with my logstash when i am running my configuration file then some logs are coming then automatically its giving some error like \[2017-11-29T11:20:51,023\]\[WARN \]\[logstash.outputs.elasticsearch\] … --- ## [Using environment variables in pipelines.yml (LS 6.0.0.)](https://discuss.elastic.co/t/using-environment-variables-in-pipelines-yml-ls-6-0-0/109674)
**Author:** [@zpraxis](https://discuss.elastic.co/u/zpraxis)\ **Replies:** 3\ **Last updated:** [December 1, 2017, 4:02am UTC](https://discuss.elastic.co/t/using-environment-variables-in-pipelines-yml-ls-6-0-0/109674 "2017-12-01T04:02:23Z")
I am trying to use an environment variable in my pipelines.yml file. It looks something like this: - pipeline.id: test path.config: '${LS\_SETTINGS\_DIR}/test.conf' I am using Linux RedHat distribution, Logstash was in… --- ## [Kibana MAP stopt working without making chances expect new input](https://discuss.elastic.co/t/kibana-map-stopt-working-without-making-chances-expect-new-input/109220)
**Author:** [@bobwilly](https://discuss.elastic.co/u/bobwilly)\ **Replies:** 3\ **Last updated:** [December 1, 2017, 2:27am UTC](https://discuss.elastic.co/t/kibana-map-stopt-working-without-making-chances-expect-new-input/109220 "2017-12-01T02:27:02Z")
For some reason my geo.ip.location stopt working. I tryed to reindex my data but the field is gone. I have not made any changes in the last days. Starting logstash output: \[2017-11-27T13:47:01,091\]\[INFO \]\[logstash.ou… --- ## [UDP, merged messages](https://discuss.elastic.co/t/udp-merged-messages/109856)
**Author:** [@robert-blankenship](https://discuss.elastic.co/u/robert-blankenship)\ **Replies:** 0\ **Last updated:** [December 1, 2017, 12:39am UTC](https://discuss.elastic.co/t/udp-merged-messages/109856 "2017-12-01T00:39:44Z")
I have a Logstash instance set up with a simple UDP listener, and a single output to Kafka input { udp { port =\> 514 codec =\> line } } filter { grok { match =\> #stuff t… --- ## [Access Log Filter](https://discuss.elastic.co/t/access-log-filter/109556)
**Author:** [@deepakkr407](https://discuss.elastic.co/u/deepakkr407)\ **Replies:** 1\ **Last updated:** [November 30, 2017, 11:40pm UTC](https://discuss.elastic.co/t/access-log-filter/109556 "2017-11-30T23:40:20Z")
I am using following filter but this is not fetching response code and timetaken as column in elastic search. Any suggestions? { "message" =\> \[ "%{TIMESTAMP\_ISO8601:log\_timestamp} %{NOTSPACE:s-sitename} %{NOTSPACE:s-com… --- ## [Logstash 6 - log rotation](https://discuss.elastic.co/t/logstash-6-log-rotation/109532)
**Author:** [@Martin\_Duris](https://discuss.elastic.co/u/Martin_Duris)\ **Replies:** 1\ **Last updated:** [November 30, 2017, 11:32pm UTC](https://discuss.elastic.co/t/logstash-6-log-rotation/109532 "2017-11-30T23:32:05Z")
Hi, i have upgraded to new version of Logstash 6, but i cannot setup logstash/log4j to keep only few log files (log files which are created by logstash, logstash installed from package dep package) (for example 7). My l… --- ## [Installing logstash-plugin - logstash-input-beats](https://discuss.elastic.co/t/installing-logstash-plugin-logstash-input-beats/109524)
**Author:** [@Juliette](https://discuss.elastic.co/u/Juliette)\ **Replies:** 1\ **Last updated:** [November 30, 2017, 11:25pm UTC](https://discuss.elastic.co/t/installing-logstash-plugin-logstash-input-beats/109524 "2017-11-30T23:25:41Z")
Hi everybody :grinning: I need your help ! When I try to install the logstash plugin, I receive an error : \[ELK Logstash\]$ bin/logstash-plugin install /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-be… --- ## [Elapsed plugin missing start events](https://discuss.elastic.co/t/elapsed-plugin-missing-start-events/109848)
**Author:** [@joconner](https://discuss.elastic.co/u/joconner)\ **Replies:** 0\ **Last updated:** [November 30, 2017, 10:29pm UTC](https://discuss.elastic.co/t/elapsed-plugin-missing-start-events/109848 "2017-11-30T22:29:52Z")
Processing potentially several thousands of events per second, the elapsed filter plugin seems to miss about 10% of my start/end tagged events. The tag message I often see in the "end" events is elapsed\_end\_without\_start.… --- ## [Logstash-salesforce-input plugin interval refresh](https://discuss.elastic.co/t/logstash-salesforce-input-plugin-interval-refresh/109830)
**Author:** [@mPanasiewicz](https://discuss.elastic.co/u/mPanasiewicz)\ **Replies:** 0\ **Last updated:** [November 30, 2017, 8:20pm UTC](https://discuss.elastic.co/t/logstash-salesforce-input-plugin-interval-refresh/109830 "2017-11-30T20:20:56Z")
Hello, When logstash starts it fetches only once data. I'd like to refresh my data in cycles/intervals/schedule. What is the best option for this. For now i would like to try cron tasks. In my opinion it is not good sol… --- ## [Logsstash filling up /var/log/syslog?](https://discuss.elastic.co/t/logsstash-filling-up-var-log-syslog/108234)
**Author:** [@mathurin68](https://discuss.elastic.co/u/mathurin68)\ **Replies:** 4\ **Last updated:** [November 30, 2017, 9:19pm UTC](https://discuss.elastic.co/t/logsstash-filling-up-var-log-syslog/108234 "2017-11-30T21:19:53Z")
syslog in /var/log filling up with logstash? Every event coming through logstash is filling up syslog - ogstash\[54646\]: "type" =\> "wineventlog", Nov 18 06:25:05 elkstack logstash\[54646\]: "opcod… --- ## [SOLVED - Logstash-output-kafka JDBC Results](https://discuss.elastic.co/t/solved-logstash-output-kafka-jdbc-results/109829)
**Author:** [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\ **Replies:** 1\ **Last updated:** [November 30, 2017, 8:36pm UTC](https://discuss.elastic.co/t/solved-logstash-output-kafka-jdbc-results/109829 "2017-11-30T20:36:35Z")
Hi All, I have a logstash config that connects to a JDBC connection and I want to output to a Kafka Cluster. I am able to connect fine , I am also seeing messages expect they look like the below rather than my actual me… --- ## [Grokking multiple line formats](https://discuss.elastic.co/t/grokking-multiple-line-formats/109814)
**Author:** [@StivOstenberg](https://discuss.elastic.co/u/StivOstenberg)\ **Replies:** 3\ **Last updated:** [November 30, 2017, 8:25pm UTC](https://discuss.elastic.co/t/grokking-multiple-line-formats/109814 "2017-11-30T20:25:08Z")
New to grokking, and trying to modify an AWS ELB log filter to handle AWS ALB logs (one additional field at the beginning, 4 at the end) if \[type\] == "{{logstash\_elb\_access\_logs\_type}}" { grok { match =\> \["message", "… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=863) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=865)