# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1154 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1155 --- ## [Is there a plugin watch a directory? ( I don't want the file contents, just the filename)](https://discuss.elastic.co/t/is-there-a-plugin-watch-a-directory-i-dont-want-the-file-contents-just-the-filename/49587)
**Author:** [@iamthealex](https://discuss.elastic.co/u/iamthealex)\ **Replies:** 6\ **Last updated:** [May 13, 2016, 9:18pm UTC](https://discuss.elastic.co/t/is-there-a-plugin-watch-a-directory-i-dont-want-the-file-contents-just-the-filename/49587 "2016-05-13T21:18:08Z")
The subject says it all. --- ## [Not able to run plugin install logstash-input-beats in Windows](https://discuss.elastic.co/t/not-able-to-run-plugin-install-logstash-input-beats-in-windows/41502)
**Author:** [@jayaram](https://discuss.elastic.co/u/jayaram)\ **Replies:** 4\ **Last updated:** [May 13, 2016, 5:59pm UTC](https://discuss.elastic.co/t/not-able-to-run-plugin-install-logstash-input-beats-in-windows/41502 "2016-05-13T17:59:51Z")
Hello, I am trying to setup the Beats--\>Logstash--\>Elasticsearch--\>Kibana for all type of beats\[Filebeat, Topbeat, Packetbeat and WinLogbeat\] In https://www.elastic.co/guide/en/beats/libbeat/current/getting-star… --- ## [Logstash restore option (ouput-s3)](https://discuss.elastic.co/t/logstash-restore-option-ouput-s3/50005)
**Author:** [@keepcosmos](https://discuss.elastic.co/u/keepcosmos)\ **Replies:** 0\ **Last updated:** [May 13, 2016, 2:27pm UTC](https://discuss.elastic.co/t/logstash-restore-option-ouput-s3/50005 "2016-05-13T14:27:41Z")
I use logstash-ouput-s3 There is restore option. Document said "IMPORTANT: if you use multiple instance of s3, you should specify on one of them the “restore=\> true” and on the others “restore =\> false”. This is hac… --- ## [Config Help with SSL between Logstash and Elasticsearch](https://discuss.elastic.co/t/config-help-with-ssl-between-logstash-and-elasticsearch/49876)
**Author:** [@Aljaz\_Gantar](https://discuss.elastic.co/u/Aljaz_Gantar)\ **Replies:** 1\ **Last updated:** [May 13, 2016, 1:29pm UTC](https://discuss.elastic.co/t/config-help-with-ssl-between-logstash-and-elasticsearch/49876 "2016-05-13T13:29:35Z")
Okei so i have this case: I have tree server which are running the whole stack, and they are set up so: Filebeat =\> Logstash =\> Elasticsearch/Kibana And I successfully configured that the Filebeat sends logs from his s… --- ## [Codec rubydebug output not showing in 'docker logs' output](https://discuss.elastic.co/t/codec-rubydebug-output-not-showing-in-docker-logs-output/47799)
**Author:** [@Alex\_Harvey](https://discuss.elastic.co/u/Alex_Harvey)\ **Replies:** 1\ **Last updated:** [May 13, 2016, 11:32am UTC](https://discuss.elastic.co/t/codec-rubydebug-output-not-showing-in-docker-logs-output/47799 "2016-05-13T11:32:21Z")
I am trying to debug the official Logstash Docker container. All available documentation I can find seems to imply that if I add an output stdout { codec =\> rubydebug }, I should be able to see this in docker logs. It'… --- ## [Logstash name fields](https://discuss.elastic.co/t/logstash-name-fields/49967)
**Author:** [@stefansaye](https://discuss.elastic.co/u/stefansaye)\ **Replies:** 3\ **Last updated:** [May 13, 2016, 9:46am UTC](https://discuss.elastic.co/t/logstash-name-fields/49967 "2016-05-13T09:46:27Z")
i have a field, the field format looks like A-B-C::E\_\[randomNum\] the field is dynamic because the randonNUM , i want to change the '-' to '\_' and remove the \[randomNUM\] and it's will be looks like as follow, A-B-C:… --- ## [Problem with lumberjack and beats](https://discuss.elastic.co/t/problem-with-lumberjack-and-beats/49975)
**Author:** [@marcmga](https://discuss.elastic.co/u/marcmga)\ **Replies:** 0\ **Last updated:** [May 13, 2016, 8:50am UTC](https://discuss.elastic.co/t/problem-with-lumberjack-and-beats/49975 "2016-05-13T08:50:05Z")
Hello, I'm receiving every X hours this error. {:timestamp=\>"2016-05-06T13:42:07.484000+0200", :message=\>"CircuitBreaker::rescuing exceptions", :name=\>"Lumberjack input", :exception=\>LogStash::SizedQueueTimeout::Timeo… --- ## [Indexing csv data in a nested field using logstash](https://discuss.elastic.co/t/indexing-csv-data-in-a-nested-field-using-logstash/49912)
**Author:** [@tamil](https://discuss.elastic.co/u/tamil)\ **Replies:** 4\ **Last updated:** [May 13, 2016, 8:44am UTC](https://discuss.elastic.co/t/indexing-csv-data-in-a-nested-field-using-logstash/49912 "2016-05-13T08:44:32Z")
With below settings im unable to index the data . Logstash pipeline starts and stands still without showing any error or producing output. Please correct me where it is going wrong Input csv: cust\_name,state1,city1,… --- ## [Logstash Throttle logic](https://discuss.elastic.co/t/logstash-throttle-logic/49963)
**Author:** [@Kishore](https://discuss.elastic.co/u/Kishore)\ **Replies:** 0\ **Last updated:** [May 13, 2016, 6:17am UTC](https://discuss.elastic.co/t/logstash-throttle-logic/49963 "2016-05-13T06:17:24Z")
I wanted to throttle events so i only receive an event after 3 occurrences and i get no more than 5 in 180 seconds. I am using below throttle condition/logic to generate an email alert. When i give the period is equ… --- ## [Does Logstash have any way of knowing what it's pid is?](https://discuss.elastic.co/t/does-logstash-have-any-way-of-knowing-what-its-pid-is/49809)
**Author:** [@David\_McClain](https://discuss.elastic.co/u/David_McClain)\ **Replies:** 1\ **Last updated:** [May 13, 2016, 6:05am UTC](https://discuss.elastic.co/t/does-logstash-have-any-way-of-knowing-what-its-pid-is/49809 "2016-05-13T06:05:30Z")
I'm trying to setup some lightweight healthchecks for my system for Logstash to get us by until 5.0 is prod ready and we can adopt it. My implementation is to use the heartbeat input plugin to generate some basic info … --- ## [File Output Pluging creates corrupted files](https://discuss.elastic.co/t/file-output-pluging-creates-corrupted-files/49953)
**Author:** [@Ingo\_Sauerzapf](https://discuss.elastic.co/u/Ingo_Sauerzapf)\ **Replies:** 0\ **Last updated:** [May 12, 2016, 11:45pm UTC](https://discuss.elastic.co/t/file-output-pluging-creates-corrupted-files/49953 "2016-05-12T23:45:37Z")
Hi everyone, we are using the file output plugin with gzip=true and we are getting quite a bit of corrupted files. At first we thought that it had to do with the size of the files but it appears quite randomly. Some of o… --- ## [Undefined method for NilClass in buffer\_flush](https://discuss.elastic.co/t/undefined-method-for-nilclass-in-buffer-flush/49944)
**Author:** [@WillBradley](https://discuss.elastic.co/u/WillBradley)\ **Replies:** 0\ **Last updated:** [May 12, 2016, 10:15pm UTC](https://discuss.elastic.co/t/undefined-method-for-nilclass-in-buffer-flush/49944 "2016-05-12T22:15:10Z")
Just encountered this error in my logs, Logstash crashed immediately afterwards. Not sure if it's an output-sqs or a stud / buffer issue. May 11 15:22:09 logstash.err: NoMethodError: undefined method \`' for nil:NilCla… --- ## [Logstash hipchat integration](https://discuss.elastic.co/t/logstash-hipchat-integration/49894)
**Author:** [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Replies:** 1\ **Last updated:** [May 12, 2016, 9:26pm UTC](https://discuss.elastic.co/t/logstash-hipchat-integration/49894 "2016-05-12T21:26:51Z")
I need to send logstash messages to hipchat. What configuration should I do in hipchat as well as in logstash.? --- ## [Kafka logstash very high memory use](https://discuss.elastic.co/t/kafka-logstash-very-high-memory-use/49939)
**Author:** [@arpan](https://discuss.elastic.co/u/arpan)\ **Replies:** 0\ **Last updated:** [May 12, 2016, 9:11pm UTC](https://discuss.elastic.co/t/kafka-logstash-very-high-memory-use/49939 "2016-05-12T21:11:08Z")
Hi, I have logstash 2.3 setup with elasticsearch 2.3. I am using the elasticsearch output and the kafka input. My input consumes from a single topic in kafka and the inputs section looks like this: input { kafka { … --- ## [Grok is not parsing GREEDYDATA field](https://discuss.elastic.co/t/grok-is-not-parsing-greedydata-field/49704)
**Author:** [@eugeniocesar](https://discuss.elastic.co/u/eugeniocesar)\ **Replies:** 5\ **Last updated:** [May 12, 2016, 8:53pm UTC](https://discuss.elastic.co/t/grok-is-not-parsing-greedydata-field/49704 "2016-05-12T20:53:47Z")
Hi Everybody! I have been facing a problem using Grok with OpenLDAP log, where it ignores a GREEDYDATA value in my match rule. Here's a working rule example: Source log line: May 9 18:53:01 openldap-master slapd\[456… --- ## [Plugin update question](https://discuss.elastic.co/t/plugin-update-question/49933)
**Author:** [@tpnoonan](https://discuss.elastic.co/u/tpnoonan)\ **Replies:** 2\ **Last updated:** [May 12, 2016, 8:30pm UTC](https://discuss.elastic.co/t/plugin-update-question/49933 "2016-05-12T20:30:42Z")
I have a question about when updates are available for newer releases of logstash plugins. There is a new release of the logstash-code-netflow plugin available. https://github.com/logstash-plugins/logstash-codec-netfl… --- ## [Logstash output file has additional fields](https://discuss.elastic.co/t/logstash-output-file-has-additional-fields/49892)
**Author:** [@vijay\_dhaks](https://discuss.elastic.co/u/vijay_dhaks)\ **Replies:** 3\ **Last updated:** [May 12, 2016, 6:13pm UTC](https://discuss.elastic.co/t/logstash-output-file-has-additional-fields/49892 "2016-05-12T18:13:10Z")
Hi, My config is as below filter { grok { match =\> { "message" =\> "%{DATE\_EU:mydate}%{SPACE}%{HOUR:myhour}.%{MINUTE:myminute}.%{SECOND:mysecond}%{SPACE}%{GREEDYDATA:realdata}"} } and my input is as be… --- ## [Not to add 'host', 'path' field](https://discuss.elastic.co/t/not-to-add-host-path-field/49889)
**Author:** [@no\_jihun](https://discuss.elastic.co/u/no_jihun)\ **Replies:** 4\ **Last updated:** [May 12, 2016, 2:52pm UTC](https://discuss.elastic.co/t/not-to-add-host-path-field/49889 "2016-05-12T14:52:21Z")
Hi. With this configuration, input { file { path =\> ... } } output { elasticsearch { ... } } logstash add files 'host' and 'path' to the document. "\_source": { ... "p… --- ## [GetToday's inside Logstash Conf](https://discuss.elastic.co/t/gettodays-inside-logstash-conf/49849)
**Author:** [@Sameer\_Panicker](https://discuss.elastic.co/u/Sameer_Panicker)\ **Replies:** 4\ **Last updated:** [May 12, 2016, 1:05pm UTC](https://discuss.elastic.co/t/gettodays-inside-logstash-conf/49849 "2016-05-12T13:05:12Z")
I am trying to invoke a service using HTTP\_Poller. The service accepts 2 parameters. 1. Today's Date 2. Today's Date - 3 Months Can anyone tell me how can I achieve this ? --- ## [Logstash persistent queueing](https://discuss.elastic.co/t/logstash-persistent-queueing/49669)
**Author:** [@mnhan](https://discuss.elastic.co/u/mnhan)\ **Replies:** 2\ **Last updated:** [May 12, 2016, 12:56pm UTC](https://discuss.elastic.co/t/logstash-persistent-queueing/49669 "2016-05-12T12:56:44Z")
Hi, I know that there is a persistent queueing for logstash in the works (https://github.com/elastic/logstash/issues/2605). Its marked as roadmap 5.1, Is there plan for this to appear before 5.1? Is there an esti… --- ## [ERR SSL client failed to connect with:](https://discuss.elastic.co/t/err-ssl-client-failed-to-connect-with/49320)
**Author:** [@dickepa](https://discuss.elastic.co/u/dickepa)\ **Replies:** 11\ **Last updated:** [May 11, 2016, 10:50am UTC](https://discuss.elastic.co/t/err-ssl-client-failed-to-connect-with/49320 "2016-05-11T10:50:25Z")
I think this is a Logstash config issue hence posting in this category since I have multiple nodes reporting same message. When starting filebeat on a node I get the above message ending with ":5044: getsockopt: connec… --- ## [Cannot parse jboss server log](https://discuss.elastic.co/t/cannot-parse-jboss-server-log/49572)
**Author:** [@dlopez](https://discuss.elastic.co/u/dlopez)\ **Replies:** 3\ **Last updated:** [May 12, 2016, 8:02am UTC](https://discuss.elastic.co/t/cannot-parse-jboss-server-log/49572 "2016-05-12T08:02:33Z")
Hi, I'm newbie on logstash, I've successfully installed and configured ElasticSearch, LogStash and Kibana as well as my first input using the Remote Log4J connector. But now I'm trying to parse the server.log file wi… --- ## [Logstash data](https://discuss.elastic.co/t/logstash-data/49765)
**Author:** [@Karl\_Trasschaert](https://discuss.elastic.co/u/Karl_Trasschaert)\ **Replies:** 3\ **Last updated:** [May 12, 2016, 7:42am UTC](https://discuss.elastic.co/t/logstash-data/49765 "2016-05-12T07:42:12Z")
Hi, I will use a already set up elk stack to send data to elastic cloud. I saw that the data storage of the server hsting the ELK is already full ( i know it's a smaal drive) / is using 99.5% of 25.23GB Where are … --- ## [Grok Pattern for optional XML tags](https://discuss.elastic.co/t/grok-pattern-for-optional-xml-tags/49667)
**Author:** [@stevedearl](https://discuss.elastic.co/u/stevedearl)\ **Replies:** 2\ **Last updated:** [May 12, 2016, 7:12am UTC](https://discuss.elastic.co/t/grok-pattern-for-optional-xml-tags/49667 "2016-05-12T07:12:06Z")
Hi All, Apologies if this is a dumb and/or basic question. I had a search through the forums but didn't find anything that quite gave me the help I was after. I need to use a grok filter to parse events with XML payl… --- ## [Multiline codec and JDBC](https://discuss.elastic.co/t/multiline-codec-and-jdbc/49727)
**Author:** [@berguiz1](https://discuss.elastic.co/u/berguiz1)\ **Replies:** 2\ **Last updated:** [May 12, 2016, 7:03am UTC](https://discuss.elastic.co/t/multiline-codec-and-jdbc/49727 "2016-05-12T07:03:46Z")
Hello everybody, I'm trying to use the multiline codec with the ids of rows from a table (Database). i've an input plugin with JDBC which give me some rows with same id (because of the SQL query). I would like to process… --- ## [Extract and add field](https://discuss.elastic.co/t/extract-and-add-field/49617)
**Author:** [@HugoBang](https://discuss.elastic.co/u/HugoBang)\ **Replies:** 7\ **Last updated:** [May 12, 2016, 5:50am UTC](https://discuss.elastic.co/t/extract-and-add-field/49617 "2016-05-12T05:50:14Z")
Hello, How can I extract a value from message field and add field with the extracted value? For example, message field is as below. "message": "This is a test.\\n\\nSubject:\\n\\tSecurity ID:\\t\\tT-E-S-T\\n\\tAccount Name… --- ## [Sychronize Index with Database in Logstash](https://discuss.elastic.co/t/sychronize-index-with-database-in-logstash/49709)
**Author:** [@echua](https://discuss.elastic.co/u/echua)\ **Replies:** 1\ **Last updated:** [May 12, 2016, 5:45am UTC](https://discuss.elastic.co/t/sychronize-index-with-database-in-logstash/49709 "2016-05-12T05:45:45Z")
I am trying to read from my database periodically on a schedule. Every 5 minutes. I noticed that each time I read from the database the index doubles in size. I only want changes in the database to be transferred to my… --- ## [File plugin not reading files as expected](https://discuss.elastic.co/t/file-plugin-not-reading-files-as-expected/49844)
**Author:** [@null](https://discuss.elastic.co/u/null)\ **Replies:** 3\ **Last updated:** [May 12, 2016, 5:28am UTC](https://discuss.elastic.co/t/file-plugin-not-reading-files-as-expected/49844 "2016-05-12T05:28:44Z")
When set 'sincedb\_path' to '/dev/null' and 'start\_position' to 'beginning', logstash does not parse files as expected, it always start and hangs at output below: Settings: Default pipeline workers: 4 Pipeline main start… --- ## [Logstash Input UDP problem](https://discuss.elastic.co/t/logstash-input-udp-problem/49718)
**Author:** [@Yim\_JunHyeok](https://discuss.elastic.co/u/Yim_JunHyeok)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 11:57pm UTC](https://discuss.elastic.co/t/logstash-input-udp-problem/49718 "2016-05-11T23:57:58Z")
First of all sorry for my bad english. I'm doing logstash input UDP performance test. Unfortunately every time when i proceed logstash, i'm facing data loss. not sure where i am going wrong. I Followed below steps. 1… --- ## [Intermediate logstash shut down stops indexing after the restart](https://discuss.elastic.co/t/intermediate-logstash-shut-down-stops-indexing-after-the-restart/49712)
**Author:** [@subbu.nv](https://discuss.elastic.co/u/subbu.nv)\ **Replies:** 1\ **Last updated:** [May 11, 2016, 11:56pm UTC](https://discuss.elastic.co/t/intermediate-logstash-shut-down-stops-indexing-after-the-restart/49712 "2016-05-11T23:56:20Z")
I was indexing 110GB of data to elastic search using logstash. after it ran for few hours, i realised , i need to add another conf file for a different indexing. so i decided to add the same and restart logsatsh. when i … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1153) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1155)