# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1244 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1245 --- ## [Elasticsearch plugin - is there a way to keep it forever listening](https://discuss.elastic.co/t/elasticsearch-plugin-is-there-a-way-to-keep-it-forever-listening/27484)
**Author:** [@Cjtow](https://discuss.elastic.co/u/Cjtow)\ **Replies:** 2\ **Last updated:** [August 17, 2015, 11:15am UTC](https://discuss.elastic.co/t/elasticsearch-plugin-is-there-a-way-to-keep-it-forever-listening/27484 "2015-08-17T11:15:01Z")
Good morning, At the moment, I have a query and it runs against the Logstash-\* index but once it's done reading from today's date it closes the connection. Is there a way to keep it always running so it will pick up an… --- ## [ELK Product Inquiry!](https://discuss.elastic.co/t/elk-product-inquiry/27486)
**Author:** [@sreeram](https://discuss.elastic.co/u/sreeram)\ **Replies:** 5\ **Last updated:** [August 17, 2015, 10:19am UTC](https://discuss.elastic.co/t/elk-product-inquiry/27486 "2015-08-17T10:19:12Z")
Hi, I am writing to inquire about ELK for centralized logging implementation in production environment. We have tested ELK in development environment and We're very much interested in implementing ELK in productio… --- ## [Logstash local install plugin](https://discuss.elastic.co/t/logstash-local-install-plugin/27443)
**Author:** [@Marcello\_A](https://discuss.elastic.co/u/Marcello_A)\ **Replies:** 1\ **Last updated:** [August 17, 2015, 6:17am UTC](https://discuss.elastic.co/t/logstash-local-install-plugin/27443 "2015-08-17T06:17:45Z")
Hi All, i would test some extra plugin on logstash 1.5.3, but i have to install this on a server without internet connection. Are there some ways to use the pugin command with the local version of the plugin downloaded o… --- ## [How to create index automatically by logstash conf](https://discuss.elastic.co/t/how-to-create-index-automatically-by-logstash-conf/27468)
**Author:** [@Jason\_Zheng](https://discuss.elastic.co/u/Jason_Zheng)\ **Replies:** 5\ **Last updated:** [August 17, 2015, 5:18am UTC](https://discuss.elastic.co/t/how-to-create-index-automatically-by-logstash-conf/27468 "2015-08-17T05:18:04Z")
Hi all, when I call $curl -XPUT -u es\_admin:iiiiii 'http://localhost:9200/franky/json/1' -d '{ "ismb.miki.host":"192.168.10.53" }' the result is {"\_index":"franky","\_type":"json","\_id":"1","\_version":2,"created":… --- ## [Mixed log with timestamp and json data](https://discuss.elastic.co/t/mixed-log-with-timestamp-and-json-data/27456)
**Author:** [@muhammadn](https://discuss.elastic.co/u/muhammadn)\ **Replies:** 1\ **Last updated:** [August 16, 2015, 11:14pm UTC](https://discuss.elastic.co/t/mixed-log-with-timestamp-and-json-data/27456 "2015-08-16T23:14:02Z")
I also have a mixed json data with the timestamp first which i followed with the following code: \`filter{ grok{ match =\> \[ "message", "%{DATESTAMP} %{GREEDYDATA:message}" \] overwrite =\> \["message"\] } … --- ## [Mixed log with json field](https://discuss.elastic.co/t/mixed-log-with-json-field/26688)
**Author:** [@Or\_Cohen](https://discuss.elastic.co/u/Or_Cohen)\ **Replies:** 8\ **Last updated:** [August 16, 2015, 3:12pm UTC](https://discuss.elastic.co/t/mixed-log-with-json-field/26688 "2015-08-16T15:12:23Z")
Hi, I have a log with mixed fields the first two fields are outsude the json the third one is a json Example: tag : 02:00:00 --\> {"v":"2.59","tagid":"2edfe14540b51a047dc8f66c219c7804","cb":"1665253253","ds":"loading"… --- ## [File input processing question](https://discuss.elastic.co/t/file-input-processing-question/27442)
**Author:** [@jjdepaul](https://discuss.elastic.co/u/jjdepaul)\ **Replies:** 2\ **Last updated:** [August 16, 2015, 2:33pm UTC](https://discuss.elastic.co/t/file-input-processing-question/27442 "2015-08-16T14:33:46Z")
I have logstash config file setup as shown below. It is monitoring demo directory for specific file pattern. It finds the log file and reads it, but once it reads it I can't ever convince it to re-read it from the be… --- ## [I/O exception (org.apache.http.conn.HttpHostConnectException)](https://discuss.elastic.co/t/i-o-exception-org-apache-http-conn-httphostconnectexception/27394)
**Author:** [@Ashok](https://discuss.elastic.co/u/Ashok)\ **Replies:** 1\ **Last updated:** [August 15, 2015, 10:44pm UTC](https://discuss.elastic.co/t/i-o-exception-org-apache-http-conn-httphostconnectexception/27394 "2015-08-15T22:44:33Z")
Hi, I am getting following error while executing logstash conf file. And i want to connect the server with protocal https to monitor the IBM wepsphere logs.But i don't have any option in protocol for https. org.apache.… --- ## [Can logstash configuration have two different inputs with the same port?](https://discuss.elastic.co/t/can-logstash-configuration-have-two-different-inputs-with-the-same-port/27406)
**Author:** [@Diman\_Ghazi](https://discuss.elastic.co/u/Diman_Ghazi)\ **Replies:** 1\ **Last updated:** [August 15, 2015, 7:00am UTC](https://discuss.elastic.co/t/can-logstash-configuration-have-two-different-inputs-with-the-same-port/27406 "2015-08-15T07:00:08Z")
Hi, I am fairly new in Logstash. I am trying to have two inputs with the same port but different type and tag. e.g: input { tcp { port =\> 5000 type =\> "x" tags =\> \["x"\] } log4j { … --- ## [Logstash jdbc input for multi fields and nested objects](https://discuss.elastic.co/t/logstash-jdbc-input-for-multi-fields-and-nested-objects/27020)
**Author:** [@vijay116](https://discuss.elastic.co/u/vijay116)\ **Replies:** 1\ **Last updated:** [August 14, 2015, 8:04pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-for-multi-fields-and-nested-objects/27020 "2015-08-14T20:04:24Z")
Does the plugin support both cases? I had issue with both. For examples with the mapping { "consumer": { "properties": { "name": { "type": "string", "fields": { "raw": { "type": "string", … --- ## [Elasticsearch output plugin fails with "undefined method \`version' for nil:NilClass"](https://discuss.elastic.co/t/elasticsearch-output-plugin-fails-with-undefined-method-version-for-nil-nilclass/27409)
**Author:** [@Sven\_Beauprez](https://discuss.elastic.co/u/Sven_Beauprez)\ **Replies:** 0\ **Last updated:** [August 14, 2015, 3:31pm UTC](https://discuss.elastic.co/t/elasticsearch-output-plugin-fails-with-undefined-method-version-for-nil-nilclass/27409 "2015-08-14T15:31:47Z")
I want to work and extend functionality of the elasticsearch plugin, but I have an issue running the original plugin. After succesfully building the gem file for the master branch of logstash-output-elasticsearch (outp… --- ## [Does the useragent filter produce two apparently identical os fields?](https://discuss.elastic.co/t/does-the-useragent-filter-produce-two-apparently-identical-os-fields/27270)
**Author:** [@LansK](https://discuss.elastic.co/u/LansK)\ **Replies:** 2\ **Last updated:** [August 14, 2015, 1:19pm UTC](https://discuss.elastic.co/t/does-the-useragent-filter-produce-two-apparently-identical-os-fields/27270 "2015-08-14T13:19:56Z")
The useragent filter (https://www.elastic.co/guide/en/logstash/current/plugins-filters-useragent.html), when applied to the user agent that is produced by apache access logs (through the built in COMBINEDAPACHELOG patter… --- ## [Is it possible to create configuration snippet for the logstash-forwarder in a directory?](https://discuss.elastic.co/t/is-it-possible-to-create-configuration-snippet-for-the-logstash-forwarder-in-a-directory/27309)
**Author:** [@michaelzoet](https://discuss.elastic.co/u/michaelzoet)\ **Replies:** 3\ **Last updated:** [August 14, 2015, 1:04pm UTC](https://discuss.elastic.co/t/is-it-possible-to-create-configuration-snippet-for-the-logstash-forwarder-in-a-directory/27309 "2015-08-14T13:04:33Z")
Hello all, I want/need to create logstash-forwarder configuration snippets based on which Puppet profile a server uses. For example a server that runs Apache and PostgreSQL should have entries in /etc/logstash-forward… --- ## [How do I tell logstash-forwarder to finish processing a log before rotation?](https://discuss.elastic.co/t/how-do-i-tell-logstash-forwarder-to-finish-processing-a-log-before-rotation/27382)
**Author:** [@chmd](https://discuss.elastic.co/u/chmd)\ **Replies:** 0\ **Last updated:** [August 14, 2015, 9:38am UTC](https://discuss.elastic.co/t/how-do-i-tell-logstash-forwarder-to-finish-processing-a-log-before-rotation/27382 "2015-08-14T09:38:47Z")
Hi, I am using logstash-forwarder to forward the content of a file. This file grows fast, and I need to rotate it often. To give you an idea, I trigger logrotate every 5 minutes. The condition for the rotation to actua… --- ## [Elasticsearch Input Plugin - Index field string to match system date](https://discuss.elastic.co/t/elasticsearch-input-plugin-index-field-string-to-match-system-date/27304)
**Author:** [@Cjtow](https://discuss.elastic.co/u/Cjtow)\ **Replies:** 3\ **Last updated:** [August 13, 2015, 10:24am UTC](https://discuss.elastic.co/t/elasticsearch-input-plugin-index-field-string-to-match-system-date/27304 "2015-08-13T10:24:51Z")
Good morning/afternoon/evening/hi, using logstash version 1.5.3 Sorry if this has been bought up before, I have been searching for a while and can't find anything. Life story, you don't need to read I've been working… --- ## [How to monitor logstash itself?](https://discuss.elastic.co/t/how-to-monitor-logstash-itself/27172)
**Author:** [@MichaelScofield](https://discuss.elastic.co/u/MichaelScofield)\ **Replies:** 11\ **Last updated:** [August 14, 2015, 7:44am UTC](https://discuss.elastic.co/t/how-to-monitor-logstash-itself/27172 "2015-08-14T07:44:45Z")
Are there any tools that can monitor logstash's health? I googled it but found no answers. --- ## [Configure stderr logs](https://discuss.elastic.co/t/configure-stderr-logs/27361)
**Author:** [@NehaSood](https://discuss.elastic.co/u/NehaSood)\ **Replies:** 1\ **Last updated:** [August 14, 2015, 6:02am UTC](https://discuss.elastic.co/t/configure-stderr-logs/27361 "2015-08-14T06:02:27Z")
Hi, I've logstash running on AWS EC2 instances, which I am using to push data to some ElasticSearch instances. Occasionally, I notice a few messages getting lost & not being published to ElasticSearch. I am using a ver… --- ## [Multiple ssl certificates in logstash-forwarder](https://discuss.elastic.co/t/multiple-ssl-certificates-in-logstash-forwarder/27337)
**Author:** [@iamrudra](https://discuss.elastic.co/u/iamrudra)\ **Replies:** 1\ **Last updated:** [August 13, 2015, 9:34pm UTC](https://discuss.elastic.co/t/multiple-ssl-certificates-in-logstash-forwarder/27337 "2015-08-13T21:34:06Z")
The basic/example config shipped with logstash-forwarder states the following for listing multiple downstream servers: ... #A list of downstream servers listening for our messages. #logstash-forwarder will pick one a… --- ## [Logstash Date filter default timezone](https://discuss.elastic.co/t/logstash-date-filter-default-timezone/27293)
**Author:** [@vilas](https://discuss.elastic.co/u/vilas)\ **Replies:** 4\ **Last updated:** [August 13, 2015, 8:52pm UTC](https://discuss.elastic.co/t/logstash-date-filter-default-timezone/27293 "2015-08-13T20:52:03Z")
Hi, Following is the timezone setting documentation for Logstash Date filter. What is meant by platform default in the description? Does it mean the timezone of the server on which the Logstash is running? Timezone … --- ## [Logstash Output Plugin Batch Events / De-Register Method](https://discuss.elastic.co/t/logstash-output-plugin-batch-events-de-register-method/27291)
**Author:** [@rishi](https://discuss.elastic.co/u/rishi)\ **Replies:** 4\ **Last updated:** [August 13, 2015, 8:35pm UTC](https://discuss.elastic.co/t/logstash-output-plugin-batch-events-de-register-method/27291 "2015-08-13T20:35:23Z")
I'm working on an output plugin for logstash and was wondering if the events that the ruby script receives can be batched (multiple events in place of the receive(event) method that gives a single event)? --- ## [Best way to change a list of not wanted values](https://discuss.elastic.co/t/best-way-to-change-a-list-of-not-wanted-values/27329)
**Author:** [@jclose](https://discuss.elastic.co/u/jclose)\ **Replies:** 3\ **Last updated:** [August 13, 2015, 5:57pm UTC](https://discuss.elastic.co/t/best-way-to-change-a-list-of-not-wanted-values/27329 "2015-08-13T17:57:21Z")
So I have some values being fed from logstash, and they are failing input into Elastic because I have a template being used, and the values sometimes don't adhere to the tablet. Notably, I get the values "-" and "(empt… --- ## [Java 7 upgrade to 8 problem with Logstash](https://discuss.elastic.co/t/java-7-upgrade-to-8-problem-with-logstash/27246)
**Author:** [@AymenChetoui](https://discuss.elastic.co/u/AymenChetoui)\ **Replies:** 2\ **Last updated:** [August 13, 2015, 11:59am UTC](https://discuss.elastic.co/t/java-7-upgrade-to-8-problem-with-logstash/27246 "2015-08-13T11:59:05Z")
Hi all, In my configuration, I have Logstash getting Syslog authentication in an EC2 instance (Instance 1) , and sending the to Elasticsearch/Kibana in a second EC2 instance (Instance 2). When Java 7 installed on (… --- ## [Basic Question : Can i convert unstructured Data into structured data for my analytical using logstash elasticsearch](https://discuss.elastic.co/t/basic-question-can-i-convert-unstructured-data-into-structured-data-for-my-analytical-using-logstash-elasticsearch/27306)
**Author:** [@rkhapre](https://discuss.elastic.co/u/rkhapre)\ **Replies:** 1\ **Last updated:** [August 13, 2015, 8:29am UTC](https://discuss.elastic.co/t/basic-question-can-i-convert-unstructured-data-into-structured-data-for-my-analytical-using-logstash-elasticsearch/27306 "2015-08-13T08:29:23Z")
Hi All I have a very basic question, i have a column in my mysql database which hold records like this below Cashflow 250 rate 12% year 2 NPV -34 Like this i have data jumbled in whole column which is like a tex… --- ## [Logstash with metrics plugin does not replace fieldnames like %{my\_field}](https://discuss.elastic.co/t/logstash-with-metrics-plugin-does-not-replace-fieldnames-like-my-field/27267)
**Author:** [@m1k3ga](https://discuss.elastic.co/u/m1k3ga)\ **Replies:** 0\ **Last updated:** [August 12, 2015, 2:53pm UTC](https://discuss.elastic.co/t/logstash-with-metrics-plugin-does-not-replace-fieldnames-like-my-field/27267 "2015-08-12T14:53:15Z")
Hi there, i want include field values into my metrics record. Therefore i used add\_field and add\_tag with my field "%{my\_field}". In stdout and Elasticsearch the content of \`add\_field\` is "New field with value: %{my\_fie… --- ## [Redis filter/mutate for LogStash](https://discuss.elastic.co/t/redis-filter-mutate-for-logstash/27285)
**Author:** [@Sesha\_Narahari](https://discuss.elastic.co/u/Sesha_Narahari)\ **Replies:** 5\ **Last updated:** [August 13, 2015, 2:19am UTC](https://discuss.elastic.co/t/redis-filter-mutate-for-logstash/27285 "2015-08-13T02:19:28Z")
I would like to use Redis as a cache, look up key:value data to enrich messages in a mutate filter. An example would be IP:MAC association. The pipeline looks like: Input from Redis -\> Filter/Mutate with regex/grok, en… --- ## [Best way to delete ElasticSearch document using Logstash configuration](https://discuss.elastic.co/t/best-way-to-delete-elasticsearch-document-using-logstash-configuration/27263)
**Author:** [@umangjain](https://discuss.elastic.co/u/umangjain)\ **Replies:** 1\ **Last updated:** [August 12, 2015, 9:42pm UTC](https://discuss.elastic.co/t/best-way-to-delete-elasticsearch-document-using-logstash-configuration/27263 "2015-08-12T21:42:01Z")
Please let me know the best practice to delete a record from ES using logstash configuration. I am currently using this, but it doesn't seem to work fine: elasticsearch { docum… --- ## [Multiple "elasticsearch" output plugins for single logstash instance](https://discuss.elastic.co/t/multiple-elasticsearch-output-plugins-for-single-logstash-instance/27275)
**Author:** [@iamrudra](https://discuss.elastic.co/u/iamrudra)\ **Replies:** 3\ **Last updated:** [August 12, 2015, 8:35pm UTC](https://discuss.elastic.co/t/multiple-elasticsearch-output-plugins-for-single-logstash-instance/27275 "2015-08-12T20:35:18Z")
I am trying to use one logstash instance to send data to multiple elasticsearch clusters. My lumberjack output config looks like output { elasticsearch { host =\> "localhost" cluster =\> "my-es-cluster" } el… --- ## [Logstash 1.5.3 issue with elasticsearch 1.6.1/1.7.0 "undefined method"](https://discuss.elastic.co/t/logstash-1-5-3-issue-with-elasticsearch-1-6-1-1-7-0-undefined-method/26566)
**Author:** [@m1k3ga](https://discuss.elastic.co/u/m1k3ga)\ **Replies:** 5\ **Last updated:** [August 12, 2015, 3:14pm UTC](https://discuss.elastic.co/t/logstash-1-5-3-issue-with-elasticsearch-1-6-1-1-7-0-undefined-method/26566 "2015-08-12T15:14:50Z")
hi there, yesterday we tried to update our production system. we have N logstash instances (as shipper) to a few redis instances. Then there are a few logstash indexer pulling from redis and putting them into the elast… --- ## [Performances degradation when using type field in inputs](https://discuss.elastic.co/t/performances-degradation-when-using-type-field-in-inputs/27111)
**Author:** [@Alessio\_Martorelli](https://discuss.elastic.co/u/Alessio_Martorelli)\ **Replies:** 8\ **Last updated:** [August 12, 2015, 1:36pm UTC](https://discuss.elastic.co/t/performances-degradation-when-using-type-field-in-inputs/27111 "2015-08-12T13:36:45Z")
Hi all, I've been trying to troubleshoot few latency issues that I have with Logstash. I'm using Redis as input source and for each input file I decided to specify a type. What I've found is that when rotating the index,… --- ## [ZeroMQ connectivity-check](https://discuss.elastic.co/t/zeromq-connectivity-check/27259)
**Author:** [@Eru](https://discuss.elastic.co/u/Eru)\ **Replies:** 0\ **Last updated:** [August 12, 2015, 1:29pm UTC](https://discuss.elastic.co/t/zeromq-connectivity-check/27259 "2015-08-12T13:29:36Z")
Hi, We are using ELK in production, and some apps are shipping to logtash via ZeroQM and files.. Occasionally our servers go offline, and sometimes logstash can't boot-up properly. We have a fallback that ships the log… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1243) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1245)