# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=954 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 955 --- ## [Why "?" character appearance in output message via filebeat input?](https://discuss.elastic.co/t/why-character-appearance-in-output-message-via-filebeat-input/91260)
**Author:** [@sonnguyenit](https://discuss.elastic.co/u/sonnguyenit)\ **Replies:** 4\ **Last updated:** [June 30, 2017, 7:30am UTC](https://discuss.elastic.co/t/why-character-appearance-in-output-message-via-filebeat-input/91260 "2017-06-30T07:30:39Z")
I have log files with lines in xml format like that: \\\http… --- ## [Upgraded to Logstash 5, bulk queue started filling up?](https://discuss.elastic.co/t/upgraded-to-logstash-5-bulk-queue-started-filling-up/88288)
**Author:** [@codyaray](https://discuss.elastic.co/u/codyaray)\ **Replies:** 12\ **Last updated:** [June 30, 2017, 3:48am UTC](https://discuss.elastic.co/t/upgraded-to-logstash-5-bulk-queue-started-filling-up/88288 "2017-06-30T03:48:37Z")
Hi all, We just upgraded to Logstash 5.4 from Logstash 1.5.6. (Yes, we were way behind.) We're using the elasticsearch output to write to our existing ES 2.4.0 cluster. After upgrading, we're receiving a bunch of 429s … --- ## [Unable to delete documents from index when using logstash output plugin](https://discuss.elastic.co/t/unable-to-delete-documents-from-index-when-using-logstash-output-plugin/91303)
**Author:** [@truesecure](https://discuss.elastic.co/u/truesecure)\ **Replies:** 2\ **Last updated:** [June 30, 2017, 2:33am UTC](https://discuss.elastic.co/t/unable-to-delete-documents-from-index-when-using-logstash-output-plugin/91303 "2017-06-30T02:33:40Z")
My logstash plugin isn't deleting the docs in the index. It appears to be ignoring the document\_id it is being passed, and instead just uses the variable name (as opposed to the value). I have the following configurati… --- ## [Running the ELK stack on windows](https://discuss.elastic.co/t/running-the-elk-stack-on-windows/91202)
**Author:** [@CDR](https://discuss.elastic.co/u/CDR)\ **Replies:** 3\ **Last updated:** [June 29, 2017, 10:35pm UTC](https://discuss.elastic.co/t/running-the-elk-stack-on-windows/91202 "2017-06-29T22:35:02Z")
Hello. I am currently running the ELK stack through the batch files on a Windows machine. I noticed when I close the batch files and then reopen the services the data remains in elasticsearch. I figured out that to delet… --- ## [Logstash not processing JSON files present on S3](https://discuss.elastic.co/t/logstash-not-processing-json-files-present-on-s3/89640)
**Author:** [@sidharth1985](https://discuss.elastic.co/u/sidharth1985)\ **Replies:** 3\ **Last updated:** [June 29, 2017, 8:16pm UTC](https://discuss.elastic.co/t/logstash-not-processing-json-files-present-on-s3/89640 "2017-06-29T20:16:41Z")
I am trying to read JSON files from my S3 bucket through logstash and output them on elasticssearch. Both logstash and elastic search are running on my local. I do know logstash is able to connect to S3 by looking at l… --- ## [Two if conditions in one line](https://discuss.elastic.co/t/two-if-conditions-in-one-line/91314)
**Author:** [@pmadhavan1](https://discuss.elastic.co/u/pmadhavan1)\ **Replies:** 2\ **Last updated:** [June 29, 2017, 6:30pm UTC](https://discuss.elastic.co/t/two-if-conditions-in-one-line/91314 "2017-06-29T18:30:19Z")
Hi, I want to add an if condition for two timestamps. I want to say that if either of them don't match then the message should be dropped. if !\[log\_timestamp\] { # drop {} # } # if !(\[log\_timestamp2\]… --- ## [Will older logstash forward to newer elasticsearch?](https://discuss.elastic.co/t/will-older-logstash-forward-to-newer-elasticsearch/91328)
**Author:** [@AmazingFERGATRON](https://discuss.elastic.co/u/AmazingFERGATRON)\ **Replies:** 1\ **Last updated:** [June 29, 2017, 6:12pm UTC](https://discuss.elastic.co/t/will-older-logstash-forward-to-newer-elasticsearch/91328 "2017-06-29T18:12:52Z")
I'm a newbie here, so apologies if I breach etiquette or protocol. I did search this category and Logstash-forwarder for a similar topic and didn't find one, so apologies again if this question has been asked before... … --- ## [Exception on parsing user agent with 5.4.2](https://discuss.elastic.co/t/exception-on-parsing-user-agent-with-5-4-2/90576)
**Author:** [@Joelp](https://discuss.elastic.co/u/Joelp)\ **Replies:** 7\ **Last updated:** [June 29, 2017, 5:59pm UTC](https://discuss.elastic.co/t/exception-on-parsing-user-agent-with-5-4-2/90576 "2017-06-29T17:59:52Z")
after update logstash to 5.4.2, I have an issue with parsing and performance of course: \[2017-06-22T09:46:42,039\]\[ERROR\]\[logstash.filters.useragent\] Uknown error while parsing user agent data {:exception=\>java.lang.Str… --- ## [Offline beats plugin install not completing - oversubscribed literal/length tree (Zlib::DataError)](https://discuss.elastic.co/t/offline-beats-plugin-install-not-completing-oversubscribed-literal-length-tree-zlib-dataerror/91316)
**Author:** [@gitsideways](https://discuss.elastic.co/u/gitsideways)\ **Replies:** 1\ **Last updated:** [June 29, 2017, 5:48pm UTC](https://discuss.elastic.co/t/offline-beats-plugin-install-not-completing-oversubscribed-literal-length-tree-zlib-dataerror/91316 "2017-06-29T17:48:33Z")
Hi all, I have a logstash server that needs an offline beats plugin install, but I get the following error when running the install: C:\\Elastic\\logstash-5.4.0\\bin\>logstash-plugin install file:///c:/logstash-offline-p… --- ## [Logstash S3 input support for folders in bucket](https://discuss.elastic.co/t/logstash-s3-input-support-for-folders-in-bucket/91325)
**Author:** [@infamous](https://discuss.elastic.co/u/infamous)\ **Replies:** 0\ **Last updated:** [June 29, 2017, 5:40pm UTC](https://discuss.elastic.co/t/logstash-s3-input-support-for-folders-in-bucket/91325 "2017-06-29T17:40:13Z")
Hi all, Does the S3 input allow for file pathing in buckets? Currently I am only able to extract files that are sitting directly in the bucket, but ideally I want to be able to navigate through folders to access certai… --- ## [Gsub can't modify "type" : "ip" to fix string literal error entries for non-ip address info](https://discuss.elastic.co/t/gsub-cant-modify-type-ip-to-fix-string-literal-error-entries-for-non-ip-address-info/91283)
**Author:** [@danno](https://discuss.elastic.co/u/danno)\ **Replies:** 12\ **Last updated:** [June 29, 2017, 3:39pm UTC](https://discuss.elastic.co/t/gsub-cant-modify-type-ip-to-fix-string-literal-error-entries-for-non-ip-address-info/91283 "2017-06-29T15:39:11Z")
I set field mappings for IP address fields to { "type" : "ip" } and things are great until a logging device (Fortigate) doesn't have an IP address and sends N/A. "error"=\>{"type"=\>"mapper\_parsing\_exception" "reason"=\>"… --- ## [Aggregate filter check value with all previous records](https://discuss.elastic.co/t/aggregate-filter-check-value-with-all-previous-records/91311)
**Author:** [@Tejas\_Ghutukade](https://discuss.elastic.co/u/Tejas_Ghutukade)\ **Replies:** 0\ **Last updated:** [June 29, 2017, 3:15pm UTC](https://discuss.elastic.co/t/aggregate-filter-check-value-with-all-previous-records/91311 "2017-06-29T15:15:55Z")
while reading the csv file in logstash i want to compare value with all the previous record and depending on the condition set a field with true or false. m able to compare with the immediate previous record but i want … --- ## [Logstash Instal CIDR Plugin failed](https://discuss.elastic.co/t/logstash-instal-cidr-plugin-failed/89058)
**Author:** [@jtool](https://discuss.elastic.co/u/jtool)\ **Replies:** 1\ **Last updated:** [June 29, 2017, 1:55pm UTC](https://discuss.elastic.co/t/logstash-instal-cidr-plugin-failed/89058 "2017-06-29T13:55:47Z")
I am a bit confused. I'm getting a production cluster ready and tried to install the CIDR plugin, but the install returned an error saying "Installation aborted, verification failed for logstash-filter-cidr". When I ra… --- ## [Convert type to timpestamp](https://discuss.elastic.co/t/convert-type-to-timpestamp/91289)
**Author:** [@mohammed\_cherif](https://discuss.elastic.co/u/mohammed_cherif)\ **Replies:** 0\ **Last updated:** [June 29, 2017, 1:41pm UTC](https://discuss.elastic.co/t/convert-type-to-timpestamp/91289 "2017-06-29T13:41:18Z")
i want to index data from table, i use this config : input { jdbc { jdbc\_connection\_string =\> "jdbc:oracle:thin:@xxx:1512/yyy" jdbc\_user =\> "test" jdbc\_password =\> "test" jdbc\_valida… --- ## [Overwrite @timestamp field](https://discuss.elastic.co/t/overwrite-timestamp-field/91280)
**Author:** [@2knarf](https://discuss.elastic.co/u/2knarf)\ **Replies:** 1\ **Last updated:** [June 29, 2017, 1:36pm UTC](https://discuss.elastic.co/t/overwrite-timestamp-field/91280 "2017-06-29T13:36:02Z")
Hello, I'm trying to overwrite the timestamp field with incoming log messages' timestamp, However I am unable to get it working, I have tried searching the forums but I do not find anything that solves my case, attaching… --- ## [Read multiple log](https://discuss.elastic.co/t/read-multiple-log/90124)
**Author:** [@mohammed\_cherif](https://discuss.elastic.co/u/mohammed_cherif)\ **Replies:** 2\ **Last updated:** [June 29, 2017, 1:31pm UTC](https://discuss.elastic.co/t/read-multiple-log/90124 "2017-06-29T13:31:30Z")
i want index multiple log with one input but logstash not read files my config : input { file { path =\> "C:/data1/log/\*.log" start\_position =\> "beginning" sincedb\_path =\> "/dev/… --- ## [Logstash mutate - convert empty string to null](https://discuss.elastic.co/t/logstash-mutate-convert-empty-string-to-null/91263)
**Author:** [@selivan](https://discuss.elastic.co/u/selivan)\ **Replies:** 3\ **Last updated:** [June 29, 2017, 11:04am UTC](https://discuss.elastic.co/t/logstash-mutate-convert-empty-string-to-null/91263 "2017-06-29T11:04:27Z")
There is a field in log file, which contains IP address or empty string if IP address is not available. Elasticsearch index has mapping, that maps "ip" type to this field. When field value is empty string, logstash can n… --- ## [SOLR\_HTTP plugin & https](https://discuss.elastic.co/t/solr-http-plugin-https/90953)
**Author:** [@marjan4jc](https://discuss.elastic.co/u/marjan4jc)\ **Replies:** 1\ **Last updated:** [June 29, 2017, 10:58am UTC](https://discuss.elastic.co/t/solr-http-plugin-https/90953 "2017-06-29T10:58:31Z")
Hi, I'm trying to index data in solr but no luck till now. I'm not even sure if solr\_http logstash plugin is capable of handling a https connection? I've putted https url and I can not see any data on solr side. Every… --- ## [Logstash-input-rackspace-3.0.1 plugin for Logstash 5.4](https://discuss.elastic.co/t/logstash-input-rackspace-3-0-1-plugin-for-logstash-5-4/91264)
**Author:** [@dprada](https://discuss.elastic.co/u/dprada)\ **Replies:** 0\ **Last updated:** [June 29, 2017, 10:15am UTC](https://discuss.elastic.co/t/logstash-input-rackspace-3-0-1-plugin-for-logstash-5-4/91264 "2017-06-29T10:15:58Z")
Hi, I am trying to install this plugin running logstash-plugin install --no-verify --local /opt/logstash/plugins/logstash-input-rackspace-3.0.1.gem command, which give me the next output: Installing logstash-input-ra… --- ## [Drupal \`watchdog.type\` field value overrides configured/default type](https://discuss.elastic.co/t/drupal-watchdog-type-field-value-overrides-configured-default-type/91128)
**Author:** [@loopsysdev](https://discuss.elastic.co/u/loopsysdev)\ **Replies:** 0\ **Last updated:** [June 28, 2017, 2:47pm UTC](https://discuss.elastic.co/t/drupal-watchdog-type-field-value-overrides-configured-default-type/91128 "2017-06-28T14:47:29Z")
I've enabled the drupal\_dblog input, and I'm getting entries (documents), but they all have the same type as the entries in /admin/reports/dblog. It seems the type field set by the plugin is overridden by the contents of… --- ## [Installing and using logstash (and other elk components) with plugins on a shared mount](https://discuss.elastic.co/t/installing-and-using-logstash-and-other-elk-components-with-plugins-on-a-shared-mount/90608)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 4\ **Last updated:** [June 29, 2017, 10:20am UTC](https://discuss.elastic.co/t/installing-and-using-logstash-and-other-elk-components-with-plugins-on-a-shared-mount/90608 "2017-06-29T10:20:23Z")
Hi Elastic Team, This question actually applies to not only logstash but actually to all components in ELK stack. Due to some constraints I have, I'm unable to run the installation process with plugin-install command… --- ## [Plugin logstash-output-lumberjack not available for Vers. 5](https://discuss.elastic.co/t/plugin-logstash-output-lumberjack-not-available-for-vers-5/91113)
**Author:** [@lordofthings](https://discuss.elastic.co/u/lordofthings)\ **Replies:** 2\ **Last updated:** [June 29, 2017, 10:08am UTC](https://discuss.elastic.co/t/plugin-logstash-output-lumberjack-not-available-for-vers-5/91113 "2017-06-29T10:08:10Z")
Hi all, The plugin logstash-output-lumberjack is still not available for Logstash 5.x Does anybody now what happen with it and if it will be available near future? This project is still active on github (https://git… --- ## [Logstash does not send apache logs to elasticsearch](https://discuss.elastic.co/t/logstash-does-not-send-apache-logs-to-elasticsearch/90797)
**Author:** [@themoisis](https://discuss.elastic.co/u/themoisis)\ **Replies:** 52\ **Last updated:** [June 29, 2017, 9:45am UTC](https://discuss.elastic.co/t/logstash-does-not-send-apache-logs-to-elasticsearch/90797 "2017-06-29T09:45:24Z")
Here is the logstash.conf input { file { path =\> "/home/test6/admin\_access.log" type =\> "apache-access" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { if \[type\] == "apach… --- ## [Use ruby variable in if filter](https://discuss.elastic.co/t/use-ruby-variable-in-if-filter/91251)
**Author:** [@kitz99](https://discuss.elastic.co/u/kitz99)\ **Replies:** 3\ **Last updated:** [June 29, 2017, 9:42am UTC](https://discuss.elastic.co/t/use-ruby-variable-in-if-filter/91251 "2017-06-29T09:42:05Z")
I have this logstash config: input { stdin { } } output { elasticsearch { hosts =\> \["localhost:9200"\] } stdout { codec =\> rubydebug } } filter { ruby { code =\> " @@exists\_pattern = \['/collaboratio… --- ## [Repeated delimiters with dissect , how to handle?](https://discuss.elastic.co/t/repeated-delimiters-with-dissect-how-to-handle/90888)
**Author:** [@Grigory\_Shamov1](https://discuss.elastic.co/u/Grigory_Shamov1)\ **Replies:** 4\ **Last updated:** [June 29, 2017, 9:09am UTC](https://discuss.elastic.co/t/repeated-delimiters-with-dissect-how-to-handle/90888 "2017-06-29T09:09:56Z")
Hi, Is there a way to change the default behaviour of dissect filter that takes repeated delimiters for single one? It causes problem with empty fields between same delimiters, like for its comma delimited string A… --- ## [Logstash solr\_http plugin not working](https://discuss.elastic.co/t/logstash-solr-http-plugin-not-working/86217)
**Author:** [@Divya\_Prakash](https://discuss.elastic.co/u/Divya_Prakash)\ **Replies:** 3\ **Last updated:** [June 29, 2017, 9:06am UTC](https://discuss.elastic.co/t/logstash-solr-http-plugin-not-working/86217 "2017-06-29T09:06:41Z")
Logstash version 5.4.0 Tried to comment out line 65 in solr\_http as suggested in another post - However, that workaround is not helping with logstash v5.4.0. (https://discuss.elastic.co/t/solr6-logstash-5-timestamp-error… --- ## [Logstash date filter won't parse my date field](https://discuss.elastic.co/t/logstash-date-filter-wont-parse-my-date-field/91146)
**Author:** [@Tomer\_Lev](https://discuss.elastic.co/u/Tomer_Lev)\ **Replies:** 7\ **Last updated:** [June 29, 2017, 9:06am UTC](https://discuss.elastic.co/t/logstash-date-filter-wont-parse-my-date-field/91146 "2017-06-29T09:06:13Z")
Hi :smile: TL;DR logstash won't parse date field; date field parsed as string... I'm trying to parse my date field using logstash date filter but with no success... looks like it is not override the @timestamp field… --- ## [ES: Content type detection for rest requests is deprecated - 5.4.X](https://discuss.elastic.co/t/es-content-type-detection-for-rest-requests-is-deprecated-5-4-x/91253)
**Author:** [@Ruben\_Ortiz](https://discuss.elastic.co/u/Ruben_Ortiz)\ **Replies:** 0\ **Last updated:** [June 29, 2017, 9:02am UTC](https://discuss.elastic.co/t/es-content-type-detection-for-rest-requests-is-deprecated-5-4-x/91253 "2017-06-29T09:02:51Z")
Hi everyone Elasticsearch version: elasticsearch-5.4.3-1.noarch Logstash version: logstash-5.4.3-1.noarch Scenario IT: beaver -\> REDIS -\> Logstash -\> ES ELK is working properly. No issues apparently. I know this… --- ## [Convert monthname to monthnumber in logstash configuration file](https://discuss.elastic.co/t/convert-monthname-to-monthnumber-in-logstash-configuration-file/91230)
**Author:** [@JyothiRaj](https://discuss.elastic.co/u/JyothiRaj)\ **Replies:** 1\ **Last updated:** [June 29, 2017, 8:59am UTC](https://discuss.elastic.co/t/convert-monthname-to-monthnumber-in-logstash-configuration-file/91230 "2017-06-29T08:59:54Z")
Hi, First of all, I am very new to this elastic search and logstash. Excuse me If it sounds simple. I am having issue with logstash conf file. Currently my date format is "Wed Feb 15 08:05:50 2017". I wrote filter to m… --- ## [Logstash stopping randomly after few hours](https://discuss.elastic.co/t/logstash-stopping-randomly-after-few-hours/90383)
**Author:** [@gautamv](https://discuss.elastic.co/u/gautamv)\ **Replies:** 10\ **Last updated:** [June 29, 2017, 8:36am UTC](https://discuss.elastic.co/t/logstash-stopping-randomly-after-few-hours/90383 "2017-06-29T08:36:03Z")
Hi, I'm getting a random error when I run logstash: 16:30:26.240 \[\[main\]\>worker0\] ERROR logstash.pipeline - Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configura… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=953) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=955)