# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=984 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 985 --- ## [Failure to replace elastic search document\_type using logstash config](https://discuss.elastic.co/t/failure-to-replace-elastic-search-document-type-using-logstash-config/85255)
**Author:** [@cganesan](https://discuss.elastic.co/u/cganesan)\ **Replies:** 1\ **Last updated:** [May 14, 2017, 8:31pm UTC](https://discuss.elastic.co/t/failure-to-replace-elastic-search-document-type-using-logstash-config/85255 "2017-05-14T20:31:50Z")
I'd like elastic search to dynamically create different document\_types based on input json variable property in the config. For quite some time I've been struggling to get logstash replace the elastic search document\_typ… --- ## [Logstash decrease write rate when rabbitmq full](https://discuss.elastic.co/t/logstash-decrease-write-rate-when-rabbitmq-full/85064)
**Author:** [@guy\_es](https://discuss.elastic.co/u/guy_es)\ **Replies:** 1\ **Last updated:** [May 14, 2017, 9:31am UTC](https://discuss.elastic.co/t/logstash-decrease-write-rate-when-rabbitmq-full/85064 "2017-05-14T09:31:41Z")
Hi all, I use logstash 2.4 to forward docs from RabbitMQ to Elasticsearch 2.4, I see correlation between how much the queue is full to logstash write rate. When the queue gets full the write rate significantly decrease… --- ## [Logstash Slow after small config change](https://discuss.elastic.co/t/logstash-slow-after-small-config-change/85174)
**Author:** [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\ **Replies:** 1\ **Last updated:** [May 14, 2017, 9:07am UTC](https://discuss.elastic.co/t/logstash-slow-after-small-config-change/85174 "2017-05-14T09:07:34Z")
Hi I have changed only one place in my log config file, by adding one field into the prune's whitelist. Then, the logstash parses super slowly, 10X roughly. Also, the added variable does not show properly in Kibana: no … --- ## [Grok doesn't work](https://discuss.elastic.co/t/grok-doesnt-work/85645)
**Author:** [@Eduard\_Kolesnikov](https://discuss.elastic.co/u/Eduard_Kolesnikov)\ **Replies:** 3\ **Last updated:** [May 14, 2017, 6:03am UTC](https://discuss.elastic.co/t/grok-doesnt-work/85645 "2017-05-14T06:03:20Z")
Hello, i have some problem with grok. my\_config.conf: input { stdin {} } filter { grok { } } output { stdout { codec =\> rubydebug } } but grok doesn't work. logs: Sending Logstash's logs to /home/em/… --- ## [How to define input based on variables from another input](https://discuss.elastic.co/t/how-to-define-input-based-on-variables-from-another-input/57399)
**Author:** [@billzy](https://discuss.elastic.co/u/billzy)\ **Replies:** 5\ **Last updated:** [May 13, 2017, 1:44pm UTC](https://discuss.elastic.co/t/how-to-define-input-based-on-variables-from-another-input/57399 "2017-05-13T13:44:58Z")
here's a case that I need to finish following steps 1. call a rest service end point to fetch a returned id 2. call another rest end point using the returned id in step 1 as parameter for step 1 , I defined a http\_poll… --- ## [Unable to view logs on Elastic/Kibana using JDBC Logstash plugin](https://discuss.elastic.co/t/unable-to-view-logs-on-elastic-kibana-using-jdbc-logstash-plugin/62220)
**Author:** [@tesaaron](https://discuss.elastic.co/u/tesaaron)\ **Replies:** 3\ **Last updated:** [May 12, 2017, 6:59pm UTC](https://discuss.elastic.co/t/unable-to-view-logs-on-elastic-kibana-using-jdbc-logstash-plugin/62220 "2017-05-12T18:59:11Z")
Hello, I have successfully connected Logstash to my back-end Vertica database using the JDBC plugin for Logstash. Anytime a new entry is made in the table I am pointing at in Vertica I can view the Logstash update in v… --- ## [Logstash 5.4.0 not writing logs or opening port](https://discuss.elastic.co/t/logstash-5-4-0-not-writing-logs-or-opening-port/85495)
**Author:** [@Geezer](https://discuss.elastic.co/u/Geezer)\ **Replies:** 4\ **Last updated:** [May 12, 2017, 5:53pm UTC](https://discuss.elastic.co/t/logstash-5-4-0-not-writing-logs-or-opening-port/85495 "2017-05-12T17:53:56Z")
Hello, I have a new install of Logstash 5.4.0 on CentOS 7 with a single beat input and output configured. The service starts, and sytemctl says it is running, but it won't open the TCP port, connect to RabbitMQ or w… --- ## [Logstash as windows service not reading log files](https://discuss.elastic.co/t/logstash-as-windows-service-not-reading-log-files/85455)
**Author:** [@lgopalakrishnan](https://discuss.elastic.co/u/lgopalakrishnan)\ **Replies:** 6\ **Last updated:** [May 12, 2017, 5:32pm UTC](https://discuss.elastic.co/t/logstash-as-windows-service-not-reading-log-files/85455 "2017-05-12T17:32:29Z")
Hello, I have logstash installed as a windows service and it is not reading the log files. When I run logstash from command prompt it works fine. Any help would be appreciated.. Thanks -Leelakrishnan --- ## [Logstash noisy and throwing errors about no elastic connection](https://discuss.elastic.co/t/logstash-noisy-and-throwing-errors-about-no-elastic-connection/84295)
**Author:** [@mut3](https://discuss.elastic.co/u/mut3)\ **Replies:** 1\ **Last updated:** [May 12, 2017, 5:05pm UTC](https://discuss.elastic.co/t/logstash-noisy-and-throwing-errors-about-no-elastic-connection/84295 "2017-05-12T17:05:02Z")
I am running logstash from a docker container built from elastic.co's 5.3.2 container (latest at time of post). As part of testing I launch this container manually. docker run --rm -it --name=testy --volume="$(pwd)/log… --- ## [Kibana not reading logstash output](https://discuss.elastic.co/t/kibana-not-reading-logstash-output/85564)
**Author:** [@Honda\_fred\_elk](https://discuss.elastic.co/u/Honda_fred_elk)\ **Replies:** 1\ **Last updated:** [May 12, 2017, 3:22pm UTC](https://discuss.elastic.co/t/kibana-not-reading-logstash-output/85564 "2017-05-12T15:22:36Z")
Hi, I have two inputs plugins in logstash.conf 1. beats & 2. file like below. I input JSON string for both. Logstash config input { file{ path =\> "/logs/ServerLogs/testing-jsonoutput.log" sincedb\_path =\> … --- ## [Logstash-output-syslog questions / improvements for central log hub towards other SIEM/Compliancy tool](https://discuss.elastic.co/t/logstash-output-syslog-questions-improvements-for-central-log-hub-towards-other-siem-compliancy-tool/84882)
**Author:** [@arnold79](https://discuss.elastic.co/u/arnold79)\ **Replies:** 8\ **Last updated:** [May 12, 2017, 2:13pm UTC](https://discuss.elastic.co/t/logstash-output-syslog-questions-improvements-for-central-log-hub-towards-other-siem-compliancy-tool/84882 "2017-05-12T14:13:55Z")
Hi All, Currently working on a project to replace a syslog-ng product ( balabit) in front of a SIEM solution, which centrally receives all messages and if needed forwards that unique event towards the SIEM in the comp… --- ## [Extracting JSON and Parsing it](https://discuss.elastic.co/t/extracting-json-and-parsing-it/85536)
**Author:** [@Vedran\_Maricevic](https://discuss.elastic.co/u/Vedran_Maricevic)\ **Replies:** 4\ **Last updated:** [May 12, 2017, 1:13pm UTC](https://discuss.elastic.co/t/extracting-json-and-parsing-it/85536 "2017-05-12T13:13:08Z")
I have a JSON log that comes to Logstash. It looks like this: \[{"orderNumber":"xxxxxxxx","externalOrderNumber":"07efc4c7d3113453427f7fe525e22a61e","operation":{"name":"CAPTURE","amount":990,"status":"PENDING","create… --- ## [Dateparsefailure: change @timestamp to timestamp for apache php error log](https://discuss.elastic.co/t/dateparsefailure-change-timestamp-to-timestamp-for-apache-php-error-log/85403)
**Author:** [@PratapSingh](https://discuss.elastic.co/u/PratapSingh)\ **Replies:** 10\ **Last updated:** [May 12, 2017, 12:09pm UTC](https://discuss.elastic.co/t/dateparsefailure-change-timestamp-to-timestamp-for-apache-php-error-log/85403 "2017-05-12T12:09:24Z")
Hi , I am using apache error log where timestamp is as "Wed May 10 17:21:26.820378 2017" Sample log is provided below: Sample log: \[Wed May 10 17:21:26.820378 2017\] \[proxy\_fcgi:error\] \[pid 6803:tid 139780532639488\] … --- ## [How to avoid jdbc database polling and make connections on demand?](https://discuss.elastic.co/t/how-to-avoid-jdbc-database-polling-and-make-connections-on-demand/82298)
**Author:** [@endersonmaia](https://discuss.elastic.co/u/endersonmaia)\ **Replies:** 2\ **Last updated:** [May 12, 2017, 11:13am UTC](https://discuss.elastic.co/t/how-to-avoid-jdbc-database-polling-and-make-connections-on-demand/82298 "2017-05-12T11:13:39Z")
I've already setup a scenario where I get all the sales from a database (input-jdbc) , by polling the database every minute via jdbc schedule parameter. I send this (output-rabbitmq) to a rabbitmq queue and I subscribe… --- ## [Logstash Twitter Input with Coordinates is not working](https://discuss.elastic.co/t/logstash-twitter-input-with-coordinates-is-not-working/46893)
**Author:** [@Omer\_Uludag](https://discuss.elastic.co/u/Omer_Uludag)\ **Replies:** 3\ **Last updated:** [May 12, 2017, 10:44am UTC](https://discuss.elastic.co/t/logstash-twitter-input-with-coordinates-is-not-working/46893 "2017-05-12T10:44:12Z")
Hello together, I am using Logstash in order to ingest data from Twitter. I am using the following configuration file: input { twitter { consumer\_key =\> "XXXXXXXXXX" con… --- ## [Logstash5.4 output kafka problem](https://discuss.elastic.co/t/logstash5-4-output-kafka-problem/85533)
**Author:** [@wongchao](https://discuss.elastic.co/u/wongchao)\ **Replies:** 1\ **Last updated:** [May 12, 2017, 9:53am UTC](https://discuss.elastic.co/t/logstash5-4-output-kafka-problem/85533 "2017-05-12T09:53:47Z")
kafka 0.9.0.1 centos 6.5 logstash 5.4 with the following config, logstash2.4 can output data to kafka topic, but failed with logatsh5.4. i don't know why. input{ file { path =\> "/home/wangchao/api2.l… --- ## [Unable to avoid JSON parsing errors in logstash log-file](https://discuss.elastic.co/t/unable-to-avoid-json-parsing-errors-in-logstash-log-file/84673)
**Author:** [@Marco\_Spadoni](https://discuss.elastic.co/u/Marco_Spadoni)\ **Replies:** 1\ **Last updated:** [May 12, 2017, 9:49am UTC](https://discuss.elastic.co/t/unable-to-avoid-json-parsing-errors-in-logstash-log-file/84673 "2017-05-12T09:49:44Z")
Hi all. I am working with logstash and elasticsearch 5.2.2 version. I had the following pipeline: input { gelf { codec =\> "json" port =\> 12201 } } filter { mutate { gsub =\> \["time… --- ## [Not able to install logstash-core-event-java gem as its giving me error](https://discuss.elastic.co/t/not-able-to-install-logstash-core-event-java-gem-as-its-giving-me-error/84589)
**Author:** [@kartikeya\_sharma\_13](https://discuss.elastic.co/u/kartikeya_sharma_13)\ **Replies:** 1\ **Last updated:** [May 12, 2017, 9:34am UTC](https://discuss.elastic.co/t/not-able-to-install-logstash-core-event-java-gem-as-its-giving-me-error/84589 "2017-05-12T09:34:52Z")
Hi All, I am not able to install the logstash-core-event-java gem as it is gining me this error Bundler::GemNotFound: Could not find gem 'logstash-filter-translate (\>= 0) java' in any of the gem sources listed in you… --- ## [Alternative of S3 Plugin](https://discuss.elastic.co/t/alternative-of-s3-plugin/85531)
**Author:** [@sheenu.celeste](https://discuss.elastic.co/u/sheenu.celeste)\ **Replies:** 0\ **Last updated:** [May 12, 2017, 9:29am UTC](https://discuss.elastic.co/t/alternative-of-s3-plugin/85531 "2017-05-12T09:29:25Z")
I have to transfer cloudfront log files from s3 bucket to my logstash running on an ec2 instance , I tried using the s3 input plugin for that but the plugin performance is not good when load on bucket is high , can you s… --- ## [How to change delimiter with json\_lines](https://discuss.elastic.co/t/how-to-change-delimiter-with-json-lines/85286)
**Author:** [@henrilabarre](https://discuss.elastic.co/u/henrilabarre)\ **Replies:** 8\ **Last updated:** [May 12, 2017, 9:21am UTC](https://discuss.elastic.co/t/how-to-change-delimiter-with-json-lines/85286 "2017-05-12T09:21:56Z")
Hi here is my conf file input { elasticsearch { hosts =\> "http://localhost:9200" index =\> "sirene" query =\> '{"query": {"query\_string" : {"query": "((DEPET:54 AND provider:sp\_mairie) OR (DEPET:55 AND provider:sp\_m… --- ## [How to re fetch data from Kafka, from the beginning of topic, without changing consumer group? (resetting consumer group offset)](https://discuss.elastic.co/t/how-to-re-fetch-data-from-kafka-from-the-beginning-of-topic-without-changing-consumer-group-resetting-consumer-group-offset/84492)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [May 12, 2017, 9:13am UTC](https://discuss.elastic.co/t/how-to-re-fetch-data-from-kafka-from-the-beginning-of-topic-without-changing-consumer-group-resetting-consumer-group-offset/84492 "2017-05-12T09:13:56Z")
Hi, I am using ELK GA 5.0.0. I have a logstash instance to pull data from a kafka topic, like below; kafka{ group\_id =\> "kafkagroupA" topics =\> \["kafkatopicA"\] bootstrap\_servers =\> "192.168.0.1:9091,192.168.0.2:90… --- ## [How to configure SSL for logstash-input-tcp when running on windows?](https://discuss.elastic.co/t/how-to-configure-ssl-for-logstash-input-tcp-when-running-on-windows/85260)
**Author:** [@Tonni\_Hult](https://discuss.elastic.co/u/Tonni_Hult)\ **Replies:** 1\ **Last updated:** [May 12, 2017, 8:36am UTC](https://discuss.elastic.co/t/how-to-configure-ssl-for-logstash-input-tcp-when-running-on-windows/85260 "2017-05-12T08:36:50Z")
Hi I need help in getting the ssl configuration correct. We are running ELK on windows servers and are now sending the logs from nxlog to logstash over tcp, port 5000. Now we are switching over to ssl but I can't seem … --- ## [Logstash elasticsearch input](https://discuss.elastic.co/t/logstash-elasticsearch-input/85276)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 1\ **Last updated:** [May 12, 2017, 8:17am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input/85276 "2017-05-12T08:17:38Z")
I am trying to use logstash to copy an elasticsearch input to an elasticsearch output. The documentation tells me I can use query =\> "\*". Using logstash 5.4, with an ES 5.2 input I get "root\_cause":\[{"type":"parse\_exce… --- ## [Logstash - complex filtering](https://discuss.elastic.co/t/logstash-complex-filtering/85433)
**Author:** [@Vedran\_Maricevic](https://discuss.elastic.co/u/Vedran_Maricevic)\ **Replies:** 2\ **Last updated:** [May 12, 2017, 7:32am UTC](https://discuss.elastic.co/t/logstash-complex-filtering/85433 "2017-05-12T07:32:32Z")
I have ELK setup. I am parsing several completely different log formats. They were all one liners. Now I need to add a different one which is a multi line. Example of two log entries: error: callback invoked exception… --- ## [Cannot install Logstash plugin from offline pack](https://discuss.elastic.co/t/cannot-install-logstash-plugin-from-offline-pack/83407)
**Author:** [@franhoeyemac](https://discuss.elastic.co/u/franhoeyemac)\ **Replies:** 5\ **Last updated:** [May 12, 2017, 7:06am UTC](https://discuss.elastic.co/t/cannot-install-logstash-plugin-from-offline-pack/83407 "2017-05-12T07:06:32Z")
Hello, I've prepared an offline pack on my computer by running the following command: bin\\logstash-plugin prepare-offline-pack --output syslog-output.zip logstash-output-syslog When I move the zip to the server with… --- ## [Need to be pointed down the right path](https://discuss.elastic.co/t/need-to-be-pointed-down-the-right-path/85488)
**Author:** [@Tim\_Flowers](https://discuss.elastic.co/u/Tim_Flowers)\ **Replies:** 1\ **Last updated:** [May 12, 2017, 3:50am UTC](https://discuss.elastic.co/t/need-to-be-pointed-down-the-right-path/85488 "2017-05-12T03:50:13Z")
I have a syslog and ELK stack setup that I am working though the final design of. A brief overview of what I am doing, I have about 100 firewalls from 10+ different vendors that I am sending data to a syslog server. Th… --- ## [Exclude Graphite fields from Elasticsearch](https://discuss.elastic.co/t/exclude-graphite-fields-from-elasticsearch/85094)
**Author:** [@Exocomp](https://discuss.elastic.co/u/Exocomp)\ **Replies:** 4\ **Last updated:** [May 12, 2017, 2:42am UTC](https://discuss.elastic.co/t/exclude-graphite-fields-from-elasticsearch/85094 "2017-05-12T02:42:49Z")
In the filter stage of logstash I'm creating some fields based on metricbeat data that are meant to be for Graphite specifically. Then in the output stage of logstash I'm sending the data to Graphite and Elasticsearch,… --- ## [Logstash mirror output](https://discuss.elastic.co/t/logstash-mirror-output/84982)
**Author:** [@PandKing](https://discuss.elastic.co/u/PandKing)\ **Replies:** 14\ **Last updated:** [May 12, 2017, 2:12am UTC](https://discuss.elastic.co/t/logstash-mirror-output/84982 "2017-05-12T02:12:11Z")
Hi, Is there a way to mirror a logstash input to 2 or more outputs ? I added 2 outputs to my winlogbeat.yml but only seems to take the first. Trying to send one pipeline to ES and a copy to another output. Th… --- ## [Logstash is throwing an error message and unable to stay up, keeps stopping and starting the pipeline](https://discuss.elastic.co/t/logstash-is-throwing-an-error-message-and-unable-to-stay-up-keeps-stopping-and-starting-the-pipeline/85487)
**Author:** [@rhealitycheck](https://discuss.elastic.co/u/rhealitycheck)\ **Replies:** 1\ **Last updated:** [May 12, 2017, 1:53am UTC](https://discuss.elastic.co/t/logstash-is-throwing-an-error-message-and-unable-to-stay-up-keeps-stopping-and-starting-the-pipeline/85487 "2017-05-12T01:53:54Z")
I'm running a slightly older version of Logstash 5.1.1 on Amazon Linux 2017.03 with openjdk 1.8.0\_131 (because I'm also running the same version of Elasticsearch and Kibana). It is working in 2 clusters with the same har… --- ## [Simple Math Functions with Ruby in Logstash 5.3](https://discuss.elastic.co/t/simple-math-functions-with-ruby-in-logstash-5-3/85162)
**Author:** [@seth.yes](https://discuss.elastic.co/u/seth.yes)\ **Replies:** 2\ **Last updated:** [May 11, 2017, 5:43pm UTC](https://discuss.elastic.co/t/simple-math-functions-with-ruby-in-logstash-5-3/85162 "2017-05-11T17:43:05Z")
I'm trying to do simple math calculations, e.g. convert a number to a decimal such as 93 =\> 0.93. I've looked around and can't find the proper syntax to do this. Furthermore, I don't get any warnings in the logs, just … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=983) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=985)