# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1214 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1215 --- ## [Buffer data in Logstash if Elasticsearch is down](https://discuss.elastic.co/t/buffer-data-in-logstash-if-elasticsearch-is-down/29127)
**Author:** [@scaarup](https://discuss.elastic.co/u/scaarup)\ **Replies:** 5\ **Last updated:** [December 3, 2015, 6:56am UTC](https://discuss.elastic.co/t/buffer-data-in-logstash-if-elasticsearch-is-down/29127 "2015-12-03T06:56:37Z")
Is there a buffer in Logstash to keep data in case of Elasticsearch (or any other output-taget) is down? If so, can I configure the size etc.? So I can estimate for how long we can accept elasticsearch is down. --- ## [Logstash Error with ES-Shield plugin](https://discuss.elastic.co/t/logstash-error-with-es-shield-plugin/36224)
**Author:** [@Kaufusihm](https://discuss.elastic.co/u/Kaufusihm)\ **Replies:** 0\ **Last updated:** [December 2, 2015, 9:32pm UTC](https://discuss.elastic.co/t/logstash-error-with-es-shield-plugin/36224 "2015-12-02T21:32:36Z")
I'm trying to get logstash configured to work with Elasticsearch-shield and I am running into the following error: After running bin/plugin install logstash-output-elasticsearch-shield I get this error: ERROR: I… --- ## [1.5.5 ThreadDeath UncaughtExceptionHandler mystery](https://discuss.elastic.co/t/1-5-5-threaddeath-uncaughtexceptionhandler-mystery/36217)
**Author:** [@bitsofinfo](https://discuss.elastic.co/u/bitsofinfo)\ **Replies:** 2\ **Last updated:** [December 2, 2015, 9:01pm UTC](https://discuss.elastic.co/t/1-5-5-threaddeath-uncaughtexceptionhandler-mystery/36217 "2015-12-02T21:01:02Z")
We just upgraded from 1.4.1 to 1.5.5. Now we see these super-un-helpful errors in the error log every few minutes. How can we get the root stack trace logged or possibly determine what is causing this??? Exception:… --- ## [Logstash forwarder vs. logback json encoder](https://discuss.elastic.co/t/logstash-forwarder-vs-logback-json-encoder/34281)
**Author:** [@rammulay](https://discuss.elastic.co/u/rammulay)\ **Replies:** 2\ **Last updated:** [December 2, 2015, 8:30pm UTC](https://discuss.elastic.co/t/logstash-forwarder-vs-logback-json-encoder/34281 "2015-12-02T20:30:20Z")
Hi, logs can be forwarded to logstash server using logback json encoder, LogstashTcpSocketAppender (https://github.com/logstash/logstash-logback-encoder) without installing the logstash forwarder. Is there a reason we st… --- ## [Suddenly slow logstash after upgrade(s)](https://discuss.elastic.co/t/suddenly-slow-logstash-after-upgrade-s/36129)
**Author:** [@hvisage](https://discuss.elastic.co/u/hvisage)\ **Replies:** 1\ **Last updated:** [December 2, 2015, 8:19pm UTC](https://discuss.elastic.co/t/suddenly-slow-logstash-after-upgrade-s/36129 "2015-12-02T20:19:23Z")
Hi there, I had this config: http://pastebin.com/aZr0uGtg running with (I recall) 1.5.4 and it was chucking along quite nicely on a Ubuntu 14.04 We've had a problem with other parts of the system, so we did an update … --- ## [Set @timestamp manually](https://discuss.elastic.co/t/set-timestamp-manually/36176)
**Author:** [@shaharmor](https://discuss.elastic.co/u/shaharmor)\ **Replies:** 1\ **Last updated:** [December 2, 2015, 5:48pm UTC](https://discuss.elastic.co/t/set-timestamp-manually/36176 "2015-12-02T17:48:53Z")
Hi, Currently we use the logstash date filter to match our timestamp field (named \_time) using this config: date { match =\> \["\_time", "UNIX\_MS"\] remove\_field =\> \["\_time"\] } I see that all this … --- ## [Logstash and http-output plugin](https://discuss.elastic.co/t/logstash-and-http-output-plugin/36203)
**Author:** [@Alvin82](https://discuss.elastic.co/u/Alvin82)\ **Replies:** 1\ **Last updated:** [December 2, 2015, 4:50pm UTC](https://discuss.elastic.co/t/logstash-and-http-output-plugin/36203 "2015-12-02T16:50:37Z")
hi all, i've made a config file like this: input { file { add\_field =\> {"\[@metadata\]\[cl\_cert\]" =\> "cert.crt"} add\_field =\> {"\[@metadata\]\[cl\_key\]" =\> "private-key.pem"} path =\> "/var/log/\*.js… --- ## [Logstash with translate and external yaml file](https://discuss.elastic.co/t/logstash-with-translate-and-external-yaml-file/36182)
**Author:** [@l1carter](https://discuss.elastic.co/u/l1carter)\ **Replies:** 5\ **Last updated:** [December 2, 2015, 4:04pm UTC](https://discuss.elastic.co/t/logstash-with-translate-and-external-yaml-file/36182 "2015-12-02T16:04:37Z")
all, I've got a logstash.conf file I'm working with and it's using a translate function like so: translate { field =\> "hostname" destination =\> "devtype" dictionary\_path =\> "/opt/elk/logst… --- ## [Logstash JDBC Plugin Driver Not Loaded Error](https://discuss.elastic.co/t/logstash-jdbc-plugin-driver-not-loaded-error/35000)
**Author:** [@jeromefroe](https://discuss.elastic.co/u/jeromefroe)\ **Replies:** 6\ **Last updated:** [December 2, 2015, 2:05pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-driver-not-loaded-error/35000 "2015-12-02T14:05:08Z")
Hello all, I have been trying to use the jdbc input plugin for logstash but keep getting a driver not loaded error. Specifically, I am using logstash-2.0.0 and the oracle jdbc jar (ojdbc6.jar). I have seen similar issues… --- ## [RabbitMQ is losing messages](https://discuss.elastic.co/t/rabbitmq-is-losing-messages/36181)
**Author:** [@billy6](https://discuss.elastic.co/u/billy6)\ **Replies:** 0\ **Last updated:** [December 2, 2015, 1:57pm UTC](https://discuss.elastic.co/t/rabbitmq-is-losing-messages/36181 "2015-12-02T13:57:37Z")
I have just installed RabbitMQ, I'm using it with logstash. I have been sending messages but I lose a lot of data. I have no idea how could I solve it. I'm using the default configuration in rabbitMQ. These are the out… --- ## [Logstash can't flush b/c of ClientProtocolException](https://discuss.elastic.co/t/logstash-cant-flush-b-c-of-clientprotocolexception/36164)
**Author:** [@treehouse](https://discuss.elastic.co/u/treehouse)\ **Replies:** 0\ **Last updated:** [December 2, 2015, 12:32pm UTC](https://discuss.elastic.co/t/logstash-cant-flush-b-c-of-clientprotocolexception/36164 "2015-12-02T12:32:25Z")
I can't quite figure out what is going on here. This error is only popping up for one particular index that I am inserting to. The vast majority of writes from LS to ES are going smoothly. Just to this one particular ind… --- ## [Index Twitter Stream with Logstash](https://discuss.elastic.co/t/index-twitter-stream-with-logstash/35905)
**Author:** [@adellarocca](https://discuss.elastic.co/u/adellarocca)\ **Replies:** 2\ **Last updated:** [December 2, 2015, 11:14am UTC](https://discuss.elastic.co/t/index-twitter-stream-with-logstash/35905 "2015-12-02T11:14:22Z")
Hi! I'm trying to index a Twitter stream using twitter plugin of Logstash. This is my configuration file input { twitter { consumer\_key =\> "omitted" consumer\_secret =\> "omitted" oauth\_token =\> "omitted" oauth\_tok… --- ## [What happens if Logstash stops indexing from Redis?](https://discuss.elastic.co/t/what-happens-if-logstash-stops-indexing-from-redis/36136)
**Author:** [@dorr](https://discuss.elastic.co/u/dorr)\ **Replies:** 1\ **Last updated:** [December 2, 2015, 10:55am UTC](https://discuss.elastic.co/t/what-happens-if-logstash-stops-indexing-from-redis/36136 "2015-12-02T10:55:50Z")
Hello, Consider this scenario: Logstash shipper ships to Redis, and a Logstash indexer consumes messages from it. Suddenly, the indexer stops due to server shutdown. What is going to happen to Redis? Will it be fed me… --- ## [Logstash "doc\_as\_upsert" --\> Error 404](https://discuss.elastic.co/t/logstash-doc-as-upsert-error-404/36146)
**Author:** [@MarcelHallmann](https://discuss.elastic.co/u/MarcelHallmann)\ **Replies:** 0\ **Last updated:** [December 2, 2015, 10:39am UTC](https://discuss.elastic.co/t/logstash-doc-as-upsert-error-404/36146 "2015-12-02T10:39:56Z")
Hi, I'm trying to upgrade to logstash 2.x and can't figure out how to use the "upsert" correctly. In my config I have: output { elasticsearch { action =\> "update" doc\_as\_upsert =\> true docume… --- ## [Logstash - Multiple Log Sources 'Syslog'](https://discuss.elastic.co/t/logstash-multiple-log-sources-syslog/35743)
**Author:** [@inzanez](https://discuss.elastic.co/u/inzanez)\ **Replies:** 2\ **Last updated:** [December 2, 2015, 8:29am UTC](https://discuss.elastic.co/t/logstash-multiple-log-sources-syslog/35743 "2015-12-02T08:29:47Z")
Hi! I'm in the process of planning an ELK setup to have a central log collection and monitoring system. I've got a few Linux clusters, firewalls and Microsoft Windows servers I would like to collect logs from. Firew… --- ## [Logstash Metrics Using Dots - problems with ES 2.x?](https://discuss.elastic.co/t/logstash-metrics-using-dots-problems-with-es-2-x/36123)
**Author:** [@ninjada](https://discuss.elastic.co/u/ninjada)\ **Replies:** 0\ **Last updated:** [December 2, 2015, 4:51am UTC](https://discuss.elastic.co/t/logstash-metrics-using-dots-problems-with-es-2-x/36123 "2015-12-02T04:51:55Z")
hello, just going through all the potential problems with my upgrade to latest elk stack versions (logstash 2.x / elasticsearch 2.x / kibana 4.x) received a number of warnings about dots in field names: Dots in field… --- ## [Logstash and NetFlow v5](https://discuss.elastic.co/t/logstash-and-netflow-v5/36104)
**Author:** [@skinner1984](https://discuss.elastic.co/u/skinner1984)\ **Replies:** 0\ **Last updated:** [December 1, 2015, 10:07pm UTC](https://discuss.elastic.co/t/logstash-and-netflow-v5/36104 "2015-12-01T22:07:25Z")
Hi guys, I'm trying to get a really basic Netflow setup with Elasticsearch and Kibana. Logstash seems to be receiving the NetFlow records OK, and I can see that Elasticsearch creates an index within Kibana \[Imgur\] … --- ## [Retry Logstash Input if it Fails](https://discuss.elastic.co/t/retry-logstash-input-if-it-fails/36092)
**Author:** [@jeromefroe](https://discuss.elastic.co/u/jeromefroe)\ **Replies:** 2\ **Last updated:** [December 1, 2015, 9:52pm UTC](https://discuss.elastic.co/t/retry-logstash-input-if-it-fails/36092 "2015-12-01T21:52:04Z")
I'm currently using the jdbc plug-in to ingest data from an Oracle database into Elasticsearch every hour on the hour. I've noticed that sometimes the data fails to be ingested and when I check the logstash log file I se… --- ## [XML filter error](https://discuss.elastic.co/t/xml-filter-error/36085)
**Author:** [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Replies:** 2\ **Last updated:** [December 1, 2015, 7:32pm UTC](https://discuss.elastic.co/t/xml-filter-error/36085 "2015-12-01T19:32:12Z")
Working with 2.1.0 logstash. With test.xml: \value1\ ...and configuration: input{ file{ path =\> "/test.xml" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } f… --- ## [Logstash service stops after few minutes](https://discuss.elastic.co/t/logstash-service-stops-after-few-minutes/35968)
**Author:** [@ananthanaidu](https://discuss.elastic.co/u/ananthanaidu)\ **Replies:** 6\ **Last updated:** [December 1, 2015, 7:26pm UTC](https://discuss.elastic.co/t/logstash-service-stops-after-few-minutes/35968 "2015-12-01T19:26:15Z")
Hello Greetings, Can some one help us to fix as we start logstash service it stops with in few minutes and logs shows with the below error, {:timestamp=\>"2015-11-30T13:04:07.450000-0600", :message=\>"The error reporte… --- ## [Logstash stop reading from RabbitMQ](https://discuss.elastic.co/t/logstash-stop-reading-from-rabbitmq/34750)
**Author:** [@srikanth.guduru](https://discuss.elastic.co/u/srikanth.guduru)\ **Replies:** 4\ **Last updated:** [December 1, 2015, 7:24pm UTC](https://discuss.elastic.co/t/logstash-stop-reading-from-rabbitmq/34750 "2015-12-01T19:24:14Z")
After sometime logstash stop reading data from RabbitMQ. Gives out no error, just halts. Messages stack-up in unacknowledged mode in RabbitMQ Messages rates can go upto 40-50 messages per second. My config for losgtas… --- ## [Need help in configuring the logstash on linux box](https://discuss.elastic.co/t/need-help-in-configuring-the-logstash-on-linux-box/35909)
**Author:** [@Bharath\_Pusuluri](https://discuss.elastic.co/u/Bharath_Pusuluri)\ **Replies:** 9\ **Last updated:** [December 1, 2015, 6:29pm UTC](https://discuss.elastic.co/t/need-help-in-configuring-the-logstash-on-linux-box/35909 "2015-12-01T18:29:22Z")
our plan is to install logstash and filter the /var/log/messages based on key words and it need to log to a file and mail should be sent if any filter matches i am not able to configure also at this moment. could any … --- ## [Multiple Logstash Docker containers sharing an S3 input](https://discuss.elastic.co/t/multiple-logstash-docker-containers-sharing-an-s3-input/36077)
**Author:** [@Paul\_Kavanagh](https://discuss.elastic.co/u/Paul_Kavanagh)\ **Replies:** 1\ **Last updated:** [December 1, 2015, 6:01pm UTC](https://discuss.elastic.co/t/multiple-logstash-docker-containers-sharing-an-s3-input/36077 "2015-12-01T18:01:30Z")
This is one of those "I can't be the first one" kinda questions. We're running Logstash server in Docker. We run anywhere between 3 and 6, depending on the logging volume. Our current input is just a Redis box (Elastic… --- ## [How anonymize some fields?](https://discuss.elastic.co/t/how-anonymize-some-fields/36062)
**Author:** [@sampasei](https://discuss.elastic.co/u/sampasei)\ **Replies:** 5\ **Last updated:** [December 1, 2015, 5:35pm UTC](https://discuss.elastic.co/t/how-anonymize-some-fields/36062 "2015-12-01T17:35:31Z")
Hi, i have several logs as input: each record log has different field. I want to pass some fields and change the remaning ones: some fields are common to all records, others are totally different. There's a way to do th… --- ## [One input two outputs](https://discuss.elastic.co/t/one-input-two-outputs/36069)
**Author:** [@jjdepaul](https://discuss.elastic.co/u/jjdepaul)\ **Replies:** 1\ **Last updated:** [December 1, 2015, 4:14pm UTC](https://discuss.elastic.co/t/one-input-two-outputs/36069 "2015-12-01T16:14:00Z")
Not sure why my earlier post disappeared.... let me try again: Our configuration: LS 2.0, ES 2.0, Kib 4.2 We have an input config file that processes trace logs produced by our application. We filter out the data w… --- ## [Logstash 2.1 : Kafka Input](https://discuss.elastic.co/t/logstash-2-1-kafka-input/36071)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 0\ **Last updated:** [December 1, 2015, 3:54pm UTC](https://discuss.elastic.co/t/logstash-2-1-kafka-input/36071 "2015-12-01T15:54:59Z")
Hi, I'm experimenting with using kafka as a broker in my elk stack. I have logstash outputting ingested mesages to kafka and then have a second logstash instance consuming that topic from the broker. The version of kafka… --- ## [How to handle logstash failures](https://discuss.elastic.co/t/how-to-handle-logstash-failures/36044)
**Author:** [@sampath\_methuku](https://discuss.elastic.co/u/sampath_methuku)\ **Replies:** 1\ **Last updated:** [December 1, 2015, 2:43pm UTC](https://discuss.elastic.co/t/how-to-handle-logstash-failures/36044 "2015-12-01T14:43:16Z")
I am using following logstash configuration to push log file content in to elastic search. input { file { path =\> "/home/testuser/apps/logs/sample-collector-app.log" start\_position =\> "beginning" … --- ## [How to measure the performance of logstash with logstash-perftool gem?](https://discuss.elastic.co/t/how-to-measure-the-performance-of-logstash-with-logstash-perftool-gem/36042)
**Author:** [@Vachan.Da](https://discuss.elastic.co/u/Vachan.Da)\ **Replies:** 0\ **Last updated:** [December 1, 2015, 12:35pm UTC](https://discuss.elastic.co/t/how-to-measure-the-performance-of-logstash-with-logstash-perftool-gem/36042 "2015-12-01T12:35:35Z")
After installing the logstash-perftool on the machine, how do you make it test the custom logstash configuration? I'm running logstash 1.4.2 for indexing and ES 1.4.4 to store. --- ## [Adding data to additional fields depending on the content of the message](https://discuss.elastic.co/t/adding-data-to-additional-fields-depending-on-the-content-of-the-message/36026)
**Author:** [@Usm](https://discuss.elastic.co/u/Usm)\ **Replies:** 1\ **Last updated:** [December 1, 2015, 11:14am UTC](https://discuss.elastic.co/t/adding-data-to-additional-fields-depending-on-the-content-of-the-message/36026 "2015-12-01T11:14:29Z")
Hello! I have next situation. I need to add fields to each log message if it has specific information (matching with some regexp). Is there any possibillity to check the message to matching some regexp and if it is match… --- ## [Losgstah configuration issue](https://discuss.elastic.co/t/losgstah-configuration-issue/36017)
**Author:** [@fbicquelet](https://discuss.elastic.co/u/fbicquelet)\ **Replies:** 2\ **Last updated:** [December 1, 2015, 10:08am UTC](https://discuss.elastic.co/t/losgstah-configuration-issue/36017 "2015-12-01T10:08:07Z")
Hello, I begin with logstash and ElasticSearch and I would like to index .pdf or .doc file type in ElasticSearch via logstash. I configured logstash using the codec multiline to get my file in a single message in Elastic… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1213) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1215)