# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1148 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1149 --- ## [Logstash can not reach ElasticSearch](https://discuss.elastic.co/t/logstash-can-not-reach-elasticsearch/51363)
**Author:** [@xueliang](https://discuss.elastic.co/u/xueliang)\ **Replies:** 2\ **Last updated:** [May 31, 2016, 3:29am UTC](https://discuss.elastic.co/t/logstash-can-not-reach-elasticsearch/51363 "2016-05-31T03:29:52Z")
I just try to use logstash to ship the log(json formate data) to elasticsearch. But I get an strange error my elasticsearch work correctly and logstash itself work correctly,but together do not work. The input and filt… --- ## [Windows Logstash shipper always crash at jffi-1.2 module](https://discuss.elastic.co/t/windows-logstash-shipper-always-crash-at-jffi-1-2-module/51364)
**Author:** [@coolzong](https://discuss.elastic.co/u/coolzong)\ **Replies:** 0\ **Last updated:** [May 31, 2016, 2:48am UTC](https://discuss.elastic.co/t/windows-logstash-shipper-always-crash-at-jffi-1-2-module/51364 "2016-05-31T02:48:34Z")
从 One Logstash shipper on windows 2012 frequently keeps crashing 继续讨论： I also meet the issue on Windows 2008, JRE8\_66. Once our one bat job start to move log files for archive at every day night 23:50, the logstash JVM … --- ## [Logstash SIGTERM received](https://discuss.elastic.co/t/logstash-sigterm-received/35377)
**Author:** [@Marco](https://discuss.elastic.co/u/Marco)\ **Replies:** 3\ **Last updated:** [May 30, 2016, 7:23pm UTC](https://discuss.elastic.co/t/logstash-sigterm-received/35377 "2016-05-30T19:23:48Z")
when I restart the logstash the logstash.log shows as below: {:timestamp=\>"2015-11-24T09:30:18.155000+0800", :message=\>"SIGTERM received. Shutting down the pipeline.", :level=\>:warn} {:timestamp=\>"2015-11-24T09:33:07.125… --- ## [Help with Grok & Regexp: effective reuse of patterns](https://discuss.elastic.co/t/help-with-grok-regexp-effective-reuse-of-patterns/51175)
**Author:** [@Bruno\_Lavoie](https://discuss.elastic.co/u/Bruno_Lavoie)\ **Replies:** 2\ **Last updated:** [May 30, 2016, 5:07pm UTC](https://discuss.elastic.co/t/help-with-grok-regexp-effective-reuse-of-patterns/51175 "2016-05-30T17:07:19Z")
Hello, I'm trying to extract fields from messages and making some reuse of information tokens but sometimes my regexps are too greedy and capture outside the end tag. I have a few different type of messages but some … --- ## [Use logcheck ignore rules](https://discuss.elastic.co/t/use-logcheck-ignore-rules/51149)
**Author:** [@quimnuss](https://discuss.elastic.co/u/quimnuss)\ **Replies:** 5\ **Last updated:** [May 30, 2016, 1:45pm UTC](https://discuss.elastic.co/t/use-logcheck-ignore-rules/51149 "2016-05-30T13:45:06Z")
I've started using ELK yesterday, moving from logcheck and I would like to re-use the ignore rules of logcheck and some custom I had added. I currently have two servers providing logs throught filebeat to logstash, who t… --- ## [Filtering/Visualising JMX fields](https://discuss.elastic.co/t/filtering-visualising-jmx-fields/51326)
**Author:** [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)\ **Replies:** 0\ **Last updated:** [May 30, 2016, 1:28pm UTC](https://discuss.elastic.co/t/filtering-visualising-jmx-fields/51326 "2016-05-30T13:28:39Z")
Hi, I am successfully able to integrate JMX plugin for logstash. Now to i am trying to Visualize the JMX data. For this i am trying to add custom fields to the parsed jmx data. Example: if \[type\] == "jmx" { if… --- ## [Logstash: Amazon S3 Trouble](https://discuss.elastic.co/t/logstash-amazon-s3-trouble/51310)
**Author:** [@Piyush\_Paritosh](https://discuss.elastic.co/u/Piyush_Paritosh)\ **Replies:** 0\ **Last updated:** [May 30, 2016, 11:20am UTC](https://discuss.elastic.co/t/logstash-amazon-s3-trouble/51310 "2016-05-30T11:20:33Z")
I am using Macbook and I am trying to take input from Amazon S3. I have a bucket in my S3 and it has a textfile. I am using the following config: input { s3 { access\_key\_id =\> "Access\_Key" bucket =\> … --- ## [Parsing multiline and capture data, also from 2nd line onwards](https://discuss.elastic.co/t/parsing-multiline-and-capture-data-also-from-2nd-line-onwards/51309)
**Author:** [@pandith\_asrar](https://discuss.elastic.co/u/pandith_asrar)\ **Replies:** 0\ **Last updated:** [May 30, 2016, 11:18am UTC](https://discuss.elastic.co/t/parsing-multiline-and-capture-data-also-from-2nd-line-onwards/51309 "2016-05-30T11:18:00Z")
Hi Magnus, I am writing a config to read from a log file and using multiline codec. I am able to read the 1st line of the pattern and the multiline codec is also working, but the pattern for the multiline does not crea… --- ## [Logstash to Redis output plugin](https://discuss.elastic.co/t/logstash-to-redis-output-plugin/51288)
**Author:** [@criestlav](https://discuss.elastic.co/u/criestlav)\ **Replies:** 2\ **Last updated:** [May 30, 2016, 8:45am UTC](https://discuss.elastic.co/t/logstash-to-redis-output-plugin/51288 "2016-05-30T08:45:23Z")
Hi, I am a newbie in ELK. I have Logstash reading from a json file. What i am trying to do is that Logstash send the entire event to ES (it works) and just one field from json event to redis (it does not work). It intro… --- ## [Question regarding template\_overwrite in logstash output to elasticsearch](https://discuss.elastic.co/t/question-regarding-template-overwrite-in-logstash-output-to-elasticsearch/50908)
**Author:** [@macymin](https://discuss.elastic.co/u/macymin)\ **Replies:** 4\ **Last updated:** [May 30, 2016, 7:59am UTC](https://discuss.elastic.co/t/question-regarding-template-overwrite-in-logstash-output-to-elasticsearch/50908 "2016-05-30T07:59:10Z")
I want to do the on the fly template overwrite for 1 of my index data (basically is to decrease the shards count). so i need to add below session: my questions is do I need to add this to all the logstash indexer's conf … --- ## [Logstash and multiple outputs: how can I circumvent unreliable outputs?](https://discuss.elastic.co/t/logstash-and-multiple-outputs-how-can-i-circumvent-unreliable-outputs/51159)
**Author:** [@ppuschmann](https://discuss.elastic.co/u/ppuschmann)\ **Replies:** 2\ **Last updated:** [May 30, 2016, 7:20am UTC](https://discuss.elastic.co/t/logstash-and-multiple-outputs-how-can-i-circumvent-unreliable-outputs/51159 "2016-05-30T07:20:18Z")
Hi, in a test scenario (still using Logstash 1.5) we like to push logevents to two separate outputs. The first one is 'elasticsearch' the second one shall be 'redis'. Right now we experience errors in the whole Logst… --- ## [Upload date as timestemp](https://discuss.elastic.co/t/upload-date-as-timestemp/51243)
**Author:** [@chenbe2204](https://discuss.elastic.co/u/chenbe2204)\ **Replies:** 2\ **Last updated:** [May 30, 2016, 4:50am UTC](https://discuss.elastic.co/t/upload-date-as-timestemp/51243 "2016-05-30T04:50:56Z")
Hi , I have a field by the name "create\_date" , the field format is: "MM/DD/YYYY" for example: 05/29/2016. I want to update timestamp as create\_date Which match format I need to put when I use Logstash ? filter{ da… --- ## [Can I configure logstash to run a refresh in elasticsearch?](https://discuss.elastic.co/t/can-i-configure-logstash-to-run-a-refresh-in-elasticsearch/51121)
**Author:** [@LaszloHont](https://discuss.elastic.co/u/LaszloHont)\ **Replies:** 5\ **Last updated:** [May 30, 2016, 4:44am UTC](https://discuss.elastic.co/t/can-i-configure-logstash-to-run-a-refresh-in-elasticsearch/51121 "2016-05-30T04:44:58Z")
I'm (mis)using logstash to load a days worth of data into elasticsearch, and then exit. Is there a way for logstash to trigger an elasticsearch refresh on the index logstash is writing to after that has happened? Even … --- ## [Is it possible to combine the entire file as one event?](https://discuss.elastic.co/t/is-it-possible-to-combine-the-entire-file-as-one-event/2743)
**Author:** [@nathaniel](https://discuss.elastic.co/u/nathaniel)\ **Replies:** 4\ **Last updated:** [May 30, 2016, 4:32am UTC](https://discuss.elastic.co/t/is-it-possible-to-combine-the-entire-file-as-one-event/2743 "2016-05-30T04:32:11Z")
Hi guys. I am facing a situation where every of my log files are single exceptions, and I do not wish to store them line by line. Instead, I would prefer to store them file by file, which means, I wish to combine the ent… --- ## [Regex not working for translate filter](https://discuss.elastic.co/t/regex-not-working-for-translate-filter/44231)
**Author:** [@bharath\_meka](https://discuss.elastic.co/u/bharath_meka)\ **Replies:** 2\ **Last updated:** [May 30, 2016, 2:19am UTC](https://discuss.elastic.co/t/regex-not-working-for-translate-filter/44231 "2016-05-30T02:19:59Z")
Hi, I'm trying to add a translate filter to parse a field using regex and populate a destination based on the value. If it is a direct text match it is working fine. But with regex it is always going to fallback value. … --- ## [Feature Request - MQTT plugin for ES8266](https://discuss.elastic.co/t/feature-request-mqtt-plugin-for-es8266/51207)
**Author:** [@spuder](https://discuss.elastic.co/u/spuder)\ **Replies:** 3\ **Last updated:** [May 30, 2016, 12:04am UTC](https://discuss.elastic.co/t/feature-request-mqtt-plugin-for-es8266/51207 "2016-05-30T00:04:15Z")
MQTT is becoming a popular message buss for IOT devices like arduinos, raspberry Pi's and ES8266. It would be great if logstash had an input and an output pluging for MQTT message busses. Example use case: https:… --- ## [Cloning events from items in an array?](https://discuss.elastic.co/t/cloning-events-from-items-in-an-array/51256)
**Author:** [@mightyschwartz](https://discuss.elastic.co/u/mightyschwartz)\ **Replies:** 2\ **Last updated:** [May 29, 2016, 10:13pm UTC](https://discuss.elastic.co/t/cloning-events-from-items-in-an-array/51256 "2016-05-29T22:13:15Z")
I've seen this done previously with JSON docs. However, I have data coming in as comma separated values like so: Created jobgroup: XYZ containing jobid(s): ABC,DEF,GHI,JKL,MNO While I want the jobgroup, I want the jo… --- ## [Logstash at 100% CPU, slow to process Redis queue to Elasticsearch](https://discuss.elastic.co/t/logstash-at-100-cpu-slow-to-process-redis-queue-to-elasticsearch/49686)
**Author:** [@RCMMike](https://discuss.elastic.co/u/RCMMike)\ **Replies:** 2\ **Last updated:** [May 29, 2016, 8:17pm UTC](https://discuss.elastic.co/t/logstash-at-100-cpu-slow-to-process-redis-queue-to-elasticsearch/49686 "2016-05-29T20:17:19Z")
Hello! I have a Logstash instance that is reading data from a Redis queue, processing it via some grok filters, and then outputting to Elasticsearch. For a day or two, this was working properly - decent CPU but not ov… --- ## [Timestamp from logfile lacks millisec](https://discuss.elastic.co/t/timestamp-from-logfile-lacks-millisec/51137)
**Author:** [@dasm](https://discuss.elastic.co/u/dasm)\ **Replies:** 4\ **Last updated:** [May 29, 2016, 8:02pm UTC](https://discuss.elastic.co/t/timestamp-from-logfile-lacks-millisec/51137 "2016-05-29T20:02:30Z")
I have logbeat installed on my windows nodes to forward logfiles to logstash. In logstash, I grab the date/time from the logline and insert it into @timestamp. This is all working, but since the date/time in my logline h… --- ## [Logstash: Problem in date converting](https://discuss.elastic.co/t/logstash-problem-in-date-converting/51050)
**Author:** [@Piyush\_Paritosh](https://discuss.elastic.co/u/Piyush_Paritosh)\ **Replies:** 4\ **Last updated:** [May 29, 2016, 6:37pm UTC](https://discuss.elastic.co/t/logstash-problem-in-date-converting/51050 "2016-05-29T18:37:03Z")
A line from my logfile: 63001,MAt,1420396131,20-SEP-11 12.12.38.000000 AM,RecordSource,null,Can not be null,null I want to convert --\> 20-SEP-11 12.12.38.000000 AM into date time format and put it into @timestamp f… --- ## [Parsing a querystring from nginx](https://discuss.elastic.co/t/parsing-a-querystring-from-nginx/51248)
**Author:** [@john\_6](https://discuss.elastic.co/u/john_6)\ **Replies:** 2\ **Last updated:** [May 29, 2016, 3:18pm UTC](https://discuss.elastic.co/t/parsing-a-querystring-from-nginx/51248 "2016-05-29T15:18:01Z")
I have the following logstash filter so far: filter { if \[type\] == "nginx-access" { grok { match =\> { "message" =\> "%{NGINXACCESS}" } } useragent { source =\> "agent" target =\> "ua" }… --- ## [Cannot install elasticsearch-head plugin](https://discuss.elastic.co/t/cannot-install-elasticsearch-head-plugin/51247)
**Author:** [@BanksySan](https://discuss.elastic.co/u/BanksySan)\ **Replies:** 1\ **Last updated:** [May 29, 2016, 2:33pm UTC](https://discuss.elastic.co/t/cannot-install-elasticsearch-head-plugin/51247 "2016-05-29T14:33:37Z")
I'm working my way through The Logstash Book and am just at the bit where I install my very first plugin. When I try to run it I get the following error though: C:\\logstash-2.3.2\\bin\>logstash-plugin install mobz/elast… --- ## [Twitter plugin retweeted is always false](https://discuss.elastic.co/t/twitter-plugin-retweeted-is-always-false/51237)
**Author:** [@Gautam\_S\_Thakur](https://discuss.elastic.co/u/Gautam_S_Thakur)\ **Replies:** 1\ **Last updated:** [May 29, 2016, 12:05pm UTC](https://discuss.elastic.co/t/twitter-plugin-retweeted-is-always-false/51237 "2016-05-29T12:05:08Z")
Trying Twitter plugin for Logstash. I am wondering why the plugin is unable to capture retweets? The mapping value for retweeted Twitter is always false --- "\> retweeted": false --- ## [Twitter Bounding - How to distinguish among more than one bounding box coordinates?](https://discuss.elastic.co/t/twitter-bounding-how-to-distinguish-among-more-than-one-bounding-box-coordinates/51183)
**Author:** [@Gautam\_S\_Thakur](https://discuss.elastic.co/u/Gautam_S_Thakur)\ **Replies:** 1\ **Last updated:** [May 29, 2016, 11:36am UTC](https://discuss.elastic.co/t/twitter-bounding-how-to-distinguish-among-more-than-one-bounding-box-coordinates/51183 "2016-05-29T11:36:38Z")
First, thank you for adding "location" featuring bounding box for Twitter. As per the description -- A comma-separated list of longitude, latitude pairs specifying a set of bounding boxes to filter tweets by. See Consu… --- ## [Input mariadb 5.5.47 in logstash 2.3](https://discuss.elastic.co/t/input-mariadb-5-5-47-in-logstash-2-3/51244)
**Author:** [@srneodani](https://discuss.elastic.co/u/srneodani)\ **Replies:** 1\ **Last updated:** [May 29, 2016, 11:06am UTC](https://discuss.elastic.co/t/input-mariadb-5-5-47-in-logstash-2-3/51244 "2016-05-29T11:06:44Z")
Hi, I was setting logstash to load data from my data base MariaDB, but no way. I installed the plugins: logstash-output-influxdb logstash-input-jdbc And I downloaded the connectors mysql-connector-java-5.1.36-bin.j… --- ## [Date format issue for two different log types into same datefield in logstash](https://discuss.elastic.co/t/date-format-issue-for-two-different-log-types-into-same-datefield-in-logstash/50952)
**Author:** [@jyothish](https://discuss.elastic.co/u/jyothish)\ **Replies:** 2\ **Last updated:** [May 29, 2016, 10:25am UTC](https://discuss.elastic.co/t/date-format-issue-for-two-different-log-types-into-same-datefield-in-logstash/50952 "2016-05-29T10:25:12Z")
I have a common timedate field named TIME\_END for two different date format for two different log types ,, in the first format it comes in UNIX type eg : 1460555937090 (millisecond+last 3 digits in nanoseconds) a… --- ## [Why doesn't this plugin add messages to the queue?](https://discuss.elastic.co/t/why-doesnt-this-plugin-add-messages-to-the-queue/51229)
**Author:** [@spuder](https://discuss.elastic.co/u/spuder)\ **Replies:** 1\ **Last updated:** [May 28, 2016, 9:54pm UTC](https://discuss.elastic.co/t/why-doesnt-this-plugin-add-messages-to-the-queue/51229 "2016-05-28T21:54:54Z")
I'm trying to write my first logstash input plugin by following the guide. It uses the mqtt ruby gem to fetch from a mqtt message bus. The method 'run' is able to fetch from mqtt, and the 'puts' statement shows the me… --- ## [Two Logstash Instances - Windows](https://discuss.elastic.co/t/two-logstash-instances-windows/50724)
**Author:** [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Replies:** 9\ **Last updated:** [May 28, 2016, 6:13am UTC](https://discuss.elastic.co/t/two-logstash-instances-windows/50724 "2016-05-28T06:13:58Z")
Hi, I'm trying to run two instances of Logstash 1.5.0 on W2012 server. Getting:- Error: Exception thrown by the agent : java.rmi.server.ExportException: Port already in use: 9010; nested exception is: java.net.… --- ## [Logstash local plugin Install Issues](https://discuss.elastic.co/t/logstash-local-plugin-install-issues/44006)
**Author:** [@Kryten](https://discuss.elastic.co/u/Kryten)\ **Replies:** 1\ **Last updated:** [May 28, 2016, 6:12am UTC](https://discuss.elastic.co/t/logstash-local-plugin-install-issues/44006 "2016-05-28T06:12:46Z")
I would really appreciate some input regarding how to install locally manufactured plugins. There seems to be a bit of a black hole in this area where Logstash (testing with 2.2) is telling me that the plugin installed o… --- ## [Pipeline batch size in logstash](https://discuss.elastic.co/t/pipeline-batch-size-in-logstash/51154)
**Author:** [@baneshbanna](https://discuss.elastic.co/u/baneshbanna)\ **Replies:** 2\ **Last updated:** [May 28, 2016, 5:00am UTC](https://discuss.elastic.co/t/pipeline-batch-size-in-logstash/51154 "2016-05-28T05:00:16Z")
How to give pipeline batch size when we run logstash as a service. When we run as a instance we can give as like -b, --pipeline-batch-size But when running as service how to give. Thanks in advance. [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1147) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1149)