# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=689 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 690 --- ## [Twitter client error: Gateway Timeout](https://discuss.elastic.co/t/twitter-client-error-gateway-timeout/147569)
**Author:** [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Replies:** 2\ **Last updated:** [September 7, 2018, 7:25am UTC](https://discuss.elastic.co/t/twitter-client-error-gateway-timeout/147569 "2018-09-07T07:25:37Z")
Hello, I am getting following error: \[2018-09-06T14:26:15,801\]\[WARN \]\[logstash.inputs.twitter \] Twitter client error {:message=\>"", :exception=\>"Twitter::Error::GatewayTimeout", :backtrace=\>\["/usr/share/logstash/ven… --- ## [Errors when processing (mutating) date values](https://discuss.elastic.co/t/errors-when-processing-mutating-date-values/147345)
**Author:** [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\ **Replies:** 13\ **Last updated:** [September 7, 2018, 7:19am UTC](https://discuss.elastic.co/t/errors-when-processing-mutating-date-values/147345 "2018-09-07T07:19:17Z")
We're attempting to import historical data in CSV format via Logstash, but it's choking on the date values. So far, we have the following config: input { stdin {} file { path =\> \["/Applications/logstash-6.2.4/data/… --- ## [How to add more than one paramter for filter](https://discuss.elastic.co/t/how-to-add-more-than-one-paramter-for-filter/147153)
**Author:** [@anon69830709](https://discuss.elastic.co/u/anon69830709)\ **Replies:** 16\ **Last updated:** [September 7, 2018, 7:09am UTC](https://discuss.elastic.co/t/how-to-add-more-than-one-paramter-for-filter/147153 "2018-09-07T07:09:12Z")
filter { if \[type\] == "access" { grok { match =\> { "message" =\> "%{IPV4:ip} (?(msg "\[A-Za-z \]+))" } } } } how to make it work. No matching is being done here for cause filed, --- ## [List of filebeats which connected to logstash](https://discuss.elastic.co/t/list-of-filebeats-which-connected-to-logstash/147663)
**Author:** [@wonki](https://discuss.elastic.co/u/wonki)\ **Replies:** 2\ **Last updated:** [September 7, 2018, 6:46am UTC](https://discuss.elastic.co/t/list-of-filebeats-which-connected-to-logstash/147663 "2018-09-07T06:46:33Z")
Hi! How can i find the list of filebeats which connected to logstash? Does Logstash have function like this? If not, which staff has these function? I want to see the list of beats that connected specific logstash. I… --- ## [Email output font format](https://discuss.elastic.co/t/email-output-font-format/147549)
**Author:** [@sureshmcsd](https://discuss.elastic.co/u/sureshmcsd)\ **Replies:** 3\ **Last updated:** [September 7, 2018, 6:39am UTC](https://discuss.elastic.co/t/email-output-font-format/147549 "2018-09-07T06:39:32Z")
I want to define the font size of email subject while sending email as output. How can I achieve it. for eg: The word Warning should be font size bold and remaining shall be 10 regular Like : Subject =\> Warning !!! Sy… --- ## [ERR Connecting error publishing events (retrying): dial tcp x.x.x.x:5044: getsockopt: connection refused](https://discuss.elastic.co/t/err-connecting-error-publishing-events-retrying-dial-tcp-x-x-x-x-getsockopt-connection-refused/147645)
**Author:** [@BronQ](https://discuss.elastic.co/u/BronQ)\ **Replies:** 3\ **Last updated:** [September 7, 2018, 6:37am UTC](https://discuss.elastic.co/t/err-connecting-error-publishing-events-retrying-dial-tcp-x-x-x-x-getsockopt-connection-refused/147645 "2018-09-07T06:37:38Z")
Hi All, Please help, Filebeat failed to send logs to logstash with the error below: ERR Connecting error publishing events (retrying): dial tcp x.x.x.x:5044: getsockopt: connection refused Telnet from the client serve… --- ## [Huawei netstream v5 or V9 logstash 6.3.1 Ignoring Netflow version](https://discuss.elastic.co/t/huawei-netstream-v5-or-v9-logstash-6-3-1-ignoring-netflow-version/146981)
**Author:** [@tigerofcn](https://discuss.elastic.co/u/tigerofcn)\ **Replies:** 3\ **Last updated:** [September 7, 2018, 5:52am UTC](https://discuss.elastic.co/t/huawei-netstream-v5-or-v9-logstash-6-3-1-ignoring-netflow-version/146981 "2018-09-07T05:52:21Z")
ip netstream export version 5 \[2018-08-16T16:10:51,220\]\[WARN \]\[logstash.codecs.netflow \] Ignoring Netflow version v32773 \[2018-08-16T16:10:56,202\]\[WARN \]\[logstash.codecs.netflow \] Ignoring Netflow version v32773 \[20… --- ## [Log stash](https://discuss.elastic.co/t/log-stash/147639)
**Author:** [@satish161418](https://discuss.elastic.co/u/satish161418)\ **Replies:** 3\ **Last updated:** [September 7, 2018, 4:18am UTC](https://discuss.elastic.co/t/log-stash/147639 "2018-09-07T04:18:09Z")
i Have logs with three different line like in message,out message and Error. Have written 3 grok patterns for parsing to kibana.pat1,pat2,pat 3. How to write a logstash if loop to parse if it is in message take pat1 ,i… --- ## [S3 input plugin is not able to handle encrypted data](https://discuss.elastic.co/t/s3-input-plugin-is-not-able-to-handle-encrypted-data/147415)
**Author:** [@Satej](https://discuss.elastic.co/u/Satej)\ **Replies:** 3\ **Last updated:** [September 6, 2018, 8:32pm UTC](https://discuss.elastic.co/t/s3-input-plugin-is-not-able-to-handle-encrypted-data/147415 "2018-09-06T20:32:08Z")
Hello All, I have setup the "Kinesis Firehose stream" which accepts the log data from "kinesis data stream" and stores these logs as backup on S3 bucket in gz format. For this configuration I have kept the KMS encrypti… --- ## [Log stash CSV Filter is not reading last line](https://discuss.elastic.co/t/log-stash-csv-filter-is-not-reading-last-line/147208)
**Author:** [@krisdinesh](https://discuss.elastic.co/u/krisdinesh)\ **Replies:** 1\ **Last updated:** [September 6, 2018, 8:09pm UTC](https://discuss.elastic.co/t/log-stash-csv-filter-is-not-reading-last-line/147208 "2018-09-06T20:09:49Z")
Log stash CSV Filter is not reading last line as it is due to the same issue outlined in the below link I added a blank row at the end of csv and saved the file, re-ran the logstash and it worked. As I am using Jira … --- ## [Error In Logstash Filter](https://discuss.elastic.co/t/error-in-logstash-filter/147237)
**Author:** [@marcos.vbp](https://discuss.elastic.co/u/marcos.vbp)\ **Replies:** 5\ **Last updated:** [September 6, 2018, 6:37pm UTC](https://discuss.elastic.co/t/error-in-logstash-filter/147237 "2018-09-06T18:37:20Z")
hi, My log : 172.16.2.211 - - \[04/Set/2018:12:09:48 -0300\] "GET /lmsa/framebottom.do HTTP/1.1" 200 4858 match =\> { message =\> "%{IP:client} %{DATA:USERNAME} %{DATA:USERNAME} \[%{DATA:logtime}\] " %{WORD:request} %{URIPA… --- ## [How can sort assorted logs to steam line logs](https://discuss.elastic.co/t/how-can-sort-assorted-logs-to-steam-line-logs/146943)
**Author:** [@ramindia](https://discuss.elastic.co/u/ramindia)\ **Replies:** 6\ **Last updated:** [September 6, 2018, 6:03pm UTC](https://discuss.elastic.co/t/how-can-sort-assorted-logs-to-steam-line-logs/146943 "2018-09-06T18:03:23Z")
I have below assorted logs, how can i sort to Time Device Source only log message and remove rest all unwanted log message Sep 1 14:40:33 VPX-02 09/01/2018:13:40:33 GMT VPX-02 0-PPE-0 : default API CMD\_EXECUTED 52… --- ## [Elapsed filter not with default timestamp field](https://discuss.elastic.co/t/elapsed-filter-not-with-default-timestamp-field/147249)
**Author:** [@jonathan.keenan](https://discuss.elastic.co/u/jonathan.keenan)\ **Replies:** 1\ **Last updated:** [September 6, 2018, 6:00pm UTC](https://discuss.elastic.co/t/elapsed-filter-not-with-default-timestamp-field/147249 "2018-09-06T18:00:36Z")
Hi there, Its a long story that i cant get into here(!) but the data we are pushing through logstash and further on into elasticsearch is not in real time and is on a regular 10 minute loop by a cron job. Therefore our … --- ## [Dead Letter Queue - Cannot Write Event to DLQ](https://discuss.elastic.co/t/dead-letter-queue-cannot-write-event-to-dlq/146540)
**Author:** [@Jay1](https://discuss.elastic.co/u/Jay1)\ **Replies:** 1\ **Last updated:** [September 6, 2018, 2:26pm UTC](https://discuss.elastic.co/t/dead-letter-queue-cannot-write-event-to-dlq/146540 "2018-09-06T14:26:43Z")
Hello , I have enabled DLQ on two different logstash instances running on AWS EC2 as docker container and everything was working fine almost 6+ months . Last week , my DLQ was filled up with 1GB messages and had to drop … --- ## [Conditional Filter only grok IF](https://discuss.elastic.co/t/conditional-filter-only-grok-if/147458)
**Author:** [@jenyphur](https://discuss.elastic.co/u/jenyphur)\ **Replies:** 4\ **Last updated:** [September 6, 2018, 1:46pm UTC](https://discuss.elastic.co/t/conditional-filter-only-grok-if/147458 "2018-09-06T13:46:48Z")
Hi there, so I am receiving syslogs with different formats. I ONLY want to grok out the ones that have the characters "\[App/" in them. The others, I would like to ignore and NOT send to ES. Here's an example of a log: … --- ## [Updating more than 17 Previous ElasticSearch records Using Logstash Output Plugin is not working as expected](https://discuss.elastic.co/t/updating-more-than-17-previous-elasticsearch-records-using-logstash-output-plugin-is-not-working-as-expected/147571)
**Author:** [@Arjun](https://discuss.elastic.co/u/Arjun)\ **Replies:** 2\ **Last updated:** [September 6, 2018, 1:13pm UTC](https://discuss.elastic.co/t/updating-more-than-17-previous-elasticsearch-records-using-logstash-output-plugin-is-not-working-as-expected/147571 "2018-09-06T13:13:38Z")
Hi Team, I am using Logstash Output Plugin and trying to update 17 previous Elasticsearch records but unfortunately Logstash able to detect and update only 15 records of Elastic search not 17. Upto 16 records Logstash … --- ## [Reverse proxy for Logstash - is this even possible now?](https://discuss.elastic.co/t/reverse-proxy-for-logstash-is-this-even-possible-now/147550)
**Author:** [@pastechecker](https://discuss.elastic.co/u/pastechecker)\ **Replies:** 1\ **Last updated:** [September 6, 2018, 12:17pm UTC](https://discuss.elastic.co/t/reverse-proxy-for-logstash-is-this-even-possible-now/147550 "2018-09-06T12:17:33Z")
Hello, I have Logstash 6.4 installed on Debian 9.5. Recently I had to move Logstash to one zone where the only option to get out to the internet is over reverse proxy. I have multiple http\_pollers in my Logstash config… --- ## [Grok filter pattern for nginx](https://discuss.elastic.co/t/grok-filter-pattern-for-nginx/147520)
**Author:** [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Replies:** 8\ **Last updated:** [September 6, 2018, 12:04pm UTC](https://discuss.elastic.co/t/grok-filter-pattern-for-nginx/147520 "2018-09-06T12:04:15Z")
Hello Team, I am using ELK6.4.0 and filebeat6.4.0. Currently i am sending only my application logs over elasticsearch using filebeat and parse via logstash. But now we want to send the nginx logs (error and access) over… --- ## [\[logstash\] Cannot find the logs that donot match the grok filter](https://discuss.elastic.co/t/logstash-cannot-find-the-logs-that-donot-match-the-grok-filter/147510)
**Author:** [@Zhao](https://discuss.elastic.co/u/Zhao)\ **Replies:** 2\ **Last updated:** [September 6, 2018, 11:22am UTC](https://discuss.elastic.co/t/logstash-cannot-find-the-logs-that-donot-match-the-grok-filter/147510 "2018-09-06T11:22:37Z")
Hello, dear all, I updated the logstash from vesrion 2.1 to 6.4, and I find there are some differences between them which caused my problem. I cannot find any logs that don't match my grok filters in the kibana for v… --- ## [Elasticsearch network configuration](https://discuss.elastic.co/t/elasticsearch-network-configuration/146714)
**Author:** [@arush\_khanna](https://discuss.elastic.co/u/arush_khanna)\ **Replies:** 9\ **Last updated:** [September 6, 2018, 10:43am UTC](https://discuss.elastic.co/t/elasticsearch-network-configuration/146714 "2018-09-06T10:43:53Z")
I'm new to elk stack trying to get my hands dirty. There are two machines - one with logstash where i want to ingest a csv file and store it on another machine where elasticsearch and kibana are installed. I tried ed… --- ## [Logstash skipped half of log file](https://discuss.elastic.co/t/logstash-skipped-half-of-log-file/147422)
**Author:** [@cosmo](https://discuss.elastic.co/u/cosmo)\ **Replies:** 4\ **Last updated:** [September 6, 2018, 10:31am UTC](https://discuss.elastic.co/t/logstash-skipped-half-of-log-file/147422 "2018-09-06T10:31:37Z")
Hello I have a daily process that downloads some compressed log files, and then extracts them into a directory logstash is watching weblogs pipeline; input { file { id =\> "weblogs\_input\_file" pat… --- ## [Logstash on Windows: Slow startup time](https://discuss.elastic.co/t/logstash-on-windows-slow-startup-time/147531)
**Author:** [@Karunakaran](https://discuss.elastic.co/u/Karunakaran)\ **Replies:** 0\ **Last updated:** [September 6, 2018, 8:42am UTC](https://discuss.elastic.co/t/logstash-on-windows-slow-startup-time/147531 "2018-09-06T08:42:41Z")
The problem: When I start Logstash from the command line on my Windows machine, it takes 3 minutes to show me the first message ("Sending Logstash logs to /logstash-6.4.0/logs..."). And it takes further 2 minutes to sta… --- ## [Logstash - Filebeat doesnt parse every line of IIS Logs](https://discuss.elastic.co/t/logstash-filebeat-doesnt-parse-every-line-of-iis-logs/147508)
**Author:** [@lina](https://discuss.elastic.co/u/lina)\ **Replies:** 0\ **Last updated:** [September 6, 2018, 7:17am UTC](https://discuss.elastic.co/t/logstash-filebeat-doesnt-parse-every-line-of-iis-logs/147508 "2018-09-06T07:17:52Z")
Hello, I am trying to parse IIS logs from a log file on the server, the grok filters are not the problem because i tried without it. Logstash parse only half of the lines in the log file, for example : The IIS Log fil… --- ## [Help with creating configuration file](https://discuss.elastic.co/t/help-with-creating-configuration-file/147316)
**Author:** [@anon69830709](https://discuss.elastic.co/u/anon69830709)\ **Replies:** 3\ **Last updated:** [September 6, 2018, 6:13am UTC](https://discuss.elastic.co/t/help-with-creating-configuration-file/147316 "2018-09-06T06:13:16Z")
Its a 2 days and cannot even break the log into section . My log is down below. \[Wed Sep 05 11:45:56.601060 2018\] \[:error\] \[pid 5303:tid 140044044035840\] \[client 192.168.1.3:51297\] \[client 192.168.1.3\] ModSecurity: War… --- ## [Filtering in Logstash for XML data](https://discuss.elastic.co/t/filtering-in-logstash-for-xml-data/147434)
**Author:** [@Bharath\_Pusuluri](https://discuss.elastic.co/u/Bharath_Pusuluri)\ **Replies:** 4\ **Last updated:** [September 6, 2018, 6:15am UTC](https://discuss.elastic.co/t/filtering-in-logstash-for-xml-data/147434 "2018-09-06T06:15:59Z")
Need help in writing the right conf file for the data displayed here. Getting the splittable errors Input file --- from the url \ \12434\ \12279\ **Author:** [@mrubin](https://discuss.elastic.co/u/mrubin)\ **Replies:** 2\ **Last updated:** [September 6, 2018, 6:05am UTC](https://discuss.elastic.co/t/how-to-tune-logstash-for-es-indexing-speed/147476 "2018-09-06T06:05:16Z") I am using Logstash 6.2.x with Kafka as input and ES 6.4.0 (hosted in Elastic Cloud) as the output. I needed to stop Logstash for some time while by ES cluster was being upgraded. Once I restarted Logstash (and there wer… --- ## [Sudo bin/logstash -f /etc/logstash/conf.d/logstash.conf --path.settings /etc/logstash](https://discuss.elastic.co/t/sudo-bin-logstash-f-etc-logstash-conf-d-logstash-conf-path-settings-etc-logstash/147445)
**Author:** [@ElastiSid](https://discuss.elastic.co/u/ElastiSid)\ **Replies:** 5\ **Last updated:** [September 5, 2018, 11:02pm UTC](https://discuss.elastic.co/t/sudo-bin-logstash-f-etc-logstash-conf-d-logstash-conf-path-settings-etc-logstash/147445 "2018-09-05T23:02:19Z")
I am getting error. Logstash is not really working - Elasticsearch has no feed. Ubuntu 16.04 LTS, Java 1.8, ES 6., LS 6. /usr/share/logstash/bin$ ls -l total 68 -rwxr-xr-x 1 logstash logstash 152 Aug 17 17:21 benc… --- ## [HTML log file / PARSE FAILURE](https://discuss.elastic.co/t/html-log-file-parse-failure/147264)
**Author:** [@robegome](https://discuss.elastic.co/u/robegome)\ **Replies:** 1\ **Last updated:** [September 5, 2018, 7:14pm UTC](https://discuss.elastic.co/t/html-log-file-parse-failure/147264 "2018-09-05T19:14:55Z")
Hello everyone! I am trying to get a dashboard in Kibana of several HTML LOG files... the patter of those logs are like this example: \ \ … --- ## [HTTP input plugin - how to avoid Logstash to pass through preflight OPTIONS to Elasticsearch](https://discuss.elastic.co/t/http-input-plugin-how-to-avoid-logstash-to-pass-through-preflight-options-to-elasticsearch/147462)
**Author:** [@tianyid](https://discuss.elastic.co/u/tianyid)\ **Replies:** 0\ **Last updated:** [September 5, 2018, 6:59pm UTC](https://discuss.elastic.co/t/http-input-plugin-how-to-avoid-logstash-to-pass-through-preflight-options-to-elasticsearch/147462 "2018-09-05T18:59:33Z")
Hello, I have web application who is using JQuery Ajax to POST event data of JSON format to Logstash's HTTP input. Here is the snippet of Ajax call. $.ajax({ url: logstashUrl, … --- ## [Help to grok](https://discuss.elastic.co/t/help-to-grok/147189)
**Author:** [@sreejiths](https://discuss.elastic.co/u/sreejiths)\ **Replies:** 2\ **Last updated:** [September 5, 2018, 7:07pm UTC](https://discuss.elastic.co/t/help-to-grok/147189 "2018-09-05T19:07:18Z")
Below is the Input log to the logstash ..How to parse this ..How to ignore spaces & special character ":" ..Can anyone provide some tips please .. Log Name: n01firewall\_fw01-vsx,Security Policy: fw01-active-14MAR2018,C… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=688) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=690)