# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=681 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 682 --- ## [SQL Server Sync with Elasticsearch through Logstash - retransfer does not happens](https://discuss.elastic.co/t/sql-server-sync-with-elasticsearch-through-logstash-retransfer-does-not-happens/148851)
**Author:** [@Ganesh\_Ram\_Ravi](https://discuss.elastic.co/u/Ganesh_Ram_Ravi)\ **Replies:** 1\ **Last updated:** [September 24, 2018, 2:18pm UTC](https://discuss.elastic.co/t/sql-server-sync-with-elasticsearch-through-logstash-retransfer-does-not-happens/148851 "2018-09-24T14:18:50Z")
Background: We are doing a POC of Sync SQL Server error log data to elasticsearch(ES) to bring a dashboard in kibana. I used Logstash with jdbc input plugin to move sql server table data to (ES), it was succeeded. In … --- ## [Trimming the URI Path from end](https://discuss.elastic.co/t/trimming-the-uri-path-from-end/147213)
**Author:** [@satish\_kumar1](https://discuss.elastic.co/u/satish_kumar1)\ **Replies:** 2\ **Last updated:** [September 24, 2018, 12:58pm UTC](https://discuss.elastic.co/t/trimming-the-uri-path-from-end/147213 "2018-09-24T12:58:31Z")
Hi All, I am new to logstash, I am working on creating visualization of top API based on AWS ELB logs, and i am stucked at this point where i need to trim URIPath. example request: https://www.google.com/test/v1/api/00… --- ## [First time trying to load data from mysql db to elasticsearch, need help with config file](https://discuss.elastic.co/t/first-time-trying-to-load-data-from-mysql-db-to-elasticsearch-need-help-with-config-file/149673)
**Author:** [@Prakti](https://discuss.elastic.co/u/Prakti)\ **Replies:** 0\ **Last updated:** [September 24, 2018, 12:35pm UTC](https://discuss.elastic.co/t/first-time-trying-to-load-data-from-mysql-db-to-elasticsearch-need-help-with-config-file/149673 "2018-09-24T12:35:21Z")
Hi all, I'm new to ELK and I installed everything. I now want to fetch my first table from a mySQL db. I created a sample.conf file whitch I run from my command window with: C:\\...\\logstash-6.4.1\\bin\\logstash.bat -f sa… --- ## [Last line of logs is not reflected in Logstash and Kibana](https://discuss.elastic.co/t/last-line-of-logs-is-not-reflected-in-logstash-and-kibana/149667)
**Author:** [@shubhimu](https://discuss.elastic.co/u/shubhimu)\ **Replies:** 0\ **Last updated:** [September 24, 2018, 12:28pm UTC](https://discuss.elastic.co/t/last-line-of-logs-is-not-reflected-in-logstash-and-kibana/149667 "2018-09-24T12:28:09Z")
Hi, I am reading log4j logs using filebeat as a service. Everything is working fine for me but last line of logs is not shown in logstash and kibana. Can someone please tell me what should i do so that last line should… --- ## [Cannot see any output on my kibana dashboard](https://discuss.elastic.co/t/cannot-see-any-output-on-my-kibana-dashboard/149650)
**Author:** [@pramodchoudhari](https://discuss.elastic.co/u/pramodchoudhari)\ **Replies:** 15\ **Last updated:** [September 24, 2018, 11:54am UTC](https://discuss.elastic.co/t/cannot-see-any-output-on-my-kibana-dashboard/149650 "2018-09-24T11:54:10Z")
This is my logstash.conf file:- input { beats { port =\> 5044 } } filter { if \[fields\]\[log\_type\] == "syslog" { grok { match =\> { "message" =\> "(#!TOT:)?(320:)(%{NUMBER:st1}:)\*%{NUMBER:st2} %{NUMBER:s… --- ## [JSON array parsing in Logstash using the ruby filter](https://discuss.elastic.co/t/json-array-parsing-in-logstash-using-the-ruby-filter/146829)
**Author:** [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Replies:** 10\ **Last updated:** [September 24, 2018, 11:26am UTC](https://discuss.elastic.co/t/json-array-parsing-in-logstash-using-the-ruby-filter/146829 "2018-09-24T11:26:55Z")
Hi All, I have to parse json logs using Logstash. I have a file with my JSON data. I am using the following config and it works perfectly fine for most cases. input { file { path =\> "/opt/logs/\*.log" sincedb\_… --- ## [Failed to receive http input](https://discuss.elastic.co/t/failed-to-receive-http-input/149640)
**Author:** [@johnson\_mui](https://discuss.elastic.co/u/johnson_mui)\ **Replies:** 1\ **Last updated:** [September 24, 2018, 9:09am UTC](https://discuss.elastic.co/t/failed-to-receive-http-input/149640 "2018-09-24T09:09:27Z")
I have the following logstash conf: input { http { port =\> 8080 ssl =\> true ssl\_certificate =\> "/etc/logstash/keys/certificate.crt" ssl\_key =\> "/etc/logstash/keys/certificate.key" ssl\_verify\_mode =\> "force\_peer" … --- ## [Logstash km filter](https://discuss.elastic.co/t/logstash-km-filter/148857)
**Author:** [@gopisa](https://discuss.elastic.co/u/gopisa)\ **Replies:** 17\ **Last updated:** [September 24, 2018, 8:52am UTC](https://discuss.elastic.co/t/logstash-km-filter/148857 "2018-09-24T08:52:13Z")
Hi All, need help on Km filter. i am using the logstash 2.3V. my logs files look like below } "version" : "123" "Keywords" : "1", "Level" : "Verbose", "Message" : "", "Opcode" : "Info", } logstash conf: input… --- ## [Logstash throws error when loading a centrally-managed pipeline](https://discuss.elastic.co/t/logstash-throws-error-when-loading-a-centrally-managed-pipeline/148818)
**Author:** [@programagor](https://discuss.elastic.co/u/programagor)\ **Replies:** 2\ **Last updated:** [September 24, 2018, 7:29am UTC](https://discuss.elastic.co/t/logstash-throws-error-when-loading-a-centrally-managed-pipeline/148818 "2018-09-24T07:29:10Z")
Greetings I am using centrally managed pipelines, so that I can change the pipeline in all my logstash nodes at once. The configuration is following: xpack.management.enabled: true xpack.management.elasticsearch.url: \[… --- ## [Multiple Grok: add filed of one grok in other groks](https://discuss.elastic.co/t/multiple-grok-add-filed-of-one-grok-in-other-groks/148964)
**Author:** [@richagautam](https://discuss.elastic.co/u/richagautam)\ **Replies:** 3\ **Last updated:** [September 24, 2018, 6:44am UTC](https://discuss.elastic.co/t/multiple-grok-add-filed-of-one-grok-in-other-groks/148964 "2018-09-24T06:44:51Z")
Hi Team, I have multiple grok to extract multiple formats from the log files.My question is how can i add filed of grok in other grok. grok { match =\> {"message" =\> "%{SYSLOGTIMESTAMP:Time} %{GREEDYDATA:host} (?%{WOR… --- ## [How to enable logstash monitoring in 6.4](https://discuss.elastic.co/t/how-to-enable-logstash-monitoring-in-6-4/149618)
**Author:** [@hegdedarsh](https://discuss.elastic.co/u/hegdedarsh)\ **Replies:** 0\ **Last updated:** [September 24, 2018, 5:21am UTC](https://discuss.elastic.co/t/how-to-enable-logstash-monitoring-in-6-4/149618 "2018-09-24T05:21:03Z")
Hello, I have enabled monitoring for elasticsearch,kibana,beats but for logstash i am not able to see anything in the kibana dashboard under monitoring. Can someone please help Please find my logstash.yml file below f… --- ## [How to log only error message from logstash container](https://discuss.elastic.co/t/how-to-log-only-error-message-from-logstash-container/149478)
**Author:** [@netvmdb](https://discuss.elastic.co/u/netvmdb)\ **Replies:** 5\ **Last updated:** [September 23, 2018, 11:14pm UTC](https://discuss.elastic.co/t/how-to-log-only-error-message-from-logstash-container/149478 "2018-09-23T23:14:43Z")
I am running logstash 6.2.4 as docker container. Use default log4j2.properties setting comes with container. But on the host, I am seeing every parsed field in the container log file /var/lib/docker/containers//-json.log… --- ## [Need plugin for G-Suite Logs](https://discuss.elastic.co/t/need-plugin-for-g-suite-logs/146542)
**Author:** [@Ryan.Ivis](https://discuss.elastic.co/u/Ryan.Ivis)\ **Replies:** 3\ **Last updated:** [September 3, 2018, 8:55pm UTC](https://discuss.elastic.co/t/need-plugin-for-g-suite-logs/146542 "2018-09-03T20:55:10Z")
Hey all my business runs the G suite by google. I would love to get the logs from this platform into elastic. They have an API. Sumo logic does this out of the gate. I know there has to be a way. Any ideas? --- ## [Update-rc.d: /etc/init.d/logstash: file does not exist](https://discuss.elastic.co/t/update-rc-d-etc-init-d-logstash-file-does-not-exist/146407)
**Author:** [@Nina\_brown](https://discuss.elastic.co/u/Nina_brown)\ **Replies:** 3\ **Last updated:** [August 29, 2018, 9:29am UTC](https://discuss.elastic.co/t/update-rc-d-etc-init-d-logstash-file-does-not-exist/146407 "2018-08-29T09:29:01Z")
I installed logstash 6.4 on Ubuntu 14, but when I try to run it on boot sudo update-rc.d logstash defaults 96 9 I get the error: 'update-rc.d: /etc/init.d/logstash: file does not exist' looks like it's not running a… --- ## [Logstash Output to Elasticsearch Alias using Rollover API](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-alias-using-rollover-api/149385)
**Author:** [@xmatt](https://discuss.elastic.co/u/xmatt)\ **Replies:** 2\ **Last updated:** [September 23, 2018, 6:14pm UTC](https://discuss.elastic.co/t/logstash-output-to-elasticsearch-alias-using-rollover-api/149385 "2018-09-23T18:14:10Z")
Hi there, I've spent several hours searching the wider internet, including a lot of the "elastic" docs and discussion topics but I still haven't been able to find a clear answer to my question. I intend to use an Index… --- ## [Logstash - With Filebeats and Netflow](https://discuss.elastic.co/t/logstash-with-filebeats-and-netflow/149075)
**Author:** [@Carlos\_Magalhaes](https://discuss.elastic.co/u/Carlos_Magalhaes)\ **Replies:** 5\ **Last updated:** [September 23, 2018, 5:50pm UTC](https://discuss.elastic.co/t/logstash-with-filebeats-and-netflow/149075 "2018-09-23T17:50:28Z")
Hi all, Been working with the ELK stack and loving it. A bit of a learning curve but finally got all working with dashboards etc. I have file beat reporting back from a FW and it is working really well to display my da… --- ## [Help Improving Performance](https://discuss.elastic.co/t/help-improving-performance/149576)
**Author:** [@TestCandidate](https://discuss.elastic.co/u/TestCandidate)\ **Replies:** 6\ **Last updated:** [September 23, 2018, 12:33pm UTC](https://discuss.elastic.co/t/help-improving-performance/149576 "2018-09-23T12:33:52Z")
So I am logging lots of requests coming into my server. However, in order to identify some details about the user making the request, I am matching the request IP and request time to my radius db, and fetching some resu… --- ## [Logstash Config change](https://discuss.elastic.co/t/logstash-config-change/149542)
**Author:** [@amritesh\_mishra](https://discuss.elastic.co/u/amritesh_mishra)\ **Replies:** 1\ **Last updated:** [September 22, 2018, 5:59pm UTC](https://discuss.elastic.co/t/logstash-config-change/149542 "2018-09-22T17:59:41Z")
Hi i am generating a config file and from that config file i am able to parse the data form the log but for the event time which is getting parsed from the log file can we change the format of the date after parsing bef… --- ## [Grok and date plugin](https://discuss.elastic.co/t/grok-and-date-plugin/149505)
**Author:** [@xyz2](https://discuss.elastic.co/u/xyz2)\ **Replies:** 5\ **Last updated:** [September 22, 2018, 5:55pm UTC](https://discuss.elastic.co/t/grok-and-date-plugin/149505 "2018-09-22T17:55:28Z")
I need to pick the time from a file and use that for @timestamp. The format of time in file is below 2018-08-30 00:01:23|abc|xy777777 match =\> \["message","%{TIMESTAMP\_ISO8601:time}|..........\] } date { match =\> \[… --- ## [Logstash 6.4.x : reading multiple files in conf.d](https://discuss.elastic.co/t/logstash-6-4-x-reading-multiple-files-in-conf-d/149515)
**Author:** [@nonozi](https://discuss.elastic.co/u/nonozi)\ **Replies:** 8\ **Last updated:** [September 22, 2018, 5:17pm UTC](https://discuss.elastic.co/t/logstash-6-4-x-reading-multiple-files-in-conf-d/149515 "2018-09-22T17:17:26Z")
Hello I encounter a very special problem on a debian 9.5: 1 my conf files are well read in /etc/logstash/conf.d but as soon as there are several, they are no longer taken into account. Example: juste one file iptables… --- ## [Logstash not processing all logfiles](https://discuss.elastic.co/t/logstash-not-processing-all-logfiles/149472)
**Author:** [@G-K-Patel](https://discuss.elastic.co/u/G-K-Patel)\ **Replies:** 0\ **Last updated:** [September 21, 2018, 1:49pm UTC](https://discuss.elastic.co/t/logstash-not-processing-all-logfiles/149472 "2018-09-21T13:49:45Z")
Dear all, I am using Logstash (6.4.0 in a Docker Image) to unzip (.gz files) and delete the processed file after EOF. My Input filter is like this, input { file { path =\> "/input/\*\*/\*.gz" mode =\> "read" file\_comple… --- ## [Type of Document\_id in logstash](https://discuss.elastic.co/t/type-of-document-id-in-logstash/149549)
**Author:** [@ritwik1994](https://discuss.elastic.co/u/ritwik1994)\ **Replies:** 1\ **Last updated:** [September 22, 2018, 2:29pm UTC](https://discuss.elastic.co/t/type-of-document-id-in-logstash/149549 "2018-09-22T14:29:12Z")
Hi , In my database I am storing primary key as Long so I want my document\_type to be of long type but it is String by default. Can we change the type of document\_type in logstash --- ## [Data and time in Kibana / ES](https://discuss.elastic.co/t/data-and-time-in-kibana-es/149139)
**Author:** [@Carlos\_Magalhaes](https://discuss.elastic.co/u/Carlos_Magalhaes)\ **Replies:** 12\ **Last updated:** [September 22, 2018, 11:24am UTC](https://discuss.elastic.co/t/data-and-time-in-kibana-es/149139 "2018-09-22T11:24:02Z")
Hi all Maybe I am being really stupid but the logs showing up seem to be stored / passed with the wrong timestamp. Let me explain: How I noticed this - When building a search filter etc I noticed there were no records… --- ## [Is Logstash executing config sequentially](https://discuss.elastic.co/t/is-logstash-executing-config-sequentially/146642)
**Author:** [@74db36a597f21b891b3f](https://discuss.elastic.co/u/74db36a597f21b891b3f)\ **Replies:** 13\ **Last updated:** [September 22, 2018, 11:11am UTC](https://discuss.elastic.co/t/is-logstash-executing-config-sequentially/146642 "2018-09-22T11:11:07Z")
Hi there. I have two Elasticsearch clusters so Logstash has two output definitions for every index in the way like this: if \[type\] == "iis" { elasticsearch { hosts =\> \["NODE1:9200", "NODE2:9200"\] index =\> "logstas… --- ## [Parsing multiline log](https://discuss.elastic.co/t/parsing-multiline-log/149539)
**Author:** [@tito32](https://discuss.elastic.co/u/tito32)\ **Replies:** 0\ **Last updated:** [September 22, 2018, 10:18am UTC](https://discuss.elastic.co/t/parsing-multiline-log/149539 "2018-09-22T10:18:25Z")
HI! I have some huge multiline logs from vendor programm with different info on each line. All lines have several patterns to match. Almost each line starts with INFO\\DEBUG word after that it has some datetime numbers p… --- ## [Rsyslog + logstash issue -rsyslog stalls when logstash -elasticsearch communication breaks](https://discuss.elastic.co/t/rsyslog-logstash-issue-rsyslog-stalls-when-logstash-elasticsearch-communication-breaks/148965)
**Author:** [@ajayraghuraj](https://discuss.elastic.co/u/ajayraghuraj)\ **Replies:** 2\ **Last updated:** [September 22, 2018, 6:35am UTC](https://discuss.elastic.co/t/rsyslog-logstash-issue-rsyslog-stalls-when-logstash-elasticsearch-communication-breaks/148965 "2018-09-22T06:35:19Z")
Hello, Here is a rsyslog-logstash integration scenario. rsyslog is pumping filtered messages to logstash . logstash in turn sends data to elasticsearch. Something like a storage full event occurs on the elasticsearch… --- ## [Grok : Extract multiple lines format occurring between two lines of log data](https://discuss.elastic.co/t/grok-extract-multiple-lines-format-occurring-between-two-lines-of-log-data/148962)
**Author:** [@richagautam](https://discuss.elastic.co/u/richagautam)\ **Replies:** 1\ **Last updated:** [September 21, 2018, 9:21pm UTC](https://discuss.elastic.co/t/grok-extract-multiple-lines-format-occurring-between-two-lines-of-log-data/148962 "2018-09-21T21:21:04Z")
Hi Team, I am trying to parse multiple log files where log syntax are not same and extract differnt lines which is coming between 2 lines of log data. Below is the sample data Feb 12 18:24:26 localhost.localdomain: TC … --- ## [Looking to consume data from a rejected message queue](https://discuss.elastic.co/t/looking-to-consume-data-from-a-rejected-message-queue/149514)
**Author:** [@reubendevries](https://discuss.elastic.co/u/reubendevries)\ **Replies:** 0\ **Last updated:** [September 21, 2018, 8:24pm UTC](https://discuss.elastic.co/t/looking-to-consume-data-from-a-rejected-message-queue/149514 "2018-09-21T20:24:22Z")
Hi we have use Rabbit MQ that sends data from point a to point b. If for some reason a message or piece of data doesn't make it from point a to point b, it goes into a retry queue. If the same message fails three times i… --- ## [Logstash or Elasticsearch slows down when sending logs to multiple indexes](https://discuss.elastic.co/t/logstash-or-elasticsearch-slows-down-when-sending-logs-to-multiple-indexes/149476)
**Author:** [@byoungb](https://discuss.elastic.co/u/byoungb)\ **Replies:** 4\ **Last updated:** [September 21, 2018, 6:57pm UTC](https://discuss.elastic.co/t/logstash-or-elasticsearch-slows-down-when-sending-logs-to-multiple-indexes/149476 "2018-09-21T18:57:30Z")
I am really not sure if this is a logstash issue or an elasticsearch issue. I have a logstash set up with 3 pipelines each with their own filebeat input, each with logs coming from a different website, logstash then doe… --- ## [Logstash filter section - "if" condition doesn't work](https://discuss.elastic.co/t/logstash-filter-section-if-condition-doesnt-work/149459)
**Author:** [@Ged](https://discuss.elastic.co/u/Ged)\ **Replies:** 1\ **Last updated:** [September 21, 2018, 2:17pm UTC](https://discuss.elastic.co/t/logstash-filter-section-if-condition-doesnt-work/149459 "2018-09-21T14:17:17Z")
Hi, I have problem with dropping events on condition that one of fields has specific value set. Fields is extracted using xml filter plugin. According to below my understanding is all events except of those with xmlReq… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=680) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=682)