# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=844 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 845 --- ## [SSL Authentication to RabbitMQ](https://discuss.elastic.co/t/ssl-authentication-to-rabbitmq/114743)
**Author:** [@Shane\_Eddy](https://discuss.elastic.co/u/Shane_Eddy)\ **Replies:** 0\ **Last updated:** [January 9, 2018, 6:13pm UTC](https://discuss.elastic.co/t/ssl-authentication-to-rabbitmq/114743 "2018-01-09T18:13:08Z")
We are trying to get our logstatsh rabbitmq plugin to authenticate to our RabbitMQ instance using PKI credentials. We've updated our logstash-rmq.config file with the following: HOST=localhost PORT=5671 VHOST=/logstas… --- ## [Regex, logstash and passwords](https://discuss.elastic.co/t/regex-logstash-and-passwords/112322)
**Author:** [@mathurin68](https://discuss.elastic.co/u/mathurin68)\ **Replies:** 4\ **Last updated:** [January 9, 2018, 3:17pm UTC](https://discuss.elastic.co/t/regex-logstash-and-passwords/112322 "2018-01-09T15:17:19Z")
So I'm looking for a way to detect when someone enters their password and then subst out the password for hashes. The 1.1.1.8 is an example, it could be any ip address. From this I want to detect if there is a password … --- ## [Graylog Spring Boot with Logback](https://discuss.elastic.co/t/graylog-spring-boot-with-logback/114721)
**Author:** [@anas90](https://discuss.elastic.co/u/anas90)\ **Replies:** 0\ **Last updated:** [January 9, 2018, 2:47pm UTC](https://discuss.elastic.co/t/graylog-spring-boot-with-logback/114721 "2018-01-09T14:47:15Z")
I create spring boot application i used logback implementation i want send the output logs to "graylog" so what i using the "logstash-gelf" as dependency , then i changes the logback.xml as below : test \ **Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 8\ **Last updated:** [January 9, 2018, 1:57pm UTC](https://discuss.elastic.co/t/why-does-logstash-not-restart-from-jdbc-i-o-errors/114422 "2018-01-09T13:57:55Z") For some reason lately I've been experiencing JDBC input I/O errors in Logstash's logs, but Logstash evidently doesn't do anything when it encounters these. It just hangs and the pipeline can't continue anymore. I have t… --- ## [Apache Log and \_geoip\_lookup\_failure error](https://discuss.elastic.co/t/apache-log-and-geoip-lookup-failure-error/114679)
**Author:** [@ssppune](https://discuss.elastic.co/u/ssppune)\ **Replies:** 2\ **Last updated:** [January 9, 2018, 11:29am UTC](https://discuss.elastic.co/t/apache-log-and-geoip-lookup-failure-error/114679 "2018-01-09T11:29:00Z")
I am using Apache behind HAProxy. The sample log file is as follows 194.1.143.146, 10.80.4.30 - - \[13/Dec/2017:15:08:12 +0530\] "GET /Portal/themes/js/language/showjs.js HTTP/1.1" 200 7106 "https://myserver.com/search/?se… --- ## [Why 1gb data of syslog files are taking hours to upload? Where I made mistake?](https://discuss.elastic.co/t/why-1gb-data-of-syslog-files-are-taking-hours-to-upload-where-i-made-mistake/114695)
**Author:** [@Hari\_Baskar](https://discuss.elastic.co/u/Hari_Baskar)\ **Replies:** 0\ **Last updated:** [January 9, 2018, 11:20am UTC](https://discuss.elastic.co/t/why-1gb-data-of-syslog-files-are-taking-hours-to-upload-where-i-made-mistake/114695 "2018-01-09T11:20:51Z")
I am trying to upload 1gb data of syslog files into elasticsearch using logstash. But It's uploading by hours. why it's taking too much time to upload? . --- ## [Multiple CSV files need to upload for same column names](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012)
**Author:** [@Hari\_Baskar](https://discuss.elastic.co/u/Hari_Baskar)\ **Replies:** 9\ **Last updated:** [January 9, 2018, 11:17am UTC](https://discuss.elastic.co/t/multiple-csv-files-need-to-upload-for-same-column-names/114012 "2018-01-09T11:17:00Z")
Hi, I need to upload monthly wise csv data(which contains the same columns) using logstash. So how to upload that and create an index. Thanks in Advance. --- ## [Logstash filter for json is not using jason file date](https://discuss.elastic.co/t/logstash-filter-for-json-is-not-using-jason-file-date/114672)
**Author:** [@rijinmp](https://discuss.elastic.co/u/rijinmp)\ **Replies:** 0\ **Last updated:** [January 9, 2018, 9:48am UTC](https://discuss.elastic.co/t/logstash-filter-for-json-is-not-using-jason-file-date/114672 "2018-01-09T09:48:50Z")
Hi All , I am uploading log from a json file . Logstash filter is working . But the same time 2 times log is uploading to elasticsearch , with log date and current date . example : jason log {"index": {"\_index":"ser… --- ## [Metricbeat and Filebeat two input in one logstash](https://discuss.elastic.co/t/metricbeat-and-filebeat-two-input-in-one-logstash/114674)
**Author:** [@Bikash\_Behuria](https://discuss.elastic.co/u/Bikash_Behuria)\ **Replies:** 1\ **Last updated:** [January 9, 2018, 9:59am UTC](https://discuss.elastic.co/t/metricbeat-and-filebeat-two-input-in-one-logstash/114674 "2018-01-09T09:59:11Z")
Hi , Can I have two input of beat in one logstash like below . The type in File beat is being set in document\_type field in filebeat yml and for metricbeat type , I am still searching how to set from Metric beat YML. Ca… --- ## [Different timezones for same kind of logs](https://discuss.elastic.co/t/different-timezones-for-same-kind-of-logs/114502)
**Author:** [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)\ **Replies:** 1\ **Last updated:** [January 9, 2018, 9:40am UTC](https://discuss.elastic.co/t/different-timezones-for-same-kind-of-logs/114502 "2018-01-09T09:40:45Z")
I have two kind of access being taken from two different locations and on applying filter I am not able to replace my @timestamp with timestamp my logs are(with different timezones) 1.IP - - \[12/Jun/2017:11:52:13 +0200… --- ## [Aggregate push\_previous\_map\_as\_event seems not work](https://discuss.elastic.co/t/aggregate-push-previous-map-as-event-seems-not-work/114498)
**Author:** [@Rayburn](https://discuss.elastic.co/u/Rayburn)\ **Replies:** 1\ **Last updated:** [January 9, 2018, 8:55am UTC](https://discuss.elastic.co/t/aggregate-push-previous-map-as-event-seems-not-work/114498 "2018-01-09T08:55:27Z")
I followed the sample 4# of logstash aggregation guide which url is https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html#plugins-filters-aggregate-example4 The log output json is right, data h… --- ## [\[logstash.outputs.nagiosnsca\] TypeError: no implicit conversion of Fixnum into String](https://discuss.elastic.co/t/logstash-outputs-nagiosnsca-typeerror-no-implicit-conversion-of-fixnum-into-string/113024)
**Author:** [@Andrey\_Maraev](https://discuss.elastic.co/u/Andrey_Maraev)\ **Replies:** 1\ **Last updated:** [January 9, 2018, 7:20am UTC](https://discuss.elastic.co/t/logstash-outputs-nagiosnsca-typeerror-no-implicit-conversion-of-fixnum-into-string/113024 "2018-01-09T07:20:58Z")
Hello there! I use logstash 6.1.0 with logstash-output-nagios-nsca plugin for sending Errors from my log to the nagios server. My config is: input { file { path =\> "/opt/mylog.log" start\_position =\> "beginning" } … --- ## [Order of events after Split](https://discuss.elastic.co/t/order-of-events-after-split/114505)
**Author:** [@Bhavani\_Ananth](https://discuss.elastic.co/u/Bhavani_Ananth)\ **Replies:** 0\ **Last updated:** [January 8, 2018, 11:11am UTC](https://discuss.elastic.co/t/order-of-events-after-split/114505 "2018-01-08T11:11:59Z")
Hello There, Find the sample log at the end. I am using the multiline codec in logstash to combine the lines without timestamp with tasks and then using split to get the following fields timestamp, processid, user, ro… --- ## [Logstash Output Websocket with SSL](https://discuss.elastic.co/t/logstash-output-websocket-with-ssl/114628)
**Author:** [@sisrael](https://discuss.elastic.co/u/sisrael)\ **Replies:** 0\ **Last updated:** [January 8, 2018, 11:56pm UTC](https://discuss.elastic.co/t/logstash-output-websocket-with-ssl/114628 "2018-01-08T23:56:08Z")
Hi, I have logstash running on my webserver with the websocket output plugin. The plugin works great over http but my website uses https. How can I set up logstash to allow me to connect via "wss://"? As far as I can te… --- ## [Centralized piplines via rest](https://discuss.elastic.co/t/centralized-piplines-via-rest/114618)
**Author:** [@jupp](https://discuss.elastic.co/u/jupp)\ **Replies:** 2\ **Last updated:** [January 8, 2018, 11:22pm UTC](https://discuss.elastic.co/t/centralized-piplines-via-rest/114618 "2018-01-08T23:22:38Z")
I am wondering if it is possible to create/update centralized pipelines via a rest api or is it only possible to manage this pipeline in kibana? --- ## [Data is sent to created index and the default index](https://discuss.elastic.co/t/data-is-sent-to-created-index-and-the-default-index/113845)
**Author:** [@motts](https://discuss.elastic.co/u/motts)\ **Replies:** 5\ **Last updated:** [January 8, 2018, 9:18pm UTC](https://discuss.elastic.co/t/data-is-sent-to-created-index-and-the-default-index/113845 "2018-01-08T21:18:09Z")
I have an interesting issue where data is being sent to the new index and also the old default index. I am using Elasticsearch 2.4 and here is my output file: output { if \[type\] == "Corporate" { elasti… --- ## [Logstash pipeline fails to start with an existing persistent queue](https://discuss.elastic.co/t/logstash-pipeline-fails-to-start-with-an-existing-persistent-queue/112923)
**Author:** [@eheinrich](https://discuss.elastic.co/u/eheinrich)\ **Replies:** 7\ **Last updated:** [January 8, 2018, 9:03pm UTC](https://discuss.elastic.co/t/logstash-pipeline-fails-to-start-with-an-existing-persistent-queue/112923 "2018-01-08T21:03:09Z")
Hi, We've had some cases where something happens, our logstash pipeline crashes and attempts to restart, but is unable to start up again because it fails to create the persistent queue. When this happens, I have to go i… --- ## [Grok Pattern DNS Querys send via syslogd](https://discuss.elastic.co/t/grok-pattern-dns-querys-send-via-syslogd/114512)
**Author:** [@A.Klos](https://discuss.elastic.co/u/A.Klos)\ **Replies:** 1\ **Last updated:** [January 8, 2018, 8:35pm UTC](https://discuss.elastic.co/t/grok-pattern-dns-querys-send-via-syslogd/114512 "2018-01-08T20:35:14Z")
Hi, we have a DNS server and send all dns queries to logstash / elastic. The message line seems to be like this: \<30\>Jan 8 12:32:35 10.243.XX.XX named\[21826\]: queries: client 10.243.YY.YY#43789 (111.236.ZZ.ZZ.in-addr… --- ## [Logstash 6.1 Fatal error crashing background running logstash when stdout output plugin is used](https://discuss.elastic.co/t/logstash-6-1-fatal-error-crashing-background-running-logstash-when-stdout-output-plugin-is-used/114586)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 0\ **Last updated:** [January 8, 2018, 5:40pm UTC](https://discuss.elastic.co/t/logstash-6-1-fatal-error-crashing-background-running-logstash-when-stdout-output-plugin-is-used/114586 "2018-01-08T17:40:33Z")
Hi Team, I'm getting this error while receiving data via the input plugin if stdout output plugin is used when logstash instance has been put to run in the background. It is not a problem per se as I do not intend to u… --- ## [Logstash Kafka Offset Handling](https://discuss.elastic.co/t/logstash-kafka-offset-handling/114575)
**Author:** [@shaharmor](https://discuss.elastic.co/u/shaharmor)\ **Replies:** 0\ **Last updated:** [January 8, 2018, 5:20pm UTC](https://discuss.elastic.co/t/logstash-kafka-offset-handling/114575 "2018-01-08T17:20:17Z")
Hi, I'm witnessing some weird (Or maybe normal) behavior when using Logstash with the Kafka input. Kafka v1.0 Logstash v6.1.1 ES v6.0 1 Logstash server 3 Kafka servers 1 Topic 50 Partitions configuration: inpu… --- ## [SFLOW Codec plugin input format](https://discuss.elastic.co/t/sflow-codec-plugin-input-format/114573)
**Author:** [@weirdaze](https://discuss.elastic.co/u/weirdaze)\ **Replies:** 0\ **Last updated:** [January 8, 2018, 5:14pm UTC](https://discuss.elastic.co/t/sflow-codec-plugin-input-format/114573 "2018-01-08T17:14:04Z")
using logstash 2.3.4 Hello! Newbie to Logstash. Trying to figure out how I should format the input that the sflow codec would accept. so far all I get is (per line of text in my sflow.input file): Ignoring Sflow versi… --- ## [Special characters not supported in delimiter of json\_lines codec](https://discuss.elastic.co/t/special-characters-not-supported-in-delimiter-of-json-lines-codec/114560)
**Author:** [@dimw](https://discuss.elastic.co/u/dimw)\ **Replies:** 0\ **Last updated:** [January 8, 2018, 4:31pm UTC](https://discuss.elastic.co/t/special-characters-not-supported-in-delimiter-of-json-lines-codec/114560 "2018-01-08T16:31:06Z")
According to the documentation of json\_lines codec plugin the default value of the delimiter option is newline (\\n). However, when \\n (or other unicode/special character like \\x00) is explicitly defined in the input conf… --- ## [Usign ruby to update local Linux environment variable](https://discuss.elastic.co/t/usign-ruby-to-update-local-linux-environment-variable/113948)
**Author:** [@moriol](https://discuss.elastic.co/u/moriol)\ **Replies:** 15\ **Last updated:** [January 8, 2018, 2:57pm UTC](https://discuss.elastic.co/t/usign-ruby-to-update-local-linux-environment-variable/113948 "2018-01-08T14:57:44Z")
I am ingesting some data with Logstash, with Kafka as an input. Is it possible with ruby, to configure a filter, which will update a Linux environment variable with what I am getting from Kafka? Lets say I will get som… --- ## [Collecting the logs from servers without installing a collection agent](https://discuss.elastic.co/t/collecting-the-logs-from-servers-without-installing-a-collection-agent/114520)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 1\ **Last updated:** [January 8, 2018, 2:21pm UTC](https://discuss.elastic.co/t/collecting-the-logs-from-servers-without-installing-a-collection-agent/114520 "2018-01-08T14:21:41Z")
Is there a non-intrusive way of gathering logs from the production system? That means no piece of software will be installed in the source systems. Hence, is there any way to collect logs from remote systems and feed to … --- ## [Grok on logstash not working, but working on grokdebugger](https://discuss.elastic.co/t/grok-on-logstash-not-working-but-working-on-grokdebugger/114527)
**Author:** [@anjali](https://discuss.elastic.co/u/anjali)\ **Replies:** 1\ **Last updated:** [January 8, 2018, 1:21pm UTC](https://discuss.elastic.co/t/grok-on-logstash-not-working-but-working-on-grokdebugger/114527 "2018-01-08T13:21:53Z")
Hello, The following grok pattern is working on grokdebugger perfectly. %{DATESTAMP:Timestamp} %{LOGLEVEL:LogLevel} \[%{JAVACLASS:Class}\] (%{GREEDYDATA:MessageID}) %{GREEDYDATA:Error}\\n(?m)%{GREEDYDATA:JavaStackTrace} \[… --- ## [Logstash jdbc input plugin not indexing all data to elasticsearch](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-not-indexing-all-data-to-elasticsearch/114383)
**Author:** [@acuero](https://discuss.elastic.co/u/acuero)\ **Replies:** 1\ **Last updated:** [January 8, 2018, 12:31pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-not-indexing-all-data-to-elasticsearch/114383 "2018-01-08T12:31:38Z")
Hi to everyone! I'm new to elasticsearch and my english is very bad so I will try to explain my issue as well as possible! :grinning: I got a mysql table with more than 300.000 records, but the logstash is indexing onl… --- ## [JDBC can't connect anymore for no apparent reason](https://discuss.elastic.co/t/jdbc-cant-connect-anymore-for-no-apparent-reason/114379)
**Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 1\ **Last updated:** [January 8, 2018, 12:14pm UTC](https://discuss.elastic.co/t/jdbc-cant-connect-anymore-for-no-apparent-reason/114379 "2018-01-08T12:14:13Z")
I keep getting this error: \[2018-01-06T14:53:40,534\]\[ERROR\]\[logstash.inputs.jdbc \] Java::OrgPostgresqlUtil::PSQLException: An I/O error occurred while sending to the backend.: SELECT count(\*) AS "count" FROM (SELEC… --- ## [How to calculate elapsedtime using logstash](https://discuss.elastic.co/t/how-to-calculate-elapsedtime-using-logstash/114481)
**Author:** [@klymptzic](https://discuss.elastic.co/u/klymptzic)\ **Replies:** 1\ **Last updated:** [January 8, 2018, 12:09pm UTC](https://discuss.elastic.co/t/how-to-calculate-elapsedtime-using-logstash/114481 "2018-01-08T12:09:54Z")
Hi Everyone, I need to extract response time of an event by calculating elapsedtime from start and end time excerpts from log: 08-12-17 04:38:17:090 \[INFO\] \[thread-1\] \[User1\] c.s.s.i.a.t.JavaClass1 - Loading Data from … --- ## [Logstash 6.1 removal of mapping types](https://discuss.elastic.co/t/logstash-6-1-removal-of-mapping-types/114321)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 2\ **Last updated:** [January 8, 2018, 10:21am UTC](https://discuss.elastic.co/t/logstash-6-1-removal-of-mapping-types/114321 "2018-01-08T10:21:10Z")
Hi Elastic Team, With info from logstash breaking changes and elasticsearch removal of types doc, does that mean it is now recommended not to use the "type" parameter in input plugins? Many thanks for your help, --- ## [Logstash : Time Field](https://discuss.elastic.co/t/logstash-time-field/114453)
**Author:** [@Rahul\_Pm](https://discuss.elastic.co/u/Rahul_Pm)\ **Replies:** 9\ **Last updated:** [January 8, 2018, 9:52am UTC](https://discuss.elastic.co/t/logstash-time-field/114453 "2018-01-08T09:52:17Z")
Hi , I'm trying to parse a CSV file on my local machine via logstash and visualize it on Kibana. But the date field ("timestamp") is being parsed as a string, and not as a date type. This is how my config looks : input… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=843) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=845)