# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=574 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 575 --- ## [Logstash monitoring info not showing in kibana](https://discuss.elastic.co/t/logstash-monitoring-info-not-showing-in-kibana/170321)
**Author:** [@varun1992](https://discuss.elastic.co/u/varun1992)\ **Replies:** 17\ **Last updated:** [April 1, 2019, 10:12pm UTC](https://discuss.elastic.co/t/logstash-monitoring-info-not-showing-in-kibana/170321 "2019-04-01T22:12:23Z")
i am using ELK version 6.5.4. what i did , i changed elastic cluster settings and added as persistent xpack.monitoring.collection.enabled : true i changed logstash.yml file as xpack.monitoring.enabled: true xpack.mo… --- ## [Logstash date timezone conversion](https://discuss.elastic.co/t/logstash-date-timezone-conversion/174840)
**Author:** [@paano](https://discuss.elastic.co/u/paano)\ **Replies:** 4\ **Last updated:** [April 1, 2019, 9:46pm UTC](https://discuss.elastic.co/t/logstash-date-timezone-conversion/174840 "2019-04-01T21:46:04Z")
in the pipeline conf, used timezone but when viewing in kibana it converts to UTC time. date { timezone =\> "America/Chicago" match =\> \[ "ALTDATE", "yyyy-MM-dd" \] target =\> "ALTDATE" } date { match =\> \[ "CRDATE", "… --- ## [Invalid JSON somewhere in pipeline](https://discuss.elastic.co/t/invalid-json-somewhere-in-pipeline/174820)
**Author:** [@OrangeDog](https://discuss.elastic.co/u/OrangeDog)\ **Replies:** 2\ **Last updated:** [April 1, 2019, 8:00pm UTC](https://discuss.elastic.co/t/invalid-json-somewhere-in-pipeline/174820 "2019-04-01T20:00:42Z")
Can't add multiple categories, but this involves APM as well. Logs flow apm-agent-java -\> stdout -\> journald -\> rsyslog -\> logstash -\> elasticsearch. At the end I end up with a document like this { "message": "{\\"@v… --- ## [Not all log entries show up in the Elasticsearch indices](https://discuss.elastic.co/t/not-all-log-entries-show-up-in-the-elasticsearch-indices/174845)
**Author:** [@Ebisu](https://discuss.elastic.co/u/Ebisu)\ **Replies:** 1\ **Last updated:** [April 1, 2019, 7:24pm UTC](https://discuss.elastic.co/t/not-all-log-entries-show-up-in-the-elasticsearch-indices/174845 "2019-04-01T19:24:55Z")
Sorry for this very general question: I'm using Logstash to parse simple log files. I notice a difference in the number of entries between between what I get by using grep and what shows up in the indices. It's not that … --- ## [Failed to parse field \[flow.bytes\] of type \[long\]](https://discuss.elastic.co/t/failed-to-parse-field-flow-bytes-of-type-long/174843)
**Author:** [@Sghtblinder](https://discuss.elastic.co/u/Sghtblinder)\ **Replies:** 1\ **Last updated:** [April 1, 2019, 7:23pm UTC](https://discuss.elastic.co/t/failed-to-parse-field-flow-bytes-of-type-long/174843 "2019-04-01T19:23:19Z")
Hey Everyone i have been working on our ELK stack the last week and since this past weekend this error has started popping up in my logstash logs and i honestly have no idea what is causing it. \[2019-04-01T13:47:28,143… --- ## [Handling failures with Logstash](https://discuss.elastic.co/t/handling-failures-with-logstash/174854)
**Author:** [@ed.sam13](https://discuss.elastic.co/u/ed.sam13)\ **Replies:** 0\ **Last updated:** [April 1, 2019, 7:09pm UTC](https://discuss.elastic.co/t/handling-failures-with-logstash/174854 "2019-04-01T19:09:12Z")
We have a logstash pipeline with JDBC input and Elastic search output. If the log stash pipeline is scheduled to pick deltas every 15 mins and in 15 mins there are 1000 updates, what happens if the logstash pipeline fai… --- ## [When is sql\_last\_value updated?](https://discuss.elastic.co/t/when-is-sql-last-value-updated/174850)
**Author:** [@ed.sam13](https://discuss.elastic.co/u/ed.sam13)\ **Replies:** 0\ **Last updated:** [April 1, 2019, 7:07pm UTC](https://discuss.elastic.co/t/when-is-sql-last-value-updated/174850 "2019-04-01T19:07:02Z")
We are using Logstash with JDBC input plugin and Elastic search as output. If - for example, we start the full data import from DB at 9am and the import completes at 11am, what gets updated as the sql last value on the f… --- ## [Running Logstash w JDBC input in cloud environment](https://discuss.elastic.co/t/running-logstash-w-jdbc-input-in-cloud-environment/174849)
**Author:** [@ed.sam13](https://discuss.elastic.co/u/ed.sam13)\ **Replies:** 0\ **Last updated:** [April 1, 2019, 7:04pm UTC](https://discuss.elastic.co/t/running-logstash-w-jdbc-input-in-cloud-environment/174849 "2019-04-01T19:04:45Z")
When we run log stash in the cloud environment with multiple nodes, with a JDBC input, how do we ensure that the same deltas are not not picked by multiple nodes? --- ## [Logstash not showing output](https://discuss.elastic.co/t/logstash-not-showing-output/174803)
**Author:** [@peter.tu](https://discuss.elastic.co/u/peter.tu)\ **Replies:** 5\ **Last updated:** [April 1, 2019, 6:28pm UTC](https://discuss.elastic.co/t/logstash-not-showing-output/174803 "2019-04-01T18:28:40Z")
Hey guys, So I'm just trying to create a simple pipeline in logstash to produce a simple output in the 'output' file based on data entered in the 'input' file. The pipeline seems to be fine with no errors, but it's not… --- ## [True/false value form filebeat is ignored in logstash](https://discuss.elastic.co/t/true-false-value-form-filebeat-is-ignored-in-logstash/174833)
**Author:** [@roblopes](https://discuss.elastic.co/u/roblopes)\ **Replies:** 2\ **Last updated:** [April 1, 2019, 6:24pm UTC](https://discuss.elastic.co/t/true-false-value-form-filebeat-is-ignored-in-logstash/174833 "2019-04-01T18:24:14Z")
Hi, I have a filebeat parameter called "keep\_message", where the value should be true or false. In filebeat, it is configured like that: paths: - c:\\temp\\ssis.txt fields\_under\_root: true fields: keep\_m… --- ## [Configure Pipelines in pipelines.yml](https://discuss.elastic.co/t/configure-pipelines-in-pipelines-yml/174251)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 1\ **Last updated:** [April 1, 2019, 5:19pm UTC](https://discuss.elastic.co/t/configure-pipelines-in-pipelines-yml/174251 "2019-04-01T17:19:16Z")
I know we can specify pipelines in pipelines.yml but can we configure them as well? Ideally we could have one config file which is flexible enough to do different things depending on fields we send to it from pipelines.y… --- ## [Replace in Logstash](https://discuss.elastic.co/t/replace-in-logstash/174331)
**Author:** [@Gabriela\_Zielinska](https://discuss.elastic.co/u/Gabriela_Zielinska)\ **Replies:** 1\ **Last updated:** [April 1, 2019, 5:13pm UTC](https://discuss.elastic.co/t/replace-in-logstash/174331 "2019-04-01T17:13:08Z")
Hi I'm trying to replace some strings in field message in with Logstash My logstash configuration: input { beats { port =\> 5044 } } filter { mutate { gsub =\> \["message", "s", "X"\] } } output { elasticsearc… --- ## [Logstash to Logstash Flow, Where to filter the logs](https://discuss.elastic.co/t/logstash-to-logstash-flow-where-to-filter-the-logs/174546)
**Author:** [@Kris.devops](https://discuss.elastic.co/u/Kris.devops)\ **Replies:** 1\ **Last updated:** [April 1, 2019, 5:07pm UTC](https://discuss.elastic.co/t/logstash-to-logstash-flow-where-to-filter-the-logs/174546 "2019-04-01T17:07:39Z")
Hello Team, I am trying to send logs from LOgstash A to Logstash B , and from logstash B to elasticsearch . My question is where do I have to filter the logs and add additional fields to the messages. In logstash A or L… --- ## ["No configuration found in the configured sources": Error when working through LogStash tutorial](https://discuss.elastic.co/t/no-configuration-found-in-the-configured-sources-error-when-working-through-logstash-tutorial/173309)
**Author:** [@Miao](https://discuss.elastic.co/u/Miao)\ **Replies:** 3\ **Last updated:** [April 1, 2019, 4:02pm UTC](https://discuss.elastic.co/t/no-configuration-found-in-the-configured-sources-error-when-working-through-logstash-tutorial/173309 "2019-04-01T16:02:06Z")
I downloaded LogStash 6.6.2 today and I am currently going through the tutorial here. On my Windows 10 machine, I followed the instructions here to install/configure/run Filebeat. I followed every single step in the tu… --- ## [Logstash service is shutting down with Invalid Frame Type 69 and 84](https://discuss.elastic.co/t/logstash-service-is-shutting-down-with-invalid-frame-type-69-and-84/174804)
**Author:** [@arunkasireddi](https://discuss.elastic.co/u/arunkasireddi)\ **Replies:** 4\ **Last updated:** [April 1, 2019, 3:43pm UTC](https://discuss.elastic.co/t/logstash-service-is-shutting-down-with-invalid-frame-type-69-and-84/174804 "2019-04-01T15:43:33Z")
Hi Team, I am trying to set up Logstash service on ECS cluster to push logs to AWS Elasticsearch service. The Logstash container shuts down frequently giving Invalid Frame Type 69 and 84 errors. Any direction on resolv… --- ## [Split array with multiple identical keys](https://discuss.elastic.co/t/split-array-with-multiple-identical-keys/174797)
**Author:** [@jbm](https://discuss.elastic.co/u/jbm)\ **Replies:** 4\ **Last updated:** [April 1, 2019, 2:26pm UTC](https://discuss.elastic.co/t/split-array-with-multiple-identical-keys/174797 "2019-04-01T14:26:14Z")
I am working on splitting out an array that has multiple identical values. The message is json, and I run the whole message through a json filter plugin before trying to parse out the "threatsInfoMap" field. Currently it… --- ## [Logstash tcp output to logstash tcp input, losing metadata?](https://discuss.elastic.co/t/logstash-tcp-output-to-logstash-tcp-input-losing-metadata/174796)
**Author:** [@bjosve](https://discuss.elastic.co/u/bjosve)\ **Replies:** 2\ **Last updated:** [April 1, 2019, 2:15pm UTC](https://discuss.elastic.co/t/logstash-tcp-output-to-logstash-tcp-input-losing-metadata/174796 "2019-04-01T14:15:51Z")
Hi, I have two logstash servers. Logstash#1 is using tcp output to send data to logstash#2 tcp input. It works fine, but the metadata and fields I add in logstash#1 never seems to reach logstash#2. Maybe I'm doing some… --- ## [Multiline logs with space between rows](https://discuss.elastic.co/t/multiline-logs-with-space-between-rows/174745)
**Author:** [@cppatel](https://discuss.elastic.co/u/cppatel)\ **Replies:** 1\ **Last updated:** [April 1, 2019, 2:02pm UTC](https://discuss.elastic.co/t/multiline-logs-with-space-between-rows/174745 "2019-04-01T14:02:24Z")
Hi, I am using filebeat version- 6.2.1. it is sending logs to the logstash to parse it. Now the issue is my log is basically an output of a script so it is not sort of structured. It is something like below. 2 lines ha… --- ## [Unable to read logs from external URIs](https://discuss.elastic.co/t/unable-to-read-logs-from-external-uris/174783)
**Author:** [@Raju\_Bhagat](https://discuss.elastic.co/u/Raju_Bhagat)\ **Replies:** 1\ **Last updated:** [April 1, 2019, 1:47pm UTC](https://discuss.elastic.co/t/unable-to-read-logs-from-external-uris/174783 "2019-04-01T13:47:45Z")
I am using windows 10, using following logstash.conf input { http{ host =\> "https://filesharestracc6870.blob.core.windows.net/log/logfile.log" port =\> 443 codec =\> plain ssl =\> true } } the url is publically acc… --- ## [Logstash Grok Issue with Multi-Line Events](https://discuss.elastic.co/t/logstash-grok-issue-with-multi-line-events/173756)
**Author:** [@cawoodm](https://discuss.elastic.co/u/cawoodm)\ **Replies:** 3\ **Last updated:** [April 1, 2019, 1:46pm UTC](https://discuss.elastic.co/t/logstash-grok-issue-with-multi-line-events/173756 "2019-04-01T13:46:56Z")
We have the following multi-line log file: INFO | jvm 1 | srvmain | 2019/03/25 15:17:00.333 | ERROR \[aaa\] 200\] \[bbb\] foo INFO | jvm 1 | srvmain | 2019/03/25 15:17:00.333 | Mar 05, 2019 2:54:39 PM bar FileBeat… --- ## [“JSON parse error, original data now in message field”](https://discuss.elastic.co/t/json-parse-error-original-data-now-in-message-field/174790)
**Author:** [@Miao](https://discuss.elastic.co/u/Miao)\ **Replies:** 1\ **Last updated:** [April 1, 2019, 1:39pm UTC](https://discuss.elastic.co/t/json-parse-error-original-data-now-in-message-field/174790 "2019-04-01T13:39:11Z")
I use LogStash 6.7.0 and ElasticSearch 6.7.0. Both of them are installed on my remote Azure VM server. I installed the Http\_poller input plugin, and I am currently following the tutorial here. As instructed by the tuto… --- ## [Unknown setting 'protocol' for elasticsearch](https://discuss.elastic.co/t/unknown-setting-protocol-for-elasticsearch/174772)
**Author:** [@Miao](https://discuss.elastic.co/u/Miao)\ **Replies:** 2\ **Last updated:** [April 1, 2019, 1:31pm UTC](https://discuss.elastic.co/t/unknown-setting-protocol-for-elasticsearch/174772 "2019-04-01T13:31:32Z")
I have just installed the HTTP Poller plugin, and I am now following the tutorial here. As instructed by the tutorial, I am using this config: input { http\_poller { urls =\> { "localhost" =\> "http://localhos… --- ## [Issue with logstash parsing mysql logs](https://discuss.elastic.co/t/issue-with-logstash-parsing-mysql-logs/174766)
**Author:** [@Jayanthi\_Bhanuji\_Rao](https://discuss.elastic.co/u/Jayanthi_Bhanuji_Rao)\ **Replies:** 2\ **Last updated:** [April 1, 2019, 1:00pm UTC](https://discuss.elastic.co/t/issue-with-logstash-parsing-mysql-logs/174766 "2019-04-01T13:00:17Z")
Hello, I am using logstash to parse my mysql error logs. I am getting an error "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"object mapping for \[host\] tried to parse field \[host\] as object, but found a concre… --- ## [Logstash - File output error - permission denied](https://discuss.elastic.co/t/logstash-file-output-error-permission-denied/173954)
**Author:** [@HaranKumar](https://discuss.elastic.co/u/HaranKumar)\ **Replies:** 7\ **Last updated:** [April 1, 2019, 12:47pm UTC](https://discuss.elastic.co/t/logstash-file-output-error-permission-denied/173954 "2019-04-01T12:47:08Z")
Hello Team, I am trying to collect a set of data from a URL and save it in a file as json format. below is my logstash configuration file . Kindly help me with troubleshooting where and what i am missing. I am gettin… --- ## [Unicode Characters](https://discuss.elastic.co/t/unicode-characters/174611)
**Author:** [@muraliv](https://discuss.elastic.co/u/muraliv)\ **Replies:** 2\ **Last updated:** [April 1, 2019, 12:46pm UTC](https://discuss.elastic.co/t/unicode-characters/174611 "2019-04-01T12:46:10Z")
Hi All, I apologize if this question has been asked already by other members of this forum. I am pulling events from Kafka and getting data similar to this. Is there any way I could replace these characters with "" ? … --- ## [No implicit conversion of Fixnum into String when adding days to a date](https://discuss.elastic.co/t/no-implicit-conversion-of-fixnum-into-string-when-adding-days-to-a-date/174690)
**Author:** [@roblopes](https://discuss.elastic.co/u/roblopes)\ **Replies:** 2\ **Last updated:** [April 1, 2019, 12:41pm UTC](https://discuss.elastic.co/t/no-implicit-conversion-of-fixnum-into-string-when-adding-days-to-a-date/174690 "2019-04-01T12:41:35Z")
Hi, I am trying to add day(s) to a date in logstash, based on a filebeat prospector configuration, but when I try adding, I get an error like this: \[2019-03-31T16:43:14,624\]\[ERROR\]\[logstash.filters.ruby \] Ruby excep… --- ## [Logstash: Issue with Encode Windows-1252](https://discuss.elastic.co/t/logstash-issue-with-encode-windows-1252/174333)
**Author:** [@pvn\_sreedhar](https://discuss.elastic.co/u/pvn_sreedhar)\ **Replies:** 1\ **Last updated:** [April 1, 2019, 10:13am UTC](https://discuss.elastic.co/t/logstash-issue-with-encode-windows-1252/174333 "2019-04-01T10:13:13Z")
Hi , I am facing an issue with Logstash to a read a file. Original data in the file: r8Dà€ - contains uni-code characters. When read the file in python with the options - (encoding='Windows-1252', errors="surrogate… --- ## [Logstash SSL config from pkcs12 to a working config my odyssey](https://discuss.elastic.co/t/logstash-ssl-config-from-pkcs12-to-a-working-config-my-odyssey/174524)
**Author:** [@gruselglatz](https://discuss.elastic.co/u/gruselglatz)\ **Replies:** 5\ **Last updated:** [April 1, 2019, 9:16am UTC](https://discuss.elastic.co/t/logstash-ssl-config-from-pkcs12-to-a-working-config-my-odyssey/174524 "2019-04-01T09:16:47Z")
Hi, I want to connect from filebeat to logastash via ssl and maybe later it should also use client auth. But the Keyformat for Logstash is an endless odyssey... I came from pkcs12 .pfx. I've read every Post here and on… --- ## [Question about my 2 pipelines](https://discuss.elastic.co/t/question-about-my-2-pipelines/174716)
**Author:** [@Thibaut603](https://discuss.elastic.co/u/Thibaut603)\ **Replies:** 2\ **Last updated:** [April 1, 2019, 8:07am UTC](https://discuss.elastic.co/t/question-about-my-2-pipelines/174716 "2019-04-01T08:07:14Z")
Hello i've some questions i've an elastic stack with many beats on windows file servers, AD, Exchanges. all events are sended to logstash into differents pipelines.for AD on ports 5044 and 5043 for file server I use 2 … --- ## [Logstash Custom Timestamp issue](https://discuss.elastic.co/t/logstash-custom-timestamp-issue/174678)
**Author:** [@lakshykar](https://discuss.elastic.co/u/lakshykar)\ **Replies:** 2\ **Last updated:** [April 1, 2019, 7:48am UTC](https://discuss.elastic.co/t/logstash-custom-timestamp-issue/174678 "2019-04-01T07:48:42Z")
Hi, I am trying to put transaction data into elasticsearch, i want to override @timestamp with actual transaction time, but after converting timestamp , getting different time. below if my conf file: input { file{ … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=573) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=575)