# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=484 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 485 --- ## [Input File plugin permissions](https://discuss.elastic.co/t/input-file-plugin-permissions/197515)
**Author:** [@nujojo](https://discuss.elastic.co/u/nujojo)\ **Replies:** 2\ **Last updated:** [September 2, 2019, 7:09am UTC](https://discuss.elastic.co/t/input-file-plugin-permissions/197515 "2019-09-02T07:09:36Z")
Hi Guys, Hope you can help. I have configured Logstash Input File plugin to read the files and send further. Application is creating the file logs.log in /home/user/logs.log with permissions: -rw-r----- Logstash is … --- ## [Using aggregate filter to merge different events](https://discuss.elastic.co/t/using-aggregate-filter-to-merge-different-events/197619)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 3\ **Last updated:** [September 1, 2019, 3:31pm UTC](https://discuss.elastic.co/t/using-aggregate-filter-to-merge-different-events/197619 "2019-09-01T15:31:42Z")
Hi all, there is a log which some evenets of it have a same task\_id A1=12;A2=gty;A3=tsbr;A4=5798 B1=adr;B2=156;B3=765 A1=12;A2=gty;A3=tsbr;A4=5798;A5=895 B1=adr;B2=156;B3=765;B4=khu as following: task\_id=1 =\> A1=… --- ## [How to read SQL Server log trace(log.trc)?](https://discuss.elastic.co/t/how-to-read-sql-server-log-trace-log-trc/197613)
**Author:** [@sundar\_elk](https://discuss.elastic.co/u/sundar_elk)\ **Replies:** 0\ **Last updated:** [September 1, 2019, 6:16am UTC](https://discuss.elastic.co/t/how-to-read-sql-server-log-trace-log-trc/197613 "2019-09-01T06:16:08Z")
Hi Team, Use case :- Reading SQL server log trace file. file name and extension :- log.trc file path :- Program Files/Microsoft SQL Server/MSSQL13.MSSQLSERVER/MSSQL/Log/log\_64.trc I tried to read .trc file directly f… --- ## [How to separate fields in different logs using logstash](https://discuss.elastic.co/t/how-to-separate-fields-in-different-logs-using-logstash/197581)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 2\ **Last updated:** [August 31, 2019, 1:08pm UTC](https://discuss.elastic.co/t/how-to-separate-fields-in-different-logs-using-logstash/197581 "2019-08-31T13:08:49Z")
hi all, I have different lines of logs for example as following: A1=5454;A2=34757354575;A3=setoh;A4=NU;A5=(555);A6=\[78925\] B1=get;B2=589641;B3=258 C1=(5472);C2=LA;C3=Mid;C4=0000 I am using a csv filter in order to sep… --- ## [How Can I See the Inside of my Logstash Event Object in a Ruby Script?](https://discuss.elastic.co/t/how-can-i-see-the-inside-of-my-logstash-event-object-in-a-ruby-script/197415)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 2\ **Last updated:** [August 30, 2019, 8:15pm UTC](https://discuss.elastic.co/t/how-can-i-see-the-inside-of-my-logstash-event-object-in-a-ruby-script/197415 "2019-08-30T20:15:50Z")
Hi Logstash Jedi Masters, I’m building my first Logstash service. I want my filter to be able to do a little processing based on the data flowing through LS. My LS setup is pretty basic. Here’s my LS config file, sim… --- ## [ELK HA with 2 logstash nodes](https://discuss.elastic.co/t/elk-ha-with-2-logstash-nodes/195860)
**Author:** [@Yashwant\_Shettigar](https://discuss.elastic.co/u/Yashwant_Shettigar)\ **Replies:** 2\ **Last updated:** [August 30, 2019, 6:23pm UTC](https://discuss.elastic.co/t/elk-ha-with-2-logstash-nodes/195860 "2019-08-30T18:23:48Z")
Hi Everyone, Trying to setup an ELK stack with high availability using below configurations. 1 Kibana node 3 Elasticsearch nodes ( 1 master and 2 child) 2 Logstash nodes Can someone please help me setting up this co… --- ## [Pipeline config error](https://discuss.elastic.co/t/pipeline-config-error/197525)
**Author:** [@Philip\_Thomson1](https://discuss.elastic.co/u/Philip_Thomson1)\ **Replies:** 8\ **Last updated:** [August 30, 2019, 5:48pm UTC](https://discuss.elastic.co/t/pipeline-config-error/197525 "2019-08-30T17:48:28Z")
Maybe I have been staring at this too long and need another set of eyes to point out something silly.. Using logstash 7.3.1 started using pipelines, I have one running, but two defined, I get an error in the debug logs… --- ## [Does using a script in elastic output plugin for Logstash impact's performance](https://discuss.elastic.co/t/does-using-a-script-in-elastic-output-plugin-for-logstash-impacts-performance/197547)
**Author:** [@Rahul\_Goyal](https://discuss.elastic.co/u/Rahul_Goyal)\ **Replies:** 1\ **Last updated:** [August 30, 2019, 4:43pm UTC](https://discuss.elastic.co/t/does-using-a-script-in-elastic-output-plugin-for-logstash-impacts-performance/197547 "2019-08-30T16:43:51Z")
Hi, I am trying to insert data from multiple Kafka topics to ES. However, there are scenarios where I need to append message in an existing array or update message in multiple JSON documents, for which I am using the "s… --- ## [Unexpected character ('\<' (code 60)) - different behavior - ES cloud](https://discuss.elastic.co/t/unexpected-character-code-60-different-behavior-es-cloud/197535)
**Author:** [@hborda](https://discuss.elastic.co/u/hborda)\ **Replies:** 2\ **Last updated:** [August 30, 2019, 3:24pm UTC](https://discuss.elastic.co/t/unexpected-character-code-60-different-behavior-es-cloud/197535 "2019-08-30T15:24:03Z")
We read most of what have been written about the topic, we tried many permutations and we can't successfully make logstash to work: Here is the console output start ----------------- C:\\Deleteme\\es\_logstash\>C:\\Softwar… --- ## [Unable to use the app generated timestamp in logstash](https://discuss.elastic.co/t/unable-to-use-the-app-generated-timestamp-in-logstash/197531)
**Author:** [@chimbu3306](https://discuss.elastic.co/u/chimbu3306)\ **Replies:** 3\ **Last updated:** [August 30, 2019, 3:11pm UTC](https://discuss.elastic.co/t/unable-to-use-the-app-generated-timestamp-in-logstash/197531 "2019-08-30T15:11:55Z")
I have the below log file created by the application and the timestamp value is recorded in "@timestamp" but logstash is unable to use that timestamp and failing to push this value to elasticsearch { "@timestamp": "2019… --- ## [Environment variables in if..then logic](https://discuss.elastic.co/t/environment-variables-in-if-then-logic/197099)
**Author:** [@pgf](https://discuss.elastic.co/u/pgf)\ **Replies:** 4\ **Last updated:** [August 30, 2019, 2:53pm UTC](https://discuss.elastic.co/t/environment-variables-in-if-then-logic/197099 "2019-08-30T14:53:06Z")
I am using a number of environment variables in my logstash config which all work well, except for one. In my output section I am trying to use a variable in an if.. then statement and it does not appear to be working. P… --- ## [Having trouble ingesting a ModSsecurity log VIA TCP..sort of](https://discuss.elastic.co/t/having-trouble-ingesting-a-modssecurity-log-via-tcp-sort-of/197527)
**Author:** [@SigmazGFX](https://discuss.elastic.co/u/SigmazGFX)\ **Replies:** 2\ **Last updated:** [August 30, 2019, 2:22pm UTC](https://discuss.elastic.co/t/having-trouble-ingesting-a-modssecurity-log-via-tcp-sort-of/197527 "2019-08-30T14:22:20Z")
Hey guys.. I'm really hoping that someone here can help me out. I'm working with a Sophos XG firewall that uses Modsecurity as the reverse proxy WAF engine. Because of support contract restrictions and also the locked … --- ## [LogStash::ConfigurationError", :message=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1)](https://discuss.elastic.co/t/logstash-configurationerror-message-expected-one-of-input-filter-output-at-line-1-column-1-byte-1/197447)
**Author:** [@shan\_usa](https://discuss.elastic.co/u/shan_usa)\ **Replies:** 1\ **Last updated:** [August 30, 2019, 1:22pm UTC](https://discuss.elastic.co/t/logstash-configurationerror-message-expected-one-of-input-filter-output-at-line-1-column-1-byte-1/197447 "2019-08-30T13:22:25Z")
Hi , I am trying pull data from oracle table to elastic search but i am getting below error, Any one please tell me how to fix this issue. conf file : input { jdbc { jdbc\_driver\_library =\> "C:\\Product\\SabaWeb\\lib\\o… --- ## [Duplicate entries while using mutate](https://discuss.elastic.co/t/duplicate-entries-while-using-mutate/197413)
**Author:** [@humartinez](https://discuss.elastic.co/u/humartinez)\ **Replies:** 4\ **Last updated:** [August 30, 2019, 1:08pm UTC](https://discuss.elastic.co/t/duplicate-entries-while-using-mutate/197413 "2019-08-30T13:08:37Z")
Hi there! Im having a issue (random I think) when using mutate and split. This is my logstash config input { kafka { bootstrap\_servers =\> "localhost:9092" topics =\> \["topic"\] } } filter { mutate { … --- ## [How to select specific field value](https://discuss.elastic.co/t/how-to-select-specific-field-value/197473)
**Author:** [@Skiloop\_Shi](https://discuss.elastic.co/u/Skiloop_Shi)\ **Replies:** 2\ **Last updated:** [August 30, 2019, 12:49pm UTC](https://discuss.elastic.co/t/how-to-select-specific-field-value/197473 "2019-08-30T12:49:52Z")
For example: How to turn {"name":"James","location":{"city":"A City","address":""}} into {"city":"A City","address":""} --- ## [Unnecessary hits from logstash to elastic index](https://discuss.elastic.co/t/unnecessary-hits-from-logstash-to-elastic-index/197034)
**Author:** [@bharatwaj](https://discuss.elastic.co/u/bharatwaj)\ **Replies:** 8\ **Last updated:** [August 30, 2019, 12:22pm UTC](https://discuss.elastic.co/t/unnecessary-hits-from-logstash-to-elastic-index/197034 "2019-08-30T12:22:55Z")
When using JDBC paging for input plugin instead of using top and order by , the SQL Last value is not getting changed after every fetch which results in the last value to be updated after all records are pushed from the… --- ## [Custom input codec configuration](https://discuss.elastic.co/t/custom-input-codec-configuration/197120)
**Author:** [@DottorP](https://discuss.elastic.co/u/DottorP)\ **Replies:** 4\ **Last updated:** [August 30, 2019, 8:24am UTC](https://discuss.elastic.co/t/custom-input-codec-configuration/197120 "2019-08-30T08:24:48Z")
Hi everybody. I'm working on my very first codec for Logstash pipeline. The code itself works (according to unit tests). This is the input configuration in LS pipeline: input { jdbc { #stuff statement =\> "select … --- ## [Problem dropping multiple log lines with match statement](https://discuss.elastic.co/t/problem-dropping-multiple-log-lines-with-match-statement/197144)
**Author:** [@Neje](https://discuss.elastic.co/u/Neje)\ **Replies:** 3\ **Last updated:** [August 30, 2019, 8:18am UTC](https://discuss.elastic.co/t/problem-dropping-multiple-log-lines-with-match-statement/197144 "2019-08-30T08:18:22Z")
I'm using LogStash to process AWS Cloudfront logs and I've got a bit of a problem with the log headers. At the start of each log file are two header lines before the main log-data itself. Both start with # and take the … --- ## [No erroe but can't set to elasticsearch](https://discuss.elastic.co/t/no-erroe-but-cant-set-to-elasticsearch/193460)
**Author:** [@SenalChen](https://discuss.elastic.co/u/SenalChen)\ **Replies:** 0\ **Last updated:** [August 2, 2019, 6:46am UTC](https://discuss.elastic.co/t/no-erroe-but-cant-set-to-elasticsearch/193460 "2019-08-02T06:46:36Z")
(topic withdrawn by author, will be automatically deleted in 24 hours unless flagged) --- ## [How to make TLS and authorization configurable by environment variables?](https://discuss.elastic.co/t/how-to-make-tls-and-authorization-configurable-by-environment-variables/197318)
**Author:** [@asp](https://discuss.elastic.co/u/asp)\ **Replies:** 6\ **Last updated:** [August 30, 2019, 6:46am UTC](https://discuss.elastic.co/t/how-to-make-tls-and-authorization-configurable-by-environment-variables/197318 "2019-08-30T06:46:17Z")
Hi, I use security module to encrypt and authorization in public dev system in kubernetes - as it will be on production later. In local developement environment I have not set up TLS. Is there any way how I can config… --- ## [How do you analyze logstash grok parsing for errors?](https://discuss.elastic.co/t/how-do-you-analyze-logstash-grok-parsing-for-errors/197298)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 3\ **Last updated:** [August 30, 2019, 6:35am UTC](https://discuss.elastic.co/t/how-do-you-analyze-logstash-grok-parsing-for-errors/197298 "2019-08-30T06:35:11Z")
I want to get systemname and typelog in the path sample： /usr/local/xxx-springboot/logs/xxx/operator.log Correct result systemname =\> xxx typelog =\> operator filter { grok { match =\> { "source" =\> "/%{WORD}/%{… --- ## [Logstash JMS Input Plugin integration with SolacePubSub](https://discuss.elastic.co/t/logstash-jms-input-plugin-integration-with-solacepubsub/197441)
**Author:** [@rohitjose](https://discuss.elastic.co/u/rohitjose)\ **Replies:** 0\ **Last updated:** [August 30, 2019, 5:47am UTC](https://discuss.elastic.co/t/logstash-jms-input-plugin-integration-with-solacepubsub/197441 "2019-08-30T05:47:21Z")
I am trying to setup logstash jms input plugin read from Solace PubSub. I am using Logstash 7.3.1 with the JMS plugin version 3.1.0. I am able to successfully connect to Solace however the pipeline fails to generate an … --- ## [Basic logstash mutations](https://discuss.elastic.co/t/basic-logstash-mutations/197244)
**Author:** [@Evan\_Something](https://discuss.elastic.co/u/Evan_Something)\ **Replies:** 3\ **Last updated:** [August 29, 2019, 11:40pm UTC](https://discuss.elastic.co/t/basic-logstash-mutations/197244 "2019-08-29T23:40:28Z")
So, for whatever reason, I'm having a hard time grasping working with certain aspects of the logstash filter portion of the pipeline. I see plenty of examples for how to do specific things, but I can't seem to come up wi… --- ## [Logstash grok parse doesn't parse error code](https://discuss.elastic.co/t/logstash-grok-parse-doesnt-parse-error-code/197425)
**Author:** [@EZprogramming](https://discuss.elastic.co/u/EZprogramming)\ **Replies:** 1\ **Last updated:** [August 29, 2019, 11:36pm UTC](https://discuss.elastic.co/t/logstash-grok-parse-doesnt-parse-error-code/197425 "2019-08-29T23:36:44Z")
Logstash version: 7.0.1 Given filename =\> "EUI106\_occurances", I want to have code =\> "EUI106". The following grok parse filter doesn't work: filter{ grok { match =\> { "filename", "%{POSTFIX\_QUEUEID:code}" } … --- ## [How to have logstash update logs as they are coming in from logstash and send to kibana graphs automatically](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383)
**Author:** [@nizztizz](https://discuss.elastic.co/u/nizztizz)\ **Replies:** 12\ **Last updated:** [August 29, 2019, 10:31pm UTC](https://discuss.elastic.co/t/how-to-have-logstash-update-logs-as-they-are-coming-in-from-logstash-and-send-to-kibana-graphs-automatically/197383 "2019-08-29T22:31:10Z")
I want logstash to automatically update and run as new logs are coming in from a web application and send them to kibana's graphs. As of right now I have to manually run logstash and once it finishes running it shows up … --- ## [Parsing F5 LTM logs with Logstash](https://discuss.elastic.co/t/parsing-f5-ltm-logs-with-logstash/197392)
**Author:** [@sarahvo](https://discuss.elastic.co/u/sarahvo)\ **Replies:** 7\ **Last updated:** [August 29, 2019, 7:09pm UTC](https://discuss.elastic.co/t/parsing-f5-ltm-logs-with-logstash/197392 "2019-08-29T19:09:34Z")
Hi, I'm trying to parse LTM logs using this example (found in the official documentation) below: filter { if \[type\] == "syslog" { grok { match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:sysl… --- ## [Logstash s3 plugin](https://discuss.elastic.co/t/logstash-s3-plugin/194895)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 4\ **Last updated:** [August 29, 2019, 6:09pm UTC](https://discuss.elastic.co/t/logstash-s3-plugin/194895 "2019-08-29T18:09:21Z")
Whether logstash s3 input plugin will read the files line by line in real time from s3 bucket similar to the filebeat reading from the files? --- ## [Grok filter date error](https://discuss.elastic.co/t/grok-filter-date-error/196599)
**Author:** [@zolthar-z](https://discuss.elastic.co/u/zolthar-z)\ **Replies:** 1\ **Last updated:** [August 29, 2019, 4:29pm UTC](https://discuss.elastic.co/t/grok-filter-date-error/196599 "2019-08-29T16:29:15Z")
Hello My current platform is composed by filebeat-\>logstash-\>elasticsearch-\>Kibana (everything in 7.3 version) and I have configured a grok in logstash, the grok extract the time from the log and put it in the @timestam… --- ## [Logstash : Extract values from a String field in Json](https://discuss.elastic.co/t/logstash-extract-values-from-a-string-field-in-json/197349)
**Author:** [@chimbu3306](https://discuss.elastic.co/u/chimbu3306)\ **Replies:** 3\ **Last updated:** [August 29, 2019, 4:15pm UTC](https://discuss.elastic.co/t/logstash-extract-values-from-a-string-field-in-json/197349 "2019-08-29T16:15:32Z")
I have the below Json submitted to my logstash from kinesis where the message string in the json contains all the required values which I need to store it as a separate field in elasticsearch . Currently its stored as a … --- ## [Decide if nested JSON is JSON or text](https://discuss.elastic.co/t/decide-if-nested-json-is-json-or-text/197338)
**Author:** [@hunsw](https://discuss.elastic.co/u/hunsw)\ **Replies:** 2\ **Last updated:** [August 29, 2019, 1:36pm UTC](https://discuss.elastic.co/t/decide-if-nested-json-is-json-or-text/197338 "2019-08-29T13:36:01Z")
I'm receiving JSON formatted events from Filebeat, parsed as JSON. I.e. Original log: {"date" : "2019...", "data" : { "subdata" : "sometext" }} (CASE1) {"date" : "2019...", "data" : { "subdata" : { "field1" : "blah" … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=483) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=485)