# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=743 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 744 --- ## [Parsing additional data in a windows event log](https://discuss.elastic.co/t/parsing-additional-data-in-a-windows-event-log/135157)
**Author:** [@Joshua\_Fenton](https://discuss.elastic.co/u/Joshua_Fenton)\ **Replies:** 3\ **Last updated:** [June 11, 2018, 5:43pm UTC](https://discuss.elastic.co/t/parsing-additional-data-in-a-windows-event-log/135157 "2018-06-11T17:43:06Z")
I am using a customized SQL audit specification to forward MSSQL logs to the windows security event log. These events come in with the event id of 33205. I would like to take the message field from the logs and parse t… --- ## [Adding a field base on others field](https://discuss.elastic.co/t/adding-a-field-base-on-others-field/135376)
**Author:** [@Pradana](https://discuss.elastic.co/u/Pradana)\ **Replies:** 2\ **Last updated:** [June 11, 2018, 3:56pm UTC](https://discuss.elastic.co/t/adding-a-field-base-on-others-field/135376 "2018-06-11T15:56:36Z")
Hi, I have field of a "operator" which consist a content of 5 digits string like : 62811 62812 62814 62817 I would like to add another field with name "operatorname" with the content is the mapping of above data. e… --- ## [Is it possible to enable paging in Logstash jdbc input without tracking\_column?](https://discuss.elastic.co/t/is-it-possible-to-enable-paging-in-logstash-jdbc-input-without-tracking-column/134576)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 3\ **Last updated:** [June 11, 2018, 3:29pm UTC](https://discuss.elastic.co/t/is-it-possible-to-enable-paging-in-logstash-jdbc-input-without-tracking-column/134576 "2018-06-11T15:29:26Z")
Hi, I am using ELK GA 5.0.0. I am using Logstash to fetch rows from my Postgres db like; jdbc { jdbc\_driver\_library =\> "/ELK/Logstash/edb-jdbc17.jar" jdbc\_driver\_class =\> "Java::com.edb.Driver" jdbc\_connection\_strin… --- ## [Grok pattern fails although it is valid](https://discuss.elastic.co/t/grok-pattern-fails-although-it-is-valid/135317)
**Author:** [@dr01](https://discuss.elastic.co/u/dr01)\ **Replies:** 1\ **Last updated:** [June 11, 2018, 9:51am UTC](https://discuss.elastic.co/t/grok-pattern-fails-although-it-is-valid/135317 "2018-06-11T09:51:33Z")
I need to extract the first numeric value from this log line: Summary for local -------------- Succeeded: 515 (changed=142) Failed: 0 -------------- Total: (etc.) so this is my grok filter: grok { match =\> { "message… --- ## [Error with dissect filter \[SOLVED\]](https://discuss.elastic.co/t/error-with-dissect-filter-solved/135368)
**Author:** [@dr01](https://discuss.elastic.co/u/dr01)\ **Replies:** 8\ **Last updated:** [June 11, 2018, 2:33pm UTC](https://discuss.elastic.co/t/error-with-dissect-filter-solved/135368 "2018-06-11T14:33:41Z")
I am trying to parse this message: Summary for local -------------- Succeeded: 478 (changed=180) Failed: 0 -------------- Total states run: 478 Total run time: 68.410 s \[Tue, 05 Jun 2018 14:33:17 +0200\] codedeploy-agent… --- ## [Creating a new field based on substring](https://discuss.elastic.co/t/creating-a-new-field-based-on-substring/135248)
**Author:** [@Pradana](https://discuss.elastic.co/u/Pradana)\ **Replies:** 7\ **Last updated:** [June 11, 2018, 1:52pm UTC](https://discuss.elastic.co/t/creating-a-new-field-based-on-substring/135248 "2018-06-11T13:52:22Z")
I have below message : 2018-06-08 21:35:44,182 DEBUG \[org.mobicents.smsc.library.CdrGenerator\] 2018-06-08 21:35:44.171,62895416500343,1,1,6282880640846,1,1,success\_esme,SS7\_HR,message,null,92044,0,null,null,null,null,62… --- ## [How to split string into array of objects using logstash?](https://discuss.elastic.co/t/how-to-split-string-into-array-of-objects-using-logstash/135370)
**Author:** [@stevefai](https://discuss.elastic.co/u/stevefai)\ **Replies:** 1\ **Last updated:** [June 11, 2018, 1:48pm UTC](https://discuss.elastic.co/t/how-to-split-string-into-array-of-objects-using-logstash/135370 "2018-06-11T13:48:18Z")
Hello, I'm trying to save some Java stacktrace data into Elasticsearch using Logstash. Each event contains a multiline string like: "exception1: Exception Message 1 exception2: Exception Message 2 exception3: Except… --- ## [How to log \_bulk request payload sent by Logstash to Elasticsearch via output plugin?](https://discuss.elastic.co/t/how-to-log-bulk-request-payload-sent-by-logstash-to-elasticsearch-via-output-plugin/135363)
**Author:** [@pankhuri](https://discuss.elastic.co/u/pankhuri)\ **Replies:** 1\ **Last updated:** [June 11, 2018, 1:24pm UTC](https://discuss.elastic.co/t/how-to-log-bulk-request-payload-sent-by-logstash-to-elasticsearch-via-output-plugin/135363 "2018-06-11T13:24:29Z")
I am getting this error on using logstash-elasticsearch output plugin: \[2018-06-11T17:46:29,075\]\[ERROR\]\[logstash.outputs.elasticsearch\] Encountered a retryable error. Will Retry with exponential backoff {:code=\>400, :ur… --- ## [Local logstash connection to cloud](https://discuss.elastic.co/t/local-logstash-connection-to-cloud/135347)
**Author:** [@amel](https://discuss.elastic.co/u/amel)\ **Replies:** 1\ **Last updated:** [June 11, 2018, 1:14pm UTC](https://discuss.elastic.co/t/local-logstash-connection-to-cloud/135347 "2018-06-11T13:14:07Z")
I've configured my local logstash instance to connect o my cloud. I've the following errror : \[2018-06-11T13:03:11,469\]\[WARN \]\[logstash.outputs.elasticsearch\] Attempted to resurrect connection to dead ES instance, but … --- ## [Network Performance Monitoring with agent](https://discuss.elastic.co/t/network-performance-monitoring-with-agent/135350)
**Author:** [@Ibrahim\_Lubis](https://discuss.elastic.co/u/Ibrahim_Lubis)\ **Replies:** 0\ **Last updated:** [June 11, 2018, 11:22am UTC](https://discuss.elastic.co/t/network-performance-monitoring-with-agent/135350 "2018-06-11T11:22:25Z")
Hi, First, I don't know what category my question is, so I put in this category. I have and idea but don't know where to start, I want to monitoring my Network performance by deploying agent/raspberry in every floor whe… --- ## [Logstash cannot connect tou cloud](https://discuss.elastic.co/t/logstash-cannot-connect-tou-cloud/135151)
**Author:** [@amel](https://discuss.elastic.co/u/amel)\ **Replies:** 3\ **Last updated:** [June 11, 2018, 10:22am UTC](https://discuss.elastic.co/t/logstash-cannot-connect-tou-cloud/135151 "2018-06-11T10:22:01Z")
Hello We have a new elastic cloud on elastic.co on my local logstash installation, I want logstash index data into our cloud cloud.id: "MYID " # I've putted my ID cloud.auth: login:password #login and password … --- ## [How to parse json data in ruby filter?](https://discuss.elastic.co/t/how-to-parse-json-data-in-ruby-filter/135295)
**Author:** [@clash\_of\_clanss](https://discuss.elastic.co/u/clash_of_clanss)\ **Replies:** 7\ **Last updated:** [June 11, 2018, 10:16am UTC](https://discuss.elastic.co/t/how-to-parse-json-data-in-ruby-filter/135295 "2018-06-11T10:16:21Z")
Hi, I am having one json file and I am trying to parse those json field in logstash. so I decided to create one ruby file from that I have to parse those json fields. is there any way to do this process? for example, … --- ## [How to match backspace in grok filter?](https://discuss.elastic.co/t/how-to-match-backspace-in-grok-filter/135326)
**Author:** [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\ **Replies:** 0\ **Last updated:** [June 11, 2018, 9:48am UTC](https://discuss.elastic.co/t/how-to-match-backspace-in-grok-filter/135326 "2018-06-11T09:48:44Z")
input: (NvCapture) Error Tried grok : (?/(NvCapture/) Error) --- ## [\[logstash.pipeline\] A plugin had an unrecoverable error](https://discuss.elastic.co/t/logstash-pipeline-a-plugin-had-an-unrecoverable-error/134627)
**Author:** [@swastime](https://discuss.elastic.co/u/swastime)\ **Replies:** 1\ **Last updated:** [June 11, 2018, 8:41am UTC](https://discuss.elastic.co/t/logstash-pipeline-a-plugin-had-an-unrecoverable-error/134627 "2018-06-11T08:41:21Z")
Hi! I'm new to the ELK stack and I am trying to use the jdbc input plugin to put data from my derby database into kibana. I am using a windows computer and I have elasticsearch, kibana and logstash installed, I am using … --- ## [Reading from first](https://discuss.elastic.co/t/reading-from-first/135216)
**Author:** [@NIloufarVafaei](https://discuss.elastic.co/u/NIloufarVafaei)\ **Replies:** 13\ **Last updated:** [June 11, 2018, 8:06am UTC](https://discuss.elastic.co/t/reading-from-first/135216 "2018-06-11T08:06:20Z")
i use logstash for reading a log file .this file store my service log ,so added some logs during a hours. the point is each time that a log add in my log file , logstash read all log file from first and send it to elasti… --- ## [I defined ducument\_type=apachelogs in filebeat.prospectors, but having problem in if\[type\] == "apachelogs"](https://discuss.elastic.co/t/i-defined-ducument-type-apachelogs-in-filebeat-prospectors-but-having-problem-in-if-type-apachelogs/135303)
**Author:** [@dakshay](https://discuss.elastic.co/u/dakshay)\ **Replies:** 1\ **Last updated:** [June 11, 2018, 7:44am UTC](https://discuss.elastic.co/t/i-defined-ducument-type-apachelogs-in-filebeat-prospectors-but-having-problem-in-if-type-apachelogs/135303 "2018-06-11T07:44:17Z")
Below is filebeat.yml filebeat: prospectors: - paths: - /home/st/Desktop/apache.log input\_type: log document\_type: apache\_logs - paths: - /home/st/Desktop/server.log input\_type: log document\_type: server\_logs … --- ## [Logstash tcp input issue](https://discuss.elastic.co/t/logstash-tcp-input-issue/134956)
**Author:** [@somu.sandeep](https://discuss.elastic.co/u/somu.sandeep)\ **Replies:** 3\ **Last updated:** [June 11, 2018, 7:16am UTC](https://discuss.elastic.co/t/logstash-tcp-input-issue/134956 "2018-06-11T07:16:22Z")
Hi, I'm trying to transfer wso2carbon logs to elk using tcp input plugin my config for wso2 log4jproperties file. #TCP logger pattern log4j.appender.tcp=org.apache.log4j.net.SocketAppender log4j.appender.tcp.layout… --- ## [Having trouble with the date filter for iis logs](https://discuss.elastic.co/t/having-trouble-with-the-date-filter-for-iis-logs/135267)
**Author:** [@Doron\_Raz](https://discuss.elastic.co/u/Doron_Raz)\ **Replies:** 3\ **Last updated:** [June 11, 2018, 6:49am UTC](https://discuss.elastic.co/t/having-trouble-with-the-date-filter-for-iis-logs/135267 "2018-06-11T06:49:51Z")
Continuing the discussion from Date Filter Not working parsing IIS logs: I get the correct timestamp in the debugger, but not in Kibana. here is an example of the parsed log in the debugger: { "uriQuery" =\> "-"… --- ## [Unable to see indexes in Logstash](https://discuss.elastic.co/t/unable-to-see-indexes-in-logstash/134389)
**Author:** [@PanneerSelvam](https://discuss.elastic.co/u/PanneerSelvam)\ **Replies:** 7\ **Last updated:** [June 8, 2018, 3:30am UTC](https://discuss.elastic.co/t/unable-to-see-indexes-in-logstash/134389 "2018-06-08T03:30:02Z")
Hi Team, I am able to execute the following code and get the below output as follows but there is no indexes creation happening in logstash. I request you to kindly assist me and let me know, in case if I miss anything… --- ## [Parse json file with logstash](https://discuss.elastic.co/t/parse-json-file-with-logstash/134795)
**Author:** [@Super8film](https://discuss.elastic.co/u/Super8film)\ **Replies:** 5\ **Last updated:** [June 11, 2018, 5:55am UTC](https://discuss.elastic.co/t/parse-json-file-with-logstash/134795 "2018-06-11T05:55:11Z")
hey I've following json file { "time":"2015-09-20;12:13:24", "bug\_code":"test", "stacktrace":1235 } I got this from a python script and logstash reading it out with following configuration: input { file{ pat… --- ## [Multiple index creation from single configuration file](https://discuss.elastic.co/t/multiple-index-creation-from-single-configuration-file/135132)
**Author:** [@cppatel](https://discuss.elastic.co/u/cppatel)\ **Replies:** 3\ **Last updated:** [June 11, 2018, 5:01am UTC](https://discuss.elastic.co/t/multiple-index-creation-from-single-configuration-file/135132 "2018-06-11T05:01:34Z")
I am trying to generate multiple indices in one .conf file under a single tag(which is also defined in filebeat.yml) with different grok filters along with respective index generation. The whole scenario is , I have in… --- ## [Explanation of the principle of aggregation and log normalization](https://discuss.elastic.co/t/explanation-of-the-principle-of-aggregation-and-log-normalization/135276)
**Author:** [@Dora](https://discuss.elastic.co/u/Dora)\ **Replies:** 0\ **Last updated:** [June 10, 2018, 11:21pm UTC](https://discuss.elastic.co/t/explanation-of-the-principle-of-aggregation-and-log-normalization/135276 "2018-06-10T23:21:53Z")
Hello, I would like to understand please at what level are the aggregation and normalization of ELK logs, is it done on logstash or Elasticsearch on the pipeline for Beats. and if possible I would like to have an exampl… --- ## [Logstash undefined local variable](https://discuss.elastic.co/t/logstash-undefined-local-variable/135243)
**Author:** [@camjoex](https://discuss.elastic.co/u/camjoex)\ **Replies:** 3\ **Last updated:** [June 10, 2018, 10:42pm UTC](https://discuss.elastic.co/t/logstash-undefined-local-variable/135243 "2018-06-10T22:42:27Z")
Hello I'm trying to transform timestamp to specific format YYYY-MM and occured some trouble. I want to convert timestamp date to format YYYY-MM in specific string crea. Here my test echo "1524227744"|logstash -e 'fil… --- ## [Using field values in Geoip database path](https://discuss.elastic.co/t/using-field-values-in-geoip-database-path/134070)
**Author:** [@Trevor\_Shires](https://discuss.elastic.co/u/Trevor_Shires)\ **Replies:** 1\ **Last updated:** [June 10, 2018, 8:35pm UTC](https://discuss.elastic.co/t/using-field-values-in-geoip-database-path/134070 "2018-06-10T20:35:19Z")
I was wondering if it is possible to use a value stored in a field as part of the path for a geoip database. I tried the same syntax that works for using fields in strings, but in the path it doesn't seem to be working.… --- ## [Logstash load multi template](https://discuss.elastic.co/t/logstash-load-multi-template/134824)
**Author:** [@giuliano](https://discuss.elastic.co/u/giuliano)\ **Replies:** 1\ **Last updated:** [June 10, 2018, 8:33pm UTC](https://discuss.elastic.co/t/logstash-load-multi-template/134824 "2018-06-10T20:33:21Z")
Hi all... I use ELK 6.2.3. In logstash conf.d directory I have prepared two different configuration files for two different firewall with geo localization with a custom fields: "fw\_SourceGEO" and "fw\_DestinationGEO". … --- ## [Installing logstash-core-5.4.1-java.gem with an error](https://discuss.elastic.co/t/installing-logstash-core-5-4-1-java-gem-with-an-error/134911)
**Author:** [@harryyou](https://discuss.elastic.co/u/harryyou)\ **Replies:** 1\ **Last updated:** [June 10, 2018, 8:24pm UTC](https://discuss.elastic.co/t/installing-logstash-core-5-4-1-java-gem-with-an-error/134911 "2018-06-10T20:24:53Z")
Command：gem install logstash-core-5.4.1-java.gem --local OS：Suse 11 SP4 ELK Vesion:5.4.1 Jruby Vesion: 9.2.0.0 Error Message: jar dependencies for logstash-core-5.4.1-java.gemspec . . . \[ERROR\] \[ERROR\] Some problem… --- ## [Map geo\_point field in Elasticsearch](https://discuss.elastic.co/t/map-geo-point-field-in-elasticsearch/135218)
**Author:** [@espogian](https://discuss.elastic.co/u/espogian)\ **Replies:** 7\ **Last updated:** [June 10, 2018, 2:01pm UTC](https://discuss.elastic.co/t/map-geo-point-field-in-elasticsearch/135218 "2018-06-10T14:01:07Z")
Hi there. I have defined a field named location which is an array with longitude and latitude. I noticed that Logstash is indexing this field as a float and is not being recognized natively as geographical point in Ela… --- ## [GeoIP with JSON](https://discuss.elastic.co/t/geoip-with-json/135250)
**Author:** [@JeMunos](https://discuss.elastic.co/u/JeMunos)\ **Replies:** 2\ **Last updated:** [June 10, 2018, 2:05am UTC](https://discuss.elastic.co/t/geoip-with-json/135250 "2018-06-10T02:05:54Z")
I'm having some difficulty pushing the correct field from my json to the GeoIP filter module. My goal is to add the GeoIP data to each event as it comes in. I've been trying to test this using the following config file. … --- ## [How to use ruby variable outside ruby block?](https://discuss.elastic.co/t/how-to-use-ruby-variable-outside-ruby-block/135194)
**Author:** [@clash\_of\_clanss](https://discuss.elastic.co/u/clash_of_clanss)\ **Replies:** 9\ **Last updated:** [June 9, 2018, 9:23pm UTC](https://discuss.elastic.co/t/how-to-use-ruby-variable-outside-ruby-block/135194 "2018-06-09T21:23:07Z")
Hi, I like to know, how to use ruby variable outside ruby code block. say for eg. input { file { path =\> "/logfile/\*" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { ruby { c… --- ## [Logstash monitoring ssl](https://discuss.elastic.co/t/logstash-monitoring-ssl/135048)
**Author:** [@empfangsfehler](https://discuss.elastic.co/u/empfangsfehler)\ **Replies:** 1\ **Last updated:** [June 9, 2018, 5:42pm UTC](https://discuss.elastic.co/t/logstash-monitoring-ssl/135048 "2018-06-09T17:42:13Z")
Hello, i do get an error message on Logstas regarding the ssl-certificate for monitoring: :error=\>"Host name '172.17.0.121' does not match the certificate subject provided by the peer (CN=node-0.example.com, OU=node, O… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=742) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=744)