# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=456 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 457 --- ## [Grok filter when logs format is not fixed](https://discuss.elastic.co/t/grok-filter-when-logs-format-is-not-fixed/205408)
**Author:** [@varunjithtk](https://discuss.elastic.co/u/varunjithtk)\ **Replies:** 1\ **Last updated:** [October 28, 2019, 7:57am UTC](https://discuss.elastic.co/t/grok-filter-when-logs-format-is-not-fixed/205408 "2019-10-28T07:57:44Z")
Am trying to parse the rest api request and response in my node logs. Below are two example logs {"level":"debug","message":"HTTP GET /1stexample/alerts/subs?userId=111","meta":{"res":{"statusCode":200},"req":{"headers"… --- ## [Enrich Win event logs with LDAP](https://discuss.elastic.co/t/enrich-win-event-logs-with-ldap/205409)
**Author:** [@AlexH](https://discuss.elastic.co/u/AlexH)\ **Replies:** 0\ **Last updated:** [October 28, 2019, 6:32am UTC](https://discuss.elastic.co/t/enrich-win-event-logs-with-ldap/205409 "2019-10-28T06:32:02Z")
I would like to enrich win event logs with LDAP info - user group, user OU, user start date, etc. What is the best way to accomplish this, given a domain and username? --- ## [Map timestamp using first time value in incoming JSON Lines?](https://discuss.elastic.co/t/map-timestamp-using-first-time-value-in-incoming-json-lines/204748)
**Author:** [@GrahamHannington](https://discuss.elastic.co/u/GrahamHannington)\ **Replies:** 4\ **Last updated:** [October 28, 2019, 6:20am UTC](https://discuss.elastic.co/t/map-timestamp-using-first-time-value-in-incoming-json-lines/204748 "2019-10-28T06:20:54Z")
I am using Logstash 7.4 to ingest JSON Lines. In this particular JSON Lines data, which is from a proprietary source, the event timestamp is the first time value in each incoming line. By first, I am referring to the s… --- ## [Logstash TCP json\_lines to elasticsearch won't stop printing output to stdout](https://discuss.elastic.co/t/logstash-tcp-json-lines-to-elasticsearch-wont-stop-printing-output-to-stdout/205407)
**Author:** [@Blake\_Loring](https://discuss.elastic.co/u/Blake_Loring)\ **Replies:** 0\ **Last updated:** [October 28, 2019, 5:59am UTC](https://discuss.elastic.co/t/logstash-tcp-json-lines-to-elasticsearch-wont-stop-printing-output-to-stdout/205407 "2019-10-28T05:59:28Z")
Hello, I am using the logstash docker image and a pipeline from tcp to elastic search. Whenever a message is processed by logstash it is also output to stdout, even though I have logging turned to warn or above. There a… --- ## [Ruby exception occurred: undefined method \`/'](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method/205389)
**Author:** [@Luan\_Ph\_m](https://discuss.elastic.co/u/Luan_Ph_m)\ **Replies:** 1\ **Last updated:** [October 27, 2019, 6:52pm UTC](https://discuss.elastic.co/t/ruby-exception-occurred-undefined-method/205389 "2019-10-27T18:52:27Z")
filter { ruby { code =\> "event.set('test', (event.get('vstorage.space.allocatable') / event.get('vstorage.license.capacity')) \* 100)" } } The error is \[ERROR\]\[logstash.filters.ruby \] Ruby ex… --- ## [S3 input plugin processing very slow](https://discuss.elastic.co/t/s3-input-plugin-processing-very-slow/205365)
**Author:** [@pami](https://discuss.elastic.co/u/pami)\ **Replies:** 0\ **Last updated:** [October 26, 2019, 6:33pm UTC](https://discuss.elastic.co/t/s3-input-plugin-processing-very-slow/205365 "2019-10-26T18:33:58Z")
Hi, We have large data in S3 bucket. one logstash server is pulling from S3 bucket. I can't configure more logstash servers due to duplicate problem. because of this there was lag of two days .... Is there any way to m… --- ## [How to parse it correctly](https://discuss.elastic.co/t/how-to-parse-it-correctly/205329)
**Author:** [@Barushka](https://discuss.elastic.co/u/Barushka)\ **Replies:** 0\ **Last updated:** [October 25, 2019, 9:10pm UTC](https://discuss.elastic.co/t/how-to-parse-it-correctly/205329 "2019-10-25T21:10:48Z")
Here is a brainier. I am trying to track the number of the requests between server and user ( I need to know the elapsed time per each request), and I am trying to catch the number and the elapsed time of the small API … --- ## [Make configuration only take effect for next daily index](https://discuss.elastic.co/t/make-configuration-only-take-effect-for-next-daily-index/205292)
**Author:** [@bruce289](https://discuss.elastic.co/u/bruce289)\ **Replies:** 3\ **Last updated:** [October 25, 2019, 5:04pm UTC](https://discuss.elastic.co/t/make-configuration-only-take-effect-for-next-daily-index/205292 "2019-10-25T17:04:54Z")
Hi, We index into daily indices from logstash e.g. my\_index-%{+YYYY.MM.dd}. I would like to adjust the value of a setting on the elasticsearch output, but I only want that setting to take effect for the next day's index… --- ## [Invalid version of beats protocol?](https://discuss.elastic.co/t/invalid-version-of-beats-protocol/205259)
**Author:** [@kavindu\_manaram](https://discuss.elastic.co/u/kavindu_manaram)\ **Replies:** 1\ **Last updated:** [October 25, 2019, 5:03pm UTC](https://discuss.elastic.co/t/invalid-version-of-beats-protocol/205259 "2019-10-25T17:03:35Z")
I try to used filebeat and logstash in local machine with cloud Elasticsearch. And some errors comes on filebeat and logstash both. \[2019-10-25T20:16:46,162\]\[INFO \]\[org.logstash.beats.BeatsHandler\]\[main\] \[local: 172.16… --- ## [Proper timestamp parsing](https://discuss.elastic.co/t/proper-timestamp-parsing/205293)
**Author:** [@nujojo](https://discuss.elastic.co/u/nujojo)\ **Replies:** 1\ **Last updated:** [October 25, 2019, 4:20pm UTC](https://discuss.elastic.co/t/proper-timestamp-parsing/205293 "2019-10-25T16:20:56Z")
Hi Guys, I have a message with timestamp 2019-10-25T10:39:44.079331-04:00. I want to use grok filter and match that time, so later on can transfer to other format. What will be the best filter for match such timestamp. … --- ## [Logstash on kubernetes. Multiple output conditions](https://discuss.elastic.co/t/logstash-on-kubernetes-multiple-output-conditions/205087)
**Author:** [@Khuman](https://discuss.elastic.co/u/Khuman)\ **Replies:** 4\ **Last updated:** [October 25, 2019, 4:17pm UTC](https://discuss.elastic.co/t/logstash-on-kubernetes-multiple-output-conditions/205087 "2019-10-25T16:17:54Z")
Hi, guys! I want to manipulate output indexes in my output section. The config example below don't work. I need insert data into one index when host\_name index is present or another if not. Field kubernetes.labels.devo… --- ## [Logstash 7 is not adding datestamp to index name](https://discuss.elastic.co/t/logstash-7-is-not-adding-datestamp-to-index-name/205162)
**Author:** [@kenwdelong](https://discuss.elastic.co/u/kenwdelong)\ **Replies:** 3\ **Last updated:** [October 25, 2019, 3:52pm UTC](https://discuss.elastic.co/t/logstash-7-is-not-adding-datestamp-to-index-name/205162 "2019-10-25T15:52:01Z")
I just updated my stack from 6.7.0 -\> 6.8.0 -\> 7.4.0. Everything seems find EXCEPT logstash is putting all the events into an index called "logstash". I used to date-stamp the index, like "logstash-2019-10-24". I canno… --- ## [Logstash and Beat float compare](https://discuss.elastic.co/t/logstash-and-beat-float-compare/205274)
**Author:** [@odlevakp](https://discuss.elastic.co/u/odlevakp)\ **Replies:** 0\ **Last updated:** [October 25, 2019, 2:00pm UTC](https://discuss.elastic.co/t/logstash-and-beat-float-compare/205274 "2019-10-25T14:00:27Z")
I am sending metricbeat data via logstash input into elasticsearch and would like to set up a notification on low disk space. I tried something like: output { if \[type\] == "beats" and \[metricset\]\[name\] == "filesystem… --- ## [Logstash 7.3 install](https://discuss.elastic.co/t/logstash-7-3-install/205270)
**Author:** [@sobeman21](https://discuss.elastic.co/u/sobeman21)\ **Replies:** 0\ **Last updated:** [October 25, 2019, 1:46pm UTC](https://discuss.elastic.co/t/logstash-7-3-install/205270 "2019-10-25T13:46:12Z")
I'm fairly new to the ELK stack world. It seems that when I upgraded Logstash to 7.4.1-1 it broke Elastiflow. How do I go about going back to either 7.4.0-0 or even 7.3.2? --- ## [Geoip with private networks fails with](https://discuss.elastic.co/t/geoip-with-private-networks-fails-with/205183)
**Author:** [@aimarjg](https://discuss.elastic.co/u/aimarjg)\ **Replies:** 1\ **Last updated:** [October 25, 2019, 1:04pm UTC](https://discuss.elastic.co/t/geoip-with-private-networks-fails-with/205183 "2019-10-25T13:04:12Z")
Hi, I'm having a trouble with mapping private IPs to their geo coordiantes, filter below always fails with error: "reason"=\>"failed to parse field \[client.geo.location\] of type \[geo\_point\]", "caused\_by"=\>{"type"=\>"pars… --- ## [Error when run logstash .conf file](https://discuss.elastic.co/t/error-when-run-logstash-conf-file/205199)
**Author:** [@Vladpov](https://discuss.elastic.co/u/Vladpov)\ **Replies:** 1\ **Last updated:** [October 25, 2019, 12:31pm UTC](https://discuss.elastic.co/t/error-when-run-logstash-conf-file/205199 "2019-10-25T12:31:27Z")
Hi guys when I run config file with sudo -u logstash /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/task.conf It gives me the following error \[INFO \] 2019-10-25 07:59:53.283 \[\[main\]\ **Author:** [@emilio](https://discuss.elastic.co/u/emilio)\ **Replies:** 3\ **Last updated:** [October 25, 2019, 12:24pm UTC](https://discuss.elastic.co/t/abnormally-cpu-utilization-by-logstash/203386 "2019-10-25T12:24:37Z") Dear all, I have just one grok filter in my logstash and it horrible utilizes CPU up to 1600% Could you, please help me to detect bottleneck in my filter and fix it? Thank you in advance. That's my filter: filter { … --- ## [Configuration elk stack](https://discuss.elastic.co/t/configuration-elk-stack/205249)
**Author:** [@Leks](https://discuss.elastic.co/u/Leks)\ **Replies:** 0\ **Last updated:** [October 25, 2019, 11:53am UTC](https://discuss.elastic.co/t/configuration-elk-stack/205249 "2019-10-25T11:53:29Z")
Hi all, I try to solve my problem with Elasticsearch Unreachable: Connection refused, while loading data by using logstash. Elasticsearch.yml: network.host: 0.0.0.0 http:port: 9200 Kibana.yml: elasticsearch.host… --- ## [Unable to install the logstash 6.6.2](https://discuss.elastic.co/t/unable-to-install-the-logstash-6-6-2/205245)
**Author:** [@krishnakumar\_s](https://discuss.elastic.co/u/krishnakumar_s)\ **Replies:** 0\ **Last updated:** [October 25, 2019, 11:37am UTC](https://discuss.elastic.co/t/unable-to-install-the-logstash-6-6-2/205245 "2019-10-25T11:37:14Z")
Hi Team I have logstash 6.6.2 version but when i ran the services getting "Logstash stopped processing because of an error:(Load Error) unsupported platform:x86\_64-windows" Kindly confirm the logstash 6.6.2 it suppor… --- ## [Filter Data With First Data in grok filter](https://discuss.elastic.co/t/filter-data-with-first-data-in-grok-filter/203426)
**Author:** [@kisuke](https://discuss.elastic.co/u/kisuke)\ **Replies:** 6\ **Last updated:** [October 25, 2019, 11:35am UTC](https://discuss.elastic.co/t/filter-data-with-first-data-in-grok-filter/203426 "2019-10-25T11:35:32Z")
All is in the title I want to filter with the first data in my grok filter to use an other grok filter --- ## [Error while configuring multiple JDBC inputs with unique sql\_last\_value in docker-compose](https://discuss.elastic.co/t/error-while-configuring-multiple-jdbc-inputs-with-unique-sql-last-value-in-docker-compose/205227)
**Author:** [@rasmus](https://discuss.elastic.co/u/rasmus)\ **Replies:** 1\ **Last updated:** [October 25, 2019, 10:44am UTC](https://discuss.elastic.co/t/error-while-configuring-multiple-jdbc-inputs-with-unique-sql-last-value-in-docker-compose/205227 "2019-10-25T10:44:07Z")
I am trying to set up multiple JDBC pipelines to be able to use multiple SQL-statements. In these pipelines I want to use unique sql\_last\_value's. I am using docker-compose and have volumed the files correctly (I think).… --- ## [How to read ealsticsearch as input in logstash only once](https://discuss.elastic.co/t/how-to-read-ealsticsearch-as-input-in-logstash-only-once/205236)
**Author:** [@Sunil\_Murali](https://discuss.elastic.co/u/Sunil_Murali)\ **Replies:** 0\ **Last updated:** [October 25, 2019, 10:36am UTC](https://discuss.elastic.co/t/how-to-read-ealsticsearch-as-input-in-logstash-only-once/205236 "2019-10-25T10:36:33Z")
Hi, Need to take the values from an elastic index\_1 through logstash then process it and insert it to another elastic index\_2. This is the process i am doing. But once the input read index\_1 and inserted to index\_2, th… --- ## [Logstash concurrency - managing multiple LS nodes that share the same pipeline configuration](https://discuss.elastic.co/t/logstash-concurrency-managing-multiple-ls-nodes-that-share-the-same-pipeline-configuration/204446)
**Author:** [@AlessandroKP](https://discuss.elastic.co/u/AlessandroKP)\ **Replies:** 2\ **Last updated:** [October 25, 2019, 9:47am UTC](https://discuss.elastic.co/t/logstash-concurrency-managing-multiple-ls-nodes-that-share-the-same-pipeline-configuration/204446 "2019-10-25T09:47:56Z")
Hi all, I have a question regarding concurrency management in Logstash. My setup is the following. I have two Logstash nodes that run the same pipeline, i.e. they use the same pipeline configuration. They perform exac… --- ## [Noob trying to process a file](https://discuss.elastic.co/t/noob-trying-to-process-a-file/204215)
**Author:** [@tronty](https://discuss.elastic.co/u/tronty)\ **Replies:** 2\ **Last updated:** [October 25, 2019, 8:56am UTC](https://discuss.elastic.co/t/noob-trying-to-process-a-file/204215 "2019-10-25T08:56:32Z")
Hi all, So I'm trying to do something that I'm guessing is pretty basic but I am stuck and cannot find the answer. Problems are with config file :frowning: I've got an input file (postfix logs) that I want to parse w… --- ## [Grokparsefailure when filtering a multiline json value](https://discuss.elastic.co/t/grokparsefailure-when-filtering-a-multiline-json-value/204803)
**Author:** [@Cluster](https://discuss.elastic.co/u/Cluster)\ **Replies:** 2\ **Last updated:** [October 25, 2019, 7:32am UTC](https://discuss.elastic.co/t/grokparsefailure-when-filtering-a-multiline-json-value/204803 "2019-10-25T07:32:11Z")
Due to memory limitation I cannot deploy Beats on my device. Therefore I pull metric data with a remote Http\_poller into Logstash. The response data of the poll-request looks like this: { "memory": { "raw": " … --- ## [Get value from message and put it into new field](https://discuss.elastic.co/t/get-value-from-message-and-put-it-into-new-field/205045)
**Author:** [@AstroSlazak](https://discuss.elastic.co/u/AstroSlazak)\ **Replies:** 2\ **Last updated:** [October 25, 2019, 4:16am UTC](https://discuss.elastic.co/t/get-value-from-message-and-put-it-into-new-field/205045 "2019-10-25T04:16:34Z")
Hi, I have a problem i want to get Longitude and Latitude from log: "2019-10-24 12:01:06,110 \[33\] INFO XUDP\_Endpoint.UdpEnpoint \[(null)\] - Data received: AS:17907,NW:77,RD:3003,DT24.10.2019 10.01.05, Longitude:10,457… --- ## ["Unable to find driver class" when use jdbc\_driver\_library =\> "/home/ubuntu/ojdbc6.jar" after upgrade to Logstash 6.8.4](https://discuss.elastic.co/t/unable-to-find-driver-class-when-use-jdbc-driver-library-home-ubuntu-ojdbc6-jar-after-upgrade-to-logstash-6-8-4/205182)
**Author:** [@Jersey\_Zhang](https://discuss.elastic.co/u/Jersey_Zhang)\ **Replies:** 0\ **Last updated:** [October 25, 2019, 5:00am UTC](https://discuss.elastic.co/t/unable-to-find-driver-class-when-use-jdbc-driver-library-home-ubuntu-ojdbc6-jar-after-upgrade-to-logstash-6-8-4/205182 "2019-10-25T05:00:46Z")
The latest release of Logstash 6.8.4 with jdbc-input-plugin 4.3.16 give us following error in syslog Oct 24 21:02:00 ip-10-40-0-99 logstash\[15520\]: 2172 job: Oct 24 21:02:00 ip-10-40-0-99 logstash\[15520\]: 2172 … --- ## [ELK/Logstash in Swarm](https://discuss.elastic.co/t/elk-logstash-in-swarm/205189)
**Author:** [@stinkfly](https://discuss.elastic.co/u/stinkfly)\ **Replies:** 0\ **Last updated:** [October 25, 2019, 6:16am UTC](https://discuss.elastic.co/t/elk-logstash-in-swarm/205189 "2019-10-25T06:16:24Z")
Hi there, I've deployed sebp/elk (https://hub.docker.com/r/sebp/elk/) in a 3 node swarm. It comes up fine. If I point winlogbeat to Elasticsearch, it works fine. However, I find that if I set Logstash as the target f… --- ## [LogStash on Docker keeps shutting down after failing to find config](https://discuss.elastic.co/t/logstash-on-docker-keeps-shutting-down-after-failing-to-find-config/205186)
**Author:** [@rvshchwl](https://discuss.elastic.co/u/rvshchwl)\ **Replies:** 0\ **Last updated:** [October 25, 2019, 5:45am UTC](https://discuss.elastic.co/t/logstash-on-docker-keeps-shutting-down-after-failing-to-find-config/205186 "2019-10-25T05:45:46Z")
I was running LogStash on Docker using the provided command, and today it crashed. After restarting the Docker, the Logstash silently shuts down after a few minutes, each time. This is my starting script: #!/bin/bash d… --- ## [How to improve logstash performance on this config](https://discuss.elastic.co/t/how-to-improve-logstash-performance-on-this-config/205138)
**Author:** [@krishna\_bhargav](https://discuss.elastic.co/u/krishna_bhargav)\ **Replies:** 1\ **Last updated:** [October 24, 2019, 8:11pm UTC](https://discuss.elastic.co/t/how-to-improve-logstash-performance-on-this-config/205138 "2019-10-24T20:11:18Z")
My current config is as below, right now below config processes 119,445,555 records from beats to logstash and sent to elastic takes 3 mins 40 sec. what other config can improve this performance. The AccountFilter1 is si… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=455) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=457)