# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=941 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 942 --- ## [Issue in parsing huge json file](https://discuss.elastic.co/t/issue-in-parsing-huge-json-file/93596)
**Author:** [@Narayanan\_Sukumar](https://discuss.elastic.co/u/Narayanan_Sukumar)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 3:45pm UTC](https://discuss.elastic.co/t/issue-in-parsing-huge-json-file/93596 "2017-07-21T15:45:06Z")
I have a huge json file in a log for one success api call. I am able to see that response at when there is no filter is applied over it.(seeing as string) \*\*\*Small json are working as intended. When I am applying the… --- ## [Cannot assign requested address to: localhost:80](https://discuss.elastic.co/t/cannot-assign-requested-address-to-localhost-80/93595)
**Author:** [@kitz99](https://discuss.elastic.co/u/kitz99)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 3:43pm UTC](https://discuss.elastic.co/t/cannot-assign-requested-address-to-localhost-80/93595 "2017-07-21T15:43:52Z")
Hello, I use filebeat and logstash to parse some existing log files \[lots of them (each file has ~ 500 Mb)\]. At some point, logstash start throwing Ruby exception occurred: Cannot assign requested address - bind - Ca… --- ## [How to write this algorithm in Logstash and display the new fields in Kibana](https://discuss.elastic.co/t/how-to-write-this-algorithm-in-logstash-and-display-the-new-fields-in-kibana/93534)
**Author:** [@Alex\_S](https://discuss.elastic.co/u/Alex_S)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 3:21pm UTC](https://discuss.elastic.co/t/how-to-write-this-algorithm-in-logstash-and-display-the-new-fields-in-kibana/93534 "2017-07-21T15:21:24Z")
Hello Geeks :slight\_smile: I have this pesudo code that I want to translate into a Logstash plugin and show the lat and long fields into Kibana ... Assume the following variables are to be used in the calculation/c… --- ## [Use logstash to modify an event previously saved to elasticsearch](https://discuss.elastic.co/t/use-logstash-to-modify-an-event-previously-saved-to-elasticsearch/93529)
**Author:** [@cmpashayan](https://discuss.elastic.co/u/cmpashayan)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 3:12pm UTC](https://discuss.elastic.co/t/use-logstash-to-modify-an-event-previously-saved-to-elasticsearch/93529 "2017-07-21T15:12:27Z")
Specifically, I'm trying to tag the last event prior to an exception. So, when you encounter an event that matches some criterion, go back and find the last event and then add a tag to it. Is this possible? I assume … --- ## [Add option to 'elapsed' plugin to use newest start event](https://discuss.elastic.co/t/add-option-to-elapsed-plugin-to-use-newest-start-event/94122)
**Author:** [@James\_Harr](https://discuss.elastic.co/u/James_Harr)\ **Replies:** 0\ **Last updated:** [July 21, 2017, 1:50pm UTC](https://discuss.elastic.co/t/add-option-to-elapsed-plugin-to-use-newest-start-event/94122 "2017-07-21T13:50:36Z")
I'm trying to use the 'elapsed' plugin to differentiate between two events. When the plugin detects multiple start events, it uses the first to measure the elapsed time. I would like to add an option for it to use the ne… --- ## [Feed Logstash with gzipped multiline inputs](https://discuss.elastic.co/t/feed-logstash-with-gzipped-multiline-inputs/92963)
**Author:** [@gilbertotcc](https://discuss.elastic.co/u/gilbertotcc)\ **Replies:** 4\ **Last updated:** [July 21, 2017, 12:21pm UTC](https://discuss.elastic.co/t/feed-logstash-with-gzipped-multiline-inputs/92963 "2017-07-21T12:21:49Z")
Hi All, I'm implementing an ELK stack for Java logs indexing and analysis. At the moment, it's only a proof of concept and I cannot feed Logstash with plain text log files; I've got gzipped past log files and I want to… --- ## [Feature request: 'logstash-output-gelf' plugin TLS support](https://discuss.elastic.co/t/feature-request-logstash-output-gelf-plugin-tls-support/94092)
**Author:** [@Bragolgirith](https://discuss.elastic.co/u/Bragolgirith)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 11:39am UTC](https://discuss.elastic.co/t/feature-request-logstash-output-gelf-plugin-tls-support/94092 "2017-07-21T11:39:28Z")
In Graylog it is possible to configure a GELF TCP Input with TLS, but from what I gather, the logstash gelf output plugin doesn't currently support sending events over secured TCP. Sounds like something important to add… --- ## [Getting Ip-Adress from Filebeat-Client](https://discuss.elastic.co/t/getting-ip-adress-from-filebeat-client/93757)
**Author:** [@Truesight](https://discuss.elastic.co/u/Truesight)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 11:33am UTC](https://discuss.elastic.co/t/getting-ip-adress-from-filebeat-client/93757 "2017-07-21T11:33:13Z")
Hello, i have my Logstash on one Server and Filebeat on many other mashines with varying IP-Adresses. I have tried the dns plugin allready, but since the hostname is not convertable to ip it doesn't work. Is there any ot… --- ## [Logstash is not picking up the heap memory settings - Linux OS](https://discuss.elastic.co/t/logstash-is-not-picking-up-the-heap-memory-settings-linux-os/94101)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 0\ **Last updated:** [July 21, 2017, 11:11am UTC](https://discuss.elastic.co/t/logstash-is-not-picking-up-the-heap-memory-settings-linux-os/94101 "2017-07-21T11:11:32Z")
I have set the LS\_JAVA\_OPTS="-Xmx4g -Xms4g" in startup options file and it didn't reflect. Also added -Xms2g -Xmx2g in jvm.options file and restarted logstash. But this is also not reflecting. --- ## [\[GROK\]\[Pattern\] Match any char except \\n](https://discuss.elastic.co/t/grok-pattern-match-any-char-except-n/94089)
**Author:** [@ThaPoox](https://discuss.elastic.co/u/ThaPoox)\ **Replies:** 2\ **Last updated:** [July 21, 2017, 10:34am UTC](https://discuss.elastic.co/t/grok-pattern-match-any-char-except-n/94089 "2017-07-21T10:34:01Z")
So I have this log data: \> 12:03:55,117 ERROR \[My.Super.Cool.Java.Class\] (main-executor-threads - 213) This is an Exception message example: package.myProject.MyException: You cannot do whatever you are trying to do … --- ## [\[ERROR\]\[logstash.filters.grok \] Error while attempti ng to check/cancel excessively long grok patterns {:message=\>"Mutex relocking by same thread", :class=\>"ThreadError",](https://discuss.elastic.co/t/error-logstash-filters-grok-error-while-attempti-ng-to-check-cancel-excessively-long-grok-patterns-message-mutex-relocking-by-same-thread-class-threaderror/92990)
**Author:** [@swatititame](https://discuss.elastic.co/u/swatititame)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 10:12am UTC](https://discuss.elastic.co/t/error-logstash-filters-grok-error-while-attempti-ng-to-check-cancel-excessively-long-grok-patterns-message-mutex-relocking-by-same-thread-class-threaderror/92990 "2017-07-21T10:12:01Z")
2017-07-13T15:59:36,280\]\[ERROR\]\[logstash.filters.grok \] Error while attempti ng to check/cancel excessively long grok patterns {:message=\>"Mutex relocking by same thread", :class=\>"ThreadError", :backtrace=\>\["org/jr… --- ## [Update single field in document in elastic from within logstash](https://discuss.elastic.co/t/update-single-field-in-document-in-elastic-from-within-logstash/93798)
**Author:** [@bterhorst](https://discuss.elastic.co/u/bterhorst)\ **Replies:** 3\ **Last updated:** [July 21, 2017, 10:31am UTC](https://discuss.elastic.co/t/update-single-field-in-document-in-elastic-from-within-logstash/93798 "2017-07-21T10:31:15Z")
Hi, How can I update a single field on an existing document in elastic from logstash? This is the docment in elastic where I want to upate field: "FEEDBACK\_RECIEVED" from "NO" to "YES" { "\_index": "portal-201… --- ## [Jdbc-streaming filter plugin seems not working properly](https://discuss.elastic.co/t/jdbc-streaming-filter-plugin-seems-not-working-properly/93022)
**Author:** [@Blue\_Moon](https://discuss.elastic.co/u/Blue_Moon)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 8:38am UTC](https://discuss.elastic.co/t/jdbc-streaming-filter-plugin-seems-not-working-properly/93022 "2017-07-21T08:38:12Z")
I want to fetch data from sqldb and want to prapared a nested json array structure input{ jdbc { jdbc\_connection\_string =\> "jdbc:sqlserver://testserver;databaseName=TEST\_DEV;user=me;password=secret;" jdbc\_user … --- ## [Logstash --debug -f](https://discuss.elastic.co/t/logstash-debug-f/94008)
**Author:** [@pzkr](https://discuss.elastic.co/u/pzkr)\ **Replies:** 1\ **Last updated:** [July 21, 2017, 7:31am UTC](https://discuss.elastic.co/t/logstash-debug-f/94008 "2017-07-21T07:31:09Z")
whats the command for? i presume its for debug option but the documents state "With this command, Logstash concatenates three config files, /tmp/one, /tmp/two, and /tmp/three, and parses them into a single config". when … --- ## [Seeing number\_format\_exception on a non-numeric data field](https://discuss.elastic.co/t/seeing-number-format-exception-on-a-non-numeric-data-field/93647)
**Author:** [@enyuhsu](https://discuss.elastic.co/u/enyuhsu)\ **Replies:** 3\ **Last updated:** [July 21, 2017, 5:55am UTC](https://discuss.elastic.co/t/seeing-number-format-exception-on-a-non-numeric-data-field/93647 "2017-07-21T05:55:58Z")
Our production logstash has been working for months until 7/13, starting almost exactly at 6pm UTC (the timestamp is significant). The error message we saw was: Failed action. {:status=\>400, :action=\>\["index", {:\_id=\>n… --- ## [Kafka input for logstash，process restart how to set offset](https://discuss.elastic.co/t/kafka-input-for-logstash-process-restart-how-to-set-offset/94048)
**Author:** [@kai\_fu](https://discuss.elastic.co/u/kai_fu)\ **Replies:** 0\ **Last updated:** [July 21, 2017, 3:14am UTC](https://discuss.elastic.co/t/kafka-input-for-logstash-process-restart-how-to-set-offset/94048 "2017-07-21T03:14:28Z")
If the logstash restarts, how to ensure that the read kafka offset begins the last one befor reboot, 'auto\_offset\_reset' just supports earliest / lastest, which may cause the data to read missing or duplicate, the old ve… --- ## [Installing Logstash Plugins failing on Windows 10](https://discuss.elastic.co/t/installing-logstash-plugins-failing-on-windows-10/93287)
**Author:** [@patrickian](https://discuss.elastic.co/u/patrickian)\ **Replies:** 8\ **Last updated:** [July 20, 2017, 10:15pm UTC](https://discuss.elastic.co/t/installing-logstash-plugins-failing-on-windows-10/93287 "2017-07-20T22:15:40Z")
Hi, I'm using logstash 5.5 running on Windows 10 and I'm trying to install the logstash elapsed filter by running the command ./logstash-plugin install logstash-filter-elapsed in the bin folder of my logstash instal… --- ## [Generate an array of fields from other fields in an Array of Hashes](https://discuss.elastic.co/t/generate-an-array-of-fields-from-other-fields-in-an-array-of-hashes/94004)
**Author:** [@corywink](https://discuss.elastic.co/u/corywink)\ **Replies:** 1\ **Last updated:** [July 20, 2017, 9:02pm UTC](https://discuss.elastic.co/t/generate-an-array-of-fields-from-other-fields-in-an-array-of-hashes/94004 "2017-07-20T21:02:07Z")
Hello, I am trying to grab fields out of an array of hashes in Logstash. Right now it looks like this: "hashes" =\> \[ \[0\] { "key1" =\> "value1", "key2"=\> "value2", … --- ## [How to map new created fields into existing index?](https://discuss.elastic.co/t/how-to-map-new-created-fields-into-existing-index/93955)
**Author:** [@moabbas](https://discuss.elastic.co/u/moabbas)\ **Replies:** 0\ **Last updated:** [July 20, 2017, 1:53pm UTC](https://discuss.elastic.co/t/how-to-map-new-created-fields-into-existing-index/93955 "2017-07-20T13:53:07Z")
I already have index in elasticsearh, i added new field in logstash to my index like the follow: mutate { add\_field =\> {"\[location\]" =\> \["%{lat}","%{lon}"\]} } mutate { … --- ## [Convert JSON formatted apache log time to datetime](https://discuss.elastic.co/t/convert-json-formatted-apache-log-time-to-datetime/93838)
**Author:** [@ndhilliprasad](https://discuss.elastic.co/u/ndhilliprasad)\ **Replies:** 0\ **Last updated:** [July 19, 2017, 8:35pm UTC](https://discuss.elastic.co/t/convert-json-formatted-apache-log-time-to-datetime/93838 "2017-07-19T20:35:55Z")
Hi, I am new to the ELK stack. We have setup a Filebeat -\> Logstash -\> Elastic -\> Kibana. Apache logs are in JSON format (This will avoid grok filter) as following { "time":"\[31/May/2017:00:12:58 -0500\]", "clientI… --- ## [What is the best approach to capture Linux(Ubuntu) server logs?](https://discuss.elastic.co/t/what-is-the-best-approach-to-capture-linux-ubuntu-server-logs/93415)
**Author:** [@amruth](https://discuss.elastic.co/u/amruth)\ **Replies:** 5\ **Last updated:** [July 20, 2017, 6:33pm UTC](https://discuss.elastic.co/t/what-is-the-best-approach-to-capture-linux-ubuntu-server-logs/93415 "2017-07-20T18:33:39Z")
Can someone suggest me the best way to capture Ubuntu server logs and send it to Elasticsearch? I am trying to use Logstash but on top of that is there any other plugin? For windows event logs Winlogbeat and Logstash ca… --- ## [Multiple if conditions in output section-Logstash conf](https://discuss.elastic.co/t/multiple-if-conditions-in-output-section-logstash-conf/93972)
**Author:** [@SandhyaRani](https://discuss.elastic.co/u/SandhyaRani)\ **Replies:** 0\ **Last updated:** [July 20, 2017, 3:14pm UTC](https://discuss.elastic.co/t/multiple-if-conditions-in-output-section-logstash-conf/93972 "2017-07-20T15:14:01Z")
Hi there! I wanted to use if conditionals, Like I dont want to load the log data with log levels TRACE, DEBUG log levels. So here is my output section: =============================================== output { if "… --- ## [ISO8601 in date filter](https://discuss.elastic.co/t/iso8601-in-date-filter/93976)
**Author:** [@Adam\_Szabo](https://discuss.elastic.co/u/Adam_Szabo)\ **Replies:** 0\ **Last updated:** [July 20, 2017, 3:37pm UTC](https://discuss.elastic.co/t/iso8601-in-date-filter/93976 "2017-07-20T15:37:32Z")
I'm pulling events in from a MariaDB database. The event that arrives to Elasticsearch looks like this: { "\_index": "test", "\_type": "logs", "\_id": "AV1gkQx-9-vDP\_XVWXk\_", "\_score": null, "\_source": { "@t… --- ## [Logstash, windows paths with spaces break plugin installation](https://discuss.elastic.co/t/logstash-windows-paths-with-spaces-break-plugin-installation/93586)
**Author:** [@IHateBugs](https://discuss.elastic.co/u/IHateBugs)\ **Replies:** 2\ **Last updated:** [July 20, 2017, 2:44pm UTC](https://discuss.elastic.co/t/logstash-windows-paths-with-spaces-break-plugin-installation/93586 "2017-07-20T14:44:20Z")
as the title suggest installing a plugin is not possible (tested with logstash 5.5.0) if the path contains any spaces. tested with logstash-filter-aggregate-2.5.2.gem in a standard "C:\\Program Files" path. i jury rigge… --- ## [Logstash Input file is not reading file](https://discuss.elastic.co/t/logstash-input-file-is-not-reading-file/92441)
**Author:** [@igbayilola](https://discuss.elastic.co/u/igbayilola)\ **Replies:** 4\ **Last updated:** [July 20, 2017, 2:43pm UTC](https://discuss.elastic.co/t/logstash-input-file-is-not-reading-file/92441 "2017-07-20T14:43:37Z")
Hello, I want first to apologize for my english level. I'm using Logstash 5.5.0 and I'm facing an issue with my configuration. It works just fine with the stdin plugin but not with the file pugin. I have the following co… --- ## [Logstash sometimes sends duplicate entry to ES](https://discuss.elastic.co/t/logstash-sometimes-sends-duplicate-entry-to-es/93618)
**Author:** [@fozboz](https://discuss.elastic.co/u/fozboz)\ **Replies:** 7\ **Last updated:** [July 20, 2017, 2:06pm UTC](https://discuss.elastic.co/t/logstash-sometimes-sends-duplicate-entry-to-es/93618 "2017-07-20T14:06:26Z")
Hi there, I have a Metricbeat configured to send Beats to a single Logstash host, which in turn sends to an ES cluster. Metricbeat config: metricbeat.modules: #------------------------------- System Module --------… --- ## [Simple string and Time stamp parse also redirect to CSV file](https://discuss.elastic.co/t/simple-string-and-time-stamp-parse-also-redirect-to-csv-file/93948)
**Author:** [@krishna1](https://discuss.elastic.co/u/krishna1)\ **Replies:** 0\ **Last updated:** [July 20, 2017, 1:23pm UTC](https://discuss.elastic.co/t/simple-string-and-time-stamp-parse-also-redirect-to-csv-file/93948 "2017-07-20T13:23:57Z")
Dear Folks, Recently I've started using logstash for parsing the logs, Please help on syntax for parsing the log in for CSV file. ex: \[Wed May 10 17:18:03.391 2017\] PassportMain 17:17:59.319: tvp\[tvp\_CloseCam/5211… --- ## [Illegal quoting in translate filter](https://discuss.elastic.co/t/illegal-quoting-in-translate-filter/93938)
**Author:** [@AlexIsFly](https://discuss.elastic.co/u/AlexIsFly)\ **Replies:** 0\ **Last updated:** [July 20, 2017, 12:50pm UTC](https://discuss.elastic.co/t/illegal-quoting-in-translate-filter/93938 "2017-07-20T12:50:00Z")
Hi, I have a dictionary for the translate filter that looks like this : 192.168.1.1,Column2;Column3;"Column4";Column5 If there is a match with the key, then I process Column2;Column3;"Column4";Column5 in a csv fi… --- ## [UDP input queue monitoring](https://discuss.elastic.co/t/udp-input-queue-monitoring/93926)
**Author:** [@Mr.Flakes](https://discuss.elastic.co/u/Mr.Flakes)\ **Replies:** 2\ **Last updated:** [July 20, 2017, 12:38pm UTC](https://discuss.elastic.co/t/udp-input-queue-monitoring/93926 "2017-07-20T12:38:03Z")
Hello guys, I was wondering if anyone knows how to monitor UDP input plugin queue? My config is: input { udp { codec =\> elog { elog\_definitions =\> "XXX" } port =\> XXX que… --- ## [Increse performance of Logstash with multiple input](https://discuss.elastic.co/t/increse-performance-of-logstash-with-multiple-input/93910)
**Author:** [@Zeeshan\_Alam](https://discuss.elastic.co/u/Zeeshan_Alam)\ **Replies:** 1\ **Last updated:** [July 20, 2017, 12:00pm UTC](https://discuss.elastic.co/t/increse-performance-of-logstash-with-multiple-input/93910 "2017-07-20T12:00:57Z")
I am using Logstash to read from Kafka. My VM is having 6 processors. I looked at following two config: pipeline.workers: Default is Number of the host’s CPU cores The number of workers that will, in parallel, execu… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=940) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=942)