# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=845 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 846 --- ## [Logstash get field from elasticsearch message](https://discuss.elastic.co/t/logstash-get-field-from-elasticsearch-message/114325)
**Author:** [@faoc](https://discuss.elastic.co/u/faoc)\ **Replies:** 4\ **Last updated:** [January 8, 2018, 9:35am UTC](https://discuss.elastic.co/t/logstash-get-field-from-elasticsearch-message/114325 "2018-01-08T09:35:18Z")
Hi, I have this message on ElasticSearch { "\_index": "audit\_201712", "\_type": "ccc", "\_id": "AWAsoVnydQqBAYNwSrNi", "\_score": null, "\_source": { "resourceType": "AuditEvent", "id": "4306b213-65a2-4ae1-b7e6-d1221… --- ## [Fatal in logstash 6.0.0](https://discuss.elastic.co/t/fatal-in-logstash-6-0-0/114469)
**Author:** [@sbampa](https://discuss.elastic.co/u/sbampa)\ **Replies:** 3\ **Last updated:** [January 8, 2018, 9:23am UTC](https://discuss.elastic.co/t/fatal-in-logstash-6-0-0/114469 "2018-01-08T09:23:46Z")
Hi all, i receive the following error in logstash logs and the process died.: Error: unknown IOException: jnr.enxio.channels.NativeException: Error closing fd 371: Stale NFS file handle Exception: Java::JavaLang::Runt… --- ## [How to Parse and Copy a Field](https://discuss.elastic.co/t/how-to-parse-and-copy-a-field/113918)
**Author:** [@MultiplierMultiplier](https://discuss.elastic.co/u/MultiplierMultiplier)\ **Replies:** 4\ **Last updated:** [January 8, 2018, 9:14am UTC](https://discuss.elastic.co/t/how-to-parse-and-copy-a-field/113918 "2018-01-08T09:14:46Z")
I have the TimeStamp field and I would like to extract the hour of the day so that I can look for out of hours user activity. Jan 3 12:09:01 filter { grok { match =\> { "TimeStamp" =\> "%{WORD}\\s%{NUMBER}\\s%{NUMBER:H… --- ## [Parsing stacktraces in log files](https://discuss.elastic.co/t/parsing-stacktraces-in-log-files/114272)
**Author:** [@anjali](https://discuss.elastic.co/u/anjali)\ **Replies:** 4\ **Last updated:** [January 8, 2018, 8:33am UTC](https://discuss.elastic.co/t/parsing-stacktraces-in-log-files/114272 "2018-01-08T08:33:03Z")
Hello, I'm new to the Elastic Stack and I'm trying to parse the following log- 06:40:13,339 ERROR \[org.hornetq.core.client\] (Thread-29 (HornetQ-server-HornetQServerImpl::serverUUID=3a64b874-09b1-11e7-af28-998b43be6a2d-7… --- ## [Logstash filter to parse json --\> json deserialized string](https://discuss.elastic.co/t/logstash-filter-to-parse-json-json-deserialized-string/114445)
**Author:** [@akustic](https://discuss.elastic.co/u/akustic)\ **Replies:** 1\ **Last updated:** [January 8, 2018, 8:18am UTC](https://discuss.elastic.co/t/logstash-filter-to-parse-json-json-deserialized-string/114445 "2018-01-08T08:18:19Z")
I have some log files I want to parse and they are in the following format. {"log":"{"@timestamp":"2018-01-08T02:11:35.651+00:00","@version":1,"message":"\\u001B\[36mFinished \\u001B\[0m:get / for 139.162.106.181 in (1 ms) … --- ## [Logstash filter condition statement not working as expected](https://discuss.elastic.co/t/logstash-filter-condition-statement-not-working-as-expected/114424)
**Author:** [@shankar1](https://discuss.elastic.co/u/shankar1)\ **Replies:** 6\ **Last updated:** [January 8, 2018, 8:17am UTC](https://discuss.elastic.co/t/logstash-filter-condition-statement-not-working-as-expected/114424 "2018-01-08T08:17:35Z")
Version Used: Logstash 5.5.0 Data Flow -\> Filebeat (5.5.0)\_ Logstash 5.5.0\_Elasticsearch 5.5.0 Input Case1: \\\\\Data 1\ \\\\ Input Case 2: \\ **Author:** [@Amir\_P](https://discuss.elastic.co/u/Amir_P)\ **Replies:** 5\ **Last updated:** [January 8, 2018, 7:27am UTC](https://discuss.elastic.co/t/set-logstash-cloud-output-hosts/114435 "2018-01-08T07:27:44Z") hello, I am trying to set Logstash configuration file output hosts as Elasticsearch cloud link output { elasticsearch { hosts =\> "https://5something3.us-east-1.aws.found.io:9243" index =\> "check1" } stdout {} Unf… --- ## [Pipelines.yml ignored on logstash running as a service](https://discuss.elastic.co/t/pipelines-yml-ignored-on-logstash-running-as-a-service/107828)
**Author:** [@dorj1234](https://discuss.elastic.co/u/dorj1234)\ **Replies:** 17\ **Last updated:** [January 8, 2018, 6:24am UTC](https://discuss.elastic.co/t/pipelines-yml-ignored-on-logstash-running-as-a-service/107828 "2018-01-08T06:24:15Z")
First, I saw and tried the solution mentioned here and commented out path.config from /etc/logstash/logstash.yml Looks like when I run logstash from command line all works well, but I need the logstash service to run. i… --- ## [How can i accelerate the queue output speed in logstash](https://discuss.elastic.co/t/how-can-i-accelerate-the-queue-output-speed-in-logstash/113881)
**Author:** [@antony.y](https://discuss.elastic.co/u/antony.y)\ **Replies:** 7\ **Last updated:** [January 8, 2018, 6:11am UTC](https://discuss.elastic.co/t/how-can-i-accelerate-the-queue-output-speed-in-logstash/113881 "2018-01-08T06:11:44Z")
We encountered an network failure yesterday,and before that we enable the persist queue .So after the network became normal,we can see my queue is full, but this cause i can not see the real-time data in my kibana, -rw… --- ## [Multi\_fields rename problem](https://discuss.elastic.co/t/multi-fields-rename-problem/114410)
**Author:** [@abu.sayeed](https://discuss.elastic.co/u/abu.sayeed)\ **Replies:** 2\ **Last updated:** [January 8, 2018, 5:28am UTC](https://discuss.elastic.co/t/multi-fields-rename-problem/114410 "2018-01-08T05:28:36Z")
MY Logformat: computer\_name NAZ-TECH-PC-005.NTDAC.nazdaqTechnologies.local event\_data.TargetServerName VNTDACWSPRP001.NTDAC.nazdaqTechnologies.local event\_data.SubjectUserName Jhon event\_data.TargetUserName Tom fi… --- ## [Bootstrap logstash 6.1.1 with Error ' sourceloader - No configuration found in the configured sources.'](https://discuss.elastic.co/t/bootstrap-logstash-6-1-1-with-error-sourceloader-no-configuration-found-in-the-configured-sources/112604)
**Author:** [@lovedebug](https://discuss.elastic.co/u/lovedebug)\ **Replies:** 2\ **Last updated:** [January 8, 2018, 3:26am UTC](https://discuss.elastic.co/t/bootstrap-logstash-6-1-1-with-error-sourceloader-no-configuration-found-in-the-configured-sources/112604 "2018-01-08T03:26:40Z")
Hi All I have run the logstash on win10 with multiple piplines and debug enabled. And i have got below errors. I have no idea about it as the debug info is not enough. Regards. \[2017-12-20T17:58:29,823\]\[DEBUG\]\[logs… --- ## [Following just regular installer instructions: Error: Could not find or load main class org.jruby.Main](https://discuss.elastic.co/t/following-just-regular-installer-instructions-error-could-not-find-or-load-main-class-org-jruby-main/114426)
**Author:** [@Kieren\_Johnstone](https://discuss.elastic.co/u/Kieren_Johnstone)\ **Replies:** 4\ **Last updated:** [January 7, 2018, 9:13pm UTC](https://discuss.elastic.co/t/following-just-regular-installer-instructions-error-could-not-find-or-load-main-class-org-jruby-main/114426 "2018-01-07T21:13:58Z")
Kierens-MBP:bin kieren$ ./logstash --? Java HotSpot(TM) 64-Bit Server VM warning: Option UseParNewGC was deprecated in version 9.0 and will likely be removed in a future release. Java HotSpot(TM) 64-Bit Server VM warni… --- ## [Multiple Files for Logstash - URGENT HELP Please](https://discuss.elastic.co/t/multiple-files-for-logstash-urgent-help-please/114402)
**Author:** [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\ **Replies:** 7\ **Last updated:** [January 7, 2018, 6:13pm UTC](https://discuss.elastic.co/t/multiple-files-for-logstash-urgent-help-please/114402 "2018-01-07T18:13:46Z")
Hi everyone so I loaded my first \*.csv into logstash and got it to come up in Kibana, but I have about 50 more files and more overtime I will need to upload. So I created a new config file. called config\_2.config and whe… --- ## [Logstash | elasticsearch - Encountered a retryable error](https://discuss.elastic.co/t/logstash-elasticsearch-encountered-a-retryable-error/114418)
**Author:** [@rrs](https://discuss.elastic.co/u/rrs)\ **Replies:** 2\ **Last updated:** [January 7, 2018, 4:42pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-encountered-a-retryable-error/114418 "2018-01-07T16:42:56Z")
We have setup ELK 5.6.3 in cluster method and everything in each layer is running fine, but logstash keep reporting below error even the port 5044 in listening state. Can anyone help me out here. \[ERROR\] 2018-01-07 11:1… --- ## [Logstash mutate and copy entire \_source object into an inner field](https://discuss.elastic.co/t/logstash-mutate-and-copy-entire-source-object-into-an-inner-field/114131)
**Author:** [@deflector](https://discuss.elastic.co/u/deflector)\ **Replies:** 1\ **Last updated:** [January 7, 2018, 4:31pm UTC](https://discuss.elastic.co/t/logstash-mutate-and-copy-entire-source-object-into-an-inner-field/114131 "2018-01-07T16:31:23Z")
Hello, I'm trying to add a layer to my template definition to wrap the existing document as an inner field, lets call it 'summary'. I have to do this for several types, each with many fields. One of the bigger issues wi… --- ## [Logstash error during setup](https://discuss.elastic.co/t/logstash-error-during-setup/114323)
**Author:** [@alpha\_gamma](https://discuss.elastic.co/u/alpha_gamma)\ **Replies:** 17\ **Last updated:** [January 7, 2018, 4:21pm UTC](https://discuss.elastic.co/t/logstash-error-during-setup/114323 "2018-01-07T16:21:34Z")
Hi The team, I am getting error and tried many ways but dont know how to resolve below error. In one of the previous blogs I found the logstash and elasticsearch should be on different VM, what if I want all on one one … --- ## [Aggregate messages with different patterns](https://discuss.elastic.co/t/aggregate-messages-with-different-patterns/114411)
**Author:** [@dooz](https://discuss.elastic.co/u/dooz)\ **Replies:** 0\ **Last updated:** [January 7, 2018, 10:56am UTC](https://discuss.elastic.co/t/aggregate-messages-with-different-patterns/114411 "2018-01-07T10:56:14Z")
Hello, An application we manage generates logs with different patterns. One event generates between 2 and 4 messages, which have the same event\_id in the lig message (in the below example: "event\_id\_1" Each message wi… --- ## [After installing X-pack, logstash can't run](https://discuss.elastic.co/t/after-installing-x-pack-logstash-cant-run/114377)
**Author:** [@f26227279](https://discuss.elastic.co/u/f26227279)\ **Replies:** 4\ **Last updated:** [January 7, 2018, 10:17am UTC](https://discuss.elastic.co/t/after-installing-x-pack-logstash-cant-run/114377 "2018-01-07T10:17:35Z")
As title, after I install x-pack, logstash can't run. and it show me some errors: 28,006\]\[WARN \]\[logstash.licensechecker.licensereader\] UNEXPECTED POOL ERROR {:e=\>#\ **Author:** [@arisbanach](https://discuss.elastic.co/u/arisbanach)\ **Replies:** 3\ **Last updated:** [January 6, 2018, 9:13pm UTC](https://discuss.elastic.co/t/json-filter-gives-warning-that-it-requires-a-target/114378 "2018-01-06T21:13:18Z") \[2018-01-06T03:41:04,492\]\[WARN \]\[logstash.filters.json \] Parsed JSON object/hash requires a target configuration option {:source=\>"extras", :raw=\>""} I don't have a target set because I read in the documentation th… --- ## [Error Installing plugin](https://discuss.elastic.co/t/error-installing-plugin/114281)
**Author:** [@faoc](https://discuss.elastic.co/u/faoc)\ **Replies:** 5\ **Last updated:** [January 6, 2018, 7:28pm UTC](https://discuss.elastic.co/t/error-installing-plugin/114281 "2018-01-06T19:28:28Z")
Hi, I'm trying to install a logstash-output-plugin, but every time a get a proxy error. I already set HTTP\_PROXY and HTTPS\_PROXY but i'm still with the same issue. this is the error: "Unable to download data from htt… --- ## [How to write grock pattern for date(\[yyyy-MM-dd HH:mm:ss.SSS z\] ) in logstash](https://discuss.elastic.co/t/how-to-write-grock-pattern-for-date-yyyy-mm-dd-hh-mm-ss-sss-z-in-logstash/114295)
**Author:** [@nitin59](https://discuss.elastic.co/u/nitin59)\ **Replies:** 1\ **Last updated:** [January 6, 2018, 5:25pm UTC](https://discuss.elastic.co/t/how-to-write-grock-pattern-for-date-yyyy-mm-dd-hh-mm-ss-sss-z-in-logstash/114295 "2018-01-06T17:25:41Z")
Hi, In my log4j error logs, the datetime is printed in below format, I was trying to write a pattern in grock to parse that, but not getting the proper result. \[yyyy-MM-dd HH:mm:ss.SSS z\] ex. \[2018-01-04 15:00:01.361 … --- ## [Document Type in version 6.1.0](https://discuss.elastic.co/t/document-type-in-version-6-1-0/113933)
**Author:** [@Frank\_Matera](https://discuss.elastic.co/u/Frank_Matera)\ **Replies:** 1\ **Last updated:** [January 6, 2018, 10:21am UTC](https://discuss.elastic.co/t/document-type-in-version-6-1-0/113933 "2018-01-06T10:21:05Z")
I have been testing ELK for a few months and was previously on 5.6. Filebeat -\> Logstash -\> Elasticsearch. In 5.6 Filebeat I would assign a document\_type to a log. I was working with 2 types of logs: filebeat.prospector… --- ## [Grok - how to add field with values from existing fields?](https://discuss.elastic.co/t/grok-how-to-add-field-with-values-from-existing-fields/114277)
**Author:** [@qubusp](https://discuss.elastic.co/u/qubusp)\ **Replies:** 5\ **Last updated:** [January 6, 2018, 9:58am UTC](https://discuss.elastic.co/t/grok-how-to-add-field-with-values-from-existing-fields/114277 "2018-01-06T09:58:20Z")
The following is a pull of metrics from a jmx application. The question is how to build a grok pattern, which will use the metric\_path: as field name(it's changing constantly) and use metric\_value\_number as the numeric … --- ## [Throttle filter picking up old instead of new value](https://discuss.elastic.co/t/throttle-filter-picking-up-old-instead-of-new-value/114351)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 1\ **Last updated:** [January 6, 2018, 4:37am UTC](https://discuss.elastic.co/t/throttle-filter-picking-up-old-instead-of-new-value/114351 "2018-01-06T04:37:47Z")
Hi, I'm struggling a bit with the throttle filter. After the period expires I expect the throttle filter to save the first new event, but instead its saving the last old event. input { file { path =\> "/home/test/… --- ## [X and Y coordinates to latitude and longitude](https://discuss.elastic.co/t/x-and-y-coordinates-to-latitude-and-longitude/113920)
**Author:** [@RoyS](https://discuss.elastic.co/u/RoyS)\ **Replies:** 6\ **Last updated:** [January 6, 2018, 4:35am UTC](https://discuss.elastic.co/t/x-and-y-coordinates-to-latitude-and-longitude/113920 "2018-01-06T04:35:04Z")
Hi, We have lots of locations with x,y coordinates but Logstash need lat, long values. Could you please show how to convert from one to the other in a .config file. Thanks Roy --- ## [Multiline XML processing using logstash](https://discuss.elastic.co/t/multiline-xml-processing-using-logstash/113915)
**Author:** [@Saket\_Kumar](https://discuss.elastic.co/u/Saket_Kumar)\ **Replies:** 6\ **Last updated:** [January 5, 2018, 9:18pm UTC](https://discuss.elastic.co/t/multiline-xml-processing-using-logstash/113915 "2018-01-05T21:18:27Z")
I have multi line XML to process more than 1000 lines. I am generating single message out of it. When using Logstash 2.4 multiline filter plugin was working perfectly. But as its been deprecated in 6.1. Therefore modi… --- ## [Unable install logstash-output-email plugin on logstash 5.5.2](https://discuss.elastic.co/t/unable-install-logstash-output-email-plugin-on-logstash-5-5-2/113808)
**Author:** [@govinda.rao](https://discuss.elastic.co/u/govinda.rao)\ **Replies:** 5\ **Last updated:** [January 5, 2018, 9:16pm UTC](https://discuss.elastic.co/t/unable-install-logstash-output-email-plugin-on-logstash-5-5-2/113808 "2018-01-05T21:16:19Z")
Hi, I have logstash 5.5.2 and when installing email plugin "logstash-output-email" all the time i am getting below error. please help ERROR: Something went wrong when installing logstash-output-email, message: 403 "Forb… --- ## [Beats input logs after upgrading to Logstash 6.1.1](https://discuss.elastic.co/t/beats-input-logs-after-upgrading-to-logstash-6-1-1/113710)
**Author:** [@system](https://discuss.elastic.co/u/system)\ **Replies:** 1\ **Last updated:** [January 5, 2018, 8:00pm UTC](https://discuss.elastic.co/t/beats-input-logs-after-upgrading-to-logstash-6-1-1/113710 "2018-01-05T20:00:48Z")
I started noticing these in our logs after upgrading to 6.1 and I'm not sure what to make of it. Is this anything that we should be concern about? Thank you in advanced! \[INFO \] 2018-01-02 01:33:20.871 \[defaultEventExec… --- ## [X-pack on logstash](https://discuss.elastic.co/t/x-pack-on-logstash/114108)
**Author:** [@SJN8](https://discuss.elastic.co/u/SJN8)\ **Replies:** 4\ **Last updated:** [January 5, 2018, 7:39pm UTC](https://discuss.elastic.co/t/x-pack-on-logstash/114108 "2018-01-05T19:39:23Z")
Hi All , I have installed X-pack on logstash but in step it is mentioned to add user and pwd in logstash.yml . but i am using pipelines which have configuration files under conf.d , so where shall i add x-pack related… --- ## [How to find JSON-Properties from LogFile as Fields in Elasticsearch?](https://discuss.elastic.co/t/how-to-find-json-properties-from-logfile-as-fields-in-elasticsearch/114266)
**Author:** [@awaitJosh](https://discuss.elastic.co/u/awaitJosh)\ **Replies:** 4\ **Last updated:** [January 5, 2018, 6:25pm UTC](https://discuss.elastic.co/t/how-to-find-json-properties-from-logfile-as-fields-in-elasticsearch/114266 "2018-01-05T18:25:39Z")
Hi, I have to create an ELK-Showcase for my team. The task is to take Log(files) from an ASP.NET-Application with Filebeat, forward them via Logstash to Elasticsearch and create a few sample-dashboards in Kibana. I ha… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=844) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=846)