# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=558 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 559 --- ## [How to creating Map View in Kibana using Latitude and Longitude imported from Logstash?](https://discuss.elastic.co/t/how-to-creating-map-view-in-kibana-using-latitude-and-longitude-imported-from-logstash/178461)
**Author:** [@Ac3](https://discuss.elastic.co/u/Ac3)\ **Replies:** 4\ **Last updated:** [April 25, 2019, 4:13pm UTC](https://discuss.elastic.co/t/how-to-creating-map-view-in-kibana-using-latitude-and-longitude-imported-from-logstash/178461 "2019-04-25T16:13:56Z")
Hi I am trying to create a map view in Kibana using Latitude and Longitude. I have been following posts online saying I need to have my Latitude and Longitude combined into 1 field. However, I am still unable to choose… --- ## [Help splitting field into two](https://discuss.elastic.co/t/help-splitting-field-into-two/178459)
**Author:** [@rblumenthal](https://discuss.elastic.co/u/rblumenthal)\ **Replies:** 4\ **Last updated:** [April 25, 2019, 3:02pm UTC](https://discuss.elastic.co/t/help-splitting-field-into-two/178459 "2019-04-25T15:02:11Z")
Hey folks, I am sending logs from Windows server using winlogbeat to logstash (6.6). The field that I am trying to split has 2 IP addresses separated by a comma. How can I go about and create 2 new fields (FirstIP and S… --- ## [Logstash parsing failure](https://discuss.elastic.co/t/logstash-parsing-failure/178326)
**Author:** [@bkasrai](https://discuss.elastic.co/u/bkasrai)\ **Replies:** 2\ **Last updated:** [April 25, 2019, 2:07pm UTC](https://discuss.elastic.co/t/logstash-parsing-failure/178326 "2019-04-25T14:07:31Z")
I am having an issue getting certain records within my log file to parse and I do not know how to resolve this. Not every record is 1-1 and for records not containing these fields, it works fine and parses\\maps properly.… --- ## [Grok pattern for capturing string with CR in the middle](https://discuss.elastic.co/t/grok-pattern-for-capturing-string-with-cr-in-the-middle/178391)
**Author:** [@royboy](https://discuss.elastic.co/u/royboy)\ **Replies:** 1\ **Last updated:** [April 25, 2019, 1:56pm UTC](https://discuss.elastic.co/t/grok-pattern-for-capturing-string-with-cr-in-the-middle/178391 "2019-04-25T13:56:53Z")
Hi All new here and in general to Logs in general i have encountered problem to generate a Grok (for an extractor) pattern so it will capture string cut in the middle (sometimes) by a CR and continues, i want to captur… --- ## [Logstash s3 input plugin not processing gz files](https://discuss.elastic.co/t/logstash-s3-input-plugin-not-processing-gz-files/177641)
**Author:** [@gurumanoh](https://discuss.elastic.co/u/gurumanoh)\ **Replies:** 1\ **Last updated:** [April 25, 2019, 1:53pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-not-processing-gz-files/177641 "2019-04-25T13:53:34Z")
Hi, I have the cofiguration for processing s3 gz files. But getting below error, \[2019-04-19T20:38:33,341\]\[ERROR\]\[logstash.inputs.s3 \] Failed to read the file. Skip processing. {:filename=\>"C:/Users/mgurupr1/hom… --- ## [How to prevent json parsing of a field in Logstash?](https://discuss.elastic.co/t/how-to-prevent-json-parsing-of-a-field-in-logstash/178270)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 3\ **Last updated:** [April 25, 2019, 1:52pm UTC](https://discuss.elastic.co/t/how-to-prevent-json-parsing-of-a-field-in-logstash/178270 "2019-04-25T13:52:50Z")
Hi, I am using Logstasg GA 6.3.0. I am taking data from a kafka topic like below; input { kafka{ group\_id =\> "mygroup" topics =\> \["mytopic"\] bootstrap\_servers =\> "192.168.0.1:9092" codec =\> "json" } } everyt… --- ## [How to deal with the UTC format by date plug-in](https://discuss.elastic.co/t/how-to-deal-with-the-utc-format-by-date-plug-in/178413)
**Author:** [@ipv6ok](https://discuss.elastic.co/u/ipv6ok)\ **Replies:** 1\ **Last updated:** [April 25, 2019, 1:47pm UTC](https://discuss.elastic.co/t/how-to-deal-with-the-utc-format-by-date-plug-in/178413 "2019-04-25T13:47:42Z")
my nginx log timestamp like this: 20190425T092956Z but use date plug-in can't deal ! logstash code: date { match =\> \["timestamp", "ISO8601","yyyyMMddHHmmssZ"\] } IN kibana show @timestamp no change logstash ve… --- ## [Filter modules execution sequence](https://discuss.elastic.co/t/filter-modules-execution-sequence/178426)
**Author:** [@Alankrit\_Aman\_Mishra](https://discuss.elastic.co/u/Alankrit_Aman_Mishra)\ **Replies:** 1\ **Last updated:** [April 25, 2019, 1:42pm UTC](https://discuss.elastic.co/t/filter-modules-execution-sequence/178426 "2019-04-25T13:42:41Z")
In which sequence does the code present in the filter section is executed. I think it is not being executed sequentially ie the Order in which it is written in the file. I know that mutate has its own order of executio… --- ## [Parsing Rule works in Grok Debugger but not when running in logstash with MSSQL](https://discuss.elastic.co/t/parsing-rule-works-in-grok-debugger-but-not-when-running-in-logstash-with-mssql/178088)
**Author:** [@jbm](https://discuss.elastic.co/u/jbm)\ **Replies:** 1\ **Last updated:** [April 25, 2019, 1:01pm UTC](https://discuss.elastic.co/t/parsing-rule-works-in-grok-debugger-but-not-when-running-in-logstash-with-mssql/178088 "2019-04-25T13:01:44Z")
I am attempting to read Microsoft SQL Error logs in via filebeat -\> logstash. Testing the rule below against the logs below works in the grok debugger, but fails when running in production. %{TIMESTAMP\_ISO8601:time} %{W… --- ## [Create Index lifecycle policy in Logstash output to Elasticsearch \[step by step\]](https://discuss.elastic.co/t/create-index-lifecycle-policy-in-logstash-output-to-elasticsearch-step-by-step/178454)
**Author:** [@Pretbc](https://discuss.elastic.co/u/Pretbc)\ **Replies:** 0\ **Last updated:** [April 25, 2019, 12:28pm UTC](https://discuss.elastic.co/t/create-index-lifecycle-policy-in-logstash-output-to-elasticsearch-step-by-step/178454 "2019-04-25T12:28:51Z")
I have read almost all the topics on the Elastic page (tutorials) and topics from this forum, but I still have doubts how to set these settings. The policy on the life cycle of the index and Rolling over. Logstash conf… --- ## [Grok GUnicorn error log date format?](https://discuss.elastic.co/t/grok-gunicorn-error-log-date-format/178290)
**Author:** [@JJB](https://discuss.elastic.co/u/JJB)\ **Replies:** 3\ **Last updated:** [April 25, 2019, 12:05pm UTC](https://discuss.elastic.co/t/grok-gunicorn-error-log-date-format/178290 "2019-04-25T12:05:42Z")
GUnicorn uses this format for the error log file: \[2019-04-24 14:53:51 +0000\] \[10\] \[INFO\] Starting gunicorn 19.4.5 \[2019-04-24 14:53:51 +0000\] \[10\] \[INFO\] Listening at: http://0.0.0.0:12007 (10) Is the timestamp a stan… --- ## [Considering the last selection log](https://discuss.elastic.co/t/considering-the-last-selection-log/175096)
**Author:** [@satyajeet\_dutt](https://discuss.elastic.co/u/satyajeet_dutt)\ **Replies:** 11\ **Last updated:** [April 25, 2019, 10:31am UTC](https://discuss.elastic.co/t/considering-the-last-selection-log/175096 "2019-04-25T10:31:55Z")
hi there, I am stuck with a specific issue related to clickstream log. It's based on the selection made by the user.If a user selects something then the request gets added up in the request parameter. I am posting a sam… --- ## [Logstash restarts continually since last upgrade](https://discuss.elastic.co/t/logstash-restarts-continually-since-last-upgrade/178415)
**Author:** [@uhlhosting](https://discuss.elastic.co/u/uhlhosting)\ **Replies:** 0\ **Last updated:** [April 25, 2019, 9:48am UTC](https://discuss.elastic.co/t/logstash-restarts-continually-since-last-upgrade/178415 "2019-04-25T09:48:13Z")
I get logstash restart every 10 seconds , and all this after the upgrade from 6.6 to 6.7 and onward. Is this normal? Both kibana and elastisearch are up and running. Yes I run elastic on local ip 172.18.111.1 \[2019-03… --- ## [GROK pattern query](https://discuss.elastic.co/t/grok-pattern-query/177464)
**Author:** [@MiddlewareTeam](https://discuss.elastic.co/u/MiddlewareTeam)\ **Replies:** 2\ **Last updated:** [April 25, 2019, 9:40am UTC](https://discuss.elastic.co/t/grok-pattern-query/177464 "2019-04-25T09:40:21Z")
I have log as below: nbresp:500 ~ nbresp1:406 ~ nbresp2: ~ nbresp3:409 I am using GROK pattern as below %{WORD}:%{WORD:nbresp} ~ %{WORD}:%{WORD:nbresp1} ~ %{WORD}:%{WORD:nbresp2} ~ %{WORD}:%{WORD:nbresp3} ~ which is … --- ## [Can't find index when i create logstash .conf for send nginx log](https://discuss.elastic.co/t/cant-find-index-when-i-create-logstash-conf-for-send-nginx-log/178407)
**Author:** [@hasanroo10](https://discuss.elastic.co/u/hasanroo10)\ **Replies:** 0\ **Last updated:** [April 25, 2019, 9:11am UTC](https://discuss.elastic.co/t/cant-find-index-when-i-create-logstash-conf-for-send-nginx-log/178407 "2019-04-25T09:11:46Z")
\[ASK\] iam confused about can't find index in kibana when i check log no error but index cant find please help me this my log in logstash iam using version 7.0 --- ## [How to Parse SAP HANA DB Audit logs?](https://discuss.elastic.co/t/how-to-parse-sap-hana-db-audit-logs/178229)
**Author:** [@Vigneshwaran\_Mohanas](https://discuss.elastic.co/u/Vigneshwaran_Mohanas)\ **Replies:** 2\ **Last updated:** [April 25, 2019, 9:07am UTC](https://discuss.elastic.co/t/how-to-parse-sap-hana-db-audit-logs/178229 "2019-04-25T09:07:07Z")
Hi Team, I am new to ELK & GROK filtering . I am trying to parse SAP hana DB audit logs . I am facing one problem .. supposing if a log line consists of " ;;;;;;values\>" , if u notice, after 2 values I have 4 semicolon… --- ## [How to specify logstash config file when running it as a service](https://discuss.elastic.co/t/how-to-specify-logstash-config-file-when-running-it-as-a-service/178266)
**Author:** [@Mick\_O\_Gorman](https://discuss.elastic.co/u/Mick_O_Gorman)\ **Replies:** 2\ **Last updated:** [April 25, 2019, 8:17am UTC](https://discuss.elastic.co/t/how-to-specify-logstash-config-file-when-running-it-as-a-service/178266 "2019-04-25T08:17:19Z")
I can use logstash with the binaries by specifying the config file like this ./logstash -f ../config/logstash-sample.conf\` But when it is installed rather than using the binaries it seems their site says you start logs… --- ## [Running multiple logstash configurations in a single node ES 6.6.1 with pipelines.yml](https://discuss.elastic.co/t/running-multiple-logstash-configurations-in-a-single-node-es-6-6-1-with-pipelines-yml/178376)
**Author:** [@pavimay24](https://discuss.elastic.co/u/pavimay24)\ **Replies:** 0\ **Last updated:** [April 25, 2019, 6:36am UTC](https://discuss.elastic.co/t/running-multiple-logstash-configurations-in-a-single-node-es-6-6-1-with-pipelines-yml/178376 "2019-04-25T06:36:23Z")
Hi, I want to run multiple logstash configurations in a single node ES 6.6.1 with pipelines.yml. What is the best practice for configuring pipelines.yml in single node? what are changes need to be done in logstash.ym… --- ## [Unit test Logstash pipeline - Need to tokenize the pipeline config file](https://discuss.elastic.co/t/unit-test-logstash-pipeline-need-to-tokenize-the-pipeline-config-file/178371)
**Author:** [@SManorathna](https://discuss.elastic.co/u/SManorathna)\ **Replies:** 0\ **Last updated:** [April 25, 2019, 6:16am UTC](https://discuss.elastic.co/t/unit-test-logstash-pipeline-need-to-tokenize-the-pipeline-config-file/178371 "2019-04-25T06:16:19Z")
Hi, I am writing a unit test for the logstash pipeline. I need to extract all the grok type filters and test those in an isolated environment. Tests will be executed for each filter separately. I Can't use java for this… --- ## [Got a bad response code from server, but this code is not considered retryable \[Status : 400\]\]](https://discuss.elastic.co/t/got-a-bad-response-code-from-server-but-this-code-is-not-considered-retryable-status-400/178366)
**Author:** [@sarasa](https://discuss.elastic.co/u/sarasa)\ **Replies:** 0\ **Last updated:** [April 25, 2019, 5:33am UTC](https://discuss.elastic.co/t/got-a-bad-response-code-from-server-but-this-code-is-not-considered-retryable-status-400/178366 "2019-04-25T05:33:39Z")
There is a error in my logstash's logs. No idea whats happening. I keep getting this error time to time. Does anyoen know whats need to be done to solve this? docker-compose configurations- elasticsearch: image: docke… --- ## [Parse XML response and extract the attributes from it to form new fields](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090)
**Author:** [@kaushik.vankayala](https://discuss.elastic.co/u/kaushik.vankayala)\ **Replies:** 6\ **Last updated:** [April 25, 2019, 5:13am UTC](https://discuss.elastic.co/t/parse-xml-response-and-extract-the-attributes-from-it-to-form-new-fields/177090 "2019-04-25T05:13:23Z")
Hi There, I am working on monitoring a specific queue in ActiveMQ server. For the same i am using the xml utility which returns the structure of the queue like below; This XML file does not appear to have any style inf… --- ## [Lostash multi-pipeline queue behaviour](https://discuss.elastic.co/t/lostash-multi-pipeline-queue-behaviour/178359)
**Author:** [@sandybro](https://discuss.elastic.co/u/sandybro)\ **Replies:** 0\ **Last updated:** [April 25, 2019, 4:37am UTC](https://discuss.elastic.co/t/lostash-multi-pipeline-queue-behaviour/178359 "2019-04-25T04:37:08Z")
Hello Guys, Could you please help us to understand queue behaviour in case of multiple pipeline. We have more than 500 pipelines. As per Documentation, page capacity is allocated to all pipelines.So if we set queue.pag… --- ## [S3 Input Plugin Sincedb Time](https://discuss.elastic.co/t/s3-input-plugin-sincedb-time/178312)
**Author:** [@stevetso](https://discuss.elastic.co/u/stevetso)\ **Replies:** 0\ **Last updated:** [April 24, 2019, 6:26pm UTC](https://discuss.elastic.co/t/s3-input-plugin-sincedb-time/178312 "2019-04-24T18:26:28Z")
I am using the Logstash S3 Input Plugin to read the gz file in the S3 bucket and ingest into Elasticsearch. Here is the Conf file under /etc/logstash/conf.d: input { s3 { aws\_credentials\_file =\> "/etc/logstash/aws\_… --- ## [PUT mapping resulting in hidden fields , not appearing in 'Discover'](https://discuss.elastic.co/t/put-mapping-resulting-in-hidden-fields-not-appearing-in-discover/177794)
**Author:** [@ajayraghuraj](https://discuss.elastic.co/u/ajayraghuraj)\ **Replies:** 0\ **Last updated:** [April 22, 2019, 5:03am UTC](https://discuss.elastic.co/t/put-mapping-resulting-in-hidden-fields-not-appearing-in-discover/177794 "2019-04-22T05:03:42Z")
I added a field by updating all the indices using the 'PUT' API. But they appear as missing fields in the 'Available fields' and do not appear in the Fields 'Table' . Can someone help with this ? PUT logstashindex-\*… --- ## [Custom logstash image fails with java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/custom-logstash-image-fails-with-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/178318)
**Author:** [@Pouria\_Fewzee](https://discuss.elastic.co/u/Pouria_Fewzee)\ **Replies:** 6\ **Last updated:** [April 25, 2019, 12:59am UTC](https://discuss.elastic.co/t/custom-logstash-image-fails-with-java-lang-illegalstateexception-logstash-stopped-processing-because-of-an-error-systemexit-exit/178318 "2019-04-25T00:59:20Z")
Hi, the following is my logstash.conf and logstash.yml files: logstash.conf input { beats { port =\> 5044 } } filter {} output { http { http\_mehod =\> "post" url =\> "https://dydx.ngrok.io" } } logstash.yml http… --- ## [Clarify how timeout\_timestamp\_field works with timeouts in aggregate filter](https://discuss.elastic.co/t/clarify-how-timeout-timestamp-field-works-with-timeouts-in-aggregate-filter/178299)
**Author:** [@lens](https://discuss.elastic.co/u/lens)\ **Replies:** 3\ **Last updated:** [April 24, 2019, 9:04pm UTC](https://discuss.elastic.co/t/clarify-how-timeout-timestamp-field-works-with-timeouts-in-aggregate-filter/178299 "2019-04-24T21:04:15Z")
I am aggregating flows using timeouts based on a timestamp field in the events (start\_time), and I push\_map\_as\_event\_on\_timeout. Is my understanding of how this works correct? An event comes in with a new task\_id (x),… --- ## [Logstash Error registering plugin](https://discuss.elastic.co/t/logstash-error-registering-plugin/178307)
**Author:** [@cbeprem](https://discuss.elastic.co/u/cbeprem)\ **Replies:** 2\ **Last updated:** [April 24, 2019, 7:09pm UTC](https://discuss.elastic.co/t/logstash-error-registering-plugin/178307 "2019-04-24T19:09:54Z")
I am getting the following error. while trying to run S3 input \[2019-04-24T17:46:30,448\]\[INFO \]\[logstash.pipeline \] Starting pipeline {:pipeline\_id=\>"main", "pipeline.workers"=\>2, "pipeline.batch.size"=\>125, "pip… --- ## [JDBC Library Logstash 7.0.0](https://discuss.elastic.co/t/jdbc-library-logstash-7-0-0/178304)
**Author:** [@Carlos\_Rojas](https://discuss.elastic.co/u/Carlos_Rojas)\ **Replies:** 5\ **Last updated:** [April 24, 2019, 6:59pm UTC](https://discuss.elastic.co/t/jdbc-library-logstash-7-0-0/178304 "2019-04-24T18:59:14Z")
HI, i'm using logstash-input-jdbc, and this is installed correctly Step 3/4 : RUN logstash-plugin install logstash-input-jdbc ---\> Running in 3fd593a57661 Validating logstash-input-jdbc Installing logstash-input-jd… --- ## [Parse string escaped JSON inside message](https://discuss.elastic.co/t/parse-string-escaped-json-inside-message/178212)
**Author:** [@jfreeman](https://discuss.elastic.co/u/jfreeman)\ **Replies:** 2\ **Last updated:** [April 24, 2019, 6:38pm UTC](https://discuss.elastic.co/t/parse-string-escaped-json-inside-message/178212 "2019-04-24T18:38:21Z")
In logstash the event I receive from http input is { "event" =\> { "@timestamp" =\>2019-04-24T09:17:29.940 Z, "headers" =\> { "http\_host" =\>"localhost:5055 … --- ## [Can it possible to single syslog server use to get the log from all Network device down the line logstatsh](https://discuss.elastic.co/t/can-it-possible-to-single-syslog-server-use-to-get-the-log-from-all-network-device-down-the-line-logstatsh/178295)
**Author:** [@sarban.kumar](https://discuss.elastic.co/u/sarban.kumar)\ **Replies:** 0\ **Last updated:** [April 24, 2019, 4:17pm UTC](https://discuss.elastic.co/t/can-it-possible-to-single-syslog-server-use-to-get-the-log-from-all-network-device-down-the-line-logstatsh/178295 "2019-04-24T16:17:15Z")
\*\*We have Forti Analyser who collects all the Network devices log \*\* and we are sending log from Forti Analyzer to Syslog server. My question is that : - Can we use single syslog server to get all the logs from Fortian… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=557) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=559)