# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=457 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 458 --- ## [Dead Letter Queue DLQ](https://discuss.elastic.co/t/dead-letter-queue-dlq/205133)
**Author:** [@cglencr](https://discuss.elastic.co/u/cglencr)\ **Replies:** 1\ **Last updated:** [October 24, 2019, 7:51pm UTC](https://discuss.elastic.co/t/dead-letter-queue-dlq/205133 "2019-10-24T19:51:29Z")
I have searched but cannot find any documentation or discussion on how to test the DLQ in 7.4. I have set the logstash.yml for DLQ as: dead\_letter\_queue.enable: true path.dead\_letter\_queue: "C:/Applications/ElasticSea… --- ## [How to add timestamp to aggregate filter](https://discuss.elastic.co/t/how-to-add-timestamp-to-aggregate-filter/205122)
**Author:** [@krishna\_bhargav](https://discuss.elastic.co/u/krishna_bhargav)\ **Replies:** 2\ **Last updated:** [October 24, 2019, 6:02pm UTC](https://discuss.elastic.co/t/how-to-add-timestamp-to-aggregate-filter/205122 "2019-10-24T18:02:15Z")
filter contents below, timeevent should display current timestamp aggregate { task\_id =\> "%{ENVIRONMENT\_ID}\_%{SPN\_FIRM\_ID}\_%{ENTITY\_TYPE}\_%{ENTITY\_ID}" code =\> ' map\["keyInformation"\] ||= {} map\["keyInformati… --- ## [Logstash Filter - creating list from multiple events](https://discuss.elastic.co/t/logstash-filter-creating-list-from-multiple-events/204561)
**Author:** [@krishna\_bhargav](https://discuss.elastic.co/u/krishna_bhargav)\ **Replies:** 3\ **Last updated:** [October 24, 2019, 5:04pm UTC](https://discuss.elastic.co/t/logstash-filter-creating-list-from-multiple-events/204561 "2019-10-24T17:04:41Z")
I am working on below schema, grouping multiple households based on the keyinformation. We could achieve this with the logstash aggregate filter. But this is having performance issue and also cannot run with multiple thr… --- ## [REST API using http\_poller](https://discuss.elastic.co/t/rest-api-using-http-poller/205107)
**Author:** [@akhilsharma.in](https://discuss.elastic.co/u/akhilsharma.in)\ **Replies:** 3\ **Last updated:** [October 24, 2019, 5:01pm UTC](https://discuss.elastic.co/t/rest-api-using-http-poller/205107 "2019-10-24T17:01:19Z")
We have to implement REST API using http\_poller if it is possible. It has two calls. 1st POST creates a token which has to be used by 2nd REST GET. Can I pass the output token of 1st call to the input of other http\_pol… --- ## [Logstash - Creating multiple pipelines for beats inputs on same port](https://discuss.elastic.co/t/logstash-creating-multiple-pipelines-for-beats-inputs-on-same-port/204562)
**Author:** [@krishna\_bhargav](https://discuss.elastic.co/u/krishna_bhargav)\ **Replies:** 2\ **Last updated:** [October 24, 2019, 4:43pm UTC](https://discuss.elastic.co/t/logstash-creating-multiple-pipelines-for-beats-inputs-on-same-port/204562 "2019-10-24T16:43:43Z")
Am trying to create multiple pipelines beatsPipeline1 and beatsPipeline2 in logstash listening to beats events from same port but getting error saying address in use for second pipeline: pipeline.id: beatsPipeline1 pa… --- ## [Split XML](https://discuss.elastic.co/t/split-xml/204949)
**Author:** [@Jim\_Thunder](https://discuss.elastic.co/u/Jim_Thunder)\ **Replies:** 12\ **Last updated:** [October 24, 2019, 4:14pm UTC](https://discuss.elastic.co/t/split-xml/204949 "2019-10-24T16:14:44Z")
I have an XML doc being sent to a logstash server/ELK server. This is the portion of the xml I'm trying to work with: \\\ \\\OMXEOC0\\\\\\100\\\\\\00E4\\\ \\\ \\\ **Author:** [@Devopsio](https://discuss.elastic.co/u/Devopsio)\ **Replies:** 1\ **Last updated:** [October 24, 2019, 3:38pm UTC](https://discuss.elastic.co/t/no-logstash-menu-in-kibana-ui/205094 "2019-10-24T15:38:09Z") I have an entire ELK stack in docker compose but only Kibana and Elasticsearch are shown. Logs show no errors and it seems, that Logstash communicates with the rest of stack. Here's my logstash.yml: xpack.monitoring.ela… --- ## [Logstash HA](https://discuss.elastic.co/t/logstash-ha/205112)
**Author:** [@dmoronta](https://discuss.elastic.co/u/dmoronta)\ **Replies:** 0\ **Last updated:** [October 24, 2019, 3:27pm UTC](https://discuss.elastic.co/t/logstash-ha/205112 "2019-10-24T15:27:37Z")
Hi all, please can someone tell me if is possible made a logstash HA solution? I can't use a third party load balancer. The logstash input is a very critical procces to me, so I am thinking in a kind of vip between two … --- ## [Logstash Filter for Specific Winlogs Value](https://discuss.elastic.co/t/logstash-filter-for-specific-winlogs-value/204962)
**Author:** [@uklipse](https://discuss.elastic.co/u/uklipse)\ **Replies:** 3\ **Last updated:** [October 24, 2019, 3:21pm UTC](https://discuss.elastic.co/t/logstash-filter-for-specific-winlogs-value/204962 "2019-10-24T15:21:34Z")
I'm trying to filter out some specific values for the Windows event id 4663. We have one computer that generates 99% of these events and we want to filter that out. Below is what I have but it's not working. The add\_tag … --- ## [Logstash create geo\_point](https://discuss.elastic.co/t/logstash-create-geo-point/205111)
**Author:** [@haythem](https://discuss.elastic.co/u/haythem)\ **Replies:** 0\ **Last updated:** [October 24, 2019, 3:16pm UTC](https://discuss.elastic.co/t/logstash-create-geo-point/205111 "2019-10-24T15:16:47Z")
hello guys , i'm having a problem when i insert a geopoint in elasticsearch,so i've created an index using this command PUT my\_index then i've created a geo point field in the created index using this command and then … --- ## [Logsatsh geopoint](https://discuss.elastic.co/t/logsatsh-geopoint/205090)
**Author:** [@haythem](https://discuss.elastic.co/u/haythem)\ **Replies:** 3\ **Last updated:** [October 24, 2019, 3:15pm UTC](https://discuss.elastic.co/t/logsatsh-geopoint/205090 "2019-10-24T15:15:12Z")
Hi guys , so i'm working on a project now when i have to load data from oracle database and convert two columns(longitude,latitude) to a geo\_point , so i've created an index in elasticsearch and i created a geo\_point map… --- ## [Logstash Grok Filter failure with match](https://discuss.elastic.co/t/logstash-grok-filter-failure-with-match/204616)
**Author:** [@fuXz](https://discuss.elastic.co/u/fuXz)\ **Replies:** 5\ **Last updated:** [October 24, 2019, 2:55pm UTC](https://discuss.elastic.co/t/logstash-grok-filter-failure-with-match/204616 "2019-10-24T14:55:41Z")
Unfortunately I'm stuck with some Logstash basics. I want to use Logstash to get logs from a Sophos UTM and do something with the data. I already tried different things from the documentation, only with little success ta… --- ## [Proxy\_url on Logstash with x-pack](https://discuss.elastic.co/t/proxy-url-on-logstash-with-x-pack/205101)
**Author:** [@hardwarehacker](https://discuss.elastic.co/u/hardwarehacker)\ **Replies:** 0\ **Last updated:** [October 24, 2019, 2:54pm UTC](https://discuss.elastic.co/t/proxy-url-on-logstash-with-x-pack/205101 "2019-10-24T14:54:59Z")
My Logstash is running behind a proxy server with all the logs successfully delivered to Elasticsearch Cloud. Now I'm trying to enable x-pack on logstash and looks like xpack is not following my proxy. Besides that I can… --- ## [Nginx useragent mapper\_parsing\_exception](https://discuss.elastic.co/t/nginx-useragent-mapper-parsing-exception/205089)
**Author:** [@stevesimpson](https://discuss.elastic.co/u/stevesimpson)\ **Replies:** 2\ **Last updated:** [October 24, 2019, 2:40pm UTC](https://discuss.elastic.co/t/nginx-useragent-mapper-parsing-exception/205089 "2019-10-24T14:40:02Z")
Hi, I'm having a bit of trouble understanding why I'm getting the below errors when trying to Index Nginx access logs. \[2019-10-24T12:52:56,876\]\[WARN \]\[logstash.outputs.elasticsearch\] Could not index event to Elasticse… --- ## [Create new field from existing field using REGEX](https://discuss.elastic.co/t/create-new-field-from-existing-field-using-regex/204709)
**Author:** [@Marta\_Zagrajek](https://discuss.elastic.co/u/Marta_Zagrajek)\ **Replies:** 3\ **Last updated:** [October 24, 2019, 2:19pm UTC](https://discuss.elastic.co/t/create-new-field-from-existing-field-using-regex/204709 "2019-10-24T14:19:46Z")
I have grok like that from apache logs: grok { match =\> { "message" =\> '%{IP:ip}\\s(%{GREEDYDATA:logname})?\\s(%{GREEDYDATA:remote\_user})?\\s\[%{HTTPDATE:timestamp}\]\\s"%{WORD:metoda}\\s%{NOTSPACE:request}\\s(HTTP/%{NUMBER:… --- ## [+FORMAT syntax works for @timestamp field only?](https://discuss.elastic.co/t/format-syntax-works-for-timestamp-field-only/205037)
**Author:** [@jetnet](https://discuss.elastic.co/u/jetnet)\ **Replies:** 3\ **Last updated:** [October 24, 2019, 2:06pm UTC](https://discuss.elastic.co/t/format-syntax-works-for-timestamp-field-only/205037 "2019-10-24T14:06:17Z")
can some other timestamp field be used with +FORMAT syntax? index =\> "${ELASTICSEARCH\_INDEX}.%{+yyyy.MM.dd}" the above config line would use @timestamp as the input for the format definition, but I'd like to use some o… --- ## [Grok parse not work](https://discuss.elastic.co/t/grok-parse-not-work/205029)
**Author:** [@111228](https://discuss.elastic.co/u/111228)\ **Replies:** 2\ **Last updated:** [October 24, 2019, 1:16pm UTC](https://discuss.elastic.co/t/grok-parse-not-work/205029 "2019-10-24T13:16:55Z")
Hello to everyone, I have a problem with grok parsing. trying to parse fiend log.file.path, I want to put out domain name for example: I Have path /var/www/httpd-logs/mydomain.net.access.log I'm using %{GREEDYDATA}/%… --- ## [Logstash pipeline problem](https://discuss.elastic.co/t/logstash-pipeline-problem/204902)
**Author:** [@basavyr](https://discuss.elastic.co/u/basavyr)\ **Replies:** 3\ **Last updated:** [October 24, 2019, 1:10pm UTC](https://discuss.elastic.co/t/logstash-pipeline-problem/204902 "2019-10-24T13:10:24Z")
So I have to achieve a Logstash setup where I have two (or maybe even more) log files, each with its own input and filters (output will always be sent to Elastic and then visualized in Kibana). I saw this is very doable… --- ## [ERROR in cmd - logstash](https://discuss.elastic.co/t/error-in-cmd-logstash/204875)
**Author:** [@Marfig](https://discuss.elastic.co/u/Marfig)\ **Replies:** 4\ **Last updated:** [October 24, 2019, 12:59pm UTC](https://discuss.elastic.co/t/error-in-cmd-logstash/204875 "2019-10-24T12:59:42Z")
Hi I have install Logstash, ElasticSearch and Kibana, without any problems. But when I going to rund the Logstash conf file through the Command Prompt or PowerShell I get the same error messages... see picture: My co… --- ## [Syslog Output Plugin](https://discuss.elastic.co/t/syslog-output-plugin/205039)
**Author:** [@jforsain](https://discuss.elastic.co/u/jforsain)\ **Replies:** 0\ **Last updated:** [October 24, 2019, 9:58am UTC](https://discuss.elastic.co/t/syslog-output-plugin/205039 "2019-10-24T09:58:01Z")
Hello, I'm using the syslog output plugin in my Logatsh file conf, but during the execution ( /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/apache.conf ) of the logstash conf file, it shows multiple errors : … --- ## [How to Fetch Data of last month from logstash](https://discuss.elastic.co/t/how-to-fetch-data-of-last-month-from-logstash/205058)
**Author:** [@vish1](https://discuss.elastic.co/u/vish1)\ **Replies:** 0\ **Last updated:** [October 24, 2019, 12:06pm UTC](https://discuss.elastic.co/t/how-to-fetch-data-of-last-month-from-logstash/205058 "2019-10-24T12:06:19Z")
I am trying to input data from mongodb of last month, so i want to know how we need to describe in conf file in filter section. --- ## [Logtash to elasticsearch connection error using certificate](https://discuss.elastic.co/t/logtash-to-elasticsearch-connection-error-using-certificate/204799)
**Author:** [@akshatBais](https://discuss.elastic.co/u/akshatBais)\ **Replies:** 23\ **Last updated:** [October 24, 2019, 6:17am UTC](https://discuss.elastic.co/t/logtash-to-elasticsearch-connection-error-using-certificate/204799 "2019-10-24T06:17:03Z")
Hi All, I am facing this isssue while connecting elasticsearch with logstash over certificate . I have mentioned the logstash.yml path in startup.options file is LS\_HOME and LS\_SETTINGS\_DIR . Still it gives me the foll… --- ## [Logstash parse arrays in array by grok](https://discuss.elastic.co/t/logstash-parse-arrays-in-array-by-grok/204777)
**Author:** [@newkate12](https://discuss.elastic.co/u/newkate12)\ **Replies:** 2\ **Last updated:** [October 24, 2019, 2:24am UTC](https://discuss.elastic.co/t/logstash-parse-arrays-in-array-by-grok/204777 "2019-10-24T02:24:39Z")
Hi I'm trying to import csv and parse array data in a specific column. My csv data is like this. TXN\_TIME|TENANT\_ID|TAG\_ID|TAG\_VALUE 2019-06-14 10:00:00.693|T00|GTW000000007-001-1|\[\[1.9744,3.0427,1.9170\],\[1.9743,3.006… --- ## [Persistent Queue not removing page files despite event count being 0](https://discuss.elastic.co/t/persistent-queue-not-removing-page-files-despite-event-count-being-0/204983)
**Author:** [@Melaniemj](https://discuss.elastic.co/u/Melaniemj)\ **Replies:** 0\ **Last updated:** [October 24, 2019, 1:33am UTC](https://discuss.elastic.co/t/persistent-queue-not-removing-page-files-despite-event-count-being-0/204983 "2019-10-24T01:33:04Z")
We are using logstash v 6.8.2 with the persistent queue enabled. Recently, one of our clusters started to show signs that the queue had data, but the event count was 0. After observing the behavior for several days, I've… --- ## [Data Import Question](https://discuss.elastic.co/t/data-import-question/204112)
**Author:** [@mbrown007](https://discuss.elastic.co/u/mbrown007)\ **Replies:** 1\ **Last updated:** [October 24, 2019, 1:17am UTC](https://discuss.elastic.co/t/data-import-question/204112 "2019-10-24T01:17:45Z")
I have data in my mariaDB/MySQL database that I would like to graph in elasticsearch. I am new to ELK and from what I have seen this is something that I would use logstash for correct? I also have data in a csv file tha… --- ## [Copy logstash template](https://discuss.elastic.co/t/copy-logstash-template/204694)
**Author:** [@SirJune](https://discuss.elastic.co/u/SirJune)\ **Replies:** 4\ **Last updated:** [October 24, 2019, 1:17am UTC](https://discuss.elastic.co/t/copy-logstash-template/204694 "2019-10-24T01:17:29Z")
i would like to copy logstash template to a newname template. I did a GET \_template/logstash , copied and edited (replace the word "logstash" with "blah") Did a PUT but got an error of: { "error": { "root\_cause… --- ## [Question about multiple inputs](https://discuss.elastic.co/t/question-about-multiple-inputs/204939)
**Author:** [@xibal](https://discuss.elastic.co/u/xibal)\ **Replies:** 4\ **Last updated:** [October 23, 2019, 9:02pm UTC](https://discuss.elastic.co/t/question-about-multiple-inputs/204939 "2019-10-23T21:02:56Z")
Hi, I'm migrating orders and their itemsin a joined SQL and using aggregate filter to list items as a nested field under each order. Each document's id is the respective orders table id. Now, in another table I have ord… --- ## [Running Logstash FILE config files "once"](https://discuss.elastic.co/t/running-logstash-file-config-files-once/204269)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 1\ **Last updated:** [October 23, 2019, 8:54pm UTC](https://discuss.elastic.co/t/running-logstash-file-config-files-once/204269 "2019-10-23T20:54:01Z")
Hello, How can I run a logstash config ingesting a csv or xml or file so that it is not continually running? The logstash config file can keep running until it is manually shut down, however, how can I have it run once … --- ## [Logstash wont start correctly with systemctl](https://discuss.elastic.co/t/logstash-wont-start-correctly-with-systemctl/204952)
**Author:** [@jmoran8888](https://discuss.elastic.co/u/jmoran8888)\ **Replies:** 0\ **Last updated:** [October 23, 2019, 8:04pm UTC](https://discuss.elastic.co/t/logstash-wont-start-correctly-with-systemctl/204952 "2019-10-23T20:04:48Z")
I have logstash 7.4 installed - when i run it from the command line via this /usr/share/logstash/bin/logstash --path.settings /etc/logstash it works fine - i see the log file and its showing up in the Kibana Monitoring… --- ## [What to be used instead of "type"](https://discuss.elastic.co/t/what-to-be-used-instead-of-type/204930)
**Author:** [@Senthil\_ak](https://discuss.elastic.co/u/Senthil_ak)\ **Replies:** 3\ **Last updated:** [October 23, 2019, 6:22pm UTC](https://discuss.elastic.co/t/what-to-be-used-instead-of-type/204930 "2019-10-23T18:22:32Z")
Hi Team, I'm using type for separating the logs category, but in the logs file I am getting error (warning) message like below. \[2019-10-22T14:48:01,104\]\[WARN \]\[logstash.outputs.elasticsearch\] Detected a 6.x and above… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=456) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=458)