# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=150 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 151 --- ## [Logstash.filters.mutate](https://discuss.elastic.co/t/logstash-filters-mutate/301241)
**Author:** [@INS](https://discuss.elastic.co/u/INS)\ **Replies:** 6\ **Last updated:** [April 1, 2022, 7:17am UTC](https://discuss.elastic.co/t/logstash-filters-mutate/301241 "2022-04-01T07:17:34Z")
Hi I'm facing the problem with filters.mutate \[2022-03-31T17:49:26,980\]\[WARN \]\[logstash.filters.mutate \]\[subscriberstrunk\]\[11ac345049d68c6fc55061d82c036ef3c26cf4e2fe68230c4df255bc3403de66\] Exception caught while apply… --- ## [Logstash 401 error](https://discuss.elastic.co/t/logstash-401-error/301247)
**Author:** [@chaishiqi](https://discuss.elastic.co/u/chaishiqi)\ **Replies:** 2\ **Last updated:** [April 1, 2022, 6:28am UTC](https://discuss.elastic.co/t/logstash-401-error/301247 "2022-04-01T06:28:08Z")
Here is my logstash config: input { file { path =\> "/var/log/nginx/access.log" codec =\> "json" } } output { elasticsearch { hosts =\> \["https://10.0.5.132:9200"\] ssl =\> true cacert =\> "/etc/logs… --- ## [No space left on device - Logstash](https://discuss.elastic.co/t/no-space-left-on-device-logstash/301154)
**Author:** [@Victorv18](https://discuss.elastic.co/u/Victorv18)\ **Replies:** 3\ **Last updated:** [March 31, 2022, 11:18pm UTC](https://discuss.elastic.co/t/no-space-left-on-device-logstash/301154 "2022-03-31T23:18:52Z")
I have an ELK stack, hosted on a Linux, an EC2 VM running at AWS and my logstash works in the background on that same environment I used to start logstash as a service using "systemctl start logstash" and it works norma… --- ## [Logstash keystore uninitialized constant I18n](https://discuss.elastic.co/t/logstash-keystore-uninitialized-constant-i18n/301270)
**Author:** [@DonKool](https://discuss.elastic.co/u/DonKool)\ **Replies:** 0\ **Last updated:** [March 31, 2022, 10:08pm UTC](https://discuss.elastic.co/t/logstash-keystore-uninitialized-constant-i18n/301270 "2022-03-31T22:08:31Z")
Running version 8.1.1 of ELK in Docker containers. Trying to secure the Logstash API on port 9600. This requires the creation of a keystore file. When trying to create a Logstash keystore file, get the following error… --- ## [Logstash conf file is not creating an index in ES](https://discuss.elastic.co/t/logstash-conf-file-is-not-creating-an-index-in-es/301243)
**Author:** [@yash\_196](https://discuss.elastic.co/u/yash_196)\ **Replies:** 1\ **Last updated:** [March 31, 2022, 6:42pm UTC](https://discuss.elastic.co/t/logstash-conf-file-is-not-creating-an-index-in-es/301243 "2022-03-31T18:42:31Z")
Logstash conf file is not creating an index in ES when using docker-compose. This is my logstash.conf code, can anyone help me figure out the solution? input{ elasticsearch { hosts =\> "elasticsearch:9200" … --- ## [Multiple grok pattern, really multiple?](https://discuss.elastic.co/t/multiple-grok-pattern-really-multiple/301166)
**Author:** [@lgllrd](https://discuss.elastic.co/u/lgllrd)\ **Replies:** 1\ **Last updated:** [March 31, 2022, 5:05pm UTC](https://discuss.elastic.co/t/multiple-grok-pattern-really-multiple/301166 "2022-03-31T17:05:04Z")
Hello, I'm trying to use the multiple grok pattern as in the documentation ( Grok filter plugin | Logstash Reference \[8.1\] | Elastic ) but I notice that it only handles the first two patterns. Here is my filter : { … --- ## [HTTP Filter --- Body Syntax](https://discuss.elastic.co/t/http-filter-body-syntax/301217)
**Author:** [@JeremyP](https://discuss.elastic.co/u/JeremyP)\ **Replies:** 0\ **Last updated:** [March 31, 2022, 2:59pm UTC](https://discuss.elastic.co/t/http-filter-body-syntax/301217 "2022-03-31T14:59:21Z")
Hello, I'm trying to use an HTTP filter with a POST and body and I'm having a hard time nailing down the syntax. The JSON body which I can get working using postman is as follows: { "filters": \[ { "day… --- ## [Elasticsearch-Input to File-Output: How to ensure query or index has been fully exported?](https://discuss.elastic.co/t/elasticsearch-input-to-file-output-how-to-ensure-query-or-index-has-been-fully-exported/300973)
**Author:** [@aru](https://discuss.elastic.co/u/aru)\ **Replies:** 2\ **Last updated:** [March 31, 2022, 12:10pm UTC](https://discuss.elastic.co/t/elasticsearch-input-to-file-output-how-to-ensure-query-or-index-has-been-fully-exported/300973 "2022-03-31T12:10:59Z")
We need to extract all documents of an index from Elasticsearch to a file. So our pipeline consists of an Elasticsearch input and a file output. At the time of querying Elasticsearch we know that there will be no further… --- ## [Json split multiple event in Logstash Pipeline](https://discuss.elastic.co/t/json-split-multiple-event-in-logstash-pipeline/301080)
**Author:** [@Purushottam22](https://discuss.elastic.co/u/Purushottam22)\ **Replies:** 3\ **Last updated:** [March 31, 2022, 1:05pm UTC](https://discuss.elastic.co/t/json-split-multiple-event-in-logstash-pipeline/301080 "2022-03-31T13:05:40Z")
Hi All, I am trying to ingest the python script output using exec input plugin filter but i am facing issue while performing split operation on message. I am not sure how can I split into fields, below sample output whi… --- ## [Docker - Setup Generated Certs - Logstash Permission Denied](https://discuss.elastic.co/t/docker-setup-generated-certs-logstash-permission-denied/301133)
**Author:** [@lluked](https://discuss.elastic.co/u/lluked)\ **Replies:** 1\ **Last updated:** [March 31, 2022, 11:28am UTC](https://discuss.elastic.co/t/docker-setup-generated-certs-logstash-permission-denied/301133 "2022-03-31T11:28:32Z")
Hi All, I'm trying to setup the elk stack with docker-compose, however Logstash cannot read the mounted certs directory, I'm guessing this is a permissions issue however I struggle to see why Elasticsearch, Kibana and F… --- ## [Data\_stream\_namespace ignored?](https://discuss.elastic.co/t/data-stream-namespace-ignored/301183)
**Author:** [@EvertM](https://discuss.elastic.co/u/EvertM)\ **Replies:** 0\ **Last updated:** [March 31, 2022, 10:51am UTC](https://discuss.elastic.co/t/data-stream-namespace-ignored/301183 "2022-03-31T10:51:49Z")
Hi all, I have the following in output: data\_stream =\> "true" data\_stream\_type =\> "logs" data\_stream\_dataset =\> "synology" data\_stream\_namespace =\> "yournamehere" However, I end up with a datastream named logs-synolog… --- ## [Unable to use grok pattern on GREEDYDATA message](https://discuss.elastic.co/t/unable-to-use-grok-pattern-on-greedydata-message/301086)
**Author:** [@rahulgupta18](https://discuss.elastic.co/u/rahulgupta18)\ **Replies:** 4\ **Last updated:** [March 31, 2022, 10:04am UTC](https://discuss.elastic.co/t/unable-to-use-grok-pattern-on-greedydata-message/301086 "2022-03-31T10:04:32Z")
Hi, I have this log message - 2022-03-08 04:16:04 \[DEBUG\] Creating linked clone: from Template-CentOS, to CentOS-001122 My logstash config. file is as follows - filter { grok { match =\> { "message" … --- ## [Logstash output to Kibana](https://discuss.elastic.co/t/logstash-output-to-kibana/300919)
**Author:** [@RomanKau](https://discuss.elastic.co/u/RomanKau)\ **Replies:** 1\ **Last updated:** [March 31, 2022, 8:28am UTC](https://discuss.elastic.co/t/logstash-output-to-kibana/300919 "2022-03-31T08:28:02Z")
Running Windows 10, Logstash 8.1.0, Elasticsearch, kibana and filebeat 8.0.0 all on the same machine. Getting the data from filebeat to kibana works great but the problem is that the whole log is in the field message an… --- ## [Ssl\_verification\_mode in http\_poller input plugin "An exception happened when converging configuration"](https://discuss.elastic.co/t/ssl-verification-mode-in-http-poller-input-plugin-an-exception-happened-when-converging-configuration/300924)
**Author:** [@Syed\_Saqlain\_Hussain](https://discuss.elastic.co/u/Syed_Saqlain_Hussain)\ **Replies:** 4\ **Last updated:** [March 31, 2022, 3:48am UTC](https://discuss.elastic.co/t/ssl-verification-mode-in-http-poller-input-plugin-an-exception-happened-when-converging-configuration/300924 "2022-03-31T03:48:41Z")
I have this configuration file using http\_poller input plugin. input { http\_poller { urls =\> { myurl2 =\> { # Supports all options supported by ruby's Manticore HTTP client method =\> get … --- ## [How to understand logstash pipeline-to-pipeline patterns?](https://discuss.elastic.co/t/how-to-understand-logstash-pipeline-to-pipeline-patterns/301044)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 2\ **Last updated:** [March 31, 2022, 12:50am UTC](https://discuss.elastic.co/t/how-to-understand-logstash-pipeline-to-pipeline-patterns/301044 "2022-03-31T00:50:15Z")
Currently I use logstash to operate pipeline.yml, the contents are as follows, according to my understanding, a line of pipeline.id should be a thread, it should be independent, but recently I found that there is a probl… --- ## [Logstash ruby code DateTime fails](https://discuss.elastic.co/t/logstash-ruby-code-datetime-fails/301137)
**Author:** [@scottfred](https://discuss.elastic.co/u/scottfred)\ **Replies:** 4\ **Last updated:** [March 30, 2022, 8:19pm UTC](https://discuss.elastic.co/t/logstash-ruby-code-datetime-fails/301137 "2022-03-30T20:19:16Z")
I've tried parsing this date string, "2022-01-25 08:30:22.126714" a few different ways in Logstash, but I keep coming up short. I don't want to use filter { date {} }... I should be able to use this ruby code because i… --- ## [Logstash aggregate filter not working for GCP MySQL slow query logs](https://discuss.elastic.co/t/logstash-aggregate-filter-not-working-for-gcp-mysql-slow-query-logs/301103)
**Author:** [@Chris\_Pinto](https://discuss.elastic.co/u/Chris_Pinto)\ **Replies:** 2\ **Last updated:** [March 30, 2022, 6:53pm UTC](https://discuss.elastic.co/t/logstash-aggregate-filter-not-working-for-gcp-mysql-slow-query-logs/301103 "2022-03-30T18:53:53Z")
Hi Team, I'm trying to send Google Cloud MySQL slow query logs from GCP Cloud Logging to elastic through Logstash pubsub input. The Slow query logs are in the JSON format and Pubsub has a limitation - it does not main… --- ## [Filter to add fiels of on nested JSON name value fields](https://discuss.elastic.co/t/filter-to-add-fiels-of-on-nested-json-name-value-fields/300848)
**Author:** [@DI\_Ralph](https://discuss.elastic.co/u/DI_Ralph)\ **Replies:** 4\ **Last updated:** [March 30, 2022, 6:26pm UTC](https://discuss.elastic.co/t/filter-to-add-fiels-of-on-nested-json-name-value-fields/300848 "2022-03-30T18:26:40Z")
Hi, In Logstash I'm trying to create new fields from Nested JSON where the value of "Name" is part of the field name and "Value" is the actual value of the new field. For example I got part of an audit log here below. … --- ## [Keep only domain.tld in field with various subdomains](https://discuss.elastic.co/t/keep-only-domain-tld-in-field-with-various-subdomains/301097)
**Author:** [@andre22](https://discuss.elastic.co/u/andre22)\ **Replies:** 1\ **Last updated:** [March 30, 2022, 5:34pm UTC](https://discuss.elastic.co/t/keep-only-domain-tld-in-field-with-various-subdomains/301097 "2022-03-30T17:34:16Z")
Hi, in my DNS logs I have a lot of very long URLs, for example those from CDNs. They can have one or more levels of subdomains, and I am looking to remove these. For example: safebrowsing.googleapis.com \> googleapis.c… --- ## [Logstash.yaml configuration xpack.management.elasticsearch.hosts (Elastic 7.10.)](https://discuss.elastic.co/t/logstash-yaml-configuration-xpack-management-elasticsearch-hosts-elastic-7-10/301114)
**Author:** [@newschapmj1](https://discuss.elastic.co/u/newschapmj1)\ **Replies:** 0\ **Last updated:** [March 30, 2022, 3:22pm UTC](https://discuss.elastic.co/t/logstash-yaml-configuration-xpack-management-elasticsearch-hosts-elastic-7-10/301114 "2022-03-30T15:22:12Z")
We have a 7 node Elasticsearch cluster (v7.10). Is it good practice to point xpack.management.Elasticsearch.hosts: to all the cluster nodes ? At the moment it is pointing to the 1st 3 nodes which was the original clus… --- ## [S3 input plugin does not recognise Glacier Flexible Retrieval](https://discuss.elastic.co/t/s3-input-plugin-does-not-recognise-glacier-flexible-retrieval/299875)
**Author:** [@banana](https://discuss.elastic.co/u/banana)\ **Replies:** 1\ **Last updated:** [March 30, 2022, 2:42pm UTC](https://discuss.elastic.co/t/s3-input-plugin-does-not-recognise-glacier-flexible-retrieval/299875 "2022-03-30T14:42:06Z")
HI All, We have noticed that the s3 input plugin does not recognise Glacier Flexible Retrieval storage classes. The result is that logstash takes forever to process our files. We would appreciate some input on how to … --- ## [Logstash pipelines reduces the number of events sent](https://discuss.elastic.co/t/logstash-pipelines-reduces-the-number-of-events-sent/300000)
**Author:** [@zpltn](https://discuss.elastic.co/u/zpltn)\ **Replies:** 1\ **Last updated:** [March 30, 2022, 8:22am UTC](https://discuss.elastic.co/t/logstash-pipelines-reduces-the-number-of-events-sent/300000 "2022-03-30T08:22:28Z")
Hi there, we have Metricbeat, Filebeat, Logstash and Elasticsearch running on an OpenShift Cluster. We are using Logstash to configure multiples pipelines to filter the logs for some namespaces and Beats types. Our sta… --- ## [Elasticsearch filter plugin "hosts" does not fail over](https://discuss.elastic.co/t/elasticsearch-filter-plugin-hosts-does-not-fail-over/299048)
**Author:** [@Zhang\_Jun\_Wu](https://discuss.elastic.co/u/Zhang_Jun_Wu)\ **Replies:** 1\ **Last updated:** [March 30, 2022, 6:37am UTC](https://discuss.elastic.co/t/elasticsearch-filter-plugin-hosts-does-not-fail-over/299048 "2022-03-30T06:37:53Z")
Hi, I am trying logstash 7.17.1 with a two-node es cluster (es 7.17.0), which are running on localhost:9200 and localhost:9201. Both logstash and Elasticsearch are running on same windows machine. I have written a simpl… --- ## [How to map timestamp with nano seconds?](https://discuss.elastic.co/t/how-to-map-timestamp-with-nano-seconds/300881)
**Author:** [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Replies:** 4\ **Last updated:** [March 30, 2022, 5:06am UTC](https://discuss.elastic.co/t/how-to-map-timestamp-with-nano-seconds/300881 "2022-03-30T05:06:42Z")
Hie , I have a time where it is of the format : 2022- 03-28T16:51:11.637003013Z I tried to map like this : date{ match =\> \["Pretime","yyyy-MM-dd'T'HH:mm:ss.SSSSSSSSSZ"\] target =\> "@timestamp" remove\_field =\> "Preti… --- ## [Logtash Giving : : Expected one of #, input, filter, output at line 1, column 1 (byte 1) Error in syntax checking](https://discuss.elastic.co/t/logtash-giving-expected-one-of-input-filter-output-at-line-1-column-1-byte-1-error-in-syntax-checking/300987)
**Author:** [@d6036de2b54af16665f4](https://discuss.elastic.co/u/d6036de2b54af16665f4)\ **Replies:** 14\ **Last updated:** [March 29, 2022, 7:54pm UTC](https://discuss.elastic.co/t/logtash-giving-expected-one-of-input-filter-output-at-line-1-column-1-byte-1-error-in-syntax-checking/300987 "2022-03-29T19:54:03Z")
\[WARN \] 2022-03-29 15:12:27.166 \[LogStash::Runner\] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified \[FATAL\] 2022-03-29 15:12:27.560 \[LogStash::Runner\] runner - The giv… --- ## [Trying to convert JSON to new JSON outputs quoted JSON](https://discuss.elastic.co/t/trying-to-convert-json-to-new-json-outputs-quoted-json/300998)
**Author:** [@geena.rollins](https://discuss.elastic.co/u/geena.rollins)\ **Replies:** 2\ **Last updated:** [March 29, 2022, 6:12pm UTC](https://discuss.elastic.co/t/trying-to-convert-json-to-new-json-outputs-quoted-json/300998 "2022-03-29T18:12:20Z")
On my macbook monterey 12.2.1, I get quoted JSON. I'm hoping for plain JSON. logstash.yml is node.name: tester file.conf is input { file { path =\> \["/Users/geena.rollins/ws/hello.json"\] start\_position =\> … --- ## [Default install of logstash 8.1 results in error](https://discuss.elastic.co/t/default-install-of-logstash-8-1-results-in-error/300854)
**Author:** [@Louis\_Bohm](https://discuss.elastic.co/u/Louis_Bohm)\ **Replies:** 5\ **Last updated:** [March 29, 2022, 4:50pm UTC](https://discuss.elastic.co/t/default-install-of-logstash-8-1-results-in-error/300854 "2022-03-29T16:50:12Z")
I am running Centos 7.9 and followed the directions on this web to install logstash. After doing a yum -y install logstash to get logstash 8.1 installed did the following: Removed the logstash-sample.conf. Ran the fol… --- ## [Collect multiple events for processing in the filter and send to output (bulk)](https://discuss.elastic.co/t/collect-multiple-events-for-processing-in-the-filter-and-send-to-output-bulk/300986)
**Author:** [@Kosodrom](https://discuss.elastic.co/u/Kosodrom)\ **Replies:** 1\ **Last updated:** [March 29, 2022, 4:25pm UTC](https://discuss.elastic.co/t/collect-multiple-events-for-processing-in-the-filter-and-send-to-output-bulk/300986 "2022-03-29T16:25:14Z")
Hi folks, I have following use case: I receive events from beat and would like to enrich them with additional information using a filter plugin in logstash. The filter plugin must send a request to an endpoint and retr… --- ## [When output from logstash to mangodb, can I specify which fields wanted in output?](https://discuss.elastic.co/t/when-output-from-logstash-to-mangodb-can-i-specify-which-fields-wanted-in-output/300817)
**Author:** [@maoxuguang](https://discuss.elastic.co/u/maoxuguang)\ **Replies:** 3\ **Last updated:** [March 29, 2022, 2:56pm UTC](https://discuss.elastic.co/t/when-output-from-logstash-to-mangodb-can-i-specify-which-fields-wanted-in-output/300817 "2022-03-29T14:56:25Z")
I config two destination in logstash output, one is Elasticsearch, the other is mongodb. I only want to out put several fields to mongodb, not all of them. How to config to achieve that? Thanks a lot. --- ## [Logstash: \[ERROR\]\[logstash.javapipeline \]\[main\] Pipeline error | Cannot get new connection from pool](https://discuss.elastic.co/t/logstash-error-logstash-javapipeline-main-pipeline-error-cannot-get-new-connection-from-pool/300696)
**Author:** [@Bavaria](https://discuss.elastic.co/u/Bavaria)\ **Replies:** 11\ **Last updated:** [March 29, 2022, 1:40pm UTC](https://discuss.elastic.co/t/logstash-error-logstash-javapipeline-main-pipeline-error-cannot-get-new-connection-from-pool/300696 "2022-03-29T13:40:02Z")
I am trying to start logstash pipeline. But after long search, I do not know what I am doing wrong. I am getting this output: C:\\Users\\Name\\ElasticStack\\logstash-8.0.1\>.\\bin\\logstash.bat -f erste-pipeline.conf "Using bu… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=149) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=151)