# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=359 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 360 --- ## [How to get detailed :error message from mutate filter?](https://discuss.elastic.co/t/how-to-get-detailed-error-message-from-mutate-filter/232333)
**Author:** [@Badger](https://discuss.elastic.co/u/Badger)\ **Replies:** 0\ **Last updated:** [May 12, 2020, 11:22pm UTC](https://discuss.elastic.co/t/how-to-get-detailed-error-message-from-mutate-filter/232333 "2020-05-12T23:22:07Z")
Detailed error messages from mutate are not logged anywhere. When the mutate filter raises a logstash.agent.configuration.invalid\_plugin\_register exception it includes in the raise a detailed exception such as "Invalid … --- ## [Unable to see data into Apache's Default Dashboard with filebeat which is sending data to logstash](https://discuss.elastic.co/t/unable-to-see-data-into-apaches-default-dashboard-with-filebeat-which-is-sending-data-to-logstash/232187)
**Author:** [@sachin\_koshti](https://discuss.elastic.co/u/sachin_koshti)\ **Replies:** 3\ **Last updated:** [May 12, 2020, 10:09pm UTC](https://discuss.elastic.co/t/unable-to-see-data-into-apaches-default-dashboard-with-filebeat-which-is-sending-data-to-logstash/232187 "2020-05-12T22:09:14Z")
Hello All, I have setup ELK stack on Windows Server 2019 and used filebeat to read Apache's log files and it's output is configured to send all data to Logstash and able to see data but when i try to see all those data … --- ## [Can Logstash Interact with a C Library?](https://discuss.elastic.co/t/can-logstash-interact-with-a-c-library/232059)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 2\ **Last updated:** [May 12, 2020, 9:15pm UTC](https://discuss.elastic.co/t/can-logstash-interact-with-a-c-library/232059 "2020-05-12T21:15:29Z")
Hi Logstash Ninja Masters, I’m using an instance of Logstash (Docker container version 7.4.0, yes, I know I need to upgrade) that is currently processing our network data. Here’s a sample data record: “10.10.10.10”, “… --- ## [Symbolic Links in logstash for Kubernetes - EEXISTS](https://discuss.elastic.co/t/symbolic-links-in-logstash-for-kubernetes-eexists/232320)
**Author:** [@jrcichra](https://discuss.elastic.co/u/jrcichra)\ **Replies:** 0\ **Last updated:** [May 12, 2020, 8:35pm UTC](https://discuss.elastic.co/t/symbolic-links-in-logstash-for-kubernetes-eexists/232320 "2020-05-12T20:35:53Z")
Hi, I'm trying to write Kubernetes pod logs out through a symlink. When trying to do so I get this error: WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules WARNING: Use … --- ## [I am getting (NoMethodError) undefined method \`to\_time' for 2016-09-09T17:57:40.873Z:LogStash::Timestamp](https://discuss.elastic.co/t/i-am-getting-nomethoderror-undefined-method-to-time-for-2016-09-09t1740-873z-timestamp/232311)
**Author:** [@Navaneetha\_KrishnanR](https://discuss.elastic.co/u/Navaneetha_KrishnanR)\ **Replies:** 0\ **Last updated:** [May 12, 2020, 7:50pm UTC](https://discuss.elastic.co/t/i-am-getting-nomethoderror-undefined-method-to-time-for-2016-09-09t1740-873z-timestamp/232311 "2020-05-12T19:50:17Z")
When i trying to get the Date field from the MSSQL via logstash I am getting the following error : Caused by: org.jruby.exceptions.NoMethodError: (NoMethodError) undefined method \`to\_time' for 2016-09-09T17:57:40.873Z:L… --- ## [Logstash s3 output doesn't support path-style url for s3 compatible storage](https://discuss.elastic.co/t/logstash-s3-output-doesnt-support-path-style-url-for-s3-compatible-storage/232113)
**Author:** [@ibmdcb](https://discuss.elastic.co/u/ibmdcb)\ **Replies:** 2\ **Last updated:** [May 12, 2020, 6:38pm UTC](https://discuss.elastic.co/t/logstash-s3-output-doesnt-support-path-style-url-for-s3-compatible-storage/232113 "2020-05-12T18:38:58Z")
logstash s3 output doesn't support path-style url for s3 compatible storage. It tries to connect to "\< bucket\>.\< endpoint\>"... instead of "\< endpoint\>/\< bucket\>". Our internal s3 implementation currently only supports … --- ## [Logstash and CGroups](https://discuss.elastic.co/t/logstash-and-cgroups/232269)
**Author:** [@mcstephens](https://discuss.elastic.co/u/mcstephens)\ **Replies:** 0\ **Last updated:** [May 12, 2020, 4:13pm UTC](https://discuss.elastic.co/t/logstash-and-cgroups/232269 "2020-05-12T16:13:59Z")
We're in the process of migrating platforms and we're looking for architectural guidance. We've been told by Elastic that Elasticsearch nodes should be using CGroups (56 core machines, multiple nodes per machine), but I'… --- ## [JSON Filter fails on valid key names](https://discuss.elastic.co/t/json-filter-fails-on-valid-key-names/232244)
**Author:** [@davehatma](https://discuss.elastic.co/u/davehatma)\ **Replies:** 1\ **Last updated:** [May 12, 2020, 2:58pm UTC](https://discuss.elastic.co/t/json-filter-fails-on-valid-key-names/232244 "2020-05-12T14:58:47Z")
I have some Kafka messages that have key names that contain reserved Logstash characters -- specifically '\[" and "\]". Additionally the json document also has arrays. My Logstash pipeline is generating: \[WARN \]\[logstas… --- ## [Trouble with aggregate filter?](https://discuss.elastic.co/t/trouble-with-aggregate-filter/230679)
**Author:** [@Emna1](https://discuss.elastic.co/u/Emna1)\ **Replies:** 16\ **Last updated:** [May 2, 2020, 8:31pm UTC](https://discuss.elastic.co/t/trouble-with-aggregate-filter/230679 "2020-05-02T20:31:57Z")
Hi,I have a csv file contain 1000 rows ,it looks like this Blockquote ent\_sort , Date , vehicle enter, 2020-04-28 05:04:02 , c1 sortie , 2020-04-28 10:04:02 , c1 enter , 2020-04-28 05:04:02 , c3 sortie , 2020-04-2… --- ## [How to Create a weekly index from filebeat and daily for Metricbeat](https://discuss.elastic.co/t/how-to-create-a-weekly-index-from-filebeat-and-daily-for-metricbeat/232161)
**Author:** [@Mai\_Waly](https://discuss.elastic.co/u/Mai_Waly)\ **Replies:** 1\ **Last updated:** [May 12, 2020, 10:50am UTC](https://discuss.elastic.co/t/how-to-create-a-weekly-index-from-filebeat-and-daily-for-metricbeat/232161 "2020-05-12T10:50:40Z")
Hi All, Could you please help in my logstash configuration Currently we use daily indexes and cluster is block as we soon hit max number of shards How to change logstash configuration to have a weekly filebeat index a… --- ## [Logstash-input-beats - File does not contain valid private key](https://discuss.elastic.co/t/logstash-input-beats-file-does-not-contain-valid-private-key/228140)
**Author:** [@quentin.legraverend](https://discuss.elastic.co/u/quentin.legraverend)\ **Replies:** 17\ **Last updated:** [May 12, 2020, 9:26am UTC](https://discuss.elastic.co/t/logstash-input-beats-file-does-not-contain-valid-private-key/228140 "2020-05-12T09:26:29Z")
Hi there, I am currently doing a stack upgrade from 7.0.1 to 7.6.2. Everything went fine on the Elasticsearch cluster and Kibana instances but I have issues upgrading Logstash. To be precise, Logstash upgrade was fine,… --- ## [Elastic Output plugin fetch status of connectivity](https://discuss.elastic.co/t/elastic-output-plugin-fetch-status-of-connectivity/230022)
**Author:** [@Sunil\_Vp](https://discuss.elastic.co/u/Sunil_Vp)\ **Replies:** 1\ **Last updated:** [May 12, 2020, 8:45am UTC](https://discuss.elastic.co/t/elastic-output-plugin-fetch-status-of-connectivity/230022 "2020-05-12T08:45:34Z")
Hi, I was looking at way to fetch the status of elastic connectivity from logstash output plugin. Is there an API or any other mechanism to fetch the same? I am basically interested in outputting the status periodicall… --- ## [How to remove spaces from log using gsub](https://discuss.elastic.co/t/how-to-remove-spaces-from-log-using-gsub/232171)
**Author:** [@tahseen\_fatima](https://discuss.elastic.co/u/tahseen_fatima)\ **Replies:** 0\ **Last updated:** [May 12, 2020, 8:42am UTC](https://discuss.elastic.co/t/how-to-remove-spaces-from-log-using-gsub/232171 "2020-05-12T08:42:24Z")
Hi, I want to remove spaces from my logs using gsub. How can I do that. Below is my multiline logs. for some of the line I am using grok, and for some of the logs I'll use KV to parse the data as it is in kv format in … --- ## [Date Filter Logstash](https://discuss.elastic.co/t/date-filter-logstash/232076)
**Author:** [@alberto1](https://discuss.elastic.co/u/alberto1)\ **Replies:** 4\ **Last updated:** [May 12, 2020, 8:07am UTC](https://discuss.elastic.co/t/date-filter-logstash/232076 "2020-05-12T08:07:26Z")
Hi everyone! I am having a little issue trying to get right the date filter in logstash. What I am trying to archive here is to change the @timestamp for the actual log time. My logs looks like this... 2020-02-06 00:… --- ## [Proper way to create new event from Logstash Ruby filter](https://discuss.elastic.co/t/proper-way-to-create-new-event-from-logstash-ruby-filter/232119)
**Author:** [@szgeri](https://discuss.elastic.co/u/szgeri)\ **Replies:** 3\ **Last updated:** [May 12, 2020, 8:10am UTC](https://discuss.elastic.co/t/proper-way-to-create-new-event-from-logstash-ruby-filter/232119 "2020-05-12T08:10:55Z")
Hi, I would like to somehow create new event with Logstash Ruby filter but actually with no success. I tried a lot of versions and syntaxes, searched for many online topics and documentations and the result is the same. … --- ## [Logstash calling a curl through exec quotations not allowed](https://discuss.elastic.co/t/logstash-calling-a-curl-through-exec-quotations-not-allowed/232050)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 4\ **Last updated:** [May 12, 2020, 8:00am UTC](https://discuss.elastic.co/t/logstash-calling-a-curl-through-exec-quotations-not-allowed/232050 "2020-05-12T08:00:37Z")
Im trying to send my logstash output to an API url with the exec output plugin. \> output { \> exec { \> command =\> "curl -u username:'password' -k https://url.com/zdmd/Events/evconsole\_router -d '{"action":"EventsRouter",… --- ## [Logstah http filter merge and add and array](https://discuss.elastic.co/t/logstah-http-filter-merge-and-add-and-array/232159)
**Author:** [@katara](https://discuss.elastic.co/u/katara)\ **Replies:** 0\ **Last updated:** [May 12, 2020, 7:58am UTC](https://discuss.elastic.co/t/logstah-http-filter-merge-and-add-and-array/232159 "2020-05-12T07:58:24Z")
Hi, I am sending my data to a http url which I want to be in the below format: { "action":"EventsRouter", "method":"add\_event", "data": \[ { "summary": Error 123", "device":"device01", "message":" Error 123 , Appli… --- ## [Configuration issue Logstash or Kibana](https://discuss.elastic.co/t/configuration-issue-logstash-or-kibana/232129)
**Author:** [@anmohamed](https://discuss.elastic.co/u/anmohamed)\ **Replies:** 1\ **Last updated:** [May 12, 2020, 5:22am UTC](https://discuss.elastic.co/t/configuration-issue-logstash-or-kibana/232129 "2020-05-12T05:22:37Z")
I have like this log format. but I am not sure i have to config filebeat or Logstash to get the after message to get the logs number to the same log. Can you help me to do this? I have given 2 sample logs. I, \[2020-05… --- ## [Get today's date that change automatically everyday](https://discuss.elastic.co/t/get-todays-date-that-change-automatically-everyday/231934)
**Author:** [@yummycake](https://discuss.elastic.co/u/yummycake)\ **Replies:** 8\ **Last updated:** [May 12, 2020, 3:02am UTC](https://discuss.elastic.co/t/get-todays-date-that-change-automatically-everyday/231934 "2020-05-12T03:02:15Z")
I am trying to add a new field, "Today's Date" to get the Today's/Current date from logstash into elasticsearch. Here is my configuration in logstash input { file { path =\> "C:/Users/Admin/Documents/dummy\_test\_da… --- ## [About import Windows File Resource management CSV file](https://discuss.elastic.co/t/about-import-windows-file-resource-management-csv-file/232128)
**Author:** [@Ryan4](https://discuss.elastic.co/u/Ryan4)\ **Replies:** 0\ **Last updated:** [May 12, 2020, 2:13am UTC](https://discuss.elastic.co/t/about-import-windows-file-resource-management-csv-file/232128 "2020-05-12T02:13:56Z")
Hello All, I need import Windows File Resource management CSV file. The CSV file like following sample. "Large Files Report" Generated at: "4/24/2020 10:03:51 AM" "Lists files that are a specified size or larger. Us… --- ## [Import csv file failed](https://discuss.elastic.co/t/import-csv-file-failed/232026)
**Author:** [@Florent\_H](https://discuss.elastic.co/u/Florent_H)\ **Replies:** 1\ **Last updated:** [May 12, 2020, 12:19am UTC](https://discuss.elastic.co/t/import-csv-file-failed/232026 "2020-05-12T00:19:30Z")
Hello, I try to insert CSV file in elastic and visualyse it to kibana but it doesn't work. I start elastic and kibana. The conf file is: input { file { path =\> "C:/Users/florenth/bureau/analyseFH/donnee\_qivivo.… --- ## [Http filter plugin authorization as parameter values](https://discuss.elastic.co/t/http-filter-plugin-authorization-as-parameter-values/232085)
**Author:** [@alxsss](https://discuss.elastic.co/u/alxsss)\ **Replies:** 6\ **Last updated:** [May 12, 2020, 12:00am UTC](https://discuss.elastic.co/t/http-filter-plugin-authorization-as-parameter-values/232085 "2020-05-12T00:00:42Z")
Hello, I try to pass user and password to http filter plugin as parameter values like. user =\> "%{user\_value}" password =\> "%{password\_value}" both values come from an input plugin as tag values. However, I get http… --- ## [Problem to replace field string "|" with mutate gsub filter](https://discuss.elastic.co/t/problem-to-replace-field-string-with-mutate-gsub-filter/232116)
**Author:** [@esportuga](https://discuss.elastic.co/u/esportuga)\ **Replies:** 1\ **Last updated:** [May 11, 2020, 10:12pm UTC](https://discuss.elastic.co/t/problem-to-replace-field-string-with-mutate-gsub-filter/232116 "2020-05-11T22:12:40Z")
Hello, My problem has two solutions that I can use. At the moment I am not achieving success in any. My log 2020-04-30T13: 48: 17.954 Int 50141 009401EA-1003849D 3553384768 custom\_var navigation|4300; 4201; 7001; 8001… --- ## [Mis-match in compatibility matrices](https://discuss.elastic.co/t/mis-match-in-compatibility-matrices/232092)
**Author:** [@spoole](https://discuss.elastic.co/u/spoole)\ **Replies:** 0\ **Last updated:** [May 11, 2020, 6:06pm UTC](https://discuss.elastic.co/t/mis-match-in-compatibility-matrices/232092 "2020-05-11T18:06:59Z")
If you look at the compatibility matrix here https://www.elastic.co/support/matrix#matrix\_compatibility you see that Logstash 7.X is compatible with Beats 6.8.X -\> 7.6.X but if you scroll down farther you see Beats 6.8.X… --- ## [Issue with setting geo shape array](https://discuss.elastic.co/t/issue-with-setting-geo-shape-array/232081)
**Author:** [@adude946](https://discuss.elastic.co/u/adude946)\ **Replies:** 0\ **Last updated:** [May 11, 2020, 5:32pm UTC](https://discuss.elastic.co/t/issue-with-setting-geo-shape-array/232081 "2020-05-11T17:32:40Z")
I'm trying to set the envelope geo shape, but cant get the array of array right. It should be: \[\[a,b\],\[c,d\]\]. I know the syntax is wrong but not sure what to change. Tried both ways using ruby and mutate. Error "Failed… --- ## [Logstash starts only with sudo using sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/postgresql.conf](https://discuss.elastic.co/t/logstash-starts-only-with-sudo-using-sudo-usr-share-logstash-bin-logstash-f-etc-logstash-conf-d-postgresql-conf/231994)
**Author:** [@Amine\_Aloui](https://discuss.elastic.co/u/Amine_Aloui)\ **Replies:** 3\ **Last updated:** [May 11, 2020, 4:58pm UTC](https://discuss.elastic.co/t/logstash-starts-only-with-sudo-using-sudo-usr-share-logstash-bin-logstash-f-etc-logstash-conf-d-postgresql-conf/231994 "2020-05-11T16:58:57Z")
with sudo , it works without any issue sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/postgresql.conf but if i run /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/postgresql.conf OpenJDK 64-Bi… --- ## [Logstash Twitter input error](https://discuss.elastic.co/t/logstash-twitter-input-error/231575)
**Author:** [@Yann\_Allandit](https://discuss.elastic.co/u/Yann_Allandit)\ **Replies:** 1\ **Last updated:** [May 11, 2020, 4:47pm UTC](https://discuss.elastic.co/t/logstash-twitter-input-error/231575 "2020-05-11T16:47:02Z")
Hi, The following logstash config file fails with the error below (credentials were sanitized): input { twitter { consumer\_key =\> "99999999999999999999" consumer\_secret =\> "99999999999999999999" … --- ## [Logstash 7.4.2 Page file size is too small to hold elements](https://discuss.elastic.co/t/logstash-7-4-2-page-file-size-is-too-small-to-hold-elements/232062)
**Author:** [@dweljali](https://discuss.elastic.co/u/dweljali)\ **Replies:** 0\ **Last updated:** [May 11, 2020, 3:43pm UTC](https://discuss.elastic.co/t/logstash-7-4-2-page-file-size-is-too-small-to-hold-elements/232062 "2020-05-11T15:43:29Z")
We are using logstash 7.4.2, we faced queue corruption issue, below error. We tried pqcheck and pqrepair logstash utility however its not fixed. Any pointer would be really helpful. Thank you. Error :\[2020-05-08T11… --- ## ["LogStash::ConfigurationError", :message=\>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after ", :backtrace](https://discuss.elastic.co/t/logstash-configurationerror-message-expected-one-of-input-filter-output-at-line-1-column-1-byte-1-after-backtrace/231818)
**Author:** [@subramanyam.setty](https://discuss.elastic.co/u/subramanyam.setty)\ **Replies:** 2\ **Last updated:** [May 11, 2020, 3:52pm UTC](https://discuss.elastic.co/t/logstash-configurationerror-message-expected-one-of-input-filter-output-at-line-1-column-1-byte-1-after-backtrace/231818 "2020-05-11T15:52:23Z")
Hi team, I am getting below exception .I have mentioned .conf file and log file which is given input to the .conf.Please let suggest. Exception: {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exceptio… --- ## [Use logstash to transform a message](https://discuss.elastic.co/t/use-logstash-to-transform-a-message/231811)
**Author:** [@dooger21](https://discuss.elastic.co/u/dooger21)\ **Replies:** 2\ **Last updated:** [May 11, 2020, 3:50pm UTC](https://discuss.elastic.co/t/use-logstash-to-transform-a-message/231811 "2020-05-11T15:50:10Z")
Hello everyone, I have a log that generates thousands of events a day, a group of events and its entire plot can be differentiated and obtained by the KEY field generated by the application as in the following example. … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=358) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=360)