# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=198 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 199 --- ## [Timezone parameter has no impact in Date filter plugin](https://discuss.elastic.co/t/timezone-parameter-has-no-impact-in-date-filter-plugin/282935)
**Author:** [@6igwig](https://discuss.elastic.co/u/6igwig)\ **Replies:** 2\ **Last updated:** [August 31, 2021, 4:10pm UTC](https://discuss.elastic.co/t/timezone-parameter-has-no-impact-in-date-filter-plugin/282935 "2021-08-31T16:10:35Z")
TLDR: my timestamp value is already in UTC, I am trying to prevent elastic from converting it to UTC again. I am using the JDBC input in logstash to query a SQL server database. When I look at a specific record in SQL S… --- ## [Retrieve value & key from json to build data table kibana](https://discuss.elastic.co/t/retrieve-value-key-from-json-to-build-data-table-kibana/282940)
**Author:** [@Mira\_9](https://discuss.elastic.co/u/Mira_9)\ **Replies:** 0\ **Last updated:** [August 31, 2021, 3:37pm UTC](https://discuss.elastic.co/t/retrieve-value-key-from-json-to-build-data-table-kibana/282940 "2021-08-31T15:37:56Z")
hey, i'm extracting data from Json file and displaying it in Kibana. i want to create a data table displaying the details for each id and for each category. How can i extract properly the key and the value from the mes… --- ## [Parsing duration with 00:00:00 format](https://discuss.elastic.co/t/parsing-duration-with-0000-format/281284)
**Author:** [@Sketchy](https://discuss.elastic.co/u/Sketchy)\ **Replies:** 6\ **Last updated:** [August 31, 2021, 3:00pm UTC](https://discuss.elastic.co/t/parsing-duration-with-0000-format/281284 "2021-08-31T15:00:34Z")
I have a CSV that has multiple durations that are in seconds but one of the columns has the duration as 00:00:30 for 30 seconds for example. I cant work out how to get that into seconds/integer easily. I can think of a f… --- ## [Convert mysql point schema to geo\_point type through logstash please tell me how](https://discuss.elastic.co/t/convert-mysql-point-schema-to-geo-point-type-through-logstash-please-tell-me-how/282879)
**Author:** [@Gi\_seong\_Lee](https://discuss.elastic.co/u/Gi_seong_Lee)\ **Replies:** 1\ **Last updated:** [August 31, 2021, 2:57pm UTC](https://discuss.elastic.co/t/convert-mysql-point-schema-to-geo-point-type-through-logstash-please-tell-me-how/282879 "2021-08-31T14:57:26Z")
In various internets, there are many data that convert longitude and latitude variables to geo\_point by separately specifying them as double type. But I want to convert one Mysql's point type variable to geo\_point. How c… --- ## [Tried to load a plugin's code, but failed. path=\>"logstash/filters/\"](https://discuss.elastic.co/t/tried-to-load-a-plugins-code-but-failed-path-logstash-filters-filter-name/282923)
**Author:** [@raiz](https://discuss.elastic.co/u/raiz)\ **Replies:** 0\ **Last updated:** [August 31, 2021, 12:57pm UTC](https://discuss.elastic.co/t/tried-to-load-a-plugins-code-but-failed-path-logstash-filters-filter-name/282923 "2021-08-31T12:57:51Z")
Hello, I'm trying to use my filter plugin in a conf file. It's being properly installed via an offline plugin pack, but errors when running logstash: ERROR registry:123 - Tried to load a plugin's code, but failed. {:ex… --- ## [Overwriting Record in Logstash Using document\_id?](https://discuss.elastic.co/t/overwriting-record-in-logstash-using-document-id/282908)
**Author:** [@padamrai](https://discuss.elastic.co/u/padamrai)\ **Replies:** 0\ **Last updated:** [August 31, 2021, 11:18am UTC](https://discuss.elastic.co/t/overwriting-record-in-logstash-using-document-id/282908 "2021-08-31T11:18:20Z")
I have data in which every record comes a three-time with attached unique referencedID. I am using document\_id=\>"referencedID". Currently, It is overwriting the record comes seconds time but It is not overwriting the rec… --- ## [Update previosuly indexed data](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765)
**Author:** [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Replies:** 8\ **Last updated:** [August 31, 2021, 9:46am UTC](https://discuss.elastic.co/t/update-previosuly-indexed-data/282765 "2021-08-31T09:46:47Z")
Hello ! Is there any way to detect with logstash jdbc if some previously indexed field value has been modified and if so, replace the old value with the new modified one in the next scheduled execution? Thanks in advan… --- ## [Logstash not creating index in secure(https) kibana](https://discuss.elastic.co/t/logstash-not-creating-index-in-secure-https-kibana/282444)
**Author:** [@kiran80511](https://discuss.elastic.co/u/kiran80511)\ **Replies:** 9\ **Last updated:** [August 31, 2021, 4:57am UTC](https://discuss.elastic.co/t/logstash-not-creating-index-in-secure-https-kibana/282444 "2021-08-31T04:57:45Z")
This is my logstash log using journalctl -b 0 -r Aug 25 11:33:18 elastic.sys systemd\[1\]: Started logstash. Aug 25 11:33:18 elastic.sys systemd\[1\]: Stopped logstash. Aug 25 11:33:18 elastic.sys systemd\[1\]: logstash.serv… --- ## [Logstash using vulnerable JDK](https://discuss.elastic.co/t/logstash-using-vulnerable-jdk/282623)
**Author:** [@Aditya931](https://discuss.elastic.co/u/Aditya931)\ **Replies:** 6\ **Last updated:** [August 31, 2021, 4:50am UTC](https://discuss.elastic.co/t/logstash-using-vulnerable-jdk/282623 "2021-08-31T04:50:14Z")
Hi, The latest logstash (7.14.0) is using a vulnerable jdk, and we've got number of security issues because of that. Below is the version of logstash and the jdk used by it. Any update on when will this be resolved? an… --- ## [Question regarding file and folder permissions when it comes to File output plugin](https://discuss.elastic.co/t/question-regarding-file-and-folder-permissions-when-it-comes-to-file-output-plugin/282873)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 0\ **Last updated:** [August 31, 2021, 4:19am UTC](https://discuss.elastic.co/t/question-regarding-file-and-folder-permissions-when-it-comes-to-file-output-plugin/282873 "2021-08-31T04:19:42Z")
Problem In my Output plugin I want to write all grok parse failures to a file else if "\_grokparsefailure" in \[tags\] { file { path =\> "/home/anotherAccount/testfolder/grokfailures" … --- ## [In logstash, I want to save logs to different outputs for each k8s namespace](https://discuss.elastic.co/t/in-logstash-i-want-to-save-logs-to-different-outputs-for-each-k8s-namespace/282867)
**Author:** [@Suyong](https://discuss.elastic.co/u/Suyong)\ **Replies:** 3\ **Last updated:** [August 31, 2021, 2:57am UTC](https://discuss.elastic.co/t/in-logstash-i-want-to-save-logs-to-different-outputs-for-each-k8s-namespace/282867 "2021-08-31T02:57:08Z")
Hello everyone!! I'm recently struggling with filebeat and logstash log handling in a kubernetes env. But my problem is that when the log below is left as logstash, I want to save it in a separate ES for each namespace… --- ## [Logstash csv date deactivate](https://discuss.elastic.co/t/logstash-csv-date-deactivate/282764)
**Author:** [@shariq](https://discuss.elastic.co/u/shariq)\ **Replies:** 6\ **Last updated:** [August 30, 2021, 1:19pm UTC](https://discuss.elastic.co/t/logstash-csv-date-deactivate/282764 "2021-08-30T13:19:07Z")
Hi, I am new to the ELK stack. I am trying to import a CSV that contains Person Data to Elastic search, everything is working fine the only problem is that I would like to change the Birth\_date to "text" so i can use… --- ## [Problems to start Logstash conf pipelines with SystemCTL](https://discuss.elastic.co/t/problems-to-start-logstash-conf-pipelines-with-systemctl/282811)
**Author:** [@Leandro\_Valim](https://discuss.elastic.co/u/Leandro_Valim)\ **Replies:** 2\ **Last updated:** [August 30, 2021, 1:19pm UTC](https://discuss.elastic.co/t/problems-to-start-logstash-conf-pipelines-with-systemctl/282811 "2021-08-30T13:19:03Z")
I have a problem with logstash, when I running like this:/usr/share/logstash/bin/logstash --debug -f /etc/logstash/conf.d/switch.conf Works fine and start this specific configuration file. But I put this conf file in p… --- ## [JSON Split](https://discuss.elastic.co/t/json-split/282642)
**Author:** [@shashikant.deshmukh](https://discuss.elastic.co/u/shashikant.deshmukh)\ **Replies:** 4\ **Last updated:** [August 30, 2021, 12:03pm UTC](https://discuss.elastic.co/t/json-split/282642 "2021-08-30T12:03:46Z")
Hi, below is my Json, can you please help to split, I want all details from "Value" as separate event in elasticsearch. { "@odata.context": "https://urlrequired/api/$metadata#Robots", "@odata.count": 116, … --- ## [Update deleted RDBMS entries in Elasticsearch using Logstash jdbc input plugin](https://discuss.elastic.co/t/update-deleted-rdbms-entries-in-elasticsearch-using-logstash-jdbc-input-plugin/282803)
**Author:** [@RRSR](https://discuss.elastic.co/u/RRSR)\ **Replies:** 0\ **Last updated:** [August 30, 2021, 11:18am UTC](https://discuss.elastic.co/t/update-deleted-rdbms-entries-in-elasticsearch-using-logstash-jdbc-input-plugin/282803 "2021-08-30T11:18:44Z")
I am trying to sync the RDBMS (postgres) data to Elasticsearch using Logstash and it works fine. Logstash configuration file: input { jdbc { jdbc\_driver\_library =\> "/home/raj/Downloads/postgresql-42.2.23.jar" … --- ## [Filebeat error- Failed to publish events caused by: lumberjack protocol error](https://discuss.elastic.co/t/filebeat-error-failed-to-publish-events-caused-by-lumberjack-protocol-error/282791)
**Author:** [@Ankit-github-26](https://discuss.elastic.co/u/Ankit-github-26)\ **Replies:** 0\ **Last updated:** [August 30, 2021, 9:26am UTC](https://discuss.elastic.co/t/filebeat-error-failed-to-publish-events-caused-by-lumberjack-protocol-error/282791 "2021-08-30T09:26:08Z")
Filebeat gives following error- 2021-08-30T08:59:03.490Z INFO \[publisher\_pipeline\_output\] pipeline/output.go:151 Connection to backoff(async(tcp://localhost:5044)) established 2021-08-30T08:59:03.565Z … --- ## [Using \_dissectfailure for matching multiple lines with dissect](https://discuss.elastic.co/t/using-dissectfailure-for-matching-multiple-lines-with-dissect/282781)
**Author:** [@nimblealliance](https://discuss.elastic.co/u/nimblealliance)\ **Replies:** 0\ **Last updated:** [August 30, 2021, 7:20am UTC](https://discuss.elastic.co/t/using-dissectfailure-for-matching-multiple-lines-with-dissect/282781 "2021-08-30T07:20:09Z")
Hello :slight\_smile: Could someone please let me know why my below filter using dissect doesn't work? I tried using the "\_dissectfailure" option to parse logs when the first dissect filter fails and returns this tag but… --- ## [How changing root of json events with logstash filter?](https://discuss.elastic.co/t/how-changing-root-of-json-events-with-logstash-filter/282755)
**Author:** [@e-fo](https://discuss.elastic.co/u/e-fo)\ **Replies:** 2\ **Last updated:** [August 30, 2021, 5:55am UTC](https://discuss.elastic.co/t/how-changing-root-of-json-events-with-logstash-filter/282755 "2021-08-30T05:55:23Z")
Hello, I have a problem with json events that importing into elasticsearch from logstash. events come from mongodb (with jdbc input plugin) and then imported into elasticsearch. after importing events into logstash th… --- ## [Apply data streams, ILM, Index template through logstash](https://discuss.elastic.co/t/apply-data-streams-ilm-index-template-through-logstash/282666)
**Author:** [@prathamesh7](https://discuss.elastic.co/u/prathamesh7)\ **Replies:** 9\ **Last updated:** [August 30, 2021, 5:52am UTC](https://discuss.elastic.co/t/apply-data-streams-ilm-index-template-through-logstash/282666 "2021-08-30T05:52:51Z")
Hi Team, I am deploying elasticsearch cluster with latest version i.e 7.14 through automation tool. The current config (v 7.4) is creating daily indices and it is not having data\_stream, ILM policy, index\_template etc.. … --- ## [Kafka Output - idempotent producer?](https://discuss.elastic.co/t/kafka-output-idempotent-producer/282754)
**Author:** [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\ **Replies:** 1\ **Last updated:** [August 29, 2021, 2:43pm UTC](https://discuss.elastic.co/t/kafka-output-idempotent-producer/282754 "2021-08-29T14:43:00Z")
I can see from Logstash logs that enable.idempotence=false by default. How do I set the value to true? --- ## [Logstash Plugins](https://discuss.elastic.co/t/logstash-plugins/282713)
**Author:** [@Markelastic](https://discuss.elastic.co/u/Markelastic)\ **Replies:** 4\ **Last updated:** [August 28, 2021, 10:53pm UTC](https://discuss.elastic.co/t/logstash-plugins/282713 "2021-08-28T22:53:52Z")
I am trying to parse a single line of unstructured data and I believe I am over complicating the matter. I am a log time SIEM engineer and content developer. That said, I am both in awe and just a tad overwhelmed in ligh… --- ## [HTTP\_Poller and Elasticsearch](https://discuss.elastic.co/t/http-poller-and-elasticsearch/282704)
**Author:** [@tofubeats](https://discuss.elastic.co/u/tofubeats)\ **Replies:** 0\ **Last updated:** [August 27, 2021, 8:03pm UTC](https://discuss.elastic.co/t/http-poller-and-elasticsearch/282704 "2021-08-27T20:03:40Z")
Hi, I am trying to query elasticsearch API with http\_poller input but keep getting the beloved PKIX building path failed error. I've set the elasticsearch.yml on the node being queried to contain the following: xpack.s… --- ## [Logstash Grok pattern match issue](https://discuss.elastic.co/t/logstash-grok-pattern-match-issue/282584)
**Author:** [@ericilavia](https://discuss.elastic.co/u/ericilavia)\ **Replies:** 3\ **Last updated:** [August 27, 2021, 3:44pm UTC](https://discuss.elastic.co/t/logstash-grok-pattern-match-issue/282584 "2021-08-27T15:44:28Z")
Hello, I am facing an issue with parsing data using Logstash using Grok filter, below is the use-case There are two types of logs I am trying to parse, //Grok patterns Type1: 04-16 07:16:35,359\[ INFO\]:Except:Process… --- ## [Logstash output http add param](https://discuss.elastic.co/t/logstash-output-http-add-param/282660)
**Author:** [@zhyp](https://discuss.elastic.co/u/zhyp)\ **Replies:** 0\ **Last updated:** [August 27, 2021, 10:36am UTC](https://discuss.elastic.co/t/logstash-output-http-add-param/282660 "2021-08-27T10:36:36Z")
Hi The following is my configuration. I want to add parameters, but there will be errors. output { stdout { codec =\> json } if(\[@metadata\]\[\_index\] == "need\_before\_load\_forecast") { http { url =\> "http://localhost… --- ## [Logstash Monitoring](https://discuss.elastic.co/t/logstash-monitoring/282641)
**Author:** [@bpandit](https://discuss.elastic.co/u/bpandit)\ **Replies:** 0\ **Last updated:** [August 27, 2021, 8:32am UTC](https://discuss.elastic.co/t/logstash-monitoring/282641 "2021-08-27T08:32:40Z")
There are no index formed for while trying to monitor logstash using legacy settings, please help, below are the logs of logstash. \[2021-08-27T15:04:57,444\]\[INFO \]\[logstash.runner \] Log4j configuration path use… --- ## [Daisy chained logstash not recieving using TCP](https://discuss.elastic.co/t/daisy-chained-logstash-not-recieving-using-tcp/282599)
**Author:** [@Spencer\_Hazell](https://discuss.elastic.co/u/Spencer_Hazell)\ **Replies:** 3\ **Last updated:** [August 27, 2021, 7:29am UTC](https://discuss.elastic.co/t/daisy-chained-logstash-not-recieving-using-tcp/282599 "2021-08-27T07:29:53Z")
Hi there I have 3 logstash servers, daisy chained. First one has 2 outputs, 1 for ES and the other for 2nd Logstash. A tcpdump shows the 2nd Logstash is receiving data. The 2nd Logstash connects to the 3rd Logstash o… --- ## [Error in using EXEC filter for curl](https://discuss.elastic.co/t/error-in-using-exec-filter-for-curl/282459)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 5\ **Last updated:** [August 27, 2021, 7:27am UTC](https://discuss.elastic.co/t/error-in-using-exec-filter-for-curl/282459 "2021-08-27T07:27:35Z")
Hi All, We have a scenario where through curl we are getting some output, when tried the curl command directly we are getting the output. curl --location --request POST 'https://demo.cloud.ie/api/v2/analytics/conversat… --- ## [Logstash keystore value not detected in configuration processing](https://discuss.elastic.co/t/logstash-keystore-value-not-detected-in-configuration-processing/281904)
**Author:** [@ramilbermejo](https://discuss.elastic.co/u/ramilbermejo)\ **Replies:** 5\ **Last updated:** [August 27, 2021, 3:47am UTC](https://discuss.elastic.co/t/logstash-keystore-value-not-detected-in-configuration-processing/281904 "2021-08-27T03:47:29Z")
Hello Everyone, I'm trying keystore to store the password for elasticsearch but for some reason I've encountered this error during configuration test. # /usr/share/logstash/bin/logstash --path.settings /etc/logstash -t… --- ## [Logstash filter for a log record from filebeats](https://discuss.elastic.co/t/logstash-filter-for-a-log-record-from-filebeats/282228)
**Author:** [@Rabin\_Bhattacharya](https://discuss.elastic.co/u/Rabin_Bhattacharya)\ **Replies:** 5\ **Last updated:** [August 26, 2021, 3:35pm UTC](https://discuss.elastic.co/t/logstash-filter-for-a-log-record-from-filebeats/282228 "2021-08-26T15:35:57Z")
Hi, I am using ELK stack+filebeats in our project and the application log is integration with filebeats. Filebeat is reading the application log. The log is successfully read by logstash. My question is how to parse the… --- ## [Logstash filter check value without fields](https://discuss.elastic.co/t/logstash-filter-check-value-without-fields/282598)
**Author:** [@dfoot](https://discuss.elastic.co/u/dfoot)\ **Replies:** 0\ **Last updated:** [August 26, 2021, 3:27pm UTC](https://discuss.elastic.co/t/logstash-filter-check-value-without-fields/282598 "2021-08-26T15:27:24Z")
Below document there are 3 values without fields I need add logstash configuration to pick up 3 values, can you help please? maybe start with loglevel as DEBUG and error N13 to confirm before pick up 3 values below. 70-… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=197) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=199)