# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=136 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 137 --- ## [Logstash filter date match not replacing @timestamp](https://discuss.elastic.co/t/logstash-filter-date-match-not-replacing-timestamp/306364)
**Author:** [@yhache](https://discuss.elastic.co/u/yhache)\ **Replies:** 20\ **Last updated:** [June 7, 2022, 12:42pm UTC](https://discuss.elastic.co/t/logstash-filter-date-match-not-replacing-timestamp/306364 "2022-06-07T12:42:31Z")
Hi, I have a bunch of PHP log to add to my ELK cluster and i cannot get the @timestamp to work. I'm pretty sure it has something to do with the config file and the date format used in the PHP logs. Here's my config file… --- ## [Stop logstash from automatically creating an index on elastic](https://discuss.elastic.co/t/stop-logstash-from-automatically-creating-an-index-on-elastic/306555)
**Author:** [@Elie\_Sbat](https://discuss.elastic.co/u/Elie_Sbat)\ **Replies:** 1\ **Last updated:** [June 7, 2022, 11:19am UTC](https://discuss.elastic.co/t/stop-logstash-from-automatically-creating-an-index-on-elastic/306555 "2022-06-07T11:19:56Z")
Hello, Is there a way to stop logstash from automatically creating an index in Elasticsearch. even if the conf file does not contain any configuration regarding the index. Logstash logs: logstash | \[WARN \] 2022-06-… --- ## [Revome over gsub()](https://discuss.elastic.co/t/revome-over-gsub/306460)
**Author:** [@Iss](https://discuss.elastic.co/u/Iss)\ **Replies:** 5\ **Last updated:** [June 7, 2022, 10:44am UTC](https://discuss.elastic.co/t/revome-over-gsub/306460 "2022-06-07T10:44:21Z")
Hi! I have nested field as : "statement" =\> { "cntxt" =\> { "extension" =\> { "https://ssss/xp/w/yy" =\> "xxxxxxx" } }, ... } the field "extension" is dynamic… --- ## [Encountered a retryable error (will retry with exponential backoff) {:code=\>400, :url=\>"http://localhost:9200/\_bulk", :content\_length=\>187}](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff-code-400-url-http-localhost-9200-bulk-content-length-187/306547)
**Author:** [@akshay\_bhardwaj](https://discuss.elastic.co/u/akshay_bhardwaj)\ **Replies:** 0\ **Last updated:** [June 7, 2022, 9:24am UTC](https://discuss.elastic.co/t/encountered-a-retryable-error-will-retry-with-exponential-backoff-code-400-url-http-localhost-9200-bulk-content-length-187/306547 "2022-06-07T09:24:03Z")
Hi i want to config logstash with incremental pipeline but i am getting below error t\[2022-06-07T14:47:45,313\]\[ERROR\]\[logstash.outputs.Elasticsearch\]\[main\]\[d24e130edb19f1038da36ee73576edcc3bb8a8df6f0207c132792ec2475e92ec… --- ## [Problem to setup sub flieds](https://discuss.elastic.co/t/problem-to-setup-sub-flieds/306134)
**Author:** [@cRaZyT](https://discuss.elastic.co/u/cRaZyT)\ **Replies:** 5\ **Last updated:** [June 7, 2022, 8:39am UTC](https://discuss.elastic.co/t/problem-to-setup-sub-flieds/306134 "2022-06-07T08:39:17Z")
Hi, I have a problem to generate sub fileds from a json message . With the following filter I try to check if the field "details" exist and add the json fields to it. If it doesn't exist the field sould be created wit… --- ## [Salesforce object NOT\_FOUND](https://discuss.elastic.co/t/salesforce-object-not-found/305937)
**Author:** [@Simone1](https://discuss.elastic.co/u/Simone1)\ **Replies:** 2\ **Last updated:** [June 7, 2022, 6:42am UTC](https://discuss.elastic.co/t/salesforce-object-not-found/305937 "2022-06-07T06:42:47Z")
I want to send data from Salesforce to Elasticsearch with Logstash. I have a case.conf file that sends data of the Case object and it works fine. But i want to send other object (WorkOrder for example) but when i start… --- ## [Can you help me to push the json data to ELK?](https://discuss.elastic.co/t/can-you-help-me-to-push-the-json-data-to-elk/306525)
**Author:** [@dineshsekar](https://discuss.elastic.co/u/dineshsekar)\ **Replies:** 1\ **Last updated:** [June 7, 2022, 6:31am UTC](https://discuss.elastic.co/t/can-you-help-me-to-push-the-json-data-to-elk/306525 "2022-06-07T06:31:14Z")
Continuing the discussion from Push json key value with ELK: File name: data.json In below json, these are the content, can you help me to push the data to ELK ? { "\_source": { "model": "commit", "tool": "bitbucke… --- ## [2 Logstash for load balancing and avoid duplication](https://discuss.elastic.co/t/2-logstash-for-load-balancing-and-avoid-duplication/306501)
**Author:** [@Devraj\_patel](https://discuss.elastic.co/u/Devraj_patel)\ **Replies:** 1\ **Last updated:** [June 6, 2022, 8:56pm UTC](https://discuss.elastic.co/t/2-logstash-for-load-balancing-and-avoid-duplication/306501 "2022-06-06T20:56:20Z")
Till now I was using only 1 server with logstash for sending syslog data to Elasticsearch. I am switching to 2 servers with logstash for load balancing and using f5 VIP, is there a way logstash from different instances c… --- ## [Not able to fetch data from oracle database using jdbc plugin](https://discuss.elastic.co/t/not-able-to-fetch-data-from-oracle-database-using-jdbc-plugin/306442)
**Author:** [@Kishan3176](https://discuss.elastic.co/u/Kishan3176)\ **Replies:** 1\ **Last updated:** [June 6, 2022, 11:26am UTC](https://discuss.elastic.co/t/not-able-to-fetch-data-from-oracle-database-using-jdbc-plugin/306442 "2022-06-06T11:26:03Z")
Hi All, I am new to logstash. There was a requirement where client want to monitor few database servers using logstash and sentinel. I am able to connect to Sentinel but I am not able to fetch data from oracle databas… --- ## [Logstash update elastic search when any update in a table](https://discuss.elastic.co/t/logstash-update-elastic-search-when-any-update-in-a-table/306133)
**Author:** [@ABHISHEK\_KUMAR\_SINGH](https://discuss.elastic.co/u/ABHISHEK_KUMAR_SINGH)\ **Replies:** 15\ **Last updated:** [June 6, 2022, 9:23am UTC](https://discuss.elastic.co/t/logstash-update-elastic-search-when-any-update-in-a-table/306133 "2022-06-06T09:23:45Z")
Hello, I have a Users table in my DB and I have successfully synced my logstash to update new records. But I want a way to update the existing records on Elasticsearch. I don't have any timestamp column in the table to … --- ## [How to receive logs from multiple filebeats running on different machine(ec2) to one logstash instance?](https://discuss.elastic.co/t/how-to-receive-logs-from-multiple-filebeats-running-on-different-machine-ec2-to-one-logstash-instance/306314)
**Author:** [@Krishna\_Prasad](https://discuss.elastic.co/u/Krishna_Prasad)\ **Replies:** 4\ **Last updated:** [June 6, 2022, 8:55am UTC](https://discuss.elastic.co/t/how-to-receive-logs-from-multiple-filebeats-running-on-different-machine-ec2-to-one-logstash-instance/306314 "2022-06-06T08:55:24Z")
i have 2 filebeats running on different machine(Ex: A & B) and I have also specified the logstash machine IP in filebeat.yaml so i have written 2 logstash .conf file . My conf file looks like this -\> A.conf input{ bea… --- ## [Logstash fields from aggregation to csv](https://discuss.elastic.co/t/logstash-fields-from-aggregation-to-csv/306376)
**Author:** [@ddcro](https://discuss.elastic.co/u/ddcro)\ **Replies:** 2\ **Last updated:** [June 6, 2022, 7:02am UTC](https://discuss.elastic.co/t/logstash-fields-from-aggregation-to-csv/306376 "2022-06-06T07:02:40Z")
Hi guys, I'm trying to export data from elastic to csv. The conf from logstash is something like: input { elasticsearch { hosts =\> "127.0.0.1:9200" index =\> "log" ... query… --- ## [Json parse - logstash VB.net](https://discuss.elastic.co/t/json-parse-logstash-vb-net/306427)
**Author:** [@Filip\_Caha](https://discuss.elastic.co/u/Filip_Caha)\ **Replies:** 0\ **Last updated:** [June 6, 2022, 6:48am UTC](https://discuss.elastic.co/t/json-parse-logstash-vb-net/306427 "2022-06-06T06:48:07Z")
Hi, I have a problem with parsing JSON data. My source code: \> Dim xPoslatUmo As String = ConfigurationManager.AppSettings.Item("UMOPoslat") \> If xPoslatUmo = "Ano" Then \> Dim request … --- ## [Logstash s3 plugin to push json files to elasticsearch](https://discuss.elastic.co/t/logstash-s3-plugin-to-push-json-files-to-elasticsearch/306325)
**Author:** [@JinnaBalu](https://discuss.elastic.co/u/JinnaBalu)\ **Replies:** 2\ **Last updated:** [June 6, 2022, 5:25am UTC](https://discuss.elastic.co/t/logstash-s3-plugin-to-push-json-files-to-elasticsearch/306325 "2022-06-06T05:25:41Z")
Usecase Push s3 JSON objects (s3 containers json files), each object will be one document) I have written a pipeline input { s3 { access\_key\_id =\> "MY\_KEY" secret\_access\_key =\> "MY\_SECRET" … --- ## [Logstash s3 input plugin and path-style access configuration](https://discuss.elastic.co/t/logstash-s3-input-plugin-and-path-style-access-configuration/306412)
**Author:** [@betheway](https://discuss.elastic.co/u/betheway)\ **Replies:** 0\ **Last updated:** [June 5, 2022, 9:41pm UTC](https://discuss.elastic.co/t/logstash-s3-input-plugin-and-path-style-access-configuration/306412 "2022-06-05T21:41:47Z")
With s3 path-style access, the plugin returns an error on the downloading step. From access logs: Found key {:key=\>"myFile"} Added to objects\[\] {:key=\>"myFile", :length=\>1} Processing {:bucket=\>"", :key=\>"myFile"} Downl… --- ## [Logstash not indexing date to the filebeat write index for data stream](https://discuss.elastic.co/t/logstash-not-indexing-date-to-the-filebeat-write-index-for-data-stream/306360)
**Author:** [@Carlos\_T](https://discuss.elastic.co/u/Carlos_T)\ **Replies:** 4\ **Last updated:** [June 4, 2022, 10:21pm UTC](https://discuss.elastic.co/t/logstash-not-indexing-date-to-the-filebeat-write-index-for-data-stream/306360 "2022-06-04T22:21:00Z")
Hi everybody. I've got a three nodes ES cluster. I've got the Kibana, one logstash and one filebeat running along in one of those three ES nodes. 'ubuntuelk'. Everything on version 8.2.1. My goal is to use that filebe… --- ## [Read column of datatype "JSON" from database using Logstash](https://discuss.elastic.co/t/read-column-of-datatype-json-from-database-using-logstash/306369)
**Author:** [@bhavani\_nandakumar](https://discuss.elastic.co/u/bhavani_nandakumar)\ **Replies:** 0\ **Last updated:** [June 3, 2022, 10:31pm UTC](https://discuss.elastic.co/t/read-column-of-datatype-json-from-database-using-logstash/306369 "2022-06-03T22:31:24Z")
I have developed a script to read table from Postgres Database and load the data in a json format inside Kafka topic. The problem is one of the columns (cust\_record) in the table is JSON type and I could not read that co… --- ## [How to slow down output ingestion rate with logstash?](https://discuss.elastic.co/t/how-to-slow-down-output-ingestion-rate-with-logstash/306363)
**Author:** [@newelastic](https://discuss.elastic.co/u/newelastic)\ **Replies:** 2\ **Last updated:** [June 3, 2022, 9:26pm UTC](https://discuss.elastic.co/t/how-to-slow-down-output-ingestion-rate-with-logstash/306363 "2022-06-03T21:26:09Z")
Hello, I've millions of records in Elasticsearch and sending outputs as individual files using file output plugin. I've noticed logstash extraction rate is high and used a sleep plugin to slow down after every few secs… --- ## [CSV data enrichment with Geolocation](https://discuss.elastic.co/t/csv-data-enrichment-with-geolocation/306354)
**Author:** [@Vitaliy.N](https://discuss.elastic.co/u/Vitaliy.N)\ **Replies:** 4\ **Last updated:** [June 3, 2022, 8:18pm UTC](https://discuss.elastic.co/t/csv-data-enrichment-with-geolocation/306354 "2022-06-03T20:18:04Z")
Hello, I am just starting with Elk stack and need a nudge in the right direction. Is there a way to convert string address into a longitude and latitude coordinates and add them back to CSV file using Logstash? Or does … --- ## [Parse Array and Send to Elastic Filter](https://discuss.elastic.co/t/parse-array-and-send-to-elastic-filter/306271)
**Author:** [@JeremyP](https://discuss.elastic.co/u/JeremyP)\ **Replies:** 2\ **Last updated:** [June 3, 2022, 12:54pm UTC](https://discuss.elastic.co/t/parse-array-and-send-to-elastic-filter/306271 "2022-06-03T12:54:23Z")
Hello, I have an array of strings which I'd like to send nested object to an Elasticsearch filter to obtain additional data in a separately stored index. I've been searching for an approach to this but I cannot seem to … --- ## [Hardware requirement for my server ELK](https://discuss.elastic.co/t/hardware-requirement-for-my-server-elk/306229)
**Author:** [@Big\_Mara](https://discuss.elastic.co/u/Big_Mara)\ **Replies:** 4\ **Last updated:** [June 3, 2022, 12:05pm UTC](https://discuss.elastic.co/t/hardware-requirement-for-my-server-elk/306229 "2022-06-03T12:05:26Z")
Hi I am new in ELK and I wonder with what can I start if I have many firewalls and I get 200gb of logs per day. The deal is ELK is already installed but the server capacity is not enough so we need to resize it and upgr… --- ## [Grok pattern construction](https://discuss.elastic.co/t/grok-pattern-construction/305850)
**Author:** [@anushka1203](https://discuss.elastic.co/u/anushka1203)\ **Replies:** 6\ **Last updated:** [June 3, 2022, 9:09am UTC](https://discuss.elastic.co/t/grok-pattern-construction/305850 "2022-06-03T09:09:02Z")
Hi, i need help in constructing the grok pattern for the following logs. had asked the same question earlier here- this pattern %{HTTPDERROR\_DATE:date}\\s:\\s%{LOGLEVEL:log}:%{GREEDYDATA:message} works for the following… --- ## [Gracefully restart logstash after adding an additional pipeline](https://discuss.elastic.co/t/gracefully-restart-logstash-after-adding-an-additional-pipeline/306284)
**Author:** [@stillfreem](https://discuss.elastic.co/u/stillfreem)\ **Replies:** 0\ **Last updated:** [June 3, 2022, 5:15am UTC](https://discuss.elastic.co/t/gracefully-restart-logstash-after-adding-an-additional-pipeline/306284 "2022-06-03T05:15:33Z")
Hi All, in my logstash.yml I enabled the config.reload.automatic: True and doing it every 30s config.reload.interval: 30s. So now I started adding new pipelines in my pipeline.yml. Once I add a pipeline what I usually… --- ## [How to configure current age using the date plugin filter from logstash](https://discuss.elastic.co/t/how-to-configure-current-age-using-the-date-plugin-filter-from-logstash/306244)
**Author:** [@Rizky\_Hudha](https://discuss.elastic.co/u/Rizky_Hudha)\ **Replies:** 0\ **Last updated:** [June 2, 2022, 2:49pm UTC](https://discuss.elastic.co/t/how-to-configure-current-age-using-the-date-plugin-filter-from-logstash/306244 "2022-06-02T14:49:34Z")
hello I'm trying to create an age range with the logstash plugin filter, so I have date of birth data, the following is an example of the data: id, Nik,Phone,Date Of Birth,Gender 1,3710144892657,6289123432134,06/06/196… --- ## [Logstash from 2 different resource in one logstash](https://discuss.elastic.co/t/logstash-from-2-different-resource-in-one-logstash/306185)
**Author:** [@Zerra\_Triani](https://discuss.elastic.co/u/Zerra_Triani)\ **Replies:** 1\ **Last updated:** [June 2, 2022, 11:22am UTC](https://discuss.elastic.co/t/logstash-from-2-different-resource-in-one-logstash/306185 "2022-06-02T11:22:30Z")
Hi, i have two differetent resource log and i want to filtered in one server with one logstash. I've created the grok filter as I want it using the grol debugger and it's working and producing the appropriate output. But… --- ## [LOGSTASH DISCOVER FILES EACH 3 HOURS APROX](https://discuss.elastic.co/t/logstash-discover-files-each-3-hours-aprox/306186)
**Author:** [@Daniel\_Lopez](https://discuss.elastic.co/u/Daniel_Lopez)\ **Replies:** 0\ **Last updated:** [June 2, 2022, 6:00am UTC](https://discuss.elastic.co/t/logstash-discover-files-each-3-hours-aprox/306186 "2022-06-02T06:00:52Z")
Hi! I'm started a new pipeline for reading xml from a cifs mounted unit from a Windows Server in my centos server, the issue I have is that logstash reads the unit each 3 hours and I don't know why, I'm indexing old fil… --- ## [Pipeline to ingest JSON files with old objects from old cluster](https://discuss.elastic.co/t/pipeline-to-ingest-json-files-with-old-objects-from-old-cluster/306163)
**Author:** [@Mary2022](https://discuss.elastic.co/u/Mary2022)\ **Replies:** 0\ **Last updated:** [June 1, 2022, 9:28pm UTC](https://discuss.elastic.co/t/pipeline-to-ingest-json-files-with-old-objects-from-old-cluster/306163 "2022-06-01T21:28:58Z")
Hi, I still very new to ELK and need help. We had an old cluster with 2 logstashs, 3 elasticsearchs, and 2 kibana. They are running version 6.23. We built a new cluster and install version 7.15. We want to configure a … --- ## [Need grok pattern for logstash](https://discuss.elastic.co/t/need-grok-pattern-for-logstash/305835)
**Author:** [@zaeemmasood](https://discuss.elastic.co/u/zaeemmasood)\ **Replies:** 6\ **Last updated:** [June 1, 2022, 6:09pm UTC](https://discuss.elastic.co/t/need-grok-pattern-for-logstash/305835 "2022-06-01T18:09:18Z")
Hi All, I need to use grok filter to parse the following log pattern: \[2022-05-24T02:15:20.979+0000\]\[info\]\[gc \] GC(187) Pause Full (G1 Evacuation Pause) 2559M-\>1698M(2560M) 724.899ms The idea is to capture… --- ## [Logstash output file plugin adds automatically hostname](https://discuss.elastic.co/t/logstash-output-file-plugin-adds-automatically-hostname/306113)
**Author:** [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)\ **Replies:** 2\ **Last updated:** [June 1, 2022, 4:51pm UTC](https://discuss.elastic.co/t/logstash-output-file-plugin-adds-automatically-hostname/306113 "2022-06-01T16:51:31Z")
Hello I am using Logstash version 7.8.0. I am having my configuration as input is logstash-plain.log file which is its logfile. It does not contain hostname , on its line, while logstash when processing data to output… --- ## [Logstash container shut down + error creating action from filter](https://discuss.elastic.co/t/logstash-container-shut-down-error-creating-action-from-filter/306140)
**Author:** [@The-morpho](https://discuss.elastic.co/u/The-morpho)\ **Replies:** 1\ **Last updated:** [June 1, 2022, 4:23pm UTC](https://discuss.elastic.co/t/logstash-container-shut-down-error-creating-action-from-filter/306140 "2022-06-01T16:23:18Z")
Hello I'm new to Elasticsearch I'm working with log files comming from filebeat and logstash and I'm trying to add a field "response\_time", and then affect the difference between timestamps to It. So I create a logstash… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=135) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=137)