# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=59 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 60 --- ## [Can Logstash use a file in an Azure BLOB container as its direct input?](https://discuss.elastic.co/t/can-logstash-use-a-file-in-an-azure-blob-container-as-its-direct-input/343080)
**Author:** [@Eugene\_Goldberg](https://discuss.elastic.co/u/Eugene_Goldberg)\ **Replies:** 0\ **Last updated:** [September 14, 2023, 9:34pm UTC](https://discuss.elastic.co/t/can-logstash-use-a-file-in-an-azure-blob-container-as-its-direct-input/343080 "2023-09-14T21:34:13Z")
Greetings, I am trying to configure Logstash to ingest changes to a JSON file which is stored in Azure BLOB storage container. There used to be an input plugin called \`\`\` logstash-input-azureblob When I attempted to i… --- ## [Joining and mapping](https://discuss.elastic.co/t/joining-and-mapping/343069)
**Author:** [@Hamed\_Ahmadi](https://discuss.elastic.co/u/Hamed_Ahmadi)\ **Replies:** 0\ **Last updated:** [September 14, 2023, 5:25pm UTC](https://discuss.elastic.co/t/joining-and-mapping/343069 "2023-09-14T17:25:51Z")
Hi guys! I have a relational Database which I have synchronised with Elasticsearch over logstash. I have a work table, article table and comment table. One work has multiple comments and articles, basically work is pare… --- ## [Transform and simplify long regex with dissect or grok?](https://discuss.elastic.co/t/transform-and-simplify-long-regex-with-dissect-or-grok/342973)
**Author:** [@sam1975](https://discuss.elastic.co/u/sam1975)\ **Replies:** 4\ **Last updated:** [September 14, 2023, 2:09pm UTC](https://discuss.elastic.co/t/transform-and-simplify-long-regex-with-dissect-or-grok/342973 "2023-09-14T14:09:35Z")
Hello, I have some long and heavy regex and i wonder if i can simplify and gain in term of performances by using dissect here are some examples "^\<%{NONNEGINT:syslog\_priority:int}\>1 (?:-|%{TIMESTAMP\_ISO8601:syslog\_t… --- ## [No Logs from logstash docker](https://discuss.elastic.co/t/no-logs-from-logstash-docker/342882)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 10\ **Last updated:** [September 14, 2023, 6:12am UTC](https://discuss.elastic.co/t/no-logs-from-logstash-docker/342882 "2023-09-14T06:12:18Z")
I call logstash from commandline /usr/share/logstash/bin/logstash --path.settings=/usr/share/logstash/config -f /usr/share/logstash/conf.d/ But no logs written bash-4.4$ ls -ld logs1 drwxrwxrwx 2 logstash logstash 6 S… --- ## [Logstash file plugin on windows](https://discuss.elastic.co/t/logstash-file-plugin-on-windows/342852)
**Author:** [@mahmoud.shsuite](https://discuss.elastic.co/u/mahmoud.shsuite)\ **Replies:** 7\ **Last updated:** [September 13, 2023, 8:03pm UTC](https://discuss.elastic.co/t/logstash-file-plugin-on-windows/342852 "2023-09-13T20:03:41Z")
I've just installed logstach version 8.9.2 on windows and tried to do first file sample but I am greeting message=\>"Unable to configure plugins: (PluginLoadingError) Couldn't find any input plugin named 'file' I insured… --- ## [Logstash crashing](https://discuss.elastic.co/t/logstash-crashing/342972)
**Author:** [@sc5283](https://discuss.elastic.co/u/sc5283)\ **Replies:** 4\ **Last updated:** [September 13, 2023, 5:49pm UTC](https://discuss.elastic.co/t/logstash-crashing/342972 "2023-09-13T17:49:37Z")
Input is from S3 layout of S3 bucket is : s3 { .... bucket =\> "bucket" prefix =\> "YYYY/MM/DD/hh/" ..... } so every hour I have to create a new conf file with the corresponding prefix… --- ## [How to aggregate conditionally logs](https://discuss.elastic.co/t/how-to-aggregate-conditionally-logs/342945)
**Author:** [@Marieta](https://discuss.elastic.co/u/Marieta)\ **Replies:** 1\ **Last updated:** [September 13, 2023, 2:30pm UTC](https://discuss.elastic.co/t/how-to-aggregate-conditionally-logs/342945 "2023-09-13T14:30:07Z")
Hi I am trying to aggregate the following logs: 2023-09-06 07:36:22,573 | INFO | Thread-934 | Config | ENTERORDER: identifier = 'Barbie', buy = false, quantity = 290000.0, price = 96.1, account = '123', reference = '',… --- ## [Substitute GROK by dissect: test of writing](https://discuss.elastic.co/t/substitute-grok-by-dissect-test-of-writing/342930)
**Author:** [@sam1975](https://discuss.elastic.co/u/sam1975)\ **Replies:** 2\ **Last updated:** [September 13, 2023, 12:07pm UTC](https://discuss.elastic.co/t/substitute-grok-by-dissect-test-of-writing/342930 "2023-09-13T12:07:54Z")
hello, I want to substitute a grok filter by a dissect In a few words, i want replace this grok filter grok { match =\> { "\[raw\_syslog\_result\]\[syslog\_message\]" =\> \[ "THREAT,%{WORD:threat\_type},%{DATA:generate\_time},%… --- ## [Logstash output to loki](https://discuss.elastic.co/t/logstash-output-to-loki/342910)
**Author:** [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\ **Replies:** 0\ **Last updated:** [September 13, 2023, 7:04am UTC](https://discuss.elastic.co/t/logstash-output-to-loki/342910 "2023-09-13T07:04:39Z")
Hi is there any way to send data from logstash to loki or promtial or grafana? --- ## [File input not sending to Elasticsearch](https://discuss.elastic.co/t/file-input-not-sending-to-elasticsearch/342891)
**Author:** [@roman-tasi](https://discuss.elastic.co/u/roman-tasi)\ **Replies:** 0\ **Last updated:** [September 13, 2023, 12:55am UTC](https://discuss.elastic.co/t/file-input-not-sending-to-elasticsearch/342891 "2023-09-13T00:55:05Z")
Hi I am just wondering if someone could look over these relevant portions of my Logstash config to see if there is an issue: input { file { path =\> "/etc/elasticsearch/scripts/otherScripts/fortune.txt" codec =\>… --- ## [Syslog to BigQuery help](https://discuss.elastic.co/t/syslog-to-bigquery-help/342816)
**Author:** [@Russ\_Starr](https://discuss.elastic.co/u/Russ_Starr)\ **Replies:** 1\ **Last updated:** [September 12, 2023, 8:53pm UTC](https://discuss.elastic.co/t/syslog-to-bigquery-help/342816 "2023-09-12T20:53:04Z")
Hi, I am new to logstash and I've been doing some reading and grok debugging. My goal is really simple. I have a Linux box with logstash and I want to receive syslog messages from all my systems and forward them to Googl… --- ## [Logstash 8.9.0 docker logs to stdout. how to provide custom path for it?](https://discuss.elastic.co/t/logstash-8-9-0-docker-logs-to-stdout-how-to-provide-custom-path-for-it/342594)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 4\ **Last updated:** [September 12, 2023, 7:07pm UTC](https://discuss.elastic.co/t/logstash-8-9-0-docker-logs-to-stdout-how-to-provide-custom-path-for-it/342594 "2023-09-12T19:07:53Z")
What is the file and path that needs to changed to provide custom log path for logstash-plain.log and so on. Please advise --- ## [Reading new data from elastic using logstash to rabbitMQ](https://discuss.elastic.co/t/reading-new-data-from-elastic-using-logstash-to-rabbitmq/342844)
**Author:** [@Shay\_Hershko](https://discuss.elastic.co/u/Shay_Hershko)\ **Replies:** 0\ **Last updated:** [September 12, 2023, 1:51pm UTC](https://discuss.elastic.co/t/reading-new-data-from-elastic-using-logstash-to-rabbitmq/342844 "2023-09-12T13:51:59Z")
Hi, I want to send every new data entered to index in elastic to a RabbitMQ queue every second. I tried using logstash for it but for some reason it send all the data and not just the new one. I saw you can use time st… --- ## [Input jdbc error handling](https://discuss.elastic.co/t/input-jdbc-error-handling/342836)
**Author:** [@inbeom\_cho](https://discuss.elastic.co/u/inbeom_cho)\ **Replies:** 0\ **Last updated:** [September 12, 2023, 1:03pm UTC](https://discuss.elastic.co/t/input-jdbc-error-handling/342836 "2023-09-12T13:03:16Z")
hi all this is my input jdbc input { jdbc { jdbc\_driver\_library =\> "/usr/share/java/postgresql.jar" jdbc\_driver\_class =\> "org.postgresql.Driver" jdbc\_connection\_string =\> "jdbc:postgresql://\*.\*.\*.\*/databa… --- ## [After upgrading logstash to version 8.9.1, it disconnects from the DB2 database after a few days](https://discuss.elastic.co/t/after-upgrading-logstash-to-version-8-9-1-it-disconnects-from-the-db2-database-after-a-few-days/342830)
**Author:** [@Lukas\_Hrcka](https://discuss.elastic.co/u/Lukas_Hrcka)\ **Replies:** 0\ **Last updated:** [September 12, 2023, 12:20pm UTC](https://discuss.elastic.co/t/after-upgrading-logstash-to-version-8-9-1-it-disconnects-from-the-db2-database-after-a-few-days/342830 "2023-09-12T12:20:33Z")
Hello, after upgrading logstash to version 8.9.1 from 7.17.x, I have problems with the automatic loss of connection to the DB2 database (DB2 ver. 11.5 Mod 7). The previous version of ELK 7.17.x had no problem, the conne… --- ## [How to build docker image from local clone copy of source](https://discuss.elastic.co/t/how-to-build-docker-image-from-local-clone-copy-of-source/342767)
**Author:** [@balakr](https://discuss.elastic.co/u/balakr)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 9:57pm UTC](https://discuss.elastic.co/t/how-to-build-docker-image-from-local-clone-copy-of-source/342767 "2023-09-11T21:57:27Z")
i have an enlistment of logstash, would like to build a docker image, what is my command for this --- ## [Netflow gigamon - Flowset id error](https://discuss.elastic.co/t/netflow-gigamon-flowset-id-error/342747)
**Author:** [@SilasMuniz1](https://discuss.elastic.co/u/SilasMuniz1)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 2:40pm UTC](https://discuss.elastic.co/t/netflow-gigamon-flowset-id-error/342747 "2023-09-11T14:40:42Z")
Hi everybody. I still have a problem about neflow gigamon. I used netflow codec for parsing logs received from gigamon however I continuous received flowset error. My logstash is 8.4.3 version. Netflow codec versio… --- ## [Aggregate - Output issues](https://discuss.elastic.co/t/aggregate-output-issues/342536)
**Author:** [@vymk](https://discuss.elastic.co/u/vymk)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 11:34am UTC](https://discuss.elastic.co/t/aggregate-output-issues/342536 "2023-09-11T11:34:44Z")
I asked for some aggregation code a while ago (Help with aggregation code) and now finally had the time to get back at this (and changed the output concept a bit). So I want to aggregate data from multiple documents with… --- ## [Only one record gets created in the Elastic search](https://discuss.elastic.co/t/only-one-record-gets-created-in-the-elastic-search/342603)
**Author:** [@almostepic](https://discuss.elastic.co/u/almostepic)\ **Replies:** 10\ **Last updated:** [September 11, 2023, 12:16pm UTC](https://discuss.elastic.co/t/only-one-record-gets-created-in-the-elastic-search/342603 "2023-09-11T12:16:26Z")
I am using Logstash and created a conf file which allows me to input data into Elasticsearch. The data is related to git statistics from azure as I am creating a dashboard which will help to see information such as tota… --- ## [How to whitelist a hunreds nested field](https://discuss.elastic.co/t/how-to-whitelist-a-hunreds-nested-field/342563)
**Author:** [@yuswanul](https://discuss.elastic.co/u/yuswanul)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 10:36am UTC](https://discuss.elastic.co/t/how-to-whitelist-a-hunreds-nested-field/342563 "2023-09-11T10:36:04Z")
I plan to whitelist around 600 fields from 3000 fields in my index pattern but how do I do it? if I use mutate rename and then use mutate remove to delete the rest of it. it will put so much work into it. I wonder if th… --- ## [Logstash input json splitted by newline “\\n”](https://discuss.elastic.co/t/logstash-input-json-splitted-by-newline-n/342719)
**Author:** [@tomaxp13](https://discuss.elastic.co/u/tomaxp13)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 8:33am UTC](https://discuss.elastic.co/t/logstash-input-json-splitted-by-newline-n/342719 "2023-09-11T08:33:45Z")
I send the bulk to logstash where the jsons are separated by a newline. it looks like that: {"tenantId":"abcdefg","userSessionId":"zxxxxxxxxxxxxxx","startTime":1234,"endTime":12345,"duration":111,"internalUserId":"1234… --- ## [Logstash Docker to write Logstash internal logs /usr/share/logstash/logs](https://discuss.elastic.co/t/logstash-docker-to-write-logstash-internal-logs-usr-share-logstash-logs/342715)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 0\ **Last updated:** [September 11, 2023, 7:19am UTC](https://discuss.elastic.co/t/logstash-docker-to-write-logstash-internal-logs-usr-share-logstash-logs/342715 "2023-09-11T07:19:17Z")
Currently docker logs to stdout. I need to write to /usr/share/logstash/logs. cat logstash.yml path.logs: /usr/share/logstash/logs logstash.yml is in /usr/share/logstash/config. I trigger logstash from command line … --- ## [Logstash Parallelism (pipeline workers) does not work with Persistent Queue](https://discuss.elastic.co/t/logstash-parallelism-pipeline-workers-does-not-work-with-persistent-queue/342673)
**Author:** [@zalseryani](https://discuss.elastic.co/u/zalseryani)\ **Replies:** 2\ **Last updated:** [September 11, 2023, 6:44am UTC](https://discuss.elastic.co/t/logstash-parallelism-pipeline-workers-does-not-work-with-persistent-queue/342673 "2023-09-11T06:44:15Z")
Issue We found that when setting pipeline.workers to 4 and having 4 CPU, filters and output were not running in parallel, while disabling the persistence queue and relying on memory , the parallel workers were working … --- ## [Logstash failed to parse field \[host\] of type \[text\] in document](https://discuss.elastic.co/t/logstash-failed-to-parse-field-host-of-type-text-in-document/342697)
**Author:** [@ranjini](https://discuss.elastic.co/u/ranjini)\ **Replies:** 1\ **Last updated:** [September 11, 2023, 6:38am UTC](https://discuss.elastic.co/t/logstash-failed-to-parse-field-host-of-type-text-in-document/342697 "2023-09-11T06:38:19Z")
\[2023-09-10T19:02:57,621\]\[WARN \]\[logstash.outputs.amazonelasticsearch\]\[main\]\[58792cc6d6e46359a72de39af72a6b76e760ccb0beb773f15a78ec2ef0b24671\] Could not index event to Elasticsearch. {:status=\>400, :action=\>\["index", {:\_… --- ## [How to Read Real time data from url (.php real time file) and send data to elasticsearch? is there any method in logstash or filebeat to read real time data from url.?](https://discuss.elastic.co/t/how-to-read-real-time-data-from-url-php-real-time-file-and-send-data-to-elasticsearch-is-there-any-method-in-logstash-or-filebeat-to-read-real-time-data-from-url/342572)
**Author:** [@bbkunbi](https://discuss.elastic.co/u/bbkunbi)\ **Replies:** 2\ **Last updated:** [September 11, 2023, 4:31am UTC](https://discuss.elastic.co/t/how-to-read-real-time-data-from-url-php-real-time-file-and-send-data-to-elasticsearch-is-there-any-method-in-logstash-or-filebeat-to-read-real-time-data-from-url/342572 "2023-09-11T04:31:31Z")
TOPIC: Fetch data from url and send to Elasticsearch. In php file (Ex: https://temeprature.co/temp.log) my real time data is written. i want to read this latest data log and send it to Elasticsearch how to do this task? … --- ## [Grok pattern matching both the logs](https://discuss.elastic.co/t/grok-pattern-matching-both-the-logs/342586)
**Author:** [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\ **Replies:** 4\ **Last updated:** [September 11, 2023, 4:26am UTC](https://discuss.elastic.co/t/grok-pattern-matching-both-the-logs/342586 "2023-09-11T04:26:19Z")
Hi i have two types of logs, which differ just by one value at the end. can i get a single grok pattern to match both the files \[08/Sep/2023:13:28:50 +0530\] | 404 | 1 ms | 773 B | 127.0.0.1 | - | - | - | "GET /… --- ## [How does Logstash know the host.hostname field?](https://discuss.elastic.co/t/how-does-logstash-know-the-host-hostname-field/342666)
**Author:** [@Poubelle\_Dirty](https://discuss.elastic.co/u/Poubelle_Dirty)\ **Replies:** 4\ **Last updated:** [September 10, 2023, 3:17pm UTC](https://discuss.elastic.co/t/how-does-logstash-know-the-host-hostname-field/342666 "2023-09-10T15:17:26Z")
Hello, I don't find the information and I think it's about ECS. I have a really simple config on logtash that uses syslog input, grok parsing and inject into elastic cluster. input { syslog { host =\> "0.0.0.0" … --- ## [Logstash 7.17 keystore - get value by variable key](https://discuss.elastic.co/t/logstash-7-17-keystore-get-value-by-variable-key/342446)
**Author:** [@bonyolult](https://discuss.elastic.co/u/bonyolult)\ **Replies:** 4\ **Last updated:** [September 8, 2023, 8:24pm UTC](https://discuss.elastic.co/t/logstash-7-17-keystore-get-value-by-variable-key/342446 "2023-09-08T20:24:59Z")
Hello, long story short: i must process logs that are encryped with EC keys. The customer requires the keys' passwords to be stored in Logstash keystore where the key is the p12's serial and the value is the password. T… --- ## [What happens when Redis output plugin can't deliver a message?](https://discuss.elastic.co/t/what-happens-when-redis-output-plugin-cant-deliver-a-message/342630)
**Author:** [@noobiewan](https://discuss.elastic.co/u/noobiewan)\ **Replies:** 2\ **Last updated:** [September 8, 2023, 6:49pm UTC](https://discuss.elastic.co/t/what-happens-when-redis-output-plugin-cant-deliver-a-message/342630 "2023-09-08T18:49:17Z")
Hello there, I'm trying to understand what happens when an output plugin can't deliver a message. We are using logstash-output-redis to send batched messages to Redis and I would like to understand what happens if Redis… --- ## [Grok pattern to account for #](https://discuss.elastic.co/t/grok-pattern-to-account-for/342616)
**Author:** [@Jim\_Thunder](https://discuss.elastic.co/u/Jim_Thunder)\ **Replies:** 1\ **Last updated:** [September 8, 2023, 4:46pm UTC](https://discuss.elastic.co/t/grok-pattern-to-account-for/342616 "2023-09-08T16:46:26Z")
I have log messages coming in and a few of them have one of three special characters: @, -, or #. How can I get grok to ignore the hashtag without removing it from the new field? Below is the code snippet I'm using. It… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=58) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=60)