# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=81 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 82 --- ## [FortiMail logs are being combined in TCP input](https://discuss.elastic.co/t/fortimail-logs-are-being-combined-in-tcp-input/329768)
**Author:** [@6igwig](https://discuss.elastic.co/u/6igwig)\ **Replies:** 4\ **Last updated:** [April 11, 2023, 7:12pm UTC](https://discuss.elastic.co/t/fortimail-logs-are-being-combined-in-tcp-input/329768 "2023-04-11T19:12:53Z")
I have configured a tcp input in logstash to receive FortiMail logs. I believe the logs are losing their new line character in transit because all of the logs come in as a single document. (If I leave the pipeline runnin… --- ## [Logstash error](https://discuss.elastic.co/t/logstash-error/329706)
**Author:** [@sks](https://discuss.elastic.co/u/sks)\ **Replies:** 1\ **Last updated:** [April 11, 2023, 3:16pm UTC](https://discuss.elastic.co/t/logstash-error/329706 "2023-04-11T15:16:08Z")
Dear sir ; i want to send a json log file from my local pc to Elasticsearch my sample json file is { "people" : \[ { "firstName": "Joe", "lastName": "Jackson", "gender": "male", "age": 28, "number": "7349282382" … --- ## [Grok filter isn't working but working in kibana grok debugger](https://discuss.elastic.co/t/grok-filter-isnt-working-but-working-in-kibana-grok-debugger/329755)
**Author:** [@ira-zaya](https://discuss.elastic.co/u/ira-zaya)\ **Replies:** 0\ **Last updated:** [April 11, 2023, 2:44pm UTC](https://discuss.elastic.co/t/grok-filter-isnt-working-but-working-in-kibana-grok-debugger/329755 "2023-04-11T14:44:21Z")
Hi. I have the following logstash configuration: filter { if "platform1" in \[tags\] { grok { match =\> { "message" =\> \['%{TIMESTAMP\_ISO8601:timestamp}? ?\\\[?L?:? ?%{LOGLEVEL:logLevel}?\\\]… --- ## [Parsing logfiles](https://discuss.elastic.co/t/parsing-logfiles/329505)
**Author:** [@SIRAJEDDINE-HAMZA](https://discuss.elastic.co/u/SIRAJEDDINE-HAMZA)\ **Replies:** 2\ **Last updated:** [April 11, 2023, 1:08pm UTC](https://discuss.elastic.co/t/parsing-logfiles/329505 "2023-04-11T13:08:16Z")
I'm new to using ElasticStack and I'm having trouble parsing a log file using Logstash. Specifically, I want to split the file using the timestamp as a separator and extract data from each block, but I'm not sure how to … --- ## [Logstash input pipelines are slow after restart](https://discuss.elastic.co/t/logstash-input-pipelines-are-slow-after-restart/329715)
**Author:** [@Amit\_Gupta2](https://discuss.elastic.co/u/Amit_Gupta2)\ **Replies:** 1\ **Last updated:** [April 11, 2023, 8:03am UTC](https://discuss.elastic.co/t/logstash-input-pipelines-are-slow-after-restart/329715 "2023-04-11T08:03:56Z")
Hi Team, I am facing slowness issue in data sync after every restart of logstash. My observation is that input pipeline are taking time to start in parallel. I am using Logstash 6.8 which is deployed on an EC2 instance… --- ## [Logs getting Merged/clubbed with each other in some cases](https://discuss.elastic.co/t/logs-getting-merged-clubbed-with-each-other-in-some-cases/329588)
**Author:** [@shadu88](https://discuss.elastic.co/u/shadu88)\ **Replies:** 10\ **Last updated:** [April 11, 2023, 7:52am UTC](https://discuss.elastic.co/t/logs-getting-merged-clubbed-with-each-other-in-some-cases/329588 "2023-04-11T07:52:05Z")
Hello Dear ELKs, I'm using logstash7.10 for forward the logs to Qradar and Azure sentinel. Have noticed some irregularities with some log source type. Log flow : heterogenous logs -\> file --\> logstash( file input) --\> … --- ## [How to created multi field parsh message from snort](https://discuss.elastic.co/t/how-to-created-multi-field-parsh-message-from-snort/329637)
**Author:** [@wisnu\_adiputra](https://discuss.elastic.co/u/wisnu_adiputra)\ **Replies:** 1\ **Last updated:** [April 10, 2023, 12:15pm UTC](https://discuss.elastic.co/t/how-to-created-multi-field-parsh-message-from-snort/329637 "2023-04-10T12:15:06Z")
Continuing the discussion from Grok pattern for snort alerts: 1/03-21:37:12.106096 \[\] \[1:249:8\] DDOS mstream client to handler \[\] \[Classification: Attempted Denial of Service\] \[Priority: 2\] {TCP} 172.16.0.5:61301 -\> 19… --- ## [Logstash pipeline error when processing a csv file](https://discuss.elastic.co/t/logstash-pipeline-error-when-processing-a-csv-file/329612)
**Author:** [@Ashutosh\_Vaidya](https://discuss.elastic.co/u/Ashutosh_Vaidya)\ **Replies:** 6\ **Last updated:** [April 10, 2023, 3:44am UTC](https://discuss.elastic.co/t/logstash-pipeline-error-when-processing-a-csv-file/329612 "2023-04-10T03:44:39Z")
Hello I am getting the below error when running the pipeline logstash conf file. Kindly let me know way to overcome this error student@elk:/$ sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/csv-read-3.co… --- ## [What is the point and purpose of ca\_trusted\_fingerprint?](https://discuss.elastic.co/t/what-is-the-point-and-purpose-of-ca-trusted-fingerprint/329623)
**Author:** [@jba](https://discuss.elastic.co/u/jba)\ **Replies:** 3\ **Last updated:** [April 9, 2023, 5:40pm UTC](https://discuss.elastic.co/t/what-is-the-point-and-purpose-of-ca-trusted-fingerprint/329623 "2023-04-09T17:40:38Z")
What is the point of adding the ca\_trusted\_fingerprint parameter to an logstash-output-elasticsearch section in an output filter? Is it purely to defend against a possible attack on DNS servers? Misconfiguration of the E… --- ## [Unable to perform airthmetic operations in ruby using logstash pipeline](https://discuss.elastic.co/t/unable-to-perform-airthmetic-operations-in-ruby-using-logstash-pipeline/329587)
**Author:** [@Sujith\_Nair](https://discuss.elastic.co/u/Sujith_Nair)\ **Replies:** 10\ **Last updated:** [April 9, 2023, 1:54pm UTC](https://discuss.elastic.co/t/unable-to-perform-airthmetic-operations-in-ruby-using-logstash-pipeline/329587 "2023-04-09T13:54:45Z")
Hi guys, I am facing an issue where i am trying to perform an airthmetic operation using ruby but in at the field section i am getting the same value not the subtracted value. event.set('\[d\]', (event.get('\[b\]').to\_f) -… --- ## [Logstash memory consumption and swap memory issues](https://discuss.elastic.co/t/logstash-memory-consumption-and-swap-memory-issues/329456)
**Author:** [@Ofek\_Agmon](https://discuss.elastic.co/u/Ofek_Agmon)\ **Replies:** 12\ **Last updated:** [April 9, 2023, 7:26am UTC](https://discuss.elastic.co/t/logstash-memory-consumption-and-swap-memory-issues/329456 "2023-04-09T07:26:56Z")
Hi all, I've been using logstash version 7.17.8 in docker, and for a while now trying to minimize its memory usage and swap usage, without much success. I have 2 file inputs and one gelf input, and 2 small filters. I … --- ## [Logstash running code](https://discuss.elastic.co/t/logstash-running-code/329408)
**Author:** [@sks](https://discuss.elastic.co/u/sks)\ **Replies:** 1\ **Last updated:** [April 5, 2023, 12:22pm UTC](https://discuss.elastic.co/t/logstash-running-code/329408 "2023-04-05T12:22:26Z")
Hyy, I am new to Elasticsearch . I am trying to send logfile from logstash to elasticsearch . for checking purspose i am running this config file as below vi logstash-simple.conf input { stdin { } } output { elasti… --- ## [Schedule , scroll , size Elasticsearch input plugin Plugin more explanation](https://discuss.elastic.co/t/schedule-scroll-size-elasticsearch-input-plugin-plugin-more-explanation/329606)
**Author:** [@alex\_petrov](https://discuss.elastic.co/u/alex_petrov)\ **Replies:** 0\ **Last updated:** [April 8, 2023, 6:11am UTC](https://discuss.elastic.co/t/schedule-scroll-size-elasticsearch-input-plugin-plugin-more-explanation/329606 "2023-04-08T06:11:51Z")
I am using elasticsearch index as my input in logstash.I read the documentation and don't understand the usage of schedule , scroll , size option.I need more explanation to understand these featues. Thanks --- ## [Can we create dependent inputs in logstash pipeline?](https://discuss.elastic.co/t/can-we-create-dependent-inputs-in-logstash-pipeline/329436)
**Author:** [@Disha\_Bodade](https://discuss.elastic.co/u/Disha_Bodade)\ **Replies:** 2\ **Last updated:** [April 7, 2023, 4:51pm UTC](https://discuss.elastic.co/t/can-we-create-dependent-inputs-in-logstash-pipeline/329436 "2023-04-07T16:51:15Z")
Hi Team, I have requirement to get the links from rss feed and extract each link and store its XML page source as a document in ES. I am trying to use rss and http\_poller input plugin together, something like below con… --- ## [Enforce Double quotes using csv codec plugin](https://discuss.elastic.co/t/enforce-double-quotes-using-csv-codec-plugin/329585)
**Author:** [@uzair13151](https://discuss.elastic.co/u/uzair13151)\ **Replies:** 1\ **Last updated:** [April 7, 2023, 4:30pm UTC](https://discuss.elastic.co/t/enforce-double-quotes-using-csv-codec-plugin/329585 "2023-04-07T16:30:09Z")
Hi All, Is it possible to wrap the data in the rows to be encapsulated by double quotes using csv codec plugin. Currently I am getting: Column1|Column2|Column3 Data1|Data2|"" Expectation: "Column1"|"Column2"|"Colum… --- ## [I don't see my index in index management](https://discuss.elastic.co/t/i-dont-see-my-index-in-index-management/329515)
**Author:** [@Hanni](https://discuss.elastic.co/u/Hanni)\ **Replies:** 2\ **Last updated:** [April 7, 2023, 8:59am UTC](https://discuss.elastic.co/t/i-dont-see-my-index-in-index-management/329515 "2023-04-07T08:59:16Z")
Hello, I'm trying to send data from a CSV file to Elasticsearch using Logstash. I have configured my input, filter, and output, but I cannot find my index in Elasticsearch. I have the impression that Logstash is not proc… --- ## [Backfill with previous indexed data](https://discuss.elastic.co/t/backfill-with-previous-indexed-data/329321)
**Author:** [@suminlim](https://discuss.elastic.co/u/suminlim)\ **Replies:** 1\ **Last updated:** [April 7, 2023, 3:02am UTC](https://discuss.elastic.co/t/backfill-with-previous-indexed-data/329321 "2023-04-07T03:02:23Z")
using logstash, is it available ? input { elasticsearch { hosts =\> "localhost" index =\> "logs" query =\> '{ "sort": \[ "timeinfo" \] }' } } filter { if \[location\] == "" { // how to get previous data … --- ## [Backfill with previous indexed data](https://discuss.elastic.co/t/backfill-with-previous-indexed-data/329278)
**Author:** [@suminlim](https://discuss.elastic.co/u/suminlim)\ **Replies:** 7\ **Last updated:** [April 7, 2023, 3:01am UTC](https://discuss.elastic.co/t/backfill-with-previous-indexed-data/329278 "2023-04-07T03:01:09Z")
When the document sorted by timestamp, is there any solution that backfill location data into next row with previous row ??? --- ## [Parse log file line by line](https://discuss.elastic.co/t/parse-log-file-line-by-line/329518)
**Author:** [@pen120](https://discuss.elastic.co/u/pen120)\ **Replies:** 3\ **Last updated:** [April 6, 2023, 1:58pm UTC](https://discuss.elastic.co/t/parse-log-file-line-by-line/329518 "2023-04-06T13:58:53Z")
I have a log file with output repetitive below, I want to parse this log line by line in fields to extract the value for each line 10:31:07 2022/10/16 ZBXTRAP 192.168.23.2 PDU INFO: messageid 0 … --- ## [Unable to authenticate user \[logstash\_internal\] for REST request \[/bulk\]](https://discuss.elastic.co/t/unable-to-authenticate-user-logstash-internal-for-rest-request-bulk/329473)
**Author:** [@oscardoudou](https://discuss.elastic.co/u/oscardoudou)\ **Replies:** 3\ **Last updated:** [April 6, 2023, 12:59pm UTC](https://discuss.elastic.co/t/unable-to-authenticate-user-logstash-internal-for-rest-request-bulk/329473 "2023-04-06T12:59:33Z")
logstash 7.17.9 \[2023-04-05T22:50:02,837\]\[ERROR\]\[logstash.outputs.elasticsearch\]\[main\] Encountered a retryable error (will retry with exponential backoff) {:code=\>401, :url=\>"http://server:9200/\_bulk", :content\_length=\>… --- ## [Logstash snmp OID](https://discuss.elastic.co/t/logstash-snmp-oid/327677)
**Author:** [@San9](https://discuss.elastic.co/u/San9)\ **Replies:** 2\ **Last updated:** [April 6, 2023, 12:30pm UTC](https://discuss.elastic.co/t/logstash-snmp-oid/327677 "2023-04-06T12:30:20Z")
Hi all. In logstash, I use the snmp module to poll the OIDs. It partially works, I get the required values. I have many OIDs and hosts to poll. The problem is that on some of the equipment certain oids do not work or … --- ## [Logstash bulk requests growing after some time](https://discuss.elastic.co/t/logstash-bulk-requests-growing-after-some-time/328486)
**Author:** [@zerzn](https://discuss.elastic.co/u/zerzn)\ **Replies:** 1\ **Last updated:** [April 6, 2023, 9:15am UTC](https://discuss.elastic.co/t/logstash-bulk-requests-growing-after-some-time/328486 "2023-04-06T09:15:47Z")
hi, I have a strange situation with logstash with elasticsearch and maybe someone can help me out. My logstash bulk requests to elasticsearch are growing after some hours. (2-24h) There are arround 1000 winlogbeat age… --- ## [In Operator Behaves Unexpectedly When Used in a Filter](https://discuss.elastic.co/t/in-operator-behaves-unexpectedly-when-used-in-a-filter/329449)
**Author:** [@foxfire-auspex](https://discuss.elastic.co/u/foxfire-auspex)\ **Replies:** 2\ **Last updated:** [April 6, 2023, 8:42am UTC](https://discuss.elastic.co/t/in-operator-behaves-unexpectedly-when-used-in-a-filter/329449 "2023-04-06T08:42:59Z")
Hello, I just had a very strange experience debugging one of our Logstash filters and would like to know whether we could have anticipated this or encountered a known quirk or bug (we run Logstash v7.17). Please see be… --- ## [Logstash s3 parsing issue](https://discuss.elastic.co/t/logstash-s3-parsing-issue/329484)
**Author:** [@Rushikesh](https://discuss.elastic.co/u/Rushikesh)\ **Replies:** 0\ **Last updated:** [April 6, 2023, 6:12am UTC](https://discuss.elastic.co/t/logstash-s3-parsing-issue/329484 "2023-04-06T06:12:30Z")
output { s3 { access\_key\_id =\> "test" secret\_access\_key =\> "test" bucket =\> "logstorage" region =\> "us-west-1" codec =\> "json\_lines" prefix =\> "%{+YYYY}/%{+MM}/%{+dd}/example.log" } } So I h… --- ## [Logstash Output Issue](https://discuss.elastic.co/t/logstash-output-issue/329319)
**Author:** [@Rushikesh](https://discuss.elastic.co/u/Rushikesh)\ **Replies:** 2\ **Last updated:** [April 6, 2023, 5:10am UTC](https://discuss.elastic.co/t/logstash-output-issue/329319 "2023-04-06T05:10:29Z")
Below is my logstash configuration file. input { beats { port =\> 5044 } } filter { json { source =\> "message" } } output { stdout { codec =\> json } } filter { mutate { add\_field =\> { … --- ## [I have a question need great god help to have a look, a data into the es and run for a period of time will appear stuck, here are some information](https://discuss.elastic.co/t/i-have-a-question-need-great-god-help-to-have-a-look-a-data-into-the-es-and-run-for-a-period-of-time-will-appear-stuck-here-are-some-information/329184)
**Author:** [@northestface](https://discuss.elastic.co/u/northestface)\ **Replies:** 1\ **Last updated:** [April 6, 2023, 3:14am UTC](https://discuss.elastic.co/t/i-have-a-question-need-great-god-help-to-have-a-look-a-data-into-the-es-and-run-for-a-period-of-time-will-appear-stuck-here-are-some-information/329184 "2023-04-06T03:14:40Z")
this is jstash information 2023-04-03 16:34:24 Full thread dump OpenJDK 64-Bit Server VM (17.0.6+10 mixed mode, sharing): Threads class SMR info: \_java\_thread\_list=0x00007f24d0004620, length=41, elements={ 0x00007f251c… --- ## [Logstash: Ingesting the data from Azure Storage](https://discuss.elastic.co/t/logstash-ingesting-the-data-from-azure-storage/329404)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 2\ **Last updated:** [April 5, 2023, 7:29pm UTC](https://discuss.elastic.co/t/logstash-ingesting-the-data-from-azure-storage/329404 "2023-04-05T19:29:46Z")
I know from AWS S3 we can ingest the data into elasticsearch using logstash. Similar way is it possible to ingest the data from Azure Storage as well? I don't see the input plugin in the documentation - Input plugins | … --- ## [Either white space or a %{WORD:\_\_\_\_\_}](https://discuss.elastic.co/t/either-white-space-or-a-word/329357)
**Author:** [@Jim\_Thunder](https://discuss.elastic.co/u/Jim_Thunder)\ **Replies:** 3\ **Last updated:** [April 5, 2023, 2:35pm UTC](https://discuss.elastic.co/t/either-white-space-or-a-word/329357 "2023-04-05T14:35:12Z")
I have two different kinds of messages that are very similar: "Rec": " 10:33:38 +HCXPCTA-E CW83 ISMDAYS ASRA"} "Rec": " 10:31:56 +HCXPCTA-E IS60 RX1 ISMDAYS ASRA"} In the REC field one message has RX1 while … --- ## [Unable to add empty field with Logstash](https://discuss.elastic.co/t/unable-to-add-empty-field-with-logstash/329306)
**Author:** [@Jirka\_Liska](https://discuss.elastic.co/u/Jirka_Liska)\ **Replies:** 2\ **Last updated:** [April 5, 2023, 9:34am UTC](https://discuss.elastic.co/t/unable-to-add-empty-field-with-logstash/329306 "2023-04-05T09:34:39Z")
Hello community! I'm trying to add empty field into Logstash parsers like this: mutate { add\_field =\> {"comments" =\> {} } } this is my mapping in Kibana: { "\_meta": { "documentation": "https://www.elastic.co/gu… --- ## [Issue with elasticsearch](https://discuss.elastic.co/t/issue-with-elasticsearch/329333)
**Author:** [@Hanni](https://discuss.elastic.co/u/Hanni)\ **Replies:** 6\ **Last updated:** [April 5, 2023, 9:11am UTC](https://discuss.elastic.co/t/issue-with-elasticsearch/329333 "2023-04-05T09:11:25Z")
\[logstash.outputs.elasticsearch\]\[main\] Attempted to resurrect connection to dead ES instance, but got an error {:url=\>"http://newadmin:xxxxxx@localhost:9200/", :exception=\>LogStash::Outputs::Elasticsearch::HttpClient::Po… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=80) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=82)