# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=101 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 102 --- ## [Convert text field to date](https://discuss.elastic.co/t/convert-text-field-to-date/320721)
**Author:** [@stefanocog](https://discuss.elastic.co/u/stefanocog)\ **Replies:** 2\ **Last updated:** [December 7, 2022, 10:36pm UTC](https://discuss.elastic.co/t/convert-text-field-to-date/320721 "2022-12-07T22:36:21Z")
Hello, I need convert a field in text format into date, i need to have another date field besides @timestamp, the @timestamp field identifies the arrival time of the log, instead the dataRisposta field (in case of respo… --- ## [Logstash: Optional fields in grok](https://discuss.elastic.co/t/logstash-optional-fields-in-grok/320604)
**Author:** [@anon99430464](https://discuss.elastic.co/u/anon99430464)\ **Replies:** 2\ **Last updated:** [December 7, 2022, 2:35pm UTC](https://discuss.elastic.co/t/logstash-optional-fields-in-grok/320604 "2022-12-07T14:35:34Z")
Hi there I'm trying to create a logstash for our logs. Our log files can look like the following 2022-11-22 10:43:59,061 INFO \[SERVER1.Subscription\] |\>\> | \[ctx:0ecaa086-f9b7-4d0e-bce4-1b8513238745\] \[req:facdaf2c-b8e6… --- ## [ILM Policy interval is later than configured](https://discuss.elastic.co/t/ilm-policy-interval-is-later-than-configured/320682)
**Author:** [@yarons](https://discuss.elastic.co/u/yarons)\ **Replies:** 1\ **Last updated:** [December 7, 2022, 1:25pm UTC](https://discuss.elastic.co/t/ilm-policy-interval-is-later-than-configured/320682 "2022-12-07T13:25:34Z")
Hi, I have created the following ILM Policy for testing: { "test-1-m": { "version": 6, "modified\_date": "2022-12-07T11:27:55.648Z", "policy": { "phases": { "hot": { "min\_age": "0ms"… --- ## [Insert Events with Changed Status Only using Logstash](https://discuss.elastic.co/t/insert-events-with-changed-status-only-using-logstash/319196)
**Author:** [@sajid](https://discuss.elastic.co/u/sajid)\ **Replies:** 2\ **Last updated:** [December 7, 2022, 1:17pm UTC](https://discuss.elastic.co/t/insert-events-with-changed-status-only-using-logstash/319196 "2022-12-07T13:17:56Z")
Hi All, Need your help to solve an issue. Below are the details: Pipeline1 : Polls multiple http endpoints every minute using http\_poller and insert the response to index1 in ES. Below is the sample response of one of… --- ## [Logstash mutate nested dynamic field path](https://discuss.elastic.co/t/logstash-mutate-nested-dynamic-field-path/320594)
**Author:** [@Shrouk\_Negm](https://discuss.elastic.co/u/Shrouk_Negm)\ **Replies:** 0\ **Last updated:** [December 6, 2022, 2:40pm UTC](https://discuss.elastic.co/t/logstash-mutate-nested-dynamic-field-path/320594 "2022-12-06T14:40:13Z")
logstash add field from third level BUT second level name is dynamic based on another field value like the bellow example entityType value depending on serviceName value so how should i do it input { kafka{ codec… --- ## [Logstash applying json filter on all messages in filter when I did not apply it](https://discuss.elastic.co/t/logstash-applying-json-filter-on-all-messages-in-filter-when-i-did-not-apply-it/320628)
**Author:** [@d14](https://discuss.elastic.co/u/d14)\ **Replies:** 1\ **Last updated:** [December 7, 2022, 12:08am UTC](https://discuss.elastic.co/t/logstash-applying-json-filter-on-all-messages-in-filter-when-i-did-not-apply-it/320628 "2022-12-07T00:08:42Z")
I have a pipeline that looks like the below, for some reason I still get \[2022-12-06T17:30:17,641\]\[ERROR\]\[logstash.codecs.json \]\[main\] \[f764d264.....\] JSON parse error, original data now in message field ...........… --- ## [How does logstash match?](https://discuss.elastic.co/t/how-does-logstash-match/320610)
**Author:** [@stefanocog](https://discuss.elastic.co/u/stefanocog)\ **Replies:** 4\ **Last updated:** [December 6, 2022, 4:57pm UTC](https://discuss.elastic.co/t/how-does-logstash-match/320610 "2022-12-06T16:57:45Z")
Hello, I need to match some logs that differ only in one fields (url), I match with grok, each grok rule matches a log, so I have different filters but with different grok rules, but now I realize that some logs after d… --- ## [DLQ Processing](https://discuss.elastic.co/t/dlq-processing/320603)
**Author:** [@dawiro](https://discuss.elastic.co/u/dawiro)\ **Replies:** 1\ **Last updated:** [December 6, 2022, 4:34pm UTC](https://discuss.elastic.co/t/dlq-processing/320603 "2022-12-06T16:34:54Z")
Hi, I'm trying to process events written to a dlq due to mapping conflicts. I'm not trying to preprocess any of those messages (yet). Instead, am just reading them in and writing them to a new index name. However, I get… --- ## [How to grok haproxy Log](https://discuss.elastic.co/t/how-to-grok-haproxy-log/320566)
**Author:** [@Roccof97](https://discuss.elastic.co/u/Roccof97)\ **Replies:** 3\ **Last updated:** [December 6, 2022, 1:59pm UTC](https://discuss.elastic.co/t/how-to-grok-haproxy-log/320566 "2022-12-06T13:59:13Z")
Hi, I'm parsing this log, but I get stuck in curly braces, I don't know what exception to add to export dns and ip. This is my log: Dec 6 10:31:43 eu01-test-test haproxy\[1311\]: eu01-test.test.lan x.x.x.x:xxxx \[06/Dec… --- ## [Cannot parse empty date](https://discuss.elastic.co/t/cannot-parse-empty-date/320575)
**Author:** [@stefanocog](https://discuss.elastic.co/u/stefanocog)\ **Replies:** 4\ **Last updated:** [December 6, 2022, 1:16pm UTC](https://discuss.elastic.co/t/cannot-parse-empty-date/320575 "2022-12-06T13:16:58Z")
Hi, I have a log that has a json field inside that can have empty fields, specifically I have a date field, the log can be like this "2022-11-28 09:24:46:705"|"+0100"|"transId: xxxxxx"|"resId: xxxxxx"|"1.1.1.1"|"https:… --- ## [Logstash input elasticsearch argument error](https://discuss.elastic.co/t/logstash-input-elasticsearch-argument-error/320538)
**Author:** [@CherryGoose](https://discuss.elastic.co/u/CherryGoose)\ **Replies:** 0\ **Last updated:** [December 6, 2022, 7:59am UTC](https://discuss.elastic.co/t/logstash-input-elasticsearch-argument-error/320538 "2022-12-06T07:59:32Z")
Hello. Im trying to setup my logstash to take docs from elastic and output them to console and im getting the following error \[ERROR\]\[logstash.agent \] Failed to execute action {:action=\>LogStash::PipelineActio… --- ## [Logstash drop filter plugin](https://discuss.elastic.co/t/logstash-drop-filter-plugin/320526)
**Author:** [@bex](https://discuss.elastic.co/u/bex)\ **Replies:** 1\ **Last updated:** [December 6, 2022, 7:19am UTC](https://discuss.elastic.co/t/logstash-drop-filter-plugin/320526 "2022-12-06T07:19:31Z")
As we know, we can drop like that: if \[log\]\[file\]\[path\] == "/var/log/messages" { drop {} } But I have case when I am getting logs in format "/var/log/messages-20221212" or "/var/log/messages-date" But logstash drop … --- ## [Xpath fails to extract](https://discuss.elastic.co/t/xpath-fails-to-extract/320479)
**Author:** [@stefanocog](https://discuss.elastic.co/u/stefanocog)\ **Replies:** 4\ **Last updated:** [December 5, 2022, 8:36pm UTC](https://discuss.elastic.co/t/xpath-fails-to-extract/320479 "2022-12-05T20:36:49Z")
Hi, i need extract one value from a parsed XML, my logstash configuration is xml { source =\> "contentRequest" target =\> "contentRequest\_field" store\_xml =\> false xpath =\> \[ "/datianagrafici/datipersonali… --- ## [Logstash multiline and clone](https://discuss.elastic.co/t/logstash-multiline-and-clone/320478)
**Author:** [@adrianfusco](https://discuss.elastic.co/u/adrianfusco)\ **Replies:** 2\ **Last updated:** [December 5, 2022, 4:28pm UTC](https://discuss.elastic.co/t/logstash-multiline-and-clone/320478 "2022-12-05T16:28:40Z")
I am sending some information from filebeat using one kind of multiline pattern. At some point, I realized I need to use another multiline pattern based on the log\_source. My idea was clone the beat and if the log\_sour… --- ## [Logstash netscaler citrix input](https://discuss.elastic.co/t/logstash-netscaler-citrix-input/319571)
**Author:** [@perezdev](https://discuss.elastic.co/u/perezdev)\ **Replies:** 2\ **Last updated:** [December 5, 2022, 1:58pm UTC](https://discuss.elastic.co/t/logstash-netscaler-citrix-input/319571 "2022-12-05T13:58:41Z")
Hello, I'm trying to collect logs from Netscaler Citrix using ipfix protocol, but I'm not able to decode properly the message. This is my input configuration file: input { udp { port =\> 9913 codec =\> netflow… --- ## [How to special date time format to timestamp format](https://discuss.elastic.co/t/how-to-special-date-time-format-to-timestamp-format/320444)
**Author:** [@akif\_bal](https://discuss.elastic.co/u/akif_bal)\ **Replies:** 6\ **Last updated:** [December 5, 2022, 1:09pm UTC](https://discuss.elastic.co/t/how-to-special-date-time-format-to-timestamp-format/320444 "2022-12-05T13:09:56Z")
I have a log type that I cannot manipulate. How can I convert the Time format in the log to timestamp format? Example log: "0001 021222 095725 00240541029896158 11 00 00 0000000000 000 0" Date and time that comes with … --- ## [Ruby logstash filter](https://discuss.elastic.co/t/ruby-logstash-filter/320408)
**Author:** [@tienld](https://discuss.elastic.co/u/tienld)\ **Replies:** 1\ **Last updated:** [December 3, 2022, 6:07pm UTC](https://discuss.elastic.co/t/ruby-logstash-filter/320408 "2022-12-03T18:07:25Z")
I have a message contains Unicode Escape Sequence I want convert it to UTF-8 character with my country language (VIetnamese) Input is from filebeat filestream I use logstash to parse the message: \\u0043\\u1ea3\\u006d\\u… --- ## [Connect logstash to elasticsearch](https://discuss.elastic.co/t/connect-logstash-to-elasticsearch/320298)
**Author:** [@vassiliy.vins](https://discuss.elastic.co/u/vassiliy.vins)\ **Replies:** 4\ **Last updated:** [December 2, 2022, 8:57pm UTC](https://discuss.elastic.co/t/connect-logstash-to-elasticsearch/320298 "2022-12-02T20:57:41Z")
Hello! I can see in logstash logs that it is not able to connect to elasticsearch to provide data The ELK documentation says: Copy the self-signed CA certificate from the Elasticsearch config/certs directory. Save it… --- ## [Fingerprint filter questions: is fingerprint source sensitive to different data types? does it include the field name when calculating fingerprint?](https://discuss.elastic.co/t/fingerprint-filter-questions-is-fingerprint-source-sensitive-to-different-data-types-does-it-include-the-field-name-when-calculating-fingerprint/320326)
**Author:** [@mikhatanu](https://discuss.elastic.co/u/mikhatanu)\ **Replies:** 1\ **Last updated:** [December 2, 2022, 4:58pm UTC](https://discuss.elastic.co/t/fingerprint-filter-questions-is-fingerprint-source-sensitive-to-different-data-types-does-it-include-the-field-name-when-calculating-fingerprint/320326 "2022-12-02T16:58:03Z")
first question: If I use a source with 2 field value of different data type (1 string field and 1 integer field) and a source with 2 string field value but same content, will it output different fingerprint? second qu… --- ## [Fingerprint filter concatenate\_all\_fields vs concatenate\_sources](https://discuss.elastic.co/t/fingerprint-filter-concatenate-all-fields-vs-concatenate-sources/320318)
**Author:** [@mikhatanu](https://discuss.elastic.co/u/mikhatanu)\ **Replies:** 1\ **Last updated:** [December 2, 2022, 4:44pm UTC](https://discuss.elastic.co/t/fingerprint-filter-concatenate-all-fields-vs-concatenate-sources/320318 "2022-12-02T16:44:15Z")
Hello, does concatenate\_sources only concat field in the source option and concatenate\_all\_fields concat all the fields from the input, e.g.: i have a log with a,b,c,d,e fields. If i use this, fingerprint{ concaten… --- ## [Setting Env Variables in logstash configuration files](https://discuss.elastic.co/t/setting-env-variables-in-logstash-configuration-files/320341)
**Author:** [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Replies:** 3\ **Last updated:** [December 2, 2022, 2:52pm UTC](https://discuss.elastic.co/t/setting-env-variables-in-logstash-configuration-files/320341 "2022-12-02T14:52:17Z")
Hello All, I would like to know how can I set up Env Variable in logstash pipleline.yml to read config files from particular folder when logstash startup. Directory structure: C-\>Logstash-\>cfg-\>Different config files … --- ## [Logstash ConfigurationError - Expect character](https://discuss.elastic.co/t/logstash-configurationerror-expect-character/320355)
**Author:** [@stefanocog](https://discuss.elastic.co/u/stefanocog)\ **Replies:** 1\ **Last updated:** [December 2, 2022, 10:47am UTC](https://discuss.elastic.co/t/logstash-configurationerror-expect-character/320355 "2022-12-02T10:47:23Z")
Hi, I've this pattern that match correctly on https://grokconstructor.appspot.com "%{TIMESTAMP\_ISO8601:timestamp}"\\|"%{DATA:tz}"\\|"%{GREEDYDATA:trans}\\: %{GREEDYDATA:transId}"\\|"%{GREEDYDATA:req}\\: %{GREEDYDATA:reqId}"\\… --- ## [Multiline logs into one event using logstash?](https://discuss.elastic.co/t/multiline-logs-into-one-event-using-logstash/320005)
**Author:** [@kurdit](https://discuss.elastic.co/u/kurdit)\ **Replies:** 9\ **Last updated:** [December 2, 2022, 10:14am UTC](https://discuss.elastic.co/t/multiline-logs-into-one-event-using-logstash/320005 "2022-12-02T10:14:08Z")
hi all! I have these logs: Nov 23 18:57:14 mx.host.cloud 18:57:14.756 2 SIPS-072111 SIPDATA-124634 REGISTER sip:111.222.333.444:65110 from udp\[555.666.777.888\]:65111 Nov 23 18:57:14 mx.host.cloud 18:57:14.756 2 SIPS-072… --- ## [Need help for creating filter pattern based on custom logs](https://discuss.elastic.co/t/need-help-for-creating-filter-pattern-based-on-custom-logs/320347)
**Author:** [@Shrikant\_Dhawale](https://discuss.elastic.co/u/Shrikant_Dhawale)\ **Replies:** 0\ **Last updated:** [December 2, 2022, 10:02am UTC](https://discuss.elastic.co/t/need-help-for-creating-filter-pattern-based-on-custom-logs/320347 "2022-12-02T10:02:55Z")
Hi All, I am new to this technology started learning, need help in writing filter plugin which will parse custom logs from the server. External User(dummy-user) is Mapped to Temp User(dummy-user) with Role(s): role1 ro… --- ## [Pipeline queue is getting filled and never able to control, using below drop filter plugin](https://discuss.elastic.co/t/pipeline-queue-is-getting-filled-and-never-able-to-control-using-below-drop-filter-plugin/320287)
**Author:** [@rravitech](https://discuss.elastic.co/u/rravitech)\ **Replies:** 2\ **Last updated:** [December 2, 2022, 5:01am UTC](https://discuss.elastic.co/t/pipeline-queue-is-getting-filled-and-never-able-to-control-using-below-drop-filter-plugin/320287 "2022-12-02T05:01:19Z")
I am trying to implement 2 pipelines, when one is about to get blocked, i am trying to drop events form that pipeline. So that the other one, which is half filled or healthy will keep receiving the events. Here in th… --- ## [Specific filter for different windows EventID](https://discuss.elastic.co/t/specific-filter-for-different-windows-eventid/320301)
**Author:** [@Marsidi](https://discuss.elastic.co/u/Marsidi)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 10:08pm UTC](https://discuss.elastic.co/t/specific-filter-for-different-windows-eventid/320301 "2022-12-01T22:08:10Z")
Hi all, I am collecting logs from different windows channels, and for different EventID I want to use different filter files. For example: If 'channel' is 'sysmon': If 'EventID' is 1: use sysmon1\_filter fi… --- ## [Logstash is not generating logs after I include the cmd under container](https://discuss.elastic.co/t/logstash-is-not-generating-logs-after-i-include-the-cmd-under-container/320271)
**Author:** [@rravitech](https://discuss.elastic.co/u/rravitech)\ **Replies:** 1\ **Last updated:** [December 1, 2022, 6:47pm UTC](https://discuss.elastic.co/t/logstash-is-not-generating-logs-after-i-include-the-cmd-under-container/320271 "2022-12-01T18:47:22Z")
How can print the logs in logstash, once after I add command and args in container. containers: - name: logstash env: - name: LOGSTASH\_PW image: docker.elastic.co/logstash/logstas… --- ## [Issue: Logstash with Opensearch get stucked after the pipeline start](https://discuss.elastic.co/t/issue-logstash-with-opensearch-get-stucked-after-the-pipeline-start/320282)
**Author:** [@elCoder](https://discuss.elastic.co/u/elCoder)\ **Replies:** 1\ **Last updated:** [December 1, 2022, 6:37pm UTC](https://discuss.elastic.co/t/issue-logstash-with-opensearch-get-stucked-after-the-pipeline-start/320282 "2022-12-01T18:37:21Z")
Hi Everyone, I hope you could help me. I need to set up Logstash reading as input from an AWS S3 bucket and sending as output to OpenSearch. The input is a folder containing many csv. The issue is that when Logstash is … --- ## [Dell iDrac Syslog Grok Logstash](https://discuss.elastic.co/t/dell-idrac-syslog-grok-logstash/320198)
**Author:** [@rcraigncs](https://discuss.elastic.co/u/rcraigncs)\ **Replies:** 2\ **Last updated:** [December 1, 2022, 5:32pm UTC](https://discuss.elastic.co/t/dell-idrac-syslog-grok-logstash/320198 "2022-12-01T17:32:24Z")
I am looking for help to grok Syslog from Dell idrac I am having problems taking the grok below and making it work with a Logstash. Your help is deeply appreciated! Extracting additional fields from iDRAC logs I found o… --- ## [Trying to send logs to AWS CloudWatch via logstash output plugin](https://discuss.elastic.co/t/trying-to-send-logs-to-aws-cloudwatch-via-logstash-output-plugin/320238)
**Author:** [@Randika\_Madhushan](https://discuss.elastic.co/u/Randika_Madhushan)\ **Replies:** 0\ **Last updated:** [December 1, 2022, 12:25pm UTC](https://discuss.elastic.co/t/trying-to-send-logs-to-aws-cloudwatch-via-logstash-output-plugin/320238 "2022-12-01T12:25:54Z")
Hello there, Currently, I'm trying to send the applications logs to AWS CloudWatch via logstash agent output plugin. So I tried the below configuration. But that did not work. Could you please help with this? input { … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=100) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=102)