# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=68 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 69 --- ## [Unable to parse grokpattern for below log in opensearch ](https://discuss.elastic.co/t/unable-to-parse-grokpattern-for-below-log-in-opensearch/337839)
**Author:** [@David\_Kumar\_Duggu](https://discuss.elastic.co/u/David_Kumar_Duggu)\ **Replies:** 2\ **Last updated:** [July 6, 2023, 8:13pm UTC](https://discuss.elastic.co/t/unable-to-parse-grokpattern-for-below-log-in-opensearch/337839 "2023-07-06T20:13:34Z")
I am trying to parse the below log using Grok pattern, Grok pattern is parsing in grokdebugger but it is not able to parse in open search. Please find the log attached {"level":"info","msg":"method:offlineActivate,name:… --- ## [Grok pattern for datadog to get everything between two curly braces {}](https://discuss.elastic.co/t/grok-pattern-for-datadog-to-get-everything-between-two-curly-braces/337665)
**Author:** [@shekhsadiq21](https://discuss.elastic.co/u/shekhsadiq21)\ **Replies:** 8\ **Last updated:** [July 6, 2023, 4:47pm UTC](https://discuss.elastic.co/t/grok-pattern-for-datadog-to-get-everything-between-two-curly-braces/337665 "2023-07-06T16:47:47Z")
Hi All, Can anyone help to get GROK pattern of logs everthing between two curly braces logsample: { CONNECTION: keep-alive X-ORIGINAL-URL: /Data/RetailItem.js?Log=1&Sync=0 X-FORWARDED-PROTO: https X-FORWARDED-PORT:… --- ## [Problem in query in logstash](https://discuss.elastic.co/t/problem-in-query-in-logstash/337781)
**Author:** [@Hind\_Alla](https://discuss.elastic.co/u/Hind_Alla)\ **Replies:** 5\ **Last updated:** [July 6, 2023, 2:28pm UTC](https://discuss.elastic.co/t/problem-in-query-in-logstash/337781 "2023-07-06T14:28:23Z")
input { jdbc { jdbc\_driver\_library =\> "XXXX" jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver" jdbc\_connection\_string =\> "XXXX" jdbc\_user =\> "XXXX" jdbc\_password =\> "XXXX" statement =\> "SELECT \* FROM MO… --- ## [Read an index and count in other index](https://discuss.elastic.co/t/read-an-index-and-count-in-other-index/337803)
**Author:** [@reed](https://discuss.elastic.co/u/reed)\ **Replies:** 0\ **Last updated:** [July 6, 2023, 2:00pm UTC](https://discuss.elastic.co/t/read-an-index-and-count-in-other-index/337803 "2023-07-06T14:00:43Z")
Hi all, I have two index: index\_master name - code m1 - c1 m2 - c2 m3 - c3 index\_details code - other fields c1 - data field c1 - data field c1 - data field c1 - data field c2 - data field c2 - dat… --- ## [Hi all i am getting error like Logstash stopped processing because of an error: (SystemExit) exit](https://discuss.elastic.co/t/hi-all-i-am-getting-error-like-logstash-stopped-processing-because-of-an-error-systemexit-exit/337536)
**Author:** [@maheswari1](https://discuss.elastic.co/u/maheswari1)\ **Replies:** 4\ **Last updated:** [July 6, 2023, 12:07pm UTC](https://discuss.elastic.co/t/hi-all-i-am-getting-error-like-logstash-stopped-processing-because-of-an-error-systemexit-exit/337536 "2023-07-06T12:07:29Z")
Logstash stopped processing because of an error: (SystemExit) exit --- ## [Logstash does not update document](https://discuss.elastic.co/t/logstash-does-not-update-document/337777)
**Author:** [@mehmetalix](https://discuss.elastic.co/u/mehmetalix)\ **Replies:** 0\ **Last updated:** [July 6, 2023, 10:24am UTC](https://discuss.elastic.co/t/logstash-does-not-update-document/337777 "2023-07-06T10:24:23Z")
Hi, I have a problem with data update in logstash. I need to update specific field in some document according to sql data. My data is looking like this: testid-field1-field2-field3-testtype-time 1-1-1-1-1-2023/05 1… --- ## [Issue's in configuring kafka input plugin with TLS](https://discuss.elastic.co/t/issues-in-configuring-kafka-input-plugin-with-tls/337761)
**Author:** [@girish.ms](https://discuss.elastic.co/u/girish.ms)\ **Replies:** 0\ **Last updated:** [July 6, 2023, 7:50am UTC](https://discuss.elastic.co/t/issues-in-configuring-kafka-input-plugin-with-tls/337761 "2023-07-06T07:50:34Z")
Description of the problem: I'm having trouble integrating Kafka with Logstash, and Kafka is configured with TLS. I am getting the following exceptions when trying to provide PKCS12 format TLS certificates in the Kafka… --- ## [JSON parse error, original data now in message field {:message=\>"Could not set field 'original' on object '' to value '{\\"event\\": \\"\\"}'](https://discuss.elastic.co/t/json-parse-error-original-data-now-in-message-field-message-could-not-set-field-original-on-object-to-value-event/337740)
**Author:** [@cosmosir](https://discuss.elastic.co/u/cosmosir)\ **Replies:** 0\ **Last updated:** [July 6, 2023, 4:09am UTC](https://discuss.elastic.co/t/json-parse-error-original-data-now-in-message-field-message-could-not-set-field-original-on-object-to-value-event/337740 "2023-07-06T04:09:53Z")
logstash8.8.1 JSON parse error, original data now in message field {:message=\>"Could not set field 'original' on object '' to value '{"event": ""}'.This is probably due to trying to set a field like \[foo\]\[bar\] = someVal… --- ## [Wildcard search for a word](https://discuss.elastic.co/t/wildcard-search-for-a-word/337718)
**Author:** [@umesh\_choudary](https://discuss.elastic.co/u/umesh_choudary)\ **Replies:** 0\ **Last updated:** [July 5, 2023, 9:03pm UTC](https://discuss.elastic.co/t/wildcard-search-for-a-word/337718 "2023-07-05T21:03:02Z")
How can i search for a word as contains while searching index. eg: if i search the index using books, i need to get the results which should contain book and books in the response.. --- ## [Deserializing Avro Records with different schemas](https://discuss.elastic.co/t/deserializing-avro-records-with-different-schemas/337701)
**Author:** [@fine\_porcupine](https://discuss.elastic.co/u/fine_porcupine)\ **Replies:** 2\ **Last updated:** [July 5, 2023, 7:43pm UTC](https://discuss.elastic.co/t/deserializing-avro-records-with-different-schemas/337701 "2023-07-05T19:43:12Z")
I'm doing the due diligence on the Avro Codec Plugin and I'm wondering if it's possible to use this if there are different types of events in the same SQS queue? For example - SQS Queue contains serialized events with s… --- ## [Installation document for ELK 8.7](https://discuss.elastic.co/t/installation-document-for-elk-8-7/337635)
**Author:** [@SivaPrasadELK](https://discuss.elastic.co/u/SivaPrasadELK)\ **Replies:** 1\ **Last updated:** [July 5, 2023, 1:59pm UTC](https://discuss.elastic.co/t/installation-document-for-elk-8-7/337635 "2023-07-05T13:59:40Z")
While trying to install the ELK 8.7 and its components such as file beat logstash kibana and Elasticsearch and trying to setup the ELK as below Filebeat =====\> Logstash ======\> elastic =======\>kibana keeping this workf… --- ## [Is it possible to use encrypted elascticsearch instead of direct username, password in Output plugin of logstash?](https://discuss.elastic.co/t/is-it-possible-to-use-encrypted-elascticsearch-instead-of-direct-username-password-in-output-plugin-of-logstash/337647)
**Author:** [@merson](https://discuss.elastic.co/u/merson)\ **Replies:** 1\ **Last updated:** [July 5, 2023, 1:24pm UTC](https://discuss.elastic.co/t/is-it-possible-to-use-encrypted-elascticsearch-instead-of-direct-username-password-in-output-plugin-of-logstash/337647 "2023-07-05T13:24:57Z")
I want to use encrypted elasticsearch, So I don't want to use directly username ,password of elasticsearch in logstash. Please provide the any answers. --- ## [Logstash build from Source failing](https://discuss.elastic.co/t/logstash-build-from-source-failing/337611)
**Author:** [@tejas7](https://discuss.elastic.co/u/tejas7)\ **Replies:** 2\ **Last updated:** [July 5, 2023, 12:58pm UTC](https://discuss.elastic.co/t/logstash-build-from-source-failing/337611 "2023-07-05T12:58:49Z")
Hello Team , I am trying to build logstash from source code from the main branch. It is failing with the following error: \* Exception is: org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':install… --- ## [Logstash JDBC Input - Time difference problem](https://discuss.elastic.co/t/logstash-jdbc-input-time-difference-problem/337542)
**Author:** [@thibaut\_a](https://discuss.elastic.co/u/thibaut_a)\ **Replies:** 2\ **Last updated:** [July 5, 2023, 7:20am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-time-difference-problem/337542 "2023-07-05T07:20:45Z")
Hi, I receive entries from a PostgreSQL database in my Logstash Docker container. I get what I want but I'm facing a problem with the timestamp. When I do a request in pgAdmin, it shows timestamps with the local time (Fr… --- ## [MongoDB to Elasticsearch?](https://discuss.elastic.co/t/mongodb-to-elasticsearch/336767)
**Author:** [@stephane\_chan](https://discuss.elastic.co/u/stephane_chan)\ **Replies:** 4\ **Last updated:** [July 5, 2023, 6:40am UTC](https://discuss.elastic.co/t/mongodb-to-elasticsearch/336767 "2023-07-05T06:40:52Z")
Is there a way to index data from mongoDB to elasticsearch? I've searched a bit but I haven't found any mongodb input on logstash i.e. part of the mongoDB collection by making an aggregation query and then storing the r… --- ## [How to use curl command to input data into logstash](https://discuss.elastic.co/t/how-to-use-curl-command-to-input-data-into-logstash/337615)
**Author:** [@vijeibarthi](https://discuss.elastic.co/u/vijeibarthi)\ **Replies:** 0\ **Last updated:** [July 5, 2023, 6:21am UTC](https://discuss.elastic.co/t/how-to-use-curl-command-to-input-data-into-logstash/337615 "2023-07-05T06:21:16Z")
Hi All, I have a curl command which works fine but I have trouble using that curl command in the json format. Please guide on how to correct this script to output the data. =============================================… --- ## [Not eligible for data streams because config contains one or more settings that are not compatible with data streams](https://discuss.elastic.co/t/not-eligible-for-data-streams-because-config-contains-one-or-more-settings-that-are-not-compatible-with-data-streams/337594)
**Author:** [@Cruz](https://discuss.elastic.co/u/Cruz)\ **Replies:** 4\ **Last updated:** [July 5, 2023, 3:27am UTC](https://discuss.elastic.co/t/not-eligible-for-data-streams-because-config-contains-one-or-more-settings-that-are-not-compatible-with-data-streams/337594 "2023-07-05T03:27:44Z")
I tried to upload my template using Logstash, but it did not. It says that \[2023-07-05T00:03:53,496\]\[INFO \]\[logstash.outputs.elasticsearch\]\[main\] Not eligible for data streams because config contains one or more settin… --- ## [Bitdefender GravityZone and Logstash Integration](https://discuss.elastic.co/t/bitdefender-gravityzone-and-logstash-integration/337582)
**Author:** [@Paulo\_Martins\_de\_Sen](https://discuss.elastic.co/u/Paulo_Martins_de_Sen)\ **Replies:** 0\ **Last updated:** [July 4, 2023, 5:24pm UTC](https://discuss.elastic.co/t/bitdefender-gravityzone-and-logstash-integration/337582 "2023-07-04T17:24:48Z")
Hey guys, has anyone done Bitdefender GravityZone and Elastic Security integration? I'm trying to do it through agent elastic, but without success so far. --- ## [Logstash @timestamp not including milliseconds](https://discuss.elastic.co/t/logstash-timestamp-not-including-milliseconds/337579)
**Author:** [@Anthony\_Zottola](https://discuss.elastic.co/u/Anthony_Zottola)\ **Replies:** 2\ **Last updated:** [July 4, 2023, 4:17pm UTC](https://discuss.elastic.co/t/logstash-timestamp-not-including-milliseconds/337579 "2023-07-04T16:17:08Z")
All of my logs have this format "@timestamp" =\> 2023-07-04T15:40:19.000Z where the milliseconds are missing, is there any way to make it included the milliseconds. I tried manually adding it but it is read only. My con… --- ## [How to parse log in “message” fieled cisco-ise](https://discuss.elastic.co/t/how-to-parse-log-in-message-fieled-cisco-ise/337492)
**Author:** [@Mbrezzy](https://discuss.elastic.co/u/Mbrezzy)\ **Replies:** 5\ **Last updated:** [July 3, 2023, 10:05pm UTC](https://discuss.elastic.co/t/how-to-parse-log-in-message-fieled-cisco-ise/337492 "2023-07-03T22:05:57Z")
How to parsing this log logstash Hey everyone i am facing a problem with logstash. I want to parse log that are coming from cisco-ise but all information i want is inside message fields : I want parsing like Net… --- ## [Csv with double quotes, spaces, special characters not inserting](https://discuss.elastic.co/t/csv-with-double-quotes-spaces-special-characters-not-inserting/336561)
**Author:** [@mohanss08](https://discuss.elastic.co/u/mohanss08)\ **Replies:** 6\ **Last updated:** [July 3, 2023, 3:06pm UTC](https://discuss.elastic.co/t/csv-with-double-quotes-spaces-special-characters-not-inserting/336561 "2023-07-03T15:06:56Z")
I'm trying to insert my csv report into elastisearch through logstash. But not able to insert successfully. My csv data look as given below. 1,1234556,30-12-2022,frank.van,SAMPLE\_PRODUCT,"\[Name\] Frank Van Puffelen JAVA… --- ## [A question about separator in logstash](https://discuss.elastic.co/t/a-question-about-separator-in-logstash/337427)
**Author:** [@caixukun](https://discuss.elastic.co/u/caixukun)\ **Replies:** 5\ **Last updated:** [July 3, 2023, 11:44am UTC](https://discuss.elastic.co/t/a-question-about-separator-in-logstash/337427 "2023-07-03T11:44:42Z")
my data is: 123456@gmail.com----john----password 123456@gmail.com----john----p@ssword 123456@gmail.com----john----123456 123456@gmail.com----john----123456 123456@gmail.com----john----123----456 123456@gmail.com---… --- ## [Secure logstash connection to elasticsearch](https://discuss.elastic.co/t/secure-logstash-connection-to-elasticsearch/337369)
**Author:** [@PeroWong](https://discuss.elastic.co/u/PeroWong)\ **Replies:** 1\ **Last updated:** [July 3, 2023, 10:12am UTC](https://discuss.elastic.co/t/secure-logstash-connection-to-elasticsearch/337369 "2023-07-03T10:12:26Z")
I follow the tutorial(Install Elasticsearch with Docker | Elasticsearch Guide \[8.8\] | Elastic) creating certs in the docker-compose.yml services setup. And I follow the guide to config logstash scure Secure your connect… --- ## [Hi All, so I have a requirement where I need logstash to capture error log messages and trigger a mail upon that , is that possible with basic open source version. Thanks in advance](https://discuss.elastic.co/t/hi-all-so-i-have-a-requirement-where-i-need-logstash-to-capture-error-log-messages-and-trigger-a-mail-upon-that-is-that-possible-with-basic-open-source-version-thanks-in-advance/337423)
**Author:** [@Akulainelastic](https://discuss.elastic.co/u/Akulainelastic)\ **Replies:** 1\ **Last updated:** [July 3, 2023, 8:33am UTC](https://discuss.elastic.co/t/hi-all-so-i-have-a-requirement-where-i-need-logstash-to-capture-error-log-messages-and-trigger-a-mail-upon-that-is-that-possible-with-basic-open-source-version-thanks-in-advance/337423 "2023-07-03T08:33:42Z")
Continuing the discussion from Timestamp problem created using dissect: --- ## [One or more required cgroup files or directories not found in logstash](https://discuss.elastic.co/t/one-or-more-required-cgroup-files-or-directories-not-found-in-logstash/337348)
**Author:** [@sanjay\_bhati](https://discuss.elastic.co/u/sanjay_bhati)\ **Replies:** 3\ **Last updated:** [July 1, 2023, 4:02pm UTC](https://discuss.elastic.co/t/one-or-more-required-cgroup-files-or-directories-not-found-in-logstash/337348 "2023-07-01T16:02:14Z")
I am getting error: One or more required cgroup files or directories not found here is my input file input { file { path =\> "/Users/spbhati/Downloads/S3BucketConfiguration.csv" start\_position =\> "beginning" sincedb… --- ## [How to refer to the whole modified event inside http output plugin](https://discuss.elastic.co/t/how-to-refer-to-the-whole-modified-event-inside-http-output-plugin/337232)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 2\ **Last updated:** [June 30, 2023, 6:23pm UTC](https://discuss.elastic.co/t/how-to-refer-to-the-whole-modified-event-inside-http-output-plugin/337232 "2023-06-30T18:23:38Z")
I am trying to map my http payload and put the whole Logstash event inside another json key/field: http { format =\> "json" http\_method =\> "post" url =\> "some url" headers =\> \["some header"\] ma… --- ## [Force Logstash Finish on Error](https://discuss.elastic.co/t/force-logstash-finish-on-error/337331)
**Author:** [@palomasun](https://discuss.elastic.co/u/palomasun)\ **Replies:** 2\ **Last updated:** [June 30, 2023, 5:03pm UTC](https://discuss.elastic.co/t/force-logstash-finish-on-error/337331 "2023-06-30T17:03:07Z")
Hi, I use logstash 7.12.1 and I would like to avoid, in case of any error, a ethernal loop: For instance, if my configuration file doesn´t have a certification path it , loops: "unreacheble elastic... " Is there a way… --- ## [Logstash duplication](https://discuss.elastic.co/t/logstash-duplication/335847)
**Author:** [@ramiwashere](https://discuss.elastic.co/u/ramiwashere)\ **Replies:** 2\ **Last updated:** [June 30, 2023, 3:06pm UTC](https://discuss.elastic.co/t/logstash-duplication/335847 "2023-06-30T15:06:12Z")
Hello I have created a logstash pipeline via the http\_poller plugin in order to collect information from an API link. In order to manage the duplication of documents, I used the 'fingerprint' plugin in the filter part … --- ## [SQS Input Plugin retries](https://discuss.elastic.co/t/sqs-input-plugin-retries/337321)
**Author:** [@fine\_porcupine](https://discuss.elastic.co/u/fine_porcupine)\ **Replies:** 0\ **Last updated:** [June 30, 2023, 2:27pm UTC](https://discuss.elastic.co/t/sqs-input-plugin-retries/337321 "2023-06-30T14:27:44Z")
I have a Logstash pipeline that receives events from an AWS SQS queue via the SQS Input Plugin. If there is a failure during data processing, will SQS retry the event, or do I need a Logstash DLQ to handle intermittent f… --- ## [Is there a way to dynamically group overlapping events?](https://discuss.elastic.co/t/is-there-a-way-to-dynamically-group-overlapping-events/337290)
**Author:** [@landre](https://discuss.elastic.co/u/landre)\ **Replies:** 0\ **Last updated:** [June 30, 2023, 9:03am UTC](https://discuss.elastic.co/t/is-there-a-way-to-dynamically-group-overlapping-events/337290 "2023-06-30T09:03:29Z")
I am importing data from MySQL using logstash, that contains events with a start and an end date. However, some of these events overlap and, in some conditions, need to be treated as a single event, starting at the start… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=67) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=69)