# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=826 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 827 --- ## [Logstash Reading Same Data](https://discuss.elastic.co/t/logstash-reading-same-data/312251)
**Author:** [@mehmetalix](https://discuss.elastic.co/u/mehmetalix)\ **Replies:** 1\ **Last updated:** [August 18, 2022, 6:49am UTC](https://discuss.elastic.co/t/logstash-reading-same-data/312251 "2022-08-18T06:49:22Z")
Hi, I have a logstash .conf: input { jdbc { jdbc\_driver\_library =\> "${LOGSTASH\_JDBC\_DRIVER\_JAR\_LOCATION}" jdbc\_driver\_class =\> "${LOGSTASH\_JDBC\_DRIVER}" jdbc\_connection\_string =\> "${LOGSTASH… --- ## [\[Filebeat\] AWS CloudTrail Processor parses incorrect AWS region from logs](https://discuss.elastic.co/t/filebeat-aws-cloudtrail-processor-parses-incorrect-aws-region-from-logs/312150)
**Author:** [@styks90](https://discuss.elastic.co/u/styks90)\ **Replies:** 3\ **Last updated:** [August 18, 2022, 6:31am UTC](https://discuss.elastic.co/t/filebeat-aws-cloudtrail-processor-parses-incorrect-aws-region-from-logs/312150 "2022-08-18T06:31:48Z")
When processing AWS CloudTrail logs, we noticed a discrepancy in the parsed AWS region compared to the region a log file originated. CloudTrail is not region specific , but monitors all configured regions and stuffs the … --- ## [Value not greater 20% of the average value of the same job](https://discuss.elastic.co/t/value-not-greater-20-of-the-average-value-of-the-same-job/310832)
**Author:** [@eddieyee](https://discuss.elastic.co/u/eddieyee)\ **Replies:** 2\ **Last updated:** [August 18, 2022, 6:23am UTC](https://discuss.elastic.co/t/value-not-greater-20-of-the-average-value-of-the-same-job/310832 "2022-08-18T06:23:11Z")
Hi I'm trying to create a data table as seem in the screenshot below. Basically I need to compare the current document duration field value not higher than 20% of the average duration of the same job. Have tried using … --- ## [Resthighlevel client unit test is not working](https://discuss.elastic.co/t/resthighlevel-client-unit-test-is-not-working/311897)
**Author:** [@Apurba](https://discuss.elastic.co/u/Apurba)\ **Replies:** 2\ **Last updated:** [August 18, 2022, 6:05am UTC](https://discuss.elastic.co/t/resthighlevel-client-unit-test-is-not-working/311897 "2022-08-18T06:05:23Z")
I have written a method, below @Inject private RestHighLevelClient esClient; public SearchResponse getData(SearchRequest searchRequest) { try { SearchResponse searchResponse = esClient.s… --- ## [Timezone problem when output to Elasticsearch](https://discuss.elastic.co/t/timezone-problem-when-output-to-elasticsearch/312252)
**Author:** [@kent010341](https://discuss.elastic.co/u/kent010341)\ **Replies:** 10\ **Last updated:** [August 18, 2022, 4:36am UTC](https://discuss.elastic.co/t/timezone-problem-when-output-to-elasticsearch/312252 "2022-08-18T04:36:14Z")
I have multiple log files with the date on their file name, and I want to read them with the file input plugin and send them to an Elasticsearch index (the index name contains the date). However, I have some logs being … --- ## [How keystore works](https://discuss.elastic.co/t/how-keystore-works/312274)
**Author:** [@GustavoPires](https://discuss.elastic.co/u/GustavoPires)\ **Replies:** 1\ **Last updated:** [August 18, 2022, 3:59am UTC](https://discuss.elastic.co/t/how-keystore-works/312274 "2022-08-18T03:59:31Z")
Hello, can someone explain me what is the purpose of the keystore and how can I configure it? --- ## [Backreference in regex](https://discuss.elastic.co/t/backreference-in-regex/312299)
**Author:** [@Lukas\_Hubl](https://discuss.elastic.co/u/Lukas_Hubl)\ **Replies:** 1\ **Last updated:** [August 18, 2022, 3:11am UTC](https://discuss.elastic.co/t/backreference-in-regex/312299 "2022-08-18T03:11:45Z")
Hi, is it possible to use backreference for regex in auditbeat/filebeat? I am trying something like this: \\"1\\":\\"(\\w+)\\",\\"2\\":\\"\\1\\" and the following string should match: "1":"string","2":"string" But I am getti… --- ## [Gcp pubsub input error](https://discuss.elastic.co/t/gcp-pubsub-input-error/312333)
**Author:** [@Gustavo\_Llermaly](https://discuss.elastic.co/u/Gustavo_Llermaly)\ **Replies:** 0\ **Last updated:** [August 18, 2022, 3:02am UTC](https://discuss.elastic.co/t/gcp-pubsub-input-error/312333 "2022-08-18T03:02:56Z")
Logstash 8.3.3, having this error with gcp pubsub input. Trying with basic settings. \[2022-08-17T22:52:45,136\]\[ERROR\]\[logstash.javapipeline \]\[main\]\[69466b68efba17b190dd16bcd12466cd5488773b860c9a0f79fa6c7c65f5ec61\] A … --- ## [Transform is not created when a particular field is not available in the index](https://discuss.elastic.co/t/transform-is-not-created-when-a-particular-field-is-not-available-in-the-index/312214)
**Author:** [@Amol](https://discuss.elastic.co/u/Amol)\ **Replies:** 2\ **Last updated:** [August 18, 2022, 2:31am UTC](https://discuss.elastic.co/t/transform-is-not-created-when-a-particular-field-is-not-available-in-the-index/312214 "2022-08-18T02:31:14Z")
Hello Team, I'm trying to create a transform, it is creating the transform index, but when the particular (Year column) is blank or not available in the Data set document, that document is not index. Could any one help… --- ## [How to generate shingles before synonyms filter?](https://discuss.elastic.co/t/how-to-generate-shingles-before-synonyms-filter/312235)
**Author:** [@mattkallo](https://discuss.elastic.co/u/mattkallo)\ **Replies:** 2\ **Last updated:** [August 17, 2022, 10:56pm UTC](https://discuss.elastic.co/t/how-to-generate-shingles-before-synonyms-filter/312235 "2022-08-17T22:56:32Z")
Hi, I understand we can't have token filters that produce multiple tokens before a synonyms filter. (Ensure TokenFilters only produce single tokens when parsing synonyms by romseygeek · Pull Request #34331 · elastic/ela… --- ## [Disk usage exceeded flood-stage watermark](https://discuss.elastic.co/t/disk-usage-exceeded-flood-stage-watermark/312294)
**Author:** [@Khaled\_Saidi](https://discuss.elastic.co/u/Khaled_Saidi)\ **Replies:** 1\ **Last updated:** [August 17, 2022, 9:53pm UTC](https://discuss.elastic.co/t/disk-usage-exceeded-flood-stage-watermark/312294 "2022-08-17T21:53:10Z")
Hello everybody, I need some help on conception and settings. I have a cluster of 6 nodes running each one an elasticsearch server instance. An error was logged in logstash log file : TOO\_MANY\_REQUESTS/12/disk usage … --- ## [Error “S02000B61A1A001\>Service 'Elasticsearch 8.1.3 (elasticsearch-service-x64)' is not running. (No automatic restart is attempted)](https://discuss.elastic.co/t/error-s02000b61a1a001-service-elasticsearch-8-1-3-elasticsearch-service-x64-is-not-running-no-automatic-restart-is-attempted/312303)
**Author:** [@Coen](https://discuss.elastic.co/u/Coen)\ **Replies:** 1\ **Last updated:** [August 17, 2022, 9:42pm UTC](https://discuss.elastic.co/t/error-s02000b61a1a001-service-elasticsearch-8-1-3-elasticsearch-service-x64-is-not-running-no-automatic-restart-is-attempted/312303 "2022-08-17T21:42:40Z")
As the title says “S02000B61A1A001\>Service 'Elasticsearch 8.1.3 (elasticsearch-service-x64)' is not running. (No automatic restart is attempted). This event seems to happen daily, the services are being started by us an… --- ## [MongoDB-SSPL-1.0 license issue](https://discuss.elastic.co/t/mongodb-sspl-1-0-license-issue/312317)
**Author:** [@xliu](https://discuss.elastic.co/u/xliu)\ **Replies:** 3\ **Last updated:** [August 17, 2022, 8:38pm UTC](https://discuss.elastic.co/t/mongodb-sspl-1-0-license-issue/312317 "2022-08-17T20:38:33Z")
We use elasticsearch 7.17.4. Nexus-iq reports 'Banned' license threat group ('MongoDB-SSPL-1.0'). Nexus IQ Server is Sonatype's policy engine. Sonatype has MongoDB-SSPL-1.0 in Banned list. The policy error comes out f… --- ## [Unable to create the repository for snapshots after mounting nfs share in the container](https://discuss.elastic.co/t/unable-to-create-the-repository-for-snapshots-after-mounting-nfs-share-in-the-container/312284)
**Author:** [@Bhagya\_09](https://discuss.elastic.co/u/Bhagya_09)\ **Replies:** 6\ **Last updated:** [August 17, 2022, 5:31pm UTC](https://discuss.elastic.co/t/unable-to-create-the-repository-for-snapshots-after-mounting-nfs-share-in-the-container/312284 "2022-08-17T17:31:31Z")
curl -k -u elastic:password -XPUT "http://localhost:9200/\_snapshot/backupes?pretty" -H 'Content-Type: application/json' -d'{"type": "fs","settings":{"location": "/es\_snapshots"}}' { "error" : { "root\_cause" : \[ { "t… --- ## [Elasticsearch into docker](https://discuss.elastic.co/t/elasticsearch-into-docker/311528)
**Author:** [@dav1](https://discuss.elastic.co/u/dav1)\ **Replies:** 1\ **Last updated:** [August 17, 2022, 4:17pm UTC](https://discuss.elastic.co/t/elasticsearch-into-docker/311528 "2022-08-17T16:17:19Z")
Hit. I would like to use docker to run my elascticsearch. I use this image : docker.elastic.co/elasticsearch/elasticsearch:7.7.0 My environment : Windows 10 Docker Desktop (Docker version 20.10.17, build 100c701) do… --- ## [Developing Custom Parser for Logs](https://discuss.elastic.co/t/developing-custom-parser-for-logs/312021)
**Author:** [@sta02](https://discuss.elastic.co/u/sta02)\ **Replies:** 2\ **Last updated:** [August 17, 2022, 3:36pm UTC](https://discuss.elastic.co/t/developing-custom-parser-for-logs/312021 "2022-08-17T15:36:26Z")
Hello, In my installation, the output of Logstash is set to Azure Sentinel. The installation is working fine. However, the client prefers to send logs from inhouse applications to Sentinel via Syslog. Is it possible to… --- ## [How to read mongoDB logs from logstash jdbc plugin](https://discuss.elastic.co/t/how-to-read-mongodb-logs-from-logstash-jdbc-plugin/312301)
**Author:** [@sundar\_elk](https://discuss.elastic.co/u/sundar_elk)\ **Replies:** 0\ **Last updated:** [August 17, 2022, 3:00pm UTC](https://discuss.elastic.co/t/how-to-read-mongodb-logs-from-logstash-jdbc-plugin/312301 "2022-08-17T15:00:28Z")
Hi All, We are trying to get logs from MongoDB database using logstash jdbc plugin and below is the Logstash version :- 7.16.2 MongoDB version :- 4.2 MongoJDBC driver version :- mongojdbc4.1.jar (download jar from Dow… --- ## [Logstash parse json child element, format and insert into elasticsearch](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230)
**Author:** [@babuzrb](https://discuss.elastic.co/u/babuzrb)\ **Replies:** 9\ **Last updated:** [August 17, 2022, 2:46pm UTC](https://discuss.elastic.co/t/logstash-parse-json-child-element-format-and-insert-into-elasticsearch/312230 "2022-08-17T14:46:50Z")
I have a json file like this: "fruits": { "fruit": \[ { "id": 1, "label": "test", "tag": "fine", "start": "4", "end": "9" }, { "id": 2, "lab… --- ## [No field "postgresql.log.level" frome Filebat Posresql module in Kibana](https://discuss.elastic.co/t/no-field-postgresql-log-level-frome-filebat-posresql-module-in-kibana/311725)
**Author:** [@sadux](https://discuss.elastic.co/u/sadux)\ **Replies:** 1\ **Last updated:** [August 17, 2022, 2:23pm UTC](https://discuss.elastic.co/t/no-field-postgresql-log-level-frome-filebat-posresql-module-in-kibana/311725 "2022-08-17T14:23:48Z")
Hello! Just install filebeat (7.15.1)+postgresql module. filebeat.yml: output.logstash: hosts: \["logstash.test.com:5001","logstash2.test.com:5001"\] loadbalance: true fields: env: prod host\_type: postgresql fi… --- ## [Event/Resource Correlation Detection](https://discuss.elastic.co/t/event-resource-correlation-detection/312196)
**Author:** [@BenB196](https://discuss.elastic.co/u/BenB196)\ **Replies:** 3\ **Last updated:** [August 16, 2022, 8:20pm UTC](https://discuss.elastic.co/t/event-resource-correlation-detection/312196 "2022-08-16T20:20:12Z")
Hi All, I was wondering if anyone has any ideas on how to achieve correlation detection between events and resources. Use Case I have a visualization which shows the 10 top process by memory usage on a system. It also… --- ## [Elasticsearch query fails intermittently](https://discuss.elastic.co/t/elasticsearch-query-fails-intermittently/312295)
**Author:** [@dsc-dev](https://discuss.elastic.co/u/dsc-dev)\ **Replies:** 0\ **Last updated:** [August 17, 2022, 2:14pm UTC](https://discuss.elastic.co/t/elasticsearch-query-fails-intermittently/312295 "2022-08-17T14:14:11Z")
I am trying to analyze why my below elastic query fails intermittently. It fails like 1 out of 3 times. The query seems correct as it works other times. Below is the python script along with the logs. we are running th… --- ## [Filebeat cant monitor the log file that file name include \[\]](https://discuss.elastic.co/t/filebeat-cant-monitor-the-log-file-that-file-name-include/311758)
**Author:** [@zyjj1](https://discuss.elastic.co/u/zyjj1)\ **Replies:** 1\ **Last updated:** [August 17, 2022, 2:13pm UTC](https://discuss.elastic.co/t/filebeat-cant-monitor-the-log-file-that-file-name-include/311758 "2022-08-17T14:13:56Z")
We monitor a log file. The name of this log file is mwsummary-log\[0\].log. We found that filebeat cannot monitor any content of this file. Also no data is sent to elasticsearch. If we remove the in the log file name, fil… --- ## [Fscrawler only indexed 59 of a 2000 page pdf](https://discuss.elastic.co/t/fscrawler-only-indexed-59-of-a-2000-page-pdf/312262)
**Author:** [@defalt](https://discuss.elastic.co/u/defalt)\ **Replies:** 1\ **Last updated:** [August 17, 2022, 1:06pm UTC](https://discuss.elastic.co/t/fscrawler-only-indexed-59-of-a-2000-page-pdf/312262 "2022-08-17T13:06:10Z")
Hello :wave: i guess @dadoonet would know best about this but if someone else can answer this it would be great. I am currently trying to index pdfs into elasticsearch. I installed the 'Ingest Attachment Processor Plug… --- ## [Do Elastic Agent (integrations) and Logstash have best practices?a](https://discuss.elastic.co/t/do-elastic-agent-integrations-and-logstash-have-best-practices-a/312279)
**Author:** [@GustavoPires](https://discuss.elastic.co/u/GustavoPires)\ **Replies:** 1\ **Last updated:** [August 17, 2022, 12:16pm UTC](https://discuss.elastic.co/t/do-elastic-agent-integrations-and-logstash-have-best-practices-a/312279 "2022-08-17T12:16:21Z")
Hello, are there any good practices for using Logstash or Elastic Agent (integrations), for example, I have Pfsense Integration for Elastic Agent, at the same time I can send data via Syslog to Logstash. In what situatio… --- ## [Comparing the values of two fields](https://discuss.elastic.co/t/comparing-the-values-of-two-fields/312280)
**Author:** [@Lukas\_Hubl](https://discuss.elastic.co/u/Lukas_Hubl)\ **Replies:** 0\ **Last updated:** [August 17, 2022, 12:10pm UTC](https://discuss.elastic.co/t/comparing-the-values-of-two-fields/312280 "2022-08-17T12:10:05Z")
Hello, I am setting up the auditbeat processor, and I am trying to figure out comparison of two fields based on theirs values and use it in condition for processor, like: equals.field1.inner\_field: field2.inner\_field I… --- ## [Test multiline pattern - random order](https://discuss.elastic.co/t/test-multiline-pattern-random-order/312275)
**Author:** [@elkuser1234](https://discuss.elastic.co/u/elkuser1234)\ **Replies:** 0\ **Last updated:** [August 17, 2022, 12:00pm UTC](https://discuss.elastic.co/t/test-multiline-pattern-random-order/312275 "2022-08-17T12:00:01Z")
Hi I have elk stack 8.3 I need to test multiline patterns. I do it so that I write a given part of the log to the .txt file, and then rewrite it to the log file monitored by filebeat example: \[xxx @ xxx logs\] # cat … --- ## [Data Stream Backing Indice Count](https://discuss.elastic.co/t/data-stream-backing-indice-count/312271)
**Author:** [@Saeed\_Mozaffari](https://discuss.elastic.co/u/Saeed_Mozaffari)\ **Replies:** 1\ **Last updated:** [August 17, 2022, 11:28am UTC](https://discuss.elastic.co/t/data-stream-backing-indice-count/312271 "2022-08-17T11:28:00Z")
Dose the number of data stream backing indice have any effect on search time performance? for example if in one year i rollover every one month and in another scenario i rollover every one day, are these two scenarios d… --- ## [KQL to elastic query string](https://discuss.elastic.co/t/kql-to-elastic-query-string/312216)
**Author:** [@Drpkton](https://discuss.elastic.co/u/Drpkton)\ **Replies:** 0\ **Last updated:** [August 16, 2022, 7:30pm UTC](https://discuss.elastic.co/t/kql-to-elastic-query-string/312216 "2022-08-16T19:30:59Z")
When I perform a Kibana Query, ie, I know it is converted to a json blob which represents a valid elasticsearch json query, ie, { "version": true, "size": 500, "sort": \[ { "@timest… --- ## [Is Elasticsearch guide available in PDF?](https://discuss.elastic.co/t/is-elasticsearch-guide-available-in-pdf/312043)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 3\ **Last updated:** [August 17, 2022, 10:23am UTC](https://discuss.elastic.co/t/is-elasticsearch-guide-available-in-pdf/312043 "2022-08-17T10:23:09Z")
Hi Team, Wondering if elasticsearch guide is available in PDF format? --- ## [Custom list of keywords](https://discuss.elastic.co/t/custom-list-of-keywords/312264)
**Author:** [@seeminglee](https://discuss.elastic.co/u/seeminglee)\ **Replies:** 0\ **Last updated:** [August 17, 2022, 9:44am UTC](https://discuss.elastic.co/t/custom-list-of-keywords/312264 "2022-08-17T09:44:35Z")
I have setup an index with the following settings: { "settings": { "analysis": { "analyzer": { "html\_analyzer": { "type": "custom", "tokeni… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=825) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=827)