# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1253 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1254 --- ## [How to calculate old date from today by subtracting given years, months, days in logstash?](https://discuss.elastic.co/t/how-to-calculate-old-date-from-today-by-subtracting-given-years-months-days-in-logstash/283349)
**Author:** [@priyaankaa](https://discuss.elastic.co/u/priyaankaa)\ **Replies:** 2\ **Last updated:** [September 4, 2021, 5:36pm UTC](https://discuss.elastic.co/t/how-to-calculate-old-date-from-today-by-subtracting-given-years-months-days-in-logstash/283349 "2021-09-04T17:36:53Z")
I want to subtract current date with given years, months, days using logstash pipeline. For example : Today = 1 Aug 2021 Given values : Years=25, Months=10, Days=10 Expected date : 22 Sept 1995 My logstash filter sec… --- ## [Logstash multiline issue - limit of size in single event](https://discuss.elastic.co/t/logstash-multiline-issue-limit-of-size-in-single-event/283335)
**Author:** [@ErSumit](https://discuss.elastic.co/u/ErSumit)\ **Replies:** 2\ **Last updated:** [September 4, 2021, 5:15pm UTC](https://discuss.elastic.co/t/logstash-multiline-issue-limit-of-size-in-single-event/283335 "2021-09-04T17:15:40Z")
Logstash file input configured with multiline pattern as codec =\> multiline { pattern =\> '^ {' negate =\> true what =\> previous } } Input file consist of multiple json objects. This mu… --- ## [Disk.size is 10 times more than disk.indices](https://discuss.elastic.co/t/disk-size-is-10-times-more-than-disk-indices/283280)
**Author:** [@mrudrego](https://discuss.elastic.co/u/mrudrego)\ **Replies:** 4\ **Last updated:** [September 4, 2021, 11:47am UTC](https://discuss.elastic.co/t/disk-size-is-10-times-more-than-disk-indices/283280 "2021-09-04T11:47:56Z")
We are facing shard allocation issues because of disk watermark and when checked allocation API, strangely disk.size is 5-6 times more than disk.indices \[root@bcmt-ovs-control-0 ~\]# curl -s elasticsearch.cgnat.svc.clust… --- ## [Logstash add field default type is string but check field in an array field must match type](https://discuss.elastic.co/t/logstash-add-field-default-type-is-string-but-check-field-in-an-array-field-must-match-type/283334)
**Author:** [@qinkkai](https://discuss.elastic.co/u/qinkkai)\ **Replies:** 1\ **Last updated:** [September 4, 2021, 7:56am UTC](https://discuss.elastic.co/t/logstash-add-field-default-type-is-string-but-check-field-in-an-array-field-must-match-type/283334 "2021-09-04T07:56:24Z")
My logstash config as follow，I want to output only when the field a in the arr. I think the condition in the output is always true. input { generator { message =\> '{"a":111}' codec =\> json add\_field =\>… --- ## [Should we adopt elasticsearch on top of postgres to handle our search/filtering?](https://discuss.elastic.co/t/should-we-adopt-elasticsearch-on-top-of-postgres-to-handle-our-search-filtering/283324)
**Author:** [@elastic\_yogi](https://discuss.elastic.co/u/elastic_yogi)\ **Replies:** 1\ **Last updated:** [September 4, 2021, 7:29am UTC](https://discuss.elastic.co/t/should-we-adopt-elasticsearch-on-top-of-postgres-to-handle-our-search-filtering/283324 "2021-09-04T07:29:02Z")
We have a web service that allows a ~10 customers to look at their data. Each customer has about 1M - 10M records. Our total datastore size will target the 10's of gigabytes. We currently store this data in postgres norm… --- ## [What does this error mean - Data too large, data for \[\\]](https://discuss.elastic.co/t/what-does-this-error-mean-data-too-large-data-for-transport-request/209345)
**Author:** [@Dmitry1](https://discuss.elastic.co/u/Dmitry1)\ **Replies:** 7\ **Last updated:** [September 4, 2021, 3:38am UTC](https://discuss.elastic.co/t/what-does-this-error-mean-data-too-large-data-for-transport-request/209345 "2021-09-04T03:38:37Z")
Hello, please explain me what this error mean ? And what I need to do to get rid of it : \[2019-11-25T00:20:58,521\]\[WARN \]\[o.e.x.m.e.l.LocalExporter\] \[host\] unexpected error while indexing monitoring document org.elastic… --- ## [Query to display the results contains speicifed string](https://discuss.elastic.co/t/query-to-display-the-results-contains-speicifed-string/48570)
**Author:** [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Replies:** 8\ **Last updated:** [September 4, 2021, 3:32am UTC](https://discuss.elastic.co/t/query-to-display-the-results-contains-speicifed-string/48570 "2021-09-04T03:32:15Z")
Hi, Application name is property in the fields list of Kibana dashboard viewlet. Suppose if I need to display logs which contains application name in the log message then what query I need to use in dashboard. Thanks… --- ## [Dynamic naming of data-streams](https://discuss.elastic.co/t/dynamic-naming-of-data-streams/283281)
**Author:** [@marcus\_lhisp](https://discuss.elastic.co/u/marcus_lhisp)\ **Replies:** 2\ **Last updated:** [September 3, 2021, 10:49pm UTC](https://discuss.elastic.co/t/dynamic-naming-of-data-streams/283281 "2021-09-03T22:49:32Z")
Hello, Currently I'm trying to simplify my pipelines but unfortunately the sprintf format for field reference according to Field References Deep Dive | Logstash Reference \[7.14\] | Elastic does not work. Am I doing some… --- ## [Date Time shows empty in lens, but not in discover pane](https://discuss.elastic.co/t/date-time-shows-empty-in-lens-but-not-in-discover-pane/283318)
**Author:** [@Tydorius](https://discuss.elastic.co/u/Tydorius)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 10:08pm UTC](https://discuss.elastic.co/t/date-time-shows-empty-in-lens-but-not-in-discover-pane/283318 "2021-09-03T22:08:50Z")
I'm working on a beat to pull in GPU/CPU temperature data into Elasticsearch. My date is correctly parsed from "yyyy'-'MM'-'dd'T'HH':'mm':'ss" into a date time. You can see my original object datetimestamp vs the same d… --- ## [Setting search.allow\_expensive\_queries to false breaks Kibana Alert Interface](https://discuss.elastic.co/t/setting-search-allow-expensive-queries-to-false-breaks-kibana-alert-interface/283199)
**Author:** [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Replies:** 3\ **Last updated:** [September 3, 2021, 9:44pm UTC](https://discuss.elastic.co/t/setting-search-allow-expensive-queries-to-false-breaks-kibana-alert-interface/283199 "2021-09-03T21:44:13Z")
Hello, I needed to set search.allow\_expensive\_queries to false because some expensive queries were putting a high load on my cluster and impacting the indexing, so temporarily I made this change. But after I applied th… --- ## [Getting missing repository exception while creating a repo in snapshot.Please help](https://discuss.elastic.co/t/getting-missing-repository-exception-while-creating-a-repo-in-snapshot-please-help/283247)
**Author:** [@yangesh4u](https://discuss.elastic.co/u/yangesh4u)\ **Replies:** 2\ **Last updated:** [September 3, 2021, 8:43pm UTC](https://discuss.elastic.co/t/getting-missing-repository-exception-while-creating-a-repo-in-snapshot-please-help/283247 "2021-09-03T20:43:35Z")
{ "error": { "root\_cause": \[ { "type": "repository\_verification\_exception", "reason": "\[my\_repo\] \[\[CZR5TlNcQCy-MvDNaUs7JA, 'RemoteTransportException\[\[ant-es-master-only-3\]\[155.165.157.81:9300\]… --- ## [How to create a new search template through node js client](https://discuss.elastic.co/t/how-to-create-a-new-search-template-through-node-js-client/283216)
**Author:** [@Gustavo\_Leite\_Silvan](https://discuss.elastic.co/u/Gustavo_Leite_Silvan)\ **Replies:** 2\ **Last updated:** [September 3, 2021, 8:08pm UTC](https://discuss.elastic.co/t/how-to-create-a-new-search-template-through-node-js-client/283216 "2021-09-03T20:08:47Z")
I'm using Javascript client. Trying to create search templates through it, but didn't found the method to do it. --- ## [Many js file being downloaded while loading dashbaord](https://discuss.elastic.co/t/many-js-file-being-downloaded-while-loading-dashbaord/283253)
**Author:** [@Lakhan\_Rathi](https://discuss.elastic.co/u/Lakhan_Rathi)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 7:40pm UTC](https://discuss.elastic.co/t/many-js-file-being-downloaded-while-loading-dashbaord/283253 "2021-09-03T19:40:21Z")
I am trying to load kibana dashboard on my website with help of ifream but while loading its downloading many js files Which on search found to be from x-pack Its almost 100 js files for one dashboard can someone sugg… --- ## [Deleting multiple kibana documents](https://discuss.elastic.co/t/deleting-multiple-kibana-documents/283267)
**Author:** [@LL\_FED](https://discuss.elastic.co/u/LL_FED)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 5:28pm UTC](https://discuss.elastic.co/t/deleting-multiple-kibana-documents/283267 "2021-09-03T17:28:52Z")
Hi! I know i can delete single documents with the "Dev-Tool", but is it possible to delete multiple documents at once? For example I want to delete all documents, which include a specific field. Thanks for any kind of… --- ## [ElasticSearch aggregation shows unexpected result for SUM](https://discuss.elastic.co/t/elasticsearch-aggregation-shows-unexpected-result-for-sum/283286)
**Author:** [@VaZa](https://discuss.elastic.co/u/VaZa)\ **Replies:** 4\ **Last updated:** [September 3, 2021, 4:39pm UTC](https://discuss.elastic.co/t/elasticsearch-aggregation-shows-unexpected-result-for-sum/283286 "2021-09-03T16:39:11Z")
Trying to apply sum aggregation and get unexpected result 1. prepare dataset $cat products.json {"index":{"\_id":"1"}} {"productId": 10,"shopId": 45,"prices": {"retailPrice": 525000000.02,"sumRetailPrice": 5250000000.2}… --- ## [Getting \_grokparsefailure when passing full log files](https://discuss.elastic.co/t/getting-grokparsefailure-when-passing-full-log-files/283207)
**Author:** [@tarabhavi](https://discuss.elastic.co/u/tarabhavi)\ **Replies:** 4\ **Last updated:** [September 3, 2021, 3:42pm UTC](https://discuss.elastic.co/t/getting-grokparsefailure-when-passing-full-log-files/283207 "2021-09-03T15:42:18Z")
have ping logs files used the excerpt of it as a sample for grok filter and tested with grok \_debugger and all good When I pass the full file will all logs instead of excerpt get \_grokparsefailure though was working w… --- ## [Filebeat holding open handles on deleted files](https://discuss.elastic.co/t/filebeat-holding-open-handles-on-deleted-files/283192)
**Author:** [@seadub](https://discuss.elastic.co/u/seadub)\ **Replies:** 2\ **Last updated:** [September 3, 2021, 3:39pm UTC](https://discuss.elastic.co/t/filebeat-holding-open-handles-on-deleted-files/283192 "2021-09-03T15:39:06Z")
v. 6.8.4 We are experiencing issues daily with lingering open file handles on files which have been deleted by another process which has completed daily compression of logs. The files seem to remain open forever until … --- ## [Point in Time with Search After missing a few logs](https://discuss.elastic.co/t/point-in-time-with-search-after-missing-a-few-logs/283296)
**Author:** [@woal\_idal](https://discuss.elastic.co/u/woal_idal)\ **Replies:** 0\ **Last updated:** [September 3, 2021, 3:01pm UTC](https://discuss.elastic.co/t/point-in-time-with-search-after-missing-a-few-logs/283296 "2021-09-03T15:01:36Z")
When I run a regular query search in elasticsearch, I will get all the values. Now, when I implement the point in time api and search after because using the regular search api can handle up to 10K hits/results the query… --- ## [Best way to create a list of all \_ids in an index (Up to date version)](https://discuss.elastic.co/t/best-way-to-create-a-list-of-all-ids-in-an-index-up-to-date-version/283195)
**Author:** [@steveh](https://discuss.elastic.co/u/steveh)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 3:16pm UTC](https://discuss.elastic.co/t/best-way-to-create-a-list-of-all-ids-in-an-index-up-to-date-version/283195 "2021-09-03T15:16:08Z")
So what's the current best way to query (and therefore dump to say a file) all the document \_ids in an index? I have a 218,000 doc database with about 8000 documents missing (according to the document counts). To inves… --- ## [Meta fields in a query](https://discuss.elastic.co/t/meta-fields-in-a-query/283217)
**Author:** [@rgopalan](https://discuss.elastic.co/u/rgopalan)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 3:13pm UTC](https://discuss.elastic.co/t/meta-fields-in-a-query/283217 "2021-09-03T15:13:41Z")
Hi, I am trying to isolate application flows that cause slow queries that are being logged in the slow log. Many of our search requests are very similar and it is hard to figure out which user was performing what actio… --- ## [Extact costum field from log](https://discuss.elastic.co/t/extact-costum-field-from-log/283288)
**Author:** [@lemahdois](https://discuss.elastic.co/u/lemahdois)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 3:13pm UTC](https://discuss.elastic.co/t/extact-costum-field-from-log/283288 "2021-09-03T15:13:26Z")
Hello, I have a compicated log that does not follow commun format ......... RType: XXX ....... I want to find an easy way to extract the RType field and the value that corresponds to it XXX (for example). note that i… --- ## [How to ignore malformed query for IP property](https://discuss.elastic.co/t/how-to-ignore-malformed-query-for-ip-property/283221)
**Author:** [@hZhYNXaCbgkgjyFBJ](https://discuss.elastic.co/u/hZhYNXaCbgkgjyFBJ)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 3:12pm UTC](https://discuss.elastic.co/t/how-to-ignore-malformed-query-for-ip-property/283221 "2021-09-03T15:12:52Z")
Hi there, I have an Elasticsearch index whose mapping has two properties - one is of type keyword and the other is of type ip: "mappings": { "properties": { "user\_id": { "… --- ## [Merging search results across multiple related indexes](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100)
**Author:** [@someguy](https://discuss.elastic.co/u/someguy)\ **Replies:** 8\ **Last updated:** [September 3, 2021, 3:01pm UTC](https://discuss.elastic.co/t/merging-search-results-across-multiple-related-indexes/282100 "2021-09-03T15:01:45Z")
Working on a project where we have a core document (one index) and each document can have multiple related files that we want searchable (a couple other indexes for different types of files). We want to implement a sear… --- ## [JAVA API not found for top\_metric aggregation](https://discuss.elastic.co/t/java-api-not-found-for-top-metric-aggregation/282880)
**Author:** [@crchaulagain](https://discuss.elastic.co/u/crchaulagain)\ **Replies:** 4\ **Last updated:** [September 3, 2021, 2:57pm UTC](https://discuss.elastic.co/t/java-api-not-found-for-top-metric-aggregation/282880 "2021-09-03T14:57:47Z")
I am using Elasticsearch 7.11.1 and going to use top\_metric aggregation to get the top records under the terms aggregation bucket. And my query is something like the following: { "size": 0, "query": { "bool": { "mu… --- ## [Deserialize different typed field to other typed property in Nest](https://discuss.elastic.co/t/deserialize-different-typed-field-to-other-typed-property-in-nest/283159)
**Author:** [@Martin\_Leonard\_Sart1](https://discuss.elastic.co/u/Martin_Leonard_Sart1)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 2:56pm UTC](https://discuss.elastic.co/t/deserialize-different-typed-field-to-other-typed-property-in-nest/283159 "2021-09-03T14:56:21Z")
Hi, We just recently upgraded from 6.3 to 7.7 version of Elasticsearch. Upon migration, we got an error something like this: "{"expected:'Number Token', actual:'"10"' at offset:458"}" Based on a blog I've read, bluew… --- ## [Watcher Timestamp showing in EPOCH](https://discuss.elastic.co/t/watcher-timestamp-showing-in-epoch/283075)
**Author:** [@arunhk3](https://discuss.elastic.co/u/arunhk3)\ **Replies:** 3\ **Last updated:** [September 3, 2021, 2:51pm UTC](https://discuss.elastic.co/t/watcher-timestamp-showing-in-epoch/283075 "2021-09-03T14:51:28Z")
Hi All, I was creating a watch in watcher which looks as below { "trigger" : { "schedule" : { "interval" : "3h" } } }, "input": { "search": { "request": { "search\_type": "query\_the… --- ## [Filebeats/logstash XML parsing](https://discuss.elastic.co/t/filebeats-logstash-xml-parsing/283249)
**Author:** [@Rabin\_Bhattacharya](https://discuss.elastic.co/u/Rabin_Bhattacharya)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 2:06pm UTC](https://discuss.elastic.co/t/filebeats-logstash-xml-parsing/283249 "2021-09-03T14:06:39Z")
Hi, I am using ELK stack+filebeats in our project and the application log is integration with filebeats. Filebeat is reading the application log. The log is successfully read by logstash. My question is how to parse the… --- ## [Where are Elastic Agent logs parsed](https://discuss.elastic.co/t/where-are-elastic-agent-logs-parsed/283173)
**Author:** [@sideyourspirit](https://discuss.elastic.co/u/sideyourspirit)\ **Replies:** 3\ **Last updated:** [September 3, 2021, 2:05pm UTC](https://discuss.elastic.co/t/where-are-elastic-agent-logs-parsed/283173 "2021-09-03T14:05:57Z")
Hello, I've got a question about elastic agent. Are the logs collected by the agent parsed on the host right away or is it done in elasticsearch ingest node? --- ## [Aggregation for Individual Query in should Array](https://discuss.elastic.co/t/aggregation-for-individual-query-in-should-array/283251)
**Author:** [@Manisekhar\_techo](https://discuss.elastic.co/u/Manisekhar_techo)\ **Replies:** 3\ **Last updated:** [September 3, 2021, 2:05pm UTC](https://discuss.elastic.co/t/aggregation-for-individual-query-in-should-array/283251 "2021-09-03T14:05:40Z")
Hi All, is there a way for me to perform the aggregation for all the Matching documents inside the bool should array individually. { "query": { "bool": { "should": \[ { … --- ## [Reinstall System Integration Assets](https://discuss.elastic.co/t/reinstall-system-integration-assets/283140)
**Author:** [@Alessandro\_Berto](https://discuss.elastic.co/u/Alessandro_Berto)\ **Replies:** 1\ **Last updated:** [September 3, 2021, 1:21pm UTC](https://discuss.elastic.co/t/reinstall-system-integration-assets/283140 "2021-09-03T13:21:56Z")
Hello, I had to delete some Dashboards that were not functioning properly, unfortunately I discovered way too late that System integrations cannot be reinstalled. Is there a way to only reinstall the assets (more speci… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1252) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1254)