# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1173 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1174 --- ## [Timestamp not being overwriten (ECS)](https://discuss.elastic.co/t/timestamp-not-being-overwriten-ecs/288544)
**Author:** [@vcaleffi](https://discuss.elastic.co/u/vcaleffi)\ **Replies:** 2\ **Last updated:** [November 6, 2021, 6:41pm UTC](https://discuss.elastic.co/t/timestamp-not-being-overwriten-ecs/288544 "2021-11-06T18:41:03Z")
Hello, I'm very new to filebeat and I'm trying to import logs created by my application (PHP - symfony). The application uses monolog and ECS formatter to write to a log file. Monolog configuration: monolog: handl… --- ## [Serval indices were created with different timestamp upon using new winlogbeat template](https://discuss.elastic.co/t/serval-indices-were-created-with-different-timestamp-upon-using-new-winlogbeat-template/288539)
**Author:** [@Alaxwora](https://discuss.elastic.co/u/Alaxwora)\ **Replies:** 1\ **Last updated:** [November 6, 2021, 4:06pm UTC](https://discuss.elastic.co/t/serval-indices-were-created-with-different-timestamp-upon-using-new-winlogbeat-template/288539 "2021-11-06T16:06:50Z")
Hello, I noted serval indices were created upon using my new winlogbeat template, and I already set ILM to rollover after 30 days or upon index size reaches 10 Giga but it seems that I made a mistake. the following scr… --- ## [How to send logs from different filebeat to logstash](https://discuss.elastic.co/t/how-to-send-logs-from-different-filebeat-to-logstash/288305)
**Author:** [@root\_linux](https://discuss.elastic.co/u/root_linux)\ **Replies:** 2\ **Last updated:** [November 6, 2021, 2:27pm UTC](https://discuss.elastic.co/t/how-to-send-logs-from-different-filebeat-to-logstash/288305 "2021-11-06T14:27:05Z")
I have configured Elasticsearch, kibana and logstash on same server. I want to send logs from multiple filebeat clients to logstash with different index name. How can I configure it? --- ## [I can not send logs to elastic cloud](https://discuss.elastic.co/t/i-can-not-send-logs-to-elastic-cloud/288411)
**Author:** [@Hugo\_Pacini](https://discuss.elastic.co/u/Hugo_Pacini)\ **Replies:** 5\ **Last updated:** [November 6, 2021, 2:16pm UTC](https://discuss.elastic.co/t/i-can-not-send-logs-to-elastic-cloud/288411 "2021-11-06T14:16:55Z")
I'm trying to send logs from Serilogs to Elastic Cloud but doesn't work and I can not see errors. Can I send logs to test using Postman? The URL is https://my-custom-domain.es.eastus2.azure.elastic-cloud.com:9243 var u… --- ## [Ingestion performance dropping by 10x daily](https://discuss.elastic.co/t/ingestion-performance-dropping-by-10x-daily/288537)
**Author:** [@TanguyB](https://discuss.elastic.co/u/TanguyB)\ **Replies:** 3\ **Last updated:** [November 6, 2021, 11:19am UTC](https://discuss.elastic.co/t/ingestion-performance-dropping-by-10x-daily/288537 "2021-11-06T11:19:46Z")
Hello, :wave: We are using Fluentbit to ingest Kubernetes cluster logs into Elasticsearch. Not sure what I am setting wrong, but every day, as the index grows, ingestion is worsening from 550k op/s to 40k op/s... Inge… --- ## [Large shard size](https://discuss.elastic.co/t/large-shard-size/288458)
**Author:** [@caseydm](https://discuss.elastic.co/u/caseydm)\ **Replies:** 3\ **Last updated:** [November 6, 2021, 4:57am UTC](https://discuss.elastic.co/t/large-shard-size/288458 "2021-11-06T04:57:56Z")
I am setting up a new Elasticsearch cluster that will hold around 300 million records in a single index to start. The documents are academic paper data. Right now I have two nodes, so I set my shards as 2 primary and 1 r… --- ## [Java Msgs in Log](https://discuss.elastic.co/t/java-msgs-in-log/287967)
**Author:** [@SteveP](https://discuss.elastic.co/u/SteveP)\ **Replies:** 8\ **Last updated:** [November 6, 2021, 2:13am UTC](https://discuss.elastic.co/t/java-msgs-in-log/287967 "2021-11-06T02:13:16Z")
Hello all, I have installed Elasticsearch for the first time and I am seeing msgs that a terminally deprecated method in java.lang.system has been called. Should I be concerned with this, even though it looks like Elast… --- ## [Fleet server setup documentation incorrect when using quickstart + (elastic with TLS)](https://discuss.elastic.co/t/fleet-server-setup-documentation-incorrect-when-using-quickstart-elastic-with-tls/288526)
**Author:** [@tfriesen](https://discuss.elastic.co/u/tfriesen)\ **Replies:** 0\ **Last updated:** [November 5, 2021, 8:38pm UTC](https://discuss.elastic.co/t/fleet-server-setup-documentation-incorrect-when-using-quickstart-elastic-with-tls/288526 "2021-11-05T20:38:48Z")
The documentation for setting up a fleet server, both the online guide and via the 'Add a fleet server', don't account for a situation where Elasticsearch is configured with TLS, but you want to use Quickstart for settin… --- ## [Python Client - unable to remove "read\_only" block](https://discuss.elastic.co/t/python-client-unable-to-remove-read-only-block/288032)
**Author:** [@philhagen](https://discuss.elastic.co/u/philhagen)\ **Replies:** 9\ **Last updated:** [November 5, 2021, 8:29pm UTC](https://discuss.elastic.co/t/python-client-unable-to-remove-read-only-block/288032 "2021-11-05T20:29:18Z")
When using the official Python client, I can't seem to find a way to change an index to normal "read-write" status once it's been marked "read\_only". This function works fine to go from read-write to read\_only, but I wa… --- ## [Build a Query based off the result of another query](https://discuss.elastic.co/t/build-a-query-based-off-the-result-of-another-query/288174)
**Author:** [@Wonjun\_Cho](https://discuss.elastic.co/u/Wonjun_Cho)\ **Replies:** 2\ **Last updated:** [November 5, 2021, 7:48pm UTC](https://discuss.elastic.co/t/build-a-query-based-off-the-result-of-another-query/288174 "2021-11-05T19:48:51Z")
Hi guys, I would like to know if it is possible to make a query based off the result of another query. For example, there are indices for school and students. School { "settings": { ... }, "mappings": { "propertie… --- ## [Some cgroup metrics are missing from output](https://discuss.elastic.co/t/some-cgroup-metrics-are-missing-from-output/288524)
**Author:** [@vPilot](https://discuss.elastic.co/u/vPilot)\ **Replies:** 0\ **Last updated:** [November 5, 2021, 7:44pm UTC](https://discuss.elastic.co/t/some-cgroup-metrics-are-missing-from-output/288524 "2021-11-05T19:44:30Z")
I've got several systems that I have recently started collecting data on via Metricbeat, and am running into an issue on several of them. Particularly, I have a couple of systems that are not reporting back system.proces… --- ## [JDBC Input plugin installation](https://discuss.elastic.co/t/jdbc-input-plugin-installation/288510)
**Author:** [@SANDRA\_ROY\_ARICATT](https://discuss.elastic.co/u/SANDRA_ROY_ARICATT)\ **Replies:** 1\ **Last updated:** [November 5, 2021, 7:00pm UTC](https://discuss.elastic.co/t/jdbc-input-plugin-installation/288510 "2021-11-05T19:00:06Z")
I have the new Mac M1 Silicon. I need to get data from MySQL workbench into Elasticsearch. I came to know that it can be achieved using Logstash, which in turn uses JDBC input plugin for getting data from database into… --- ## [How do I add a filter to an Index Threshold alerting rule and alert per server?](https://discuss.elastic.co/t/how-do-i-add-a-filter-to-an-index-threshold-alerting-rule-and-alert-per-server/288519)
**Author:** [@nsouth](https://discuss.elastic.co/u/nsouth)\ **Replies:** 0\ **Last updated:** [November 5, 2021, 6:51pm UTC](https://discuss.elastic.co/t/how-do-i-add-a-filter-to-an-index-threshold-alerting-rule-and-alert-per-server/288519 "2021-11-05T18:51:41Z")
I want to be alerted when our logs indicate that the average duration of commands is over 1 second. However, I cannot figure out how to do the following: Filter the query to only log events which have duration informa… --- ## [Missing Beats data in Kibana Stack monitoring interface](https://discuss.elastic.co/t/missing-beats-data-in-kibana-stack-monitoring-interface/288514)
**Author:** [@Fangy](https://discuss.elastic.co/u/Fangy)\ **Replies:** 1\ **Last updated:** [November 5, 2021, 6:31pm UTC](https://discuss.elastic.co/t/missing-beats-data-in-kibana-stack-monitoring-interface/288514 "2021-11-05T18:31:32Z")
Hello. I'm trying to monitor beat agents on a brand new cluster. Metricbeat is configured to send data to a separate one-node monitoring cluster. Elastic nodes and Kibana monitoring data are displayed correctly, but th… --- ## [Painless script: can not write type \[class java.math.BigDecimal\]](https://discuss.elastic.co/t/painless-script-can-not-write-type-class-java-math-bigdecimal/288517)
**Author:** [@catelling](https://discuss.elastic.co/u/catelling)\ **Replies:** 0\ **Last updated:** [November 5, 2021, 6:30pm UTC](https://discuss.elastic.co/t/painless-script-can-not-write-type-class-java-math-bigdecimal/288517 "2021-11-05T18:30:43Z")
I am facing this exception on my discover page when trying to view my logs. Previously, I had a field involving BigDecimal and worked fine. this was the script: if(!(doc\[‘obyts.keyword’\].size()==0)) { String b\_str… --- ## [Dealing with invalid json](https://discuss.elastic.co/t/dealing-with-invalid-json/288179)
**Author:** [@caseydm](https://discuss.elastic.co/u/caseydm)\ **Replies:** 6\ **Last updated:** [November 5, 2021, 6:01pm UTC](https://discuss.elastic.co/t/dealing-with-invalid-json/288179 "2021-11-05T18:01:32Z")
I'm trying to import some json data that is in a column in redshift. Some of the records have a field that includes invalid json escape sequences, such as this: "work\_title": "The Discrete and Semi-continuous Fr\\'echet … --- ## [Logstash group time series by decade](https://discuss.elastic.co/t/logstash-group-time-series-by-decade/288462)
**Author:** [@caseydm](https://discuss.elastic.co/u/caseydm)\ **Replies:** 4\ **Last updated:** [November 5, 2021, 5:48pm UTC](https://discuss.elastic.co/t/logstash-group-time-series-by-decade/288462 "2021-11-05T17:48:03Z")
I am ingesting academic articles and splitting the index by year: output { elasticsearch { hosts =\> \["${ES\_HOST\_PROD}"\] index =\> "works-%{year}" user =\> "${ES\_USER\_PROD}" password =\> … --- ## [Continuous Transform query, does it update?](https://discuss.elastic.co/t/continuous-transform-query-does-it-update/288443)
**Author:** [@iambatman](https://discuss.elastic.co/u/iambatman)\ **Replies:** 4\ **Last updated:** [November 5, 2021, 5:26pm UTC](https://discuss.elastic.co/t/continuous-transform-query-does-it-update/288443 "2021-11-05T17:26:23Z")
If you use a query when creating a continuous transform will it only ever work on the query subset that is created when the transform is started? { "bool": { "should": \[ { "match": { "state… --- ## [Recurring Date - What is the best patterns?](https://discuss.elastic.co/t/recurring-date-what-is-the-best-patterns/288515)
**Author:** [@Remy\_Letient](https://discuss.elastic.co/u/Remy_Letient)\ **Replies:** 0\ **Last updated:** [November 5, 2021, 5:19pm UTC](https://discuss.elastic.co/t/recurring-date-what-is-the-best-patterns/288515 "2021-11-05T17:19:11Z")
Hi, I need to represent "recurring event" like "each month" with the goal to create a calendar view of them. The events can be unlimited in time. What is the best way to represent this kind of data to permit query like… --- ## [Add tag to filebeat modules](https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314)
**Author:** [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Replies:** 5\ **Last updated:** [November 5, 2021, 4:53pm UTC](https://discuss.elastic.co/t/add-tag-to-filebeat-modules/288314 "2021-11-05T16:53:46Z")
hello! I am trying to send the logs with the filebeat mysql module to my logstash and it already obtains the pipeline and does the parsing correctly. How can I differentiate both inputs ( Slowlog and errror ) to apply… --- ## [Elastic Agent Crashes Status Code: 0](https://discuss.elastic.co/t/elastic-agent-crashes-status-code-0/287264)
**Author:** [@Ramguru\_Nageswaran](https://discuss.elastic.co/u/Ramguru_Nageswaran)\ **Replies:** 1\ **Last updated:** [November 5, 2021, 4:42pm UTC](https://discuss.elastic.co/t/elastic-agent-crashes-status-code-0/287264 "2021-11-05T16:42:09Z")
09:32:28.601 elastic\_agent \[elastic\_agent\]\[error\] Could not communicate with fleet-server Checking API will retry, error: status code: 0, fleet-server returned an error: , message: Unknown resource. 09:41:24.952 elastic\_… --- ## [How to make a nested term query](https://discuss.elastic.co/t/how-to-make-a-nested-term-query/288506)
**Author:** [@Araf\_Shihab](https://discuss.elastic.co/u/Araf_Shihab)\ **Replies:** 0\ **Last updated:** [November 5, 2021, 2:58pm UTC](https://discuss.elastic.co/t/how-to-make-a-nested-term-query/288506 "2021-11-05T14:58:59Z")
Hey Elasticsearch community. I have a nested query as such { "size": 20, "\_source": \["tags"\], "sort": \[ { "@timestamp": { "order": "desc" } } \], "query": { "nested" : { "path" : "tags… --- ## [\[Filebeat\] Override Module Ingest Pipeline at Runtime](https://discuss.elastic.co/t/filebeat-override-module-ingest-pipeline-at-runtime/288432)
**Author:** [@DougR](https://discuss.elastic.co/u/DougR)\ **Replies:** 6\ **Last updated:** [November 5, 2021, 2:50pm UTC](https://discuss.elastic.co/t/filebeat-override-module-ingest-pipeline-at-runtime/288432 "2021-11-05T14:50:43Z")
TL;DR Can I override the ingest pipeline for a filebeat module at runtime from modules.d/module.yml? The Rest of the Story I use the apache module with filebeat to capture logs on the Apache servers in our environment. … --- ## [Canvas and Dashboard show different values](https://discuss.elastic.co/t/canvas-and-dashboard-show-different-values/288418)
**Author:** [@r.pollak](https://discuss.elastic.co/u/r.pollak)\ **Replies:** 8\ **Last updated:** [November 5, 2021, 2:38pm UTC](https://discuss.elastic.co/t/canvas-and-dashboard-show-different-values/288418 "2021-11-05T14:38:24Z")
Hi together, I have created a metric visualization on a dashboard. The metric just counts the number of records (1.436) see picture below. I have created a metric visualization on a canvas (some index) and the metri… --- ## [Elasticsearch Java Client (elasticsearch-java) ready for production use?](https://discuss.elastic.co/t/elasticsearch-java-client-elasticsearch-java-ready-for-production-use/284759)
**Author:** [@roytmana](https://discuss.elastic.co/u/roytmana)\ **Replies:** 5\ **Last updated:** [November 5, 2021, 2:16pm UTC](https://discuss.elastic.co/t/elasticsearch-java-client-elasticsearch-java-ready-for-production-use/284759 "2021-11-05T14:16:40Z")
Hello, We have been using low level rest client avoiding high level rest client because its dependencies on elastic server. I noticed that the high level client is being deprecated in favor of a new elasticsearch-java c… --- ## [Docker replication does not work - error - Master is not discoverable](https://discuss.elastic.co/t/docker-replication-does-not-work-error-master-is-not-discoverable/288455)
**Author:** [@jitendragupta2](https://discuss.elastic.co/u/jitendragupta2)\ **Replies:** 3\ **Last updated:** [November 5, 2021, 1:59pm UTC](https://discuss.elastic.co/t/docker-replication-does-not-work-error-master-is-not-discoverable/288455 "2021-11-05T13:59:27Z")
Docker replication does not work - error - Master is not discoverable Detailed Error - "master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eli… --- ## [Logstash is reloading the same documents again and again from elasticsearch index](https://discuss.elastic.co/t/logstash-is-reloading-the-same-documents-again-and-again-from-elasticsearch-index/288315)
**Author:** [@Husnain](https://discuss.elastic.co/u/Husnain)\ **Replies:** 5\ **Last updated:** [November 5, 2021, 1:47pm UTC](https://discuss.elastic.co/t/logstash-is-reloading-the-same-documents-again-and-again-from-elasticsearch-index/288315 "2021-11-05T13:47:20Z")
Hii.I am pulling documents from one Elasticsearch index into another Elasticsearch index using logstash.I am running logstash as a service by configuring "logstash.yml" file.The index from which I am pulling data,that in… --- ## [Query unique documents after my query](https://discuss.elastic.co/t/query-unique-documents-after-my-query/288496)
**Author:** [@Araf\_Shihab](https://discuss.elastic.co/u/Araf_Shihab)\ **Replies:** 0\ **Last updated:** [November 5, 2021, 1:41pm UTC](https://discuss.elastic.co/t/query-unique-documents-after-my-query/288496 "2021-11-05T13:41:02Z")
Hey stack overflow I have an Elasticsearch document that looks like below. I'm only interested in the 'tags' key. "\_index": "graph\_20211025t0909", "\_type": "\_doc", "\_id": "E12201A5-CC50-… --- ## [Upgrading Kibana from 7.14.2 to 7.15.X results in error TypeError: Cannot read property 'schema' of undefined](https://discuss.elastic.co/t/upgrading-kibana-from-7-14-2-to-7-15-x-results-in-error-typeerror-cannot-read-property-schema-of-undefined/288417)
**Author:** [@kking-biometrica](https://discuss.elastic.co/u/kking-biometrica)\ **Replies:** 7\ **Last updated:** [November 5, 2021, 1:38pm UTC](https://discuss.elastic.co/t/upgrading-kibana-from-7-14-2-to-7-15-x-results-in-error-typeerror-cannot-read-property-schema-of-undefined/288417 "2021-11-05T13:38:46Z")
I'm trying to upgrade Kibana on a Linux platform. When upgrading, I am performing the following actions: Download and extract linux tar.gz file to a temp location Stop Kibana Backup existing Kibana install: sudo cp -R… --- ## [How to aggregate multiple events coming from different logs with slight different timestamp, when the only field is timestamp to combine those?](https://discuss.elastic.co/t/how-to-aggregate-multiple-events-coming-from-different-logs-with-slight-different-timestamp-when-the-only-field-is-timestamp-to-combine-those/287848)
**Author:** [@Indigo\_Star](https://discuss.elastic.co/u/Indigo_Star)\ **Replies:** 10\ **Last updated:** [November 5, 2021, 1:05pm UTC](https://discuss.elastic.co/t/how-to-aggregate-multiple-events-coming-from-different-logs-with-slight-different-timestamp-when-the-only-field-is-timestamp-to-combine-those/287848 "2021-11-05T13:05:14Z")
Hi, I have this situation: "How to aggregate multiple events coming from different logs with slight different timestamp, when the only common field could be the timesamp? I need to combine all data points into one dat… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1172) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1174)