# Latest **URL:** https://discuss.elastic.co/latest.md?page=1576 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1577 --- ## [What is filebeat read offset? Why does it change both up and down?](https://discuss.elastic.co/t/what-is-filebeat-read-offset-why-does-it-change-both-up-and-down/278528)
**Author:** [@Matthew\_Field](https://discuss.elastic.co/u/Matthew_Field)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 11:19am UTC](https://discuss.elastic.co/t/what-is-filebeat-read-offset-why-does-it-change-both-up-and-down/278528 "2021-07-13T11:19:23Z")
I am running filebeat 7.6.2 in docker sending files to logstash. The output performance seems to be limited to around 150 events per second, but the file is being written to around 400 events per second. I am looking at… --- ## [Windowsアプリケーションログをlogstashで解析する方法](https://discuss.elastic.co/t/windows-logstash/278527)
**Author:** [@t-nakata](https://discuss.elastic.co/u/t-nakata)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 11:18am UTC](https://discuss.elastic.co/t/windows-logstash/278527 "2021-07-13T11:18:10Z")
以下のようなWindowsアプリケーションログをlogstashで1つのドキュメントとして取り込みたいです。 ログの名前: Application ソース: Microsoft-Windows-Security-SPP 日付: 2021/06/28 12:53:18 イベント ID: 16394 タスクのカテゴリ: なし レベル: 情報… --- ## [Elasticsearch update by query - update ONLY the last document](https://discuss.elastic.co/t/elasticsearch-update-by-query-update-only-the-last-document/278506)
**Author:** [@Savas12.01](https://discuss.elastic.co/u/Savas12.01)\ **Replies:** 2\ **Last updated:** [July 13, 2021, 11:08am UTC](https://discuss.elastic.co/t/elasticsearch-update-by-query-update-only-the-last-document/278506 "2021-07-13T11:08:06Z")
Hello community, I am new to the elasticsearch and I am trying to get the last document from my index and update its list of object. I tried several queries e.g: POST localhost:9200/test-index/\_update\_by\_query { "si… --- ## [Logstash "Duplicate field 'match'](https://discuss.elastic.co/t/logstash-duplicate-field-match/278524)
**Author:** [@sahere37](https://discuss.elastic.co/u/sahere37)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 10:42am UTC](https://discuss.elastic.co/t/logstash-duplicate-field-match/278524 "2021-07-13T10:42:39Z")
Hi all I am using following input in logstash conf file to checking matching of a field. input { elasticsearch { hosts =\> \["http://localhost:9200/"\] index =\> "log\*" query =\> '{ "query": { "bool" : { … --- ## [Watcher is throwing timeout\_exception](https://discuss.elastic.co/t/watcher-is-throwing-timeout-exception/278514)
**Author:** [@JF\_LTQ](https://discuss.elastic.co/u/JF_LTQ)\ **Replies:** 2\ **Last updated:** [July 13, 2021, 10:41am UTC](https://discuss.elastic.co/t/watcher-is-throwing-timeout-exception/278514 "2021-07-13T10:41:49Z")
Hello, We've been having some issues with watchers throwing timeout exceptions: "result": { "execution\_time": "2021-07-13T07:45:00.144Z", "execution\_duration": 60004, "input": { "type": "search", … --- ## [How to prevent duplicate and has null value documents with fingerprint](https://discuss.elastic.co/t/how-to-prevent-duplicate-and-has-null-value-documents-with-fingerprint/278150)
**Author:** [@Busra\_Duygu](https://discuss.elastic.co/u/Busra_Duygu)\ **Replies:** 16\ **Last updated:** [July 13, 2021, 10:33am UTC](https://discuss.elastic.co/t/how-to-prevent-duplicate-and-has-null-value-documents-with-fingerprint/278150 "2021-07-13T10:33:20Z")
my csv file =\> name,surname,age,email,phone Harry,Potter,18,NULL,NULL Harry,Potter,NULL,harrypotter@gmail.com,+955555555 Harry,Potter,NULL,harrypotter@gmail.com,NULL Harry,Potter,NULL,NULL,+955555555 When I want to dete… --- ## [Add own snapshot repository in elastic.co](https://discuss.elastic.co/t/add-own-snapshot-repository-in-elastic-co/278519)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 1\ **Last updated:** [July 13, 2021, 10:28am UTC](https://discuss.elastic.co/t/add-own-snapshot-repository-in-elastic-co/278519 "2021-07-13T10:28:16Z")
I am using a managed/hosted ES cluster in elastic.co I want to register a GCS repository using my own bucket. I suppose at some point in time throughout the registration process, I will have to pass the json document h… --- ## [Collapsed results participating in sorting](https://discuss.elastic.co/t/collapsed-results-participating-in-sorting/278512)
**Author:** [@Markos\_Fragkakis](https://discuss.elastic.co/u/Markos_Fragkakis)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 9:28am UTC](https://discuss.elastic.co/t/collapsed-results-participating-in-sorting/278512 "2021-07-13T09:28:32Z")
I am using Elasticsearch and I want to group our results by a specific field, returning only the most recent document per group. When scoring and sorting, I want the documents I am not returning (the ones that are older)… --- ## [Elasticsearch 7.13](https://discuss.elastic.co/t/elasticsearch-7-13/275410)
**Author:** [@Krunal](https://discuss.elastic.co/u/Krunal)\ **Replies:** 25\ **Last updated:** [July 13, 2021, 9:03am UTC](https://discuss.elastic.co/t/elasticsearch-7-13/275410 "2021-07-13T09:03:06Z")
\`rypoint\[1835\]: Exception in thread "main" java.lang.RuntimeException: starting java failed with \[1\] rypoint\[1835\]: output: rypoint\[1835\]: \[0.029s\]\[error\]\[logging\] Error opening log file '/var/log/elasticsearch/gc.log': … --- ## [Using certificates from Certificate Authority instead of Self-Signed Certificates](https://discuss.elastic.co/t/using-certificates-from-certificate-authority-instead-of-self-signed-certificates/278480)
**Author:** [@siyey14049](https://discuss.elastic.co/u/siyey14049)\ **Replies:** 1\ **Last updated:** [July 13, 2021, 8:44am UTC](https://discuss.elastic.co/t/using-certificates-from-certificate-authority-instead-of-self-signed-certificates/278480 "2021-07-13T08:44:04Z")
Hi there, Currently we are using self-signed certificates generated by a basic Docker container using elasticsearch-certgen. What would be the process to move to using certificates from a Certificate Authority (whether… --- ## [serverMap Module data exceptions](https://discuss.elastic.co/t/servermap-module-data-exceptions/278012)
**Author:** [@tttoad](https://discuss.elastic.co/u/tttoad)\ **Replies:** 6\ **Last updated:** [July 13, 2021, 8:35am UTC](https://discuss.elastic.co/t/servermap-module-data-exceptions/278012 "2021-07-13T08:35:35Z")
APM Server version 7.13.2 elasticSearch: 7.13.2 kibana: 7.13.2 The service node of the serverMap module on the Kibana Apm interface cannot be linked up. 1.run apm,es,kibana: docker network create -d bridge my-jaeger-n… --- ## [Elasticsearch Azure Snapshot Plugin](https://discuss.elastic.co/t/elasticsearch-azure-snapshot-plugin/278504)
**Author:** [@robinhood\_lko](https://discuss.elastic.co/u/robinhood_lko)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 8:26am UTC](https://discuss.elastic.co/t/elasticsearch-azure-snapshot-plugin/278504 "2021-07-13T08:26:43Z")
Hi Team, We are facing one issue while configuring Snapshots using Azure repository plugin. The plugin is not able to connect to Azure container. Out Network team wants to test the connectivity calls from container. Pl… --- ## [Opentery reported the data from Trace, Metrics data at Prometheus. How to connect?](https://discuss.elastic.co/t/opentery-reported-the-data-from-trace-metrics-data-at-prometheus-how-to-connect/278502)
**Author:** [@lisiyu](https://discuss.elastic.co/u/lisiyu)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 8:13am UTC](https://discuss.elastic.co/t/opentery-reported-the-data-from-trace-metrics-data-at-prometheus-how-to-connect/278502 "2021-07-13T08:13:32Z")
I deployed the Prometheus、apm-server、elastic、kibana 、metricbeat in k8s. I want to show apm metrics from prometheus data. what should I do? Kibana version: 7.13.0 Elasticsearch version: 7.13.0 APM Server version: 7.1… --- ## [Keep older data in kibana when using a python script to upload data](https://discuss.elastic.co/t/keep-older-data-in-kibana-when-using-a-python-script-to-upload-data/278408)
**Author:** [@Adrienb02](https://discuss.elastic.co/u/Adrienb02)\ **Replies:** 2\ **Last updated:** [July 13, 2021, 8:06am UTC](https://discuss.elastic.co/t/keep-older-data-in-kibana-when-using-a-python-script-to-upload-data/278408 "2021-07-13T08:06:40Z")
Hello there ! Sorry if my question seems stupid, it is my first time touching ELK and also JSON files, I'm doing an internship related to those topics and I need a bit of help. So I have a python script that open a jso… --- ## [Reindex from remote very slow](https://discuss.elastic.co/t/reindex-from-remote-very-slow/278497)
**Author:** [@bacquelmore](https://discuss.elastic.co/u/bacquelmore)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 7:31am UTC](https://discuss.elastic.co/t/reindex-from-remote-very-slow/278497 "2021-07-13T07:31:52Z")
Hi, I am trying to reindex from remote an index of ~40Gb, single shard, single node, to an index of 8 shards, 3 nodes. The operation is very slow (now a couple of days) and seems to only progress during specific hours o… --- ## [Elasticsearch nodes automatically disconnected](https://discuss.elastic.co/t/elasticsearch-nodes-automatically-disconnected/278266)
**Author:** [@Siva\_Karan](https://discuss.elastic.co/u/Siva_Karan)\ **Replies:** 1\ **Last updated:** [July 13, 2021, 7:39am UTC](https://discuss.elastic.co/t/elasticsearch-nodes-automatically-disconnected/278266 "2021-07-13T07:39:34Z")
Hi Team, we have an 3 nodes(data cum master) cluster,it has 487 primary indices and one replica for the same. ES version : 7.3.2 but sometimes nodes get disconnected and joined later.it happens 2 to 3 times in a month… --- ## [Fetch and store data from an index every hour](https://discuss.elastic.co/t/fetch-and-store-data-from-an-index-every-hour/278332)
**Author:** [@mrsac7](https://discuss.elastic.co/u/mrsac7)\ **Replies:** 2\ **Last updated:** [July 13, 2021, 7:11am UTC](https://discuss.elastic.co/t/fetch-and-store-data-from-an-index-every-hour/278332 "2021-07-13T07:11:01Z")
Dear community, I want to create a script that can - Fetch data from a given source index Process the data and store into a destination index The script will be scheduled to run once every hour. The data contains t… --- ## [Questions about query latency](https://discuss.elastic.co/t/questions-about-query-latency/278483)
**Author:** [@yeziblo](https://discuss.elastic.co/u/yeziblo)\ **Replies:** 4\ **Last updated:** [July 13, 2021, 6:13am UTC](https://discuss.elastic.co/t/questions-about-query-latency/278483 "2021-07-13T06:13:38Z")
Hi everyone. I have a strange problem. We have an ES cluster with three nodes, each node's heap memory is 8 GB and the es version is 7.8. Now I have two indexes: test@20210711125233 and test@20210712075544. There mapp… --- ## [Elasticsearch GC timeout on data node](https://discuss.elastic.co/t/elasticsearch-gc-timeout-on-data-node/277943)
**Author:** [@atakjudin](https://discuss.elastic.co/u/atakjudin)\ **Replies:** 1\ **Last updated:** [July 13, 2021, 4:50am UTC](https://discuss.elastic.co/t/elasticsearch-gc-timeout-on-data-node/277943 "2021-07-13T04:50:54Z")
Hello, I try to find out an answer to my problem but I don't find a solution in existing threads I have read. I have a cluster of 7 elasticsearch nodes (2 master, 4 ingest, all data). The latest node was added two week… --- ## [The problems of memory of ES7.12](https://discuss.elastic.co/t/the-problems-of-memory-of-es7-12/278030)
**Author:** [@lichuandev](https://discuss.elastic.co/u/lichuandev)\ **Replies:** 1\ **Last updated:** [July 13, 2021, 4:42am UTC](https://discuss.elastic.co/t/the-problems-of-memory-of-es7-12/278030 "2021-07-13T04:42:49Z")
After I upgraded to version 7.13 and used the default configuration, things got better, but I still had to restart it once a week.Version 7.4 used previously did not have this problem.Is there any good solution? Thank y… --- ## [Network Management System with ELK (Like Open NMS etc)](https://discuss.elastic.co/t/network-management-system-with-elk-like-open-nms-etc/278485)
**Author:** [@mohammadawaisjavaid](https://discuss.elastic.co/u/mohammadawaisjavaid)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 4:41am UTC](https://discuss.elastic.co/t/network-management-system-with-elk-like-open-nms-etc/278485 "2021-07-13T04:41:27Z")
Hi all, I have configured 7.10.0 Stack Management. Now I wished to see the devices attached to my network in Elasticsearch. In this regard I am searching for a integration that helps me visualize the whole network in KI… --- ## [Got 302 code when curl kibana ingress URL](https://discuss.elastic.co/t/got-302-code-when-curl-kibana-ingress-url/278378)
**Author:** [@ls4074](https://discuss.elastic.co/u/ls4074)\ **Replies:** 2\ **Last updated:** [July 13, 2021, 3:33am UTC](https://discuss.elastic.co/t/got-302-code-when-curl-kibana-ingress-url/278378 "2021-07-13T03:33:53Z")
my ELK stack is running in AKS, when I try to curl the Kibana ingress URL, I got 302 code and and I see it is redirected to /spaces/enter. I wonder why and how I can make it to work. curl http:/app/kibana About to con… --- ## [PCF (CloudFoundry) Application rename is not being detected](https://discuss.elastic.co/t/pcf-cloudfoundry-application-rename-is-not-being-detected/278460)
**Author:** [@pausebreathefly](https://discuss.elastic.co/u/pausebreathefly)\ **Replies:** 4\ **Last updated:** [July 13, 2021, 2:02am UTC](https://discuss.elastic.co/t/pcf-cloudfoundry-application-rename-is-not-being-detected/278460 "2021-07-13T02:02:21Z")
Hi, We have configured the below processors in our filebeat application deployed on TAS or PCF but when we rename any application the changes are not being honored. Is there any additional setting that needs to be confi… --- ## [Data Pipeline - Sub traces](https://discuss.elastic.co/t/data-pipeline-sub-traces/277882)
**Author:** [@Koby\_Bass](https://discuss.elastic.co/u/Koby_Bass)\ **Replies:** 3\ **Last updated:** [July 13, 2021, 1:37am UTC](https://discuss.elastic.co/t/data-pipeline-sub-traces/277882 "2021-07-13T01:37:00Z")
Hey! We are integrating elastic APM in our data pipeline, using the latest python APM client. There are several components that analyze an item one after another. Each component (microservice) receives traceparent and… --- ## [Block in initialize'", block in healthcheck!' \[LOGSTASH\]](https://discuss.elastic.co/t/block-in-initialize-block-in-healthcheck-logstash/278432)
**Author:** [@Mesut\_Yilmaz](https://discuss.elastic.co/u/Mesut_Yilmaz)\ **Replies:** 4\ **Last updated:** [July 13, 2021, 12:32am UTC](https://discuss.elastic.co/t/block-in-initialize-block-in-healthcheck-logstash/278432 "2021-07-13T00:32:34Z")
I installed brand new logstash and I got the same error. logstash.conf is working config and systax is ok but I got pipeline error. systemctl start logstash - is running but on logstash-plain.log there are some errors as… --- ## [Search After NEST - ElasticSearch](https://discuss.elastic.co/t/search-after-nest-elasticsearch/278478)
**Author:** [@NoviceESCoder](https://discuss.elastic.co/u/NoviceESCoder)\ **Replies:** 0\ **Last updated:** [July 13, 2021, 12:06am UTC](https://discuss.elastic.co/t/search-after-nest-elasticsearch/278478 "2021-07-13T00:06:20Z")
After realizing that the search api in ElasticSearch will only handle up to 10,000 results, my next option to retrieve all the logs (more than 10,000 results) was through creating a point in time id and passing that id t… --- ## [Where do we define the password for cert generated using elastic certutil in yml file?](https://discuss.elastic.co/t/where-do-we-define-the-password-for-cert-generated-using-elastic-certutil-in-yml-file/278448)
**Author:** [@devashishsingh](https://discuss.elastic.co/u/devashishsingh)\ **Replies:** 1\ **Last updated:** [July 12, 2021, 11:55pm UTC](https://discuss.elastic.co/t/where-do-we-define-the-password-for-cert-generated-using-elastic-certutil-in-yml-file/278448 "2021-07-12T23:55:18Z")
Hello, In order to enable SSL security, I did follow steps and generated the certificate first. While generating it asked for a password(to create one for the cert protection) Please enter the desired output file \[elas… --- ## [Data nodes are connected to the master but not to each other during upgrade](https://discuss.elastic.co/t/data-nodes-are-connected-to-the-master-but-not-to-each-other-during-upgrade/278472)
**Author:** [@163satish](https://discuss.elastic.co/u/163satish)\ **Replies:** 0\ **Last updated:** [July 12, 2021, 10:35pm UTC](https://discuss.elastic.co/t/data-nodes-are-connected-to-the-master-but-not-to-each-other-during-upgrade/278472 "2021-07-12T22:35:13Z")
I am doing ES upgrade from 6.8.2 to 7.9.3 and seeing issues with 2 data nodes. Shard movements are going from "index" to "init" stage and logs show that the data nodes are not connected to each other. On the new node I … --- ## [JSON input for url in Data Table](https://discuss.elastic.co/t/json-input-for-url-in-data-table/278399)
**Author:** [@catalins](https://discuss.elastic.co/u/catalins)\ **Replies:** 2\ **Last updated:** [July 12, 2021, 8:44pm UTC](https://discuss.elastic.co/t/json-input-for-url-in-data-table/278399 "2021-07-12T20:44:53Z")
Hello I'm having some problems at this part with JSON language.... I want to make that "Split Row" customized as a clickable URL Can you guys help me please ? --- ## [Is there an API that will return how many logs are there](https://discuss.elastic.co/t/is-there-an-api-that-will-return-how-many-logs-are-there/278453)
**Author:** [@NoviceESCoder](https://discuss.elastic.co/u/NoviceESCoder)\ **Replies:** 2\ **Last updated:** [July 12, 2021, 8:18pm UTC](https://discuss.elastic.co/t/is-there-an-api-that-will-return-how-many-logs-are-there/278453 "2021-07-12T20:18:08Z")
Is there an api that returns how many results/logs there are? or like a built in feature with NEST that will tell you how many results there is like i.e. count? Because I have a NEST query search that will do the followi… [Previous page](https://discuss.elastic.co/latest.md?page=1575) [Next page](https://discuss.elastic.co/latest.md?page=1577)