# \#none **URL:** https://discuss.elastic.co/tag/none.md?page=1351 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1352 --- ## [I have to combine certain aggregation and query to extract data from elastic search . Can someone help with this following query](https://discuss.elastic.co/t/i-have-to-combine-certain-aggregation-and-query-to-extract-data-from-elastic-search-can-someone-help-with-this-following-query/239746)
**Author:** [@charvi23](https://discuss.elastic.co/u/charvi23)\ **Replies:** 7\ **Last updated:** [July 3, 2020, 10:01am UTC](https://discuss.elastic.co/t/i-have-to-combine-certain-aggregation-and-query-to-extract-data-from-elastic-search-can-someone-help-with-this-following-query/239746 "2020-07-03T10:01:38Z")
From a set of documents do the following: Extract documents which match the following criteria: a. date\_field \> given certain date and b. second\_date\_field \< given certain date From the above extracted documents,… --- ## [Continuous tranforms stop indexing after initial indexing](https://discuss.elastic.co/t/continuous-tranforms-stop-indexing-after-initial-indexing/238473)
**Author:** [@Germain\_Pavot](https://discuss.elastic.co/u/Germain_Pavot)\ **Replies:** 3\ **Last updated:** [July 3, 2020, 9:39am UTC](https://discuss.elastic.co/t/continuous-tranforms-stop-indexing-after-initial-indexing/238473 "2020-07-03T09:39:29Z")
Hy guys, I try to setup a continuous transforms strategy on a new cluster for optimize queries and index size with an 1h aggregation. The stack is setup like that : 1 index input-\[0\_9\]{6} with a write alias "input".… --- ## [Visualization of controls kibana 7.2](https://discuss.elastic.co/t/visualization-of-controls-kibana-7-2/239741)
**Author:** [@sanjaygunda1](https://discuss.elastic.co/u/sanjaygunda1)\ **Replies:** 0\ **Last updated:** [July 3, 2020, 6:58am UTC](https://discuss.elastic.co/t/visualization-of-controls-kibana-7-2/239741 "2020-07-03T06:58:00Z")
Hi all, i am trying to filter data using controls in visualization tool. example:- if i have an set of emails who is having multiple cars set emails \[a@a.com,b@b.com,c@c.com\] set cars \[ford,Honda,BMW\] a@a.com is … --- ## [Store dense vectors of size more than 2048](https://discuss.elastic.co/t/store-dense-vectors-of-size-more-than-2048/239438)
**Author:** [@techytushar](https://discuss.elastic.co/u/techytushar)\ **Replies:** 4\ **Last updated:** [July 3, 2020, 9:26am UTC](https://discuss.elastic.co/t/store-dense-vectors-of-size-more-than-2048/239438 "2020-07-03T09:26:55Z")
I am trying to build a semantic text search using ElasticSearch. For this I have decided to use the cosine similarity between the embeddings of the query paragraph and the embeddings of the search paragraphs. The proble… --- ## [How to do Logstash JDBC Output with SSH](https://discuss.elastic.co/t/how-to-do-logstash-jdbc-output-with-ssh/239760)
**Author:** [@JF-GIAMMARI](https://discuss.elastic.co/u/JF-GIAMMARI)\ **Replies:** 0\ **Last updated:** [July 3, 2020, 9:05am UTC](https://discuss.elastic.co/t/how-to-do-logstash-jdbc-output-with-ssh/239760 "2020-07-03T09:05:49Z")
Hi, I'm using JDBC as output to store logs in a mySQL database, but the mySQL server is not on the same server, so I need to launch an SSH connection before making my command, otherwise the connection fails. I've been … --- ## [Cluster.routing.allocation.awareness.attributes doesn't seem to work](https://discuss.elastic.co/t/cluster-routing-allocation-awareness-attributes-doesnt-seem-to-work/238548)
**Author:** [@mannoj87](https://discuss.elastic.co/u/mannoj87)\ **Replies:** 9\ **Last updated:** [July 3, 2020, 8:20am UTC](https://discuss.elastic.co/t/cluster-routing-allocation-awareness-attributes-doesnt-seem-to-work/238548 "2020-07-03T08:20:23Z")
Hi Team, cluster.routing.allocation.awareness.attributes: 34 And I have 2 VMs with this same value. prd-es107.com and prd-es108.com. Both has 34. I have almost 86 nodes in my cluster. Below is the shard allocation … --- ## [Get ip source TCP plugin](https://discuss.elastic.co/t/get-ip-source-tcp-plugin/239743)
**Author:** [@Thomas74](https://discuss.elastic.co/u/Thomas74)\ **Replies:** 1\ **Last updated:** [July 3, 2020, 8:05am UTC](https://discuss.elastic.co/t/get-ip-source-tcp-plugin/239743 "2020-07-03T08:05:39Z")
Hi, I wonder if it's possible to get the ip source of a sender when using tcp input in logstash. Is there a way to get this ip even if I don't use http requests ? Best regards, Thomas R. --- ## [Unable to ship data to logstash 6.8 via Python](https://discuss.elastic.co/t/unable-to-ship-data-to-logstash-6-8-via-python/239677)
**Author:** [@Swetha\_B](https://discuss.elastic.co/u/Swetha_B)\ **Replies:** 10\ **Last updated:** [July 3, 2020, 4:32am UTC](https://discuss.elastic.co/t/unable-to-ship-data-to-logstash-6-8-via-python/239677 "2020-07-03T04:32:34Z")
We have a Python script to collect data and ship to Logstash. Please find the python script and logstash config below. import socket import json import logging from datetime import datetime import sys print("starting t… --- ## [Design indexes with big data](https://discuss.elastic.co/t/design-indexes-with-big-data/239235)
**Author:** [@Ankur\_Mahajan](https://discuss.elastic.co/u/Ankur_Mahajan)\ **Replies:** 15\ **Last updated:** [July 3, 2020, 7:23am UTC](https://discuss.elastic.co/t/design-indexes-with-big-data/239235 "2020-07-03T07:23:15Z")
I am creating an elastic cluster with 2.5TB dataset. Questions: What is the best way to design shards for best performance? The only limit is 2.1Billion documents per shard (it's Lucene core limitation), 20 - 40GB… --- ## [Crypto/rsa: verification error - using heartbeat](https://discuss.elastic.co/t/crypto-rsa-verification-error-using-heartbeat/239427)
**Author:** [@Peter\_Steenbergen](https://discuss.elastic.co/u/Peter_Steenbergen)\ **Replies:** 6\ **Last updated:** [July 3, 2020, 7:21am UTC](https://discuss.elastic.co/t/crypto-rsa-verification-error-using-heartbeat/239427 "2020-07-03T07:21:31Z")
Hi, I am getting a crypto/rsa error: verification error with heartbeat version 7.4.2. The website is using LetsEncrypt and the SSL was verified with ssllabs with a grade of A. What can be the issue here? - type: http… --- ## [Filebeat encrypted communication to Logstash](https://discuss.elastic.co/t/filebeat-encrypted-communication-to-logstash/239739)
**Author:** [@d.silwon](https://discuss.elastic.co/u/d.silwon)\ **Replies:** 2\ **Last updated:** [July 3, 2020, 7:10am UTC](https://discuss.elastic.co/t/filebeat-encrypted-communication-to-logstash/239739 "2020-07-03T07:10:50Z")
Dears, I have to enable SSL/TLS in Filebeat configuration to encrypt communication to Logstash. Right now my ELK cluster use SSL/TLS configuration between nodes with self-signed certificates. Logstash is started on ea… --- ## [How to reduce collection time](https://discuss.elastic.co/t/how-to-reduce-collection-time/239730)
**Author:** [@jang](https://discuss.elastic.co/u/jang)\ **Replies:** 3\ **Last updated:** [July 3, 2020, 6:32am UTC](https://discuss.elastic.co/t/how-to-reduce-collection-time/239730 "2020-07-03T06:32:48Z")
hello, I'm using elk. After a day of testing, there was a problem. The difference between collection time and log generation time is widening. For example, the collection time and log generation time were exactly th… --- ## [Keep index template and ILM when using logstash](https://discuss.elastic.co/t/keep-index-template-and-ilm-when-using-logstash/239736)
**Author:** [@sref](https://discuss.elastic.co/u/sref)\ **Replies:** 0\ **Last updated:** [July 3, 2020, 6:31am UTC](https://discuss.elastic.co/t/keep-index-template-and-ilm-when-using-logstash/239736 "2020-07-03T06:31:51Z")
Im looking for guidance on how to keep ILM working, when sending logs via Logstash to a custom index. I do the classic setup with Winlogbeat-\>Logstash-\>ES. Logstash output config: output { if "CUSTOMTAG" in \[tags\] {… --- ## [Should I be worried for this warn message when installing Elastic?](https://discuss.elastic.co/t/should-i-be-worried-for-this-warn-message-when-installing-elastic/239734)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 0\ **Last updated:** [July 3, 2020, 6:15am UTC](https://discuss.elastic.co/t/should-i-be-worried-for-this-warn-message-when-installing-elastic/239734 "2020-07-03T06:15:39Z")
after a successful instalation, I get this line warning: elasticsearch-7.8.0-x86\_64.rpm: Header V4 RSA/SHA512 Signature, key ID XXXXXX: NOKEY\` I have instaled elastic downloading the file from elastic website, and then… --- ## [Help with converting String to Integer in xml parsing](https://discuss.elastic.co/t/help-with-converting-string-to-integer-in-xml-parsing/239672)
**Author:** [@jyotsna.ganji](https://discuss.elastic.co/u/jyotsna.ganji)\ **Replies:** 1\ **Last updated:** [July 3, 2020, 5:56am UTC](https://discuss.elastic.co/t/help-with-converting-string-to-integer-in-xml-parsing/239672 "2020-07-03T05:56:46Z")
I am parsing a nested xml using the below conf file and it parses individual tag elements perfectly. But when I try to convert string elements to Integer using XPath, it forms arrays of values. input{ http\_poller{ … --- ## [JVM error in elasticsearch](https://discuss.elastic.co/t/jvm-error-in-elasticsearch/239701)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 6\ **Last updated:** [July 3, 2020, 5:27am UTC](https://discuss.elastic.co/t/jvm-error-in-elasticsearch/239701 "2020-07-03T05:27:37Z")
HI Team, when i try to star the elasticsearch service it throwing below error \> Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit\_memory(0x00002b76c9990000, 33661845504, 0) failed; error='Cannot allocate … --- ## [How to query data from elasticsearch in kibana plugin in version 7.7.1?](https://discuss.elastic.co/t/how-to-query-data-from-elasticsearch-in-kibana-plugin-in-version-7-7-1/239727)
**Author:** [@K\_Chaitanya](https://discuss.elastic.co/u/K_Chaitanya)\ **Replies:** 0\ **Last updated:** [July 3, 2020, 4:43am UTC](https://discuss.elastic.co/t/how-to-query-data-from-elasticsearch-in-kibana-plugin-in-version-7-7-1/239727 "2020-07-03T04:43:00Z")
Hi, I'm trying to build Kibana plugin and I am using basic free authentication in kibana. I need to query GET /\_security/user/\_has\_privileges to get the privileges of the current logged in user api further I need to per… --- ## [Dead Letter - cannot write event to DLQ: reached maxQueueSize of 2147483648](https://discuss.elastic.co/t/dead-letter-cannot-write-event-to-dlq-reached-maxqueuesize-of-2147483648/239100)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 6\ **Last updated:** [July 3, 2020, 4:24am UTC](https://discuss.elastic.co/t/dead-letter-cannot-write-event-to-dlq-reached-maxqueuesize-of-2147483648/239100 "2020-07-03T04:24:55Z")
Hi Team, I facing syncing issue in Elasticsearch and when i check my logstash log and i observer the below error "cannot write event to DLQ: reached maxQueueSize of 2147483648". whether its causing the syncing issue … --- ## [【filebeat】SQSのキュー削除に関して](https://discuss.elastic.co/t/filebeat-sqs/239580)
**Author:** [@harue](https://discuss.elastic.co/u/harue)\ **Replies:** 2\ **Last updated:** [July 3, 2020, 2:15am UTC](https://discuss.elastic.co/t/filebeat-sqs/239580 "2020-07-03T02:15:12Z")
お世話になります。 filebeatのS3 inputに関して質問させて頂きます。 ①filebeat⇔S3(SQS)の処理にて失敗した場合、 SQSのキューにあるメッセージは残されたままでしょうか？ filebeatからキューのメッセージを削除（DeleteMessageを実施）するのでしょうか？ ② ①の回答がデータを削除するの場合、 処理失敗時にキューを削除するかしないかのパラメータはありますでしょうか？ お手数です… --- ## [Optimize logstash configuration](https://discuss.elastic.co/t/optimize-logstash-configuration/239574)
**Author:** [@ohoroyoi](https://discuss.elastic.co/u/ohoroyoi)\ **Replies:** 5\ **Last updated:** [July 3, 2020, 12:32am UTC](https://discuss.elastic.co/t/optimize-logstash-configuration/239574 "2020-07-03T00:32:00Z")
Hello. I'm syncing 4 tables of geo data from postgres to elasticsearch using logstash. I am updating the data to be changed using the doc\_as\_upsert option. One table has more than 700,000 rows. Not many, but sometimes … --- ## [Dissect json array of strings](https://discuss.elastic.co/t/dissect-json-array-of-strings/239711)
**Author:** [@theflakes](https://discuss.elastic.co/u/theflakes)\ **Replies:** 1\ **Last updated:** [July 2, 2020, 9:54pm UTC](https://discuss.elastic.co/t/dissect-json-array-of-strings/239711 "2020-07-02T21:54:52Z")
I have a Json array of strings. I want to find the one entry that starts with “xx” and create a new field with it as the value. The text after “xx” can vary. Not sure how to use the dissect directive on an array of stri… --- ## [Logstash Runner Error](https://discuss.elastic.co/t/logstash-runner-error/239704)
**Author:** [@Kevin\_f](https://discuss.elastic.co/u/Kevin_f)\ **Replies:** 1\ **Last updated:** [July 2, 2020, 9:45pm UTC](https://discuss.elastic.co/t/logstash-runner-error/239704 "2020-07-02T21:45:18Z")
Hi All, I am seeing this error in my logstash logs and not sure how to resolve. Some people say to install xpack plugin. My logstash is on version 6.8 and I believe it already comes with xpack by default with this vers… --- ## [Can I change an URL template with JSON input field](https://discuss.elastic.co/t/can-i-change-an-url-template-with-json-input-field/239671)
**Author:** [@Goishin](https://discuss.elastic.co/u/Goishin)\ **Replies:** 1\ **Last updated:** [July 2, 2020, 8:18pm UTC](https://discuss.elastic.co/t/can-i-change-an-url-template-with-json-input-field/239671 "2020-07-02T20:18:16Z")
I'm using Kibana to report my test pass results. I have now added the test devices to my dashboard. The problem is that some of my test devices have generic names, but may be on different servers. So, if I want to see th… --- ## [Pinned query limit of 100 configurable?](https://discuss.elastic.co/t/pinned-query-limit-of-100-configurable/239667)
**Author:** [@Karl\_Kevilus](https://discuss.elastic.co/u/Karl_Kevilus)\ **Replies:** 12\ **Last updated:** [July 2, 2020, 8:11pm UTC](https://discuss.elastic.co/t/pinned-query-limit-of-100-configurable/239667 "2020-07-02T20:11:23Z")
How can I configure the maximum # of items returned in a 'pinned query'? I'm getting the following error: "caused\_by": { "type": "x\_content\_parse\_exception", "reason": "Failed to build \[pinned\] after l… --- ## [Signals](https://discuss.elastic.co/t/signals/239666)
**Author:** [@xennn](https://discuss.elastic.co/u/xennn)\ **Replies:** 1\ **Last updated:** [July 2, 2020, 7:49pm UTC](https://discuss.elastic.co/t/signals/239666 "2020-07-02T19:49:03Z")
Hello, can i modify the signal count? I get a lot Signals for 1 Event. Thats Spam my Overview --- ## [Capturing Start and Stop Times in An Aggregated Data Record?](https://discuss.elastic.co/t/capturing-start-and-stop-times-in-an-aggregated-data-record/239668)
**Author:** [@redapplesonly](https://discuss.elastic.co/u/redapplesonly)\ **Replies:** 4\ **Last updated:** [July 2, 2020, 7:37pm UTC](https://discuss.elastic.co/t/capturing-start-and-stop-times-in-an-aggregated-data-record/239668 "2020-07-02T19:37:54Z")
Hi Logstash Superheroes, Not long ago, this forum helped me with the syntax for an Aggregate Filter Plugin. I got it working, but now I need to append Start and Stop timestamps to the aggregated records. To explain… … --- ## [Beat - logstash - logserver](https://discuss.elastic.co/t/beat-logstash-logserver/239692)
**Author:** [@slawek](https://discuss.elastic.co/u/slawek)\ **Replies:** 4\ **Last updated:** [July 2, 2020, 7:26pm UTC](https://discuss.elastic.co/t/beat-logstash-logserver/239692 "2020-07-02T19:26:47Z")
Hello, Can I configure logstash as a receiver from winlogbeat and send these logs in CEF format to another log server? If yes. How should I do that? Regards --- ## [Kibana ciphers not bringing up the port 5601](https://discuss.elastic.co/t/kibana-ciphers-not-bringing-up-the-port-5601/239542)
**Author:** [@funlearning321](https://discuss.elastic.co/u/funlearning321)\ **Replies:** 2\ **Last updated:** [July 2, 2020, 6:16pm UTC](https://discuss.elastic.co/t/kibana-ciphers-not-bringing-up-the-port-5601/239542 "2020-07-02T18:16:28Z")
Hi, I have added the below configuration in kibana.yml but however, when the service is restarted, kibana is throwing error. server.ssl.cipherSuites: "ECDHE-RSA-AES128-GCM-SHA256,ECDHE-ECDSA-AES128-GCM-SHA256,ECDHE-R… --- ## [Can I put chirpstack’s data into Kibana?](https://discuss.elastic.co/t/can-i-put-chirpstack-s-data-into-kibana/239683)
**Author:** [@miuiuiuiuiu](https://discuss.elastic.co/u/miuiuiuiuiu)\ **Replies:** 1\ **Last updated:** [July 2, 2020, 5:45pm UTC](https://discuss.elastic.co/t/can-i-put-chirpstack-s-data-into-kibana/239683 "2020-07-02T17:45:54Z")
I am using lora wan to transmit data, the data transmits to chirpstack and i want to use Kibana to visualize and analyze my data. But turns out Kibana does not support and chirpstack do not support EC just wondering … --- ## [How to inject variables to input plugin](https://discuss.elastic.co/t/how-to-inject-variables-to-input-plugin/239597)
**Author:** [@ohoroyoi](https://discuss.elastic.co/u/ohoroyoi)\ **Replies:** 1\ **Last updated:** [July 2, 2020, 5:32pm UTC](https://discuss.elastic.co/t/how-to-inject-variables-to-input-plugin/239597 "2020-07-02T17:32:19Z")
according to, this answer https://discuss.elastic.co/t/optimize-logstash-configuration/239574/3 I realized I have to compare lastest row date and last run date. but the problem is I don't know that. I can find last ru… [Previous page](https://discuss.elastic.co/tag/none.md?no_tags=true&page=1350) [Next page](https://discuss.elastic.co/tag/none.md?no_tags=true&page=1352)