# Latest **URL:** https://discuss.elastic.co/latest.md?page=1527 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1528 --- ## [Use logstash to synchronize json files to elactissearch without data](https://discuss.elastic.co/t/use-logstash-to-synchronize-json-files-to-elactissearch-without-data/281138)
**Author:** [@zhyp](https://discuss.elastic.co/u/zhyp)\ **Replies:** 3\ **Last updated:** [August 13, 2021, 2:53pm UTC](https://discuss.elastic.co/t/use-logstash-to-synchronize-json-files-to-elactissearch-without-data/281138 "2021-08-13T14:53:30Z")
There is no data, even when the file is changed, the console does not output changes. This is my conf file, test.conf input { file { path =\> "D:\\logstash\\test\\demo.log" codec =\> "json" } } output { elasticsearc… --- ## [Enroll and start the Elastic Agent - Windows](https://discuss.elastic.co/t/enroll-and-start-the-elastic-agent-windows/281275)
**Author:** [@alenraz05](https://discuss.elastic.co/u/alenraz05)\ **Replies:** 6\ **Last updated:** [August 13, 2021, 2:05pm UTC](https://discuss.elastic.co/t/enroll-and-start-the-elastic-agent-windows/281275 "2021-08-13T14:05:42Z")
Hi I can not find the elastic-agent.exe in the winlogbeat-7.14.0-windows-x86\_64 folder --- ## [Elasticsearch query and aggregation](https://discuss.elastic.co/t/elasticsearch-query-and-aggregation/281356)
**Author:** [@masakazu.ohtsuka](https://discuss.elastic.co/u/masakazu.ohtsuka)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 12:55pm UTC](https://discuss.elastic.co/t/elasticsearch-query-and-aggregation/281356 "2021-08-13T12:55:01Z")
Elasticsearch responds with buckets that should be filtered out by the query. What's wrong? The request is: curl -kv --max-time 60 -H "Content-Type: application/json" "http://elasticsearch.example.com/jp-2021.0\*/\_searc… --- ## [ERROR - Kibana Canvas email Reporting via Watcher](https://discuss.elastic.co/t/error-kibana-canvas-email-reporting-via-watcher/281234)
**Author:** [@keshara](https://discuss.elastic.co/u/keshara)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 5:18pm UTC](https://discuss.elastic.co/t/error-kibana-canvas-email-reporting-via-watcher/281234 "2021-08-12T17:18:30Z")
Hi, My use case is to automate email report via Watcher from a kibana Canvas report. ELK Stack =\> 7.14.0 I have already build a Canvas report. And from this same Canvas report I copied automatically generated Watcher … --- ## [Характеристики виртуальной машины для logstash](https://discuss.elastic.co/t/logstash/281348)
**Author:** [@111498](https://discuss.elastic.co/u/111498)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 12:12pm UTC](https://discuss.elastic.co/t/logstash/281348 "2021-08-13T12:12:38Z")
Нужна виртуальная машина для logstash. Также будет использоваться плагин lumberjack. Сколько CPU, RAM, и диска надо для этого, если учитывать что: EPS=1000, Bandwidth usage=400kb/s PPS=15 ? --- ## [Logstash import file from url](https://discuss.elastic.co/t/logstash-import-file-from-url/281067)
**Author:** [@ChinigamiHunter](https://discuss.elastic.co/u/ChinigamiHunter)\ **Replies:** 8\ **Last updated:** [August 13, 2021, 12:10pm UTC](https://discuss.elastic.co/t/logstash-import-file-from-url/281067 "2021-08-13T12:10:22Z")
I am new to Elasticsearch and Logstash, i just need to parse json data from url. Json link is : https://www.circl.lu/doc/misp/feed-osint/0165e5d7-51e6-4c2e-a382-1dd1e706f7bb.json this is my logstash conf file: input {… --- ## [Current timestamp in Kibana Dev Tools](https://discuss.elastic.co/t/current-timestamp-in-kibana-dev-tools/281247)
**Author:** [@Luca\_Frost](https://discuss.elastic.co/u/Luca_Frost)\ **Replies:** 1\ **Last updated:** [August 13, 2021, 11:43am UTC](https://discuss.elastic.co/t/current-timestamp-in-kibana-dev-tools/281247 "2021-08-13T11:43:13Z")
Hi All, Apologies if this question has already been answered. Is it possible to create a new document using the Kibana Dev Console and set the timestamp field to now, much in the same way as datetime.now() in python. P… --- ## [Cannot install osquery-manager](https://discuss.elastic.co/t/cannot-install-osquery-manager/281311)
**Author:** [@cheapsupps](https://discuss.elastic.co/u/cheapsupps)\ **Replies:** 2\ **Last updated:** [August 13, 2021, 11:36am UTC](https://discuss.elastic.co/t/cannot-install-osquery-manager/281311 "2021-08-13T11:36:39Z")
Hi, I have deployed elasticsearch + kibana + package registry in the air-gap environment. However, I am having problem deploying the osquery-manager to my agent. Below is the error at my agent. {"log.level":"info","@ti… --- ## [Logstash 7.14.0 "invalid byte sequence in UTF-8" in logstash.javapipeline](https://discuss.elastic.co/t/logstash-7-14-0-invalid-byte-sequence-in-utf-8-in-logstash-javapipeline/280976)
**Author:** [@cknz](https://discuss.elastic.co/u/cknz)\ **Replies:** 4\ **Last updated:** [August 13, 2021, 10:56am UTC](https://discuss.elastic.co/t/logstash-7-14-0-invalid-byte-sequence-in-utf-8-in-logstash-javapipeline/280976 "2021-08-13T10:56:25Z")
Came into work today and noticed that our Logstash pipeline (on all two hosts) had halted with the following error in the logs. Aug 11 00:22:40 its-elk-p03.uod.otago.ac.nz logstash\[20536\]: \[2021-08-11T00:22:40,141\]\[ERRO… --- ## [Plugin error (tanium)](https://discuss.elastic.co/t/plugin-error-tanium/281300)
**Author:** [@travlest](https://discuss.elastic.co/u/travlest)\ **Replies:** 2\ **Last updated:** [August 13, 2021, 9:14am UTC](https://discuss.elastic.co/t/plugin-error-tanium/281300 "2021-08-13T09:14:51Z")
Hi, before this, I have a problem with connecting ELK stack to Tanium. But it has already been solving with an index pattern on kibana. the server that I've tried is a dummy server with little data. now, I am trying t… --- ## [System integration navigation visuals use all the wrong URLs / IDs?](https://discuss.elastic.co/t/system-integration-navigation-visuals-use-all-the-wrong-urls-ids/281180)
**Author:** [@c1jt](https://discuss.elastic.co/u/c1jt)\ **Replies:** 3\ **Last updated:** [August 13, 2021, 9:39am UTC](https://discuss.elastic.co/t/system-integration-navigation-visuals-use-all-the-wrong-urls-ids/281180 "2021-08-13T09:39:15Z")
Hello! I have linked a "System" integration to a Windows hosted agent policy, this looks to have instaleld some dashboards akin to what Winlogbeat does. However, the navigation visuals ("Dashboard links \[Windows System … --- ## [ML multi metric split filed only have keywork](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888)
**Author:** [@quyennguyen](https://discuss.elastic.co/u/quyennguyen)\ **Replies:** 12\ **Last updated:** [August 13, 2021, 9:38am UTC](https://discuss.elastic.co/t/ml-multi-metric-split-filed-only-have-keywork/280888 "2021-08-13T09:38:37Z")
I'm just a beginner of ML so I didn't know why this happened in my ELK. I want to create new job in multi metric so I following those step Pick "Fail login" search index Then pick multi metric -\> time range "use ful… --- ## [Partial bulk insert into elasticsearch using rest API](https://discuss.elastic.co/t/partial-bulk-insert-into-elasticsearch-using-rest-api/281295)
**Author:** [@santosh\_varma](https://discuss.elastic.co/u/santosh_varma)\ **Replies:** 6\ **Last updated:** [August 13, 2021, 9:38am UTC](https://discuss.elastic.co/t/partial-bulk-insert-into-elasticsearch-using-rest-api/281295 "2021-08-13T09:38:32Z")
Hi All, I am trying to bulk insert data into elastic search batch wise(say 10 records at once). Out of these I am having one record which is invalid. The rest client is rolling back all the 10 records. I want to save th… --- ## [How to ingest complex JSON objects with tables in ES?](https://discuss.elastic.co/t/how-to-ingest-complex-json-objects-with-tables-in-es/281327)
**Author:** [@AbelAlejandro](https://discuss.elastic.co/u/AbelAlejandro)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 9:24am UTC](https://discuss.elastic.co/t/how-to-ingest-complex-json-objects-with-tables-in-es/281327 "2021-08-13T09:24:13Z")
I'm trying to define a format for my logs and these must include lists of objects. When I upload my log file to ElasticSearch, it doesn't know how to parse the lists and I don't see how to configure this. Example NDJSON… --- ## [\[Elastic Agent 7.14\] \[Kubernetes provider\] How to dedot labels and annotations](https://discuss.elastic.co/t/elastic-agent-7-14-kubernetes-provider-how-to-dedot-labels-and-annotations/281230)
**Author:** [@mohsen0](https://discuss.elastic.co/u/mohsen0)\ **Replies:** 2\ **Last updated:** [August 13, 2021, 9:05am UTC](https://discuss.elastic.co/t/elastic-agent-7-14-kubernetes-provider-how-to-dedot-labels-and-annotations/281230 "2021-08-13T09:05:36Z")
I deploy the stand-alone elastic agent using the following (Run Elastic Agent standalone on Kubernetes | Fleet User Guide \[7.14\] | Elastic). if the pods are labeled or annotated by labels with . in their name. it will … --- ## [Question about failed authentication logging](https://discuss.elastic.co/t/question-about-failed-authentication-logging/281319)
**Author:** [@borna\_talebi](https://discuss.elastic.co/u/borna_talebi)\ **Replies:** 1\ **Last updated:** [August 13, 2021, 8:53am UTC](https://discuss.elastic.co/t/question-about-failed-authentication-logging/281319 "2021-08-13T08:53:53Z")
Hi I want to enable logging when a user failed to log in via Kibana web UI. But I'm sure where should I enable it. The documents were a bit confusing so I'm not sure should I make changes to elasticsearch.yml or kibana… --- ## [Unable to choose Elasticsearch Query as Alert Type](https://discuss.elastic.co/t/unable-to-choose-elasticsearch-query-as-alert-type/281132)
**Author:** [@andrewgodwinB](https://discuss.elastic.co/u/andrewgodwinB)\ **Replies:** 4\ **Last updated:** [August 13, 2021, 8:40am UTC](https://discuss.elastic.co/t/unable-to-choose-elasticsearch-query-as-alert-type/281132 "2021-08-13T08:40:24Z")
Hi, I'm trying to grant a role access to use Elasticsearch Query as an alert type. The users with that role are unable to see the Elasticsearch Query when they go to create a new alert in Kibana. I can see it but I hav… --- ## [Problems connecting logstash-input-mongodb](https://discuss.elastic.co/t/problems-connecting-logstash-input-mongodb/281186)
**Author:** [@LourdesCrespo](https://discuss.elastic.co/u/LourdesCrespo)\ **Replies:** 0\ **Last updated:** [August 12, 2021, 10:18am UTC](https://discuss.elastic.co/t/problems-connecting-logstash-input-mongodb/281186 "2021-08-12T10:18:43Z")
In the project it is intended that the logs obtained from logstash are sent to mongodb. Docker-compose is used for this and although the containers are connected, the logs are not sent. The part of docker-compose that co… --- ## [Logstash multiline codec ignores last line of log file](https://discuss.elastic.co/t/logstash-multiline-codec-ignores-last-line-of-log-file/281318)
**Author:** [@san2597](https://discuss.elastic.co/u/san2597)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 8:19am UTC](https://discuss.elastic.co/t/logstash-multiline-codec-ignores-last-line-of-log-file/281318 "2021-08-13T08:19:51Z")
My logstash config has a mix of multiline and single line events that are being parsed correctly, reading a log file. I'm using multiline codec as follows: input { file { ..... ..... codec =\> multiline {… --- ## [Elastic search commands](https://discuss.elastic.co/t/elastic-search-commands/281139)
**Author:** [@chris.aw](https://discuss.elastic.co/u/chris.aw)\ **Replies:** 5\ **Last updated:** [August 13, 2021, 8:18am UTC](https://discuss.elastic.co/t/elastic-search-commands/281139 "2021-08-13T08:18:56Z")
Hi, is there a way to allow search commands like: "/help, /categories" etc etc? Thank you for the help in advance --- ## [Pushing Logs form Azure Storage to Elastic via Logstash](https://discuss.elastic.co/t/pushing-logs-form-azure-storage-to-elastic-via-logstash/281312)
**Author:** [@Kosodrom](https://discuss.elastic.co/u/Kosodrom)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 7:55am UTC](https://discuss.elastic.co/t/pushing-logs-form-azure-storage-to-elastic-via-logstash/281312 "2021-08-13T07:55:51Z")
Hi there, currently I am evaluating the possibility to collect logs, which reside in azure blob storage to Logstash. I came across this plug-in here: azure-diagnostics-tools/Logstash/logstash-input-azureblob at master … --- ## [Объединение 3 анализаторов в 1](https://discuss.elastic.co/t/3-1/280764)
**Author:** [@ForeverRED66](https://discuss.elastic.co/u/ForeverRED66)\ **Replies:** 12\ **Last updated:** [August 13, 2021, 7:44am UTC](https://discuss.elastic.co/t/3-1/280764 "2021-08-13T07:44:55Z")
Добрый день, есть цель объединить 3 анализатора в 1 для будущих индексов, я с этим не сталкивался в процессе работы да и с эластиком в принципе то же редко, по идее у меня цель 3 анализатора. Может кто поможет с этим де… --- ## [Unpredictable work of terminate\_after](https://discuss.elastic.co/t/unpredictable-work-of-terminate-after/281306)
**Author:** [@romanenkoss](https://discuss.elastic.co/u/romanenkoss)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 7:20am UTC](https://discuss.elastic.co/t/unpredictable-work-of-terminate-after/281306 "2021-08-13T07:20:28Z")
Hi team, I have an index with 4 billion docs, 200 shards, I want to use terminate\_after=500000 if I correctly understood request must terminate after 100kk docs, when the count of docs is more than 100kk it's working fi… --- ## [More than 1 shard not getting assigned to a node](https://discuss.elastic.co/t/more-than-1-shard-not-getting-assigned-to-a-node/279797)
**Author:** [@srivatsa](https://discuss.elastic.co/u/srivatsa)\ **Replies:** 2\ **Last updated:** [August 13, 2021, 7:11am UTC](https://discuss.elastic.co/t/more-than-1-shard-not-getting-assigned-to-a-node/279797 "2021-08-13T07:11:36Z")
ES version - 6.8 Setup - 3 nodes with default rack\_id I am trying to logically separate out clusters inside the single cluster by using the rack\_id awareness attribute, the problem I am facing is that only one shard is… --- ## [서브쿼리같은 elastice query 구문 만들기](https://discuss.elastic.co/t/elastice-query/281000)
**Author:** [@ruddms](https://discuss.elastic.co/u/ruddms)\ **Replies:** 2\ **Last updated:** [August 13, 2021, 6:44am UTC](https://discuss.elastic.co/t/elastice-query/281000 "2021-08-13T06:44:18Z")
안녕하세요 document id값으로 해당 document의 embedVector 필드 (type은 dense\_vector)의 값을 알아내고 알아낸 embedVector 필드 값으로 script\_score 사용해서 index에 존재하는 모든 document의 embedVector 필드와 cosineSimilarity를 계산하고 싶은데 이런 과정을 하나의 쿼리로 작성할 수 있을까요? 코드… --- ## [Update documents and field mapping](https://discuss.elastic.co/t/update-documents-and-field-mapping/280277)
**Author:** [@Dheeraj\_Gupta](https://discuss.elastic.co/u/Dheeraj_Gupta)\ **Replies:** 3\ **Last updated:** [August 3, 2021, 10:11am UTC](https://discuss.elastic.co/t/update-documents-and-field-mapping/280277 "2021-08-03T10:11:26Z")
I am using elasticsearch for some batch processing. The general gist of steps involved are: 1 Fetche each document from index A 2 Search index B by filtering data using attributes of document returned in step 1. 3. P… --- ## [Elasticsearch engineer certification - new topics](https://discuss.elastic.co/t/elasticsearch-engineer-certification-new-topics/281276)
**Author:** [@jerrymj](https://discuss.elastic.co/u/jerrymj)\ **Replies:** 1\ **Last updated:** [August 13, 2021, 5:33am UTC](https://discuss.elastic.co/t/elasticsearch-engineer-certification-new-topics/281276 "2021-08-13T05:33:55Z")
How can I find / learn topics that are marked in asterisk ( \* ) ? I am looking for free options to learn these topics. --- ## [Мониторинг очереди Postfix](https://discuss.elastic.co/t/postfix/281280)
**Author:** [@KaPBaJIOJI](https://discuss.elastic.co/u/KaPBaJIOJI)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 3:35am UTC](https://discuss.elastic.co/t/postfix/281280 "2021-08-13T03:35:38Z")
Хочу мониторить очередь писем, для этого пишу в файл следующую инфу "data", "hostname", "count" Wed Aug 11 11:18:44 smtp01 72 Wed Aug 11 11:43:45 smtp01 71 Wed Aug 11 13:49:38 smtp01 72 Wed Aug 11 13:51:09 smtp01 7… --- ## [Logstash does not consume Kafka](https://discuss.elastic.co/t/logstash-does-not-consume-kafka/281278)
**Author:** [@ktpktr0](https://discuss.elastic.co/u/ktpktr0)\ **Replies:** 0\ **Last updated:** [August 13, 2021, 2:47am UTC](https://discuss.elastic.co/t/logstash-does-not-consume-kafka/281278 "2021-08-13T02:47:32Z")
I created 3 logstash + 3 Kafka, and the initial size of each topic is 1. After a log grows greatly, the Kafka disk reaches 100%. After the disk is expanded, logstash does not consume Kafka. Only when Kafka is restarted c… --- ## [Attribute Based Security](https://discuss.elastic.co/t/attribute-based-security/280592)
**Author:** [@workenehg](https://discuss.elastic.co/u/workenehg)\ **Replies:** 4\ **Last updated:** [August 13, 2021, 1:46am UTC](https://discuss.elastic.co/t/attribute-based-security/280592 "2021-08-13T01:46:42Z")
I have a use case where I need to restrict access to documents in an Elasticsearch index based on the user executing the query ACL and the security attributes on the document. i was told to use Attribute based Security … [Previous page](https://discuss.elastic.co/latest.md?page=1526) [Next page](https://discuss.elastic.co/latest.md?page=1528)