# Latest **URL:** https://discuss.elastic.co/latest.md?page=1245 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1246 --- ## [Add fields with same format but different value problem with grok logstash](https://discuss.elastic.co/t/add-fields-with-same-format-but-different-value-problem-with-grok-logstash/298043)
**Author:** [@alex\_vermex](https://discuss.elastic.co/u/alex_vermex)\ **Replies:** 1\ **Last updated:** [February 23, 2022, 7:51pm UTC](https://discuss.elastic.co/t/add-fields-with-same-format-but-different-value-problem-with-grok-logstash/298043 "2022-02-23T19:51:31Z")
Hi, log file example: 2021-01-01 11:15:02 - it is a f.... 2021-01-01 11:15:04 - format is =\> 000000: 00 11 22 a5 12 12 23 2a 55 12 00 22 33 44 | ?1451202000 . . . 00114: 00 11 55 77 9a 11 0 2021-12-23 10:14:22 … --- ## [Connection Error (NodejsClient 8.0)](https://discuss.elastic.co/t/connection-error-nodejsclient-8-0/298028)
**Author:** [@dev\_test](https://discuss.elastic.co/u/dev_test)\ **Replies:** 4\ **Last updated:** [February 24, 2022, 7:35am UTC](https://discuss.elastic.co/t/connection-error-nodejsclient-8-0/298028 "2022-02-24T07:35:36Z")
Hi, I am trying to connect to my Elasticsearch 8.0 cluster through a Javascript client 8.0 and I am having this error ConnectionError: other side closed - Local: 172.31.0.1:60684, Remote: 172.31.0.4:9200 at Sniffin… --- ## [Upgrading/Updating SIEM rules](https://discuss.elastic.co/t/upgrading-updating-siem-rules/298014)
**Author:** [@alaine](https://discuss.elastic.co/u/alaine)\ **Replies:** 2\ **Last updated:** [February 24, 2022, 7:13am UTC](https://discuss.elastic.co/t/upgrading-updating-siem-rules/298014 "2022-02-24T07:13:16Z")
Good Morning, I am in the process of migrating a SOC from Splunk to Elastic and am going through some of the growing pains at the moment. This question is in regards to upgrading/updating SIEM rules as we upgrade our el… --- ## [Logstash won't start pipeline after moving to 7.6.13 from 6.5.x](https://discuss.elastic.co/t/logstash-wont-start-pipeline-after-moving-to-7-6-13-from-6-5-x/298035)
**Author:** [@fribse](https://discuss.elastic.co/u/fribse)\ **Replies:** 2\ **Last updated:** [February 24, 2022, 6:45am UTC](https://discuss.elastic.co/t/logstash-wont-start-pipeline-after-moving-to-7-6-13-from-6-5-x/298035 "2022-02-24T06:45:09Z")
Hi All, I've gotten a problem with my old logstash pipeline. My config looks like this (everything is commented out during debugging): \[root@ALPHA00035 pipeline\]# cat logstash.conf input { beats { … --- ## [Kibana SAML SSO Issue with spaces - "unauthorized to GetAll spaces"](https://discuss.elastic.co/t/kibana-saml-sso-issue-with-spaces-unauthorized-to-getall-spaces/297199)
**Author:** [@techguy20](https://discuss.elastic.co/u/techguy20)\ **Replies:** 3\ **Last updated:** [February 24, 2022, 6:25am UTC](https://discuss.elastic.co/t/kibana-saml-sso-issue-with-spaces-unauthorized-to-getall-spaces/297199 "2022-02-24T06:25:51Z")
Kibana 7.17 - self managed instance We have SAML SSL working well Azure. We have 3 roles defined kibana. user, superuser, restricted\_spaces\_user, Within the Enterprise App, the values are all assigned to the different… --- ## [How to fill area between 2 point with Kibana](https://discuss.elastic.co/t/how-to-fill-area-between-2-point-with-kibana/297883)
**Author:** [@alparslan](https://discuss.elastic.co/u/alparslan)\ **Replies:** 3\ **Last updated:** [February 24, 2022, 6:25am UTC](https://discuss.elastic.co/t/how-to-fill-area-between-2-point-with-kibana/297883 "2022-02-24T06:25:19Z")
Hi, I need to visualize "busy | available" graph for my data. Sample data and sketchy graph here is below. What is best approach to catch something like this graph? I tried something with also TSVB. But i just wonder… --- ## [Filebeat 7.10 aws cloudtrail is not parsing the output as shown in documentation](https://discuss.elastic.co/t/filebeat-7-10-aws-cloudtrail-is-not-parsing-the-output-as-shown-in-documentation/298111)
**Author:** [@Anwar](https://discuss.elastic.co/u/Anwar)\ **Replies:** 1\ **Last updated:** [February 24, 2022, 5:21am UTC](https://discuss.elastic.co/t/filebeat-7-10-aws-cloudtrail-is-not-parsing-the-output-as-shown-in-documentation/298111 "2022-02-24T05:21:19Z")
Hi, I have configured aws clodutrail module using filebeat to push the logs to Elasticsearch. Logs are pushed to the elk. But am facing two issues. Messages are showing in string instead of json If i used processor an… --- ## [Kibana Portal not loading when configured to a 3 node ES cluster](https://discuss.elastic.co/t/kibana-portal-not-loading-when-configured-to-a-3-node-es-cluster/297969)
**Author:** [@Aditya93](https://discuss.elastic.co/u/Aditya93)\ **Replies:** 4\ **Last updated:** [February 24, 2022, 4:13am UTC](https://discuss.elastic.co/t/kibana-portal-not-loading-when-configured-to-a-3-node-es-cluster/297969 "2022-02-24T04:13:21Z")
I've configured a 3 node ES cluster and installed kibana and made it point to the cluster. Both ES an Kibana services are up, but the kibana portal is still not loading. I dont see any errors in ES logs, on error log in… --- ## [Whole data from db not updated in elastic index](https://discuss.elastic.co/t/whole-data-from-db-not-updated-in-elastic-index/298030)
**Author:** [@Akhil\_Chandran](https://discuss.elastic.co/u/Akhil_Chandran)\ **Replies:** 3\ **Last updated:** [February 24, 2022, 3:53am UTC](https://discuss.elastic.co/t/whole-data-from-db-not-updated-in-elastic-index/298030 "2022-02-24T03:53:17Z")
input { jdbc { tags =\> "staff" jdbc\_connection\_string =\> "jdbc:postgresql://${DB\_HOST}:${DB\_PORT}/${DB\_NAME}" jdbc\_user =\> "${DB\_USER}" jdbc\_password =\> "${DB\_PASSWORD}" schedule =\> "\* \* \* \* \*" … --- ## [I want to rebuild the old runtime environment](https://discuss.elastic.co/t/i-want-to-rebuild-the-old-runtime-environment/297829)
**Author:** [@TANIGUCHIK](https://discuss.elastic.co/u/TANIGUCHIK)\ **Replies:** 5\ **Last updated:** [February 24, 2022, 1:49am UTC](https://discuss.elastic.co/t/i-want-to-rebuild-the-old-runtime-environment/297829 "2022-02-24T01:49:20Z")
I want to install an old plugin for 1.5.x to reproduce an old validation environment, but I could not install the plugin from the command line. Are there any pre-built plugin URLs left? Elasticsearch-cloud-aws 2.5.x E… --- ## [Elasticsearch 8.0 RPM Install error when adding new node](https://discuss.elastic.co/t/elasticsearch-8-0-rpm-install-error-when-adding-new-node/297967)
**Author:** [@soarendev](https://discuss.elastic.co/u/soarendev)\ **Replies:** 4\ **Last updated:** [February 24, 2022, 1:46am UTC](https://discuss.elastic.co/t/elasticsearch-8-0-rpm-install-error-when-adding-new-node/297967 "2022-02-24T01:46:02Z")
I'm installing Elasticsearch 8.0 on a new 3-node cluster on Amazon Linux 2. I have Elasticsearch running on the first node. I created a new enrollment key on this node. When I try to enroll the 2nd node I get the follow… --- ## [Metricbeat Custom Module Registration issue](https://discuss.elastic.co/t/metricbeat-custom-module-registration-issue/298101)
**Author:** [@tweet2santosh86](https://discuss.elastic.co/u/tweet2santosh86)\ **Replies:** 0\ **Last updated:** [February 24, 2022, 1:02am UTC](https://discuss.elastic.co/t/metricbeat-custom-module-registration-issue/298101 "2022-02-24T01:02:55Z")
Hello Team, We are trying to create custom metricbeat module by following the link Creating a Metricset | Beats Developer Guide \[master\] | Elastic.. but it seems the new module is not getting registered. We are not get… --- ## [Large amounts of bulk data sent from Logstash to Elasticsearch after using pipelines](https://discuss.elastic.co/t/large-amounts-of-bulk-data-sent-from-logstash-to-elasticsearch-after-using-pipelines/298092)
**Author:** [@userR](https://discuss.elastic.co/u/userR)\ **Replies:** 3\ **Last updated:** [February 24, 2022, 1:18am UTC](https://discuss.elastic.co/t/large-amounts-of-bulk-data-sent-from-logstash-to-elasticsearch-after-using-pipelines/298092 "2022-02-24T01:18:41Z")
I recently changed my configuration from one logstash.config file to pipelines to make it easier. My pipelines.yml (replaced actual names with letters, main re-routes data to correct pipeline based on tag): - pipeline.… --- ## [Design Question - Better to have a summary text field or index each field as keyword and text](https://discuss.elastic.co/t/design-question-better-to-have-a-summary-text-field-or-index-each-field-as-keyword-and-text/297815)
**Author:** [@johnrodey](https://discuss.elastic.co/u/johnrodey)\ **Replies:** 1\ **Last updated:** [February 24, 2022, 1:03am UTC](https://discuss.elastic.co/t/design-question-better-to-have-a-summary-text-field-or-index-each-field-as-keyword-and-text/297815 "2022-02-24T01:03:39Z")
So I have a general design question. I have long been indexing documents where all string fields generally are stored as keywords. However now we are wanting to add the capability to allow users to enter a string and t… --- ## [Data enrichment](https://discuss.elastic.co/t/data-enrichment/298099)
**Author:** [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\ **Replies:** 2\ **Last updated:** [February 24, 2022, 12:35am UTC](https://discuss.elastic.co/t/data-enrichment/298099 "2022-02-24T00:35:51Z")
How is best way for data enrichment? I have events with IDs of employees in ES, and want change them on their surname. How do it best way? Should I do it in logstash? Most helpful would be to change static lookup but I … --- ## [Should Data Nodes still be the same size?](https://discuss.elastic.co/t/should-data-nodes-still-be-the-same-size/298059)
**Author:** [@bunste](https://discuss.elastic.co/u/bunste)\ **Replies:** 2\ **Last updated:** [February 23, 2022, 11:20pm UTC](https://discuss.elastic.co/t/should-data-nodes-still-be-the-same-size/298059 "2022-02-23T23:20:53Z")
Hello all, In our company we run a small Elasticsearch cluster with meanwhile 6 data nodes. These have been added over time and are therefore equipped quite differently - especially in terms of disk capacity. Now we ar… --- ## [Beatsで収集したログを日次指定してlogstashに送信するには](https://discuss.elastic.co/t/beats-logstash/297841)
**Author:** [@gou\_gemma](https://discuss.elastic.co/u/gou_gemma)\ **Replies:** 4\ **Last updated:** [February 23, 2022, 11:17pm UTC](https://discuss.elastic.co/t/beats-logstash/297841 "2022-02-23T23:17:15Z")
winlogbeat,filebeatで収集したログをリアルタイムもしくは日次指定してlogstashに送信する方法が知りたいです。 どのソフトのymlファイルで設定すれば宜しいのでしょうか？ 宜しくお願い致します。 --- ## [Systemctlで起動する際、pipelines.ymlを読み込み起動する方法](https://discuss.elastic.co/t/systemctl-pipelines-yml/298094)
**Author:** [@gou\_gemma](https://discuss.elastic.co/u/gou_gemma)\ **Replies:** 0\ **Last updated:** [February 23, 2022, 11:16pm UTC](https://discuss.elastic.co/t/systemctl-pipelines-yml/298094 "2022-02-23T23:16:02Z")
logstashをsystemctl start logstashで起動する際、デフォルトでpipelines.ymlを読み込み起動すると認識していますが、pipelines.ymlを読み込んで起動しません。 /etc/systemd/system/logstash.serviceの内容は以下 \[Unit\] Description=logstash \[Service\] Type=simple User=root Group=roo… --- ## [Pivot transform copying values](https://discuss.elastic.co/t/pivot-transform-copying-values/297919)
**Author:** [@iamtheschmitzer](https://discuss.elastic.co/u/iamtheschmitzer)\ **Replies:** 10\ **Last updated:** [February 23, 2022, 11:05pm UTC](https://discuss.elastic.co/t/pivot-transform-copying-values/297919 "2022-02-23T23:05:21Z")
I'm attempting my first pivot transform with a goal of calculating duration between timestamps. In the group\_by clause, it results in a two documents, each with a timestamp. I have been able to calculate the duration u… --- ## [Winlogbeatで複数ファイルを振り分ける方法](https://discuss.elastic.co/t/winlogbeat/298093)
**Author:** [@gou\_gemma](https://discuss.elastic.co/u/gou_gemma)\ **Replies:** 0\ **Last updated:** [February 23, 2022, 11:01pm UTC](https://discuss.elastic.co/t/winlogbeat/298093 "2022-02-23T23:01:51Z")
複数ディレクトリに格納されたファイルをwinlogbeatからlogstashにファイルを送信した後で、ディレクトリ毎にファイルを格納したいです。 filebeatであれば、tagsで振り分けができたのですがwinlogbeatではfilebeatでいう、tagsのような振り分けに適したパラメータがあるのでしょうか？ ちなみにfilebeatでは以下のようにディレクトリ毎に振り分けをしています。 output { if "fi… --- ## [Windows Elastic Agent System Integration not pulling Security or Application events](https://discuss.elastic.co/t/windows-elastic-agent-system-integration-not-pulling-security-or-application-events/296507)
**Author:** [@n2x4](https://discuss.elastic.co/u/n2x4)\ **Replies:** 3\ **Last updated:** [February 23, 2022, 10:45pm UTC](https://discuss.elastic.co/t/windows-elastic-agent-system-integration-not-pulling-security-or-application-events/296507 "2022-02-23T22:45:42Z")
I have an Elastic Agent installed on a Windows Server and I've configured the system integration to pull Application, System, and Security logs. Only System logs are being collected and I can't find anything that indica… --- ## [Kibana Controls shows only top 10 values](https://discuss.elastic.co/t/kibana-controls-shows-only-top-10-values/297949)
**Author:** [@Eduard\_Abril](https://discuss.elastic.co/u/Eduard_Abril)\ **Replies:** 4\ **Last updated:** [February 23, 2022, 10:14pm UTC](https://discuss.elastic.co/t/kibana-controls-shows-only-top-10-values/297949 "2022-02-23T22:14:15Z")
Hi guys, I'm building some kibana dashbaords using control panel and I have a problem because control panel is showing only the first 10 values or at least it seems so. I understand that control panel does a terms aggr… --- ## [Python Elasticsearch](https://discuss.elastic.co/t/python-elasticsearch/298053)
**Author:** [@Achraf\_MENSI](https://discuss.elastic.co/u/Achraf_MENSI)\ **Replies:** 4\ **Last updated:** [February 23, 2022, 10:02pm UTC](https://discuss.elastic.co/t/python-elasticsearch/298053 "2022-02-23T22:02:53Z")
Hello it's possible to update ILM policy directly with python? --- ## [Logstash 7.x support on Debian 10 or 11](https://discuss.elastic.co/t/logstash-7-x-support-on-debian-10-or-11/298087)
**Author:** [@jeroen.antsec](https://discuss.elastic.co/u/jeroen.antsec)\ **Replies:** 0\ **Last updated:** [February 23, 2022, 9:48pm UTC](https://discuss.elastic.co/t/logstash-7-x-support-on-debian-10-or-11/298087 "2022-02-23T21:48:06Z")
Hi all. Do any of you know the reason why Logstash 7.x (all versions up to 7.17) are not officially supported on Debian 10 or 11? Elastic and Kibana are, but Logstash not. Only version 8.0 is supported on Debian 10. Wil… --- ## [Logstash parshing unicode characters](https://discuss.elastic.co/t/logstash-parshing-unicode-characters/296890)
**Author:** [@vee](https://discuss.elastic.co/u/vee)\ **Replies:** 16\ **Last updated:** [February 23, 2022, 8:46pm UTC](https://discuss.elastic.co/t/logstash-parshing-unicode-characters/296890 "2022-02-23T20:46:48Z")
hi, we are running into parsing errors when sending data through logstash pipelines. After digging a little, found that the source system has been sending a lot of unicode special characters - something like below: \\u00… --- ## [Explanation of terms in a search](https://discuss.elastic.co/t/explanation-of-terms-in-a-search/297523)
**Author:** [@MrFantastic2022](https://discuss.elastic.co/u/MrFantastic2022)\ **Replies:** 7\ **Last updated:** [February 23, 2022, 8:36pm UTC](https://discuss.elastic.co/t/explanation-of-terms-in-a-search/297523 "2022-02-23T20:36:19Z")
Hello. I am trying to understand this passage, which is in the 6.8 REST API docs: By default, all terms are optional, as long as one term matches. A search for foo bar baz will find any document that contains one or mo… --- ## [Is the q parameter available for search with App Search API](https://discuss.elastic.co/t/is-the-q-parameter-available-for-search-with-app-search-api/298078)
**Author:** [@David\_Robbins](https://discuss.elastic.co/u/David_Robbins)\ **Replies:** 0\ **Last updated:** [February 23, 2022, 7:40pm UTC](https://discuss.elastic.co/t/is-the-q-parameter-available-for-search-with-app-search-api/298078 "2022-02-23T19:40:54Z")
I have a high volume set of documents where I need to export the results into a data set in batches of 3 to 4000 documents. I'm using the App Search API with the Reactjs SearchUI for the front end, and our users will ne… --- ## [Can not connect to localhost:9600 for logstash](https://discuss.elastic.co/t/can-not-connect-to-localhost-9600-for-logstash/297687)
**Author:** [@TomYang1993](https://discuss.elastic.co/u/TomYang1993)\ **Replies:** 13\ **Last updated:** [February 23, 2022, 7:23pm UTC](https://discuss.elastic.co/t/can-not-connect-to-localhost-9600-for-logstash/297687 "2022-02-23T19:23:22Z")
logstash 7.17.0 Elasticsearch 7.17.0 logstash with all default settings, fresh install use sudo systemctl start logstash to start, sudo status shows logstash actively running However, curl -XGET 'localhost:9600/?pret… --- ## [Geoip stopped consolidating coordinates after logstash upgrade to 8.0](https://discuss.elastic.co/t/geoip-stopped-consolidating-coordinates-after-logstash-upgrade-to-8-0/298067)
**Author:** [@Shreesh\_Narayanan](https://discuss.elastic.co/u/Shreesh_Narayanan)\ **Replies:** 2\ **Last updated:** [February 23, 2022, 6:46pm UTC](https://discuss.elastic.co/t/geoip-stopped-consolidating-coordinates-after-logstash-upgrade-to-8-0/298067 "2022-02-23T18:46:22Z")
My logstash for geo tagging IPs are failing after logstash upgrade 8.0. Here's a snippet , i get the lat and lon values , but cant seem to get into the "destlocation" field "Dest\_IP" =\> { "geo" =\> { … --- ## [Using eql to query for custom log](https://discuss.elastic.co/t/using-eql-to-query-for-custom-log/297834)
**Author:** [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Replies:** 1\ **Last updated:** [February 23, 2022, 5:20pm UTC](https://discuss.elastic.co/t/using-eql-to-query-for-custom-log/297834 "2022-02-23T17:20:40Z")
Hi all, I have question on the ability to query data for eql, Can eql be used to query for any log source other than from elastic-agent or winlogbeat. For example: i use filebeat to collect log from iis server then pu… [Previous page](https://discuss.elastic.co/latest.md?page=1244) [Next page](https://discuss.elastic.co/latest.md?page=1246)