# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1023 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1024 --- ## [Send custom data to Elastic](https://discuss.elastic.co/t/send-custom-data-to-elastic/299060)
**Author:** [@nfeliciano](https://discuss.elastic.co/u/nfeliciano)\ **Replies:** 0\ **Last updated:** [March 8, 2022, 9:31am UTC](https://discuss.elastic.co/t/send-custom-data-to-elastic/299060 "2022-03-08T09:31:41Z")
Hello, I've been using metricbeat to monitor machine standard values successfully, like cpu memory and io. I can plot the values on the Kibana dashboards. Now I also want to send application custom data. I can extract … --- ## [Need for syntax and semantix - GROK filter](https://discuss.elastic.co/t/need-for-syntax-and-semantix-grok-filter/298404)
**Author:** [@Clyton](https://discuss.elastic.co/u/Clyton)\ **Replies:** 1\ **Last updated:** [March 8, 2022, 8:27am UTC](https://discuss.elastic.co/t/need-for-syntax-and-semantix-grok-filter/298404 "2022-03-08T08:27:32Z")
Hi All, I would like to grep the below numeric value before the keyword "DEBUG" from the message field and map it to a new field called OrderID. Kindly let us know the pattern we can apply in order to achieve this conte… --- ## [Convert a dashboard from 7.16 to 7.14](https://discuss.elastic.co/t/convert-a-dashboard-from-7-16-to-7-14/298946)
**Author:** [@khouloud1](https://discuss.elastic.co/u/khouloud1)\ **Replies:** 1\ **Last updated:** [March 8, 2022, 8:05am UTC](https://discuss.elastic.co/t/convert-a-dashboard-from-7-16-to-7-14/298946 "2022-03-08T08:05:35Z")
Hello , i did import a dashboard 7.16 to my kibana 7.14 but it doesn't work because it isn't compatible. Can you help me please to convert this dashboard to be compatible with my EK 7.14 https://raw.githubusercon… --- ## [Some indices do not show datetime filter](https://discuss.elastic.co/t/some-indices-do-not-show-datetime-filter/299018)
**Author:** [@markand.bhatt](https://discuss.elastic.co/u/markand.bhatt)\ **Replies:** 2\ **Last updated:** [March 8, 2022, 7:59am UTC](https://discuss.elastic.co/t/some-indices-do-not-show-datetime-filter/299018 "2022-03-08T07:59:21Z")
I wonder some of my indices show datetime filter in KQL kibana, some not. Could someone please guide me, any setting in kibana or elastic to show datetime filter? --- ## [Query for IP External IP Addresses Only](https://discuss.elastic.co/t/query-for-ip-external-ip-addresses-only/299024)
**Author:** [@Wilks](https://discuss.elastic.co/u/Wilks)\ **Replies:** 1\ **Last updated:** [March 8, 2022, 7:55am UTC](https://discuss.elastic.co/t/query-for-ip-external-ip-addresses-only/299024 "2022-03-08T07:55:43Z")
Hi, I am trying to use a KQL query to search for non private (rfc1918) addresses. I am using the following query AND NOT destination.ip:10.0.0.0/8 OR NOT destination.ip:192.168.0.0/16 OR NOT destination.ip :172.16.0.0… --- ## [Elasticsearch index](https://discuss.elastic.co/t/elasticsearch-index/298257)
**Author:** [@umangpatel](https://discuss.elastic.co/u/umangpatel)\ **Replies:** 29\ **Last updated:** [March 8, 2022, 5:42am UTC](https://discuss.elastic.co/t/elasticsearch-index/298257 "2022-03-08T05:42:10Z")
I have to remove old data of apiman-indices in Elasticsearch which contain .pos, .cf or any other extension. So how can we remove the old indices? --- ## [Elasticsearch bulk error - must be terminated by a newline](https://discuss.elastic.co/t/elasticsearch-bulk-error-must-be-terminated-by-a-newline/299037)
**Author:** [@kobalt](https://discuss.elastic.co/u/kobalt)\ **Replies:** 1\ **Last updated:** [March 8, 2022, 5:20am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-error-must-be-terminated-by-a-newline/299037 "2022-03-08T05:20:41Z")
trying to use a elasticsearch bulk api. e.g. the following bulk request succeeds $ curl -X POST "localhost:9200/test/\_bulk?pretty" -H 'Content-Type: application/json' -d ' {"index":{"\_id":1}} {"foo":"foo"} {"index":{"\_i… --- ## [Elasticsearch Log deletion not happening as per defined days in log4j.pproperties file](https://discuss.elastic.co/t/elasticsearch-log-deletion-not-happening-as-per-defined-days-in-log4j-pproperties-file/299038)
**Author:** [@pratikshatiwari](https://discuss.elastic.co/u/pratikshatiwari)\ **Replies:** 0\ **Last updated:** [March 8, 2022, 5:07am UTC](https://discuss.elastic.co/t/elasticsearch-log-deletion-not-happening-as-per-defined-days-in-log4j-pproperties-file/299038 "2022-03-08T05:07:12Z")
Dear Team, We are facing an issue where Elasticsearch logging with log4j2.properties not performing the log rotation on the basis of lfLastmodified age condition, we want to rotate the logs every after 7days but it is n… --- ## [How to Properly Use The Roles in values.yaml?](https://discuss.elastic.co/t/how-to-properly-use-the-roles-in-values-yaml/298962)
**Author:** [@lloyd01sanchez12](https://discuss.elastic.co/u/lloyd01sanchez12)\ **Replies:** 3\ **Last updated:** [March 8, 2022, 4:43am UTC](https://discuss.elastic.co/t/how-to-properly-use-the-roles-in-values-yaml/298962 "2022-03-08T04:43:58Z")
Hi Team, I've been having a problem correcting the deprecation warnings related to node.roles: "message": "legacy role settings \[node.data, node.remote\_cluster\_client, node.ingest, node.master, node.ml\] are deprecated,… --- ## [Strange data type error after upgraded ELK to 8.0 - "failed to parse date field \[...\]"](https://discuss.elastic.co/t/strange-data-type-error-after-upgraded-elk-to-8-0-failed-to-parse-date-field/299031)
**Author:** [@pinehk](https://discuss.elastic.co/u/pinehk)\ **Replies:** 0\ **Last updated:** [March 8, 2022, 3:07am UTC](https://discuss.elastic.co/t/strange-data-type-error-after-upgraded-elk-to-8-0-failed-to-parse-date-field/299031 "2022-03-08T03:07:12Z")
Upgraded ELK from 7.15 to 8.0, basic configurations are unchanged. But in 8.0, a CSV filter suddenly broke at unit test, failing to process a line that worked perfectly fine in 7. The error is "caused\_by"=\>{"type"=\>"i… --- ## [Kibana logstash Snapshot Storage Size continuously growing its size](https://discuss.elastic.co/t/kibana-logstash-snapshot-storage-size-continuously-growing-its-size/299023)
**Author:** [@renatoa12](https://discuss.elastic.co/u/renatoa12)\ **Replies:** 3\ **Last updated:** [March 8, 2022, 2:08am UTC](https://discuss.elastic.co/t/kibana-logstash-snapshot-storage-size-continuously-growing-its-size/299023 "2022-03-08T02:08:24Z")
Hi Community, Does anyone here experience that their Snapshot storage size continuously growing, currently i have 200gb snapshot size and still growing for march 3 logs, normally the size per day of snapshot ranging 50-… --- ## [Certificate doesn't match any of the subject alternative names](https://discuss.elastic.co/t/certificate-doesnt-match-any-of-the-subject-alternative-names/298091)
**Author:** [@cyberzlo](https://discuss.elastic.co/u/cyberzlo)\ **Replies:** 2\ **Last updated:** [March 8, 2022, 1:20am UTC](https://discuss.elastic.co/t/certificate-doesnt-match-any-of-the-subject-alternative-names/298091 "2022-03-08T01:20:38Z")
Since update to ELK 8 there is following problem: \[2022-02-23T23:18:41,619\]\[WARN \]\[logstash.outputs.elasticsearch\]\[main\] Failed to perform request {:message=\>"Certificate for \ doesn't match any of the sub… --- ## [Functionbeat Cannot index event - security\_exception](https://discuss.elastic.co/t/functionbeat-cannot-index-event-security-exception/299016)
**Author:** [@carl0s](https://discuss.elastic.co/u/carl0s)\ **Replies:** 0\ **Last updated:** [March 7, 2022, 11:05pm UTC](https://discuss.elastic.co/t/functionbeat-cannot-index-event-security-exception/299016 "2022-03-07T23:05:40Z")
After upgrading to 7.17 none of my logs are being processed, lots of 403 errors: { "type": "security\_exception", "reason": "action \[indices:data/write/bulk\[s\]\] is unauthorized for API key id \[\\] of user … --- ## [Cannot convert created field to integer](https://discuss.elastic.co/t/cannot-convert-created-field-to-integer/299005)
**Author:** [@aerodynamic](https://discuss.elastic.co/u/aerodynamic)\ **Replies:** 6\ **Last updated:** [March 7, 2022, 10:00pm UTC](https://discuss.elastic.co/t/cannot-convert-created-field-to-integer/299005 "2022-03-07T22:00:25Z")
Currently I'm dealing with a situation where a field name is dynamically created and needs to be set to an integer like so mutate { add\_field =\> { "%{fieldName}" =\> "%{dataInteger}" } convert … --- ## [Logstash pipeline processing](https://discuss.elastic.co/t/logstash-pipeline-processing/298921)
**Author:** [@KarlWolf](https://discuss.elastic.co/u/KarlWolf)\ **Replies:** 5\ **Last updated:** [March 7, 2022, 9:52pm UTC](https://discuss.elastic.co/t/logstash-pipeline-processing/298921 "2022-03-07T21:52:43Z")
Hello, I am having a problem with Logstash configuration. I can't get rid with pipelines configuraton. My scenario: logstash.yml - only lines which are uncommented/changed path.data: /usr/share/logstash pipeline.worke… --- ## [ML datafeed bucket aggregation/script challenge](https://discuss.elastic.co/t/ml-datafeed-bucket-aggregation-script-challenge/297552)
**Author:** [@rcowart](https://discuss.elastic.co/u/rcowart)\ **Replies:** 6\ **Last updated:** [March 7, 2022, 9:35pm UTC](https://discuss.elastic.co/t/ml-datafeed-bucket-aggregation-script-challenge/297552 "2022-03-07T21:35:41Z")
I would like to create an anomaly detection job for the ratio of two counts. The logic is as follows... A = count filtered by condition X B = count filtered by condition Y C = A/B The datafeed would need to produce C… --- ## [Issue with installing metric beat & collecting data](https://discuss.elastic.co/t/issue-with-installing-metric-beat-collecting-data/298855)
**Author:** [@Vinay\_Menon1](https://discuss.elastic.co/u/Vinay_Menon1)\ **Replies:** 4\ **Last updated:** [March 7, 2022, 9:08pm UTC](https://discuss.elastic.co/t/issue-with-installing-metric-beat-collecting-data/298855 "2022-03-07T21:08:07Z")
Hi, Please help I am trying to execute this PowerShell script in my windows system to get Metricbeat data to elastic but it does not work as intended , The script runs properly by downloading the metric beat file to c:\\… --- ## [Multiline grok filter not working with specific log](https://discuss.elastic.co/t/multiline-grok-filter-not-working-with-specific-log/298923)
**Author:** [@Mihailo\_Stanarevic](https://discuss.elastic.co/u/Mihailo_Stanarevic)\ **Replies:** 2\ **Last updated:** [March 7, 2022, 8:42pm UTC](https://discuss.elastic.co/t/multiline-grok-filter-not-working-with-specific-log/298923 "2022-03-07T20:42:49Z")
Hello, I have logs that i have to process that look something like this As far as my knowledge goes I have to set up logstash to support multiline logs so I created the following configuration input { file { pat… --- ## [Best practices for Elastic DB deployment for 100TB data](https://discuss.elastic.co/t/best-practices-for-elastic-db-deployment-for-100tb-data/298918)
**Author:** [@noamsh88](https://discuss.elastic.co/u/noamsh88)\ **Replies:** 4\ **Last updated:** [March 7, 2022, 8:35pm UTC](https://discuss.elastic.co/t/best-practices-for-elastic-db-deployment-for-100tb-data/298918 "2022-03-07T20:35:19Z")
Hello, We are designing new product that based on Elasticsearch that suppose to have around 100TB data Deployment will be on-prem (bare metal with Linux OS installed on it) and would like to ask: What is best practic… --- ## [Error while starting ES7.17.0](https://discuss.elastic.co/t/error-while-starting-es7-17-0/298989)
**Author:** [@nandatibco](https://discuss.elastic.co/u/nandatibco)\ **Replies:** 1\ **Last updated:** [March 7, 2022, 7:52pm UTC](https://discuss.elastic.co/t/error-while-starting-es7-17-0/298989 "2022-03-07T19:52:11Z")
Hello team, While starting Elasticsearch 7.17.0 with security enabled, getting below error. Please suggest where we need set the password and how. Thanks in advance. Error: \[o.e.b.ElasticsearchUncaughtExceptionHandl… --- ## [Filter Ratio in Maps](https://discuss.elastic.co/t/filter-ratio-in-maps/299004)
**Author:** [@sushmamagesh](https://discuss.elastic.co/u/sushmamagesh)\ **Replies:** 1\ **Last updated:** [March 7, 2022, 7:46pm UTC](https://discuss.elastic.co/t/filter-ratio-in-maps/299004 "2022-03-07T19:46:13Z")
Is it possible to use filter ratio with a numerator/value in maps (similar to average) Something similar to Filter Ratio within Kibana TSVB. --- ## [Find all the saved objects for a given tag](https://discuss.elastic.co/t/find-all-the-saved-objects-for-a-given-tag/298486)
**Author:** [@dao](https://discuss.elastic.co/u/dao)\ **Replies:** 1\ **Last updated:** [March 7, 2022, 6:47pm UTC](https://discuss.elastic.co/t/find-all-the-saved-objects-for-a-given-tag/298486 "2022-03-07T18:47:38Z")
Hello, I try to call a REST request to get all the saved objects tagged with a given tag FS. I'd like to call something like GET api/kibana/management/saved\_objects/\_find?type=dashboard&search\_fields=tag&search=FS It … --- ## [How to transfer data from mongoDB to elasticsearch using mongodb-plugin-input](https://discuss.elastic.co/t/how-to-transfer-data-from-mongodb-to-elasticsearch-using-mongodb-plugin-input/299001)
**Author:** [@Douglas\_Andrade](https://discuss.elastic.co/u/Douglas_Andrade)\ **Replies:** 1\ **Last updated:** [March 7, 2022, 6:42pm UTC](https://discuss.elastic.co/t/how-to-transfer-data-from-mongodb-to-elasticsearch-using-mongodb-plugin-input/299001 "2022-03-07T18:42:35Z")
I would like to know if it is possible to transfer a complete database from mongoDB to Elasticsearch, not just new data, using mongoDB-plugin-input --- ## [Two Http\_poller inputs with access token](https://discuss.elastic.co/t/two-http-poller-inputs-with-access-token/298939)
**Author:** [@rodri.gz](https://discuss.elastic.co/u/rodri.gz)\ **Replies:** 7\ **Last updated:** [March 7, 2022, 5:51pm UTC](https://discuss.elastic.co/t/two-http-poller-inputs-with-access-token/298939 "2022-03-07T17:51:03Z")
Hello! I am trying to get data from an api that needs to be accessed using a token that we get from another api. How can I pass this access token to get the data from the other api? I had thought of doing something si… --- ## [Compare two time periods for same timeseries](https://discuss.elastic.co/t/compare-two-time-periods-for-same-timeseries/297196)
**Author:** [@achint](https://discuss.elastic.co/u/achint)\ **Replies:** 2\ **Last updated:** [March 7, 2022, 5:50pm UTC](https://discuss.elastic.co/t/compare-two-time-periods-for-same-timeseries/297196 "2022-03-07T17:50:35Z")
Hi, I run performance tests on different time periods and i get the application response time in ELK. I want to compare these time intervals so that i can compare the response time for different build. Response time dat… --- ## [SQL match function](https://discuss.elastic.co/t/sql-match-function/298922)
**Author:** [@avnere](https://discuss.elastic.co/u/avnere)\ **Replies:** 3\ **Last updated:** [March 7, 2022, 5:39pm UTC](https://discuss.elastic.co/t/sql-match-function/298922 "2022-03-07T17:39:23Z")
Hi, I am looking how to perform LIKE on text type. I found a way by MATCH. Is MATCH is Elasticsearch SQL unique function or native SQL? for example: MATCH(foo, 'bar') Are there more options to perform LIKE on text… --- ## [Metricbeat sql module fields query - change to date/time](https://discuss.elastic.co/t/metricbeat-sql-module-fields-query-change-to-date-time/296723)
**Author:** [@Garry](https://discuss.elastic.co/u/Garry)\ **Replies:** 1\ **Last updated:** [March 7, 2022, 5:17pm UTC](https://discuss.elastic.co/t/metricbeat-sql-module-fields-query-change-to-date-time/296723 "2022-03-07T17:17:28Z")
When I ingest metricbeat sql module fields they are stored as a string. How can I change some of the date/time fields returned from the sql query from string to date/time to be used in Kibana? --- ## [Is it possible to filter by agent policy?](https://discuss.elastic.co/t/is-it-possible-to-filter-by-agent-policy/297392)
**Author:** [@Teckinfor](https://discuss.elastic.co/u/Teckinfor)\ **Replies:** 1\ **Last updated:** [March 7, 2022, 5:12pm UTC](https://discuss.elastic.co/t/is-it-possible-to-filter-by-agent-policy/297392 "2022-03-07T17:12:56Z")
Hello, I would like to be able to monitor 2 infrastructures at the same time, I have created a space for each infrastructure as well as 2 agent policies linked to the agents of their own infrastructure. I was wondering… --- ## [Time zones in query string query not working](https://discuss.elastic.co/t/time-zones-in-query-string-query-not-working/298995)
**Author:** [@Bostjan\_Kovac](https://discuss.elastic.co/u/Bostjan_Kovac)\ **Replies:** 0\ **Last updated:** [March 7, 2022, 5:08pm UTC](https://discuss.elastic.co/t/time-zones-in-query-string-query-not-working/298995 "2022-03-07T17:08:36Z")
I believe that the query string queries are not acknowledging the time\_zone property in the query. I have tested with range query and they are working as expected. My test index: PUT test { "mappings": { "dynamic… --- ## [Failed to connect to postgreslq via logsatash](https://discuss.elastic.co/t/failed-to-connect-to-postgreslq-via-logsatash/298772)
**Author:** [@francieliton\_araujo](https://discuss.elastic.co/u/francieliton_araujo)\ **Replies:** 1\ **Last updated:** [March 7, 2022, 4:37pm UTC](https://discuss.elastic.co/t/failed-to-connect-to-postgreslq-via-logsatash/298772 "2022-03-07T16:37:33Z")
it's the first time I'm trying to access a postgresql database through logstash. however there are dnado errors, can you help me. Below is conf and error. input{ jdbc{ jdbc\_driver\_library =\> "/usr/share/java/pos… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1022) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1024)