# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1163 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1164 --- ## [Metric libbeat.output.write.bytes, libbeat.output.read.bytes, libbeat.output.write.errors and libbeat.output.read.errors shows 0 always](https://discuss.elastic.co/t/metric-libbeat-output-write-bytes-libbeat-output-read-bytes-libbeat-output-write-errors-and-libbeat-output-read-errors-shows-0-always/289260)
**Author:** [@mbhavana](https://discuss.elastic.co/u/mbhavana)\ **Replies:** 0\ **Last updated:** [November 16, 2021, 5:30am UTC](https://discuss.elastic.co/t/metric-libbeat-output-write-bytes-libbeat-output-read-bytes-libbeat-output-write-errors-and-libbeat-output-read-errors-shows-0-always/289260 "2021-11-16T05:30:31Z")
Hello there, i have used REST API "curl -XGET localhost:5066/stats" to get the stats of filebeat.Metrics, libbeat.output.write.bytes, libbeat.output.read.bytes, libbeat.output.write.errors and libbeat.output.read.errors … --- ## [Logstash Nested JSON Parsing and Transforming data](https://discuss.elastic.co/t/logstash-nested-json-parsing-and-transforming-data/289255)
**Author:** [@zx8086](https://discuss.elastic.co/u/zx8086)\ **Replies:** 2\ **Last updated:** [November 16, 2021, 5:18am UTC](https://discuss.elastic.co/t/logstash-nested-json-parsing-and-transforming-data/289255 "2021-11-16T05:18:58Z")
How would I using the json filter to split the following json document and target / modify the time stamp (event.ts) ? {"peer":"EC2LEBOQ77XSF46QEKYYYNKBTVATU33K","agent\_name":"test-device","event":{"ts":1637034155000,"t… --- ## [Error creating runner from config: metricset 'activemq/(broker, queue, topic)' not found](https://discuss.elastic.co/t/error-creating-runner-from-config-metricset-activemq-broker-queue-topic-not-found/289258)
**Author:** [@Tina\_Rawat](https://discuss.elastic.co/u/Tina_Rawat)\ **Replies:** 0\ **Last updated:** [November 16, 2021, 5:04am UTC](https://discuss.elastic.co/t/error-creating-runner-from-config-metricset-activemq-broker-queue-topic-not-found/289258 "2021-11-16T05:04:47Z")
Hi Team, I've started metricbeat and enabled activemq and jolokia module. I'm getting---- Error : 2021-11-15T16:19:36.032+0800 ERROR \[reload\] cfgfile/list.go:96 Error creating runner from config: 1 error: 3 errors: me… --- ## [ES 7.15 FileBeats Sophos XG module not separating data into variables](https://discuss.elastic.co/t/es-7-15-filebeats-sophos-xg-module-not-separating-data-into-variables/288577)
**Author:** [@Samy\_Weee](https://discuss.elastic.co/u/Samy_Weee)\ **Replies:** 23\ **Last updated:** [November 16, 2021, 4:59am UTC](https://discuss.elastic.co/t/es-7-15-filebeats-sophos-xg-module-not-separating-data-into-variables/288577 "2021-11-16T04:59:55Z")
Following This link to configure sophos Module. www.elastic.co/.../filebeat-module-sophos.html I followed the documentation at www.elastic.co/.../filebeat-module-sophos.html and was able to get the data into ES, issue… --- ## [The load of ES cluster CPU is high, but the utilization rate is not high](https://discuss.elastic.co/t/the-load-of-es-cluster-cpu-is-high-but-the-utilization-rate-is-not-high/289246)
**Author:** [@ktpktr0](https://discuss.elastic.co/u/ktpktr0)\ **Replies:** 6\ **Last updated:** [November 16, 2021, 3:37am UTC](https://discuss.elastic.co/t/the-load-of-es-cluster-cpu-is-high-but-the-utilization-rate-is-not-high/289246 "2021-11-16T03:37:54Z")
I have an elk cluster with 3 master nodes, 4 hot data nodes and 4 cold data nodes, and the disks are SSDs. Recently, it was found that the CPU load of the ES cluster is high, but the utilization rate is not high. The fol… --- ## [Heartbeat fake timeout](https://discuss.elastic.co/t/heartbeat-fake-timeout/287278)
**Author:** [@guillaumetel10](https://discuss.elastic.co/u/guillaumetel10)\ **Replies:** 5\ **Last updated:** [November 16, 2021, 3:01am UTC](https://discuss.elastic.co/t/heartbeat-fake-timeout/287278 "2021-11-16T03:01:56Z")
Hi, We use hearbeat to monitor a hundred applications. In general it works well, but sometimes some applications are down (only for a single ping) according to the tool. However, this is not the case according to our t… --- ## [Expand limit of chars in field (truncated to 100,000 chars)](https://discuss.elastic.co/t/expand-limit-of-chars-in-field-truncated-to-100-000-chars/289209)
**Author:** [@christiancj](https://discuss.elastic.co/u/christiancj)\ **Replies:** 2\ **Last updated:** [November 16, 2021, 1:22am UTC](https://discuss.elastic.co/t/expand-limit-of-chars-in-field-truncated-to-100-000-chars/289209 "2021-11-16T01:22:18Z")
Hi, I'm facing an issue with one of our fields in Elasticsearch, I'm using FSCrawler to extract content and then ingest in Elasticsearch, however we've few large size files where the content extracted exceed 100,000 c… --- ## [Create track using datastreams](https://discuss.elastic.co/t/create-track-using-datastreams/289097)
**Author:** [@kvmuralidhar](https://discuss.elastic.co/u/kvmuralidhar)\ **Replies:** 2\ **Last updated:** [November 15, 2021, 11:33pm UTC](https://discuss.elastic.co/t/create-track-using-datastreams/289097 "2021-11-15T23:33:05Z")
Hi, I am trying to create a custom track using an data stream index and it's failing because of the datastream is an hidden index. What's the work around to create this custom track ? Thanks Murali --- ## [Effectively partial update array field in ElasticSearch](https://discuss.elastic.co/t/effectively-partial-update-array-field-in-elasticsearch/288984)
**Author:** [@fatcloud](https://discuss.elastic.co/u/fatcloud)\ **Replies:** 1\ **Last updated:** [November 15, 2021, 11:22pm UTC](https://discuss.elastic.co/t/effectively-partial-update-array-field-in-elasticsearch/288984 "2021-11-15T23:22:40Z")
Hi elastic experts, I'm wondering if there is a way to effectively partial update an element in an array field? For example, a document has a commands field like this: "commands": \[ { "id": 1000, "content": "a… --- ## [Is it ok to delete data (by query) on an index that was force merged to a single segment?](https://discuss.elastic.co/t/is-it-ok-to-delete-data-by-query-on-an-index-that-was-force-merged-to-a-single-segment/289221)
**Author:** [@kirk1](https://discuss.elastic.co/u/kirk1)\ **Replies:** 1\ **Last updated:** [November 15, 2021, 10:23pm UTC](https://discuss.elastic.co/t/is-it-ok-to-delete-data-by-query-on-an-index-that-was-force-merged-to-a-single-segment/289221 "2021-11-15T22:23:32Z")
When our indices are no longer receiving new documents we run a force-merge down to a single segment. We now find ourselves needing to delete a very small percentage of the docs in several of these these single-segment i… --- ## [Machine Learning - Default value for buckets with no documents](https://discuss.elastic.co/t/machine-learning-default-value-for-buckets-with-no-documents/289233)
**Author:** [@bpsullivan3](https://discuss.elastic.co/u/bpsullivan3)\ **Replies:** 1\ **Last updated:** [November 15, 2021, 10:22pm UTC](https://discuss.elastic.co/t/machine-learning-default-value-for-buckets-with-no-documents/289233 "2021-11-15T22:22:08Z")
I am attempting to set up a machine learning job that detects high\_sum of a field for a daily ingest. However, the data I am ingesting is conditional, and we may not actually receive any documents for a given day (most d… --- ## [Set geopoint field from string field correctly for map visualization](https://discuss.elastic.co/t/set-geopoint-field-from-string-field-correctly-for-map-visualization/288513)
**Author:** [@ccfc1986](https://discuss.elastic.co/u/ccfc1986)\ **Replies:** 10\ **Last updated:** [November 15, 2021, 10:01pm UTC](https://discuss.elastic.co/t/set-geopoint-field-from-string-field-correctly-for-map-visualization/288513 "2021-11-15T22:01:34Z")
I have the following field in my documents: "COORDINATES" : "-11.661306;-47.506833" I have created a update\_by\_query that takes this field, extracts the lat/lon and sets to my actual geopoint defined field 'GeoLocation… --- ## [Time series datastore altering](https://discuss.elastic.co/t/time-series-datastore-altering/288478)
**Author:** [@ailengcon](https://discuss.elastic.co/u/ailengcon)\ **Replies:** 5\ **Last updated:** [November 14, 2021, 9:15pm UTC](https://discuss.elastic.co/t/time-series-datastore-altering/288478 "2021-11-14T21:15:39Z")
I want to use Elasticsearch as a time series datastore. Where each event comes in with a timestamp and serial and all other data. I was told by @warkolm I can graph changes over time, either at an individual serial lev… --- ## [Custom analyzer with token replacement](https://discuss.elastic.co/t/custom-analyzer-with-token-replacement/289236)
**Author:** [@pistocop](https://discuss.elastic.co/u/pistocop)\ **Replies:** 0\ **Last updated:** [November 15, 2021, 9:33pm UTC](https://discuss.elastic.co/t/custom-analyzer-with-token-replacement/289236 "2021-11-15T21:33:23Z")
2021-11-14T23:00:00Z (UTC) Hi to everyone, I was playing with custom analyzers and I had trouble with the following example. Basically, I want to index a document with a field named "user\_opinion" in two ways: with… --- ## [Working with JSON array of objects via Ruby code block](https://discuss.elastic.co/t/working-with-json-array-of-objects-via-ruby-code-block/289009)
**Author:** [@Colin\_K](https://discuss.elastic.co/u/Colin_K)\ **Replies:** 10\ **Last updated:** [November 15, 2021, 9:14pm UTC](https://discuss.elastic.co/t/working-with-json-array-of-objects-via-ruby-code-block/289009 "2021-11-15T21:14:29Z")
I have a JSON array of nested objects. I wrote a filter json { source =\> \['body'\] target =\> "parsed" } ruby { code =\> "mapped = event.get('parsed').map{ |h| \[h\['k1'\], h\['k2'\],h\['k3'\],h\['k4'\]\] }; event.set(… --- ## [Connecting to Elasticsearch node in Elastic cloud through python promt](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028)
**Author:** [@Sentel\_AI](https://discuss.elastic.co/u/Sentel_AI)\ **Replies:** 6\ **Last updated:** [November 15, 2021, 8:54pm UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028 "2021-11-15T20:54:35Z")
Hi, In order to create an ingestion pipeline into the Elasticsearch node running in Elastic Cloud, I'm using the following code (API authentication): # API authentication config = configparser.ConfigParser() config.read… --- ## [LogStash Error](https://discuss.elastic.co/t/logstash-error/288444)
**Author:** [@John\_snow](https://discuss.elastic.co/u/John_snow)\ **Replies:** 18\ **Last updated:** [November 15, 2021, 7:30pm UTC](https://discuss.elastic.co/t/logstash-error/288444 "2021-11-15T19:30:37Z")
Hi Team, I'm getting exception=\>#\ … --- ## [Rally-eventdata-track with custom parameter\_sources](https://discuss.elastic.co/t/rally-eventdata-track-with-custom-parameter-sources/289098)
**Author:** [@BlueLegend](https://discuss.elastic.co/u/BlueLegend)\ **Replies:** 2\ **Last updated:** [November 15, 2021, 6:43pm UTC](https://discuss.elastic.co/t/rally-eventdata-track-with-custom-parameter-sources/289098 "2021-11-15T18:43:53Z")
I am run rally-eventdata-track but want to change to my own mapping. I am curious as to how the data in parameter\_sources was generated so I can create my own from my data. Are there any docs on how I can do this? --- ## [Cannot access environment variables](https://discuss.elastic.co/t/cannot-access-environment-variables/288928)
**Author:** [@tofubeats](https://discuss.elastic.co/u/tofubeats)\ **Replies:** 7\ **Last updated:** [November 15, 2021, 6:15pm UTC](https://discuss.elastic.co/t/cannot-access-environment-variables/288928 "2021-11-15T18:15:16Z")
Hi, My config is really simple, I'm just trying to get a simple envornment variable to be accessed and used by a Logstash pipeline. It looks like this: input{ file { path =\> "${test}" } } filter {} output {… --- ## [How to set indices.lifecycle.poll\_interval?](https://discuss.elastic.co/t/how-to-set-indices-lifecycle-poll-interval/289143)
**Author:** [@shi](https://discuss.elastic.co/u/shi)\ **Replies:** 4\ **Last updated:** [November 15, 2021, 6:13pm UTC](https://discuss.elastic.co/t/how-to-set-indices-lifecycle-poll-interval/289143 "2021-11-15T18:13:36Z")
How to set indices.lifecycle.poll\_interval? Read that it a cluster level setting. Thanks and regards shini --- ## [Calling on elasticsearch API endpoint directly](https://discuss.elastic.co/t/calling-on-elasticsearch-api-endpoint-directly/289207)
**Author:** [@tleduong](https://discuss.elastic.co/u/tleduong)\ **Replies:** 1\ **Last updated:** [November 15, 2021, 5:26pm UTC](https://discuss.elastic.co/t/calling-on-elasticsearch-api-endpoint-directly/289207 "2021-11-15T17:26:49Z")
I'm using NEST client to talk to Elasticsearch and it is working fine. However, I have one endpoint where I couldn't figure out how to use NEST client and call on this API. Essentially I'd like to call on "GET my-inde… --- ## [Logstash Date Parse failure](https://discuss.elastic.co/t/logstash-date-parse-failure/289185)
**Author:** [@Stephy\_Jacob](https://discuss.elastic.co/u/Stephy_Jacob)\ **Replies:** 4\ **Last updated:** [November 15, 2021, 5:18pm UTC](https://discuss.elastic.co/t/logstash-date-parse-failure/289185 "2021-11-15T17:18:41Z")
Hello, I am banging my head for quite sometime to understand why am I getting dateparsefailure after trying multiple match combinations. Logstash Version = 7.9.2 sudo echo '2021-11-15T11:17:56.831Z' | /usr/share/logst… --- ## [Shard Failed- The data you are seeing might be incomplete or wrong](https://discuss.elastic.co/t/shard-failed-the-data-you-are-seeing-might-be-incomplete-or-wrong/288670)
**Author:** [@ptanguturi](https://discuss.elastic.co/u/ptanguturi)\ **Replies:** 1\ **Last updated:** [November 15, 2021, 5:08pm UTC](https://discuss.elastic.co/t/shard-failed-the-data-you-are-seeing-might-be-incomplete-or-wrong/288670 "2021-11-15T17:08:17Z")
I am getting below error message. not sure what when wrong and didn't make any changes to index. script\_exception at shard 0index default-2021.11node weAPTkoJTm6njIweWotMLw Type script\_exception Reason runtime error … --- ## [Elastic query scipting issue using painless ( Too many dynamic script compilations )](https://discuss.elastic.co/t/elastic-query-scipting-issue-using-painless-too-many-dynamic-script-compilations/288693)
**Author:** [@Agnivesh\_Sharma](https://discuss.elastic.co/u/Agnivesh_Sharma)\ **Replies:** 3\ **Last updated:** [November 15, 2021, 4:53pm UTC](https://discuss.elastic.co/t/elastic-query-scipting-issue-using-painless-too-many-dynamic-script-compilations/288693 "2021-11-15T16:53:32Z")
A functional script query is working fine when i dont add certain scripts to it but it fails and returns \> \[script\] Too many dynamic script compilations within, max: \[75/5m\]; please use indexed, or scripts with parameter… --- ## [Kibana Canvas SQL LEFT JOIN WHERE](https://discuss.elastic.co/t/kibana-canvas-sql-left-join-where/288474)
**Author:** [@timka23](https://discuss.elastic.co/u/timka23)\ **Replies:** 1\ **Last updated:** [November 15, 2021, 4:04pm UTC](https://discuss.elastic.co/t/kibana-canvas-sql-left-join-where/288474 "2021-11-15T16:04:17Z")
I want to insert the data in the Kibana Canvas table, but I do not work query. How to get data using such a request? SELECT \* FROM A LEFT JOIN B ON A.key = B.key My SQL: SELECT a.id as idn, COUNT(\*) as count, b.title… --- ## [Canvas integrate in Iframe](https://discuss.elastic.co/t/canvas-integrate-in-iframe/288975)
**Author:** [@AlexM566](https://discuss.elastic.co/u/AlexM566)\ **Replies:** 1\ **Last updated:** [November 15, 2021, 3:43pm UTC](https://discuss.elastic.co/t/canvas-integrate-in-iframe/288975 "2021-11-15T15:43:39Z")
Hi everyone, I want to share my Canvas into a web-page using iframe and I want to get rid off by the top panel and the right panel and also display into a fullscreen mode. I know I can use the share on website option, b… --- ## [Elasticsearch is recreating indexes after they were deleted](https://discuss.elastic.co/t/elasticsearch-is-recreating-indexes-after-they-were-deleted/288039)
**Author:** [@kamillacrozara](https://discuss.elastic.co/u/kamillacrozara)\ **Replies:** 5\ **Last updated:** [November 15, 2021, 2:27pm UTC](https://discuss.elastic.co/t/elasticsearch-is-recreating-indexes-after-they-were-deleted/288039 "2021-11-15T14:27:34Z")
I am trying to sync relational data with Elasticsearch using Logstash. To do that I have crated a pipeline to Logstash like this: input { jdbc { tags =\> \["vehicles"\] jdbc\_driver\_library =\> "/usr/share/logstash… --- ## [Master not discovered yet, this node has not previously joined a bootstrapped](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075)
**Author:** [@Maks\_7v](https://discuss.elastic.co/u/Maks_7v)\ **Replies:** 15\ **Last updated:** [November 15, 2021, 2:29pm UTC](https://discuss.elastic.co/t/master-not-discovered-yet-this-node-has-not-previously-joined-a-bootstrapped/289075 "2021-11-15T14:29:59Z")
Hi there. I'm trying to deploy 3 masters, 2 data, and 2 ingest nodes to GCP k8s. There is a problem that masters struggle to discover each other. Here are the logs I get: "master not discovered yet, this node has not… --- ## [JDBC input/output with additional data transforming](https://discuss.elastic.co/t/jdbc-input-output-with-additional-data-transforming/289194)
**Author:** [@Stateros](https://discuss.elastic.co/u/Stateros)\ **Replies:** 2\ **Last updated:** [November 15, 2021, 2:16pm UTC](https://discuss.elastic.co/t/jdbc-input-output-with-additional-data-transforming/289194 "2021-11-15T14:16:55Z")
Hello folks. I am pretty new in logstash. I have a task reload lot of data 5B records from mysql to another mysql with some aggregation. I need to load all fields param1, date group them and take count rows in main table… --- ## [Strange Issue : Logs not getting indexed and error shows nothing](https://discuss.elastic.co/t/strange-issue-logs-not-getting-indexed-and-error-shows-nothing/289166)
**Author:** [@ankitdevnalkar](https://discuss.elastic.co/u/ankitdevnalkar)\ **Replies:** 2\ **Last updated:** [November 15, 2021, 2:11pm UTC](https://discuss.elastic.co/t/strange-issue-logs-not-getting-indexed-and-error-shows-nothing/289166 "2021-11-15T14:11:22Z")
Facing a strange issue, I am using filebeat module to fetch AWS and Azure logs. Since few days I saw some logs are getting indexed properly and some are not. I saw error logs of Elasticsearch and logstash both but nothin… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1162) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1164)