# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1176 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1177 --- ## [Kibana dashboard table without aggregation](https://discuss.elastic.co/t/kibana-dashboard-table-without-aggregation/287968)
**Author:** [@111505](https://discuss.elastic.co/u/111505)\ **Replies:** 5\ **Last updated:** [November 3, 2021, 4:54am UTC](https://discuss.elastic.co/t/kibana-dashboard-table-without-aggregation/287968 "2021-11-03T04:54:09Z")
Hello. I want to create a Data Table similar to Discover Tab in Kibana's Dashobard, but I can not find a way. Do not have a way to create a table that only shows Data without aggregation? --- ## [Elastic Agents 7.15 - Not sending data](https://discuss.elastic.co/t/elastic-agents-7-15-not-sending-data/288342)
**Author:** [@Trinity\_Rolling](https://discuss.elastic.co/u/Trinity_Rolling)\ **Replies:** 0\ **Last updated:** [November 3, 2021, 3:29pm UTC](https://discuss.elastic.co/t/elastic-agents-7-15-not-sending-data/288342 "2021-11-03T15:29:28Z")
Initial installation and setup of the server and agents was successful. I was troubleshooting some issues with the APM Agents not sending data to the server and broke all Fleet and agents. I attempt to undo what I had … --- ## [No data has been received from Functionbeat yet](https://discuss.elastic.co/t/no-data-has-been-received-from-functionbeat-yet/288378)
**Author:** [@spo](https://discuss.elastic.co/u/spo)\ **Replies:** 0\ **Last updated:** [November 3, 2021, 11:52pm UTC](https://discuss.elastic.co/t/no-data-has-been-received-from-functionbeat-yet/288378 "2021-11-03T23:52:56Z")
Hi, I have installed and configured functionbeat on the elk server. Installation and configuration was successful but unable to pull logs on kibana. functionbeat.yml : functionbeat.provider.aws.endpoint: "s3.amazona… --- ## ["all shards failed"](https://discuss.elastic.co/t/all-shards-failed/288230)
**Author:** [@mmuzammil45](https://discuss.elastic.co/u/mmuzammil45)\ **Replies:** 3\ **Last updated:** [November 3, 2021, 11:44pm UTC](https://discuss.elastic.co/t/all-shards-failed/288230 "2021-11-03T23:44:14Z")
I am indexing nested JSON object. The issue is when the number of fields in the object approached to 1200, it started giving the error "all shards failed". I have tried multiple solutions but nothing has worked so far.… --- ## [Add a field from one grok match to another](https://discuss.elastic.co/t/add-a-field-from-one-grok-match-to-another/288246)
**Author:** [@mwas](https://discuss.elastic.co/u/mwas)\ **Replies:** 2\ **Last updated:** [November 3, 2021, 11:38pm UTC](https://discuss.elastic.co/t/add-a-field-from-one-grok-match-to-another/288246 "2021-11-03T23:38:14Z")
I've got log that starts with a jobid on our build cluster live following: "Job \<7073381\> is submitted to queue " I'd like to extract the job id from that line and add it to all other matched lines that look like this: … --- ## [Logstash-jdbc-input plugin issue when time reverts from Daylight Savings to Standard Time](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-issue-when-time-reverts-from-daylight-savings-to-standard-time/288365)
**Author:** [@AlexandrosPan](https://discuss.elastic.co/u/AlexandrosPan)\ **Replies:** 1\ **Last updated:** [November 3, 2021, 9:25pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-plugin-issue-when-time-reverts-from-daylight-savings-to-standard-time/288365 "2021-11-03T21:25:58Z")
Hello, I use logstash with jdbc input plugin to create an Elasticsearch index and keep it synced with a Database table. My database's timezone is Asia/Nicosia and so I've set jdbc\_default\_timezone accordingly. The date… --- ## [Date to string using localtime](https://discuss.elastic.co/t/date-to-string-using-localtime/288360)
**Author:** [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Replies:** 2\ **Last updated:** [November 3, 2021, 9:16pm UTC](https://discuss.elastic.co/t/date-to-string-using-localtime/288360 "2021-11-03T21:16:46Z")
Hi, Im trying to convert a date to string with timezone different from UTC tz = "-03:00" t = event.get("lastHeartbeat") event.set("last\_heartbeat\_text", Time.at(t.to\_f).localtime(tz)) and no matter what time is in las… --- ## [Filbeat enable field message visualization](https://discuss.elastic.co/t/filbeat-enable-field-message-visualization/288367)
**Author:** [@Sergio\_Ferreyra](https://discuss.elastic.co/u/Sergio_Ferreyra)\ **Replies:** 0\ **Last updated:** [November 3, 2021, 8:43pm UTC](https://discuss.elastic.co/t/filbeat-enable-field-message-visualization/288367 "2021-11-03T20:43:13Z")
Hi Team. having an issue with message field in kibana , I need to create a visualization to use in this dashboard, but looks like is not enabled. Example: I saw there is a processor called dissect (Dissect strings | F… --- ## [What are the best practices for reading logs/data from cold or frozen data (more than 3, 6 or 12 months old)](https://discuss.elastic.co/t/what-are-the-best-practices-for-reading-logs-data-from-cold-or-frozen-data-more-than-3-6-or-12-months-old/288167)
**Author:** [@parasavelli](https://discuss.elastic.co/u/parasavelli)\ **Replies:** 2\ **Last updated:** [November 3, 2021, 8:16pm UTC](https://discuss.elastic.co/t/what-are-the-best-practices-for-reading-logs-data-from-cold-or-frozen-data-more-than-3-6-or-12-months-old/288167 "2021-11-03T20:16:29Z")
I would like to explore the best practices to search frozen data. Can you please help me to understand all the available options and best approaches. Thanks. --- ## [Need to make logs in order](https://discuss.elastic.co/t/need-to-make-logs-in-order/287845)
**Author:** [@cool999](https://discuss.elastic.co/u/cool999)\ **Replies:** 15\ **Last updated:** [November 3, 2021, 8:05pm UTC](https://discuss.elastic.co/t/need-to-make-logs-in-order/287845 "2021-11-03T20:05:23Z")
Hi, I have below app logs, like below but they are not in oder in kibana i.e the last log line below is not the first in kibana discover. 2021-10-27 16:51:14.864 | INFO | | | | | \[http-nio-8080-exec-32\] | next-con… --- ## [Logstash fails to start: "no such file to load -- concurrent"](https://discuss.elastic.co/t/logstash-fails-to-start-no-such-file-to-load-concurrent/288268)
**Author:** [@dwasss](https://discuss.elastic.co/u/dwasss)\ **Replies:** 9\ **Last updated:** [November 3, 2021, 7:47pm UTC](https://discuss.elastic.co/t/logstash-fails-to-start-no-such-file-to-load-concurrent/288268 "2021-11-03T19:47:06Z")
I recently upgraded my Elastic stack from 7.12 to 7.15.1 on Ubuntu 18.04. Logstash is running via systemd. Now Logstash fails to start with the following error: Nov 02 19:58:28 server logstash\[30340\]: Using bundled JDK:… --- ## [Best similarity for retrieving exact matches first](https://discuss.elastic.co/t/best-similarity-for-retrieving-exact-matches-first/288319)
**Author:** [@Giovanni\_Montobbio](https://discuss.elastic.co/u/Giovanni_Montobbio)\ **Replies:** 7\ **Last updated:** [November 3, 2021, 7:04pm UTC](https://discuss.elastic.co/t/best-similarity-for-retrieving-exact-matches-first/288319 "2021-11-03T19:04:14Z")
Hi I'm working on a project with elasticseach. I have an index with 1 million phrases inside and I want to retrieve phrases from the index which match with some query phrases. The phrases are in italian and I use an ital… --- ## [Weird kibana exception after authentication](https://discuss.elastic.co/t/weird-kibana-exception-after-authentication/288347)
**Author:** [@pkaramol](https://discuss.elastic.co/u/pkaramol)\ **Replies:** 1\ **Last updated:** [November 3, 2021, 6:56pm UTC](https://discuss.elastic.co/t/weird-kibana-exception-after-authentication/288347 "2021-11-03T18:56:27Z")
I am able to access the initial page of my kibana instance. After however providing username and password, I see this in my web browser page {"statusCode":500,"error":"Internal Server Error","message":"illegal\_state\_ex… --- ## [Replace paths in every field](https://discuss.elastic.co/t/replace-paths-in-every-field/287106)
**Author:** [@AnneHermann](https://discuss.elastic.co/u/AnneHermann)\ **Replies:** 5\ **Last updated:** [November 3, 2021, 6:15pm UTC](https://discuss.elastic.co/t/replace-paths-in-every-field/287106 "2021-11-03T18:15:59Z")
Hello, I've been trying to edit my logstash.config so that all file paths in all fields are changed to "anonym". input { http { port =\> 5044 codec =\> json } } filter { split { field… --- ## [Logstash + Syslog + Security](https://discuss.elastic.co/t/logstash-syslog-security/288337)
**Author:** [@MKirby](https://discuss.elastic.co/u/MKirby)\ **Replies:** 2\ **Last updated:** [November 3, 2021, 4:29pm UTC](https://discuss.elastic.co/t/logstash-syslog-security/288337 "2021-11-03T16:29:28Z")
Hello ELK gurus. I have been working on getting my first production ELK SIEM working and for the most part is it doing what is required. The one area that is really not functioning the way it needs is the Logstash and … --- ## [Logstash (Running as Windows Service) - cannot obtain logs from Windows Network Drive](https://discuss.elastic.co/t/logstash-running-as-windows-service-cannot-obtain-logs-from-windows-network-drive/288290)
**Author:** [@coldycc](https://discuss.elastic.co/u/coldycc)\ **Replies:** 3\ **Last updated:** [November 3, 2021, 4:12pm UTC](https://discuss.elastic.co/t/logstash-running-as-windows-service-cannot-obtain-logs-from-windows-network-drive/288290 "2021-11-03T16:12:08Z")
Hi Everyone, I'm having issue to get Logstash to send logs from Windows network drive while running it as Windows Service (using NSSM), it works perfectly fine using the CLI. Here's the logstash.conf: input { file {… --- ## [Nodejs ingest pdf](https://discuss.elastic.co/t/nodejs-ingest-pdf/288335)
**Author:** [@uros87](https://discuss.elastic.co/u/uros87)\ **Replies:** 3\ **Last updated:** [November 3, 2021, 3:58pm UTC](https://discuss.elastic.co/t/nodejs-ingest-pdf/288335 "2021-11-03T15:58:20Z")
Hello, I've tried to inject pdf file to elastic. PUT \_ingest/pipeline/pdfnew { "description": "Extract attachment information encoded in Base64 with UTF-8 charset", "processors": \[ { "attachment": { … --- ## [Enable Elastic Security prebuilt ML jobs \[2\]](https://discuss.elastic.co/t/enable-elastic-security-prebuilt-ml-jobs-2/288345)
**Author:** [@bil\_15](https://discuss.elastic.co/u/bil_15)\ **Replies:** 0\ **Last updated:** [November 3, 2021, 3:36pm UTC](https://discuss.elastic.co/t/enable-elastic-security-prebuilt-ml-jobs-2/288345 "2021-11-03T15:36:20Z")
Hello! Reference: \[Enable Elastic Security prebuilt rules and ML jobs\] So, I've uploaded ML Jobs bundled with datafeed through API using the request: PUT \_ml/anomaly\_detectors/custom\_rare\_process\_by\_host\_linux\_ecs { … --- ## [Call synchronous a pipeline (ex. jdbc/http input plugin) logstash from Java code and get Reponse](https://discuss.elastic.co/t/call-synchronous-a-pipeline-ex-jdbc-http-input-plugin-logstash-from-java-code-and-get-reponse/288344)
**Author:** [@ANWARIM](https://discuss.elastic.co/u/ANWARIM)\ **Replies:** 0\ **Last updated:** [November 3, 2021, 3:31pm UTC](https://discuss.elastic.co/t/call-synchronous-a-pipeline-ex-jdbc-http-input-plugin-logstash-from-java-code-and-get-reponse/288344 "2021-11-03T15:31:43Z")
Hi, I am completely new in Logstash & ES world. Is it possible to call a pipleline synchronous from java code and wait till get the reponse, in this case we want to block the recored till update is completed and Elastic… --- ## [Logstash getting 429 error and data processing getting delayed](https://discuss.elastic.co/t/logstash-getting-429-error-and-data-processing-getting-delayed/288325)
**Author:** [@Sarat\_Khilar](https://discuss.elastic.co/u/Sarat_Khilar)\ **Replies:** 5\ **Last updated:** [November 3, 2021, 3:08pm UTC](https://discuss.elastic.co/t/logstash-getting-429-error-and-data-processing-getting-delayed/288325 "2021-11-03T15:08:52Z")
We are using Elasticsearch 6.8.2 version. We are manging Elasticsearch cluster through kubernetes. We have 3 master nodes and 25 data nodes. Kubernetes cluster is being setup in bare metal AWS EC2 instance. r4.4xlarge(16… --- ## [Plugin building issue](https://discuss.elastic.co/t/plugin-building-issue/287802)
**Author:** [@pchakour](https://discuss.elastic.co/u/pchakour)\ **Replies:** 5\ **Last updated:** [November 3, 2021, 3:08pm UTC](https://discuss.elastic.co/t/plugin-building-issue/287802 "2021-11-03T15:08:44Z")
Hi ! I created a visualization plugin for Kibana 7.14.0. When I try to build it using the kibana plugin helper package, I get the following error: SassError: File to import not found or unreadable: src/core/public/mix… --- ## [While creating control visualization we are getting warning "Terms list might be incomplete"](https://discuss.elastic.co/t/while-creating-control-visualization-we-are-getting-warning-terms-list-might-be-incomplete/288187)
**Author:** [@prashant1](https://discuss.elastic.co/u/prashant1)\ **Replies:** 1\ **Last updated:** [November 3, 2021, 3:04pm UTC](https://discuss.elastic.co/t/while-creating-control-visualization-we-are-getting-warning-terms-list-might-be-incomplete/288187 "2021-11-03T15:04:12Z")
Hi We are using elk version 7.10.2. We have created a control visualisation in kibana UI. We noticed that when we selected the options list and after adding the fields it gives the below warning "Terms list might be inc… --- ## [Metricbeat shard size](https://discuss.elastic.co/t/metricbeat-shard-size/288339)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 0\ **Last updated:** [November 3, 2021, 3:04pm UTC](https://discuss.elastic.co/t/metricbeat-shard-size/288339 "2021-11-03T15:04:04Z")
I have a quick question about the best roll over shard size for metricbeat data. I understand that it's best to test with your own system but I figure that metricbeat data is so standardized that it would make sense for … --- ## [How does can\_match functionality work](https://discuss.elastic.co/t/how-does-can-match-functionality-work/288164)
**Author:** [@seanziee](https://discuss.elastic.co/u/seanziee)\ **Replies:** 7\ **Last updated:** [November 3, 2021, 2:21pm UTC](https://discuss.elastic.co/t/how-does-can-match-functionality-work/288164 "2021-11-03T14:21:36Z")
Just finished watching the video(@ruflin ):Deep dive into the new Elastic Indexing Strategy - YouTube and it's mentioned that because of the can\_match functionality, indices that don't have relevant documents won't be qu… --- ## [High Load ELK on docker](https://discuss.elastic.co/t/high-load-elk-on-docker/288262)
**Author:** [@Juan\_Fernandez\_Bernt](https://discuss.elastic.co/u/Juan_Fernandez_Bernt)\ **Replies:** 2\ **Last updated:** [November 3, 2021, 1:03pm UTC](https://discuss.elastic.co/t/high-load-elk-on-docker/288262 "2021-11-03T13:03:56Z")
I've been having issues with my ELK deployment. Its on docker and the CPU/Load usage is always high for some reason I still cannot find. Each ES node has 4GB memory assigned, total memory on the machine is 32GB. I'v… --- ## [Terms query slowing down at specific size](https://discuss.elastic.co/t/terms-query-slowing-down-at-specific-size/288005)
**Author:** [@PatrickB](https://discuss.elastic.co/u/PatrickB)\ **Replies:** 6\ **Last updated:** [November 3, 2021, 2:08pm UTC](https://discuss.elastic.co/t/terms-query-slowing-down-at-specific-size/288005 "2021-11-03T14:08:28Z")
Hi, we were noticing some odd query runtime behaviour, while investigating ways to improve our ES queries. We would like to ask you about some insights about that. Given the following mapping: { "settings": { ... … --- ## [An index for each host](https://discuss.elastic.co/t/an-index-for-each-host/287333)
**Author:** [@enp2s6](https://discuss.elastic.co/u/enp2s6)\ **Replies:** 4\ **Last updated:** [November 3, 2021, 1:59pm UTC](https://discuss.elastic.co/t/an-index-for-each-host/287333 "2021-11-03T13:59:06Z")
Hello, I would like to have a separate index created for each host. During a test to include a second host with Filebeat, the data was included in the already existing index. I could not find anything suitable in the … --- ## [Redis pipeline is recognized but not receiving logs](https://discuss.elastic.co/t/redis-pipeline-is-recognized-but-not-receiving-logs/288327)
**Author:** [@nyquillus](https://discuss.elastic.co/u/nyquillus)\ **Replies:** 0\ **Last updated:** [November 3, 2021, 12:45pm UTC](https://discuss.elastic.co/t/redis-pipeline-is-recognized-but-not-receiving-logs/288327 "2021-11-03T12:45:31Z")
Hi, I'm trying to add redis as a buffer to my stack in docker-compose. My logstash.conf input section is like below: input { redis { host =\> "redistest" port =\> "6379" ssl =\> false data\_type =\> "list"… --- ## [Create timestamp from nested JSON elements](https://discuss.elastic.co/t/create-timestamp-from-nested-json-elements/287992)
**Author:** [@alt-glitch](https://discuss.elastic.co/u/alt-glitch)\ **Replies:** 1\ **Last updated:** [November 3, 2021, 11:44am UTC](https://discuss.elastic.co/t/create-timestamp-from-nested-json-elements/287992 "2021-11-03T11:44:02Z")
Hi all, I am indexing nested JSON files using Logstash and want to extract the timestamp from them. Here are a couple of examples. Example 1 { "ip": "127.0.0.1", "data": { "postgres": { "status": "succes… --- ## [Use of length function in split processor](https://discuss.elastic.co/t/use-of-length-function-in-split-processor/287972)
**Author:** [@varunbsgaur](https://discuss.elastic.co/u/varunbsgaur)\ **Replies:** 5\ **Last updated:** [November 3, 2021, 11:05am UTC](https://discuss.elastic.co/t/use-of-length-function-in-split-processor/287972 "2021-11-03T11:05:38Z")
How can we use "length" to only extract last two values of the string in the given script:- POST \_ingest/pipeline/\_simulate { "pipeline" : { "description": "description", "processors": \[ { "split": { "field": "f… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1175) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1177)