# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1393 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1394 --- ## [Logstash filter based on reference set or database](https://discuss.elastic.co/t/logstash-filter-based-on-reference-set-or-database/274302)
**Author:** [@redfish462](https://discuss.elastic.co/u/redfish462)\ **Replies:** 3\ **Last updated:** [May 28, 2021, 8:07pm UTC](https://discuss.elastic.co/t/logstash-filter-based-on-reference-set-or-database/274302 "2021-05-28T20:07:16Z")
Hello, I have to redirect some logs based on their username (the usernames are contains in the logs) to different outputs. This list has 80 entry, so it's a lot to write this in the configuration file. I would like to … --- ## [Calculating fields in logstash on upsert](https://discuss.elastic.co/t/calculating-fields-in-logstash-on-upsert/274371)
**Author:** [@tumbledwyer](https://discuss.elastic.co/u/tumbledwyer)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 7:59pm UTC](https://discuss.elastic.co/t/calculating-fields-in-logstash-on-upsert/274371 "2021-05-28T19:59:01Z")
Hi I have a pipeline used to enrich some json data from different sources to build up an index for a report. I would like to do a calculation based on 2 fields from the different sources, but since the sources are diffe… --- ## [API Endpoint for Elastic Stack Monitoring?](https://discuss.elastic.co/t/api-endpoint-for-elastic-stack-monitoring/274373)
**Author:** [@Sumit\_Gupta2](https://discuss.elastic.co/u/Sumit_Gupta2)\ **Replies:** 0\ **Last updated:** [May 28, 2021, 7:30pm UTC](https://discuss.elastic.co/t/api-endpoint-for-elastic-stack-monitoring/274373 "2021-05-28T19:30:28Z")
Which is the API Endpoint for Elastic Stack Monitoring? --- ## [Installing filebeat on dockers question](https://discuss.elastic.co/t/installing-filebeat-on-dockers-question/274370)
**Author:** [@Kaleb](https://discuss.elastic.co/u/Kaleb)\ **Replies:** 0\ **Last updated:** [May 28, 2021, 6:37pm UTC](https://discuss.elastic.co/t/installing-filebeat-on-dockers-question/274370 "2021-05-28T18:37:03Z")
I'm new Elastic Stack. I've been able to install Elasticsearch and Kibana via Docker using the instructions on elastic.co. However, I'm having some difficulty installing filebeats using the directions on elastic.co. Afte… --- ## [Multiline: flush downstream error java.io.IOException since upgrading from 6.8 to 7.12](https://discuss.elastic.co/t/multiline-flush-downstream-error-java-io-ioexception-since-upgrading-from-6-8-to-7-12/274236)
**Author:** [@soulless](https://discuss.elastic.co/u/soulless)\ **Replies:** 3\ **Last updated:** [May 28, 2021, 5:51pm UTC](https://discuss.elastic.co/t/multiline-flush-downstream-error-java-io-ioexception-since-upgrading-from-6-8-to-7-12/274236 "2021-05-28T17:51:49Z")
We have a multiline config that we use for capturing exceptions. codec =\> multiline { pattern =\> "^%{TIMESTAMP\_ISO8601}" negate =\> true what =\> "previous" } It appears to still be working, but since upgradi… --- ## [Kibana: representing evolution of boolean field in time](https://discuss.elastic.co/t/kibana-representing-evolution-of-boolean-field-in-time/274336)
**Author:** [@mostpha456](https://discuss.elastic.co/u/mostpha456)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 5:14pm UTC](https://discuss.elastic.co/t/kibana-representing-evolution-of-boolean-field-in-time/274336 "2021-05-28T17:14:47Z")
Hello, I am trying to represent the evolution over time of a boolean field. Do you have any suggestion how to do it ? Thank you --- ## [Using already provided time period](https://discuss.elastic.co/t/using-already-provided-time-period/274299)
**Author:** [@111238](https://discuss.elastic.co/u/111238)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 5:03pm UTC](https://discuss.elastic.co/t/using-already-provided-time-period/274299 "2021-05-28T17:03:35Z")
Hello. I have a bit of a problem here - i'm reseiving via syslog some data including uptime/downtime periods in format "HHhr:MMmin:SSsec". There's no problem parsing it in an object like "downtime.\
", but ho… --- ## [Get the name of the log file and create a tag or a custom field with that value](https://discuss.elastic.co/t/get-the-name-of-the-log-file-and-create-a-tag-or-a-custom-field-with-that-value/274344)
**Author:** [@mar-ro](https://discuss.elastic.co/u/mar-ro)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 4:58pm UTC](https://discuss.elastic.co/t/get-the-name-of-the-log-file-and-create-a-tag-or-a-custom-field-with-that-value/274344 "2021-05-28T16:58:12Z")
Good afternoon. I would like to know how I could in logstash create a tag (or a custom field) with a part of the value of the log.file.path.keyword field. Considering that this field stores the absolute path, I would be… --- ## [Filebeat to logstash encoding error](https://discuss.elastic.co/t/filebeat-to-logstash-encoding-error/274314)
**Author:** [@jezemery](https://discuss.elastic.co/u/jezemery)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 4:39pm UTC](https://discuss.elastic.co/t/filebeat-to-logstash-encoding-error/274314 "2021-05-28T16:39:54Z")
Hey, I'm attempting to use docker-elk from GitHub to set up a local ELK stack for testing purposes. I have the containers running fine. I also configured filebeat on my local machine to monitor nginx logs. For some rea… --- ## [Matching patterns from existing older templates ... will take precedence during new index creation](https://discuss.elastic.co/t/matching-patterns-from-existing-older-templates-will-take-precedence-during-new-index-creation/273764)
**Author:** [@6NMgfDwZ3](https://discuss.elastic.co/u/6NMgfDwZ3)\ **Replies:** 2\ **Last updated:** [May 28, 2021, 4:38pm UTC](https://discuss.elastic.co/t/matching-patterns-from-existing-older-templates-will-take-precedence-during-new-index-creation/273764 "2021-05-28T16:38:27Z")
Hello, Environment: Elasticsearch 7.12.1 (Basic License) Kibana 7.12.1 (Basic License) Filebeat 7.12.1 (Basic License) Ubuntu 21.04 I have an index template named myapp the index pattern myapp-\*. This index template… --- ## [How does elasticsearch rank matching results?](https://discuss.elastic.co/t/how-does-elasticsearch-rank-matching-results/274325)
**Author:** [@Later](https://discuss.elastic.co/u/Later)\ **Replies:** 5\ **Last updated:** [May 28, 2021, 3:36pm UTC](https://discuss.elastic.co/t/how-does-elasticsearch-rank-matching-results/274325 "2021-05-28T15:36:35Z")
Hello. Can you explain lucidly (My knowledge in this field is almost zero) how the search engine rank matching results when a user performs a search (I think this is called search engine algorithm)? What are ranking fac… --- ## [Postgresql -\> Jdbc input plugin -\> logstash - ERROR: relation does not exist](https://discuss.elastic.co/t/postgresql-jdbc-input-plugin-logstash-error-relation-does-not-exist/274296)
**Author:** [@FALEN](https://discuss.elastic.co/u/FALEN)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 3:03pm UTC](https://discuss.elastic.co/t/postgresql-jdbc-input-plugin-logstash-error-relation-does-not-exist/274296 "2021-05-28T15:03:06Z")
Hello everyone, It seems im missing something, but couldn't found anything related to this. Im using jdbc input plugin to collect some data from postgresql db. \[2021-05-28T03:24:12,032\]\[ERROR\]\[logstash.inputs.jdbc … --- ## [Import data from the TheDude server softweare to elastiscearch](https://discuss.elastic.co/t/import-data-from-the-thedude-server-softweare-to-elastiscearch/273977)
**Author:** [@Adriann](https://discuss.elastic.co/u/Adriann)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 2:27pm UTC](https://discuss.elastic.co/t/import-data-from-the-thedude-server-softweare-to-elastiscearch/273977 "2021-05-28T14:27:01Z")
Hello, I wonder If anyone managed to connect/fetch data capture by TheDude server into elasticserach. I do not see much about that on the internet. --- ## [Url.extension not populated](https://discuss.elastic.co/t/url-extension-not-populated/274278)
**Author:** [@peasead](https://discuss.elastic.co/u/peasead)\ **Replies:** 2\ **Last updated:** [May 28, 2021, 2:24pm UTC](https://discuss.elastic.co/t/url-extension-not-populated/274278 "2021-05-28T14:24:35Z")
I am trying to get Packetbeat to populate the url.extension field. It is in ECS and reported as an exported field for Packetbeat, but I cannot get it to work. In the below example, I would think it should be url.extensi… --- ## [Enhanced Table](https://discuss.elastic.co/t/enhanced-table/274306)
**Author:** [@Krunal](https://discuss.elastic.co/u/Krunal)\ **Replies:** 2\ **Last updated:** [May 28, 2021, 2:15pm UTC](https://discuss.elastic.co/t/enhanced-table/274306 "2021-05-28T14:15:41Z")
Hi @fbaligand i would like to use enhanced table in 7.13 version .... will you able help me? --- ## [Export Dashboard into ELK](https://discuss.elastic.co/t/export-dashboard-into-elk/274335)
**Author:** [@ITchandj](https://discuss.elastic.co/u/ITchandj)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 2:15pm UTC](https://discuss.elastic.co/t/export-dashboard-into-elk/274335 "2021-05-28T14:15:05Z")
Hello everybody... Please how I can export /import dashboard into ELk. I looking for the Workflow to do it. Thank you for answers in advance... --- ## [Tagging errors parsing json messages](https://discuss.elastic.co/t/tagging-errors-parsing-json-messages/274326)
**Author:** [@alfredo.deluca](https://discuss.elastic.co/u/alfredo.deluca)\ **Replies:** 2\ **Last updated:** [May 28, 2021, 2:10pm UTC](https://discuss.elastic.co/t/tagging-errors-parsing-json-messages/274326 "2021-05-28T14:10:46Z")
Hi all we have elastic cloud 7.12 with logstash 7.12 parsing json messages and send them to elasticsearch, but as we are having issues with mismatch type I d like to tag the error when that happen. Do you have any idea… --- ## [Is it possible to include other fields in the top level term aggregation](https://discuss.elastic.co/t/is-it-possible-to-include-other-fields-in-the-top-level-term-aggregation/273528)
**Author:** [@chapmantrain](https://discuss.elastic.co/u/chapmantrain)\ **Replies:** 4\ **Last updated:** [May 28, 2021, 2:01pm UTC](https://discuss.elastic.co/t/is-it-possible-to-include-other-fields-in-the-top-level-term-aggregation/273528 "2021-05-28T14:01:27Z")
I'm doing an aggregation on devices and calculating a metric on each. Is there a way to include other fields for the device in that top bucket like id, city, state? Each document in the vn-aggr index has those fields in … --- ## [Index problem after using curator](https://discuss.elastic.co/t/index-problem-after-using-curator/273827)
**Author:** [@Weathmious](https://discuss.elastic.co/u/Weathmious)\ **Replies:** 2\ **Last updated:** [May 28, 2021, 1:51pm UTC](https://discuss.elastic.co/t/index-problem-after-using-curator/273827 "2021-05-28T13:51:52Z")
Hello, I use a graylog server, with elasticsearch and kibana installed on it. I recently used curator to send indexes to aws S3, then to delete them, in order to save space on my server. All of this works fine, but I h… --- ## [Top 10 result in anomaly explorer tab (machine learning)](https://discuss.elastic.co/t/top-10-result-in-anomaly-explorer-tab-machine-learning/273231)
**Author:** [@abhishek30](https://discuss.elastic.co/u/abhishek30)\ **Replies:** 6\ **Last updated:** [May 28, 2021, 1:02pm UTC](https://discuss.elastic.co/t/top-10-result-in-anomaly-explorer-tab-machine-learning/273231 "2021-05-28T13:02:33Z")
Hi, I have a small setup of elk stack of 7.9.0. I created a ML job on the basis of my data and wanted to remove pagination from anomaly explorer tab. Also want to show top 10 anomaly in anomaly explorer. Also attached… --- ## [ES Scroll is returning duplicate/same results](https://discuss.elastic.co/t/es-scroll-is-returning-duplicate-same-results/274327)
**Author:** [@vedantkdesai](https://discuss.elastic.co/u/vedantkdesai)\ **Replies:** 7\ **Last updated:** [May 28, 2021, 12:59pm UTC](https://discuss.elastic.co/t/es-scroll-is-returning-duplicate-same-results/274327 "2021-05-28T12:59:15Z")
Hi Team, I was using the ES scroll mechanism for pagination. I found that we are getting the same/duplicate results in the subsequent scroll request made. Could you please help us here? Thanks in advance! --- ## [I am trying to change the format of date and time using pipeline but doesn't work](https://discuss.elastic.co/t/i-am-trying-to-change-the-format-of-date-and-time-using-pipeline-but-doesnt-work/274329)
**Author:** [@aswanth\_jabba](https://discuss.elastic.co/u/aswanth_jabba)\ **Replies:** 2\ **Last updated:** [May 28, 2021, 12:30pm UTC](https://discuss.elastic.co/t/i-am-trying-to-change-the-format-of-date-and-time-using-pipeline-but-doesnt-work/274329 "2021-05-28T12:30:54Z")
Here is the pipeline : PUT \_ingest/pipeline/dateformat { "description" : "To change Date format", "processors" : \[ { "date" : { "field" : "EndDate", "target\_field" : "newEndDate", "formats" : \["date\_time"\] } } … --- ## [Filebeat setup -e fails Error kibana fail to get version](https://discuss.elastic.co/t/filebeat-setup-e-fails-error-kibana-fail-to-get-version/274337)
**Author:** [@Peter\_Boos](https://discuss.elastic.co/u/Peter_Boos)\ **Replies:** 0\ **Last updated:** [May 28, 2021, 12:15pm UTC](https://discuss.elastic.co/t/filebeat-setup-e-fails-error-kibana-fail-to-get-version/274337 "2021-05-28T12:15:53Z")
I have Kibana and elastic search running, in a non clustered local environment. This has been running filebeat for maybe a year or a little less. Between a test machine and a Elk server. Its a pretty basic config, log… --- ## [Index pattern for data stream](https://discuss.elastic.co/t/index-pattern-for-data-stream/274264)
**Author:** [@anmmok](https://discuss.elastic.co/u/anmmok)\ **Replies:** 2\ **Last updated:** [May 28, 2021, 11:23am UTC](https://discuss.elastic.co/t/index-pattern-for-data-stream/274264 "2021-05-28T11:23:15Z")
I'm trying to visualize the data from the data stream in Kibana. So I’ve created an index template “api-logs” with the index pattern “api-logs\*” and add some documents to the new data stream: PUT api-logs/\_bulk { "cr… --- ## [Accessing event data in an Kibana Alerts](https://discuss.elastic.co/t/accessing-event-data-in-an-kibana-alerts/271541)
**Author:** [@dcamozzato](https://discuss.elastic.co/u/dcamozzato)\ **Replies:** 4\ **Last updated:** [May 28, 2021, 10:35am UTC](https://discuss.elastic.co/t/accessing-event-data-in-an-kibana-alerts/271541 "2021-05-28T10:35:06Z")
I am trying to create a Kibana alert for a syslog that is captured by logstash and minimally parsed. The format is this: \process\_name\[pid\]: message The "message" portion is not parsed further, but it … --- ## [Search for Aggregation results](https://discuss.elastic.co/t/search-for-aggregation-results/274030)
**Author:** [@krishnapss](https://discuss.elastic.co/u/krishnapss)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 10:33am UTC](https://discuss.elastic.co/t/search-for-aggregation-results/274030 "2021-05-28T10:33:18Z")
Hi, Could someone help me to achieve this functionality. Aggregation results are 500 to 5000 on different fields (data is in millions). I have used CompositeAggregation for pagination. But for user it is not great t… --- ## [Get a page of products but return all product\_tags for the whole unpaginated result set](https://discuss.elastic.co/t/get-a-page-of-products-but-return-all-product-tags-for-the-whole-unpaginated-result-set/274283)
**Author:** [@kiko](https://discuss.elastic.co/u/kiko)\ **Replies:** 5\ **Last updated:** [May 28, 2021, 10:28am UTC](https://discuss.elastic.co/t/get-a-page-of-products-but-return-all-product-tags-for-the-whole-unpaginated-result-set/274283 "2021-05-28T10:28:19Z")
Hi I have a query, which filters products and paginates the results (by ?size URL argument). Is there a way to return just a page (?size=12) of products, but return the available product\_tags for the whole lot (hits.tot… --- ## [Load Cloud Kibana dashboard in iframe with API KEY or credentials](https://discuss.elastic.co/t/load-cloud-kibana-dashboard-in-iframe-with-api-key-or-credentials/274307)
**Author:** [@kwoxer](https://discuss.elastic.co/u/kwoxer)\ **Replies:** 2\ **Last updated:** [May 28, 2021, 9:59am UTC](https://discuss.elastic.co/t/load-cloud-kibana-dashboard-in-iframe-with-api-key-or-credentials/274307 "2021-05-28T09:59:11Z")
Well there are different way how to do the auth in on-prem Elastic and Kibana: But is this somehow also possible in Elastic Cloud? I cannot find a way doing that. I simply need a iframe that links to a specific Dashb… --- ## [Omit pinned filters that are inappropriate to a dashboard?](https://discuss.elastic.co/t/omit-pinned-filters-that-are-inappropriate-to-a-dashboard/273901)
**Author:** [@GrahamHannington](https://discuss.elastic.co/u/GrahamHannington)\ **Replies:** 6\ **Last updated:** [May 28, 2021, 9:38am UTC](https://discuss.elastic.co/t/omit-pinned-filters-that-are-inappropriate-to-a-dashboard/273901 "2021-05-28T09:38:14Z")
I have developed a set of dashboards. At the top of each dashboard is a Markdown panel that contains a row of links to those dashboards. Some dashboards can be usefully filtered by the job\_name field, so it makes sense … --- ## [When trying to run filebeat setup Exiting: 1 error: error loading config file](https://discuss.elastic.co/t/when-trying-to-run-filebeat-setup-exiting-1-error-error-loading-config-file/274233)
**Author:** [@thrak](https://discuss.elastic.co/u/thrak)\ **Replies:** 1\ **Last updated:** [May 28, 2021, 9:12am UTC](https://discuss.elastic.co/t/when-trying-to-run-filebeat-setup-exiting-1-error-error-loading-config-file/274233 "2021-05-28T09:12:24Z")
Hello I just want to preface this with the fact that I am running an older version of beats, version 7.7.1, it's in production and we don't have a schedule yet to upgrade so we are stuck at this version for now. Wheneve… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1392) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1394)