# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=960 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 961 --- ## [Best practices on generating queries with SDK vs storing queries as files](https://discuss.elastic.co/t/best-practices-on-generating-queries-with-sdk-vs-storing-queries-as-files/303328)
**Author:** [@darrenoc3](https://discuss.elastic.co/u/darrenoc3)\ **Replies:** 2\ **Last updated:** [April 27, 2022, 2:05pm UTC](https://discuss.elastic.co/t/best-practices-on-generating-queries-with-sdk-vs-storing-queries-as-files/303328 "2022-04-27T14:05:24Z")
Hi there, I was wondering what the communities opinion was on best practices for generating queries in a complex application. For our date histogram queries we have tried both: Store DSL queries as text files, use str… --- ## [Default values for missing fields](https://discuss.elastic.co/t/default-values-for-missing-fields/303333)
**Author:** [@tfinan](https://discuss.elastic.co/u/tfinan)\ **Replies:** 6\ **Last updated:** [April 27, 2022, 1:47pm UTC](https://discuss.elastic.co/t/default-values-for-missing-fields/303333 "2022-04-27T13:47:00Z")
Can I create a field in a mapping that will be indexed with some default value if that field is not present in the document being indexed? In other words, if I have a mapping with the field {"Rating": "integer"}, and th… --- ## [Replace value from one field in another field using an Elasticsearch Ingest Pipeline](https://discuss.elastic.co/t/replace-value-from-one-field-in-another-field-using-an-elasticsearch-ingest-pipeline/302963)
**Author:** [@DougR](https://discuss.elastic.co/u/DougR)\ **Replies:** 3\ **Last updated:** [April 27, 2022, 1:27pm UTC](https://discuss.elastic.co/t/replace-value-from-one-field-in-another-field-using-an-elasticsearch-ingest-pipeline/302963 "2022-04-27T13:27:03Z")
I need to remove a substring contained in one field from another field. However, I don't seem to be able to use the gsub or grok processors to accomplish this. E.g., I have the following fields: { "foo": "foo", … --- ## [Filebeat Harvester not starting](https://discuss.elastic.co/t/filebeat-harvester-not-starting/302891)
**Author:** [@Bhanu\_Praveen](https://discuss.elastic.co/u/Bhanu_Praveen)\ **Replies:** 3\ **Last updated:** [April 27, 2022, 1:05pm UTC](https://discuss.elastic.co/t/filebeat-harvester-not-starting/302891 "2022-04-27T13:05:36Z")
Hello, We have installed Filebeat on docker container connecting to logstah on linux server. It was working fine till a week back. AWS machine got restarted and post that, logstash is not harvesting new files. Harvester… --- ## [Parse Mcafee audit log file with Logstash](https://discuss.elastic.co/t/parse-mcafee-audit-log-file-with-logstash/302946)
**Author:** [@Xor44](https://discuss.elastic.co/u/Xor44)\ **Replies:** 4\ **Last updated:** [April 27, 2022, 12:46pm UTC](https://discuss.elastic.co/t/parse-mcafee-audit-log-file-with-logstash/302946 "2022-04-27T12:46:59Z")
Hi Folks , I need your help to find a right way to parse this non-structured file \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ Timestamp : 19/Apr/2022:05:50:02.771 +0200 User … --- ## [Flattened fields with query\_string and wildcards](https://discuss.elastic.co/t/flattened-fields-with-query-string-and-wildcards/303400)
**Author:** [@dor](https://discuss.elastic.co/u/dor)\ **Replies:** 0\ **Last updated:** [April 27, 2022, 12:29pm UTC](https://discuss.elastic.co/t/flattened-fields-with-query-string-and-wildcards/303400 "2022-04-27T12:29:46Z")
Hi, I'm using Elasticsearch 8, and I'm having weird behaviors with flattened fields and query strings with wildcards. Let's have this test index: PUT test\_index { "mappings": { "properties": { "flattened\_f… --- ## [Elastalert-kibana plugin for kibana 6.8.23](https://discuss.elastic.co/t/elastalert-kibana-plugin-for-kibana-6-8-23/303006)
**Author:** [@Vishnu\_Vijay](https://discuss.elastic.co/u/Vishnu_Vijay)\ **Replies:** 1\ **Last updated:** [April 27, 2022, 11:58am UTC](https://discuss.elastic.co/t/elastalert-kibana-plugin-for-kibana-6-8-23/303006 "2022-04-27T11:58:24Z")
Hello, We have upgraded our kibana to 6.8.23. We require compatible elastalert plugin for connecting to our elastalert system. Please let us know, whether the elastalert plugin for kibana 6.8.23 is available --- ## [Performance issue in record insertion in elastic search on upgrading to ElasticSearch version from 7.1.1 to 7.16.3 in linux](https://discuss.elastic.co/t/performance-issue-in-record-insertion-in-elastic-search-on-upgrading-to-elasticsearch-version-from-7-1-1-to-7-16-3-in-linux/303394)
**Author:** [@rahul.kushwah](https://discuss.elastic.co/u/rahul.kushwah)\ **Replies:** 1\ **Last updated:** [April 27, 2022, 11:11am UTC](https://discuss.elastic.co/t/performance-issue-in-record-insertion-in-elastic-search-on-upgrading-to-elasticsearch-version-from-7-1-1-to-7-16-3-in-linux/303394 "2022-04-27T11:11:49Z")
We are using default transport client (i.e transport client 7.16.3) for communication with Elasticsearch. Previously on Elasticsearch 7.1.1 it takes around 3 sec to insert 5000 records but after upgrading it to Elasticse… --- ## [Geo-IP Map visualization](https://discuss.elastic.co/t/geo-ip-map-visualization/302914)
**Author:** [@Bo-Wyatt](https://discuss.elastic.co/u/Bo-Wyatt)\ **Replies:** 10\ **Last updated:** [April 27, 2022, 11:07am UTC](https://discuss.elastic.co/t/geo-ip-map-visualization/302914 "2022-04-27T11:07:03Z")
Hello, I have different fields with Geo-IP data: Latitude, Longitude, Country Code, Continent code and more. How can I build a map based on that? I've tried the "Create new Maps" option but all the visualization options … --- ## [Can I transform rows into columns or vice versa with the help of pivot transformation](https://discuss.elastic.co/t/can-i-transform-rows-into-columns-or-vice-versa-with-the-help-of-pivot-transformation/302617)
**Author:** [@sukhwinder\_kumar](https://discuss.elastic.co/u/sukhwinder_kumar)\ **Replies:** 3\ **Last updated:** [April 27, 2022, 10:46am UTC](https://discuss.elastic.co/t/can-i-transform-rows-into-columns-or-vice-versa-with-the-help-of-pivot-transformation/302617 "2022-04-27T10:46:40Z")
Can I transform rows into columns or vice versa with the help of pivot transformations or with the help of any other visualization. Attached image is my requirement: --- ## [Uniqueness in filter aggregations](https://discuss.elastic.co/t/uniqueness-in-filter-aggregations/303272)
**Author:** [@Lucass](https://discuss.elastic.co/u/Lucass)\ **Replies:** 2\ **Last updated:** [April 27, 2022, 10:12am UTC](https://discuss.elastic.co/t/uniqueness-in-filter-aggregations/303272 "2022-04-27T10:12:41Z")
Hi, kindly ask you ideas to solve following: imagine having users filling personal surveys, eg. filling what they usually smoke. Each user can decide one or more possibilities from the checklist survey. On results, I am… --- ## [Logstash cipher plugin w/ KMS](https://discuss.elastic.co/t/logstash-cipher-plugin-w-kms/303374)
**Author:** [@MrAtheist](https://discuss.elastic.co/u/MrAtheist)\ **Replies:** 0\ **Last updated:** [April 27, 2022, 9:33am UTC](https://discuss.elastic.co/t/logstash-cipher-plugin-w-kms/303374 "2022-04-27T09:33:19Z")
Hello, Not sure if anyone has got the cipher filter plugin to work with kms? would love some assistance on it if there is any hack for this... --- ## [Filebeat is connected to Kafka, but it doesn't send Log](https://discuss.elastic.co/t/filebeat-is-connected-to-kafka-but-it-doesnt-send-log/303302)
**Author:** [@CemG](https://discuss.elastic.co/u/CemG)\ **Replies:** 6\ **Last updated:** [April 27, 2022, 9:41am UTC](https://discuss.elastic.co/t/filebeat-is-connected-to-kafka-but-it-doesnt-send-log/303302 "2022-04-27T09:41:37Z")
Hello, It is been 2 days that I have this issue. I would like to send log from a computer to elastic by the following way : Filebeat → Kafka → Logstash → Elasticsearch But Filebeat do not send log to Kafka and I don'… --- ## [Auto\_flush\_interval in multiline codec plugin errors out](https://discuss.elastic.co/t/auto-flush-interval-in-multiline-codec-plugin-errors-out/303376)
**Author:** [@ritesh811](https://discuss.elastic.co/u/ritesh811)\ **Replies:** 0\ **Last updated:** [April 27, 2022, 9:40am UTC](https://discuss.elastic.co/t/auto-flush-interval-in-multiline-codec-plugin-errors-out/303376 "2022-04-27T09:40:55Z")
Hi, I am using auto\_flush\_interval in the multiline codec plugin. It works fine for all the events but when it reaches the end of file, it is throwing grokparsefailure error. And when I remove auto\_flush\_interval, there… --- ## [ES Source code compilation](https://discuss.elastic.co/t/es-source-code-compilation/303137)
**Author:** [@casterQ](https://discuss.elastic.co/u/casterQ)\ **Replies:** 3\ **Last updated:** [April 27, 2022, 8:26am UTC](https://discuss.elastic.co/t/es-source-code-compilation/303137 "2022-04-27T08:26:50Z")
Should the parameter be specified with - E or - D? When i used -D like this ：-Des.path.conf=/Users/caster/Desktop/elasticsearch-7.10.2-demo/config，i got: ERROR: D is not a recognized option When i used -E like this ：-E… --- ## [Getting error while creating template in kibana 5.3](https://discuss.elastic.co/t/getting-error-while-creating-template-in-kibana-5-3/303289)
**Author:** [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Replies:** 2\ **Last updated:** [April 27, 2022, 8:22am UTC](https://discuss.elastic.co/t/getting-error-while-creating-template-in-kibana-5-3/303289 "2022-04-27T08:22:16Z")
Hello team, I am using below query in kibana 5.3 to create template but getting below error: { "error": { "root\_cause": \[ { "type": "action\_request\_validation\_exception", "reason": "Validati… --- ## [How i can store hot, warm and cold indices on serval disk partition?](https://discuss.elastic.co/t/how-i-can-store-hot-warm-and-cold-indices-on-serval-disk-partition/303356)
**Author:** [@Mohsin\_Ashraf](https://discuss.elastic.co/u/Mohsin_Ashraf)\ **Replies:** 2\ **Last updated:** [April 27, 2022, 8:10am UTC](https://discuss.elastic.co/t/how-i-can-store-hot-warm-and-cold-indices-on-serval-disk-partition/303356 "2022-04-27T08:10:28Z")
Hi, I want to store hot, warm and cold indices in different disk partitions on the same node. for example, I have different partitions on the same node like 100GB, 250GB and 300GB. So, how can I store indices across dif… --- ## [Elastic ILM rollover not working as expected](https://discuss.elastic.co/t/elastic-ilm-rollover-not-working-as-expected/303012)
**Author:** [@alytkowski](https://discuss.elastic.co/u/alytkowski)\ **Replies:** 2\ **Last updated:** [April 27, 2022, 7:45am UTC](https://discuss.elastic.co/t/elastic-ilm-rollover-not-working-as-expected/303012 "2022-04-27T07:45:13Z")
Hello, I use data streams for my input data and the expected behavior of the ILM policy is to delete data which is 10 days old. To do this, I setup a policy that should create a new index every day and move the data to … --- ## [Wildcards with query\_string query and custom analyzer](https://discuss.elastic.co/t/wildcards-with-query-string-query-and-custom-analyzer/303330)
**Author:** [@mej101](https://discuss.elastic.co/u/mej101)\ **Replies:** 1\ **Last updated:** [April 27, 2022, 5:15am UTC](https://discuss.elastic.co/t/wildcards-with-query-string-query-and-custom-analyzer/303330 "2022-04-27T05:15:16Z")
I have a query\_string query I'm using with wildcards and a custom analyzer. The query behaves as expected for some words and not for others. Here's the query: "query\_string": { "query": "Abdulla\*", "an… --- ## [Parse Date from Script\_Fields Params](https://discuss.elastic.co/t/parse-date-from-script-fields-params/303329)
**Author:** [@prashant\_zeta](https://discuss.elastic.co/u/prashant_zeta)\ **Replies:** 1\ **Last updated:** [April 27, 2022, 3:53am UTC](https://discuss.elastic.co/t/parse-date-from-script-fields-params/303329 "2022-04-27T03:53:34Z")
I am trying to find the elapsed time elapsed time: duration of time b/w now and createdTime To pass the current\_time (ie now) I have added it to the params, and I can access that in the source field by writing params… --- ## [Monitoring or Distributed 2 Elastic stack to 1 elastic](https://discuss.elastic.co/t/monitoring-or-distributed-2-elastic-stack-to-1-elastic/303338)
**Author:** [@alipujaistopo](https://discuss.elastic.co/u/alipujaistopo)\ **Replies:** 3\ **Last updated:** [April 27, 2022, 3:48am UTC](https://discuss.elastic.co/t/monitoring-or-distributed-2-elastic-stack-to-1-elastic/303338 "2022-04-27T03:48:31Z")
I have two branch offices that use elaticsearch. but I want both of them to be monitored centrally at the head office. Can it? and how? thank you --- ## [Setup elastic fleet server with wildcard cert](https://discuss.elastic.co/t/setup-elastic-fleet-server-with-wildcard-cert/303301)
**Author:** [@philshilling](https://discuss.elastic.co/u/philshilling)\ **Replies:** 0\ **Last updated:** [April 26, 2022, 2:52pm UTC](https://discuss.elastic.co/t/setup-elastic-fleet-server-with-wildcard-cert/303301 "2022-04-26T14:52:35Z")
I've looked at the documentation and i'm unsure where to start to setup a fleet server using a custom wildcard certificate. I currently have the certificate in PFX format. Any suggestion, advice or how to guides would be… --- ## [Referencing captured groups inside yaml file](https://discuss.elastic.co/t/referencing-captured-groups-inside-yaml-file/303337)
**Author:** [@Draken](https://discuss.elastic.co/u/Draken)\ **Replies:** 0\ **Last updated:** [April 27, 2022, 1:36am UTC](https://discuss.elastic.co/t/referencing-captured-groups-inside-yaml-file/303337 "2022-04-27T01:36:23Z")
Hello, I'm running the postgresql module in metricbeat with the statement metricset active, i want to be able to determine the type of the query (insert, update, select or delete), so i add an add\_fields processor with … --- ## [Docker logs includes unreadable in Kibana](https://discuss.elastic.co/t/docker-logs-includes-unreadable-in-kibana/303196)
**Author:** [@dev9](https://discuss.elastic.co/u/dev9)\ **Replies:** 4\ **Last updated:** [April 27, 2022, 1:56am UTC](https://discuss.elastic.co/t/docker-logs-includes-unreadable-in-kibana/303196 "2022-04-27T01:56:55Z")
Hi, I am using filebeat version 7.17.3 running on Ubuntu 18.04.6 LTS to ship docker logs using filebeat installation on the host OS. I am not using a filebeat container. I noticed a couple of things that I would like t… --- ## [I have an issue with the elastic 8.1.3 settings](https://discuss.elastic.co/t/i-have-an-issue-with-the-elastic-8-1-3-settings/303102)
**Author:** [@Mahmoud](https://discuss.elastic.co/u/Mahmoud)\ **Replies:** 3\ **Last updated:** [April 27, 2022, 1:17am UTC](https://discuss.elastic.co/t/i-have-an-issue-with-the-elastic-8-1-3-settings/303102 "2022-04-27T01:17:43Z")
I have a startup problem with elastic version 8.1.3 i trying to configure elastic cluster its have single node Apr 24 16:50:40 elastic systemd-entrypoint\[3419\]: ERROR: Elasticsearch did not exit normally - check the log… --- ## [Any relationship between refresh\_interval & bulk\_write task time?](https://discuss.elastic.co/t/any-relationship-between-refresh-interval-bulk-write-task-time/303112)
**Author:** [@linkerc](https://discuss.elastic.co/u/linkerc)\ **Replies:** 4\ **Last updated:** [April 27, 2022, 1:13am UTC](https://discuss.elastic.co/t/any-relationship-between-refresh-interval-bulk-write-task-time/303112 "2022-04-27T01:13:31Z")
Is it normal to have longer bulk\_write time if I increase the refresh\_interval to say 300 seconds? I have noticed an index bulk\_write is taking more than 4 seconds. Not sure if this is normal or cause for concern. I g… --- ## [After upgrading to 7.17.3 or 8.1.3 from 7.16.3 I get cannot get crawler to start with logs](https://discuss.elastic.co/t/after-upgrading-to-7-17-3-or-8-1-3-from-7-16-3-i-get-cannot-get-crawler-to-start-with-logs/303332)
**Author:** [@Tony\_Powell](https://discuss.elastic.co/u/Tony_Powell)\ **Replies:** 1\ **Last updated:** [April 27, 2022, 12:52am UTC](https://discuss.elastic.co/t/after-upgrading-to-7-17-3-or-8-1-3-from-7-16-3-i-get-cannot-get-crawler-to-start-with-logs/303332 "2022-04-27T00:52:39Z")
in my helm values file I have the following and I was going to comment out the second log to see if it makes a difference processors: # Add Kubernetes metadata | Filebeat Reference \[7.7\] | Elastic - add\_kubernetes\_met… --- ## [Logstash UDP input keeps losing data](https://discuss.elastic.co/t/logstash-udp-input-keeps-losing-data/303215)
**Author:** [@co88liwan](https://discuss.elastic.co/u/co88liwan)\ **Replies:** 2\ **Last updated:** [April 27, 2022, 12:24am UTC](https://discuss.elastic.co/t/logstash-udp-input-keeps-losing-data/303215 "2022-04-27T00:24:41Z")
Hi there, We encountered an issue that Logstash input UDP plugin consistently losing data. From tcpdump analysis we know the data indeed reached the VM, I also tested with Logstash file output plugin and I saw the data… --- ## [Kibana server is not ready yet](https://discuss.elastic.co/t/kibana-server-is-not-ready-yet/303252)
**Author:** [@Achraf\_MENSI](https://discuss.elastic.co/u/Achraf_MENSI)\ **Replies:** 1\ **Last updated:** [April 26, 2022, 11:01pm UTC](https://discuss.elastic.co/t/kibana-server-is-not-ready-yet/303252 "2022-04-26T23:01:48Z")
Hello , i depoyed a multi node cluster with 3 azure vm, the cluster worked on ES and nodes are connected. but i can't work on them with kibana " Kibana server is not ready yet " \[2022-04-25T00:00:01,091\]\[WARN \]\[o.e.c.… --- ## [Java 8.1 bulk request](https://discuss.elastic.co/t/java-8-1-bulk-request/302423)
**Author:** [@3xil3](https://discuss.elastic.co/u/3xil3)\ **Replies:** 6\ **Last updated:** [April 26, 2022, 10:40pm UTC](https://discuss.elastic.co/t/java-8-1-bulk-request/302423 "2022-04-26T22:40:34Z")
Hi, I'm having trouble doing a bulk index request using the java 8.1.1 client. I get an unclear error message and I'm basically stuck. String test= "{ \\"test\\",\\"test\\"}"; final InputStream jsonInputStr… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=959) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=961)