# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1150 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1151 --- ## [How to access nested field in ingest node pipeline?](https://discuss.elastic.co/t/how-to-access-nested-field-in-ingest-node-pipeline/290231)
**Author:** [@sidharth\_vijayakumar](https://discuss.elastic.co/u/sidharth_vijayakumar)\ **Replies:** 0\ **Last updated:** [November 26, 2021, 5:51am UTC](https://discuss.elastic.co/t/how-to-access-nested-field-in-ingest-node-pipeline/290231 "2021-11-26T05:51:27Z")
I need to access the field "kind" and it's nested I have shared how the data in json looks like. I want to use a set processor in ingest node pipelines and add a new field if this has a value pod. How do I achieve this? … --- ## [How to ignore a field from search if the field is not part of mappings?](https://discuss.elastic.co/t/how-to-ignore-a-field-from-search-if-the-field-is-not-part-of-mappings/290228)
**Author:** [@Raghavan\_TV](https://discuss.elastic.co/u/Raghavan_TV)\ **Replies:** 0\ **Last updated:** [November 26, 2021, 4:28am UTC](https://discuss.elastic.co/t/how-to-ignore-a-field-from-search-if-the-field-is-not-part-of-mappings/290228 "2021-11-26T04:28:00Z")
The document in elastic has field mappings only for the fields catalog and @type . now if a user sends an non-existing field ( countryCode ) in the search, is there a way in elastic to instruct to ignore the field in the… --- ## [Need help to create logstash json.conf file](https://discuss.elastic.co/t/need-help-to-create-logstash-json-conf-file/290225)
**Author:** [@hagaluly](https://discuss.elastic.co/u/hagaluly)\ **Replies:** 3\ **Last updated:** [November 26, 2021, 4:10am UTC](https://discuss.elastic.co/t/need-help-to-create-logstash-json-conf-file/290225 "2021-11-26T04:10:42Z")
i have jenkins job that outputs a json file. i want to know how can i build the logstash conf file in order to export it to Elasticsearch i have tried different configs but always failing on parsing errors. json file … --- ## [How to config esrally metric store so that "meta" of rally-metrics-\* can record "took" of each query](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024)
**Author:** [@kingswan](https://discuss.elastic.co/u/kingswan)\ **Replies:** 7\ **Last updated:** [November 26, 2021, 3:02am UTC](https://discuss.elastic.co/t/how-to-config-esrally-metric-store-so-that-meta-of-rally-metrics-can-record-took-of-each-query/290024 "2021-11-26T03:02:32Z")
I use metric store to save esrally results. Some of my results, the "meta" includes "took" of each query: "meta": { "plugins": \[ "test\_plugin" \], "attribute\_xpack.installed"… --- ## [How to compare fields of subevents in Logstash](https://discuss.elastic.co/t/how-to-compare-fields-of-subevents-in-logstash/290227)
**Author:** [@Rahul\_Singh1](https://discuss.elastic.co/u/Rahul_Singh1)\ **Replies:** 0\ **Last updated:** [November 26, 2021, 2:39am UTC](https://discuss.elastic.co/t/how-to-compare-fields-of-subevents-in-logstash/290227 "2021-11-26T02:39:35Z")
Hi All, I am new to ELK and i need one help with Logstash. I am getting log from source and I have splitted that into sub events. Events have one field in common and I want to compare current field to previous one. P… --- ## [How to implement deflate (best compression)?](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984)
**Author:** [@Linuxuser](https://discuss.elastic.co/u/Linuxuser)\ **Replies:** 23\ **Last updated:** [November 25, 2021, 11:00pm UTC](https://discuss.elastic.co/t/how-to-implement-deflate-best-compression/289984 "2021-11-25T23:00:24Z")
Hello, kindly help me out. My elastic version is 6.3 how can i implement best compression for the log stored by elastic. Thanks --- ## [Unable to update ui setting request failed with status code: 429](https://discuss.elastic.co/t/unable-to-update-ui-setting-request-failed-with-status-code-429/290001)
**Author:** [@ftester](https://discuss.elastic.co/u/ftester)\ **Replies:** 5\ **Last updated:** [November 25, 2021, 10:59pm UTC](https://discuss.elastic.co/t/unable-to-update-ui-setting-request-failed-with-status-code-429/290001 "2021-11-25T22:59:19Z")
Hello guys, The Scinario: I have 4 dockers, elasticsearch, kibana, filebeat, dvwa. So basically, I should attack dvwa and see the apache logs on kibana dashboard. I use Ubuntu server 20.4 to set up the docker on it an… --- ## [How to make WebStorm / TypeScript completion work?](https://discuss.elastic.co/t/how-to-make-webstorm-typescript-completion-work/290222)
**Author:** [@davestewart](https://discuss.elastic.co/u/davestewart)\ **Replies:** 0\ **Last updated:** [November 25, 2021, 10:09pm UTC](https://discuss.elastic.co/t/how-to-make-webstorm-typescript-completion-work/290222 "2021-11-25T22:09:09Z")
Hi there, I'm pretty new to Elastic, been using it on and off for about a year, using the JavaScript client. I finally seem to be at the stage where I understand how to build complex queries, bool, match\_phrase\_prefix,… --- ## [Logstash-codec-plain is broken dependency in Logstash version 7.15.2](https://discuss.elastic.co/t/logstash-codec-plain-is-broken-dependency-in-logstash-version-7-15-2/290212)
**Author:** [@Saida\_Meftah](https://discuss.elastic.co/u/Saida_Meftah)\ **Replies:** 1\ **Last updated:** [November 25, 2021, 8:23pm UTC](https://discuss.elastic.co/t/logstash-codec-plain-is-broken-dependency-in-logstash-version-7-15-2/290212 "2021-11-25T20:23:08Z")
I am trying to install Logstash-output-loki plugin, the logstash-codec-plain causes a dependency conflict, looks like this plugin is a broken dependency. My Logstash version is 7.15.2, this problem is not happening in L… --- ## [Drop metadata fields using drop\_fields processor in elastic agent config](https://discuss.elastic.co/t/drop-metadata-fields-using-drop-fields-processor-in-elastic-agent-config/290215)
**Author:** [@clewo](https://discuss.elastic.co/u/clewo)\ **Replies:** 0\ **Last updated:** [November 25, 2021, 8:04pm UTC](https://discuss.elastic.co/t/drop-metadata-fields-using-drop-fields-processor-in-elastic-agent-config/290215 "2021-11-25T20:04:19Z")
Hi guys, I've issues to drop some metadata fields in the elastic agent config using the drop\_fields processor. I tried to configure the processor in the agent.yml inputs section: inputs: - id: b21f2d23-1bf5-4e0b-852… --- ## [Stats aggregation returning inconsistent results](https://discuss.elastic.co/t/stats-aggregation-returning-inconsistent-results/289377)
**Author:** [@Doug\_Coburn](https://discuss.elastic.co/u/Doug_Coburn)\ **Replies:** 1\ **Last updated:** [November 25, 2021, 7:27pm UTC](https://discuss.elastic.co/t/stats-aggregation-returning-inconsistent-results/289377 "2021-11-25T19:27:00Z")
Using version 7.9.2 curl -X POST 172.31.26.133:9200/5cab6838-5c3a-455d-9486-a708f6cb20a7/\_search --data-raw '{"aggs":{"Stats":{"stats":{"field":"Date & Time"}}},"size":0}' -H 'content-type: application/json' { "too… --- ## [How to configure ELK for geoip](https://discuss.elastic.co/t/how-to-configure-elk-for-geoip/289590)
**Author:** [@root\_linux](https://discuss.elastic.co/u/root_linux)\ **Replies:** 21\ **Last updated:** [November 25, 2021, 6:50pm UTC](https://discuss.elastic.co/t/how-to-configure-elk-for-geoip/289590 "2021-11-25T18:50:21Z")
Hi, I want to configure ELK stack. Elasticsearch, Kibana and Logstash will configure at one server and there will be multiple filebeat client. How can I configure for geoip? What need to configure in Elasticsearch, Kiba… --- ## [Group data and create a new index whit it](https://discuss.elastic.co/t/group-data-and-create-a-new-index-whit-it/290208)
**Author:** [@lino\_alejandro](https://discuss.elastic.co/u/lino_alejandro)\ **Replies:** 0\ **Last updated:** [November 25, 2021, 6:19pm UTC](https://discuss.elastic.co/t/group-data-and-create-a-new-index-whit-it/290208 "2021-11-25T18:19:54Z")
I have this data in my index { "filekey": "book1", "word": "carlos", "page": 1 }, { "filekey": "book1", "word": "license", "page": 1 }, { "filekey": "book1", "word": "carlos", … --- ## [Filebeat sending duplicates events](https://discuss.elastic.co/t/filebeat-sending-duplicates-events/289998)
**Author:** [@josepho](https://discuss.elastic.co/u/josepho)\ **Replies:** 1\ **Last updated:** [November 25, 2021, 4:56pm UTC](https://discuss.elastic.co/t/filebeat-sending-duplicates-events/289998 "2021-11-25T16:56:15Z")
Hi, Im running Elastic/Logstash/Filebeat v 7.15. I have beats installed on a few debian 9 machines, shipping log files. Everything is working well, except that we get duplicate entries. Seems like filebeat keep reading … --- ## [Csv filter : how to deal with missing field?](https://discuss.elastic.co/t/csv-filter-how-to-deal-with-missing-field/290147)
**Author:** [@Travis](https://discuss.elastic.co/u/Travis)\ **Replies:** 2\ **Last updated:** [November 25, 2021, 4:09pm UTC](https://discuss.elastic.co/t/csv-filter-how-to-deal-with-missing-field/290147 "2021-11-25T16:09:17Z")
Hello ! I have the following data : \<188\>Nov 25 07:38:31 172.28.192.111 ts-swsan-p11 raslogd: 2021/11/25-07:38:31, \[MAPS-1003\], 28921, WWN 10:00:88:94:71:c4:c0:e0 | FID 128, WARNING, ts-swsan-p11, SW11\_TS\_VPLEX\_P01\_E1… --- ## [Issue when create custom index with filebeat in elasticsearch](https://discuss.elastic.co/t/issue-when-create-custom-index-with-filebeat-in-elasticsearch/287976)
**Author:** [@inazo](https://discuss.elastic.co/u/inazo)\ **Replies:** 1\ **Last updated:** [November 25, 2021, 3:44pm UTC](https://discuss.elastic.co/t/issue-when-create-custom-index-with-filebeat-in-elasticsearch/287976 "2021-11-25T15:44:03Z")
Hello, I need to create custom index for my filebeat agent for each host. Need to separate the data collected for each host. So i'll do this config in my filebeat.yml on the monitored host, after a large number of rese… --- ## [Is it possible to mask field name in mappings?](https://discuss.elastic.co/t/is-it-possible-to-mask-field-name-in-mappings/290193)
**Author:** [@ddoroshenko](https://discuss.elastic.co/u/ddoroshenko)\ **Replies:** 1\ **Last updated:** [November 25, 2021, 3:21pm UTC](https://discuss.elastic.co/t/is-it-possible-to-mask-field-name-in-mappings/290193 "2021-11-25T15:21:58Z")
Hello! I'd like to set all fields type as keyword except some custom fields. "properties": { "field1": { "type" : "text" }, "\*": { "type" : "keyword" } } doesn't work ant the fields mapping dynamically. Is there… --- ## [Heartbeat monitoring for a particular URL fails](https://discuss.elastic.co/t/heartbeat-monitoring-for-a-particular-url-fails/289219)
**Author:** [@maheshe](https://discuss.elastic.co/u/maheshe)\ **Replies:** 1\ **Last updated:** [November 25, 2021, 2:58pm UTC](https://discuss.elastic.co/t/heartbeat-monitoring-for-a-particular-url-fails/289219 "2021-11-25T14:58:35Z")
Hi, We are using heartbeat 7.9 client to dump the monitoring logs to Elasticsearch/Kibana 7.11 version. One particular url is returning monitor.status down even though the site is up and is returning 200. This is the s… --- ## [Pagination with PIT query required on each request?](https://discuss.elastic.co/t/pagination-with-pit-query-required-on-each-request/290144)
**Author:** [@tamis-laan](https://discuss.elastic.co/u/tamis-laan)\ **Replies:** 1\ **Last updated:** [November 25, 2021, 2:49pm UTC](https://discuss.elastic.co/t/pagination-with-pit-query-required-on-each-request/290144 "2021-11-25T14:49:44Z")
When you start a query and request a PIT ID do you need to send the query along in subsequent requests? Is the search index and query not cached and referenced by the PIT id? And if the query is required, what happens … --- ## [Concatenating array objects in elasticsearch transform aggregations](https://discuss.elastic.co/t/concatenating-array-objects-in-elasticsearch-transform-aggregations/289980)
**Author:** [@kbirk](https://discuss.elastic.co/u/kbirk)\ **Replies:** 2\ **Last updated:** [November 25, 2021, 2:42pm UTC](https://discuss.elastic.co/t/concatenating-array-objects-in-elasticsearch-transform-aggregations/289980 "2021-11-25T14:42:12Z")
I'm currently using an Elasticsearch transform in order to "merge" similar documents. It works very well, however I've run into an issue attempting to concatenate fields that contain an array of objects. For example, I… --- ## [Process in stall state while shutdown](https://discuss.elastic.co/t/process-in-stall-state-while-shutdown/290014)
**Author:** [@Vinutha\_A\_H](https://discuss.elastic.co/u/Vinutha_A_H)\ **Replies:** 3\ **Last updated:** [November 25, 2021, 2:03pm UTC](https://discuss.elastic.co/t/process-in-stall-state-while-shutdown/290014 "2021-11-25T14:03:44Z")
We have a script to stop the logstash, which inturn runs 'kill -15 '. This script is being called as part of system init file which in turn stops as part of server reboot. However, recently we see issues in prod while s… --- ## [Elasticsearch cluster load balancing](https://discuss.elastic.co/t/elasticsearch-cluster-load-balancing/290187)
**Author:** [@dooger21](https://discuss.elastic.co/u/dooger21)\ **Replies:** 0\ **Last updated:** [November 25, 2021, 1:56pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-load-balancing/290187 "2021-11-25T13:56:41Z")
Hi everyone, Please I need your help with the following: I have an Elasticsearch cluster (3 nodes) in version 7.14 configured and working, as it is in a productive environment, the following safeguards were implemented … --- ## [In NEST how to make the match query conditional?](https://discuss.elastic.co/t/in-nest-how-to-make-the-match-query-conditional/290178)
**Author:** [@S\_Zaman](https://discuss.elastic.co/u/S_Zaman)\ **Replies:** 0\ **Last updated:** [November 25, 2021, 1:10pm UTC](https://discuss.elastic.co/t/in-nest-how-to-make-the-match-query-conditional/290178 "2021-11-25T13:10:52Z")
How to make the match conditionally added/removed in c# .NET NEST? var response = client.Search\(s =\> s .Index("mytweetindex") //or specify index via settings.DefaultIndex("mytweetindex"); .From(0) .Si… --- ## [Replacing \_id field](https://discuss.elastic.co/t/replacing-id-field/290167)
**Author:** [@mangeshs](https://discuss.elastic.co/u/mangeshs)\ **Replies:** 2\ **Last updated:** [November 25, 2021, 1:10pm UTC](https://discuss.elastic.co/t/replacing-id-field/290167 "2021-11-25T13:10:39Z")
there is \_id field auto generated by Elasticsearch which differ the each doc So I have an unique field to set as \_id field I want to that field as \_id for doc in Logstash conf How do to that --- ## [User keyword and zero\_terms\_query in the same query](https://discuss.elastic.co/t/user-keyword-and-zero-terms-query-in-the-same-query/290177)
**Author:** [@fabry](https://discuss.elastic.co/u/fabry)\ **Replies:** 0\ **Last updated:** [November 25, 2021, 1:06pm UTC](https://discuss.elastic.co/t/user-keyword-and-zero-terms-query-in-the-same-query/290177 "2021-11-25T13:06:52Z")
Hi we would like to build a query like the following ... ... bool: { must: \[ match : { myfield.myfield2 : { query: "myvalue", zero\_terms\_query: "all" } } \] } ... .… --- ## [Synchronization of Elasticsearch and relational databases with Logstash and JDBC disconnected EC2](https://discuss.elastic.co/t/synchronization-of-elasticsearch-and-relational-databases-with-logstash-and-jdbc-disconnected-ec2/290140)
**Author:** [@Soongu\_Jeon](https://discuss.elastic.co/u/Soongu_Jeon)\ **Replies:** 2\ **Last updated:** [November 25, 2021, 11:43am UTC](https://discuss.elastic.co/t/synchronization-of-elasticsearch-and-relational-databases-with-logstash-and-jdbc-disconnected-ec2/290140 "2021-11-25T11:43:00Z")
hi. After reading the above site article, I set the schedule for jdbc and mysql every 5 minutes, but the next day I turned on the background and checked, and the server was disconnected. How can I solve this problem? --- ## [Encoded bytes are incorrect length issue](https://discuss.elastic.co/t/encoded-bytes-are-incorrect-length-issue/289547)
**Author:** [@Akaren](https://discuss.elastic.co/u/Akaren)\ **Replies:** 3\ **Last updated:** [November 25, 2021, 10:57am UTC](https://discuss.elastic.co/t/encoded-bytes-are-incorrect-length-issue/289547 "2021-11-25T10:57:32Z")
Hi, my Elasticsearch 7.7.1 sometimes returns encoded bytes are incorrect length error with 400. It's only sometimes, when I do the same search again, it suddently works. Can you tell me the meaning? What can be couse of… --- ## [Nodes restarting with shards initializing](https://discuss.elastic.co/t/nodes-restarting-with-shards-initializing/289489)
**Author:** [@Ayca](https://discuss.elastic.co/u/Ayca)\ **Replies:** 3\ **Last updated:** [November 25, 2021, 10:06am UTC](https://discuss.elastic.co/t/nodes-restarting-with-shards-initializing/289489 "2021-11-25T10:06:01Z")
Our elastic cluster had some nodes restarted with a network outage and when they were back, we were monitoring the shard reallocations. The question is, even on unaffected nodes were there shards in the state of "INITI… --- ## [Error with Logstash: error=\>"cannot link Java class io.netty.handler.ssl.OpenSsl, probable missing dependency: io.netty.internal.tcnative.SSLContext.setCipherSuite](https://discuss.elastic.co/t/error-with-logstash-error-cannot-link-java-class-io-netty-handler-ssl-openssl-probable-missing-dependency-io-netty-internal-tcnative-sslcontext-setciphersuite/290155)
**Author:** [@sarath.sarepaka](https://discuss.elastic.co/u/sarath.sarepaka)\ **Replies:** 0\ **Last updated:** [November 25, 2021, 10:03am UTC](https://discuss.elastic.co/t/error-with-logstash-error-cannot-link-java-class-io-netty-handler-ssl-openssl-probable-missing-dependency-io-netty-internal-tcnative-sslcontext-setciphersuite/290155 "2021-11-25T10:03:03Z")
We've upgraded our ELK stack from 6.4 version to 6.8.20 version. Elasticsearch and Kibana were upgraded without any issues. But we observed that Logstash service is restarting continuously after the upgradation. Elastic… --- ## [Machine learning error: persistent task is awating node assignment](https://discuss.elastic.co/t/machine-learning-error-persistent-task-is-awating-node-assignment/290043)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 3\ **Last updated:** [November 25, 2021, 9:54am UTC](https://discuss.elastic.co/t/machine-learning-error-persistent-task-is-awating-node-assignment/290043 "2021-11-25T09:54:43Z")
Hello, I hope this message finds Elastic community and their loved ones safe and healthy. I have a three node cluster: Node-1 and Node-2 hold data & carry out computation: both have has 24 GB of RAM with jvm set to… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1149) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1151)