# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1895 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1896 --- ## [How do I add a new field based on the contents of a file?](https://discuss.elastic.co/t/how-do-i-add-a-new-field-based-on-the-contents-of-a-file/243340)
**Author:** [@blueren](https://discuss.elastic.co/u/blueren)\ **Replies:** 1\ **Last updated:** [July 31, 2020, 11:14am UTC](https://discuss.elastic.co/t/how-do-i-add-a-new-field-based-on-the-contents-of-a-file/243340 "2020-07-31T11:14:38Z")
Hi, I'm currently using the netflow codec to decode netflow data and store in elastic. I want to introduce a new field called "site\_id". The site ID is going to be present in a file called /tmp/site.id This file will co… --- ## [Run elasticsearch code source without docker](https://discuss.elastic.co/t/run-elasticsearch-code-source-without-docker/243060)
**Author:** [@TheHunter1](https://discuss.elastic.co/u/TheHunter1)\ **Replies:** 4\ **Last updated:** [July 31, 2020, 10:03am UTC](https://discuss.elastic.co/t/run-elasticsearch-code-source-without-docker/243060 "2020-07-31T10:03:30Z")
Hello everybody, I am trying to run elasticsearch code source, and when I try to run the command ./gradlew assemble I am getting this exception: ======================================= Elasticsearch Build Hamster says… --- ## [Logstash jdbc input schedule](https://discuss.elastic.co/t/logstash-jdbc-input-schedule/243311)
**Author:** [@Luismg](https://discuss.elastic.co/u/Luismg)\ **Replies:** 2\ **Last updated:** [July 31, 2020, 9:57am UTC](https://discuss.elastic.co/t/logstash-jdbc-input-schedule/243311 "2020-07-31T09:57:55Z")
Hi, I'm currently trying to import data from a DB into ES using the jdbc input plugin. I want to run the pipeline during the night between 9pm and 4am. How can I define my schedule? I've tried '\*/1 21-04 \* \* \*' but th… --- ## [JDBC input type "store output into single field"](https://discuss.elastic.co/t/jdbc-input-type-store-output-into-single-field/243169)
**Author:** [@rguptarg](https://discuss.elastic.co/u/rguptarg)\ **Replies:** 2\ **Last updated:** [July 31, 2020, 7:29am UTC](https://discuss.elastic.co/t/jdbc-input-type-store-output-into-single-field/243169 "2020-07-31T07:29:39Z")
Hi, I am using jdbc input type and execute the oracle query, is it possible to store query output into a single variable/field ? Query Output : - ID 100 101 102 Above output store into three different JSON into ela… --- ## [Reading just once in Elastic (Multi-instance)](https://discuss.elastic.co/t/reading-just-once-in-elastic-multi-instance/243317)
**Author:** [@Ullas7](https://discuss.elastic.co/u/Ullas7)\ **Replies:** 1\ **Last updated:** [July 31, 2020, 9:15am UTC](https://discuss.elastic.co/t/reading-just-once-in-elastic-multi-instance/243317 "2020-07-31T09:15:22Z")
Is it possible to read only once in elastic? Assume I have 4 microservices which reads by timestamp, assuming four microservice has same query and timestamp as a query param. What I am trying to achieve is if any micros… --- ## [Error fetching fields for index pattern metricbeat-\* (ID: metricbeat-\*) Request Timeout](https://discuss.elastic.co/t/error-fetching-fields-for-index-pattern-metricbeat-id-metricbeat-request-timeout/241313)
**Author:** [@Peter\_Kasper](https://discuss.elastic.co/u/Peter_Kasper)\ **Replies:** 1\ **Last updated:** [July 31, 2020, 9:01am UTC](https://discuss.elastic.co/t/error-fetching-fields-for-index-pattern-metricbeat-id-metricbeat-request-timeout/241313 "2020-07-31T09:01:52Z")
Kibana, Elastic are 7.8, using Centos 7.7 and sending VM is Centos is 8.0, non secure single node. Whenever I run metricbeat setup, it loads the items but when i view the index pattern i get the error. It seems to load … --- ## [Filebeat doesn't sent the logs to elastic cloud (circuit\_breaking\_exception)](https://discuss.elastic.co/t/filebeat-doesnt-sent-the-logs-to-elastic-cloud-circuit-breaking-exception/243204)
**Author:** [@David\_Oceans](https://discuss.elastic.co/u/David_Oceans)\ **Replies:** 9\ **Last updated:** [July 31, 2020, 8:50am UTC](https://discuss.elastic.co/t/filebeat-doesnt-sent-the-logs-to-elastic-cloud-circuit-breaking-exception/243204 "2020-07-31T08:50:10Z")
Hi! I don't know why I get this error (agents filebeat in kubernetes cluster), if we increased the size of the cluster so that it had more performance and changed the indexes so that instead of 1 shard and 1 replica th… --- ## [Repository-gcs plugin 5.2.2 and Google Global HTTP batch endpoint](https://discuss.elastic.co/t/repository-gcs-plugin-5-2-2-and-google-global-http-batch-endpoint/243229)
**Author:** [@MMH](https://discuss.elastic.co/u/MMH)\ **Replies:** 1\ **Last updated:** [July 31, 2020, 8:01am UTC](https://discuss.elastic.co/t/repository-gcs-plugin-5-2-2-and-google-global-http-batch-endpoint/243229 "2020-07-31T08:01:56Z")
Hi, Google is deprecating Global HTTP batch endpoint on Aug 12th: https://www.google.com/url?q=https://developers.googleblog.com/2018/03/discontinuing-support-for-json-rpc-and.html On the ES 5.2 cluster, the repository… --- ## [The allocation of machines for different in ELK](https://discuss.elastic.co/t/the-allocation-of-machines-for-different-in-elk/243238)
**Author:** [@iammanmale](https://discuss.elastic.co/u/iammanmale)\ **Replies:** 2\ **Last updated:** [July 31, 2020, 7:41am UTC](https://discuss.elastic.co/t/the-allocation-of-machines-for-different-in-elk/243238 "2020-07-31T07:41:45Z")
I have 5 machines to form a elasticsearch cluster. How should I use them in order to maximise the performance? Should I install elasticsearch and logstash on each machine and install kibana on one of them? or should I i… --- ## [Logstash - if - indexOf](https://discuss.elastic.co/t/logstash-if-indexof/243314)
**Author:** [@daemon](https://discuss.elastic.co/u/daemon)\ **Replies:** 1\ **Last updated:** [July 31, 2020, 7:14am UTC](https://discuss.elastic.co/t/logstash-if-indexof/243314 "2020-07-31T07:14:05Z")
Java if("str||aaa").indexOf("||") logstash filter if? ps. What should I do? --- ## [Way to add proxy host settings for repository-gcs plugin in ES 7.5.2?](https://discuss.elastic.co/t/way-to-add-proxy-host-settings-for-repository-gcs-plugin-in-es-7-5-2/243309)
**Author:** [@nishant.saini](https://discuss.elastic.co/u/nishant.saini)\ **Replies:** 0\ **Last updated:** [July 31, 2020, 6:31am UTC](https://discuss.elastic.co/t/way-to-add-proxy-host-settings-for-repository-gcs-plugin-in-es-7-5-2/243309 "2020-07-31T06:31:16Z")
Is there a way to provide proxy.host and proxy.port for repository-gcs plugin. The same can be easily done for repository-s3 by setting the following in elasticsearch.yml: s3.client.default.protocol: http s3.client.defa… --- ## [Autocomplete that matches phrase prefix](https://discuss.elastic.co/t/autocomplete-that-matches-phrase-prefix/243318)
**Author:** [@emanu](https://discuss.elastic.co/u/emanu)\ **Replies:** 0\ **Last updated:** [July 31, 2020, 7:03am UTC](https://discuss.elastic.co/t/autocomplete-that-matches-phrase-prefix/243318 "2020-07-31T07:03:20Z")
Hello, I'm trying to achive an autocomplete that will match an exact phrase prefix. But I don't care where this phrase will be a.k.a in the start of the sentence, in the middle or the end. For exapmle: "Brown f" -\> "… --- ## [Seperate filebeats for different groups of servers](https://discuss.elastic.co/t/seperate-filebeats-for-different-groups-of-servers/242908)
**Author:** [@vape](https://discuss.elastic.co/u/vape)\ **Replies:** 4\ **Last updated:** [July 31, 2020, 6:25am UTC](https://discuss.elastic.co/t/seperate-filebeats-for-different-groups-of-servers/242908 "2020-07-31T06:25:17Z")
Hello, I am very new to ELK, please help me to clarify something here. So I have installed an instace of ELK all in one server for testing. I could ship the log to logstash from my servers using filebeat. I also enable … --- ## [Logstash - Filter - Error message(LogStash::ConfigurationError)](https://discuss.elastic.co/t/logstash-filter-error-message-logstash-configurationerror/243307)
**Author:** [@daemon](https://discuss.elastic.co/u/daemon)\ **Replies:** 1\ **Last updated:** [July 31, 2020, 6:06am UTC](https://discuss.elastic.co/t/logstash-filter-error-message-logstash-configurationerror/243307 "2020-07-31T06:06:09Z")
When I start Logstash I get the below error. filter{ mutate { } grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:requestTime}||%{WORD:type}||%{WORD:jobId}||%{DATA:edgeId}||%{GREEDYDATA:other}"} #match =\> { "mess… --- ## [Elasticsearch Delete Index Performance](https://discuss.elastic.co/t/elasticsearch-delete-index-performance/243301)
**Author:** [@es\_is\_hard](https://discuss.elastic.co/u/es_is_hard)\ **Replies:** 4\ **Last updated:** [July 31, 2020, 6:03am UTC](https://discuss.elastic.co/t/elasticsearch-delete-index-performance/243301 "2020-07-31T06:03:20Z")
Hello! I'm looking into the performance of the delete index API and had a few questions: How fast is deleting an Elasticsearch index? Any estimates on how long it would take to delete an index with 100s of GB of data? … --- ## [Elasticsearch service gets killed](https://discuss.elastic.co/t/elasticsearch-service-gets-killed/243039)
**Author:** [@Abdul\_Hameed](https://discuss.elastic.co/u/Abdul_Hameed)\ **Replies:** 7\ **Last updated:** [July 31, 2020, 5:14am UTC](https://discuss.elastic.co/t/elasticsearch-service-gets-killed/243039 "2020-07-31T05:14:07Z")
Hi, I have a setup of ELK on one server. Elasticsearch service and kibana run fine until I start the logstash service. When I start logstash, elasticsearch gets killed instantly and I see this error: elasticsearch sys… --- ## [To exclude around 350 fields in JSON in Logstash](https://discuss.elastic.co/t/to-exclude-around-350-fields-in-json-in-logstash/239019)
**Author:** [@tamilarasanbravo](https://discuss.elastic.co/u/tamilarasanbravo)\ **Replies:** 15\ **Last updated:** [July 31, 2020, 5:12am UTC](https://discuss.elastic.co/t/to-exclude-around-350-fields-in-json-in-logstash/239019 "2020-07-31T05:12:43Z")
Hi, I am trying to exclude many Keys in the JSON field. The data is actually of a Github payload. As the contributors are bit high, the details it carries is also high. For example, the total fields/kvp are around 720 b… --- ## [Security index goes yellow when 1 of 3 masters go down](https://discuss.elastic.co/t/security-index-goes-yellow-when-1-of-3-masters-go-down/243036)
**Author:** [@sudeera](https://discuss.elastic.co/u/sudeera)\ **Replies:** 7\ **Last updated:** [July 31, 2020, 5:07am UTC](https://discuss.elastic.co/t/security-index-goes-yellow-when-1-of-3-masters-go-down/243036 "2020-07-31T05:07:21Z")
Hi Support, We have 6 node elasticsearch cluster. Node 1 is master and Node 2 and 6 are master eligible. Node 3, 4, 5, 6 are data nodes. Node 1,2,3,4,5 have same rack\_id and Node6 has a different rack\_id. When we have… --- ## [Issue with Kibana Time Filter-7.5.2](https://discuss.elastic.co/t/issue-with-kibana-time-filter-7-5-2/241888)
**Author:** [@Sanjana17](https://discuss.elastic.co/u/Sanjana17)\ **Replies:** 1\ **Last updated:** [July 20, 2020, 10:19pm UTC](https://discuss.elastic.co/t/issue-with-kibana-time-filter-7-5-2/241888 "2020-07-20T22:19:43Z")
Hi, We have 2 servers set- one for Elasticsearch and the other for Kibana (ES version-7.5.1 and Kibana's version- 7.5.2 respectively). Both servers have their timezone set to IST. No filebeat or collector is being used … --- ## [After reindex, new index has same documents but data size is much less](https://discuss.elastic.co/t/after-reindex-new-index-has-same-documents-but-data-size-is-much-less/243068)
**Author:** [@mikewillis](https://discuss.elastic.co/u/mikewillis)\ **Replies:** 3\ **Last updated:** [July 31, 2020, 4:45am UTC](https://discuss.elastic.co/t/after-reindex-new-index-has-same-documents-but-data-size-is-much-less/243068 "2020-07-31T04:45:26Z")
As part of preparation for moving from Elasticsearch 6 to 7, I've reindexed an index which the Kibana Upgade Assistant flagged as "Index has more than one mapping type". The reindex completed without apparent error, the… --- ## [Need to understand difference between alias and rollover\_alias while creating index template for ILM implementation to multiple environments](https://discuss.elastic.co/t/need-to-understand-difference-between-alias-and-rollover-alias-while-creating-index-template-for-ilm-implementation-to-multiple-environments/242599)
**Author:** [@adeel109](https://discuss.elastic.co/u/adeel109)\ **Replies:** 0\ **Last updated:** [July 25, 2020, 11:19am UTC](https://discuss.elastic.co/t/need-to-understand-difference-between-alias-and-rollover-alias-while-creating-index-template-for-ilm-implementation-to-multiple-environments/242599 "2020-07-25T11:19:40Z")
I have one ILM policy named haproxy-ilm for all haprox logs. There are 4 different indices to which logs from each environment are written, i.e., dev, prestaging, staging, and production. Below is the configuration of l… --- ## [Can I filter logstash logs with my own log file fields?](https://discuss.elastic.co/t/can-i-filter-logstash-logs-with-my-own-log-file-fields/243300)
**Author:** [@Bhavin\_Varsur](https://discuss.elastic.co/u/Bhavin_Varsur)\ **Replies:** 0\ **Last updated:** [July 31, 2020, 4:02am UTC](https://discuss.elastic.co/t/can-i-filter-logstash-logs-with-my-own-log-file-fields/243300 "2020-07-31T04:02:52Z")
I have my own log file layout so i want to filter logstash filter in pipepines so that i can see my fields and view my own layout logs data . Is that Possible? if yes then how can i filterize ? --- ## [Geo shape self-intersection](https://discuss.elastic.co/t/geo-shape-self-intersection/242584)
**Author:** [@DenisDeis](https://discuss.elastic.co/u/DenisDeis)\ **Replies:** 4\ **Last updated:** [July 31, 2020, 3:56am UTC](https://discuss.elastic.co/t/geo-shape-self-intersection/242584 "2020-07-31T03:56:13Z")
I have the following feature: { "type": "Feature", "properties": { }, "geometry": { "type": "Polygon", "coordinates": \[\[ \[77.500, 13.500\], \[77.600, 13.500\], \[77.600, 13.400\], \[77.590, 13.400\], \[77.590, 13.390\], \[77.600,… --- ## [Elasticsearch master(8.0.0) nodes can't form into a cluster](https://discuss.elastic.co/t/elasticsearch-master-8-0-0-nodes-cant-form-into-a-cluster/243153)
**Author:** [@vsop\_479](https://discuss.elastic.co/u/vsop_479)\ **Replies:** 2\ **Last updated:** [July 31, 2020, 3:02am UTC](https://discuss.elastic.co/t/elasticsearch-master-8-0-0-nodes-cant-form-into-a-cluster/243153 "2020-07-31T03:02:20Z")
I configured three nodes with elasticsearch master branch（8.0.0）,but they can't form into a cluster. It seems they don't attempt to connection each other. Here is my configuration: node1: cluster.name: es-cluster node… --- ## [Capturing all messages in a log](https://discuss.elastic.co/t/capturing-all-messages-in-a-log/243261)
**Author:** [@eyts](https://discuss.elastic.co/u/eyts)\ **Replies:** 2\ **Last updated:** [July 31, 2020, 2:13am UTC](https://discuss.elastic.co/t/capturing-all-messages-in-a-log/243261 "2020-07-31T02:13:24Z")
I'm trying to capture all the messages in a log. For example, we get this log that contains 6 messages. "message":\["18.59.43 STC47873 +LRGPSEVRI\\u000007/29/20 18:59:43 I West Logged on to Tandem/Base24.","18.59.43 STC4… --- ## [Is there any way to find the "xxx" that produces the most geo data？](https://discuss.elastic.co/t/is-there-any-way-to-find-the-xxx-that-produces-the-most-geo-data/243142)
**Author:** [@wajika](https://discuss.elastic.co/u/wajika)\ **Replies:** 4\ **Last updated:** [July 31, 2020, 1:48am UTC](https://discuss.elastic.co/t/is-there-any-way-to-find-the-xxx-that-produces-the-most-geo-data/243142 "2020-07-31T01:48:13Z")
I did not find the visitor information for this location in Zhengzhou in the city list. I want to find out the "XX" serivce traffic information corresponding to geo\_localtion. serive.name geo\_localtion xxxx … --- ## [Trying to create too many scroll contexts for .security index](https://discuss.elastic.co/t/trying-to-create-too-many-scroll-contexts-for-security-index/243188)
**Author:** [@munkyu](https://discuss.elastic.co/u/munkyu)\ **Replies:** 0\ **Last updated:** [July 30, 2020, 8:59am UTC](https://discuss.elastic.co/t/trying-to-create-too-many-scroll-contexts-for-security-index/243188 "2020-07-30T08:59:19Z")
Hello, I'm using elastic stack 7.6.2 My Kibana failed to log in with superuser which was done until yesterday. I checked kibana and elasticsearch verbose log. Here's the log messages in kibana {"type":"log","@times… --- ## [Converting TimeStamp to epoch in LogStash](https://discuss.elastic.co/t/converting-timestamp-to-epoch-in-logstash/243168)
**Author:** [@Shanmuka\_Chowdary](https://discuss.elastic.co/u/Shanmuka_Chowdary)\ **Replies:** 9\ **Last updated:** [July 31, 2020, 12:22am UTC](https://discuss.elastic.co/t/converting-timestamp-to-epoch-in-logstash/243168 "2020-07-31T00:22:24Z")
Hi Team, I am trying to get some records from SQLServer through logstash. In the database the record is like this : 2002-09-03 04:00:00.000 , But it's saving in elastic like : "fileddate" : "2002-09-03T04:00:00.000Z" … --- ## [How to assign back bulk numbers of Unassigned shards back into cluster](https://discuss.elastic.co/t/how-to-assign-back-bulk-numbers-of-unassigned-shards-back-into-cluster/243093)
**Author:** [@suddhasil.sarkar](https://discuss.elastic.co/u/suddhasil.sarkar)\ **Replies:** 4\ **Last updated:** [July 30, 2020, 11:35pm UTC](https://discuss.elastic.co/t/how-to-assign-back-bulk-numbers-of-unassigned-shards-back-into-cluster/243093 "2020-07-30T23:35:57Z")
Hi All, any help is much appreciated. due to unexpected shutdown of my master nodes in elastic cluster I see more than 400 unassigned shards at this point, and wanted to check if there is any way I can assign them back a… --- ## [Build a visualization based on gerenralization of data](https://discuss.elastic.co/t/build-a-visualization-based-on-gerenralization-of-data/243265)
**Author:** [@Carlos\_Zottmann](https://discuss.elastic.co/u/Carlos_Zottmann)\ **Replies:** 2\ **Last updated:** [July 30, 2020, 9:45pm UTC](https://discuss.elastic.co/t/build-a-visualization-based-on-gerenralization-of-data/243265 "2020-07-30T21:45:09Z")
Hi, Everyone! Sorry for the newbie question! I have data of an inventory of my virtual servers. So far it has been a piece of cake building a visualization of my servers grouped by Operating Systems, so I´ve got a nice… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1894) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1896)