# Elasticsearch **URL:** https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1493 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1494 --- ## [Grok format for \[10-May-2017 16:16:33\] - || Response = 400](https://discuss.elastic.co/t/grok-format-for-10-may-2017-1633-response-400/155409)
**Author:** [@satyaban](https://discuss.elastic.co/u/satyaban)\ **Replies:** 3\ **Last updated:** [November 6, 2018, 2:54pm UTC](https://discuss.elastic.co/t/grok-format-for-10-may-2017-1633-response-400/155409 "2018-11-06T14:54:48Z")
I want to use this pattern in logstash but unable to find pattern for \[10-May-2017 16:16:33\] timestamp Kingly help me if anyone knew it. --- ## [How to add a new field with a default value to all docs in an index?](https://discuss.elastic.co/t/how-to-add-a-new-field-with-a-default-value-to-all-docs-in-an-index/155555)
**Author:** [@elhamroshan](https://discuss.elastic.co/u/elhamroshan)\ **Replies:** 4\ **Last updated:** [November 6, 2018, 2:54pm UTC](https://discuss.elastic.co/t/how-to-add-a-new-field-with-a-default-value-to-all-docs-in-an-index/155555 "2018-11-06T14:54:11Z")
Hi, I need to add a new field with null default value to all documents in an index. I know the following query, but how to add a default value to it? PUT customer/\_mapping/\_doc { "properties": { "major": { "type":… --- ## [Extracting raw logs from ELK to SPlunk](https://discuss.elastic.co/t/extracting-raw-logs-from-elk-to-splunk/155560)
**Author:** [@mguttula](https://discuss.elastic.co/u/mguttula)\ **Replies:** 0\ **Last updated:** [November 6, 2018, 2:36pm UTC](https://discuss.elastic.co/t/extracting-raw-logs-from-elk-to-splunk/155560 "2018-11-06T14:36:41Z")
Hi, We have a requirement in our company where security team is planning to use Splunk module for Audit related records and we want use ELK as enterprise solution. Given this scenario, how can we extract security relat… --- ## [2 nodes with different Cluster UUID issue causing issue in Monitoring](https://discuss.elastic.co/t/2-nodes-with-different-cluster-uuid-issue-causing-issue-in-monitoring/155474)
**Author:** [@askids](https://discuss.elastic.co/u/askids)\ **Replies:** 4\ **Last updated:** [November 6, 2018, 1:56pm UTC](https://discuss.elastic.co/t/2-nodes-with-different-cluster-uuid-issue-causing-issue-in-monitoring/155474 "2018-11-06T13:56:37Z")
hi, We had a 1 node cluster. So we wanted to temporarily expand it to 2 node cluster. I know that minimum 3 nodes are needed for full sized cluster. But keeping that aside, I have a different issue. We are running 5.5.1… --- ## [Multiple hosts on curator](https://discuss.elastic.co/t/multiple-hosts-on-curator/155534)
**Author:** [@hage42](https://discuss.elastic.co/u/hage42)\ **Replies:** 1\ **Last updated:** [November 6, 2018, 1:15pm UTC](https://discuss.elastic.co/t/multiple-hosts-on-curator/155534 "2018-11-06T13:15:43Z")
When setting more than one host (all participating in the same cluster) in curator.yml, will curator perform / run on every single of that hosts, or will curator make sure to be executed only on one of those ? --- ## [Unclear documentation for xpack....flush\_interval](https://discuss.elastic.co/t/unclear-documentation-for-xpack-flush-interval/155537)
**Author:** [@jacobot](https://discuss.elastic.co/u/jacobot)\ **Replies:** 2\ **Last updated:** [November 6, 2018, 1:12pm UTC](https://discuss.elastic.co/t/unclear-documentation-for-xpack-flush-interval/155537 "2018-11-06T13:12:40Z")
The latest docs https://www.elastic.co/guide/en/x-pack/current/auditing.html for the options of xpack I want to set a value for xpack.security.audit.index.flush\_interval but is it milliseconds, seconds or a string lik… --- ## [Hadoop-ES Structured Streaming-write Sink Partition Issue](https://discuss.elastic.co/t/hadoop-es-structured-streaming-write-sink-partition-issue/155550)
**Author:** [@grifith](https://discuss.elastic.co/u/grifith)\ **Replies:** 0\ **Last updated:** [November 6, 2018, 1:11pm UTC](https://discuss.elastic.co/t/hadoop-es-structured-streaming-write-sink-partition-issue/155550 "2018-11-06T13:11:42Z")
Hello, We have a structured streaming application which write the data from Spark to ES.The flow is follows- Kinesis Stream( 8 shards)----\>(Databricks Library for Kinesis-readstream)----\>Spark structured Stream(transfo… --- ## [Watcher - Alert on any new data](https://discuss.elastic.co/t/watcher-alert-on-any-new-data/155539)
**Author:** [@kpover](https://discuss.elastic.co/u/kpover)\ **Replies:** 1\ **Last updated:** [November 6, 2018, 12:57pm UTC](https://discuss.elastic.co/t/watcher-alert-on-any-new-data/155539 "2018-11-06T12:57:21Z")
Hello, I'm just getting started with Watcher. As a test I would like to create a very simple watcher that alerts whenever new data is added to an index. To provide some context, we have an index that is taking alerts fr… --- ## [Elasticsearch Error Localization](https://discuss.elastic.co/t/elasticsearch-error-localization/155495)
**Author:** [@NIKHIL\_DUBEY](https://discuss.elastic.co/u/NIKHIL_DUBEY)\ **Replies:** 3\ **Last updated:** [November 6, 2018, 12:53pm UTC](https://discuss.elastic.co/t/elasticsearch-error-localization/155495 "2018-11-06T12:53:59Z")
I have an application which is using elasticsearch. In some cases, I am passing the error message from elasticsearch directly to the frontend. For example, when trying to create a repository for data archiving, I get an … --- ## [How to access \_id from Painless in Query context?](https://discuss.elastic.co/t/how-to-access-id-from-painless-in-query-context/155211)
**Author:** [@m9aertner](https://discuss.elastic.co/u/m9aertner)\ **Replies:** 2\ **Last updated:** [November 6, 2018, 10:45am UTC](https://discuss.elastic.co/t/how-to-access-id-from-painless-in-query-context/155211 "2018-11-06T10:45:49Z")
I would like to sanity-check indexed data. Is it possible to access the \_id field in a Painless script condition, in query context? That is, something like (ES 5.3.x): { "query": { "bool": { "must": \[ … --- ## [Painless empty block?](https://discuss.elastic.co/t/painless-empty-block/155213)
**Author:** [@m9aertner](https://discuss.elastic.co/u/m9aertner)\ **Replies:** 2\ **Last updated:** [November 6, 2018, 10:29am UTC](https://discuss.elastic.co/t/painless-empty-block/155213 "2018-11-06T10:29:19Z")
I find it odd that Painless appears to not allow brackets-only empty blocks, e.g. for if statements (ES 5.3). The following works OK: "script" : { "script" : { "inline": "if(doc !… --- ## [Disable field index](https://discuss.elastic.co/t/disable-field-index/153950)
**Author:** [@hitesh-inbetween](https://discuss.elastic.co/u/hitesh-inbetween)\ **Replies:** 6\ **Last updated:** [November 6, 2018, 9:53am UTC](https://discuss.elastic.co/t/disable-field-index/153950 "2018-11-06T09:53:39Z")
Is it possible to toggle searchable field. My index contains around 3000 fields and 1200k documents, at times I see memory issues. On further analysis, I found that due to large number of fields, not all fields are avai… --- ## [Aggregate results before scoring](https://discuss.elastic.co/t/aggregate-results-before-scoring/155044)
**Author:** [@G82EtHAjnY4U](https://discuss.elastic.co/u/G82EtHAjnY4U)\ **Replies:** 1\ **Last updated:** [November 6, 2018, 9:41am UTC](https://discuss.elastic.co/t/aggregate-results-before-scoring/155044 "2018-11-06T09:41:12Z")
I'm storing an event calendar where events are often repeated at different times and locations. For simplicity, let's say I'm storing these fields: title description when where uid (I defined uid to be a identifier fo… --- ## [Cannot connect to elasticsearch cluster when it runs as a system service, works if I run it as me](https://discuss.elastic.co/t/cannot-connect-to-elasticsearch-cluster-when-it-runs-as-a-system-service-works-if-i-run-it-as-me/155166)
**Author:** [@Oddo\_Da](https://discuss.elastic.co/u/Oddo_Da)\ **Replies:** 1\ **Last updated:** [November 6, 2018, 9:17am UTC](https://discuss.elastic.co/t/cannot-connect-to-elasticsearch-cluster-when-it-runs-as-a-system-service-works-if-i-run-it-as-me/155166 "2018-11-06T09:17:58Z")
Hello, I have a centos machine where I can run elasticsearch as a service (via systemctl). My elasticsearch.yml states that network.host, transport.host, http.host all be 0.0.0.0. However, when I try to run basic code us… --- ## [Server requirements](https://discuss.elastic.co/t/server-requirements/155516)
**Author:** [@Dragos\_Moruz](https://discuss.elastic.co/u/Dragos_Moruz)\ **Replies:** 0\ **Last updated:** [November 6, 2018, 9:13am UTC](https://discuss.elastic.co/t/server-requirements/155516 "2018-11-06T09:13:10Z")
A minimum configuration to ingest from 15 devices - 12 reads/second per device - 180 reads / second in total from 15 devices Approx - 1,4GB per device on per day To clarify 1 read is a pack of data sent from the devic… --- ## [Elastic Search Data format conversion](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603)
**Author:** [@abhinav.dwivedi](https://discuss.elastic.co/u/abhinav.dwivedi)\ **Replies:** 10\ **Last updated:** [November 6, 2018, 9:11am UTC](https://discuss.elastic.co/t/elastic-search-data-format-conversion/154603 "2018-11-06T09:11:42Z")
Team I have 1 TB Elastic search data across 4 node cluster is there a way we can convert that data into CSV or Json? Regards Abhinav D --- ## [Script-expert-scoring question](https://discuss.elastic.co/t/script-expert-scoring-question/155385)
**Author:** [@CristianoFerreira](https://discuss.elastic.co/u/CristianoFerreira)\ **Replies:** 1\ **Last updated:** [November 6, 2018, 9:01am UTC](https://discuss.elastic.co/t/script-expert-scoring-question/155385 "2018-11-06T09:01:56Z")
Greetings, I have a question regarding the script-expert-scoring plugin example. Is there a way to get the total number of hits in this part of the code: @Override public ScoreScript newInstance(LeafReaderCont… --- ## [Https://discuss.elastic.co/t/an-routing-missing-exception-is-obtained-when-reindex-sets-the-routing-value/155140/3](https://discuss.elastic.co/t/https-discuss-elastic-co-t-an-routing-missing-exception-is-obtained-when-reindex-sets-the-routing-value-155140-3/155507)
**Author:** [@lwpk110](https://discuss.elastic.co/u/lwpk110)\ **Replies:** 0\ **Last updated:** [November 6, 2018, 8:32am UTC](https://discuss.elastic.co/t/https-discuss-elastic-co-t-an-routing-missing-exception-is-obtained-when-reindex-sets-the-routing-value-155140-3/155507 "2018-11-06T08:32:58Z")
Continuing the discussion from How to mitigate "\[routing\] is missing for join field \[join\_field\]: --- ## [Using third party API to modify/encrypt value of existing fields of document](https://discuss.elastic.co/t/using-third-party-api-to-modify-encrypt-value-of-existing-fields-of-document/155478)
**Author:** [@kauzhik](https://discuss.elastic.co/u/kauzhik)\ **Replies:** 2\ **Last updated:** [November 6, 2018, 7:25am UTC](https://discuss.elastic.co/t/using-third-party-api-to-modify-encrypt-value-of-existing-fields-of-document/155478 "2018-11-06T07:25:46Z")
I have to encrypt/tokenize an existing field (Integer type) of documents using a specific API that encrypts/decrypts a given value. I have a million document in my index and need to encrypt the value of that particular f… --- ## [Hardware for searching and visualizing 3 billion documents?](https://discuss.elastic.co/t/hardware-for-searching-and-visualizing-3-billion-documents/155355)
**Author:** [@Sjaak01](https://discuss.elastic.co/u/Sjaak01)\ **Replies:** 3\ **Last updated:** [November 6, 2018, 5:04am UTC](https://discuss.elastic.co/t/hardware-for-searching-and-visualizing-3-billion-documents/155355 "2018-11-06T05:04:14Z")
Hi, I know this might be a bit of a vague question, but hopefully I can get some insight into the hardware I'll need. I'm collected netflow data and use Kibana to make visualizations. My dashboard has about 20 visualiz… --- ## [How to estimate the max query throughput](https://discuss.elastic.co/t/how-to-estimate-the-max-query-throughput/155475)
**Author:** [@Jianzhou\_Z](https://discuss.elastic.co/u/Jianzhou_Z)\ **Replies:** 0\ **Last updated:** [November 6, 2018, 4:14am UTC](https://discuss.elastic.co/t/how-to-estimate-the-max-query-throughput/155475 "2018-11-06T04:14:30Z")
My query QPS is less than 100, and was wondering what is the max possible throughput for the query. The index has 16 shards (1 primary + 1 replica). The cluster has 11 nodes. Each node has 32 CPUs, 70G memory, 900G SSD … --- ## [How to disable typed\_keys using java dsl?](https://discuss.elastic.co/t/how-to-disable-typed-keys-using-java-dsl/155471)
**Author:** [@Larry\_Meadors](https://discuss.elastic.co/u/Larry_Meadors)\ **Replies:** 0\ **Last updated:** [November 6, 2018, 12:05am UTC](https://discuss.elastic.co/t/how-to-disable-typed-keys-using-java-dsl/155471 "2018-11-06T00:05:37Z")
I'm using the Java DSL to create and execute a query - when using the default request options, the typed\_keys header appears to be set, and I can't find a way to get rid of it - it's not in the headers, but the behavior … --- ## [Disk space per node in for ES cluster is not balanced across the nodes](https://discuss.elastic.co/t/disk-space-per-node-in-for-es-cluster-is-not-balanced-across-the-nodes/155459)
**Author:** [@Akshay\_Deshpande](https://discuss.elastic.co/u/Akshay_Deshpande)\ **Replies:** 3\ **Last updated:** [November 5, 2018, 10:35pm UTC](https://discuss.elastic.co/t/disk-space-per-node-in-for-es-cluster-is-not-balanced-across-the-nodes/155459 "2018-11-05T22:35:26Z")
I have a ES cluster with a large amount of nodes. There are thousands of shards. The indices are time based. The data is ingested during work-week, almost 24x7. The problem I am facing is some of the nodes in cluster… --- ## [Elastic Search Partial Search](https://discuss.elastic.co/t/elastic-search-partial-search/155318)
**Author:** [@Nush](https://discuss.elastic.co/u/Nush)\ **Replies:** 7\ **Last updated:** [November 5, 2018, 9:39pm UTC](https://discuss.elastic.co/t/elastic-search-partial-search/155318 "2018-11-05T21:39:36Z")
Hey, I am trying to create a partial search with elasticsearch but it's not working then I expected. But full word its work fine. $esQuery = \[ 'index' =\> $category, 'type' =\> $category, 'body' =\> \[ "from" =\> ($… --- ## [Weird match query behaviour](https://discuss.elastic.co/t/weird-match-query-behaviour/155443)
**Author:** [@mathee](https://discuss.elastic.co/u/mathee)\ **Replies:** 1\ **Last updated:** [November 5, 2018, 9:12pm UTC](https://discuss.elastic.co/t/weird-match-query-behaviour/155443 "2018-11-05T21:12:58Z")
Hi everyone! I have a question regarding the query match (bare in mind I am new to elasticsearch). When I run on localhost and create a test index, put some data and do (followed the getting started video) GET /bank/\_… --- ## [Elasticsearch document size](https://discuss.elastic.co/t/elasticsearch-document-size/155456)
**Author:** [@rajisankar](https://discuss.elastic.co/u/rajisankar)\ **Replies:** 1\ **Last updated:** [November 5, 2018, 9:11pm UTC](https://discuss.elastic.co/t/elasticsearch-document-size/155456 "2018-11-05T21:11:33Z")
Hi, I know there has been multiple threads on finding size of a document in Elasticsearch. I have not found a conclusive answer for ELK 6.x. For my benchmarking purposes, I would like to know the average size of docume… --- ## [Import from BigQuery to Elasticsearch](https://discuss.elastic.co/t/import-from-bigquery-to-elasticsearch/155454)
**Author:** [@melissachang](https://discuss.elastic.co/u/melissachang)\ **Replies:** 0\ **Last updated:** [November 5, 2018, 8:42pm UTC](https://discuss.elastic.co/t/import-from-bigquery-to-elasticsearch/155454 "2018-11-05T20:42:31Z")
This is a follow-up to Import from BigQuery to Elasticsearch? In https://github.com/DataBiosphere/data-explorer-indexers, we're developing a solution that is pretty fast: Copy BigQuery table to JSON files on GCS. It o… --- ## ["unable to install test security manager" when using ESTestCase](https://discuss.elastic.co/t/unable-to-install-test-security-manager-when-using-estestcase/150031)
**Author:** [@uwieuwe4](https://discuss.elastic.co/u/uwieuwe4)\ **Replies:** 6\ **Last updated:** [November 5, 2018, 8:36pm UTC](https://discuss.elastic.co/t/unable-to-install-test-security-manager-when-using-estestcase/150031 "2018-11-05T20:36:52Z")
I am testing my code using ESTestCase. When running the test I get the following error: java.lang.RuntimeException: unable to install test security manager at org.elasticsearch.bootstrap.BootstrapForTesting.\(B… --- ## [Installing Elasticsearch on Azure Vm Scaleset](https://discuss.elastic.co/t/installing-elasticsearch-on-azure-vm-scaleset/153481)
**Author:** [@vijayakrishna.rg](https://discuss.elastic.co/u/vijayakrishna.rg)\ **Replies:** 13\ **Last updated:** [November 5, 2018, 8:34pm UTC](https://discuss.elastic.co/t/installing-elasticsearch-on-azure-vm-scaleset/153481 "2018-11-05T20:34:56Z")
i am trying to Configure Elasticsearch on Red Hat Enterprise Linux Server release 7.3 (Maipo) with Azure VM Scaleset, service is unable to start, can someone please help me with it, Thanks. sudo systemctl status elastic… --- ## [Input elasticsearch log files into ELK](https://discuss.elastic.co/t/input-elasticsearch-log-files-into-elk/155362)
**Author:** [@cowensel](https://discuss.elastic.co/u/cowensel)\ **Replies:** 3\ **Last updated:** [November 5, 2018, 8:24pm UTC](https://discuss.elastic.co/t/input-elasticsearch-log-files-into-elk/155362 "2018-11-05T20:24:43Z")
Is there a way to either pass the elasticsearch logs into the Elk Stack or output it into a readable format. I have neglected the elasticsearch log files and would like them to be put out in a readable format for users … [Previous page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1492) [Next page](https://discuss.elastic.co/c/elastic-stack/elasticsearch/6.md?page=1494)