# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1500 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1501 --- ## [Logstash file input/output with multiline codec](https://discuss.elastic.co/t/logstash-file-input-output-with-multiline-codec/267552)
**Author:** [@liuke](https://discuss.elastic.co/u/liuke)\ **Replies:** 8\ **Last updated:** [March 19, 2021, 3:03pm UTC](https://discuss.elastic.co/t/logstash-file-input-output-with-multiline-codec/267552 "2021-03-19T15:03:50Z")
Hello, i'm new in logstash and i'm encountering some troubles on building a config that works properly with my needings. What i want to do is: -Input files from multiples folders (application logs, so with java excepti… --- ## [\[ERROR\]\[logstash.config.sourceloader\] No configuration found in the configured sources](https://discuss.elastic.co/t/error-logstash-config-sourceloader-no-configuration-found-in-the-configured-sources/267798)
**Author:** [@sourabhjain104](https://discuss.elastic.co/u/sourabhjain104)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 2:57pm UTC](https://discuss.elastic.co/t/error-logstash-config-sourceloader-no-configuration-found-in-the-configured-sources/267798 "2021-03-19T14:57:13Z")
Hello, I am very new to ELK, I am trying to setup ELK (7.11.2) in my local windows 10 laptop. I am able to successfully run Elasticsearch and Kibana but I have issue with Logstash. I tried to figure out this issue by … --- ## [Add local json file into elasticsearch](https://discuss.elastic.co/t/add-local-json-file-into-elasticsearch/267819)
**Author:** [@feyn\_mak](https://discuss.elastic.co/u/feyn_mak)\ **Replies:** 1\ **Last updated:** [March 19, 2021, 2:48pm UTC](https://discuss.elastic.co/t/add-local-json-file-into-elasticsearch/267819 "2021-03-19T14:48:13Z")
Hi, how do i add a json file e.g { "0": "passage 1", "1":"passage 2", "2":"passage 3" } into elasticsearch? Thank you so much for your help! --- ## [Multiple spliting in a single string using logstash](https://discuss.elastic.co/t/multiple-spliting-in-a-single-string-using-logstash/267817)
**Author:** [@Kiran\_Gupta](https://discuss.elastic.co/u/Kiran_Gupta)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 2:12pm UTC](https://discuss.elastic.co/t/multiple-spliting-in-a-single-string-using-logstash/267817 "2021-03-19T14:12:37Z")
I had a string called regions\_of\_interest which holds the following value "California - San Diego,New Zealand - Auckland,Asia - Hong Kong" I want to split this string multiple times, First with the comma(,) and then wit… --- ## [Search in all given filed but it return only match filed. so i can send those value on front end](https://discuss.elastic.co/t/search-in-all-given-filed-but-it-return-only-match-filed-so-i-can-send-those-value-on-front-end/267770)
**Author:** [@mayank\_hardiya](https://discuss.elastic.co/u/mayank_hardiya)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 9:16am UTC](https://discuss.elastic.co/t/search-in-all-given-filed-but-it-return-only-match-filed-so-i-can-send-those-value-on-front-end/267770 "2021-03-19T09:16:53Z")
My query - client.search({ index: indexname, body: { query: { multi\_match: { query: req.query.search, fields: \["description","name","hostname","username","category\_name"\], } }, } }, it return - \[ { "\_index"… --- ## [Latest Timestamp Machine learning doesn't change](https://discuss.elastic.co/t/latest-timestamp-machine-learning-doesnt-change/267362)
**Author:** [@Angelos](https://discuss.elastic.co/u/Angelos)\ **Replies:** 5\ **Last updated:** [March 19, 2021, 1:52pm UTC](https://discuss.elastic.co/t/latest-timestamp-machine-learning-doesnt-change/267362 "2021-03-19T13:52:14Z")
Elasticsearch: Platinum 7.10.1 Kibana: 7.10.1 Hi, I am creating Machine learning jobs in real time. The questions are: What kind of information Latest Timestamp provides? and Why the Latest Timestamp remains the sa… --- ## [Data tiering (hot, warm and cold storage)](https://discuss.elastic.co/t/data-tiering-hot-warm-and-cold-storage/267814)
**Author:** [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 1:29pm UTC](https://discuss.elastic.co/t/data-tiering-hot-warm-and-cold-storage/267814 "2021-03-19T13:29:35Z")
Hello, I am a self-funded student with a homelab wherein I am capturing honeypot logs. I have a workstation with SSD storage wherein one disk holds the VMs OS (Ubuntu server 20.04.2 LTS), and the second SSD contains a da… --- ## [Modsecurity Json log parsing](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807)
**Author:** [@mirko.spezie](https://discuss.elastic.co/u/mirko.spezie)\ **Replies:** 4\ **Last updated:** [March 19, 2021, 1:35pm UTC](https://discuss.elastic.co/t/modsecurity-json-log-parsing/267807 "2021-03-19T13:35:01Z")
Hi, I'm trying to parse with no success the modsecurity json log. This is the output: { "\_index": "modsecurity-2021.03.17", "\_type": "\_doc", "\_id": "oQRWQngB6w34lfYfAQp6", "\_version": 1, … --- ## [Control visualization is not working as expected](https://discuss.elastic.co/t/control-visualization-is-not-working-as-expected/264602)
**Author:** [@Treki](https://discuss.elastic.co/u/Treki)\ **Replies:** 4\ **Last updated:** [March 19, 2021, 1:14pm UTC](https://discuss.elastic.co/t/control-visualization-is-not-working-as-expected/264602 "2021-03-19T13:14:07Z")
Hello everybody, I am comparing the same control visualisation for two different timespans. No other filter is being applied. With control visualisation, I mean this visualisation type: At first, I choose a timespa… --- ## [Can elasticsearch help in seo](https://discuss.elastic.co/t/can-elasticsearch-help-in-seo/267622)
**Author:** [@fatheyabelslam](https://discuss.elastic.co/u/fatheyabelslam)\ **Replies:** 3\ **Last updated:** [March 19, 2021, 12:10pm UTC](https://discuss.elastic.co/t/can-elasticsearch-help-in-seo/267622 "2021-03-19T12:10:39Z")
Hi, I working with yoast.js The main use of it is to analyze content and return score of content based on google requirements for best content structure And i think :thinking: that elasticsearch is the best in unders… --- ## [How to restrict access to specific indices when the user get the space privilege](https://discuss.elastic.co/t/how-to-restrict-access-to-specific-indices-when-the-user-get-the-space-privilege/266876)
**Author:** [@empire](https://discuss.elastic.co/u/empire)\ **Replies:** 3\ **Last updated:** [March 19, 2021, 11:39am UTC](https://discuss.elastic.co/t/how-to-restrict-access-to-specific-indices-when-the-user-get-the-space-privilege/266876 "2021-03-19T11:39:19Z")
Hi, I have a role defined as below. would like to know if the indices restriction still work when the user is allowed to get access to the "vulcano" space. e.g will the user in this role still be restricted to only hav… --- ## [Watcher Script for Monitoring Shard Size](https://discuss.elastic.co/t/watcher-script-for-monitoring-shard-size/267761)
**Author:** [@Learner4](https://discuss.elastic.co/u/Learner4)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 7:53am UTC](https://discuss.elastic.co/t/watcher-script-for-monitoring-shard-size/267761 "2021-03-19T07:53:17Z")
I'm trying to create a Watcher Script to monitor the Shard size for all the indices. Example - If shard size reaches more than 30 GB, an alert should get trigger from Watcher \*\*GET \_cat/shards?v&s=store:desc&format=json&… --- ## [Elasticsearch bulk insertion issue: 403 request throttled due to too many requests](https://discuss.elastic.co/t/elasticsearch-bulk-insertion-issue-403-request-throttled-due-to-too-many-requests/267748)
**Author:** [@divakar](https://discuss.elastic.co/u/divakar)\ **Replies:** 16\ **Last updated:** [March 19, 2021, 11:13am UTC](https://discuss.elastic.co/t/elasticsearch-bulk-insertion-issue-403-request-throttled-due-to-too-many-requests/267748 "2021-03-19T11:13:36Z")
I have following cluster configuration of ES on AWS: Instance Type: t2.small.elasticsearch Number of Nodes: 1 EBS Volume Type: GP2 EBS Volume Size: 10 GiB Heap size: 1 GiB I am trying to index around 20000 document… --- ## [Mulitline pattern in docker autodiscover](https://discuss.elastic.co/t/mulitline-pattern-in-docker-autodiscover/267647)
**Author:** [@d01](https://discuss.elastic.co/u/d01)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 11:06am UTC](https://discuss.elastic.co/t/mulitline-pattern-in-docker-autodiscover/267647 "2021-03-19T11:06:39Z")
I have the following logs coming from a docker container: 2021-03-17T13:43:43+0000 DEBUG \[\_\_main\_\_.\\] Debug 2021-03-17T13:43:43+0000 INFO \[\_\_main\_\_.\\] Info 2021-03-17T13:43:43+0000 WARNI… --- ## [Filebeat error when importing logs](https://discuss.elastic.co/t/filebeat-error-when-importing-logs/267395)
**Author:** [@Liliana\_Novais](https://discuss.elastic.co/u/Liliana_Novais)\ **Replies:** 8\ **Last updated:** [March 19, 2021, 10:27am UTC](https://discuss.elastic.co/t/filebeat-error-when-importing-logs/267395 "2021-03-19T10:27:33Z")
Hello, I configured my elasticsearch with file beat. When my file is changed I get the following error from filebeat: can someone helpme understand this error? --- ## [BEATS input: org.logstash.FieldReference$IllegalSyntaxException: Invalid FieldReference: \`string\[\]\`](https://discuss.elastic.co/t/beats-input-org-logstash-fieldreference-illegalsyntaxexception-invalid-fieldreference-string/267781)
**Author:** [@zarak](https://discuss.elastic.co/u/zarak)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 9:46am UTC](https://discuss.elastic.co/t/beats-input-org-logstash-fieldreference-illegalsyntaxexception-invalid-fieldreference-string/267781 "2021-03-19T09:46:57Z")
Hello, I've setup a basic filebeat to logstash infra, but I keep on getting this kind of error in logstash' logs: Mar 19 10:42:33 logstash\[5912\]: \[2021-03-19T10:42:33,892\]\[INFO \]\[org.logstash.beats.BeatsHandler\]\[http-i… --- ## [Removing unwanted data in Filebeats / Logstash](https://discuss.elastic.co/t/removing-unwanted-data-in-filebeats-logstash/267774)
**Author:** [@h\_q](https://discuss.elastic.co/u/h_q)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 9:29am UTC](https://discuss.elastic.co/t/removing-unwanted-data-in-filebeats-logstash/267774 "2021-03-19T09:29:13Z")
I started ingesting Kubernetes logs into ElasticSearch, using the Filebeat Input Container: Filebeat -\> Logstash -\> ElasticSearch My filebeat.yml looks like this: - type: container paths: - /var/lib/docker/conta… --- ## [Is it possible to Return only one filed with Id](https://discuss.elastic.co/t/is-it-possible-to-return-only-one-filed-with-id/267664)
**Author:** [@mayank\_hardiya](https://discuss.elastic.co/u/mayank_hardiya)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 9:03am UTC](https://discuss.elastic.co/t/is-it-possible-to-return-only-one-filed-with-id/267664 "2021-03-19T09:03:18Z")
\[ { "\_index": "onlive", "\_type": "\_doc", "\_id": "584", "\_score": 1, "\_source": { "description": "Free group Yoga session !!! for all levels Experience authentic Yoga from ancient scriptures", "name": "Free Yoga 1… --- ## [Is the below sizing fits for mentioned use case?](https://discuss.elastic.co/t/is-the-below-sizing-fits-for-mentioned-use-case/266855)
**Author:** [@Jax\_dev](https://discuss.elastic.co/u/Jax_dev)\ **Replies:** 12\ **Last updated:** [March 19, 2021, 9:01am UTC](https://discuss.elastic.co/t/is-the-below-sizing-fits-for-mentioned-use-case/266855 "2021-03-19T09:01:39Z")
I have been reading many documents to understand about sizing about elastic search... Data load: 9k transactions per second Each transaction contains around 1kb of data. Per second, 9k inserts and 50k search queries w… --- ## [Filebeat PostgreSQL module not parsing message](https://discuss.elastic.co/t/filebeat-postgresql-module-not-parsing-message/267464)
**Author:** [@martinfrancois](https://discuss.elastic.co/u/martinfrancois)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 9:01am UTC](https://discuss.elastic.co/t/filebeat-postgresql-module-not-parsing-message/267464 "2021-03-19T09:01:37Z")
We are using filebeat 7.11.0 along with the postgresql module. However for some reason, the message attribute seems not to be touched, although some metadata seems to be added by the module. Our log messages look to be … --- ## [Got error "No processor type exists with name \[attachment\]](https://discuss.elastic.co/t/got-error-no-processor-type-exists-with-name-attachment/267741)
**Author:** [@lxia](https://discuss.elastic.co/u/lxia)\ **Replies:** 4\ **Last updated:** [March 19, 2021, 8:57am UTC](https://discuss.elastic.co/t/got-error-no-processor-type-exists-with-name-attachment/267741 "2021-03-19T08:57:20Z")
have installed ingest-attachment on the host, and also restart the service "sudo systemctl restart elasticsearch.service". and get the message on the host: elasticsearch@CNSHZ-Amarisoft-OptiPlex-7050:/usr/local/elastic… --- ## [ClusterBlockException : All shards failed](https://discuss.elastic.co/t/clusterblockexception-all-shards-failed/267755)
**Author:** [@onkark](https://discuss.elastic.co/u/onkark)\ **Replies:** 4\ **Last updated:** [March 19, 2021, 8:34am UTC](https://discuss.elastic.co/t/clusterblockexception-all-shards-failed/267755 "2021-03-19T08:34:09Z")
Hi, I am getting the error like org.elasticsearch.cluster.block.ClusterBlockException : blocked by : \[SERVICE\_UNAVAILABLE/1/state not recovered/initialized\] where the cluster health status is YELLOW and primary shards … --- ## [Synonyms and/or synonym graph for mapping product names to other names](https://discuss.elastic.co/t/synonyms-and-or-synonym-graph-for-mapping-product-names-to-other-names/267678)
**Author:** [@Robert\_SL](https://discuss.elastic.co/u/Robert_SL)\ **Replies:** 0\ **Last updated:** [March 18, 2021, 2:05pm UTC](https://discuss.elastic.co/t/synonyms-and-or-synonym-graph-for-mapping-product-names-to-other-names/267678 "2021-03-18T14:05:56Z")
(Elastic Cloud 6.8) I have a requirement when searching for a product name to also return a different product that will replace it in the future. For example, when the user searches for "Product ABC", I also want to re… --- ## [How to implement join-fields with Spring Data Elasticsearch 4.0.0](https://discuss.elastic.co/t/how-to-implement-join-fields-with-spring-data-elasticsearch-4-0-0/267661)
**Author:** [@matt4](https://discuss.elastic.co/u/matt4)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 7:50am UTC](https://discuss.elastic.co/t/how-to-implement-join-fields-with-spring-data-elasticsearch-4-0-0/267661 "2021-03-19T07:50:49Z")
Hi, I am using Spring Data Elasticsearch 4.0.0 and I would like to establish a parent-child-relationship between my entities. I found out that @JoinTypeRelation comes with version 4.1.X, but unfortunately, I am stuck wi… --- ## [Help please: logstash start error](https://discuss.elastic.co/t/help-please-logstash-start-error/267630)
**Author:** [@toufiq\_khan](https://discuss.elastic.co/u/toufiq_khan)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 8:08am UTC](https://discuss.elastic.co/t/help-please-logstash-start-error/267630 "2021-03-19T08:08:15Z")
logstash showing this error Using bundled JDK: /usr/share/logstash/jdk OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. WARNING:… --- ## [No results found in kabana discover page](https://discuss.elastic.co/t/no-results-found-in-kabana-discover-page/267242)
**Author:** [@Ravikumar.P](https://discuss.elastic.co/u/Ravikumar.P)\ **Replies:** 2\ **Last updated:** [March 19, 2021, 8:05am UTC](https://discuss.elastic.co/t/no-results-found-in-kabana-discover-page/267242 "2021-03-19T08:05:05Z")
Hi I am new to elastic search and kibana I am facing the below issues. When I select 30days time range only I could see the data until February 19th 2021, 14:01:51. How to get the latest data --- ## [Bytes can be at most 32766 in length got 36716](https://discuss.elastic.co/t/bytes-can-be-at-most-32766-in-length-got-36716/267463)
**Author:** [@ajaythakor](https://discuss.elastic.co/u/ajaythakor)\ **Replies:** 3\ **Last updated:** [March 19, 2021, 7:56am UTC](https://discuss.elastic.co/t/bytes-can-be-at-most-32766-in-length-got-36716/267463 "2021-03-19T07:56:36Z")
I have created new index named tbxmlcs Then I have added mapping: { "properties": { "34084-4": { "type": "keyword", "fields": { "keyword":{ "type":"keyword", "igno… --- ## [Include/Exclude for cardinality aggregation](https://discuss.elastic.co/t/include-exclude-for-cardinality-aggregation/267759)
**Author:** [@robmartin11](https://discuss.elastic.co/u/robmartin11)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 7:50am UTC](https://discuss.elastic.co/t/include-exclude-for-cardinality-aggregation/267759 "2021-03-19T07:50:01Z")
I have a terms aggregation with an IncludeExclude filter: { "size": 0, "query": { ... }, "aggregations": { "facets": { "terms": { "field": "type\_id", "size": 20, "order": \[ … --- ## [Upgrading Elasticsearch in Kubernetes using Helm](https://discuss.elastic.co/t/upgrading-elasticsearch-in-kubernetes-using-helm/267752)
**Author:** [@am\_Pg](https://discuss.elastic.co/u/am_Pg)\ **Replies:** 0\ **Last updated:** [March 19, 2021, 5:52am UTC](https://discuss.elastic.co/t/upgrading-elasticsearch-in-kubernetes-using-helm/267752 "2021-03-19T05:52:53Z")
Hello! Is it possible to upgrade Elasticsearch in Kubernetes using Helm from version 5.4.2 to 6.8? I saw this guide from Elastic https://www.elastic.co/guide/en/elasticsearch/reference/6.8/rolling-upgrades.html. I was w… --- ## [Cloud aws module IAM permissions for filebeat](https://discuss.elastic.co/t/cloud-aws-module-iam-permissions-for-filebeat/267617)
**Author:** [@AJ18](https://discuss.elastic.co/u/AJ18)\ **Replies:** 3\ **Last updated:** [March 19, 2021, 5:21am UTC](https://discuss.elastic.co/t/cloud-aws-module-iam-permissions-for-filebeat/267617 "2021-03-19T05:21:44Z")
What are the minimum set of permissions the cloud-aws plugin requires to work with filebeat? [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1499) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1501)