# Latest **URL:** https://discuss.elastic.co/latest.md?page=1493 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1494 --- ## [Index management](https://discuss.elastic.co/t/index-management/283393)
**Author:** [@alipujaistopo](https://discuss.elastic.co/u/alipujaistopo)\ **Replies:** 2\ **Last updated:** [September 6, 2021, 5:31am UTC](https://discuss.elastic.co/t/index-management/283393 "2021-09-06T05:31:53Z")
what does indices mean? and why do i have so many indices? and what is the difference between xxxx-000004 and xxxx-000005? here's my indices for example: \* winlogbeat-7.10.2-2021.08.22-000005 \* auditbeat-7.11.2-2021.0… --- ## [\[Beat Monitoring\] Beat name is differ with state](https://discuss.elastic.co/t/beat-monitoring-beat-name-is-differ-with-state/283239)
**Author:** [@InJong\_Park](https://discuss.elastic.co/u/InJong_Park)\ **Replies:** 2\ **Last updated:** [September 6, 2021, 2:35am UTC](https://discuss.elastic.co/t/beat-monitoring-beat-name-is-differ-with-state/283239 "2021-09-06T02:35:35Z")
Hi. Thanks for your attention. metricbeat.yml name: "sample" metricbeat.log 2021-09-03T14:46:12.876+0900 INFO \[publisher\] pipeline/module.go:113 Beat name: sample curl state \[root@ip-01-02-03-04 15:00:04… --- ## ["Unable to connect to Elasticsearch. Error: \[search\_phase\_execution\_exception\] all shards failed"](https://discuss.elastic.co/t/unable-to-connect-to-elasticsearch-error-search-phase-execution-exception-all-shards-failed/283344)
**Author:** [@ShashikanthKomandoor](https://discuss.elastic.co/u/ShashikanthKomandoor)\ **Replies:** 1\ **Last updated:** [September 5, 2021, 10:22pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-elasticsearch-error-search-phase-execution-exception-all-shards-failed/283344 "2021-09-05T22:22:02Z")
Hi Team, I have been running ELK Cluster of version 7.8 with 3 master nodes and 5 data nodes. I am running this on RHEL 7.5. I have installed httpd to proxy the requests to Kibana. I installed httpd, Kibana, Elasticsear… --- ## [Fighting with ingest pipeline create](https://discuss.elastic.co/t/fighting-with-ingest-pipeline-create/283353)
**Author:** [@Russell\_Fulton](https://discuss.elastic.co/u/Russell_Fulton)\ **Replies:** 7\ **Last updated:** [September 5, 2021, 9:40pm UTC](https://discuss.elastic.co/t/fighting-with-ingest-pipeline-create/283353 "2021-09-05T21:40:43Z")
Version 7.14 I am trying to create a simple ingest pipeline in Kibana but can not figure out what syntax the condition will accept and there is no documentation that I can find. I know how to specify what I want in the… --- ## [Curl query to delete index patterns, and, parsefailures to stdout](https://discuss.elastic.co/t/curl-query-to-delete-index-patterns-and-parsefailures-to-stdout/280110)
**Author:** [@kriss332](https://discuss.elastic.co/u/kriss332)\ **Replies:** 6\ **Last updated:** [September 5, 2021, 6:35pm UTC](https://discuss.elastic.co/t/curl-query-to-delete-index-patterns-and-parsefailures-to-stdout/280110 "2021-09-05T18:35:38Z")
Hi all... I have 2 questions- I have created more than 150 index patterns based on the indexes I have. Yesterday I changed the names of indexes for better visibility. Now I need to delete all old index-patterns using… --- ## [Using memcached for logstash](https://discuss.elastic.co/t/using-memcached-for-logstash/283381)
**Author:** [@kriss332](https://discuss.elastic.co/u/kriss332)\ **Replies:** 0\ **Last updated:** [September 5, 2021, 5:57pm UTC](https://discuss.elastic.co/t/using-memcached-for-logstash/283381 "2021-09-05T17:57:12Z")
Hi all. I have to work with memcached for logstash to get the alerts about the destination IPs (belonging to C2 IPs) from my firewall's destination IPs. I have MISP for finding malicious/suspicious IPs about the traffic. … --- ## [Msip threat intel import not working](https://discuss.elastic.co/t/msip-threat-intel-import-not-working/283210)
**Author:** [@finbarr996](https://discuss.elastic.co/u/finbarr996)\ **Replies:** 2\ **Last updated:** [September 5, 2021, 6:05pm UTC](https://discuss.elastic.co/t/msip-threat-intel-import-not-working/283210 "2021-09-05T18:05:07Z")
I've edited the /etc/filebeat/modules.d/threatintel.yml and updated the entry for misp to enable it and also to add the API key from my local misp server. My config looks like this: misp: enabled: true var.in… --- ## [Created simple filebeat module for ProxySQL](https://discuss.elastic.co/t/created-simple-filebeat-module-for-proxysql/283380)
**Author:** [@altuhovsu](https://discuss.elastic.co/u/altuhovsu)\ **Replies:** 0\ **Last updated:** [September 5, 2021, 5:16pm UTC](https://discuss.elastic.co/t/created-simple-filebeat-module-for-proxysql/283380 "2021-09-05T17:16:02Z")
If anyone needed, I have created (cloned from mysql module) a simple Filebeat module for parsing ProxySQL logs and ship them to ElasticSearch. https://github.com/altuhovsu/filebeat-proxysql-module --- ## [Ruby filter plugin does not read new field values](https://discuss.elastic.co/t/ruby-filter-plugin-does-not-read-new-field-values/283343)
**Author:** [@priyaankaa](https://discuss.elastic.co/u/priyaankaa)\ **Replies:** 8\ **Last updated:** [September 5, 2021, 2:22pm UTC](https://discuss.elastic.co/t/ruby-filter-plugin-does-not-read-new-field-values/283343 "2021-09-05T14:22:25Z")
How to get correct value of the field inside ruby code in logstash pipeline? sample 1: input { elasticsearch { hosts =\> "http://localhost:9200" index =\> "test1" } } filter { mutate { add\_field =\>… --- ## [Highlighting of exact search with simple\_query\_string](https://discuss.elastic.co/t/highlighting-of-exact-search-with-simple-query-string/283292)
**Author:** [@Wonder\_Garance](https://discuss.elastic.co/u/Wonder_Garance)\ **Replies:** 1\ **Last updated:** [September 5, 2021, 12:53pm UTC](https://discuss.elastic.co/t/highlighting-of-exact-search-with-simple-query-string/283292 "2021-09-05T12:53:23Z")
I use Elasticsearch 7.9 and I want to highlight an exact search filed with simple query string query, for a document. The following query doesn't highlight words in the document, although these words are in the document… --- ## [Influxdb and elasticsearch benchmarking for timeseries data](https://discuss.elastic.co/t/influxdb-and-elasticsearch-benchmarking-for-timeseries-data/283371)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 0\ **Last updated:** [September 5, 2021, 11:48am UTC](https://discuss.elastic.co/t/influxdb-and-elasticsearch-benchmarking-for-timeseries-data/283371 "2021-09-05T11:48:48Z")
Do we have the benchmarking stats for Elasticsearch and Influxdb considering the time series data (Metrics) Looking into the performance aspects as recent releases of elasticsearch have the additional optimization strat… --- ## [Using Kafka Stream for data transform and enrichment](https://discuss.elastic.co/t/using-kafka-stream-for-data-transform-and-enrichment/283370)
**Author:** [@mruthyu](https://discuss.elastic.co/u/mruthyu)\ **Replies:** 0\ **Last updated:** [September 5, 2021, 11:40am UTC](https://discuss.elastic.co/t/using-kafka-stream-for-data-transform-and-enrichment/283370 "2021-09-05T11:40:04Z")
Is it possible to use Kafka Streaming for data transform and enrichment instead of logstash? What are the advantages and disadvantages? --- ## [Virustotal Module MISP attributes don't store in ELK](https://discuss.elastic.co/t/virustotal-module-misp-attributes-dont-store-in-elk/282834)
**Author:** [@Abanob\_Medhat](https://discuss.elastic.co/u/Abanob_Medhat)\ **Replies:** 3\ **Last updated:** [September 5, 2021, 11:20am UTC](https://discuss.elastic.co/t/virustotal-module-misp-attributes-dont-store-in-elk/282834 "2021-09-05T11:20:55Z")
hello, I integrated Virustotal with MISP and run virustotal on events and all virustotal attributes are saved in MISP. also , I integrated MISP with Elastic using Filebeats. the events are stored in Elastic and I can … --- ## [Elasticsearch is displaying correct result only after a space](https://discuss.elastic.co/t/elasticsearch-is-displaying-correct-result-only-after-a-space/283009)
**Author:** [@Drsaud](https://discuss.elastic.co/u/Drsaud)\ **Replies:** 9\ **Last updated:** [September 5, 2021, 10:43am UTC](https://discuss.elastic.co/t/elasticsearch-is-displaying-correct-result-only-after-a-space/283009 "2021-09-05T10:43:22Z")
I build react app the has search bar using elasticsearch, but the problem is when I search in my search bar earthier I have to write the word not complete or I write as complete word but after I click space. for example… --- ## [TSVB line chart has gaps](https://discuss.elastic.co/t/tsvb-line-chart-has-gaps/282529)
**Author:** [@vijay\_kaali](https://discuss.elastic.co/u/vijay_kaali)\ **Replies:** 3\ **Last updated:** [September 5, 2021, 6:45am UTC](https://discuss.elastic.co/t/tsvb-line-chart-has-gaps/282529 "2021-09-05T06:45:20Z")
i have create a line chart and tsvb . line chart. in visualization , tsvb has gaps while normal line chart looks fine . 1st chart tsvb 2nd chart normal line chart . with same data, same avg( runqueue) formula wh… --- ## [S3 output plugin "upload\_worker\_count" logs](https://discuss.elastic.co/t/s3-output-plugin-upload-worker-count-logs/283357)
**Author:** [@Varsha1](https://discuss.elastic.co/u/Varsha1)\ **Replies:** 0\ **Last updated:** [September 5, 2021, 5:15am UTC](https://discuss.elastic.co/t/s3-output-plugin-upload-worker-count-logs/283357 "2021-09-05T05:15:32Z")
I've realized I don't see the detailed s3 log in console when running logstash for kafka input and s3 output. Also I've configured the S3 output plugin to run with multiple workers, how could I make sure the 5 workers ar… --- ## [I am trying to pass a fieldname in jdbc\_streaming plugin to the jdbc\_user parameter, but it fails to read the value of the field](https://discuss.elastic.co/t/i-am-trying-to-pass-a-fieldname-in-jdbc-streaming-plugin-to-the-jdbc-user-parameter-but-it-fails-to-read-the-value-of-the-field/283241)
**Author:** [@Varsha1](https://discuss.elastic.co/u/Varsha1)\ **Replies:** 4\ **Last updated:** [September 5, 2021, 5:07am UTC](https://discuss.elastic.co/t/i-am-trying-to-pass-a-fieldname-in-jdbc-streaming-plugin-to-the-jdbc-user-parameter-but-it-fails-to-read-the-value-of-the-field/283241 "2021-09-05T05:07:31Z")
jdbc\_streaming { jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver" jdbc\_driver\_library =\> "/home/user/logstash-7.14.0/logstash-core/lib/jars/ojdbc8.jar" jdbc\_connection\_string =\> "jdbc:oracle:thin:@XXXXX" … --- ## [Many deleted indices, no recovery of disk space?](https://discuss.elastic.co/t/many-deleted-indices-no-recovery-of-disk-space/283328)
**Author:** [@NetwarSystem](https://discuss.elastic.co/u/NetwarSystem)\ **Replies:** 3\ **Last updated:** [September 5, 2021, 3:51am UTC](https://discuss.elastic.co/t/many-deleted-indices-no-recovery-of-disk-space/283328 "2021-09-05T03:51:21Z")
I have a 1.2TB backup of Elasticsearch data that I have restored to a standalone machine running 7.14.0. The restore worked fine and I began grooming the data - about half of the indices in the snapshot are not needed fo… --- ## [Filebeat retrieve log with current time stamp](https://discuss.elastic.co/t/filebeat-retrieve-log-with-current-time-stamp/283355)
**Author:** [@josh12](https://discuss.elastic.co/u/josh12)\ **Replies:** 0\ **Last updated:** [September 5, 2021, 1:10am UTC](https://discuss.elastic.co/t/filebeat-retrieve-log-with-current-time-stamp/283355 "2021-09-05T01:10:36Z")
Can I know whether i can retrieve logs bsed on current date. I am planning for \/\*.log and set ignore\_older : 1h if this the case, how it works. will it scan all logs and take based on current date and time … --- ## [Why does every log use a different format for the Timestamp?](https://discuss.elastic.co/t/why-does-every-log-use-a-different-format-for-the-timestamp/283346)
**Author:** [@mhare](https://discuss.elastic.co/u/mhare)\ **Replies:** 10\ **Last updated:** [September 4, 2021, 9:19pm UTC](https://discuss.elastic.co/t/why-does-every-log-use-a-different-format-for-the-timestamp/283346 "2021-09-04T21:19:40Z")
I am using Stack and Filebeat version 7.13.4 on Windows 10 I have a new log to process and it has a new way to represent the timestamp. As soon as I figure one out, they throw me another This one is starts as: 2021/0… --- ## [Apply aggregation result to values in the document \[Elasticsearch\]](https://discuss.elastic.co/t/apply-aggregation-result-to-values-in-the-document-elasticsearch/283350)
**Author:** [@me.mohammed](https://discuss.elastic.co/u/me.mohammed)\ **Replies:** 0\ **Last updated:** [September 4, 2021, 6:59pm UTC](https://discuss.elastic.co/t/apply-aggregation-result-to-values-in-the-document-elasticsearch/283350 "2021-09-04T18:59:58Z")
so what I'm trying to do is change the actual values in the document depending on the aggregation result, I'm not sure if this is possible or not but here are some examples to what I'm trying to do { "deals": \[ { … --- ## [Mixed kinds of rename option in single mutate filter won't work](https://discuss.elastic.co/t/mixed-kinds-of-rename-option-in-single-mutate-filter-wont-work/283278)
**Author:** [@tasneem-hyder](https://discuss.elastic.co/u/tasneem-hyder)\ **Replies:** 4\ **Last updated:** [September 4, 2021, 6:23pm UTC](https://discuss.elastic.co/t/mixed-kinds-of-rename-option-in-single-mutate-filter-wont-work/283278 "2021-09-04T18:23:11Z")
In a single mutate filter, we can't use the mixed kinds of rename option. This is not configuration issue but a runtime issue so it is hard to debug. As this kind of config setup doesn't compile to an expected config so … --- ## [How to calculate old date from today by subtracting given years, months, days in logstash?](https://discuss.elastic.co/t/how-to-calculate-old-date-from-today-by-subtracting-given-years-months-days-in-logstash/283349)
**Author:** [@priyaankaa](https://discuss.elastic.co/u/priyaankaa)\ **Replies:** 2\ **Last updated:** [September 4, 2021, 5:36pm UTC](https://discuss.elastic.co/t/how-to-calculate-old-date-from-today-by-subtracting-given-years-months-days-in-logstash/283349 "2021-09-04T17:36:53Z")
I want to subtract current date with given years, months, days using logstash pipeline. For example : Today = 1 Aug 2021 Given values : Years=25, Months=10, Days=10 Expected date : 22 Sept 1995 My logstash filter sec… --- ## [Logstash multiline issue - limit of size in single event](https://discuss.elastic.co/t/logstash-multiline-issue-limit-of-size-in-single-event/283335)
**Author:** [@ErSumit](https://discuss.elastic.co/u/ErSumit)\ **Replies:** 2\ **Last updated:** [September 4, 2021, 5:15pm UTC](https://discuss.elastic.co/t/logstash-multiline-issue-limit-of-size-in-single-event/283335 "2021-09-04T17:15:40Z")
Logstash file input configured with multiline pattern as codec =\> multiline { pattern =\> '^ {' negate =\> true what =\> previous } } Input file consist of multiple json objects. This mu… --- ## [Disk.size is 10 times more than disk.indices](https://discuss.elastic.co/t/disk-size-is-10-times-more-than-disk-indices/283280)
**Author:** [@mrudrego](https://discuss.elastic.co/u/mrudrego)\ **Replies:** 4\ **Last updated:** [September 4, 2021, 11:47am UTC](https://discuss.elastic.co/t/disk-size-is-10-times-more-than-disk-indices/283280 "2021-09-04T11:47:56Z")
We are facing shard allocation issues because of disk watermark and when checked allocation API, strangely disk.size is 5-6 times more than disk.indices \[root@bcmt-ovs-control-0 ~\]# curl -s elasticsearch.cgnat.svc.clust… --- ## [Logstash add field default type is string but check field in an array field must match type](https://discuss.elastic.co/t/logstash-add-field-default-type-is-string-but-check-field-in-an-array-field-must-match-type/283334)
**Author:** [@qinkkai](https://discuss.elastic.co/u/qinkkai)\ **Replies:** 1\ **Last updated:** [September 4, 2021, 7:56am UTC](https://discuss.elastic.co/t/logstash-add-field-default-type-is-string-but-check-field-in-an-array-field-must-match-type/283334 "2021-09-04T07:56:24Z")
My logstash config as follow，I want to output only when the field a in the arr. I think the condition in the output is always true. input { generator { message =\> '{"a":111}' codec =\> json add\_field =\>… --- ## [Should we adopt elasticsearch on top of postgres to handle our search/filtering?](https://discuss.elastic.co/t/should-we-adopt-elasticsearch-on-top-of-postgres-to-handle-our-search-filtering/283324)
**Author:** [@elastic\_yogi](https://discuss.elastic.co/u/elastic_yogi)\ **Replies:** 1\ **Last updated:** [September 4, 2021, 7:29am UTC](https://discuss.elastic.co/t/should-we-adopt-elasticsearch-on-top-of-postgres-to-handle-our-search-filtering/283324 "2021-09-04T07:29:02Z")
We have a web service that allows a ~10 customers to look at their data. Each customer has about 1M - 10M records. Our total datastore size will target the 10's of gigabytes. We currently store this data in postgres norm… --- ## [Web defacement monitoring](https://discuss.elastic.co/t/web-defacement-monitoring/283327)
**Author:** [@Lakshan\_Chamidu](https://discuss.elastic.co/u/Lakshan_Chamidu)\ **Replies:** 0\ **Last updated:** [September 4, 2021, 4:55am UTC](https://discuss.elastic.co/t/web-defacement-monitoring/283327 "2021-09-04T04:55:48Z")
how can we monitor web defacement using elk stack? --- ## [What does this error mean - Data too large, data for \[\\]](https://discuss.elastic.co/t/what-does-this-error-mean-data-too-large-data-for-transport-request/209345)
**Author:** [@Dmitry1](https://discuss.elastic.co/u/Dmitry1)\ **Replies:** 7\ **Last updated:** [September 4, 2021, 3:38am UTC](https://discuss.elastic.co/t/what-does-this-error-mean-data-too-large-data-for-transport-request/209345 "2021-09-04T03:38:37Z")
Hello, please explain me what this error mean ? And what I need to do to get rid of it : \[2019-11-25T00:20:58,521\]\[WARN \]\[o.e.x.m.e.l.LocalExporter\] \[host\] unexpected error while indexing monitoring document org.elastic… --- ## [Query to display the results contains speicifed string](https://discuss.elastic.co/t/query-to-display-the-results-contains-speicifed-string/48570)
**Author:** [@nagesh](https://discuss.elastic.co/u/nagesh)\ **Replies:** 8\ **Last updated:** [September 4, 2021, 3:32am UTC](https://discuss.elastic.co/t/query-to-display-the-results-contains-speicifed-string/48570 "2021-09-04T03:32:15Z")
Hi, Application name is property in the fields list of Kibana dashboard viewlet. Suppose if I need to display logs which contains application name in the log message then what query I need to use in dashboard. Thanks… [Previous page](https://discuss.elastic.co/latest.md?page=1492) [Next page](https://discuss.elastic.co/latest.md?page=1494)