# \#none **URL:** https://discuss.elastic.co/tag/none.md?page=1886 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1887 --- ## [Import only arrays that match conditions in Elastic Search](https://discuss.elastic.co/t/import-only-arrays-that-match-conditions-in-elastic-search/199899)
**Author:** [@111211](https://discuss.elastic.co/u/111211)\ **Replies:** 2\ **Last updated:** [September 20, 2019, 1:24am UTC](https://discuss.elastic.co/t/import-only-arrays-that-match-conditions-in-elastic-search/199899 "2019-09-20T01:24:07Z")
I'm dealing with nested nested data in ElasticSearch. I want it to work like a SELECT \* from where in an RDBMS. If you have the following data POST test-stack/test/1234\_5678 { "Id" : 1234, "availables":\[ { … --- ## [Logstash - ruby script to create array of nested fields](https://discuss.elastic.co/t/logstash-ruby-script-to-create-array-of-nested-fields/200312)
**Author:** [@Adithyaram](https://discuss.elastic.co/u/Adithyaram)\ **Replies:** 1\ **Last updated:** [September 19, 2019, 10:50pm UTC](https://discuss.elastic.co/t/logstash-ruby-script-to-create-array-of-nested-fields/200312 "2019-09-19T22:50:08Z")
Hello, I currently have the below JSON structure "Unit": { "Unit": \["QA", "FA"\], "QTY": \["2", "5"\], "ID": \["1", "2"\], "Item": \["Shirt", "Trousers"\], "Price": \["10", "50"\] } and want … --- ## [Removing part of message giving grokparsefailure](https://discuss.elastic.co/t/removing-part-of-message-giving-grokparsefailure/200234)
**Author:** [@jlim0930](https://discuss.elastic.co/u/jlim0930)\ **Replies:** 6\ **Last updated:** [September 19, 2019, 10:41pm UTC](https://discuss.elastic.co/t/removing-part-of-message-giving-grokparsefailure/200234 "2019-09-19T22:41:45Z")
while sending logs from a central syslog server it adds "Message forwarded from ...." in the message field. I added if "Message forwarded from" in \[message\] { mutate { gsub =\> \[ "message", "Message forwarded from ",… --- ## [Javascript heap out of memeory](https://discuss.elastic.co/t/javascript-heap-out-of-memeory/200164)
**Author:** [@saroj\_shrestha](https://discuss.elastic.co/u/saroj_shrestha)\ **Replies:** 1\ **Last updated:** [September 19, 2019, 9:46pm UTC](https://discuss.elastic.co/t/javascript-heap-out-of-memeory/200164 "2019-09-19T21:46:18Z")
Hey guys. Have you faced "Javascript heap out of memory" after you have increased the heap size for the nodejs. --- ## [Do I have to configure Logstash (to use GeoIP) if I am using a VM?](https://discuss.elastic.co/t/do-i-have-to-configure-logstash-to-use-geoip-if-i-am-using-a-vm/200313)
**Author:** [@Rosho](https://discuss.elastic.co/u/Rosho)\ **Replies:** 0\ **Last updated:** [September 19, 2019, 9:05pm UTC](https://discuss.elastic.co/t/do-i-have-to-configure-logstash-to-use-geoip-if-i-am-using-a-vm/200313 "2019-09-19T21:05:58Z")
Continuing the discussion from How does geoIP for private IPaddresses work?: The idea #1 of correlating IPs is difficult to implement because the public ones change dinamically. The idea #2: I need the range of IP… --- ## [Remove fields from documents based on their value](https://discuss.elastic.co/t/remove-fields-from-documents-based-on-their-value/200270)
**Author:** [@MMH](https://discuss.elastic.co/u/MMH)\ **Replies:** 7\ **Last updated:** [September 19, 2019, 7:54pm UTC](https://discuss.elastic.co/t/remove-fields-from-documents-based-on-their-value/200270 "2019-09-19T19:54:49Z")
I am using Logstash to read JSON data and load it to ElasticSearch. I am using a combination of update with doc\_as\_upsert to update existing documents and insert new. The specific application need is that in Elastic the… --- ## [Full dashboard visualization](https://discuss.elastic.co/t/full-dashboard-visualization/200268)
**Author:** [@nickgregz](https://discuss.elastic.co/u/nickgregz)\ **Replies:** 1\ **Last updated:** [September 19, 2019, 7:40pm UTC](https://discuss.elastic.co/t/full-dashboard-visualization/200268 "2019-09-19T19:40:46Z")
Is there a way to embed the whole Kibana dashboard (with menu and navigation) instead of just certain visualizations like it's done on https://logz.io/ or https://sematext.com/logsene/? --- ## [Split Series by Array Values (instead of whole Array)](https://discuss.elastic.co/t/split-series-by-array-values-instead-of-whole-array/200221)
**Author:** [@kaihil](https://discuss.elastic.co/u/kaihil)\ **Replies:** 1\ **Last updated:** [September 19, 2019, 7:09pm UTC](https://discuss.elastic.co/t/split-series-by-array-values-instead-of-whole-array/200221 "2019-09-19T19:09:13Z")
Guess I have timeseries data roughly like this (four examples): timestamp: 1 systems: \[A, C\] value: 10 timestamp: 1 systems: \[A, B\] value: 9 timestamp: 2 systems: \[A\] value: 12 timestamp: 2 systems: \[B, C\] va… --- ## [Logstash pipeline output | duplicate messages ending up indexes](https://discuss.elastic.co/t/logstash-pipeline-output-duplicate-messages-ending-up-indexes/199878)
**Author:** [@deeps](https://discuss.elastic.co/u/deeps)\ **Replies:** 4\ **Last updated:** [September 19, 2019, 6:19pm UTC](https://discuss.elastic.co/t/logstash-pipeline-output-duplicate-messages-ending-up-indexes/199878 "2019-09-19T18:19:53Z")
In the /etc/logstash/conf.d/ directory I have configured 2 pipeline files to read messages from kafka topic/s and send it to data nodes in the cluster. (1.conf and 2.conf) \[root@ingest1 conf.d\]# egrep -w "topics|index" … --- ## [Kibana and Sonicwall sys logs](https://discuss.elastic.co/t/kibana-and-sonicwall-sys-logs/199905)
**Author:** [@RebeccaStanke](https://discuss.elastic.co/u/RebeccaStanke)\ **Replies:** 0\ **Last updated:** [September 18, 2019, 3:22am UTC](https://discuss.elastic.co/t/kibana-and-sonicwall-sys-logs/199905 "2019-09-18T03:22:14Z")
anyone have instructions on how to send sonicwall sys logs to Kibana for a newby? --- ## [Format index pattern field does not work](https://discuss.elastic.co/t/format-index-pattern-field-does-not-work/199503)
**Author:** [@bodo.te](https://discuss.elastic.co/u/bodo.te)\ **Replies:** 5\ **Last updated:** [September 19, 2019, 5:31pm UTC](https://discuss.elastic.co/t/format-index-pattern-field-does-not-work/199503 "2019-09-19T17:31:53Z")
I have setup metricbeat, logstash, ES and kibana according to the tutorial https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-elastic-stack.html which works so far as expected. Now I try to u… --- ## [Is logstash right choice for this?](https://discuss.elastic.co/t/is-logstash-right-choice-for-this/200280)
**Author:** [@plexcellmedia](https://discuss.elastic.co/u/plexcellmedia)\ **Replies:** 0\ **Last updated:** [September 19, 2019, 5:29pm UTC](https://discuss.elastic.co/t/is-logstash-right-choice-for-this/200280 "2019-09-19T17:29:27Z")
Hey, I'm having hard time here to go with right choice. I'm not sure if logstash is right way to go, hope you guys can help me out a bit. So, currently all our user tracking data is sent to elasticsearch directly which… --- ## [Ruby filter - compare a field against a mongodb and if exists the value add a new field](https://discuss.elastic.co/t/ruby-filter-compare-a-field-against-a-mongodb-and-if-exists-the-value-add-a-new-field/200279)
**Author:** [@Ricardo\_Pertuz](https://discuss.elastic.co/u/Ricardo_Pertuz)\ **Replies:** 0\ **Last updated:** [September 19, 2019, 5:20pm UTC](https://discuss.elastic.co/t/ruby-filter-compare-a-field-against-a-mongodb-and-if-exists-the-value-add-a-new-field/200279 "2019-09-19T17:20:48Z")
Hi, In my filebeat I'm adding a field called token, then I want to create a filter to check if the token value exists in my mongodb, if not then drop the entry and if so then mutate the token value to something else. Is… --- ## [Logstash output to file and s3 is different](https://discuss.elastic.co/t/logstash-output-to-file-and-s3-is-different/200272)
**Author:** [@shazbot](https://discuss.elastic.co/u/shazbot)\ **Replies:** 2\ **Last updated:** [September 19, 2019, 5:12pm UTC](https://discuss.elastic.co/t/logstash-output-to-file-and-s3-is-different/200272 "2019-09-19T17:12:49Z")
I'm trying to build something that would put logs into s3. In this case i'm working with netflow and my first step was to just get something out there. input { udp { port =\> 9995 codec =\> netflow } } outp… --- ## [Term query not working](https://discuss.elastic.co/t/term-query-not-working/200277)
**Author:** [@Ramesh\_Kanakarajan](https://discuss.elastic.co/u/Ramesh_Kanakarajan)\ **Replies:** 0\ **Last updated:** [September 19, 2019, 5:12pm UTC](https://discuss.elastic.co/t/term-query-not-working/200277 "2019-09-19T17:12:40Z")
I am trying to get data through term query, but it not returning any data. I searched in Google...came to know, need to append "keyword" in search variable when we have "text" and "keyword" defined for a field in mappi… --- ## [Limited indexing rate from one host](https://discuss.elastic.co/t/limited-indexing-rate-from-one-host/199151)
**Author:** [@Dmitry1](https://discuss.elastic.co/u/Dmitry1)\ **Replies:** 4\ **Last updated:** [September 19, 2019, 5:12pm UTC](https://discuss.elastic.co/t/limited-indexing-rate-from-one-host/199151 "2019-09-19T17:12:07Z")
Hello, i see strange ES behavior on my test cluster. Cluster has one node and has only one index for logs that have 10 primary shards no replicas. Some tuning of this index: "index.refresh\_interval": "30s", "index.tran… --- ## [Fscrawler Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: Unrecognized character escape 'U'](https://discuss.elastic.co/t/fscrawler-exception-in-thread-main-com-fasterxml-jackson-databind-jsonmappingexception-unrecognized-character-escape-u/199868)
**Author:** [@vstevenson](https://discuss.elastic.co/u/vstevenson)\ **Replies:** 2\ **Last updated:** [September 19, 2019, 5:09pm UTC](https://discuss.elastic.co/t/fscrawler-exception-in-thread-main-com-fasterxml-jackson-databind-jsonmappingexception-unrecognized-character-escape-u/199868 "2019-09-19T17:09:01Z")
Hi, I'm trying to use fscrawler to index pdf files. When I try to run it, I get the following error message: C:\\Users\\vincent.stevenson\\Desktop\\fscrawler-2.5\\fscrawler-2.5\>bin\\fscrawler --config\_dir .\\config books Exce… --- ## [Logstash JDBC input filter - Pagination](https://discuss.elastic.co/t/logstash-jdbc-input-filter-pagination/200274)
**Author:** [@sid1990](https://discuss.elastic.co/u/sid1990)\ **Replies:** 0\ **Last updated:** [September 19, 2019, 5:03pm UTC](https://discuss.elastic.co/t/logstash-jdbc-input-filter-pagination/200274 "2019-09-19T17:03:23Z")
Hi, I need a way to replicate what paging does , I'll simplify and explain only the problem part that we are struck on consider I have a table with ID's \[ 1 to 2000 \] and \[4000 - 1M\] ( ID's from 2001 to 3999 are not… --- ## [Elastic Node failure observations](https://discuss.elastic.co/t/elastic-node-failure-observations/199874)
**Author:** [@Parvez](https://discuss.elastic.co/u/Parvez)\ **Replies:** 10\ **Last updated:** [September 19, 2019, 4:19pm UTC](https://discuss.elastic.co/t/elastic-node-failure-observations/199874 "2019-09-19T16:19:42Z")
We recently conducted a Node failure test in our test environment. we have , Nodes: 3 Indices: 14 Total Shards: 94 Unassigned Shards:0 Questions: When one of the node which is a master node was brought down we st… --- ## [Fresh instalation - Logstash cannot connect to ElasticSearch](https://discuss.elastic.co/t/fresh-instalation-logstash-cannot-connect-to-elasticsearch/200085)
**Author:** [@MMH](https://discuss.elastic.co/u/MMH)\ **Replies:** 6\ **Last updated:** [September 19, 2019, 3:39pm UTC](https://discuss.elastic.co/t/fresh-instalation-logstash-cannot-connect-to-elasticsearch/200085 "2019-09-19T15:39:19Z")
Hi all, I installed Elasticsearch, Kibana, and Logstash from zip files on Windows. Elasticsearch seems to work - I can connect via Kibana and insert documents to the index. My browser returns JSON response when directe… --- ## [Filebeat & docker containers, stdout контейнера как input для Filebeat](https://discuss.elastic.co/t/filebeat-docker-containers-stdout-input-filebeat/200163)
**Author:** [@ep4sh](https://discuss.elastic.co/u/ep4sh)\ **Replies:** 1\ **Last updated:** [September 19, 2019, 3:24pm UTC](https://discuss.elastic.co/t/filebeat-docker-containers-stdout-input-filebeat/200163 "2019-09-19T15:24:31Z")
Привет всем, вопрос про Filebeat и Docker контейнеры. Нужно сделать одну вещь, но пока не понимаю как. Есть docker и логи контейнеров он пишет в stdout /proc/self/fd/2, а Filebeat должен их читать и отправлять в Logsta… --- ## [Query that finds duplicate log lines and their respective count](https://discuss.elastic.co/t/query-that-finds-duplicate-log-lines-and-their-respective-count/199937)
**Author:** [@AndriesN](https://discuss.elastic.co/u/AndriesN)\ **Replies:** 3\ **Last updated:** [September 19, 2019, 3:21pm UTC](https://discuss.elastic.co/t/query-that-finds-duplicate-log-lines-and-their-respective-count/199937 "2019-09-19T15:21:25Z")
Hi all, I am doing a school project where I am using your product for a log management system. I have lots of data and I want to know which log lines are duplicate and how many duplicates there are for that particular … --- ## [Why does logstash do not appear in the "Managament" option?](https://discuss.elastic.co/t/why-does-logstash-do-not-appear-in-the-managament-option/200075)
**Author:** [@Rosho](https://discuss.elastic.co/u/Rosho)\ **Replies:** 3\ **Last updated:** [September 19, 2019, 3:09pm UTC](https://discuss.elastic.co/t/why-does-logstash-do-not-appear-in-the-managament-option/200075 "2019-09-19T15:09:12Z")
Hi I have just viewed a tutorial about Logstash https://www.elastic.co/fr/webinars/getting-started-logstash?baymax=default&elektra=docs&storm=top-video I am using a VM to test the X-pack capabilities and the GeoIP wit… --- ## [Group by time interval](https://discuss.elastic.co/t/group-by-time-interval/200246)
**Author:** [@Akhil\_M\_S](https://discuss.elastic.co/u/Akhil_M_S)\ **Replies:** 0\ **Last updated:** [September 19, 2019, 3:05pm UTC](https://discuss.elastic.co/t/group-by-time-interval/200246 "2019-09-19T15:05:43Z")
I have a one hour data in my index.I want to get group the data by its time field ('logged\_time\_unix').here is my query \[aggs\] =\> Array ( \[viewTimeBy\] =\> Array ( \[terms\] =\> Array ( \[field\] =\> logged\_time\_unix \[si… --- ## [Dateparsefailure on logstash](https://discuss.elastic.co/t/dateparsefailure-on-logstash/200235)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 2\ **Last updated:** [September 19, 2019, 2:48pm UTC](https://discuss.elastic.co/t/dateparsefailure-on-logstash/200235 "2019-09-19T14:48:40Z")
I am pulling data from sql and I have date field I am still getting this error on output. and can't figure out where this is coming from. "tags" =\> \[ \[0\] "\_dateparsefailure" \], I try without a… --- ## [Настройка модуля nginx](https://discuss.elastic.co/t/nginx/199945)
**Author:** [@KOTOXJle6](https://discuss.elastic.co/u/KOTOXJle6)\ **Replies:** 2\ **Last updated:** [September 19, 2019, 2:45pm UTC](https://discuss.elastic.co/t/nginx/199945 "2019-09-19T14:45:10Z")
Добрый день. Имеется кластер из 3х нод ES 7.1, отдельно Kibana 7.1, отдельно Logstash 7.1. Решил добавить логи с серверов с Nginx 1.14 с помощью соответствующего модуля. Поставил filebeat 7.3.2. Включил модуль nginx, … --- ## [Data from CSV is not inserting to Elastic search using Logstash](https://discuss.elastic.co/t/data-from-csv-is-not-inserting-to-elastic-search-using-logstash/200176)
**Author:** [@SudevPK](https://discuss.elastic.co/u/SudevPK)\ **Replies:** 1\ **Last updated:** [September 19, 2019, 2:39pm UTC](https://discuss.elastic.co/t/data-from-csv-is-not-inserting-to-elastic-search-using-logstash/200176 "2019-09-19T14:39:34Z")
Data is not Inserting to Elastic search using LogStash Config input { file { path =\> \["C:\\PR Files\\Test\\Account1.csv"\] start\_position =\> "beginning" sincedb\_path =\> "nul" } } filter { csv { separator =\> "|" columns =\> … --- ## [Kibana 7.3 - Default Dashboard for IIS - 3 of 5 Elements Have Visualization Errors](https://discuss.elastic.co/t/kibana-7-3-default-dashboard-for-iis-3-of-5-elements-have-visualization-errors/200080)
**Author:** [@antonio84](https://discuss.elastic.co/u/antonio84)\ **Replies:** 2\ **Last updated:** [September 19, 2019, 2:26pm UTC](https://discuss.elastic.co/t/kibana-7-3-default-dashboard-for-iis-3-of-5-elements-have-visualization-errors/200080 "2019-09-19T14:26:50Z")
Fresh install ELK Stack v7.3, with Filebeat agent on a Win 2012 Server. I'm using the \[Filebeat IIS\] Access and Error Logs ECS dashboard, and each time it loads, I get three errors about visualization. Two of the Eleme… --- ## [Logstash only ingesting data when shutdown/closing files](https://discuss.elastic.co/t/logstash-only-ingesting-data-when-shutdown-closing-files/200067)
**Author:** [@edster](https://discuss.elastic.co/u/edster)\ **Replies:** 5\ **Last updated:** [September 19, 2019, 2:20pm UTC](https://discuss.elastic.co/t/logstash-only-ingesting-data-when-shutdown-closing-files/200067 "2019-09-19T14:20:02Z")
My logstash config file seems to running successfully, however, it doesn't pull in any data. It isn't until I manually shutdown the process using 'Ctrl+C', that it pulls in data. This doesn't seem to normal behavior. Is … --- ## [Failed to create alias: \](https://discuss.elastic.co/t/failed-to-create-alias-doctype-html-public-ietf-dtd-html-2-0-en/200226)
**Author:** [@jaiganesh06](https://discuss.elastic.co/u/jaiganesh06)\ **Replies:** 0\ **Last updated:** [September 19, 2019, 2:18pm UTC](https://discuss.elastic.co/t/failed-to-create-alias-doctype-html-public-ietf-dtd-html-2-0-en/200226 "2019-09-19T14:18:23Z")
Hi All, Version - 7.3.x I have ES behind Apache 2.4 reverse proxy. At first instance, when no index is created in ES, I am getting below error from filebeat 2019-09-19T19:35:53.281+0530 ERROR pipeline/output.go:100 Fa… [Previous page](https://discuss.elastic.co/tag/none.md?no_tags=true&page=1885) [Next page](https://discuss.elastic.co/tag/none.md?no_tags=true&page=1887)