# Elastic Stack **URL:** https://discuss.elastic.co/c/elastic-stack/81.md?page=1696 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1697 --- ## [Why Filebeat is not picking up multiline ISO timestamp pattern but picking when pattern is defined manually?](https://discuss.elastic.co/t/why-filebeat-is-not-picking-up-multiline-iso-timestamp-pattern-but-picking-when-pattern-is-defined-manually/255036)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 2\ **Last updated:** [November 16, 2020, 12:17pm UTC](https://discuss.elastic.co/t/why-filebeat-is-not-picking-up-multiline-iso-timestamp-pattern-but-picking-when-pattern-is-defined-manually/255036 "2020-11-16T12:17:24Z")
Hi, I have log lines like below; \[2020-11-11T11:19:00+05:30\] \[INFO\] \[msg1\] \[msg2\] \[msg3\] \[msg4\] \[msg5\] \[msg6\] \[msg7\] \[msg8\] \[msg9\] The msg9 attribute may contain multi-line message, so I have decided to configure mult… --- ## [Cluster with separate roles docker/ansible](https://discuss.elastic.co/t/cluster-with-separate-roles-docker-ansible/254977)
**Author:** [@vinci](https://discuss.elastic.co/u/vinci)\ **Replies:** 1\ **Last updated:** [November 16, 2020, 11:25am UTC](https://discuss.elastic.co/t/cluster-with-separate-roles-docker-ansible/254977 "2020-11-16T11:25:41Z")
Hello, I'm trying to set up an elasticsearch cluster with 3 nodes, one master, the other two data. I'm doing this all through ansible and in order to also set up the ssl connection between the nodes, I'm making use of a… --- ## [Kibana Geo\_json Map](https://discuss.elastic.co/t/kibana-geo-json-map/255482)
**Author:** [@Farid\_N](https://discuss.elastic.co/u/Farid_N)\ **Replies:** 5\ **Last updated:** [November 16, 2020, 11:25am UTC](https://discuss.elastic.co/t/kibana-geo-json-map/255482 "2020-11-16T11:25:31Z")
Hi Unfortunately, I can not import a JSON file to kibana map Could you please help me what should I do for this issue: I have downloaded many many JSON file... but every time I faced with this message :frowning: --- ## [Boolean Logic in Synonym Set?](https://discuss.elastic.co/t/boolean-logic-in-synonym-set/255384)
**Author:** [@ALK1](https://discuss.elastic.co/u/ALK1)\ **Replies:** 1\ **Last updated:** [November 16, 2020, 11:20am UTC](https://discuss.elastic.co/t/boolean-logic-in-synonym-set/255384 "2020-11-16T11:20:02Z")
Hi all, I’m trying to create a synonym set for an acronym so it will match against an exact phrase, like "United Stated of America" for USA. It seems I can’t put quotation marks in the synonym set fields. Does anyone kn… --- ## [Behaviour of ES when using multiple path in path.data](https://discuss.elastic.co/t/behaviour-of-es-when-using-multiple-path-in-path-data/255483)
**Author:** [@anon17386273](https://discuss.elastic.co/u/anon17386273)\ **Replies:** 1\ **Last updated:** [November 16, 2020, 11:05am UTC](https://discuss.elastic.co/t/behaviour-of-es-when-using-multiple-path-in-path-data/255483 "2020-11-16T11:05:31Z")
I would like to know how data is stored in ES when using multiple paths Each folder paths are RAID0. If a hard disk fails, how does ES allocate the data? Any help is appreciated. Thank you --- ## [Large file chunking with Ingest-Attachment](https://discuss.elastic.co/t/large-file-chunking-with-ingest-attachment/255134)
**Author:** [@MBResolution](https://discuss.elastic.co/u/MBResolution)\ **Replies:** 1\ **Last updated:** [November 16, 2020, 10:49am UTC](https://discuss.elastic.co/t/large-file-chunking-with-ingest-attachment/255134 "2020-11-16T10:49:44Z")
We have recently upgraded ElasticSearch from version 2 to version 7.9. We index attachments like PDFs, Word and Excel files. We previously used the “mapper-attachment” plugin for this. Mapper-attachment is not supported … --- ## [eTLD+1 seems to be eTLD+2 in some cases](https://discuss.elastic.co/t/etld-1-seems-to-be-etld-2-in-some-cases/250532)
**Author:** [@willemdh](https://discuss.elastic.co/u/willemdh)\ **Replies:** 3\ **Last updated:** [November 16, 2020, 10:45am UTC](https://discuss.elastic.co/t/etld-1-seems-to-be-etld-2-in-some-cases/250532 "2020-11-16T10:45:14Z")
Hello, Noticed in the packetbeat\_dns\_tunneling ml job that some etld+1's seem incorrect: The third column is dns.question.etld\_plus\_one Afaik an etld+1 should consist of 2 parts and 1 dot? For example dns.question.… --- ## [Regex Pattern to read log from application server](https://discuss.elastic.co/t/regex-pattern-to-read-log-from-application-server/255492)
**Author:** [@Ganesh2303](https://discuss.elastic.co/u/Ganesh2303)\ **Replies:** 0\ **Last updated:** [November 16, 2020, 10:40am UTC](https://discuss.elastic.co/t/regex-pattern-to-read-log-from-application-server/255492 "2020-11-16T10:40:22Z")
HI Team, WE are using ELK 6.4.0 in production and by using Rsyslog service we are sending log from application server to logstash. In application log we have multline data and we are not sure which we can use to send t… --- ## [32-BIT Metricbeat (7.8.0) return abnormal value for system.network.in.bytes](https://discuss.elastic.co/t/32-bit-metricbeat-7-8-0-return-abnormal-value-for-system-network-in-bytes/254369)
**Author:** [@Razby](https://discuss.elastic.co/u/Razby)\ **Replies:** 4\ **Last updated:** [November 16, 2020, 10:00am UTC](https://discuss.elastic.co/t/32-bit-metricbeat-7-8-0-return-abnormal-value-for-system-network-in-bytes/254369 "2020-11-16T10:00:35Z")
Hi, 32-BIT Metricbeat (7.8.0) return abnormal high value for system.network.in.bytes such as "6,424,932,420,939,677,696". :worried: With 64-BIT version I got acceptable value such as "3,652,142,427" I really don’t kno… --- ## [Create Elastic index from Python script](https://discuss.elastic.co/t/create-elastic-index-from-python-script/255354)
**Author:** [@Jose\_Angel\_Morena\_Si](https://discuss.elastic.co/u/Jose_Angel_Morena_Si)\ **Replies:** 2\ **Last updated:** [November 16, 2020, 9:50am UTC](https://discuss.elastic.co/t/create-elastic-index-from-python-script/255354 "2020-11-16T09:50:41Z")
Hello, I am trying to create an index in ES 7.9.1 with python 3. In order to do so I am using elasticsearch python package. Script: doc\_id=hex(random.getrandbits(128))\[2:\] doc = { 'response\_time': … --- ## [Assigning ILM for metricbeat](https://discuss.elastic.co/t/assigning-ilm-for-metricbeat/253160)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 5\ **Last updated:** [November 16, 2020, 9:49am UTC](https://discuss.elastic.co/t/assigning-ilm-for-metricbeat/253160 "2020-11-16T09:49:08Z")
I have started metricbeat used custome field.yml file to load template. I have following in metricbeat.yml setup.template.fields: "fields.yml" setup.ilm.enabled: auto setup.template.overwrite: true now I started met… --- ## [On Windows system.memory.swap.total shows 2GB although there is no swap](https://discuss.elastic.co/t/on-windows-system-memory-swap-total-shows-2gb-although-there-is-no-swap/254663)
**Author:** [@goodmirek5849](https://discuss.elastic.co/u/goodmirek5849)\ **Replies:** 4\ **Last updated:** [November 16, 2020, 9:29am UTC](https://discuss.elastic.co/t/on-windows-system-memory-swap-total-shows-2gb-although-there-is-no-swap/254663 "2020-11-16T09:29:13Z")
We run metricbeat-oss-7.9.2-windows-x86\_64.zip on Windows Server 2016 and send data to Kafka. From Kafka, we consume data with Logstash and send them to ElasticCloud. We observe an issue with system.memory.swap.\* metric… --- ## [Best query to only return documents matching a nested unique id 'id' and not the ES \_id field?](https://discuss.elastic.co/t/best-query-to-only-return-documents-matching-a-nested-unique-id-id-and-not-the-es-id-field/254986)
**Author:** [@M\_Gibbs](https://discuss.elastic.co/u/M_Gibbs)\ **Replies:** 8\ **Last updated:** [November 16, 2020, 9:26am UTC](https://discuss.elastic.co/t/best-query-to-only-return-documents-matching-a-nested-unique-id-id-and-not-the-es-id-field/254986 "2020-11-16T09:26:30Z")
Hey all, I've been pulling my hair out a bit with this. We are running a basic query which matches a set of unique id's - this works perfectly when querying the ES \_id field. But when we try to match a nested id field… --- ## [LambdaConversionException Invalid receiver type interface org.apache.http.Header -from RestHighLevelClient in Logstash Java plugin](https://discuss.elastic.co/t/lambdaconversionexception-invalid-receiver-type-interface-org-apache-http-header-from-resthighlevelclient-in-logstash-java-plugin/255393)
**Author:** [@hellopcz](https://discuss.elastic.co/u/hellopcz)\ **Replies:** 4\ **Last updated:** [November 16, 2020, 9:17am UTC](https://discuss.elastic.co/t/lambdaconversionexception-invalid-receiver-type-interface-org-apache-http-header-from-resthighlevelclient-in-logstash-java-plugin/255393 "2020-11-16T09:17:55Z")
My RestHighLevelClient can run well in Win10 IDEA enviroment for testing, but when I "gradlew gem" the gem package, and run it in CentOS8 logstash-plugin enviroment(Logstash 7.9.0), there will be: \[2020-11-10T00:09:04,… --- ## ["reason": "\[nested\] nested path \[variants.string\_facet\] is not nested"](https://discuss.elastic.co/t/reason-nested-nested-path-variants-string-facet-is-not-nested/255448)
**Author:** [@Guillaume\_R](https://discuss.elastic.co/u/Guillaume_R)\ **Replies:** 1\ **Last updated:** [November 16, 2020, 8:28am UTC](https://discuss.elastic.co/t/reason-nested-nested-path-variants-string-facet-is-not-nested/255448 "2020-11-16T08:28:20Z")
Hi all, I'm facing an error and I really don't understand the problem. I'm using ES6.8. If any one could help me, I would really appreciate. PUT prestashop\_ch { "settings": { "number\_of\_shards": 1, … --- ## [Ship login history](https://discuss.elastic.co/t/ship-login-history/255437)
**Author:** [@headtea](https://discuss.elastic.co/u/headtea)\ **Replies:** 1\ **Last updated:** [November 15, 2020, 2:57pm UTC](https://discuss.elastic.co/t/ship-login-history/255437 "2020-11-15T14:57:22Z")
On all servers, I use Filebeat and Auditbeat. While I use filebeat for all sorts of things - I only use auditbeat to ship system logins (who attempted to log-on, from where, successful or not). It comes with Auditbeat's … --- ## [Elasticsearch 7.10.0 support Tableau with JDBC driver?](https://discuss.elastic.co/t/elasticsearch-7-10-0-support-tableau-with-jdbc-driver/255364)
**Author:** [@FTLJ](https://discuss.elastic.co/u/FTLJ)\ **Replies:** 1\ **Last updated:** [November 16, 2020, 8:11am UTC](https://discuss.elastic.co/t/elasticsearch-7-10-0-support-tableau-with-jdbc-driver/255364 "2020-11-16T08:11:50Z")
Will 7.10 support JDBC driver and Elasticsearch Connector for Tableau as same as 7.9? Version 7.9.0 Tableau Desktop uses JDBC driver and Elasticsearch Connector for Tableau. But Version 7.10.0 Tableau Desktop uses … --- ## [Scripted field from sql api not working](https://discuss.elastic.co/t/scripted-field-from-sql-api-not-working/253464)
**Author:** [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)\ **Replies:** 6\ **Last updated:** [November 16, 2020, 8:08am UTC](https://discuss.elastic.co/t/scripted-field-from-sql-api-not-working/253464 "2020-11-16T08:08:51Z")
Hi All, I recently started using sql from elasticsearch. i am trying to use below API POST \_sql?format=csv { "query": "SELECT SERVICE\_NAME,Success,count() From"my\_inde" WHERE "@timestamp" \>= NOW() - INTERVAL 3 HOURS… --- ## [Using s3 input plugin logstash file\_descriptors aren't closed leading to crash of system](https://discuss.elastic.co/t/using-s3-input-plugin-logstash-file-descriptors-arent-closed-leading-to-crash-of-system/254270)
**Author:** [@petersedivec](https://discuss.elastic.co/u/petersedivec)\ **Replies:** 4\ **Last updated:** [November 16, 2020, 8:03am UTC](https://discuss.elastic.co/t/using-s3-input-plugin-logstash-file-descriptors-arent-closed-leading-to-crash-of-system/254270 "2020-11-16T08:03:22Z")
Been trying to figure out our problem for the past week and haven't figured out a solution other than restart logstash multiple times when it runs out of resources. Looking for any help or suggestions on what we can try… --- ## [Shipping all ElasticSearch logs to AWS S3 on Regular Basis](https://discuss.elastic.co/t/shipping-all-elasticsearch-logs-to-aws-s3-on-regular-basis/252776)
**Author:** [@inf](https://discuss.elastic.co/u/inf)\ **Replies:** 2\ **Last updated:** [November 16, 2020, 7:06am UTC](https://discuss.elastic.co/t/shipping-all-elasticsearch-logs-to-aws-s3-on-regular-basis/252776 "2020-11-16T07:06:44Z")
As title goes, I am currently looking for a solution to ship all ElasticSearch logs to AWS S3 on a regular basis, with the aim of having my EC2 instance hosting the ElasticStack (yes, I have only got one as a trial into … --- ## [Docker pull / error: authenticationrequired](https://discuss.elastic.co/t/docker-pull-error-authenticationrequired/255220)
**Author:** [@ricki1234](https://discuss.elastic.co/u/ricki1234)\ **Replies:** 4\ **Last updated:** [November 16, 2020, 5:59am UTC](https://discuss.elastic.co/t/docker-pull-error-authenticationrequired/255220 "2020-11-16T05:59:12Z")
I tried to pull the latest docker-images and got this error: docker pull docker.elastic.co/elasticsearch/elasticsearch:7.10.0 Error response from daemon: Get https://docker.elastic.co/v2/: authenticationrequired We're… --- ## [Disk Usage problem](https://discuss.elastic.co/t/disk-usage-problem/255292)
**Author:** [@since1976](https://discuss.elastic.co/u/since1976)\ **Replies:** 3\ **Last updated:** [November 16, 2020, 5:26am UTC](https://discuss.elastic.co/t/disk-usage-problem/255292 "2020-11-16T05:26:52Z")
Hi, A bit of background in our environment, our logstash is installed with elastiflow. 1 cluster of 4 nodes elasticsearch with 2 P and 2 R in 2 zones. The setup is collecting logs from 2 WAN routers with netflow configu… --- ## [Secure Communications from Beats to Elasticsearch](https://discuss.elastic.co/t/secure-communications-from-beats-to-elasticsearch/255460)
**Author:** [@inf](https://discuss.elastic.co/u/inf)\ **Replies:** 3\ **Last updated:** [November 16, 2020, 4:27am UTC](https://discuss.elastic.co/t/secure-communications-from-beats-to-elasticsearch/255460 "2020-11-16T04:27:27Z")
Using Filebeat's article on ... found here as reference, we see that it says that "Filebeat can use any of the following authentication methods". 3 authentication methods were then listed. My question is: are we only ab… --- ## [Logstash to log file processing start and end time](https://discuss.elastic.co/t/logstash-to-log-file-processing-start-and-end-time/255462)
**Author:** [@sam281](https://discuss.elastic.co/u/sam281)\ **Replies:** 2\ **Last updated:** [November 16, 2020, 3:59am UTC](https://discuss.elastic.co/t/logstash-to-log-file-processing-start-and-end-time/255462 "2020-11-16T03:59:41Z")
Hello, I am trying to forward like 365 gzip historical data files to another log management tool through syslog output plugin and using file input with Read mode. I need to track and log timestamp for every file when i… --- ## [Logstash saving data into two indices (Duplicated Data)](https://discuss.elastic.co/t/logstash-saving-data-into-two-indices-duplicated-data/255457)
**Author:** [@gonzalo2kx](https://discuss.elastic.co/u/gonzalo2kx)\ **Replies:** 1\ **Last updated:** [November 16, 2020, 12:16am UTC](https://discuss.elastic.co/t/logstash-saving-data-into-two-indices-duplicated-data/255457 "2020-11-16T00:16:00Z")
Good day, I am currently processing the data from filebeat's system module using Logstash. The data provided by filebeat's system module (syslog and auth) is properly parsed but unfortunately it was saving the structure… --- ## [How to write json array to elasticsearch's objetct field by es rest client?](https://discuss.elastic.co/t/how-to-write-json-array-to-elasticsearchs-objetct-field-by-es-rest-client/255224)
**Author:** [@KaimingWan](https://discuss.elastic.co/u/KaimingWan)\ **Replies:** 0\ **Last updated:** [November 12, 2020, 2:24pm UTC](https://discuss.elastic.co/t/how-to-write-json-array-to-elasticsearchs-objetct-field-by-es-rest-client/255224 "2020-11-12T14:24:03Z")
I use elasticsearch6.8 and jdk8. The key problem is hot to write json array string to elasticsearch's object type field. Following code show how I tried to write to es. My mapping is as follows: { "mapping": { "\_… --- ## [Stack monitoring issues on single node cluster](https://discuss.elastic.co/t/stack-monitoring-issues-on-single-node-cluster/255355)
**Author:** [@DanLS](https://discuss.elastic.co/u/DanLS)\ **Replies:** 3\ **Last updated:** [November 16, 2020, 12:02am UTC](https://discuss.elastic.co/t/stack-monitoring-issues-on-single-node-cluster/255355 "2020-11-16T00:02:18Z")
Hi, trying to set up node monitoring with metricbeat on a single node cluster (i know its best practise to have a monitoring node but this is just for testing etc) i've followed the install and looking at the logs it ap… --- ## [this.uiSettings is undefined](https://discuss.elastic.co/t/this-uisettings-is-undefined/251689)
**Author:** [@Kenneth\_M\_Kolano](https://discuss.elastic.co/u/Kenneth_M_Kolano)\ **Replies:** 9\ **Last updated:** [November 15, 2020, 10:10pm UTC](https://discuss.elastic.co/t/this-uisettings-is-undefined/251689 "2020-11-15T22:10:12Z")
After updating to a recent release of ELK and importing my old discover pane and visualizations, besides one using Timelion, all fail to display showing only the error... "this.uiSettings is undefined"? --- ## [Filebeat Nginx ingress\_controller doesn't parse message properly](https://discuss.elastic.co/t/filebeat-nginx-ingress-controller-doesnt-parse-message-properly/255423)
**Author:** [@zbioe](https://discuss.elastic.co/u/zbioe)\ **Replies:** 0\ **Last updated:** [November 15, 2020, 5:25am UTC](https://discuss.elastic.co/t/filebeat-nginx-ingress-controller-doesnt-parse-message-properly/255423 "2020-11-15T05:25:36Z")
We setted the pipeline for filebeat to get the logs from ingress-nginx inside kubernetes. In output it appears with event.dataset: "nginx.ingress\_controller", but it don't parse properlly. We can't see any dashboard, a… --- ## [Node cold but phase hot](https://discuss.elastic.co/t/node-cold-but-phase-hot/254826)
**Author:** [@alexfritz](https://discuss.elastic.co/u/alexfritz)\ **Replies:** 7\ **Last updated:** [November 15, 2020, 8:55pm UTC](https://discuss.elastic.co/t/node-cold-but-phase-hot/254826 "2020-11-15T20:55:20Z")
Hello I am running Version: 6.8 when I look at \_cat/shards/$index it says that the node is cold. When I look at $index/\_ilm/explain it says the index is in phase hot: { "indices": { "7a900401ea8845819a5cf5931… [Previous page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1695) [Next page](https://discuss.elastic.co/c/elastic-stack/81.md?page=1697)