# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=373 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 374 --- ## [Unable to rename host to \[host\]\[name\]](https://discuss.elastic.co/t/unable-to-rename-host-to-host-name/227824)
**Author:** [@boyhittscar](https://discuss.elastic.co/u/boyhittscar)\ **Replies:** 4\ **Last updated:** [April 16, 2020, 5:05pm UTC](https://discuss.elastic.co/t/unable-to-rename-host-to-host-name/227824 "2020-04-16T17:05:17Z")
Hello - I feel like this should be easy, and I'm not really sure why this isn't working. I have the following data host: MYPC endpoint\_type: computer To follow ECS, I'm attempting to do a mutate and rename /nest the… --- ## [Migrate to ECS](https://discuss.elastic.co/t/migrate-to-ecs/228356)
**Author:** [@inandi](https://discuss.elastic.co/u/inandi)\ **Replies:** 0\ **Last updated:** [April 16, 2020, 3:38pm UTC](https://discuss.elastic.co/t/migrate-to-ecs/228356 "2020-04-16T15:38:05Z")
Hi, I want to migrate ECS. my current project log system is below my sample data in kibana is { "\_index": "winlogbeat-6.6.1-2020.04.16", "\_type": "doc", "\_id": "ayaGg3EBM9cZYkQ41G7F", "\_version": 1, "\_scor… --- ## [How to check multiple values using if else condition in field and add new values to another field?](https://discuss.elastic.co/t/how-to-check-multiple-values-using-if-else-condition-in-field-and-add-new-values-to-another-field/228331)
**Author:** [@Emna1](https://discuss.elastic.co/u/Emna1)\ **Replies:** 2\ **Last updated:** [April 16, 2020, 3:10pm UTC](https://discuss.elastic.co/t/how-to-check-multiple-values-using-if-else-condition-in-field-and-add-new-values-to-another-field/228331 "2020-04-16T15:10:55Z")
HI, I want to add a code which contains if else conditions which helps me to add values ​​to a new field according to conditions on another field. for example; Blockquote if \[Duration\] \>= 360 { mutate { add\_field … --- ## [Parsing events before forwarding](https://discuss.elastic.co/t/parsing-events-before-forwarding/228330)
**Author:** [@calebhoch](https://discuss.elastic.co/u/calebhoch)\ **Replies:** 3\ **Last updated:** [April 16, 2020, 2:55pm UTC](https://discuss.elastic.co/t/parsing-events-before-forwarding/228330 "2020-04-16T14:55:00Z")
Hi there, we're looking into leveraging Logstash to forward our system logs to our SIEM, QRadar. I wanted to know if there is a way to filter out specific events with Logstash before forwarding onto QRadar. Any help or … --- ## [How to prevent an occurence of \_aggregatefinalflush tag?](https://discuss.elastic.co/t/how-to-prevent-an-occurence-of-aggregatefinalflush-tag/228284)
**Author:** [@vasek](https://discuss.elastic.co/u/vasek)\ **Replies:** 3\ **Last updated:** [April 16, 2020, 2:23pm UTC](https://discuss.elastic.co/t/how-to-prevent-an-occurence-of-aggregatefinalflush-tag/228284 "2020-04-16T14:23:33Z")
Hello, can someone help me how to prevent an occurence of \_aggregatefinalflush tag? aggregate { ... timeout =\> 5 timeout\_tags =\> \[\] } --- ## [Conditional Filter issue](https://discuss.elastic.co/t/conditional-filter-issue/227990)
**Author:** [@bikramsingh](https://discuss.elastic.co/u/bikramsingh)\ **Replies:** 3\ **Last updated:** [April 16, 2020, 1:54pm UTC](https://discuss.elastic.co/t/conditional-filter-issue/227990 "2020-04-16T13:54:14Z")
I have this filter in the config file, i trying to validate it using in build test config functionality. I am getting error. filter { if( "Network Login" in \[message\]){ if("caps\_fail" in \[message\]){ grok{ … --- ## [How to parse/load JSON file or .TXT file containing JSON to logstash using GROK](https://discuss.elastic.co/t/how-to-parse-load-json-file-or-txt-file-containing-json-to-logstash-using-grok/228246)
**Author:** [@Dhruv\_Mevada](https://discuss.elastic.co/u/Dhruv_Mevada)\ **Replies:** 4\ **Last updated:** [April 16, 2020, 10:47am UTC](https://discuss.elastic.co/t/how-to-parse-load-json-file-or-txt-file-containing-json-to-logstash-using-grok/228246 "2020-04-16T10:47:44Z")
I'm new in ELK & I have logs in JSON format & I can save it to JSON file or txt file. Now, Which will be a better approach should I save logs to JSON file or txt file? How should I parse/load the logs to Logstash, what… --- ## [I want to create grok pattern for custom log](https://discuss.elastic.co/t/i-want-to-create-grok-pattern-for-custom-log/227143)
**Author:** [@Atul\_Gunjal](https://discuss.elastic.co/u/Atul_Gunjal)\ **Replies:** 6\ **Last updated:** [April 16, 2020, 10:19am UTC](https://discuss.elastic.co/t/i-want-to-create-grok-pattern-for-custom-log/227143 "2020-04-16T10:19:15Z")
Hi, Following is a log-pattern and I tried creating one but it didn't work. Log: 2020-04-07 00:00:02 Local0 Info 00:00:01:911 DMSTST-TBOX.DEV.cloud.companyname.com DEV\\DMSTST-TBOX$|17352|DICOM Service VMSDBD\_SVC\_DICOM… --- ## [Cron jobs in http\_poller](https://discuss.elastic.co/t/cron-jobs-in-http-poller/227305)
**Author:** [@akhilsharma.in](https://discuss.elastic.co/u/akhilsharma.in)\ **Replies:** 2\ **Last updated:** [April 16, 2020, 8:47am UTC](https://discuss.elastic.co/t/cron-jobs-in-http-poller/227305 "2020-04-16T08:47:43Z")
Hi, I'm using http\_poller in input and running it via cron job. I have scheduled it for everyday as per my understanding but it is running every 48 hours. Please assist. Cron job: cron =\> "0 5 1-31 \* \* UTC" Platform: … --- ## [Parse and query K8S HAproxy logs by Logstash pipeline filter, filebeat and Kibana](https://discuss.elastic.co/t/parse-and-query-k8s-haproxy-logs-by-logstash-pipeline-filter-filebeat-and-kibana/228276)
**Author:** [@lsambolino](https://discuss.elastic.co/u/lsambolino)\ **Replies:** 0\ **Last updated:** [April 16, 2020, 8:24am UTC](https://discuss.elastic.co/t/parse-and-query-k8s-haproxy-logs-by-logstash-pipeline-filter-filebeat-and-kibana/228276 "2020-04-16T08:24:21Z")
Hi, I am relatively new to ELK. I am trying to parse the K8S HAproxy containers logs via GROK inside the filter config in Logstash. My goal is to query haproxy logs fields easily. On the Kubernetes side, I've deployed… --- ## [How Logstash or filebeat determins the type of logs?](https://discuss.elastic.co/t/how-logstash-or-filebeat-determins-the-type-of-logs/227779)
**Author:** [@gauravbhutani30](https://discuss.elastic.co/u/gauravbhutani30)\ **Replies:** 1\ **Last updated:** [April 16, 2020, 7:33am UTC](https://discuss.elastic.co/t/how-logstash-or-filebeat-determins-the-type-of-logs/227779 "2020-04-16T07:33:33Z")
How does the logstash or the filebeat determine the type of logs, we just provide "type = syslogs" or any other. I want to parse a custom log file, which has text in it. Please help! --- ## [Missing event while report to elastic search](https://discuss.elastic.co/t/missing-event-while-report-to-elastic-search/228262)
**Author:** [@gyana\_nayak](https://discuss.elastic.co/u/gyana_nayak)\ **Replies:** 0\ **Last updated:** [April 16, 2020, 7:06am UTC](https://discuss.elastic.co/t/missing-event-while-report-to-elastic-search/228262 "2020-04-16T07:06:21Z")
Hi All, i could see some events are missing while reporting logs to elastic search. Take an example i am sending 5 logs event only 4 or 3 are reporting. Basically i am using logstash 7.4 to read my log messages and sto… --- ## [How to set index as interface ip dynamically in elasticsearch](https://discuss.elastic.co/t/how-to-set-index-as-interface-ip-dynamically-in-elasticsearch/228114)
**Author:** [@mmk](https://discuss.elastic.co/u/mmk)\ **Replies:** 0\ **Last updated:** [April 15, 2020, 12:22pm UTC](https://discuss.elastic.co/t/how-to-set-index-as-interface-ip-dynamically-in-elasticsearch/228114 "2020-04-15T12:22:33Z")
I am using below config to create index (IP and date), actually i will be running logstash in different host and i want to differentiate each host logs with index value. so i have decided to use interface ip and host na… --- ## [Logstash syslog plugin SSL](https://discuss.elastic.co/t/logstash-syslog-plugin-ssl/228256)
**Author:** [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)\ **Replies:** 0\ **Last updated:** [April 16, 2020, 6:30am UTC](https://discuss.elastic.co/t/logstash-syslog-plugin-ssl/228256 "2020-04-16T06:30:39Z")
Can I implement basic SSL security between my source and logstash Syslog plugin without any third party tools? --- ## [Splitting a string into chunks of a certain size](https://discuss.elastic.co/t/splitting-a-string-into-chunks-of-a-certain-size/228252)
**Author:** [@Sakthi\_Svm](https://discuss.elastic.co/u/Sakthi_Svm)\ **Replies:** 0\ **Last updated:** [April 16, 2020, 6:05am UTC](https://discuss.elastic.co/t/splitting-a-string-into-chunks-of-a-certain-size/228252 "2020-04-16T06:05:06Z")
Hi, I'm in need to split a string in to multiple sub-string of same size. Is it possible to split by same or min length using ruby code? Pasted my message below, {"Name1 True N… --- ## [Input blocking when output is unavailable](https://discuss.elastic.co/t/input-blocking-when-output-is-unavailable/227661)
**Author:** [@orweinberger](https://discuss.elastic.co/u/orweinberger)\ **Replies:** 1\ **Last updated:** [April 16, 2020, 5:25am UTC](https://discuss.elastic.co/t/input-blocking-when-output-is-unavailable/227661 "2020-04-16T05:25:26Z")
I have a TCP input that receives messages, a few filters and a single Elasticsearch output. I'm running a small script that sends messages to the LS TCP port every X miliseconds. After experimenting a bit, I have a few … --- ## [Logstash Memory Requirements and JVM Sizing](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072)
**Author:** [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)\ **Replies:** 8\ **Last updated:** [April 16, 2020, 5:02am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072 "2020-04-16T05:02:01Z")
Hello I am getting approx 5000 events per second and a lot of Grok, KV, DNS filters are being used. My current memory is 16 GB, 8 Core thread per core is 2. I have set JVM size to 8GB. But My JVM is getting full and… --- ## [Logstash and filebeat picking up the old files again and again](https://discuss.elastic.co/t/logstash-and-filebeat-picking-up-the-old-files-again-and-again/228163)
**Author:** [@Ananya\_Mohiley](https://discuss.elastic.co/u/Ananya_Mohiley)\ **Replies:** 2\ **Last updated:** [April 16, 2020, 3:51am UTC](https://discuss.elastic.co/t/logstash-and-filebeat-picking-up-the-old-files-again-and-again/228163 "2020-04-16T03:51:06Z")
i have configured logstash and filebeat on localhost however even when there is no updation of files in the folder the elasticsearch index is being populated by same files repetitively pl suggest remedial measures --- ## [Kibana displays wrong time](https://discuss.elastic.co/t/kibana-displays-wrong-time/228056)
**Author:** [@vikramaddagulla](https://discuss.elastic.co/u/vikramaddagulla)\ **Replies:** 3\ **Last updated:** [April 16, 2020, 3:04am UTC](https://discuss.elastic.co/t/kibana-displays-wrong-time/228056 "2020-04-16T03:04:43Z")
Hello, I have a log file and I am sending that from beats --\> logstash --\> elastic --\> kibana. Below are two lines in the log file : 172.27.88.123 2020-04-15T02:54:15.054Z GET /em 302 311 0.00… --- ## [Date filter can't read from fieldnames which contain dots (created by the JSON filter)](https://discuss.elastic.co/t/date-filter-cant-read-from-fieldnames-which-contain-dots-created-by-the-json-filter/228200)
**Author:** [@dreamspy](https://discuss.elastic.co/u/dreamspy)\ **Replies:** 2\ **Last updated:** [April 16, 2020, 12:25am UTC](https://discuss.elastic.co/t/date-filter-cant-read-from-fieldnames-which-contain-dots-created-by-the-json-filter/228200 "2020-04-16T00:25:46Z")
Hi I'm having some problems reading timestamps from nested JSON fields. Let's say I have this JSON document: {"time" : "2020-04-15T19:17:03.195641"} and I read it with logstash using this filter: filter { json { … --- ## [Cannot send logs from logstash to elasticsearch using systemctl](https://discuss.elastic.co/t/cannot-send-logs-from-logstash-to-elasticsearch-using-systemctl/228209)
**Author:** [@bikramsingh](https://discuss.elastic.co/u/bikramsingh)\ **Replies:** 2\ **Last updated:** [April 15, 2020, 11:19pm UTC](https://discuss.elastic.co/t/cannot-send-logs-from-logstash-to-elasticsearch-using-systemctl/228209 "2020-04-15T23:19:29Z")
Hi when i send using "bin/logstash -f /etc/logstash/conf.d/logstash.conf" I am able to send logs to elasticsearch and see the index in kibana. But I am not able to send logs from logstash to elasticsearch when I am star… --- ## [How to do a conditional regex on or?](https://discuss.elastic.co/t/how-to-do-a-conditional-regex-on-or/228172)
**Author:** [@Amine\_Maalfi](https://discuss.elastic.co/u/Amine_Maalfi)\ **Replies:** 4\ **Last updated:** [April 15, 2020, 9:57pm UTC](https://discuss.elastic.co/t/how-to-do-a-conditional-regex-on-or/228172 "2020-04-15T21:57:48Z")
i have three files named: - foo.log.2020-02-17.log - bar.log - foo.log.17-02-2020 i need to extract the whole file name and the date aswell i tried this expression but it's not working for me: (?\< app\_name \>.\*(?\< lo… --- ## [Config field as date](https://discuss.elastic.co/t/config-field-as-date/228112)
**Author:** [@peretz\_shlomi](https://discuss.elastic.co/u/peretz_shlomi)\ **Replies:** 2\ **Last updated:** [April 15, 2020, 9:16pm UTC](https://discuss.elastic.co/t/config-field-as-date/228112 "2020-04-15T21:16:33Z")
hi Hi I'm trying to import a file csv that in one of the fields has a field of time according to the above format 20200402 133001 Also attached is the definition in the file filter { csv { separator =\> "," column… --- ## [Filter a CSV with a CSV](https://discuss.elastic.co/t/filter-a-csv-with-a-csv/228091)
**Author:** [@bazza](https://discuss.elastic.co/u/bazza)\ **Replies:** 3\ **Last updated:** [April 15, 2020, 9:14pm UTC](https://discuss.elastic.co/t/filter-a-csv-with-a-csv/228091 "2020-04-15T21:14:33Z")
Hi, I have a log entry from mysql\_audit.so that creates a CSV based log file. I have been able to filter this in to ELK no problems, but I have one type of log entry that contains comma delimited entries within the col… --- ## [Configuring GeoIP information in Logstash for Kibana SIEM Network Map](https://discuss.elastic.co/t/configuring-geoip-information-in-logstash-for-kibana-siem-network-map/228203)
**Author:** [@rgeisman](https://discuss.elastic.co/u/rgeisman)\ **Replies:** 0\ **Last updated:** [April 15, 2020, 8:55pm UTC](https://discuss.elastic.co/t/configuring-geoip-information-in-logstash-for-kibana-siem-network-map/228203 "2020-04-15T20:55:25Z")
Hello All, I am working with a previous ELK stack that was setup to just use beats to go to Elasticsearch. I have upgraded he cluster and now using Logstash. Previously the SIEM Network Map worked because a "geoip-inf… --- ## [Marking url as dead. Last error: HostUnreachableError](https://discuss.elastic.co/t/marking-url-as-dead-last-error-hostunreachableerror/228202)
**Author:** [@Chris\_Wilmott](https://discuss.elastic.co/u/Chris_Wilmott)\ **Replies:** 0\ **Last updated:** [April 15, 2020, 8:37pm UTC](https://discuss.elastic.co/t/marking-url-as-dead-last-error-hostunreachableerror/228202 "2020-04-15T20:37:37Z")
Hi, I have updated my ElasticSearch cluster from 6.6 to 7.6.2, as part of this upgrade the \_all field and the \_default\_ fields were removed. I updated the cluster, checked the logs, and sure enough, one of my templates… --- ## [Ruby code to push whole map to event upon aggregate end\_of\_task](https://discuss.elastic.co/t/ruby-code-to-push-whole-map-to-event-upon-aggregate-end-of-task/228174)
**Author:** [@kevin\_branch](https://discuss.elastic.co/u/kevin_branch)\ **Replies:** 2\ **Last updated:** [April 15, 2020, 8:17pm UTC](https://discuss.elastic.co/t/ruby-code-to-push-whole-map-to-event-upon-aggregate-end-of-task/228174 "2020-04-15T20:17:00Z")
I understand with aggregate that push\_map\_as\_event\_on\_timeout can be used to push the whole aggregated map to the event when there is a timeout, but that there is nothing like a push\_map\_as\_event\_on\_end\_of\_task. What wo… --- ## [Translate with array fields](https://discuss.elastic.co/t/translate-with-array-fields/228196)
**Author:** [@mtudisco](https://discuss.elastic.co/u/mtudisco)\ **Replies:** 0\ **Last updated:** [April 15, 2020, 7:22pm UTC](https://discuss.elastic.co/t/translate-with-array-fields/228196 "2020-04-15T19:22:20Z")
Hi, Im using winlogbeat to send information to logstash and then all the information goes to a log in filesystem and some information goes to elasticsearch. To identify which information to store in ES, I identify cert… --- ## [Could not index event to Elasticsearch. Root mapping definition has unsupported parameters](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-root-mapping-definition-has-unsupported-parameters/228176)
**Author:** [@Chris\_Wilmott](https://discuss.elastic.co/u/Chris_Wilmott)\ **Replies:** 4\ **Last updated:** [April 15, 2020, 6:32pm UTC](https://discuss.elastic.co/t/could-not-index-event-to-elasticsearch-root-mapping-definition-has-unsupported-parameters/228176 "2020-04-15T18:32:51Z")
Hi, I have recently upgraded from 6.6 to 7.6.2 and it all went smoothly enough. However I checked my logs this morning and have found the following: \[2020-04-15T15:50:50,851\]\[WARN \]\[logstash.outputs.elasticsearch\] Coul… --- ## [Nginx LoadBalancing to Logstash](https://discuss.elastic.co/t/nginx-loadbalancing-to-logstash/228177)
**Author:** [@luiz.miguelsl](https://discuss.elastic.co/u/luiz.miguelsl)\ **Replies:** 1\ **Last updated:** [April 15, 2020, 5:54pm UTC](https://discuss.elastic.co/t/nginx-loadbalancing-to-logstash/228177 "2020-04-15T17:54:38Z")
Hi there guys, I hope you're all safe. I have two Logstash instances behind an Nginx as the load balancer. The problem is that load balancing isn't happening properly. Always, one server is receiving more traffic than t… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=372) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=374)