# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=371 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 372 --- ## [Memory or heap size](https://discuss.elastic.co/t/memory-or-heap-size/228886)
**Author:** [@Lynd](https://discuss.elastic.co/u/Lynd)\ **Replies:** 4\ **Last updated:** [April 20, 2020, 3:17pm UTC](https://discuss.elastic.co/t/memory-or-heap-size/228886 "2020-04-20T15:17:57Z")
What is ideal heap size of elastic search for nearly millions of records --- ## [Rename dynamic nested field](https://discuss.elastic.co/t/rename-dynamic-nested-field/228827)
**Author:** [@banst](https://discuss.elastic.co/u/banst)\ **Replies:** 2\ **Last updated:** [April 20, 2020, 3:13pm UTC](https://discuss.elastic.co/t/rename-dynamic-nested-field/228827 "2020-04-20T15:13:03Z")
Hello there, Giving this event : { field\_name : "foo" nested: { foo: "bar" } } Is there a way with a mutate filter (or another solution) to transform it to : { field\_name : "foo" nested: { foo: "bar… --- ## [Connect to Telegram and get logs in real time](https://discuss.elastic.co/t/connect-to-telegram-and-get-logs-in-real-time/228890)
**Author:** [@Holguinmora](https://discuss.elastic.co/u/Holguinmora)\ **Replies:** 0\ **Last updated:** [April 20, 2020, 2:47pm UTC](https://discuss.elastic.co/t/connect-to-telegram-and-get-logs-in-real-time/228890 "2020-04-20T14:47:05Z")
Hi I´m new and hope to get some help before moving into coding. Is it possible to connect logstash to Telegram and get real time logs? By real time I mean that each time that an account post something in Telegram it tr… --- ## [\[es7.6.2\]\[logstash7.6.2\]Why logstash maps to the default index template when the index name not match the index\_patterns?](https://discuss.elastic.co/t/es7-6-2-logstash7-6-2-why-logstash-maps-to-the-default-index-template-when-the-index-name-not-match-the-index-patterns/228769)
**Author:** [@cheriemilk](https://discuss.elastic.co/u/cheriemilk)\ **Replies:** 1\ **Last updated:** [April 20, 2020, 2:42pm UTC](https://discuss.elastic.co/t/es7-6-2-logstash7-6-2-why-logstash-maps-to-the-default-index-template-when-the-index-name-not-match-the-index-patterns/228769 "2020-04-20T14:42:29Z")
Hi Team, When starting logstash, I see below log which means that my data will be indexed according to default template structure. \[2020-04-16T13:05:48,661\]\[INFO \] \[logstash.outputs.elasticsearch\]\[main\] Using defa… --- ## [Struck Grok for Linux Application Log](https://discuss.elastic.co/t/struck-grok-for-linux-application-log/228884)
**Author:** [@BALA\_BALA](https://discuss.elastic.co/u/BALA_BALA)\ **Replies:** 0\ **Last updated:** [April 20, 2020, 2:24pm UTC](https://discuss.elastic.co/t/struck-grok-for-linux-application-log/228884 "2020-04-20T14:24:18Z")
Hi Friends, I am writing grok for Linux application logs and able to achieve half. Need your suggestions on how to get the remaining data(optional data). Or do we have any other way to get these data? Application lo… --- ## [Logstash taking too long to run](https://discuss.elastic.co/t/logstash-taking-too-long-to-run/228536)
**Author:** [@Robert\_Ga](https://discuss.elastic.co/u/Robert_Ga)\ **Replies:** 7\ **Last updated:** [April 20, 2020, 1:24pm UTC](https://discuss.elastic.co/t/logstash-taking-too-long-to-run/228536 "2020-04-20T13:24:29Z")
Hello everyone, I started to index aprox. 1.000.000 json files into elastic seach using logstash. The logstash config contains some lowercase, renaming operations and if-else statements. I\`m trying to index the data … --- ## [Split nested json array from log](https://discuss.elastic.co/t/split-nested-json-array-from-log/228668)
**Author:** [@rajender](https://discuss.elastic.co/u/rajender)\ **Replies:** 12\ **Last updated:** [April 20, 2020, 12:42pm UTC](https://discuss.elastic.co/t/split-nested-json-array-from-log/228668 "2020-04-20T12:42:31Z")
I would like to to retrieve every element in below JSON to be a field so as to visualize in kibana by applying metrics in dashboard. I need each object in the array msg to be a separate entry in Elasticsearch and ever… --- ## [Confusion with making Logstash HA](https://discuss.elastic.co/t/confusion-with-making-logstash-ha/228855)
**Author:** [@Egert143](https://discuss.elastic.co/u/Egert143)\ **Replies:** 0\ **Last updated:** [April 20, 2020, 11:53am UTC](https://discuss.elastic.co/t/confusion-with-making-logstash-ha/228855 "2020-04-20T11:53:44Z")
Hello First time Elk Stack user. I could use some help with understanding Logstash HA. Currently i have started up 3 nodes (old HW, but they will do for testing). Lets call them Node-A,B,C. Each node is in different phy… --- ## [Syslog with line Codec delimiter change fail the grok parser](https://discuss.elastic.co/t/syslog-with-line-codec-delimiter-change-fail-the-grok-parser/228829)
**Author:** [@realgam3](https://discuss.elastic.co/u/realgam3)\ **Replies:** 0\ **Last updated:** [April 20, 2020, 9:43am UTC](https://discuss.elastic.co/t/syslog-with-line-codec-delimiter-change-fail-the-grok-parser/228829 "2020-04-20T09:43:02Z")
H all, when using this configuration: input { syslog { port =\> 1337 host =\> "0.0.0.0" codec =\> line { delimiter =\> "\\n" } } } with kv filter it fails with … --- ## [Database stable connection](https://discuss.elastic.co/t/database-stable-connection/228808)
**Author:** [@jagan](https://discuss.elastic.co/u/jagan)\ **Replies:** 0\ **Last updated:** [April 20, 2020, 8:24am UTC](https://discuss.elastic.co/t/database-stable-connection/228808 "2020-04-20T08:24:30Z")
Hi guru's, when a kafka job is run i got the following erro: \[ERROR\]\[logstash.pipeline \] er.PhysicalConnection.close(oracle/jdbc/driver/PhysicalConnection.java:2502)", "java.lang.reflect.Method.invoke(java/lang/… --- ## [Can someone please help? Aggregation error. How to use existing index and add additional info?](https://discuss.elastic.co/t/can-someone-please-help-aggregation-error-how-to-use-existing-index-and-add-additional-info/228748)
**Author:** [@Dinesh\_Gupta](https://discuss.elastic.co/u/Dinesh_Gupta)\ **Replies:** 8\ **Last updated:** [April 20, 2020, 12:05am UTC](https://discuss.elastic.co/t/can-someone-please-help-aggregation-error-how-to-use-existing-index-and-add-additional-info/228748 "2020-04-20T00:05:40Z")
I have an index created say, student\_master. I now have a CSV from which I need to map with the ID of the index present and combine them and populate it in a new index. Both have the same ID. I tried using aggregation … --- ## [How to gather logs from a router](https://discuss.elastic.co/t/how-to-gather-logs-from-a-router/228660)
**Author:** [@Hari\_Krishna](https://discuss.elastic.co/u/Hari_Krishna)\ **Replies:** 12\ **Last updated:** [April 20, 2020, 7:45am UTC](https://discuss.elastic.co/t/how-to-gather-logs-from-a-router/228660 "2020-04-20T07:45:54Z")
I have a Juniper router from which i have to send logs to logstash. What should i do in this case --- ## [How to forward logs to newrelic dashboard from logstash](https://discuss.elastic.co/t/how-to-forward-logs-to-newrelic-dashboard-from-logstash/228798)
**Author:** [@ppantang](https://discuss.elastic.co/u/ppantang)\ **Replies:** 0\ **Last updated:** [April 20, 2020, 7:25am UTC](https://discuss.elastic.co/t/how-to-forward-logs-to-newrelic-dashboard-from-logstash/228798 "2020-04-20T07:25:33Z")
anyone help me to to configure the logstash to forward the logs to newrelic dashboard. --- ## [Error in gathering syslogs from router](https://discuss.elastic.co/t/error-in-gathering-syslogs-from-router/228781)
**Author:** [@Hari\_Krishna](https://discuss.elastic.co/u/Hari_Krishna)\ **Replies:** 2\ **Last updated:** [April 20, 2020, 6:21am UTC](https://discuss.elastic.co/t/error-in-gathering-syslogs-from-router/228781 "2020-04-20T06:21:47Z")
input { syslog { host =\> "10.216.123.213" type =\> "syslog" } } exception=\>#\, :backtrace=\>\["… --- ## [Mapping multiple values in single CSV row output](https://discuss.elastic.co/t/mapping-multiple-values-in-single-csv-row-output/227653)
**Author:** [@karnamonkster](https://discuss.elastic.co/u/karnamonkster)\ **Replies:** 9\ **Last updated:** [April 20, 2020, 6:21am UTC](https://discuss.elastic.co/t/mapping-multiple-values-in-single-csv-row-output/227653 "2020-04-20T06:21:29Z")
We have an ES index that has different register for Meters Example: Event1 meter: FT1234 register:FLOW RATE value: somevalue1 Event2 meter:FT1234 register:REV RATE value: somevalue2 Event3 meter:FT1234 register:VOLUM… --- ## [Design options for ingesting syslog data](https://discuss.elastic.co/t/design-options-for-ingesting-syslog-data/228619)
**Author:** [@sera](https://discuss.elastic.co/u/sera)\ **Replies:** 4\ **Last updated:** [April 20, 2020, 6:16am UTC](https://discuss.elastic.co/t/design-options-for-ingesting-syslog-data/228619 "2020-04-20T06:16:20Z")
Greetings! Fairly new to Elastic Stack, so am looking for some high level design advice. Regarding syslog input, my requirement is for network devices (Cisco, Juniper, Pulse Secure, etc) to syslog to Logstash, with a fi… --- ## [Loop through fields to find a matching value](https://discuss.elastic.co/t/loop-through-fields-to-find-a-matching-value/227601)
**Author:** [@krille.com](https://discuss.elastic.co/u/krille.com)\ **Replies:** 3\ **Last updated:** [April 12, 2020, 2:01pm UTC](https://discuss.elastic.co/t/loop-through-fields-to-find-a-matching-value/227601 "2020-04-12T14:01:31Z")
I have 2 fields (integer) called Year and Month e.g. Year = 2016 Month = 9 I have from an array created seqRate.## fields e.g. "rateSeq": { "0": "2016;8;0.0001285760", "1": "2016;9;0.0001287969", "2": "2016;10;0.… --- ## [How can I remove dot from nested object in logstash](https://discuss.elastic.co/t/how-can-i-remove-dot-from-nested-object-in-logstash/228159)
**Author:** [@esfandani](https://discuss.elastic.co/u/esfandani)\ **Replies:** 2\ **Last updated:** [April 20, 2020, 3:50am UTC](https://discuss.elastic.co/t/how-can-i-remove-dot-from-nested-object-in-logstash/228159 "2020-04-20T03:50:21Z")
what goes to elasticsearch is something like this: { "result": "https://www.yahoo.com", "tags": { "url": "https://www.yahoo.com", "projectName": "monitor", "host": "ttt", "dd": 12345, … --- ## [Logstash filter combine](https://discuss.elastic.co/t/logstash-filter-combine/228764)
**Author:** [@VamPikmin](https://discuss.elastic.co/u/VamPikmin)\ **Replies:** 2\ **Last updated:** [April 20, 2020, 12:53am UTC](https://discuss.elastic.co/t/logstash-filter-combine/228764 "2020-04-20T00:53:59Z")
Is it possible to use match and mutate in the same filter? Message is in the following format: \<164\>%ASA-4-722051: Group \ User \ IP \ IPv4 Address \ IPv6 address \<::\> assigned t… --- ## [Logstash Error with configuration](https://discuss.elastic.co/t/logstash-error-with-configuration/228756)
**Author:** [@tom\_marshall](https://discuss.elastic.co/u/tom_marshall)\ **Replies:** 2\ **Last updated:** [April 19, 2020, 9:59pm UTC](https://discuss.elastic.co/t/logstash-error-with-configuration/228756 "2020-04-19T21:59:29Z")
Hi, Can anyone please help with this issue, I am running logstack. I made lesson on Udemy course (https://www.udemy.com/course/elasticsearch-7-and-elastic-stack/learn/lecture/14729000#announcements) data: wget http://… --- ## [Can someone pls help me with my logstash parsers](https://discuss.elastic.co/t/can-someone-pls-help-me-with-my-logstash-parsers/228705)
**Author:** [@Blason](https://discuss.elastic.co/u/Blason)\ **Replies:** 3\ **Last updated:** [April 19, 2020, 7:49pm UTC](https://discuss.elastic.co/t/can-someone-pls-help-me-with-my-logstash-parsers/228705 "2020-04-19T19:49:59Z")
Hi Guys, Somehow my grok pattern is not working and can someone please help? Here is original message 2020-04-18 13:02:20,391 INFO \[ImapSSLServer-345\] \[ip=171.51.237.87;ua=com.samsung.android.email.provider;\] securit… --- ## [JDBC\_Static filter is not mapping the data from postgresql](https://discuss.elastic.co/t/jdbc-static-filter-is-not-mapping-the-data-from-postgresql/228710)
**Author:** [@anjilinga](https://discuss.elastic.co/u/anjilinga)\ **Replies:** 2\ **Last updated:** [April 19, 2020, 7:29pm UTC](https://discuss.elastic.co/t/jdbc-static-filter-is-not-mapping-the-data-from-postgresql/228710 "2020-04-19T19:29:08Z")
I have loded lookup table in postgresql and used jdbc\_static filter to enrich the data. while executing i am getting the below error. In staging directory the data is not loaded. Can some one please help what is the is… --- ## [Can't send data to logstash via tcp port 5959](https://discuss.elastic.co/t/cant-send-data-to-logstash-via-tcp-port-5959/228758)
**Author:** [@ilang](https://discuss.elastic.co/u/ilang)\ **Replies:** 0\ **Last updated:** [April 19, 2020, 7:17pm UTC](https://discuss.elastic.co/t/cant-send-data-to-logstash-via-tcp-port-5959/228758 "2020-04-19T19:17:26Z")
Hi guys, I am trying to send to logstash using tcp port, It is working when the code is running on localhost, however, it should also run remotely. Maybe I need to use different plugin? (or just try to use rest api?) … --- ## [Saving logfiles in another server](https://discuss.elastic.co/t/saving-logfiles-in-another-server/228726)
**Author:** [@Luis\_Alfredo\_Gonzale](https://discuss.elastic.co/u/Luis_Alfredo_Gonzale)\ **Replies:** 0\ **Last updated:** [April 19, 2020, 12:12pm UTC](https://discuss.elastic.co/t/saving-logfiles-in-another-server/228726 "2020-04-19T12:12:21Z")
Hello, I currently im saving my logsfiles to this location, but I need to save in another server I'd like to know what is rigth way to do this.. Im trying to save to this path but it's not working. file{ … --- ## [I need create logfile every 5 minutes](https://discuss.elastic.co/t/i-need-create-logfile-every-5-minutes/228724)
**Author:** [@Luis\_Alfredo\_Gonzale](https://discuss.elastic.co/u/Luis_Alfredo_Gonzale)\ **Replies:** 0\ **Last updated:** [April 19, 2020, 11:44am UTC](https://discuss.elastic.co/t/i-need-create-logfile-every-5-minutes/228724 "2020-04-19T11:44:08Z")
Hello. I need to create a logfile every 5 minutes. My settings is generating a log file every minute but i don't know how to set every 5 minutes. It is: file{ path =\> "/home/govindo/Escritorio/ftpTest-… --- ## [Ruby loop on arrayof hashes - how to add another entry to the hash](https://discuss.elastic.co/t/ruby-loop-on-arrayof-hashes-how-to-add-another-entry-to-the-hash/228639)
**Author:** [@stuart475898](https://discuss.elastic.co/u/stuart475898)\ **Replies:** 2\ **Last updated:** [April 19, 2020, 7:20am UTC](https://discuss.elastic.co/t/ruby-loop-on-arrayof-hashes-how-to-add-another-entry-to-the-hash/228639 "2020-04-19T07:20:19Z")
Hello, I've burnt a few hours on this.... I want to ingest information via SNMP from a network switch. I have successfully configured logstash to grab these data, and now I want to do some filtering before firing it in… --- ## [Logstash and journald](https://discuss.elastic.co/t/logstash-and-journald/228672)
**Author:** [@elastic\_user1](https://discuss.elastic.co/u/elastic_user1)\ **Replies:** 0\ **Last updated:** [April 18, 2020, 6:31pm UTC](https://discuss.elastic.co/t/logstash-and-journald/228672 "2020-04-18T18:31:33Z")
I noticed I could not see all Logstash logs in journalctl -f -u logstash. I have some pipelines with stdout (ruby debug). How can I get all my separate pipelines' logs into the files. I can see only the messages that Lo… --- ## [Load data from mysql to elasticsearch with logstash](https://discuss.elastic.co/t/load-data-from-mysql-to-elasticsearch-with-logstash/228665)
**Author:** [@hiba](https://discuss.elastic.co/u/hiba)\ **Replies:** 0\ **Last updated:** [April 18, 2020, 5:43pm UTC](https://discuss.elastic.co/t/load-data-from-mysql-to-elasticsearch-with-logstash/228665 "2020-04-18T17:43:27Z")
Hi, I want to load data from mysql ( mysql wampserver ) into elasticsearch via logstash. After downloading elasticsearch ,kibana and logstash ,i installed "com.mysql.jdbc\_5.1.5" (JDBC). test.conf input { jdbc { jdb… --- ## [Logstash Add Tag if Message Starts With](https://discuss.elastic.co/t/logstash-add-tag-if-message-starts-with/228607)
**Author:** [@ericbarnes](https://discuss.elastic.co/u/ericbarnes)\ **Replies:** 1\ **Last updated:** [April 18, 2020, 5:08pm UTC](https://discuss.elastic.co/t/logstash-add-tag-if-message-starts-with/228607 "2020-04-18T17:08:56Z")
We are using the Jenkins logstash plugin. So the logstash config is just an input and ouput. The plugin sends pre-formatted json, so not much to do besides collect and forward. We want to enrich based off the type of log… --- ## [How do I take multiple CSV files into a single Elasticsearch index using logstash?](https://discuss.elastic.co/t/how-do-i-take-multiple-csv-files-into-a-single-elasticsearch-index-using-logstash/228661)
**Author:** [@Drashti\_Shah](https://discuss.elastic.co/u/Drashti_Shah)\ **Replies:** 0\ **Last updated:** [April 18, 2020, 4:53pm UTC](https://discuss.elastic.co/t/how-do-i-take-multiple-csv-files-into-a-single-elasticsearch-index-using-logstash/228661 "2020-04-18T16:53:16Z")
I have two CSV files in which, CSV 1 (student\_master) has the following fields: Student ID, First Name, Gender, State and in CSV 2,(student\_marks\_data) I have Student ID, Date, Math, Physics, Chemistry, Total, Percenta… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=370) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=372)