# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=602 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 603 --- ## [Distinguish information from different logstash running netflow modules](https://discuss.elastic.co/t/distinguish-information-from-different-logstash-running-netflow-modules/168147)
**Author:** [@folderman](https://discuss.elastic.co/u/folderman)\ **Replies:** 0\ **Last updated:** [February 13, 2019, 5:41am UTC](https://discuss.elastic.co/t/distinguish-information-from-different-logstash-running-netflow-modules/168147 "2019-02-13T05:41:36Z")
Hi All, I have multiple logstash running netflow module, and all logstash will send the data back to the same elasticsearch cluster to the same index. My question is, how can I distinguish the data sending from differe… --- ## [High CPU even with low events](https://discuss.elastic.co/t/high-cpu-even-with-low-events/168143)
**Author:** [@epacke](https://discuss.elastic.co/u/epacke)\ **Replies:** 0\ **Last updated:** [February 13, 2019, 4:50am UTC](https://discuss.elastic.co/t/high-cpu-even-with-low-events/168143 "2019-02-13T04:50:59Z")
Hi! I'm indexing Apache logs sent with filebeat (around 3000 documents per day). Although low amounts of data is still was using up all the CPU it could get: Turns out it was a configuration problem: I had started l… --- ## [How to Handle Metadata in File Headers](https://discuss.elastic.co/t/how-to-handle-metadata-in-file-headers/167544)
**Author:** [@dips](https://discuss.elastic.co/u/dips)\ **Replies:** 12\ **Last updated:** [February 12, 2019, 10:57pm UTC](https://discuss.elastic.co/t/how-to-handle-metadata-in-file-headers/167544 "2019-02-12T22:57:55Z")
I am using Elasticsearch to collect data from log files. The log files contain metadata in the header relating to the user that logged on. I want to be able to search for events using the metadata in the file header (exa… --- ## [Send syslog directly to logstash](https://discuss.elastic.co/t/send-syslog-directly-to-logstash/168118)
**Author:** [@ray\_zuniga](https://discuss.elastic.co/u/ray_zuniga)\ **Replies:** 0\ **Last updated:** [February 12, 2019, 9:52pm UTC](https://discuss.elastic.co/t/send-syslog-directly-to-logstash/168118 "2019-02-12T21:52:02Z")
I have a dev elk stack up and running. each service on a diff srv. It currently receives logs from a server that has filbert installed on it and it receives the logs that way. My question is for sending logs from a devi… --- ## [Add filter to send specific fields to elasticsearch](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096)
**Author:** [@adityak248](https://discuss.elastic.co/u/adityak248)\ **Replies:** 11\ **Last updated:** [February 12, 2019, 9:22pm UTC](https://discuss.elastic.co/t/add-filter-to-send-specific-fields-to-elasticsearch/168096 "2019-02-12T21:22:33Z")
Hello Team, Greetings I am looking to add only specific fields to send via filebeat or I am looking to receive only specific fields in elasticsearch or I am looking for Logstash to filter the incoming fields and stor… --- ## [Wrong values are populated into date field using Logstash](https://discuss.elastic.co/t/wrong-values-are-populated-into-date-field-using-logstash/167865)
**Author:** [@Ram18](https://discuss.elastic.co/u/Ram18)\ **Replies:** 10\ **Last updated:** [February 12, 2019, 7:16pm UTC](https://discuss.elastic.co/t/wrong-values-are-populated-into-date-field-using-logstash/167865 "2019-02-12T19:16:33Z")
Hello, I could see the date field are populated wrongly into Elasticsearch. I am generating CSV file and inserting the data through logstash conf file. Below is the screenshot where I have highlighted the date columns … --- ## [Logstash Error 403](https://discuss.elastic.co/t/logstash-error-403/167917)
**Author:** [@danivanrock](https://discuss.elastic.co/u/danivanrock)\ **Replies:** 8\ **Last updated:** [February 12, 2019, 6:19pm UTC](https://discuss.elastic.co/t/logstash-error-403/167917 "2019-02-12T18:19:33Z")
Hi everyone Logstash keeps crashing with 403 error, I know this error its caused by no available space on disk, on that moment I had 30 Gb, now I attached 100 Gb on disk and keeps crashing I tried removing Logstash, in… --- ## [Can't get logstash-output-icinga to work](https://discuss.elastic.co/t/cant-get-logstash-output-icinga-to-work/168077)
**Author:** [@HansM](https://discuss.elastic.co/u/HansM)\ **Replies:** 0\ **Last updated:** [February 12, 2019, 4:24pm UTC](https://discuss.elastic.co/t/cant-get-logstash-output-icinga-to-work/168077 "2019-02-12T16:24:30Z")
Hello everybody, hope you can give m ea hand with this, I have installed Logstash 6.6.0 some days ago. And everything is working except the icinga pluging I have been testing the logstash-output-icinga and is giving me… --- ## [Setup Logstash to run via SSL (No FileBeats)](https://discuss.elastic.co/t/setup-logstash-to-run-via-ssl-no-filebeats/167636)
**Author:** [@titan1978](https://discuss.elastic.co/u/titan1978)\ **Replies:** 9\ **Last updated:** [February 12, 2019, 2:17pm UTC](https://discuss.elastic.co/t/setup-logstash-to-run-via-ssl-no-filebeats/167636 "2019-02-12T14:17:12Z")
Team, Am looking to see how I can enable TLS/SSL (similar to how Kibana can be run via SSL). The intent is to startup Logstash on an EC2 on SSL so that the ALB that the EC2 is registered does traffic/health on HTTPS and … --- ## [Rename fields with index suffix in field name](https://discuss.elastic.co/t/rename-fields-with-index-suffix-in-field-name/167996)
**Author:** [@rudok](https://discuss.elastic.co/u/rudok)\ **Replies:** 2\ **Last updated:** [February 12, 2019, 1:38pm UTC](https://discuss.elastic.co/t/rename-fields-with-index-suffix-in-field-name/167996 "2019-02-12T13:38:18Z")
Hi, Is it possible to rename field containing .index suffix to different name? Like IF-MIB::ifAdminStatus.10 =\> AdminStatus First try ( of course not workig" was: filter { mutate { rename =\> \[ "IF-MIB::ifAdminSta… --- ## [Unable to push incremental data from Cassandra DB to daily index using logstash](https://discuss.elastic.co/t/unable-to-push-incremental-data-from-cassandra-db-to-daily-index-using-logstash/166324)
**Author:** [@Sapna](https://discuss.elastic.co/u/Sapna)\ **Replies:** 24\ **Last updated:** [February 12, 2019, 10:42am UTC](https://discuss.elastic.co/t/unable-to-push-incremental-data-from-cassandra-db-to-daily-index-using-logstash/166324 "2019-02-12T10:42:23Z")
Hello, I am trying to create a daily index to fetch data from Cassandra DB using JDBC driver logstash. As soon it creates a new index with new date it is again pulling the whole data , which is already available in yes… --- ## [Parsing complicated XML - extracting nested fields and splitting](https://discuss.elastic.co/t/parsing-complicated-xml-extracting-nested-fields-and-splitting/167863)
**Author:** [@Skwilly](https://discuss.elastic.co/u/Skwilly)\ **Replies:** 4\ **Last updated:** [February 12, 2019, 10:00am UTC](https://discuss.elastic.co/t/parsing-complicated-xml-extracting-nested-fields-and-splitting/167863 "2019-02-12T10:00:15Z")
Hello, I have the following multiline XML: \ \ \ \z2\ \ \z2\… --- ## [Indexing data from kafka to elasticsearch](https://discuss.elastic.co/t/indexing-data-from-kafka-to-elasticsearch/168006)
**Author:** [@B123](https://discuss.elastic.co/u/B123)\ **Replies:** 0\ **Last updated:** [February 12, 2019, 9:57am UTC](https://discuss.elastic.co/t/indexing-data-from-kafka-to-elasticsearch/168006 "2019-02-12T09:57:19Z")
Hi, I am trying to index data from an existing topic in kafka to elasticserach. I am using the kafka input plugin in my logstash pipeline . Providing the below bootstrap\_servers =\> "localhost:port" topics =\> "topic\_na… --- ## [At least once delivery setup with logstash kafka](https://discuss.elastic.co/t/at-least-once-delivery-setup-with-logstash-kafka/167998)
**Author:** [@JohanRask](https://discuss.elastic.co/u/JohanRask)\ **Replies:** 4\ **Last updated:** [February 12, 2019, 9:54am UTC](https://discuss.elastic.co/t/at-least-once-delivery-setup-with-logstash-kafka/167998 "2019-02-12T09:54:18Z")
Hi, Recently, we had an issue where all our servers, including kafka + logstash where restarted at the same time. I was hoping that after restart we would get all messages but unfortunately some messages where lost. Aft… --- ## [Using Xpath Dynamically](https://discuss.elastic.co/t/using-xpath-dynamically/166903)
**Author:** [@erdo](https://discuss.elastic.co/u/erdo)\ **Replies:** 7\ **Last updated:** [February 8, 2019, 6:44am UTC](https://discuss.elastic.co/t/using-xpath-dynamically/166903 "2019-02-08T06:44:04Z")
I want to use xpath in logstash with dynamic way My log file: \ \ **Author:** [@RamyaGowda](https://discuss.elastic.co/u/RamyaGowda)\ **Replies:** 6\ **Last updated:** [February 12, 2019, 7:33am UTC](https://discuss.elastic.co/t/metricbeat-data-is-not-read-by-logstash/167798 "2019-02-12T07:33:15Z") Hi Team, I have configured Metricbeat to capture system level metrics, Metricbeat is intsalled as a service in Redhat linux , i installed metricbeat using rpm package. Bellow is my metricbeat configuration . \> #=====… --- ## [Date Field in in Logstash](https://discuss.elastic.co/t/date-field-in-in-logstash/167963)
**Author:** [@Alok](https://discuss.elastic.co/u/Alok)\ **Replies:** 2\ **Last updated:** [February 12, 2019, 6:54am UTC](https://discuss.elastic.co/t/date-field-in-in-logstash/167963 "2019-02-12T06:54:24Z")
I have some date fields in CSV which I would like to have as Type Date in the Index. When the index is created the type remains 'text' despite many ways of handling it. The config file looks like as under : input { … --- ## [Patterns issue](https://discuss.elastic.co/t/patterns-issue/167964)
**Author:** [@aabababba](https://discuss.elastic.co/u/aabababba)\ **Replies:** 0\ **Last updated:** [February 12, 2019, 6:31am UTC](https://discuss.elastic.co/t/patterns-issue/167964 "2019-02-12T06:31:02Z")
\[2019-01-28 00:00:10.925\]-INFO-\[biz:unknow539233145125633\]\[sys:unknow539233165125632\]\[?-?\]\[\]\[ID：32 time：10841ms\] \[2019-01-28 00:30:12.173\]-INFO-\[biz:\]\[sys:\]\[?-?\]\[\]\[DynamicServerListLoadBalancer for client dataservice-o… --- ## [Log stash filter for multi line occurrences and capture in custom field](https://discuss.elastic.co/t/log-stash-filter-for-multi-line-occurrences-and-capture-in-custom-field/167633)
**Author:** [@vasudevan](https://discuss.elastic.co/u/vasudevan)\ **Replies:** 4\ **Last updated:** [February 12, 2019, 5:35am UTC](https://discuss.elastic.co/t/log-stash-filter-for-multi-line-occurrences-and-capture-in-custom-field/167633 "2019-02-12T05:35:49Z")
We are having a huge log file in which we are applying many filter and storing it as a specific filed in that below line has, more than one occurance(3 to 5 times), In that we wanted to capture last occurance under filed… --- ## [KV Filter not extracting data](https://discuss.elastic.co/t/kv-filter-not-extracting-data/167602)
**Author:** [@vivek14](https://discuss.elastic.co/u/vivek14)\ **Replies:** 4\ **Last updated:** [February 12, 2019, 1:14am UTC](https://discuss.elastic.co/t/kv-filter-not-extracting-data/167602 "2019-02-12T01:14:03Z")
After using grok and mutate filters, I have a field called headers containing following value accept-encoding=\[gzip,deflate\], breadcrumbId=\[ID-fuse1-51739-1547625048671-0-6180\], Host=\[localhost:9000\], JMSCorrelationID=\[… --- ## [Logstash-filter-xml - Fixed empty event set](https://discuss.elastic.co/t/logstash-filter-xml-fixed-empty-event-set/167947)
**Author:** [@Inammathe\_Inna](https://discuss.elastic.co/u/Inammathe_Inna)\ **Replies:** 0\ **Last updated:** [February 12, 2019, 1:02am UTC](https://discuss.elastic.co/t/logstash-filter-xml-fixed-empty-event-set/167947 "2019-02-12T01:02:54Z")
This fixes a bug introduced in https://github.com/logstash-plugins/logstash-filter-xml/commit/1620c37c3f2adbaa482bccb9c3f6ccdb21b57dde#diff-46633c67f32820575bd30a6a35b252c7R160 I opened the PR a while ago however, I d… --- ## [Handling multiple tags and types in LS configuration file](https://discuss.elastic.co/t/handling-multiple-tags-and-types-in-ls-configuration-file/167904)
**Author:** [@drivera](https://discuss.elastic.co/u/drivera)\ **Replies:** 6\ **Last updated:** [February 11, 2019, 11:45pm UTC](https://discuss.elastic.co/t/handling-multiple-tags-and-types-in-ls-configuration-file/167904 "2019-02-11T23:45:10Z")
Hello, I would like to use my existing working LS config file to handle other log sources but I'm not sure of the syntax to handle multiple tags and types. I tried to the second config file below but it did not work. An… --- ## [Maximising Logstash CPU Utilisation](https://discuss.elastic.co/t/maximising-logstash-cpu-utilisation/167620)
**Author:** [@Aditya\_Kumar](https://discuss.elastic.co/u/Aditya_Kumar)\ **Replies:** 1\ **Last updated:** [February 11, 2019, 11:15pm UTC](https://discuss.elastic.co/t/maximising-logstash-cpu-utilisation/167620 "2019-02-11T23:15:22Z")
Hi I have installed logstash 6.6.0 on a dedicated Debian 9 machine. This machine has 2 x Quad Core cpus. My workflow is I start with blank Elasticsearch indexs (on another machine) and I process logfiles with logstash… --- ## [Logstash cannot Index to Elasticsearch for http\_poller](https://discuss.elastic.co/t/logstash-cannot-index-to-elasticsearch-for-http-poller/167931)
**Author:** [@imortalsolitude](https://discuss.elastic.co/u/imortalsolitude)\ **Replies:** 3\ **Last updated:** [February 11, 2019, 10:39pm UTC](https://discuss.elastic.co/t/logstash-cannot-index-to-elasticsearch-for-http-poller/167931 "2019-02-11T22:39:52Z")
I am running ELK 6.5.4 and trying to use the http\_poller to poll data into ELK but my logstash is unable to index to Elasticsearch and throws error message \]logstash.outputs.elasticsearch\] Could not index event to Elasti… --- ## [Logstash consumers not splitting kafka partitions to consume from](https://discuss.elastic.co/t/logstash-consumers-not-splitting-kafka-partitions-to-consume-from/167467)
**Author:** [@oats\_gal](https://discuss.elastic.co/u/oats_gal)\ **Replies:** 2\ **Last updated:** [February 11, 2019, 9:26pm UTC](https://discuss.elastic.co/t/logstash-consumers-not-splitting-kafka-partitions-to-consume-from/167467 "2019-02-11T21:26:38Z")
The Elastic stack that I support was recently upgraded from using logstash 2.4 to 6.4. I've been troubleshooting a decrease in efficiency and found an interesting difference in how the versions are handling reading off o… --- ## [Not able to load data in elasticsearch using logstash](https://discuss.elastic.co/t/not-able-to-load-data-in-elasticsearch-using-logstash/167884)
**Author:** [@pathri](https://discuss.elastic.co/u/pathri)\ **Replies:** 2\ **Last updated:** [February 11, 2019, 4:48pm UTC](https://discuss.elastic.co/t/not-able-to-load-data-in-elasticsearch-using-logstash/167884 "2019-02-11T16:48:45Z")
Hi experts, i am really new to elk and looking for the starting help. i am trying importing data from csv to elasticsearch using logstash this is my csv below is my logstash-config file below is the output displa… --- ## [Possible to Parse Nested XML Data structures and combine parent with child](https://discuss.elastic.co/t/possible-to-parse-nested-xml-data-structures-and-combine-parent-with-child/167854)
**Author:** [@adiefatlady](https://discuss.elastic.co/u/adiefatlady)\ **Replies:** 6\ **Last updated:** [February 11, 2019, 4:25pm UTC](https://discuss.elastic.co/t/possible-to-parse-nested-xml-data-structures-and-combine-parent-with-child/167854 "2019-02-11T16:25:44Z")
I have a nested xml document with multiple parents and children and i want to break this down to effectively be a linear list like... \ \Child 1\ .... \ so i … --- ## [Logstash cannot connect to Azure MS Sql database](https://discuss.elastic.co/t/logstash-cannot-connect-to-azure-ms-sql-database/167831)
**Author:** [@soumrock123](https://discuss.elastic.co/u/soumrock123)\ **Replies:** 1\ **Last updated:** [February 11, 2019, 4:12pm UTC](https://discuss.elastic.co/t/logstash-cannot-connect-to-azure-ms-sql-database/167831 "2019-02-11T16:12:44Z")
I am trying to extract info from all databases using Logstash and inserting them into Elasticsearch. While it works for all databases with specified localhost addresses, I am having trouble connecting to databases in AZU… --- ## [Logstash did not parse weblogic log from local ELK stack server](https://discuss.elastic.co/t/logstash-did-not-parse-weblogic-log-from-local-elk-stack-server/167880)
**Author:** [@anguyen98](https://discuss.elastic.co/u/anguyen98)\ **Replies:** 0\ **Last updated:** [February 11, 2019, 3:08pm UTC](https://discuss.elastic.co/t/logstash-did-not-parse-weblogic-log-from-local-elk-stack-server/167880 "2019-02-11T15:08:31Z")
Please help me out I have configure full ELK Stack to parse our Weblogic log but somehow it did not pickup the log and parse it. Everything is working but when I ran. curl -XGET 'http://testweblogic.net:9200/filebeat-\*/… --- ## [Parsing input in to logstash](https://discuss.elastic.co/t/parsing-input-in-to-logstash/167539)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 9\ **Last updated:** [February 11, 2019, 2:27pm UTC](https://discuss.elastic.co/t/parsing-input-in-to-logstash/167539 "2019-02-11T14:27:39Z")
I configure snmp to get information from PDU Run only two get to test, and output is long. { "iso.org.dod.internet.private.enterprises.apc.products.hardware.rPDU2.rPDU2Device.rPDU2DeviceConfigTable.rPDU2DeviceConfigEn… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=601) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=603)