# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1043 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1044 --- ## [How to use Logstash monitoring API?](https://discuss.elastic.co/t/how-to-use-logstash-monitoring-api/73658)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 10\ **Last updated:** [February 3, 2017, 7:39am UTC](https://discuss.elastic.co/t/how-to-use-logstash-monitoring-api/73658 "2017-02-03T07:39:05Z")
Hi, I have multiple Logstash instances and it will be fine if I can see their status in a Kibana dashboard. I got to know about Logstash monitoring API here https://www.elastic.co/guide/en/logstash/current/monitoring.h… --- ## [Converting time Zone](https://discuss.elastic.co/t/converting-time-zone/73795)
**Author:** [@Dan](https://discuss.elastic.co/u/Dan)\ **Replies:** 1\ **Last updated:** [February 3, 2017, 6:34am UTC](https://discuss.elastic.co/t/converting-time-zone/73795 "2017-02-03T06:34:24Z")
I am using JDBC input and Elastic Search as output. Writing query to get data from JDBC.Where i have one date column with time. in my DB column value as 24/07/2016 09:00.30.000 when i am posting this value into elast… --- ## [Using filters on nested fields](https://discuss.elastic.co/t/using-filters-on-nested-fields/73562)
**Author:** [@shambar](https://discuss.elastic.co/u/shambar)\ **Replies:** 3\ **Last updated:** [February 3, 2017, 6:32am UTC](https://discuss.elastic.co/t/using-filters-on-nested-fields/73562 "2017-02-03T06:32:12Z")
I am having an issue trying to get logstash to see the items in the array beyond the first item. the data will come in like. {"id": "430229930", "item\_tbl" :\[{"itemid": 1,"itemname":"eggs","quantity": 10},{"itemid"… --- ## [Sql server db error log data export to logstash](https://discuss.elastic.co/t/sql-server-db-error-log-data-export-to-logstash/73340)
**Author:** [@prosenjit](https://discuss.elastic.co/u/prosenjit)\ **Replies:** 2\ **Last updated:** [February 3, 2017, 6:08am UTC](https://discuss.elastic.co/t/sql-server-db-error-log-data-export-to-logstash/73340 "2017-02-03T06:08:53Z")
Hi, I have a db table in sql server which logs the errors and maintains in a table. I want to be able to send it to elastic search after parsing it through logstash. Now the problem is the server points to a different po… --- ## [Logstash taking too long to process data](https://discuss.elastic.co/t/logstash-taking-too-long-to-process-data/72096)
**Author:** [@jstar](https://discuss.elastic.co/u/jstar)\ **Replies:** 21\ **Last updated:** [February 2, 2017, 10:09pm UTC](https://discuss.elastic.co/t/logstash-taking-too-long-to-process-data/72096 "2017-02-02T22:09:41Z")
Hi, I have setup the following architecture filebeat -\> kafka -\> logstash -\> elasticsearch. Filebeat is sending data to kafka without any issue. I can validate that by using the command line of kafka. Logstash is consu… --- ## [Logstash filter not working for @timestamp](https://discuss.elastic.co/t/logstash-filter-not-working-for-timestamp/73728)
**Author:** [@justx](https://discuss.elastic.co/u/justx)\ **Replies:** 1\ **Last updated:** [February 2, 2017, 9:16pm UTC](https://discuss.elastic.co/t/logstash-filter-not-working-for-timestamp/73728 "2017-02-02T21:16:02Z")
I am having a devil of a time figuring out how to configure my Logstash filter options to correctly parse the timestamp from the event into the @timestamp field. Im sure im doing something wrong that is simple, I just c… --- ## [How to quikly parsing 1 million xml](https://discuss.elastic.co/t/how-to-quikly-parsing-1-million-xml/73141)
**Author:** [@111126](https://discuss.elastic.co/u/111126)\ **Replies:** 19\ **Last updated:** [February 2, 2017, 8:56pm UTC](https://discuss.elastic.co/t/how-to-quikly-parsing-1-million-xml/73141 "2017-02-02T20:56:45Z")
Hello. I have 1 million file with xml. Each file size from a few kilobytes to 20mb. I need parsing their and put to elastic. I tried to use the next config: file { path =\> "/opt/lun1/data-unzip/ftp/228-pur-plan/.xm… --- ## [Convert a field from string to ip type](https://discuss.elastic.co/t/convert-a-field-from-string-to-ip-type/1865)
**Author:** [@Prachi](https://discuss.elastic.co/u/Prachi)\ **Replies:** 4\ **Last updated:** [February 2, 2017, 8:16pm UTC](https://discuss.elastic.co/t/convert-a-field-from-string-to-ip-type/1865 "2017-02-02T20:16:30Z")
Hello, I am trying to upload my text file on Elasticsearch using Logstash. Below is how my config file look like. When I upload my text files on elasticsearch, the src\_ip and dst\_ip fields are by default taken as strin… --- ## [Codec json is not working](https://discuss.elastic.co/t/codec-json-is-not-working/73637)
**Author:** [@jmcjmcjmc](https://discuss.elastic.co/u/jmcjmcjmc)\ **Replies:** 6\ **Last updated:** [February 2, 2017, 8:00pm UTC](https://discuss.elastic.co/t/codec-json-is-not-working/73637 "2017-02-02T20:00:57Z")
I am trying to read data from Kafka (fed by beats) and send to ES. Using codec =\> "json" does not work at all. input { kafka { bootstrap\_servers =\> "10.100.177.100:9999" topics … --- ## [Cannot update throttle filter plugin to 4.0.1 on logstash 2.3.4](https://discuss.elastic.co/t/cannot-update-throttle-filter-plugin-to-4-0-1-on-logstash-2-3-4/73741)
**Author:** [@tcouto](https://discuss.elastic.co/u/tcouto)\ **Replies:** 0\ **Last updated:** [February 2, 2017, 6:58pm UTC](https://discuss.elastic.co/t/cannot-update-throttle-filter-plugin-to-4-0-1-on-logstash-2-3-4/73741 "2017-02-02T18:58:24Z")
\[root@app101 ~\]# /opt/logstash/bin/logstash-plugin update logstash-filter-throttle You are updating logstash-filter-throttle to a new version 4.0.1, which may not be compatible with 2.0.4. are you sure you want to proce… --- ## [Persisted queue never clearing](https://discuss.elastic.co/t/persisted-queue-never-clearing/73732)
**Author:** [@dachenbach](https://discuss.elastic.co/u/dachenbach)\ **Replies:** 0\ **Last updated:** [February 2, 2017, 5:58pm UTC](https://discuss.elastic.co/t/persisted-queue-never-clearing/73732 "2017-02-02T17:58:26Z")
I'm having an issue with persisted queues in logstash. After about a day of running, the persisted queue fills up to the max bytes and old page files are not being removed. I have inputs set up for beats and syslog and a… --- ## [How can I add an array object to tags using add\_Tag](https://discuss.elastic.co/t/how-can-i-add-an-array-object-to-tags-using-add-tag/73676)
**Author:** [@trondhindenes](https://discuss.elastic.co/u/trondhindenes)\ **Replies:** 3\ **Last updated:** [February 2, 2017, 5:28pm UTC](https://discuss.elastic.co/t/how-can-i-add-an-array-object-to-tags-using-add-tag/73676 "2017-02-02T17:28:02Z")
I have the following logstash snippet: elseif \[Properties\]\[Telemetry\] { mutate { update =\> { "logtype" =\> "telemetry" } add\_field =\> { "key" =\> "%{\[Properties\]\[Telemetry\]\[Key\]}" "value" =\> "%{\[Properties… --- ## [Logstash failed to put logs to java ee application](https://discuss.elastic.co/t/logstash-failed-to-put-logs-to-java-ee-application/73692)
**Author:** [@Maher.Glenza](https://discuss.elastic.co/u/Maher.Glenza)\ **Replies:** 4\ **Last updated:** [February 2, 2017, 2:09pm UTC](https://discuss.elastic.co/t/logstash-failed-to-put-logs-to-java-ee-application/73692 "2017-02-02T14:09:01Z")
hi plz i m trying to filter logs with logstash then to put them in a java ee application that i m building with http requests output { http { url =\> "http://localhost:8080/LogRest/router/router" http… --- ## [5.2 Offline Plugin Install](https://discuss.elastic.co/t/5-2-offline-plugin-install/73665)
**Author:** [@NirKa](https://discuss.elastic.co/u/NirKa)\ **Replies:** 1\ **Last updated:** [February 2, 2017, 1:02pm UTC](https://discuss.elastic.co/t/5-2-offline-plugin-install/73665 "2017-02-02T13:02:55Z")
Hi, I'm trying to use the new offline plugin install feature and it doesn't seem to work... looks like I'm missing something. I created a custom plugin that pushes events to google pub sub. Steps: Installed the pl… --- ## [How to update denormalized data in Elasticsearch?](https://discuss.elastic.co/t/how-to-update-denormalized-data-in-elasticsearch/73662)
**Author:** [@lmazgon](https://discuss.elastic.co/u/lmazgon)\ **Replies:** 0\ **Last updated:** [February 2, 2017, 11:31am UTC](https://discuss.elastic.co/t/how-to-update-denormalized-data-in-elasticsearch/73662 "2017-02-02T11:31:01Z")
Hi logstash community! Actually I have asked this question on StackOverflow, but noone seems to know the answer. Maybe you can help me. Use case explanation We have a relational database with data about our day-to-d… --- ## [Logstash: Error jdbc\_driver\_library not loaded](https://discuss.elastic.co/t/logstash-error-jdbc-driver-library-not-loaded/73654)
**Author:** [@zegdene](https://discuss.elastic.co/u/zegdene)\ **Replies:** 3\ **Last updated:** [February 2, 2017, 11:29am UTC](https://discuss.elastic.co/t/logstash-error-jdbc-driver-library-not-loaded/73654 "2017-02-02T11:29:40Z")
Hi, I need to get data from a PostgreSQL DB and index it into Elasticsearch. When I run /opt/logstash-2.3.3/bin/logstash -f contacts-index-logstash.conf I receive the following error: Could not find log4j2 configurati… --- ## [Ending aggregation with multiple conditions](https://discuss.elastic.co/t/ending-aggregation-with-multiple-conditions/73650)
**Author:** [@egoldemberg](https://discuss.elastic.co/u/egoldemberg)\ **Replies:** 0\ **Last updated:** [February 2, 2017, 10:22am UTC](https://discuss.elastic.co/t/ending-aggregation-with-multiple-conditions/73650 "2017-02-02T10:22:47Z")
Hi, I'm trying to make an aggregation that will end when several phrases are seen, but I can't tell in advance their incoming order, and I want to wait for all of them to arrive (until a minimal timeout.) However, I don'… --- ## [Add "type" to my custom input plugin](https://discuss.elastic.co/t/add-type-to-my-custom-input-plugin/73358)
**Author:** [@itzikHan](https://discuss.elastic.co/u/itzikHan)\ **Replies:** 3\ **Last updated:** [February 2, 2017, 10:16am UTC](https://discuss.elastic.co/t/add-type-to-my-custom-input-plugin/73358 "2017-02-02T10:16:58Z")
Hi, I'm trying to add "type" field that i would be able to use as tag when filtering to my input plugin. For example, I can add "type" in the 'file' input plugin, and filter it later. Code: input { file { path =\> … --- ## [Email Plugin in logstash](https://discuss.elastic.co/t/email-plugin-in-logstash/73355)
**Author:** [@bini](https://discuss.elastic.co/u/bini)\ **Replies:** 2\ **Last updated:** [February 2, 2017, 8:34am UTC](https://discuss.elastic.co/t/email-plugin-in-logstash/73355 "2017-02-02T08:34:50Z")
Hi, I want to trigger an email, if \[fs.device\_name\] == "C:\\" and \[fs.used\] \>= "25,000,000,000". How can this be done? Any help would be appreciated. :slight\_smile: --- ## [Logstash keep crushing to an uknown reason](https://discuss.elastic.co/t/logstash-keep-crushing-to-an-uknown-reason/73619)
**Author:** [@sancroth](https://discuss.elastic.co/u/sancroth)\ **Replies:** 3\ **Last updated:** [February 2, 2017, 7:30am UTC](https://discuss.elastic.co/t/logstash-keep-crushing-to-an-uknown-reason/73619 "2017-02-02T07:30:15Z")
My logstash completely failed and stoped working.I dont think i messed anything up since the configs are back to the same place , service logstash configtest gives a pass. The error coming out is this : {:timestamp=\>"… --- ## [How to move log-files to another location after it is processed by Logstash and sent to ElasticSearch](https://discuss.elastic.co/t/how-to-move-log-files-to-another-location-after-it-is-processed-by-logstash-and-sent-to-elasticsearch/73459)
**Author:** [@anisen](https://discuss.elastic.co/u/anisen)\ **Replies:** 3\ **Last updated:** [February 2, 2017, 6:52am UTC](https://discuss.elastic.co/t/how-to-move-log-files-to-another-location-after-it-is-processed-by-logstash-and-sent-to-elasticsearch/73459 "2017-02-02T06:52:47Z")
Hi, I am processing multiple log-files from a folder using logstash and sending the data to ES. Now, I want to move a file to other location once the processing is done. How to do this? --- ## [Unable to parse JSON file and send it to elasticsearch](https://discuss.elastic.co/t/unable-to-parse-json-file-and-send-it-to-elasticsearch/73634)
**Author:** [@Sharath\_Vutpala](https://discuss.elastic.co/u/Sharath_Vutpala)\ **Replies:** 1\ **Last updated:** [February 2, 2017, 6:44am UTC](https://discuss.elastic.co/t/unable-to-parse-json-file-and-send-it-to-elasticsearch/73634 "2017-02-02T06:44:16Z")
Hi, I am trying to parse a JSON log file generated by an application and send it to elasticsearch and from there to kibana. I have tried so many methods, but there is no luck. Below is the logstash config file i am usi… --- ## [Certificate issues lumberjack as output plugin](https://discuss.elastic.co/t/certificate-issues-lumberjack-as-output-plugin/72836)
**Author:** [@abhinigam](https://discuss.elastic.co/u/abhinigam)\ **Replies:** 7\ **Last updated:** [February 2, 2017, 6:31am UTC](https://discuss.elastic.co/t/certificate-issues-lumberjack-as-output-plugin/72836 "2017-02-02T06:31:58Z")
I am using a self signed client certificate and configuring the same in my lumberjack output plugin. However, when my logstash forwarder on the client machines tries to talk to the logstash aggregator I am getting the fo… --- ## [To\_char converted time error](https://discuss.elastic.co/t/to-char-converted-time-error/73635)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 0\ **Last updated:** [February 2, 2017, 6:17am UTC](https://discuss.elastic.co/t/to-char-converted-time-error/73635 "2017-02-02T06:17:40Z")
Hi, I am pulling rows from oracle db using the query; SELECT request,response,reqtime,TO\_CHAR(reqtime, 'DD-MM-YYYY hh:mi:ss.FF3 AM') as req\_time,restime FROM logtable The JSON I have found in kibana for the correspo… --- ## [Multiline - match whole file](https://discuss.elastic.co/t/multiline-match-whole-file/73630)
**Author:** [@kazan](https://discuss.elastic.co/u/kazan)\ **Replies:** 0\ **Last updated:** [February 2, 2017, 5:42am UTC](https://discuss.elastic.co/t/multiline-match-whole-file/73630 "2017-02-02T05:42:23Z")
Hi there, I'm having a problem with matching the whole file. So basically I want to have content of the whole in my "message" field. So I'm trying to use multiline in filter section. What pattern should I set? I tried… --- ## [Logstash id fetching different date / time stamp from DB](https://discuss.elastic.co/t/logstash-id-fetching-different-date-time-stamp-from-db/73513)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 1\ **Last updated:** [February 2, 2017, 5:03am UTC](https://discuss.elastic.co/t/logstash-id-fetching-different-date-time-stamp-from-db/73513 "2017-02-02T05:03:37Z")
Hi, I am fetching data from postgres table which contains 2 date fields. The result JSON (fetched using Kibana) is like below; { "\_index": "mylogs-2017-02-01", "\_type": "logs", "\_id": "AVn5ff7dDzMvfhs004x9", … --- ## [Logstash use 100%+ cpu with simple beat input](https://discuss.elastic.co/t/logstash-use-100-cpu-with-simple-beat-input/73620)
**Author:** [@hckim](https://discuss.elastic.co/u/hckim)\ **Replies:** 0\ **Last updated:** [February 2, 2017, 3:59am UTC](https://discuss.elastic.co/t/logstash-use-100-cpu-with-simple-beat-input/73620 "2017-02-02T03:59:51Z")
Hi I am setting up filebeat -\> logstash -\> rabbitMQ -\>elasticsearch some how logstash simple config is using 100%+ cpu is this normal ? or am I missing some thing filebeat ( V1.3.1) logstash ( V2.4.1) cpu : AMD Opt… --- ## [Logstash to Elasticsearch Bulk Request , SSL peer shut down incorrectly- Manticore::ClientProtocolException logstash](https://discuss.elastic.co/t/logstash-to-elasticsearch-bulk-request-ssl-peer-shut-down-incorrectly-manticore-clientprotocolexception-logstash/73558)
**Author:** [@Bhavana](https://discuss.elastic.co/u/Bhavana)\ **Replies:** 2\ **Last updated:** [February 2, 2017, 2:52am UTC](https://discuss.elastic.co/t/logstash-to-elasticsearch-bulk-request-ssl-peer-shut-down-incorrectly-manticore-clientprotocolexception-logstash/73558 "2017-02-02T02:52:36Z")
ES version - 2.3.5 , Logstash - 2.4 'Attempted to send bulk request to Elasticsearch, configured at \["xxxx.com:9200"\] , An error occurred and it failed! Are you sure you can reach elasticsearch from this machine using … --- ## [Non Delimited String search](https://discuss.elastic.co/t/non-delimited-string-search/73270)
**Author:** [@vinathan](https://discuss.elastic.co/u/vinathan)\ **Replies:** 1\ **Last updated:** [February 1, 2017, 8:42pm UTC](https://discuss.elastic.co/t/non-delimited-string-search/73270 "2017-02-01T20:42:20Z")
I have an incoming field designated as source with the following "/mnt/rsyslog\_custom/nagios-prod-na2-san3/nagios\_na2\_6065". I want to use logstash to remove all but "nagios-prod-na2-san3". Is there a way to accomplish t… --- ## [Nest a json object from csv](https://discuss.elastic.co/t/nest-a-json-object-from-csv/73361)
**Author:** [@Dan\_Harper](https://discuss.elastic.co/u/Dan_Harper)\ **Replies:** 1\ **Last updated:** [February 1, 2017, 8:41pm UTC](https://discuss.elastic.co/t/nest-a-json-object-from-csv/73361 "2017-02-01T20:41:05Z")
I have a csv which has a field called cart in it and although I can import correctly the cart field creates new fields in relation to the nested products inside the cart rather just a single cart field. 431| 2017-01-2… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1042) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1044)