# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=645 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 646 --- ## [Input from file output to file](https://discuss.elastic.co/t/input-from-file-output-to-file/157664)
**Author:** [@Viti](https://discuss.elastic.co/u/Viti)\ **Replies:** 7\ **Last updated:** [November 21, 2018, 12:39pm UTC](https://discuss.elastic.co/t/input-from-file-output-to-file/157664 "2018-11-21T12:39:32Z")
Hi, I want to take a file log and process the data to other output file. I thinck than it's a easy proccedure, but my code don´t work :frowning: it's my first time with logstash. Sorry if the ask is very obvius. My fi… --- ## [Hosts as name instead of IP?](https://discuss.elastic.co/t/hosts-as-name-instead-of-ip/157536)
**Author:** [@BigAlbert](https://discuss.elastic.co/u/BigAlbert)\ **Replies:** 9\ **Last updated:** [November 21, 2018, 11:33am UTC](https://discuss.elastic.co/t/hosts-as-name-instead-of-ip/157536 "2018-11-21T11:33:46Z")
in my conf file, can I use the Elasticsearch machine "name" instead of IP? tried using hosts =\> \["Elasticsearch:9200"\] yet failed. notice that I CAN ping Elasticsearch .. THX -Albert. --- ## [Loading class \`com.mysql.jdbc.Driver'. This is deprecated. The new driver class is \`com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary](https://discuss.elastic.co/t/loading-class-com-mysql-jdbc-driver-this-is-deprecated-the-new-driver-class-is-com-mysql-cj-jdbc-driver-the-driver-is-automatically-registered-via-the-spi-and-manual-loading-of-the-driver-class-is-generally-unnecessary/157687)
**Author:** [@muyuqj](https://discuss.elastic.co/u/muyuqj)\ **Replies:** 2\ **Last updated:** [November 21, 2018, 11:18am UTC](https://discuss.elastic.co/t/loading-class-com-mysql-jdbc-driver-this-is-deprecated-the-new-driver-class-is-com-mysql-cj-jdbc-driver-the-driver-is-automatically-registered-via-the-spi-and-manual-loading-of-the-driver-class-is-generally-unnecessary/157687 "2018-11-21T11:18:29Z")
Loading class com.mysql.jdbc.Driver'. This is deprecated. The new driver class iscom.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessa… --- ## [Problem with logstash jdbc input plugin. Date field data is incorrect](https://discuss.elastic.co/t/problem-with-logstash-jdbc-input-plugin-date-field-data-is-incorrect/157682)
**Author:** [@elasticheart](https://discuss.elastic.co/u/elasticheart)\ **Replies:** 4\ **Last updated:** [November 21, 2018, 11:08am UTC](https://discuss.elastic.co/t/problem-with-logstash-jdbc-input-plugin-date-field-data-is-incorrect/157682 "2018-11-21T11:08:41Z")
Hi, I am using ELK GA 6.3.0. I have the logstash configuration like below; input { jdbc { jdbc\_driver\_library =\> "ojdbc6.jar" jdbc\_driver\_class =\> "Java::oracle.jdbc.driver.OracleDriver" jdbc\_connection\_s… --- ## [Create dictionary file using logstash file output](https://discuss.elastic.co/t/create-dictionary-file-using-logstash-file-output/157661)
**Author:** [@serhii.zaidlin](https://discuss.elastic.co/u/serhii.zaidlin)\ **Replies:** 1\ **Last updated:** [November 21, 2018, 10:31am UTC](https://discuss.elastic.co/t/create-dictionary-file-using-logstash-file-output/157661 "2018-11-21T10:31:17Z")
Hi everyone. I have a scheduled input which pulls API and recieves batch of device details and refreshes device info in Elastcisearch index (so if I have 100 devices, i will always have 100 docs in elastic index). This … --- ## [How can I set current time in field](https://discuss.elastic.co/t/how-can-i-set-current-time-in-field/157542)
**Author:** [@2501492a6edd02b1386b](https://discuss.elastic.co/u/2501492a6edd02b1386b)\ **Replies:** 3\ **Last updated:** [November 21, 2018, 10:19am UTC](https://discuss.elastic.co/t/how-can-i-set-current-time-in-field/157542 "2018-11-21T10:19:44Z")
Hi, Could you please share with me info how I can to set current system time in field "@timestamp". In my case, I successfully copy field with @timestamp in filed "real\_timestamp". And then I would like to set @times… --- ## [Removing annoying characters before a Json formated string](https://discuss.elastic.co/t/removing-annoying-characters-before-a-json-formated-string/157522)
**Author:** [@Josef](https://discuss.elastic.co/u/Josef)\ **Replies:** 3\ **Last updated:** [November 21, 2018, 9:24am UTC](https://discuss.elastic.co/t/removing-annoying-characters-before-a-json-formated-string/157522 "2018-11-21T09:24:57Z")
Hello, I'm just a beginner in ELK, so please forgive me in advance. Here is the kind of input I have in logstash : E...(.@.@..b......................=...B.{"rxpk":\[{"tmst":297436019,"time":"2018-11-20T10:54:01.237397Z… --- ## [Bulk Data upload in logstash](https://discuss.elastic.co/t/bulk-data-upload-in-logstash/156804)
**Author:** [@Gauti](https://discuss.elastic.co/u/Gauti)\ **Replies:** 6\ **Last updated:** [November 21, 2018, 7:00am UTC](https://discuss.elastic.co/t/bulk-data-upload-in-logstash/156804 "2018-11-21T07:00:58Z")
Hi All, We have been trying to index data using logstash from servicenow through restapi's, we dont see any errors and data is getting indexed only issue we are facing is we do have around 2lakh records to be indexed bu… --- ## [XML Parser failing -Logstash --- xsi:nil="true"](https://discuss.elastic.co/t/xml-parser-failing-logstash-xsi-nil-true/157600)
**Author:** [@bonsshah](https://discuss.elastic.co/u/bonsshah)\ **Replies:** 1\ **Last updated:** [November 21, 2018, 7:18am UTC](https://discuss.elastic.co/t/xml-parser-failing-logstash-xsi-nil-true/157600 "2018-11-21T07:18:31Z")
I am trying to parse complete xml without xpath as there are many attributes so can not use xpath. Logstash XML parser is failing at below line of xml. ERROR : Could not index event to Elasticsearch. {:status=\>400, :ac… --- ## [Convert a Json array of object (\[{},{},{}\] - this format) to {\[\],\[\],\[\]}](https://discuss.elastic.co/t/convert-a-json-array-of-object-this-format-to/157058)
**Author:** [@Panda8](https://discuss.elastic.co/u/Panda8)\ **Replies:** 1\ **Last updated:** [November 21, 2018, 7:13am UTC](https://discuss.elastic.co/t/convert-a-json-array-of-object-this-format-to/157058 "2018-11-21T07:13:48Z")
Hi I have a Json I/P which looks like: { "data": { "arrayField" :\[ {"id" : 7,"data" : "a"},{"id" : 3,"data" : "b"}\], "field1" : "value1" } } Where arrayField can contain any number of element. I want to acces… --- ## [CSV data upload problem](https://discuss.elastic.co/t/csv-data-upload-problem/156956)
**Author:** [@f24b83985b804d90965e](https://discuss.elastic.co/u/f24b83985b804d90965e)\ **Replies:** 8\ **Last updated:** [November 21, 2018, 5:07am UTC](https://discuss.elastic.co/t/csv-data-upload-problem/156956 "2018-11-21T05:07:51Z")
hi there. I Have 2 issue about Logstash. but first, I let you know that test base. Windows 10 Search, kibana, Logstash v6.2.2 CSV data upload it is work in v6.2.2 family. but 6.4 or 6.5.0 doesn't work in same meth… --- ## [Ruby filter causing logstash to continually restart](https://discuss.elastic.co/t/ruby-filter-causing-logstash-to-continually-restart/157612)
**Author:** [@agx](https://discuss.elastic.co/u/agx)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 9:18pm UTC](https://discuss.elastic.co/t/ruby-filter-causing-logstash-to-continually-restart/157612 "2018-11-20T21:18:07Z")
Hello, I have the following ruby filter in my logstash config: ruby { code =\> "unless event.get('delivered') == 0 event.set('openrate', event.get('opens')/event.get('delivered')) end" } when I uncomment this filter, I … --- ## [Missing/Skipped Data with Logstash JDBC Input Plugin](https://discuss.elastic.co/t/missing-skipped-data-with-logstash-jdbc-input-plugin/157605)
**Author:** [@tgeraghty](https://discuss.elastic.co/u/tgeraghty)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 7:11pm UTC](https://discuss.elastic.co/t/missing-skipped-data-with-logstash-jdbc-input-plugin/157605 "2018-11-20T19:11:07Z")
We currently have Logstash running as a service and are running a query using the “schedule” property in the Logstash configuration to retrieve data using a JDBC plugin. This query pulls previous (System Time – 15 minute… --- ## [Still problems with grok parsing](https://discuss.elastic.co/t/still-problems-with-grok-parsing/156703)
**Author:** [@heskez](https://discuss.elastic.co/u/heskez)\ **Replies:** 3\ **Last updated:** [November 20, 2018, 6:36pm UTC](https://discuss.elastic.co/t/still-problems-with-grok-parsing/156703 "2018-11-20T18:36:30Z")
Hi there, I've still problems parsing with grok. This is the raw data coming in: 1: date=2018-10-17 time=03:19:48 logid=0000000011 type=traffic subtype=forward level=warning vd=root srcip=10.10.1.150 srcport=52120 srci… --- ## [Logstash JDBC plugin index rotation](https://discuss.elastic.co/t/logstash-jdbc-plugin-index-rotation/157599)
**Author:** [@chandra0651](https://discuss.elastic.co/u/chandra0651)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 5:48pm UTC](https://discuss.elastic.co/t/logstash-jdbc-plugin-index-rotation/157599 "2018-11-20T17:48:54Z")
Hello, I am using the logstash JDBC plugin to import the data in oracle table to elasticsearch index and using the modified time stamp as sql last run and using the primary key of oracle table as elasticsearch document … --- ## [Why am I getting such poor performance?](https://discuss.elastic.co/t/why-am-i-getting-such-poor-performance/155567)
**Author:** [@monk](https://discuss.elastic.co/u/monk)\ **Replies:** 13\ **Last updated:** [November 20, 2018, 4:39pm UTC](https://discuss.elastic.co/t/why-am-i-getting-such-poor-performance/155567 "2018-11-20T16:39:30Z")
I've got a large (23M record) CSV file I want to upload once to ES. I've made a few small tweaks to my LS config (10 workers, batch size 5000, Xms and Xmx 5 Gb), but it still uploads only about 1000 records/s, compared t… --- ## [How can I identify back pressure in logstash](https://discuss.elastic.co/t/how-can-i-identify-back-pressure-in-logstash/157581)
**Author:** [@meetdave2611997](https://discuss.elastic.co/u/meetdave2611997)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 4:04pm UTC](https://discuss.elastic.co/t/how-can-i-identify-back-pressure-in-logstash/157581 "2018-11-20T16:04:40Z")
Hi, we are trying to create some alerting in elasticsearch side which would notify us when logstash puts back pressure on inputs and stops accepting data. we are currently using three types of inputs: beats, tcp and htt… --- ## [Not all Java classes can be parsed using GROK pattern %{JAVACLASS}](https://discuss.elastic.co/t/not-all-java-classes-can-be-parsed-using-grok-pattern-javaclass/157545)
**Author:** [@ovanekem](https://discuss.elastic.co/u/ovanekem)\ **Replies:** 1\ **Last updated:** [November 20, 2018, 1:34pm UTC](https://discuss.elastic.co/t/not-all-java-classes-can-be-parsed-using-grok-pattern-javaclass/157545 "2018-11-20T13:34:14Z")
I'm writing a Logstash filter (using Logstash 6.4.0) that will filter lots of different applications (some are Spring Boot, some are Wildfly, some uses generated classes from wsdl2java,...). An issue I face is that the … --- ## [Follow up on Logstash 6.3.2 - File input plugin 4.1.5 stops writing to .sincedb after some time](https://discuss.elastic.co/t/follow-up-on-logstash-6-3-2-file-input-plugin-4-1-5-stops-writing-to-sincedb-after-some-time/157553)
**Author:** [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 1:08pm UTC](https://discuss.elastic.co/t/follow-up-on-logstash-6-3-2-file-input-plugin-4-1-5-stops-writing-to-sincedb-after-some-time/157553 "2018-11-20T13:08:49Z")
Continuing the discussion from Logstash 6.3.2 - File input plugin 4.1.5 stops writing to .sincedb after some time: Just want to follow up on this - looks like the cause was due to a file handle leak: https://www.elastic… --- ## [Logstash not working when started via systemctl](https://discuss.elastic.co/t/logstash-not-working-when-started-via-systemctl/157409)
**Author:** [@Marlon\_Ruttmann](https://discuss.elastic.co/u/Marlon_Ruttmann)\ **Replies:** 1\ **Last updated:** [November 20, 2018, 12:25pm UTC](https://discuss.elastic.co/t/logstash-not-working-when-started-via-systemctl/157409 "2018-11-20T12:25:32Z")
Hello, I have all my environment configured and tested using the command bin/logstash -f myfile.conf --config.reload.automatic and everything works fine. Elasticsearch generates indices, all the filters defined in my .c… --- ## [Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main](https://discuss.elastic.co/t/failed-to-execute-action-action-logstash-create-pipeline-id-main/157419)
**Author:** [@BigAlbert](https://discuss.elastic.co/u/BigAlbert)\ **Replies:** 3\ **Last updated:** [November 20, 2018, 12:01pm UTC](https://discuss.elastic.co/t/failed-to-execute-action-action-logstash-create-pipeline-id-main/157419 "2018-11-20T12:01:21Z")
Hi I'm using logstash-6.5.0, following is cat /usr/local/logstash-6.5.0/config/logstash.conf input { file { path =\> "/usr/src/myprod/mylog.startup-1.0.0.log" type =\> "logs" start\_position =\> "beginning" } } f… --- ## [Parsing Different Date Format From File Name](https://discuss.elastic.co/t/parsing-different-date-format-from-file-name/157518)
**Author:** [@goutamdas](https://discuss.elastic.co/u/goutamdas)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 10:49am UTC](https://discuss.elastic.co/t/parsing-different-date-format-from-file-name/157518 "2018-11-20T10:49:33Z")
Hi all, I have a requirement where there are no date getting captured in the logs, whereas time has been captured. But the log file name has date + time in it. And there are different types of file name. For ex. I have… --- ## [How to parse the json content using Json filter](https://discuss.elastic.co/t/how-to-parse-the-json-content-using-json-filter/157496)
**Author:** [@Sanj](https://discuss.elastic.co/u/Sanj)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 8:33am UTC](https://discuss.elastic.co/t/how-to-parse-the-json-content-using-json-filter/157496 "2018-11-20T08:33:42Z")
Hi Team, I'm trying to parse a file containing data in Json format. Initially i tried filebeat to parse the data, but came across a discussion in this forum that logstash would be the best place to parse the nested json… --- ## [Large Logstash Config File](https://discuss.elastic.co/t/large-logstash-config-file/157485)
**Author:** [@Priyaranjan\_Mudliar](https://discuss.elastic.co/u/Priyaranjan_Mudliar)\ **Replies:** 1\ **Last updated:** [November 20, 2018, 7:35am UTC](https://discuss.elastic.co/t/large-logstash-config-file/157485 "2018-11-20T07:35:17Z")
What is the file size limit of the logstash config file ? Actually I have many grok patterns to fill in it and it might big and cumbersome to handle. Please guide me how to overcome this. Will it slow down the pipeline p… --- ## [Arcsight connector not able to connect](https://discuss.elastic.co/t/arcsight-connector-not-able-to-connect/157478)
**Author:** [@jagan](https://discuss.elastic.co/u/jagan)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 6:38am UTC](https://discuss.elastic.co/t/arcsight-connector-not-able-to-connect/157478 "2018-11-20T06:38:47Z")
Hi mates, Could you please help me out how to get connected with arcsight connector with logstash. I got the error:java.lang.IllegalStateException: Logstash stopped processing because of an error: (NameError) cannot lo… --- ## [Grok filter not working with if condition](https://discuss.elastic.co/t/grok-filter-not-working-with-if-condition/157475)
**Author:** [@ashok9177](https://discuss.elastic.co/u/ashok9177)\ **Replies:** 0\ **Last updated:** [November 20, 2018, 6:21am UTC](https://discuss.elastic.co/t/grok-filter-not-working-with-if-condition/157475 "2018-11-20T06:21:53Z")
\> filter { \> if "id4444" in \[fields\]\[component\] { \> grok { \> match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:timestamp} (\\\[%{WORD:loglevel}\\\]) %{DATA} - %{DATA:method} processing time for transactionId : %{S3\_REQUEST\_… --- ## [How to install logstash plugins Leef and syslog plugin](https://discuss.elastic.co/t/how-to-install-logstash-plugins-leef-and-syslog-plugin/157448)
**Author:** [@van](https://discuss.elastic.co/u/van)\ **Replies:** 1\ **Last updated:** [November 19, 2018, 10:16pm UTC](https://discuss.elastic.co/t/how-to-install-logstash-plugins-leef-and-syslog-plugin/157448 "2018-11-19T22:16:26Z")
How to install logstash plugins Leef and syslog plugin Tried using the commands : export http\_proxy=http://proxy.kohls.com:3128 export https\_proxy=http://proxy.kohls.com:3128 bin/logstash-plugin install logstash-… --- ## [Multiple CSV files and Multiple Header data \[identifiers\]](https://discuss.elastic.co/t/multiple-csv-files-and-multiple-header-data-identifiers/157077)
**Author:** [@wheelq](https://discuss.elastic.co/u/wheelq)\ **Replies:** 3\ **Last updated:** [November 19, 2018, 10:04pm UTC](https://discuss.elastic.co/t/multiple-csv-files-and-multiple-header-data-identifiers/157077 "2018-11-19T22:04:47Z")
I have read @theuntergeek responses in the Parse 1st Line of Multiple CSV files and set as Columns thread But I didn't fully understand this comment: csv\_type\_1 identifies a stream (by file type, or whatever you ident… --- ## [Event API ; event.remove() does not work on key with special character ? encoding problem?](https://discuss.elastic.co/t/event-api-event-remove-does-not-work-on-key-with-special-character-encoding-problem/157432)
**Author:** [@ljak](https://discuss.elastic.co/u/ljak)\ **Replies:** 0\ **Last updated:** [November 19, 2018, 8:35pm UTC](https://discuss.elastic.co/t/event-api-event-remove-does-not-work-on-key-with-special-character-encoding-problem/157432 "2018-11-19T20:35:03Z")
Hi there, From the CouchDB input plugin, logstash receives this document: { "\_id": "542de4069541c1fc76a649a9fe0377c0", "\_rev": "28-8fb537085cf219b74d13d58400245267", "type": "test", "message": "message test", … --- ## [How should I modify logstash.conf to get the field I want?](https://discuss.elastic.co/t/how-should-i-modify-logstash-conf-to-get-the-field-i-want/157328)
**Author:** [@wangyu](https://discuss.elastic.co/u/wangyu)\ **Replies:** 0\ **Last updated:** [November 19, 2018, 10:14am UTC](https://discuss.elastic.co/t/how-should-i-modify-logstash-conf-to-get-the-field-i-want/157328 "2018-11-19T10:14:48Z")
I use the ELK + Filebeat , all version is 6.4.3 , the OS is windows 10 I add custom field in filebeat.yml , the key name is log\_type , the value of log\_type is nginx-access The picture show part of filebeat.yml. … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=644) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=646)