# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1136 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1137 --- ## [Sincedb\_path under windows](https://discuss.elastic.co/t/sincedb-path-under-windows/53491)
**Author:** [@pittoch](https://discuss.elastic.co/u/pittoch)\ **Replies:** 1\ **Last updated:** [June 23, 2016, 12:27pm UTC](https://discuss.elastic.co/t/sincedb-path-under-windows/53491 "2016-06-23T12:27:09Z")
Hi Where is this directory sincedb\_path on logstash windows installation ? Thanks --- ## [Pulling Nested XML Data from a http API for loading into Elasticsearch](https://discuss.elastic.co/t/pulling-nested-xml-data-from-a-http-api-for-loading-into-elasticsearch/52492)
**Author:** [@Johnnycc](https://discuss.elastic.co/u/Johnnycc)\ **Replies:** 1\ **Last updated:** [June 23, 2016, 11:07am UTC](https://discuss.elastic.co/t/pulling-nested-xml-data-from-a-http-api-for-loading-into-elasticsearch/52492 "2016-06-23T11:07:38Z")
Ok first time poster - so please go easy on me - two questions. I would like to be able to pull data with logstash from an API. The API is at an address that looks like the following; http://domain.com/reports/1000000 … --- ## [Count messages per host](https://discuss.elastic.co/t/count-messages-per-host/53743)
**Author:** [@marcmga](https://discuss.elastic.co/u/marcmga)\ **Replies:** 0\ **Last updated:** [June 23, 2016, 6:59am UTC](https://discuss.elastic.co/t/count-messages-per-host/53743 "2016-06-23T06:59:09Z")
Hi there, I'm not capable of making this work. I receive a lot of msg from several servers. The kind of messages are the same through servers. What i want to do is count this kind of msg of each host (isolated by host)… --- ## [Oracle column date value is automatically getting converted by logstash to UTC zone](https://discuss.elastic.co/t/oracle-column-date-value-is-automatically-getting-converted-by-logstash-to-utc-zone/53669)
**Author:** [@KiranG](https://discuss.elastic.co/u/KiranG)\ **Replies:** 2\ **Last updated:** [June 23, 2016, 5:06am UTC](https://discuss.elastic.co/t/oracle-column-date-value-is-automatically-getting-converted-by-logstash-to-utc-zone/53669 "2016-06-23T05:06:42Z")
Hi, Logstash is modifying the data while writing to elastic search. There is a date column in oracle database with time in it. when I am pulling it using logstash2.1.0 to Elasticsearch, the time value is getting conv… --- ## [How does logstash generate '\_id' when the outputting to elastic search?](https://discuss.elastic.co/t/how-does-logstash-generate--id-when-the-outputting-to-elastic-search/53722)
**Author:** [@Matthew\_Laue](https://discuss.elastic.co/u/Matthew_Laue)\ **Replies:** 1\ **Last updated:** [June 23, 2016, 4:49am UTC](https://discuss.elastic.co/t/how-does-logstash-generate--id-when-the-outputting-to-elastic-search/53722 "2016-06-23T04:49:30Z")
The elasticsearch \_id parameter is set to some hash value. e.g. "\_id": "AVVpVdoIEU8QqcRWkA9P", How is this calculated? And where? matt@elk:~$ cat /etc/logstash/conf.d/apache.conf input { file { path =\>… --- ## [Logstash v2.1.3: A plugin had an unrecoverable error](https://discuss.elastic.co/t/logstash-v2-1-3-a-plugin-had-an-unrecoverable-error/53590)
**Author:** [@tuankun](https://discuss.elastic.co/u/tuankun)\ **Replies:** 1\ **Last updated:** [June 23, 2016, 1:01am UTC](https://discuss.elastic.co/t/logstash-v2-1-3-a-plugin-had-an-unrecoverable-error/53590 "2016-06-23T01:01:41Z")
Hi, I push windows eventlog using winlogbeat-dev(Version: 5.0.0-nightly2bfc1c4) to logstash v2.1.3,the logstash configuration is input{ beats { add\_field =\>\["itcategory","it\_security\_windo… --- ## [Windows Event Log filtering](https://discuss.elastic.co/t/windows-event-log-filtering/53448)
**Author:** [@brandonmcgrath1](https://discuss.elastic.co/u/brandonmcgrath1)\ **Replies:** 6\ **Last updated:** [June 22, 2016, 11:01pm UTC](https://discuss.elastic.co/t/windows-event-log-filtering/53448 "2016-06-22T23:01:17Z")
So, I'm trying to upgrade my logstash conf file to filter the windows event logs and make them more useful. At the moment the log dumps loads of information into one field called "message", like so: {"EventTime":"2016-… --- ## [An input plugin for HDFS](https://discuss.elastic.co/t/an-input-plugin-for-hdfs/53704)
**Author:** [@ni4ni](https://discuss.elastic.co/u/ni4ni)\ **Replies:** 0\ **Last updated:** [June 22, 2016, 7:45pm UTC](https://discuss.elastic.co/t/an-input-plugin-for-hdfs/53704 "2016-06-22T19:45:48Z")
Hello I need a HDFS input plugin that will enable logstash to read files from HDFS and index them in Elasticsearch. I looked everywhere but all I can find is the reverse operation suggested by webHdfs. Does such plugin… --- ## [Logstash crashing unexpectedly](https://discuss.elastic.co/t/logstash-crashing-unexpectedly/53188)
**Author:** [@tcouto](https://discuss.elastic.co/u/tcouto)\ **Replies:** 4\ **Last updated:** [June 22, 2016, 6:24pm UTC](https://discuss.elastic.co/t/logstash-crashing-unexpectedly/53188 "2016-06-22T18:24:32Z")
Hello, logstash keeps failing and spitting out this error: NoMethodError: undefined method \`join' for "\_grokparsefailure":String receive at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-gelf-1.0.0/lib/lo… --- ## [Elasticsearch convert filter not converting type to float](https://discuss.elastic.co/t/elasticsearch-convert-filter-not-converting-type-to-float/53675)
**Author:** [@apomerenk](https://discuss.elastic.co/u/apomerenk)\ **Replies:** 3\ **Last updated:** [June 22, 2016, 3:48pm UTC](https://discuss.elastic.co/t/elasticsearch-convert-filter-not-converting-type-to-float/53675 "2016-06-22T15:48:28Z")
I am trying to run a script and take the output of the script in the command line as an input to elasticsearch. I am then trying to convert this input into a float, and log that through logstash. My code currently loo… --- ## [Cannot use elasticsearch output module twice](https://discuss.elastic.co/t/cannot-use-elasticsearch-output-module-twice/53112)
**Author:** [@maxlock](https://discuss.elastic.co/u/maxlock)\ **Replies:** 3\ **Last updated:** [June 22, 2016, 2:48pm UTC](https://discuss.elastic.co/t/cannot-use-elasticsearch-output-module-twice/53112 "2016-06-22T14:48:26Z")
Hi All, I'm trying to configure Logstash 2.3.2 to call the elasticsearch output module twice as I'm migrating to a new cluster. Calling the module once for either cluster works fine, but if I call it twice no packets a… --- ## [Logstash-input-s3 assistance](https://discuss.elastic.co/t/logstash-input-s3-assistance/53670)
**Author:** [@Akumar](https://discuss.elastic.co/u/Akumar)\ **Replies:** 0\ **Last updated:** [June 22, 2016, 2:20pm UTC](https://discuss.elastic.co/t/logstash-input-s3-assistance/53670 "2016-06-22T14:20:15Z")
I am trying to read in cloudtrail logs from an S3 bucket and need assistance with the configuration and am hamstrung by limited example documentation. The bucket data is in: All buckets/auditLogs//Cloudtrail//// s3 … --- ## [Question about Multiline codec in combining the lines](https://discuss.elastic.co/t/question-about-multiline-codec-in-combining-the-lines/42360)
**Author:** [@guojingjing](https://discuss.elastic.co/u/guojingjing)\ **Replies:** 5\ **Last updated:** [June 22, 2016, 2:08pm UTC](https://discuss.elastic.co/t/question-about-multiline-codec-in-combining-the-lines/42360 "2016-06-22T14:08:54Z")
I have a fake data file: ST 1 EN 1 ST 2 EN 2 ST 3 EN 3 My config file is input { stdin {} file { # type =\> "tomcat" path =\> "C:\\1LOGS\\XXX\\data\\fakedata4.log" start\_position =\> beginning… --- ## [Logstash Service doesn't load .conf file](https://discuss.elastic.co/t/logstash-service-doesnt-load-conf-file/53434)
**Author:** [@S\_M](https://discuss.elastic.co/u/S_M)\ **Replies:** 1\ **Last updated:** [June 22, 2016, 2:03pm UTC](https://discuss.elastic.co/t/logstash-service-doesnt-load-conf-file/53434 "2016-06-22T14:03:35Z")
Dear all, I have problem with my Logstash (I don't know it is really problem or not). I have installed Logstash on my UbuntuServer with recommended manner described on Logstash official page (I use apt-get install to … --- ## [Replace content of host field with part from message field](https://discuss.elastic.co/t/replace-content-of-host-field-with-part-from-message-field/53636)
**Author:** [@himbeere](https://discuss.elastic.co/u/himbeere)\ **Replies:** 3\ **Last updated:** [June 22, 2016, 1:59pm UTC](https://discuss.elastic.co/t/replace-content-of-host-field-with-part-from-message-field/53636 "2016-06-22T13:59:48Z")
Hello. I'm trying to replace the content of the host field with part from message field. My message field contains something like that: message:\<85\>Jun 22 11:24:33 backup02.bla.domain.de sudo: ..... Now i wanna tak… --- ## [Multiple elapsed.time on a single event](https://discuss.elastic.co/t/multiple-elapsed-time-on-a-single-event/53664)
**Author:** [@kumarhanu](https://discuss.elastic.co/u/kumarhanu)\ **Replies:** 0\ **Last updated:** [June 22, 2016, 1:57pm UTC](https://discuss.elastic.co/t/multiple-elapsed-time-on-a-single-event/53664 "2016-06-22T13:57:38Z")
//Below statement creates elapsed.time on end event of task1 elapsed { start\_tag =\> "task1Started" end\_tag =\> "task1Completed" unique\_id\_field =\> "transactionId" } //Below statement creates elapsed.time on end eve… --- ## [Read .asl Log-Files on Mac OS X from "/var/log/servermetricsd/"](https://discuss.elastic.co/t/read-asl-log-files-on-mac-os-x-from-var-log-servermetricsd/53619)
**Author:** [@beelastic](https://discuss.elastic.co/u/beelastic)\ **Replies:** 2\ **Last updated:** [June 22, 2016, 1:30pm UTC](https://discuss.elastic.co/t/read-asl-log-files-on-mac-os-x-from-var-log-servermetricsd/53619 "2016-06-22T13:30:05Z")
Hello everyone! I want to use my ELK stack to monitor couple of Mac Servers. The log files are located in "/var/log/servermetricsd/" and have the file extension "\*.asl". I use Logstash with the following configuratio… --- ## [Can logstash handle array of nested objects?](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558)
**Author:** [@ChrisR](https://discuss.elastic.co/u/ChrisR)\ **Replies:** 6\ **Last updated:** [June 22, 2016, 12:44pm UTC](https://discuss.elastic.co/t/can-logstash-handle-array-of-nested-objects/53558 "2016-06-22T12:44:44Z")
I am unable to configure logstash to load an array of nested objects as described in the elasticsearch guide. I can map and load the array of nested objects via the elasticsearch REST API (as shown below), but I am una… --- ## [Aggregate logs without special pattern or "flag" for the last of the group](https://discuss.elastic.co/t/aggregate-logs-without-special-pattern-or-flag-for-the-last-of-the-group/53171)
**Author:** [@Akita](https://discuss.elastic.co/u/Akita)\ **Replies:** 2\ **Last updated:** [June 22, 2016, 9:03am UTC](https://discuss.elastic.co/t/aggregate-logs-without-special-pattern-or-flag-for-the-last-of-the-group/53171 "2016-06-22T09:03:07Z")
Might be linked with the discussion from Aggregate data based on some field: Hi everyone, I have incoming logs in Logsash looking like these: id1 valueA, id1 valueB, id1 valueC, id1 valueD, id2 valueAA, id2 valueBB… --- ## [Twitter and Logstash config files](https://discuss.elastic.co/t/twitter-and-logstash-config-files/53655)
**Author:** [@jamesl](https://discuss.elastic.co/u/jamesl)\ **Replies:** 0\ **Last updated:** [June 22, 2016, 12:21pm UTC](https://discuss.elastic.co/t/twitter-and-logstash-config-files/53655 "2016-06-22T12:21:50Z")
Hi. Trying to get Twitter feeds into my ES System but its not working like the way it should? If I remove the 'full\_tweet = true' from the input file I get some of the message (see attached) but not any Geo or other goo… --- ## [Exclude particular domain name in logstash grok filter section](https://discuss.elastic.co/t/exclude-particular-domain-name-in-logstash-grok-filter-section/52851)
**Author:** [@shan](https://discuss.elastic.co/u/shan)\ **Replies:** 5\ **Last updated:** [June 22, 2016, 11:55am UTC](https://discuss.elastic.co/t/exclude-particular-domain-name-in-logstash-grok-filter-section/52851 "2016-06-22T11:55:29Z")
Hi, I want to exclude the some domain name \*.com in my messages while storing into elasticsearch. How to mention in grok filter section ? Thanks, --- ## [Field name start with](https://discuss.elastic.co/t/field-name-start-with/53513)
**Author:** [@Nizar\_Khalifa](https://discuss.elastic.co/u/Nizar_Khalifa)\ **Replies:** 1\ **Last updated:** [June 22, 2016, 11:26am UTC](https://discuss.elastic.co/t/field-name-start-with/53513 "2016-06-22T11:26:24Z")
How can i check if a field name (not value) start with some caracter ? { params{ firstname : value1, $lastname : value2 } } How can i verify if lastname field start with $ ? And if yes remove it. B.R --- ## [Default and Customer Input and Filter Config in diffrent Files on diffrent Servers and System](https://discuss.elastic.co/t/default-and-customer-input-and-filter-config-in-diffrent-files-on-diffrent-servers-and-system/53283)
**Author:** [@mehrentraut](https://discuss.elastic.co/u/mehrentraut)\ **Replies:** 1\ **Last updated:** [June 22, 2016, 11:09am UTC](https://discuss.elastic.co/t/default-and-customer-input-and-filter-config-in-diffrent-files-on-diffrent-servers-and-system/53283 "2016-06-22T11:09:10Z")
Hello, I think I've just not found the right approach, but I am stuck here's my question: I wonder how I get it, that applies a default input and filter config all scanned files. It nevertheless is possible Custom conf… --- ## [Logstash Elapsed filter 3.0.2 expired event issue](https://discuss.elastic.co/t/logstash-elapsed-filter-3-0-2-expired-event-issue/50204)
**Author:** [@hirakj](https://discuss.elastic.co/u/hirakj)\ **Replies:** 1\ **Last updated:** [June 22, 2016, 9:13am UTC](https://discuss.elastic.co/t/logstash-elapsed-filter-3-0-2-expired-event-issue/50204 "2016-06-22T09:13:50Z")
Hello, I have been using the elapsed filter on version 3.0.2. Elapsed filter doesn't seem to generate an expired event. Is there a new configuration for the filter I'm not seeing or is this a bug? I'm running on a wi… --- ## [Logstash config example to index a couchdb database](https://discuss.elastic.co/t/logstash-config-example-to-index-a-couchdb-database/53627)
**Author:** [@veto](https://discuss.elastic.co/u/veto)\ **Replies:** 0\ **Last updated:** [June 22, 2016, 8:51am UTC](https://discuss.elastic.co/t/logstash-config-example-to-index-a-couchdb-database/53627 "2016-06-22T08:51:06Z")
can anybody show a working example of a very simply logstash config file for the couchdb\_changes plugin for indexing a couchdb database to the elasticsearch search 5.0 --- ## [Integration of MSSQL](https://discuss.elastic.co/t/integration-of-mssql/40341)
**Author:** [@praveen\_siem](https://discuss.elastic.co/u/praveen_siem)\ **Replies:** 4\ **Last updated:** [June 22, 2016, 6:10am UTC](https://discuss.elastic.co/t/integration-of-mssql/40341 "2016-06-22T06:10:03Z")
Dear Team, Has anyone at any of the customer environment has integrated Microsoft SQL Database to monitor the logs from the database. If yes, kindly share the necessary steps to perform the same. Best Regards- Pra… --- ## [Sync new entries in Oracle DB using sql\_last\_value - Not fetching the new entries](https://discuss.elastic.co/t/sync-new-entries-in-oracle-db-using-sql-last-value-not-fetching-the-new-entries/53271)
**Author:** [@SKumarMN](https://discuss.elastic.co/u/SKumarMN)\ **Replies:** 1\ **Last updated:** [June 22, 2016, 5:40am UTC](https://discuss.elastic.co/t/sync-new-entries-in-oracle-db-using-sql-last-value-not-fetching-the-new-entries/53271 "2016-06-22T05:40:12Z")
input { jdbc { jdbc\_connection\_string =\> "jdbc:oracle:thin:@host:port/service" jdbc\_user =\> "user" jdbc\_password =\> "pass" jdbc\_validate\_connection =\> "true" jdbc\_driver\_library =\> "C:\\logstash\\lib\\oj… --- ## [Logstash error after filebeat restarted](https://discuss.elastic.co/t/logstash-error-after-filebeat-restarted/52819)
**Author:** [@ivanandrianto95](https://discuss.elastic.co/u/ivanandrianto95)\ **Replies:** 2\ **Last updated:** [June 22, 2016, 2:43am UTC](https://discuss.elastic.co/t/logstash-error-after-filebeat-restarted/52819 "2016-06-22T02:43:13Z")
I got error in Logstash every time I restart Filebeat. To make logstash work again, I have to kll the Logstash process and restart it. How to fix it? Here is the error {:timestamp=\>"2016-06-15T00:48:30.697000-0400", :… --- ## [WQL is supported in Logstash](https://discuss.elastic.co/t/wql-is-supported-in-logstash/53562)
**Author:** [@raakeshtrip](https://discuss.elastic.co/u/raakeshtrip)\ **Replies:** 1\ **Last updated:** [June 22, 2016, 2:33am UTC](https://discuss.elastic.co/t/wql-is-supported-in-logstash/53562 "2016-06-22T02:33:49Z")
Is WQL is supported in Logstash .. I tried Using WMI Plugin. But its not working. My goal is to integrate Windows Performance Monitor with Kibana using ELK --- ## [:expected\_charset=\>"UTF-8", :level=\>:warn}](https://discuss.elastic.co/t/expected-charset-utf-8-level-warn/53587)
**Author:** [@santyuste](https://discuss.elastic.co/u/santyuste)\ **Replies:** 0\ **Last updated:** [June 22, 2016, 12:04am UTC](https://discuss.elastic.co/t/expected-charset-utf-8-level-warn/53587 "2016-06-22T00:04:52Z")
When i recibed a event windows log from agent windows server: :expected\_charset=\>"UTF-8", :level=\>:warn} Received an event that has a different character encoding than you configured. {:text=\>"{\\\\"rule\\...... hown … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1135) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1137)