# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1133 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1134 --- ## [Is there any api like ElasticSearch nest api for Logstash](https://discuss.elastic.co/t/is-there-any-api-like-elasticsearch-nest-api-for-logstash/54321)
**Author:** [@Rajeev\_Gaurav](https://discuss.elastic.co/u/Rajeev_Gaurav)\ **Replies:** 5\ **Last updated:** [July 1, 2016, 6:43am UTC](https://discuss.elastic.co/t/is-there-any-api-like-elasticsearch-nest-api-for-logstash/54321 "2016-07-01T06:43:32Z")
Hi, Is there any api like ElasticSearch nest api for Logstash so that I can automate the loading of data programmatic. As of now I am using process and the triggering the bin and passing the command "logstash -f " Th… --- ## [New to ELK (Syslog issue)](https://discuss.elastic.co/t/new-to-elk-syslog-issue/54141)
**Author:** [@Tommo](https://discuss.elastic.co/u/Tommo)\ **Replies:** 2\ **Last updated:** [July 1, 2016, 6:42am UTC](https://discuss.elastic.co/t/new-to-elk-syslog-issue/54141 "2016-07-01T06:42:23Z")
I've setup ELK on Windows 2012 and can't seem to get the syslog working. So at this stage my logstash.json file has a simple input for syslog. input { beats { port =\> 5044 type =\> log } syslog { } I… --- ## [No field for severity , for logs coming from log4j](https://discuss.elastic.co/t/no-field-for-severity-for-logs-coming-from-log4j/54506)
**Author:** [@babeesh](https://discuss.elastic.co/u/babeesh)\ **Replies:** 0\ **Last updated:** [July 1, 2016, 6:35am UTC](https://discuss.elastic.co/t/no-field-for-severity-for-logs-coming-from-log4j/54506 "2016-07-01T06:35:57Z")
There is no field for Severity/Priority in log4j logs. Logstash version 2.2.x log4j config: log4j.rootLogger=stdout log4j.appender.stdout=org.apache.log4j.net.SocketAppender log4j.appender.stdout.port=5… --- ## [How to ship/fetch data from Client server which is not directly accessible to Logstash](https://discuss.elastic.co/t/how-to-ship-fetch-data-from-client-server-which-is-not-directly-accessible-to-logstash/54130)
**Author:** [@shubham\_deodia](https://discuss.elastic.co/u/shubham_deodia)\ **Replies:** 3\ **Last updated:** [July 1, 2016, 5:37am UTC](https://discuss.elastic.co/t/how-to-ship-fetch-data-from-client-server-which-is-not-directly-accessible-to-logstash/54130 "2016-07-01T05:37:48Z")
I had implemented ELK stack on my Localhost. I want to Ship Data from Client server to ELK server. I am able to do so , if client server is directly accessible from ELK server. But, in production environment , the … --- ## [Can Logstash process messages AS-IS?](https://discuss.elastic.co/t/can-logstash-process-messages-as-is/54433)
**Author:** [@urover](https://discuss.elastic.co/u/urover)\ **Replies:** 2\ **Last updated:** [July 1, 2016, 5:25am UTC](https://discuss.elastic.co/t/can-logstash-process-messages-as-is/54433 "2016-07-01T05:25:23Z")
We're trying to use Logstash to process messages from Kafka and output them to multiple sinks. However, Logstash seems to default to adding some fields to the event such as @version, @timestamp, @source (or host?), etc. … --- ## [Logstash can't read files ( Permission denied )](https://discuss.elastic.co/t/logstash-cant-read-files-permission-denied/53728)
**Author:** [@kayyazka](https://discuss.elastic.co/u/kayyazka)\ **Replies:** 5\ **Last updated:** [July 1, 2016, 1:37am UTC](https://discuss.elastic.co/t/logstash-cant-read-files-permission-denied/53728 "2016-07-01T01:37:21Z")
Hi All, I have problem about Logstash,. Log from logstash.log {:timestamp=\>"2016-06-23T09:17:41.150000+0700", :message=\>"failed to open /var/log/messages: Permission denied - /var/log/messages", :level=\>:warn} {:ti… --- ## [Remove data from json array in Logstash](https://discuss.elastic.co/t/remove-data-from-json-array-in-logstash/54204)
**Author:** [@Geezer](https://discuss.elastic.co/u/Geezer)\ **Replies:** 3\ **Last updated:** [June 30, 2016, 9:04pm UTC](https://discuss.elastic.co/t/remove-data-from-json-array-in-logstash/54204 "2016-06-30T21:04:44Z")
I have a data source which is sending in data in json format. Some of this data is held in an array, which makes it difficult to visualize. The array only holds one set of data however. For example, the "docs" array b… --- ## [Launching a multi-file logstash config](https://discuss.elastic.co/t/launching-a-multi-file-logstash-config/54344)
**Author:** [@3ygun](https://discuss.elastic.co/u/3ygun)\ **Replies:** 4\ **Last updated:** [June 30, 2016, 9:05pm UTC](https://discuss.elastic.co/t/launching-a-multi-file-logstash-config/54344 "2016-06-30T21:05:24Z")
I have a multi-file configuration however when launching it I with -f I get different outputs from: bin/logstash -f PATH\_TO\_CONFIG\_DIR/ and bin/logstash -f PATH\_TO\_CONFIG\_DIR/\*.conf I was under the impression these … --- ## [Logstash mutate add\_field](https://discuss.elastic.co/t/logstash-mutate-add-field/54459)
**Author:** [@Sridhar\_Yadav\_Manoha](https://discuss.elastic.co/u/Sridhar_Yadav_Manoha)\ **Replies:** 8\ **Last updated:** [June 30, 2016, 7:37pm UTC](https://discuss.elastic.co/t/logstash-mutate-add-field/54459 "2016-06-30T19:37:47Z")
I have an xml which I have an attached I am parsing this xml using xml filters in logstash. I have a field t ="296" in my xml. This represents response time. And logstash is parsing each field as an array like "respo… --- ## [Can't install any plugins on 5.0.0-alpha3](https://discuss.elastic.co/t/cant-install-any-plugins-on-5-0-0-alpha3/54463)
**Author:** [@Samir\_Rahman](https://discuss.elastic.co/u/Samir_Rahman)\ **Replies:** 0\ **Last updated:** [June 30, 2016, 6:34pm UTC](https://discuss.elastic.co/t/cant-install-any-plugins-on-5-0-0-alpha3/54463 "2016-06-30T18:34:45Z")
Whenever I run ./bin/logstash-plugin, I keep getting error: Error opening script file: /.../Downloads/logstash-5.0.0-alpha3/lib/pluginmanager/main.rb (No such file or directory), even though if I cd or cat to the path it… --- ## [SIEM deployment design advise](https://discuss.elastic.co/t/siem-deployment-design-advise/54041)
**Author:** [@Tal\_Bar\_Or](https://discuss.elastic.co/u/Tal_Bar_Or)\ **Replies:** 1\ **Last updated:** [June 30, 2016, 6:31pm UTC](https://discuss.elastic.co/t/siem-deployment-design-advise/54041 "2016-06-30T18:31:40Z")
Hello All, I am in the process of building of SIEM (Security information and event management ) based on the ELK technology. For that purpose i would like to carefully design the deployment so it can accommodate to med… --- ## [HTTP input plugin - turn the headers out](https://discuss.elastic.co/t/http-input-plugin-turn-the-headers-out/53996)
**Author:** [@michaltptk](https://discuss.elastic.co/u/michaltptk)\ **Replies:** 1\ **Last updated:** [June 30, 2016, 6:28pm UTC](https://discuss.elastic.co/t/http-input-plugin-turn-the-headers-out/53996 "2016-06-30T18:28:47Z")
Is there a possibility to remove request headers without interference to grok or any other filter plugin? My guess is to edit response\_headers but I didn't find the proper way to do it. --- ## [Do I need to run in background logstash process for schedule jdbc input plugin?](https://discuss.elastic.co/t/do-i-need-to-run-in-background-logstash-process-for-schedule-jdbc-input-plugin/53714)
**Author:** [@metload](https://discuss.elastic.co/u/metload)\ **Replies:** 5\ **Last updated:** [June 30, 2016, 5:47pm UTC](https://discuss.elastic.co/t/do-i-need-to-run-in-background-logstash-process-for-schedule-jdbc-input-plugin/53714 "2016-06-30T17:47:15Z")
Hi everybody I am trying to run logstash conf file through Jenkins, I am using a jdbc input plugin with a schedule property to run each morning at 6.00 AM and I have a question, Do I need to leave the process run in ba… --- ## [Getting grokfailure, but grok constructor test works](https://discuss.elastic.co/t/getting-grokfailure-but-grok-constructor-test-works/54331)
**Author:** [@stecino](https://discuss.elastic.co/u/stecino)\ **Replies:** 2\ **Last updated:** [June 30, 2016, 5:17pm UTC](https://discuss.elastic.co/t/getting-grokfailure-but-grok-constructor-test-works/54331 "2016-06-30T17:17:40Z")
Hello, I am getting this discrepancy between http://grokconstructor.appspot.com/do/match and logstash grok filter. In the grok constructor site I am getting following parsed: "10.1.40.20 22.77.167.65 - \[29/Jun/2016:… --- ## [Re-running /opt/logstash/bin/logstash with same conf file does not produce output](https://discuss.elastic.co/t/re-running-opt-logstash-bin-logstash-with-same-conf-file-does-not-produce-output/53791)
**Author:** [@ZillaG](https://discuss.elastic.co/u/ZillaG)\ **Replies:** 2\ **Last updated:** [June 30, 2016, 4:26pm UTC](https://discuss.elastic.co/t/re-running-opt-logstash-bin-logstash-with-same-conf-file-does-not-produce-output/53791 "2016-06-30T16:26:36Z")
Using logstash v2.3.2 I have a \*.conf file that I use for this command /opt/logstash/bin/logstash -f /path/mylogstash.conf I was able to run this yesterday and was able to produce my desired mylogstash.json output … --- ## [Logstash kafka input](https://discuss.elastic.co/t/logstash-kafka-input/54424)
**Author:** [@somu1276](https://discuss.elastic.co/u/somu1276)\ **Replies:** 0\ **Last updated:** [June 30, 2016, 3:19pm UTC](https://discuss.elastic.co/t/logstash-kafka-input/54424 "2016-06-30T15:19:34Z")
I am trying logstash kafka input. generally all plugins wrap the message in "message", but kafka input showing without message field. {"msg" =\> "validationStatus : S","hostName" =\> "xxxxccrc-val","level" =\> "INFO","lo… --- ## [Help with GROK pattern - Cisco ASA CX](https://discuss.elastic.co/t/help-with-grok-pattern-cisco-asa-cx/50126)
**Author:** [@Mark\_Collins](https://discuss.elastic.co/u/Mark_Collins)\ **Replies:** 7\ **Last updated:** [June 30, 2016, 2:40pm UTC](https://discuss.elastic.co/t/help-with-grok-pattern-cisco-asa-cx/50126 "2016-06-30T14:40:22Z")
Hi All I have a new ELK installation and all is going well, have a couple of windows servers and firewall sending logs. Once I had told the installation to use more RAM all is running really good. Trying to send logs f… --- ## [Logstash-filter-elasticsearch-2.1.0 Argument error](https://discuss.elastic.co/t/logstash-filter-elasticsearch-2-1-0-argument-error/54406)
**Author:** [@pjanzen](https://discuss.elastic.co/u/pjanzen)\ **Replies:** 2\ **Last updated:** [June 30, 2016, 2:23pm UTC](https://discuss.elastic.co/t/logstash-filter-elasticsearch-2-1-0-argument-error/54406 "2016-06-30T14:23:58Z")
Hi All, I have installed logstash-filter-elasticsearch-2.1.0 and after configuring this filter plugin I end up with: "exception" =\> ArgumentError: Cabin::Channel#warn(message, data={}) takes at most 2 arguments I am… --- ## [File input: ignore\_older quick question](https://discuss.elastic.co/t/file-input-ignore-older-quick-question/54388)
**Author:** [@Alex\_6](https://discuss.elastic.co/u/Alex_6)\ **Replies:** 3\ **Last updated:** [June 30, 2016, 11:16am UTC](https://discuss.elastic.co/t/file-input-ignore-older-quick-question/54388 "2016-06-30T11:16:55Z")
I just want to check the behaviour of the ignore\_older parameter. Logstash 2.3.3. Logstash has a file input and is set start\_position: beginning. ignore\_older set to default of 24h. Logstash has been down for 48h. … --- ## [Input plugin issue in maintaining JDBC connections](https://discuss.elastic.co/t/input-plugin-issue-in-maintaining-jdbc-connections/53967)
**Author:** [@KiranG](https://discuss.elastic.co/u/KiranG)\ **Replies:** 6\ **Last updated:** [June 30, 2016, 9:45am UTC](https://discuss.elastic.co/t/input-plugin-issue-in-maintaining-jdbc-connections/53967 "2016-06-30T09:45:00Z")
Hi, I am using JDBC input plugin for a connection to oracle database. There are several sql files one for each jdbc input config file. Now, the issue is, logstash is successfully establishing a connection to oracle … --- ## [Trying to parse Hadoop log with logstash](https://discuss.elastic.co/t/trying-to-parse-hadoop-log-with-logstash/53894)
**Author:** [@siasic](https://discuss.elastic.co/u/siasic)\ **Replies:** 5\ **Last updated:** [June 30, 2016, 8:54am UTC](https://discuss.elastic.co/t/trying-to-parse-hadoop-log-with-logstash/53894 "2016-06-30T08:54:46Z")
hello, I have a Hadoop cluster (Hortonworks) and i try to send hdfs/yarn/.... logs to logstash for parsing them. The sent is made by filebeat. My logs look like that (i think it is log4j type) : 2016-06-12 03:00:20… --- ## [Date filter recognizing custom grok pattern?](https://discuss.elastic.co/t/date-filter-recognizing-custom-grok-pattern/24363)
**Author:** [@DavidL](https://discuss.elastic.co/u/DavidL)\ **Replies:** 11\ **Last updated:** [June 30, 2016, 7:44am UTC](https://discuss.elastic.co/t/date-filter-recognizing-custom-grok-pattern/24363 "2016-06-30T07:44:16Z")
I am dealing with a log file that has a special timestamp, 2014-12-14 23:59:40.227 -8 ISO8601 will recognize the date and time but not the time zone, so I built my own, and when I tried to use the date filter, a… --- ## [Parsing string and assigning fields](https://discuss.elastic.co/t/parsing-string-and-assigning-fields/54086)
**Author:** [@apomerenk](https://discuss.elastic.co/u/apomerenk)\ **Replies:** 7\ **Last updated:** [June 30, 2016, 7:24am UTC](https://discuss.elastic.co/t/parsing-string-and-assigning-fields/54086 "2016-06-30T07:24:45Z")
How would I use a logstash filter to parse the message: 0.00 pool 3041 and assign fields CPU\_USAGE, PROCESS\_NAME, and PROCESS\_ID to it? For example CPU\_USAGE: 0.00 (type float) PROCESS\_… --- ## [Gsub specific characters but not all of them in value](https://discuss.elastic.co/t/gsub-specific-characters-but-not-all-of-them-in-value/53909)
**Author:** [@Spacemansam](https://discuss.elastic.co/u/Spacemansam)\ **Replies:** 3\ **Last updated:** [June 30, 2016, 7:21am UTC](https://discuss.elastic.co/t/gsub-specific-characters-but-not-all-of-them-in-value/53909 "2016-06-30T07:21:26Z")
Hi, I am desperately trying to figure out a way to replace full stops in an alphabetic string in a value without affecting decimal values. I am faced with two problems, I have a field value with an unknown quantity of … --- ## [Replace @timestamp with some other field](https://discuss.elastic.co/t/replace-timestamp-with-some-other-field/28284)
**Author:** [@ajays](https://discuss.elastic.co/u/ajays)\ **Replies:** 10\ **Last updated:** [June 30, 2016, 7:18am UTC](https://discuss.elastic.co/t/replace-timestamp-with-some-other-field/28284 "2016-06-30T07:18:41Z")
Hi, I want to replace @timestamp with some other timestamp field having format like this 11/Jan/2014:05:06:24 +05:30. Can anybody please help? Regards, AjayS --- ## [No output from logstash](https://discuss.elastic.co/t/no-output-from-logstash/54339)
**Author:** [@a\_soong](https://discuss.elastic.co/u/a_soong)\ **Replies:** 1\ **Last updated:** [June 30, 2016, 6:15am UTC](https://discuss.elastic.co/t/no-output-from-logstash/54339 "2016-06-30T06:15:28Z")
im using the example from the documentation. input { file { path =\> "H:\\logstash\\zipped\_folders\\logstash-2.3.1\\bin\\first-pipeline-data" start\_position =\> beginning #1 #ignore\_older =\> 0 } } output { stdout{} }… --- ## [Gork patterns with | does not work](https://discuss.elastic.co/t/gork-patterns-with-does-not-work/54335)
**Author:** [@keamas](https://discuss.elastic.co/u/keamas)\ **Replies:** 1\ **Last updated:** [June 30, 2016, 6:04am UTC](https://discuss.elastic.co/t/gork-patterns-with-does-not-work/54335 "2016-06-30T06:04:08Z")
Hi, I am using gork to extract the fields of this kind of syslog messages: Test-Cluster/IM-6/IM-HQ-01/box\_Firewall\_Activity: Info IM-HQ-01 Remove: type=FWD|proto=TCP|srcIF=port7.122|srcIP=10.244.122.23|srcPort=62946|sr… --- ## [Error Outputting data to kafka](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192)
**Author:** [@naiktej13](https://discuss.elastic.co/u/naiktej13)\ **Replies:** 7\ **Last updated:** [June 30, 2016, 4:07am UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192 "2016-06-30T04:07:43Z")
I want to publish my script data to kafka cluster using logstash. For that I configured logstash on centOS hear is the config file i wrote: input { exec { command =\> "python /home/elastic/test.p… --- ## [Logstash influxdb writes tags with wrong datatype?](https://discuss.elastic.co/t/logstash-influxdb-writes-tags-with-wrong-datatype/54355)
**Author:** [@rbrandstaedter](https://discuss.elastic.co/u/rbrandstaedter)\ **Replies:** 0\ **Last updated:** [June 30, 2016, 3:31am UTC](https://discuss.elastic.co/t/logstash-influxdb-writes-tags-with-wrong-datatype/54355 "2016-06-30T03:31:02Z")
I don't know if that top makes sense at all but I'm writing tag fields to influxdb from logstash using influxdb { host =\> "localhost" db =\> "dynatrace" allow\_time\_override =\> true measurement =\> "%{\[busine… --- ## [Connection refused for logstash configuration file](https://discuss.elastic.co/t/connection-refused-for-logstash-configuration-file/54307)
**Author:** [@a\_soong](https://discuss.elastic.co/u/a_soong)\ **Replies:** 1\ **Last updated:** [June 30, 2016, 3:01am UTC](https://discuss.elastic.co/t/connection-refused-for-logstash-configuration-file/54307 "2016-06-30T03:01:57Z")
input { file { path =\> "H:\\logstash\\zipped\_folders\\logstash-2.3.1\\bin\\first-pipeline-data" start\_position =\> beginning #1 #ignore\_older =\> 0 } } filter { grok { # Grok looks for patterns in incoming log data … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1132) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1134)