# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1233 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1234 --- ## [Logstash http\_poller input from server with self-signed SSL certificate](https://discuss.elastic.co/t/logstash-http-poller-input-from-server-with-self-signed-ssl-certificate/31407)
**Author:** [@vikbon](https://discuss.elastic.co/u/vikbon)\ **Replies:** 5\ **Last updated:** [September 30, 2015, 7:34pm UTC](https://discuss.elastic.co/t/logstash-http-poller-input-from-server-with-self-signed-ssl-certificate/31407 "2015-09-30T19:34:01Z")
Hi All, I’m new to Elasticsearch and attempting to configure logstash http\_poller input plugin to make api calls against a web server for performance data. Currently getting an “http\_request\_failure.error, handshake\_f… --- ## [Logstash Elasticsearch template for doc\_values](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423)
**Author:** [@elvarb](https://discuss.elastic.co/u/elvarb)\ **Replies:** 8\ **Last updated:** [September 30, 2015, 5:21pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-template-for-doc-values/31423 "2015-09-30T17:21:38Z")
How do I go about modifying the default template for Logstash for it to use doc\_values for as many fields as possible? With Elasticsearch 2.0 doc\_values will be default for all fields instead of optional, so by then thi… --- ## [Parsing syslog from linux rsyslog](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267)
**Author:** [@cnozmn](https://discuss.elastic.co/u/cnozmn)\ **Replies:** 24\ **Last updated:** [September 30, 2015, 3:16pm UTC](https://discuss.elastic.co/t/parsing-syslog-from-linux-rsyslog/31267 "2015-09-30T15:16:54Z")
Hi, I setup ELK stack on my centos machine. In addition, I'm getting syslogs from rsyslog of another centos, So I can see it with "tcpdump" but I wanna see that on Kibana. I think my problem is "logstash.conf file". I… --- ## [Log Shipping Format Poll Results + Docker Logging Webinar](https://discuss.elastic.co/t/log-shipping-format-poll-results-docker-logging-webinar/31415)
**Author:** [@otisg](https://discuss.elastic.co/u/otisg)\ **Replies:** 0\ **Last updated:** [September 30, 2015, 2:40pm UTC](https://discuss.elastic.co/t/log-shipping-format-poll-results-docker-logging-webinar/31415 "2015-09-30T14:40:23Z")
Hi, Thanks to anyone who voted! The results are in, if you're curious: http://blog.sematext.com/2015/09/30/log-shipping-poll-results/ Personally, I'm surprised we didn't see more people sending logs in JSON. Seems… --- ## [Logstash don't parse logfile, but don't produce errors](https://discuss.elastic.co/t/logstash-dont-parse-logfile-but-dont-produce-errors/31229)
**Author:** [@roland](https://discuss.elastic.co/u/roland)\ **Replies:** 7\ **Last updated:** [September 30, 2015, 1:55pm UTC](https://discuss.elastic.co/t/logstash-dont-parse-logfile-but-dont-produce-errors/31229 "2015-09-30T13:55:04Z")
Hi, I have recently installed logstash + kibana + elasticsearch on my laptop. My laptop hosts a Fedora Linux 21 x86\_64 Logstash is version 1.5.4 After configuring logstash and starting it, it was not creating indices… --- ## [Logstash-input-jmx new feature proposal](https://discuss.elastic.co/t/logstash-input-jmx-new-feature-proposal/31388)
**Author:** [@graziano1955](https://discuss.elastic.co/u/graziano1955)\ **Replies:** 1\ **Last updated:** [September 30, 2015, 9:18am UTC](https://discuss.elastic.co/t/logstash-input-jmx-new-feature-proposal/31388 "2015-09-30T09:18:21Z")
Hi, I used the JMX plugin to capture data from a WSO2 platform nodes, reading the attributes exposed by their JMX Beans. But not all data can be captured reading attributes, many data can be read only invoking specific… --- ## [Add map to ELK with pfsense](https://discuss.elastic.co/t/add-map-to-elk-with-pfsense/31250)
**Author:** [@megarobo](https://discuss.elastic.co/u/megarobo)\ **Replies:** 6\ **Last updated:** [September 30, 2015, 9:01am UTC](https://discuss.elastic.co/t/add-map-to-elk-with-pfsense/31250 "2015-09-30T09:01:02Z")
i use ELK logstash 1.4 and kibana 3 and pfsense my ELK work very good and show map that IP blocked by countries i want to add map like this but with map countries with passed IP i possible and witch configuration need … --- ## [Problems with CVS: first fieldname in CVS columns has the entire message](https://discuss.elastic.co/t/problems-with-cvs-first-fieldname-in-cvs-columns-has-the-entire-message/31362)
**Author:** [@Jeff\_Silverman](https://discuss.elastic.co/u/Jeff_Silverman)\ **Replies:** 3\ **Last updated:** [September 30, 2015, 7:28am UTC](https://discuss.elastic.co/t/problems-with-cvs-first-fieldname-in-cvs-columns-has-the-entire-message/31362 "2015-09-30T07:28:12Z")
I am trying to parse a file delimited by @ characters. I am using a CSV filter, and it works properly for every field except the first field. The first field contains the entire message, including the delimiter. cu… --- ## [Logstash agent CPU spikes up to 100%](https://discuss.elastic.co/t/logstash-agent-cpu-spikes-up-to-100/31317)
**Author:** [@Augustin\_Chen](https://discuss.elastic.co/u/Augustin_Chen)\ **Replies:** 2\ **Last updated:** [September 30, 2015, 6:51am UTC](https://discuss.elastic.co/t/logstash-agent-cpu-spikes-up-to-100/31317 "2015-09-30T06:51:25Z")
I install the redis + logstsh indexer + elasticserch + kibana. In each of my application server, the logstash agent is installed to pick up the logs. Some of my logstash agent CPU spike up to 100%, some don't. Is there … --- ## [S3 output using iam roles that gives access to aws security credentials](https://discuss.elastic.co/t/s3-output-using-iam-roles-that-gives-access-to-aws-security-credentials/29302)
**Author:** [@trekr5](https://discuss.elastic.co/u/trekr5)\ **Replies:** 1\ **Last updated:** [September 30, 2015, 2:56am UTC](https://discuss.elastic.co/t/s3-output-using-iam-roles-that-gives-access-to-aws-security-credentials/29302 "2015-09-30T02:56:28Z")
Hello, I really would appreciate assistance regarding this matter. I'm using Logstash 1.5 on Ubuntu 14.04 and am trying to configure my logstash installation to upload data to an s3 bucket via an iam role. Currentl… --- ## [Problem with mapping, upserts and multiple configs](https://discuss.elastic.co/t/problem-with-mapping-upserts-and-multiple-configs/31369)
**Author:** [@griddell](https://discuss.elastic.co/u/griddell)\ **Replies:** 0\ **Last updated:** [September 29, 2015, 10:30pm UTC](https://discuss.elastic.co/t/problem-with-mapping-upserts-and-multiple-configs/31369 "2015-09-29T22:30:36Z")
Using: ES 1.7.1 LS 1.5.4 I am trying the upsert feature and it does not work as I expected (but maybe my expectation is wrong). Here is what I am doing: I created a mapping file for my elasticsearch index. (Let's say… --- ## [Prevent %{field\_name} in output, for fields not always present](https://discuss.elastic.co/t/prevent-field-name-in-output-for-fields-not-always-present/31295)
**Author:** [@nekatreven](https://discuss.elastic.co/u/nekatreven)\ **Replies:** 2\ **Last updated:** [September 29, 2015, 4:47pm UTC](https://discuss.elastic.co/t/prevent-field-name-in-output-for-fields-not-always-present/31295 "2015-09-29T16:47:41Z")
Hello, Basically, the problem is that I need to include fields in my Logstash outputs that aren't always present in the event, but are sometimes present. By not present I mean that the field name is not a defined field … --- ## [The Logstash Metric filter and reality](https://discuss.elastic.co/t/the-logstash-metric-filter-and-reality/29555)
**Author:** [@elvarb](https://discuss.elastic.co/u/elvarb)\ **Replies:** 2\ **Last updated:** [September 29, 2015, 3:59pm UTC](https://discuss.elastic.co/t/the-logstash-metric-filter-and-reality/29555 "2015-09-29T15:59:56Z")
I have been monitoring log flow by just using a simple dashboard with lots of histogram charts showing the flow of incoming logs by tags. This way I can see if a certain Logstash input has stopped or is experiencing heav… --- ## [Transport protocol not working for multiple hosts](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292)
**Author:** [@bennett](https://discuss.elastic.co/u/bennett)\ **Replies:** 9\ **Last updated:** [September 29, 2015, 3:35pm UTC](https://discuss.elastic.co/t/transport-protocol-not-working-for-multiple-hosts/31292 "2015-09-29T15:35:41Z")
can someone tell me why this isn't communicating with my elasticsearch cluster? if i put one host in the host field it communicates but not with multiple. output { elasticsearch { index =\> "logstash-%{+… --- ## [S3 input and Elasticsearch output in Logstash](https://discuss.elastic.co/t/s3-input-and-elasticsearch-output-in-logstash/31340)
**Author:** [@AymenChetoui](https://discuss.elastic.co/u/AymenChetoui)\ **Replies:** 2\ **Last updated:** [September 29, 2015, 2:37pm UTC](https://discuss.elastic.co/t/s3-input-and-elasticsearch-output-in-logstash/31340 "2015-09-29T14:37:49Z")
Hi all, I am trying to get familiar with S3 plugin in Logstash in two steps : 1 - Pushing logs to S3 as output 2 - Getting logs from S3 as input 1 - The Logstash conf file looks like this : output { s3{ … --- ## [Multiline filter fails on one record when handling a large file](https://discuss.elastic.co/t/multiline-filter-fails-on-one-record-when-handling-a-large-file/27401)
**Author:** [@stevedearl](https://discuss.elastic.co/u/stevedearl)\ **Replies:** 11\ **Last updated:** [September 29, 2015, 1:50pm UTC](https://discuss.elastic.co/t/multiline-filter-fails-on-one-record-when-handling-a-large-file/27401 "2015-09-29T13:50:34Z")
Hi Logstasher's, I'm seeing some odd behaviour with Logstash when trying to read files with a multiline filter enabled. I noticed that occasionally I'll see a multiline log entry fail to get detected/tagged as multilin… --- ## [Removing square brackets from JSON output](https://discuss.elastic.co/t/removing-square-brackets-from-json-output/31321)
**Author:** [@maisyweb](https://discuss.elastic.co/u/maisyweb)\ **Replies:** 2\ **Last updated:** [September 29, 2015, 11:47am UTC](https://discuss.elastic.co/t/removing-square-brackets-from-json-output/31321 "2015-09-29T11:47:46Z")
Hi, Logstash newbe here and I think it rocks. I'm converting xml files to output JSON but I noticed that in the JSON all values that are not objects themselves are arrays. For example {"externalID":\["67"\],"latitud… --- ## [Obscuring configuration entries](https://discuss.elastic.co/t/obscuring-configuration-entries/29572)
**Author:** [@kornicameister](https://discuss.elastic.co/u/kornicameister)\ **Replies:** 1\ **Last updated:** [September 29, 2015, 9:09am UTC](https://discuss.elastic.co/t/obscuring-configuration-entries/29572 "2015-09-29T09:09:46Z")
In Logstash base plugin there are several places where plugin writes down its current state upon certain events. That's indeed very useful but problematic if such configuration contains vulnerable data. My questions … --- ## [Logstash stops loading text file into ES](https://discuss.elastic.co/t/logstash-stops-loading-text-file-into-es/31288)
**Author:** [@ankmathur14](https://discuss.elastic.co/u/ankmathur14)\ **Replies:** 1\ **Last updated:** [September 29, 2015, 8:22am UTC](https://discuss.elastic.co/t/logstash-stops-loading-text-file-into-es/31288 "2015-09-29T08:22:15Z")
I have many csv files each of 350MB in size and each having 1 Million lines. Data present in csv file is in json format e.g {"operatingsystem":"windows7","Arch":"64Bit","Program":"some.exe","IP":"0.0.0.0"} {"operatingsys… --- ## [Logstash performance testing using logstash-performance-testing](https://discuss.elastic.co/t/logstash-performance-testing-using-logstash-performance-testing/31254)
**Author:** [@astro](https://discuss.elastic.co/u/astro)\ **Replies:** 0\ **Last updated:** [September 28, 2015, 11:42am UTC](https://discuss.elastic.co/t/logstash-performance-testing-using-logstash-performance-testing/31254 "2015-09-28T11:42:44Z")
Hi All, I came across a tool named logstash-performance-testing which can used to test our logstash configuration with respect to performance , how we can use this if we have not installed logstash rather we are using … --- ## [Local filter plugin pretended to be installed but not listed and not working](https://discuss.elastic.co/t/local-filter-plugin-pretended-to-be-installed-but-not-listed-and-not-working/2016)
**Author:** [@eravelo](https://discuss.elastic.co/u/eravelo)\ **Replies:** 3\ **Last updated:** [September 28, 2015, 8:50am UTC](https://discuss.elastic.co/t/local-filter-plugin-pretended-to-be-installed-but-not-listed-and-not-working/2016 "2015-09-28T08:50:28Z")
I tried writing a simple filter plugin, following this Howto: https://www.elastic.co/guide/en/logstash/current/\_how\_to\_write\_a\_logstash\_filter\_plugin.html When installing the plugin, it was reported to be installed: #… --- ## [Custom logging format](https://discuss.elastic.co/t/custom-logging-format/31151)
**Author:** [@deepak1](https://discuss.elastic.co/u/deepak1)\ **Replies:** 1\ **Last updated:** [September 28, 2015, 7:46am UTC](https://discuss.elastic.co/t/custom-logging-format/31151 "2015-09-28T07:46:12Z")
lets say i have a custom logging format like below:- format is:- year timestamp thread log\_level class\_name method\_name message 2015-09-24 12:06:37.051 \[catalina-exec-10\] DEBUG ActClientServerDAO - \[retSubsKeyInfo(St… --- ## [Logstash ElasticSearch IOException :](https://discuss.elastic.co/t/logstash-elasticsearch-ioexception/29664)
**Author:** [@BenjaminD](https://discuss.elastic.co/u/BenjaminD)\ **Replies:** 1\ **Last updated:** [September 28, 2015, 7:29am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-ioexception/29664 "2015-09-28T07:29:47Z")
Hi all, I'm trying to use Logstash and ElasticSearch for the first time and I get some troubles doing so. ES version : 1.7.2 (also tried with 2.0.0 beta2 for the same result). Logstash version : 1.5.4 The context :… --- ## [Logstash multiline codec not working with my pattern](https://discuss.elastic.co/t/logstash-multiline-codec-not-working-with-my-pattern/30085)
**Author:** [@Jigar\_Sheth](https://discuss.elastic.co/u/Jigar_Sheth)\ **Replies:** 1\ **Last updated:** [September 27, 2015, 6:39pm UTC](https://discuss.elastic.co/t/logstash-multiline-codec-not-working-with-my-pattern/30085 "2015-09-27T18:39:00Z")
I had been using the multline codec of logstash for my java exceptions. However, recently I wanted to capture more things and hence used another pattern. This causes my logstash not to read file even though I am using… --- ## [Input-jdbc : Error to connect to DB](https://discuss.elastic.co/t/input-jdbc-error-to-connect-to-db/29349)
**Author:** [@Kiranpatil](https://discuss.elastic.co/u/Kiranpatil)\ **Replies:** 2\ **Last updated:** [September 26, 2015, 7:44pm UTC](https://discuss.elastic.co/t/input-jdbc-error-to-connect-to-db/29349 "2015-09-26T19:44:05Z")
Hi, I am using jdbc input plugin (version : 1.0.0) to push the data from postgres to elastic. But I get an error as: Java::OrgPostgresqlUtil::PSQLException: The server requested password-based authentication, but n… --- ## [Installing "logstash-input-jdbc" I think I'm missing something](https://discuss.elastic.co/t/installing-logstash-input-jdbc-i-think-im-missing-something/29635)
**Author:** [@madCow](https://discuss.elastic.co/u/madCow)\ **Replies:** 1\ **Last updated:** [September 26, 2015, 7:41pm UTC](https://discuss.elastic.co/t/installing-logstash-input-jdbc-i-think-im-missing-something/29635 "2015-09-26T19:41:35Z")
Hi, I'm running logstash 1.4.2 and/or 1.5.0 on windows server 2008. I'm trying to use the logstash-input-jdbc plugin. Running the "bin/plugin install logstash-input-jdbc" leads me to believe that the plugin is not avai… --- ## [Too many open files?](https://discuss.elastic.co/t/too-many-open-files/29969)
**Author:** [@paulkeogh](https://discuss.elastic.co/u/paulkeogh)\ **Replies:** 1\ **Last updated:** [September 26, 2015, 7:38pm UTC](https://discuss.elastic.co/t/too-many-open-files/29969 "2015-09-26T19:38:09Z")
I am seeing errors in myLF log file around too many open files and observe that no events are subsequently forwarded after this error. I am processing files that are effectively closed - if I set the "dead time" variab… --- ## [Doing mathematical ops using logstash filters](https://discuss.elastic.co/t/doing-mathematical-ops-using-logstash-filters/30058)
**Author:** [@Udai\_Mehra](https://discuss.elastic.co/u/Udai_Mehra)\ **Replies:** 1\ **Last updated:** [September 26, 2015, 10:50am UTC](https://discuss.elastic.co/t/doing-mathematical-ops-using-logstash-filters/30058 "2015-09-26T10:50:20Z")
Hi, I am facing a weired issue and need help of experts. I am trying to do numerical (INT) comparison in logstash filter file. Basically, I want to register only the haproxy events ( from its log file ) which take above … --- ## [Lumberjack input needs to support specifying a certificate chain for ssl](https://discuss.elastic.co/t/lumberjack-input-needs-to-support-specifying-a-certificate-chain-for-ssl/30048)
**Author:** [@nneul](https://discuss.elastic.co/u/nneul)\ **Replies:** 1\ **Last updated:** [September 26, 2015, 9:25am UTC](https://discuss.elastic.co/t/lumberjack-input-needs-to-support-specifying-a-certificate-chain-for-ssl/30048 "2015-09-26T09:25:53Z")
You should allow specifying path to a certificate chain. Pointing ssl\_certificate at crt and intermediates does not work - it just grabs the first entry only. --- ## [Index not creating from couchdb with logstash](https://discuss.elastic.co/t/index-not-creating-from-couchdb-with-logstash/27848)
**Author:** [@imad](https://discuss.elastic.co/u/imad)\ **Replies:** 13\ **Last updated:** [September 25, 2015, 7:54pm UTC](https://discuss.elastic.co/t/index-not-creating-from-couchdb-with-logstash/27848 "2015-09-25T19:54:33Z")
I am using this config file to import data from couchdb. input { couchdb\_changes { db =\> "roles" host =\> "localhost" port =\> 5984 } } output { elasticsearch { document\_id =\> "%{… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1232) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1234)