# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=963 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 964 --- ## [Logstash Not Connecting to Elasticsearch](https://discuss.elastic.co/t/logstash-not-connecting-to-elasticsearch/88336)
**Author:** [@relder](https://discuss.elastic.co/u/relder)\ **Replies:** 9\ **Last updated:** [June 16, 2017, 4:05pm UTC](https://discuss.elastic.co/t/logstash-not-connecting-to-elasticsearch/88336 "2017-06-16T16:05:28Z")
Hello everyone, I am running into a problem when I try to output Logstash to Elasticsearch. Everything works well when I'm merely outputting to stdout, but once I try to output to Elasticsearch, I am hit with this warni… --- ## [Write every incoming data to separate files in S3](https://discuss.elastic.co/t/write-every-incoming-data-to-separate-files-in-s3/89745)
**Author:** [@Polocapt](https://discuss.elastic.co/u/Polocapt)\ **Replies:** 0\ **Last updated:** [June 16, 2017, 3:59pm UTC](https://discuss.elastic.co/t/write-every-incoming-data-to-separate-files-in-s3/89745 "2017-06-16T15:59:29Z")
Hi, I've been trying to put file in S3 using the logstash-output-plugin using Logstash 5.4.1. I could be wrong, which is why I'm asking this here. From what I've seen, your data will be "rotated" (pushed to S3) only wh… --- ## [Encrypt data output jdbc](https://discuss.elastic.co/t/encrypt-data-output-jdbc/89741)
**Author:** [@rocavalcante](https://discuss.elastic.co/u/rocavalcante)\ **Replies:** 0\ **Last updated:** [June 16, 2017, 3:44pm UTC](https://discuss.elastic.co/t/encrypt-data-output-jdbc/89741 "2017-06-16T15:44:55Z")
Hi Everyone! Is it possible to hide or encrypt the output data of jdbc, example username, password, database? connection\_string =\> 'jdbc:postgresql://localhost:5432/database?user=myuser&password=mypassword' Thank Yo… --- ## [Converting Un-Pivoted SQL Data to Nested Datatype using Logstash JDBC Input](https://discuss.elastic.co/t/converting-un-pivoted-sql-data-to-nested-datatype-using-logstash-jdbc-input/88301)
**Author:** [@Rory](https://discuss.elastic.co/u/Rory)\ **Replies:** 2\ **Last updated:** [June 16, 2017, 2:44pm UTC](https://discuss.elastic.co/t/converting-un-pivoted-sql-data-to-nested-datatype-using-logstash-jdbc-input/88301 "2017-06-16T14:44:58Z")
Hi, I have a table in SQL Server which looks like this: The data is in an un-pivoted format: CustomerID, AddressNumber, AddressValue 1, 1, a 1, 2, b 1, 3, c 1, 4, d 1, 5, e 2, 1, f 2, 2, g 2, 2, h 2, 2, i 2, 2, k .… --- ## [We are getting logstash 5.4 error](https://discuss.elastic.co/t/we-are-getting-logstash-5-4-error/88360)
**Author:** [@bvnages](https://discuss.elastic.co/u/bvnages)\ **Replies:** 13\ **Last updated:** [June 16, 2017, 2:36pm UTC](https://discuss.elastic.co/t/we-are-getting-logstash-5-4-error/88360 "2017-06-16T14:36:55Z")
\[2017-06-06T06:43:36,425\]\[FATAL\]\[logstash.runner \] Logstash could not be started because there is already another instance using the configured data directory. If you wish to run multiple instances, you must ch… --- ## [Central interface for managing/monitoring data shippers, logstash etc](https://discuss.elastic.co/t/central-interface-for-managing-monitoring-data-shippers-logstash-etc/89628)
**Author:** [@rahul.dhammy](https://discuss.elastic.co/u/rahul.dhammy)\ **Replies:** 2\ **Last updated:** [June 16, 2017, 2:33pm UTC](https://discuss.elastic.co/t/central-interface-for-managing-monitoring-data-shippers-logstash-etc/89628 "2017-06-16T14:33:00Z")
I have my filebeat running on different server boxes, I also have a logstash running on a different server. Is there a centeal management and monitoring UI by which I can monitor all my filebeats and logstash services. i… --- ## [Logstash split xml fields](https://discuss.elastic.co/t/logstash-split-xml-fields/89070)
**Author:** [@Makra](https://discuss.elastic.co/u/Makra)\ **Replies:** 21\ **Last updated:** [June 16, 2017, 1:45pm UTC](https://discuss.elastic.co/t/logstash-split-xml-fields/89070 "2017-06-16T13:45:12Z")
With reference to the xml file from https://discuss.elastic.co/t/logstash-split-event-from-an-xml-file-in-multiples-documents-keeping-information-from-root-tags/59801/9 given at the end I am trying to split the xml tags … --- ## [Logstash breaks when disabling certain X-pack features](https://discuss.elastic.co/t/logstash-breaks-when-disabling-certain-x-pack-features/89511)
**Author:** [@bjorn](https://discuss.elastic.co/u/bjorn)\ **Replies:** 3\ **Last updated:** [June 16, 2017, 1:44pm UTC](https://discuss.elastic.co/t/logstash-breaks-when-disabling-certain-x-pack-features/89511 "2017-06-16T13:44:43Z")
When installing x-pack on a fresh Ubuntu 17.04 install, (Logstash 5.4.1, Elasticsearch 5.4.1, Kibana 5.4.1 from Elastic repo), Logstash breaks when disabling monitoring (xpack.monitoring.enabled: false in elasticsearch.y… --- ## [\[Logstash 2.4.1\] Emails doesn't sent directly](https://discuss.elastic.co/t/logstash-2-4-1-emails-doesnt-sent-directly/89664)
**Author:** [@t.h](https://discuss.elastic.co/u/t.h)\ **Replies:** 1\ **Last updated:** [June 16, 2017, 12:59pm UTC](https://discuss.elastic.co/t/logstash-2-4-1-emails-doesnt-sent-directly/89664 "2017-06-16T12:59:51Z")
Hello, I am upgrade logstash from 1.5.4 to 2.4.1. I am not sure it is an new or old error. In my output section i have email section defined. I am simluated log-Messages in a while loop. I can see in this simulatio… --- ## [Logstash configuration file to execute a command?](https://discuss.elastic.co/t/logstash-configuration-file-to-execute-a-command/89680)
**Author:** [@cilzzz](https://discuss.elastic.co/u/cilzzz)\ **Replies:** 5\ **Last updated:** [June 16, 2017, 12:58pm UTC](https://discuss.elastic.co/t/logstash-configuration-file-to-execute-a-command/89680 "2017-06-16T12:58:41Z")
hey everyone, i need to configure logstash to execute the output of the command "ifconfig" in elastic search then to visualize on kibana here is my logstash.conf file: input { exec { command =\> "/sbin/ifconfig" in… --- ## [Logstash + logstash.service holdoff time over](https://discuss.elastic.co/t/logstash-logstash-service-holdoff-time-over/89713)
**Author:** [@Clement\_Ros](https://discuss.elastic.co/u/Clement_Ros)\ **Replies:** 0\ **Last updated:** [June 16, 2017, 12:26pm UTC](https://discuss.elastic.co/t/logstash-logstash-service-holdoff-time-over/89713 "2017-06-16T12:26:44Z")
Hi, I use Logstash 5.1. My Logstash restart alone. In /var/log/message i see : logstash: Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties systemd: logstash.service holdo… --- ## [Logstash Fails to insert the data in Elasticseach](https://discuss.elastic.co/t/logstash-fails-to-insert-the-data-in-elasticseach/89709)
**Author:** [@moni15moni](https://discuss.elastic.co/u/moni15moni)\ **Replies:** 3\ **Last updated:** [June 16, 2017, 12:21pm UTC](https://discuss.elastic.co/t/logstash-fails-to-insert-the-data-in-elasticseach/89709 "2017-06-16T12:21:26Z")
Hi While Logstash getting started with the following config elasticsearch { manage\_template =\> false codec =\> "json" hosts =\>\["10.4.1.70:9299"\] index … --- ## [Conflicting conf files](https://discuss.elastic.co/t/conflicting-conf-files/89608)
**Author:** [@Callahan](https://discuss.elastic.co/u/Callahan)\ **Replies:** 4\ **Last updated:** [June 16, 2017, 11:58am UTC](https://discuss.elastic.co/t/conflicting-conf-files/89608 "2017-06-16T11:58:30Z")
Hi, So I've discovered that my latest conf file I've written conflicts with an existing one I designed to take logs from my FW. I'm struggling to understand why having them both together in my confs folder should cause … --- ## [Field conflits ;-(](https://discuss.elastic.co/t/field-conflits/89562)
**Author:** [@pmohan](https://discuss.elastic.co/u/pmohan)\ **Replies:** 1\ **Last updated:** [June 16, 2017, 10:24am UTC](https://discuss.elastic.co/t/field-conflits/89562 "2017-06-16T10:24:47Z")
All of a sudden my filed have been marked a conflicts !!!! marked some as number and some as string ! Any idea what would have caused this ? Thanks, Prabu --- ## [Applying conditionals to grok filter](https://discuss.elastic.co/t/applying-conditionals-to-grok-filter/89662)
**Author:** [@vinod\_hy](https://discuss.elastic.co/u/vinod_hy)\ **Replies:** 2\ **Last updated:** [June 16, 2017, 10:13am UTC](https://discuss.elastic.co/t/applying-conditionals-to-grok-filter/89662 "2017-06-16T10:13:22Z")
Hi All, I have log in the below format, 2017-06-08 19:05:56.222 loglevel=DEBUG,sysId=467,package=com.MyPackage1,class=MyClass.java,method="sendMessage",Id=679,message=Hello First Message. Below is my filter, filter{ … --- ## [Logstash elasticsearch error with x pack](https://discuss.elastic.co/t/logstash-elasticsearch-error-with-x-pack/89556)
**Author:** [@raghvendra](https://discuss.elastic.co/u/raghvendra)\ **Replies:** 11\ **Last updated:** [June 16, 2017, 7:45am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-error-with-x-pack/89556 "2017-06-16T07:45:51Z")
Hi.. Can anyone help me in solving this error .. I have been trying to solve this for four days .. I am new to ELK stack so. I have installed x pack in my ELK stack . Elasticsearch and kibana is working fine but when I… --- ## [GeoIP logstash 5.3 with filebeat, elasticsearch and Kibana - geoip lookup failure](https://discuss.elastic.co/t/geoip-logstash-5-3-with-filebeat-elasticsearch-and-kibana-geoip-lookup-failure/89499)
**Author:** [@blackOcean](https://discuss.elastic.co/u/blackOcean)\ **Replies:** 8\ **Last updated:** [June 16, 2017, 7:37am UTC](https://discuss.elastic.co/t/geoip-logstash-5-3-with-filebeat-elasticsearch-and-kibana-geoip-lookup-failure/89499 "2017-06-16T07:37:48Z")
Using filebeat to send apache logs from Windows System and to my logstash server in linux EC2 and then to elastic search and Kibana. Elastic search and Kibana - 5.3 Logstash and filebeat - 5.3 filebeat.yml : fil… --- ## [Error string not present and S3 input plugin restarts](https://discuss.elastic.co/t/error-string-not-present-and-s3-input-plugin-restarts/89659)
**Author:** [@mukesh2802](https://discuss.elastic.co/u/mukesh2802)\ **Replies:** 0\ **Last updated:** [June 16, 2017, 7:24am UTC](https://discuss.elastic.co/t/error-string-not-present-and-s3-input-plugin-restarts/89659 "2017-06-16T07:24:36Z")
Infrastructure details , Running logstash 5.4.1 in a docker container. Following is my logstash.conf input { http { type =\> 'webhook' } s3 { bucket =\> "abcd" region =\> "xyz" prefix =\> "1234" exclude\_pattern… --- ## [What Would be the filetype for our custom patterns in ./pattern directory](https://discuss.elastic.co/t/what-would-be-the-filetype-for-our-custom-patterns-in-pattern-directory/88029)
**Author:** [@yash\_mangla](https://discuss.elastic.co/u/yash_mangla)\ **Replies:** 8\ **Last updated:** [June 16, 2017, 7:22am UTC](https://discuss.elastic.co/t/what-would-be-the-filetype-for-our-custom-patterns-in-pattern-directory/88029 "2017-06-16T07:22:23Z")
Hi guys, i read document of logstash where document said you can define your custom pattern in ./pattern directory. but when i download latest logstash then There is not any ./pattern directory available in logstas… --- ## [Gelfd failed to parse a message](https://discuss.elastic.co/t/gelfd-failed-to-parse-a-message/89462)
**Author:** [@stecino](https://discuss.elastic.co/u/stecino)\ **Replies:** 3\ **Last updated:** [June 16, 2017, 5:24am UTC](https://discuss.elastic.co/t/gelfd-failed-to-parse-a-message/89462 "2017-06-16T05:24:21Z")
Hello, I have a gelf input gelf { host =\> "0.0.0.0" port =\> 555 type =\> "gelf" codec =\> "json" } } I am getting this error: \[2017-06-14T20:53:03,518\]\[WARN \]\[logstash.input… --- ## [Output email plugin](https://discuss.elastic.co/t/output-email-plugin/89580)
**Author:** [@baha1](https://discuss.elastic.co/u/baha1)\ **Replies:** 8\ **Last updated:** [June 15, 2017, 9:32pm UTC](https://discuss.elastic.co/t/output-email-plugin/89580 "2017-06-15T21:32:42Z")
Hi there i want to use email plugin.i get this error: Cannot create pipeline {:reason=\>"Couldn't find any output plugin named 'email' this is pipeline.conf: output { if "ERROR" in \[LEVEL\] { elasticsearch { hosts… --- ## [Geoip: How to map any index to a template?](https://discuss.elastic.co/t/geoip-how-to-map-any-index-to-a-template/89612)
**Author:** [@DrPhlux](https://discuss.elastic.co/u/DrPhlux)\ **Replies:** 1\ **Last updated:** [June 15, 2017, 9:04pm UTC](https://discuss.elastic.co/t/geoip-how-to-map-any-index-to-a-template/89612 "2017-06-15T21:04:44Z")
Hello. I'm trying to "convert" the geoip.location field from a number to a "geo\_point". My index from my test servers is called "test". When I get this to work, I need to map index: "prod" to the same template. My … --- ## [Exec input filter in logstash not capturing entire output from python script](https://discuss.elastic.co/t/exec-input-filter-in-logstash-not-capturing-entire-output-from-python-script/89626)
**Author:** [@Jaspreet\_Singh](https://discuss.elastic.co/u/Jaspreet_Singh)\ **Replies:** 3\ **Last updated:** [June 15, 2017, 8:58pm UTC](https://discuss.elastic.co/t/exec-input-filter-in-logstash-not-capturing-entire-output-from-python-script/89626 "2017-06-15T20:58:27Z")
Here is what my configuration looks like ... input { exec { command =\> 'c:\\Python27\\python C:\\J\\logstash\\pythonscripts\\test.py' interval =\> 10000 } } output { stdout { codec =\> rubydebug } } When I execute… --- ## [How logstash uploads SQL data](https://discuss.elastic.co/t/how-logstash-uploads-sql-data/89609)
**Author:** [@Lian\_Jiang](https://discuss.elastic.co/u/Lian_Jiang)\ **Replies:** 3\ **Last updated:** [June 15, 2017, 8:48pm UTC](https://discuss.elastic.co/t/how-logstash-uploads-sql-data/89609 "2017-06-15T20:48:14Z")
I am trying to understand how logstash design sql data ingestion. For example, logstash can compare the key of a record to avoid ingesting duplicate records. It can use version to update an existing record. Does this… --- ## [Index Name pattern](https://discuss.elastic.co/t/index-name-pattern/89623)
**Author:** [@rahul.dhammy](https://discuss.elastic.co/u/rahul.dhammy)\ **Replies:** 3\ **Last updated:** [June 15, 2017, 8:30pm UTC](https://discuss.elastic.co/t/index-name-pattern/89623 "2017-06-15T20:30:13Z")
I am trying to give a name to my index, surprisingly the supported pattern is not working as well. When I give the name in format appName-%{+YYYY.MM} - This works When I try to append the week of the year by using a… --- ## [Issue parsing date](https://discuss.elastic.co/t/issue-parsing-date/89569)
**Author:** [@Peter\_Dwyer](https://discuss.elastic.co/u/Peter_Dwyer)\ **Replies:** 2\ **Last updated:** [June 15, 2017, 7:40pm UTC](https://discuss.elastic.co/t/issue-parsing-date/89569 "2017-06-15T19:40:20Z")
i'm having trouble figuring out what is wrong with my date filter. "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse \[datetime\]", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"In… --- ## [LogStash for .Net core applications](https://discuss.elastic.co/t/logstash-for-net-core-applications/89471)
**Author:** [@sslv](https://discuss.elastic.co/u/sslv)\ **Replies:** 3\ **Last updated:** [June 15, 2017, 4:50pm UTC](https://discuss.elastic.co/t/logstash-for-net-core-applications/89471 "2017-06-15T16:50:53Z")
Hi, Everyone I am a newbie to Logstash. I found it very interesting. But currently I am dealing with logging related to .netcore application. Am I right in using Log stash for my logging needs? I am confused after seein… --- ## [Mapping fields from logstash](https://discuss.elastic.co/t/mapping-fields-from-logstash/89251)
**Author:** [@akuznetsov](https://discuss.elastic.co/u/akuznetsov)\ **Replies:** 2\ **Last updated:** [June 15, 2017, 3:53pm UTC](https://discuss.elastic.co/t/mapping-fields-from-logstash/89251 "2017-06-15T15:53:33Z")
hello, colleagues, We got a problem with logstash events. we can't save it in elasticsearch when we use ruby filter version elasticsearch is 5.2.2 logstash config ... filter { json { source … --- ## [%{\[@metadata\]\[beat\]} style index in elastic with snmptrap input](https://discuss.elastic.co/t/metadata-beat-style-index-in-elastic-with-snmptrap-input/89579)
**Author:** [@lcorsini](https://discuss.elastic.co/u/lcorsini)\ **Replies:** 1\ **Last updated:** [June 15, 2017, 3:24pm UTC](https://discuss.elastic.co/t/metadata-beat-style-index-in-elastic-with-snmptrap-input/89579 "2017-06-15T15:24:51Z")
Hi, I'm experimenting with logstash and the snmptrap input. The first problem I'm encountering is how to send them to elasticsearch right now I'm using filebeat and icingabeat so, my elasticsearch output creates differen… --- ## [Discover: Field data loading is forbidden on \[loggingTime\]](https://discuss.elastic.co/t/discover-field-data-loading-is-forbidden-on-loggingtime/89452)
**Author:** [@Honda\_fred\_elk](https://discuss.elastic.co/u/Honda_fred_elk)\ **Replies:** 1\ **Last updated:** [June 15, 2017, 2:21pm UTC](https://discuss.elastic.co/t/discover-field-data-loading-is-forbidden-on-loggingtime/89452 "2017-06-15T14:21:19Z")
Hello, I use Filebeat to export JSON String to --\> logstash --\> elasticsearch --\> Kibana My application's log4j creates each log as JSON String like below.. it has loggingTime key & pair which is actual timestamp of… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=962) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=964)