# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=679 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 680 --- ## [Need help on logging solution](https://discuss.elastic.co/t/need-help-on-logging-solution/150004)
**Author:** [@anra137](https://discuss.elastic.co/u/anra137)\ **Replies:** 6\ **Last updated:** [September 27, 2018, 11:45am UTC](https://discuss.elastic.co/t/need-help-on-logging-solution/150004 "2018-09-27T11:45:16Z")
At present, we have ELK stack running on Ubuntu and filebeat on windows to collect logs from different applications running on it. If we want to send logs directly to logstash instead of using filebeat as data shipper, D… --- ## [Problem with my grok-need assistance](https://discuss.elastic.co/t/problem-with-my-grok-need-assistance/150163)
**Author:** [@Nithani25](https://discuss.elastic.co/u/Nithani25)\ **Replies:** 3\ **Last updated:** [September 27, 2018, 11:12am UTC](https://discuss.elastic.co/t/problem-with-my-grok-need-assistance/150163 "2018-09-27T11:12:11Z")
Hi Team, I am working on some simillar to weblogic trace logs and it has following inputs in combined . 2018-09-27 02:36:11,474 DEBUG \[\[ACTIVE\] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'\] Se… --- ## [Event enrichment using logstash](https://discuss.elastic.co/t/event-enrichment-using-logstash/150179)
**Author:** [@annk](https://discuss.elastic.co/u/annk)\ **Replies:** 1\ **Last updated:** [September 27, 2018, 11:02am UTC](https://discuss.elastic.co/t/event-enrichment-using-logstash/150179 "2018-09-27T11:02:16Z")
Hi, are there any plugins I can use to enrich my log entries , eg , querying other sources or files to get more data and create fields based on that ? Thanks --- ## [Best input file type for logstash](https://discuss.elastic.co/t/best-input-file-type-for-logstash/150157)
**Author:** [@Mohamed.Yousry](https://discuss.elastic.co/u/Mohamed.Yousry)\ **Replies:** 1\ **Last updated:** [September 27, 2018, 10:23am UTC](https://discuss.elastic.co/t/best-input-file-type-for-logstash/150157 "2018-09-27T10:23:12Z")
Hi I am new to elastic, and trying to make dashboard for logs, I want to what's the best type for file input (txt, CSV, or reading from DB ) --- ## [Logstash geo\_point](https://discuss.elastic.co/t/logstash-geo-point/149434)
**Author:** [@DhavalKansara](https://discuss.elastic.co/u/DhavalKansara)\ **Replies:** 15\ **Last updated:** [September 27, 2018, 8:34am UTC](https://discuss.elastic.co/t/logstash-geo-point/149434 "2018-09-27T08:34:01Z")
Hello, I want to create a geographical coordinate map. Can you please help me guide through the process of creating the geo\_point or any other way to create coordinate map. I have tried the option from some website. I r… --- ## [How to apply multiple grok patterns to single log file?](https://discuss.elastic.co/t/how-to-apply-multiple-grok-patterns-to-single-log-file/150128)
**Author:** [@pramodchoudhari](https://discuss.elastic.co/u/pramodchoudhari)\ **Replies:** 1\ **Last updated:** [September 27, 2018, 8:32am UTC](https://discuss.elastic.co/t/how-to-apply-multiple-grok-patterns-to-single-log-file/150128 "2018-09-27T08:32:48Z")
I want to apply two grok patterns to a single log file. This is my logstash.confd file :- input { beats { port =\> 5044 } } filter { if \[fields\]\[log\_type\] == "syslog" { grok { match =\> { "message" =\>… --- ## [Hypen "-" in grok fields](https://discuss.elastic.co/t/hypen-in-grok-fields/149981)
**Author:** [@ssi](https://discuss.elastic.co/u/ssi)\ **Replies:** 5\ **Last updated:** [September 27, 2018, 8:25am UTC](https://discuss.elastic.co/t/hypen-in-grok-fields/149981 "2018-09-27T08:25:55Z")
Hi im trying to apply the solution described here to the follwoing part of the Exchange message tracking logs source\_context MDB:04111c61-c212-4078-bf65-369a8cd3080c, Mailbox:257ff165-97e3-498b-8e11-b5b735da312b, Ev… --- ## [Logstash-filter-rest Client Error](https://discuss.elastic.co/t/logstash-filter-rest-client-error/150083)
**Author:** [@Sid\_Sathi](https://discuss.elastic.co/u/Sid_Sathi)\ **Replies:** 2\ **Last updated:** [September 27, 2018, 6:17am UTC](https://discuss.elastic.co/t/logstash-filter-rest-client-error/150083 "2018-09-27T06:17:40Z")
I installed logstash-filter-rest plugin in to make an post request to a webservice but I am getting the following error. \[ERROR\]\[logstash.filters.rest \] error in rest filter :client\_error=\>#\ **Author:** [@capsnumber1](https://discuss.elastic.co/u/capsnumber1)\ **Replies:** 3\ **Last updated:** [September 26, 2018, 9:05pm UTC](https://discuss.elastic.co/t/how-to-create-fields-from-data-in-json-object/150066 "2018-09-26T21:05:06Z") I have JSON that looks like this: "httpMessage":{ "method":"GET", "query":"v=1.020.107", "start":"1537967152", "path":"/themes/main/js/jscript.js", "protocol":"HTTP/1.1", "requestHeaders":"Host: www.example.com Re… --- ## [Logstash Could not index event to Elasticsearch, mapper\_parsing\_exception, illegal\_argument\_exception](https://discuss.elastic.co/t/logstash-could-not-index-event-to-elasticsearch-mapper-parsing-exception-illegal-argument-exception/149876)
**Author:** [@Simon\_Becker](https://discuss.elastic.co/u/Simon_Becker)\ **Replies:** 5\ **Last updated:** [September 26, 2018, 7:08pm UTC](https://discuss.elastic.co/t/logstash-could-not-index-event-to-elasticsearch-mapper-parsing-exception-illegal-argument-exception/149876 "2018-09-26T19:08:01Z")
Hi there, I try to get a JSON Formatted Datastream from Kafka processed by logstash towards Elasticsearch. My Json Stream has multiple Arrays filled with objects. All of the arrays except of one are working perfectly f… --- ## [Powershell the events stats logstash](https://discuss.elastic.co/t/powershell-the-events-stats-logstash/149823)
**Author:** [@cowensel](https://discuss.elastic.co/u/cowensel)\ **Replies:** 3\ **Last updated:** [September 26, 2018, 6:50pm UTC](https://discuss.elastic.co/t/powershell-the-events-stats-logstash/149823 "2018-09-26T18:50:22Z")
I wonder if you can help, Has anyone build a powershell that connects to the logstash monitoring API to get events in stats and then checks whether the events in is less than what it was x minutes ago and if it is send … --- ## [Logstash elasticsearch input re-running query infinately](https://discuss.elastic.co/t/logstash-elasticsearch-input-re-running-query-infinately/150038)
**Author:** [@JeremyinNC](https://discuss.elastic.co/u/JeremyinNC)\ **Replies:** 0\ **Last updated:** [September 26, 2018, 3:00pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-input-re-running-query-infinately/150038 "2018-09-26T15:00:39Z")
Config below, I expect this to run once at 2:55 PM but it's never stopping. I can see it's getting the right data but once it completes the query it just starts it back up again. This is using logstash 5.6.12. Any sugges… --- ## [Elapsed time from the start of reading from a db to the end of writing in ES](https://discuss.elastic.co/t/elapsed-time-from-the-start-of-reading-from-a-db-to-the-end-of-writing-in-es/150051)
**Author:** [@Nafiseh\_Faghidno](https://discuss.elastic.co/u/Nafiseh_Faghidno)\ **Replies:** 1\ **Last updated:** [September 26, 2018, 5:37pm UTC](https://discuss.elastic.co/t/elapsed-time-from-the-start-of-reading-from-a-db-to-the-end-of-writing-in-es/150051 "2018-09-26T17:37:13Z")
Hi, I have a logstash job (Version 2.2.4) which connects to db and writes the data in ES (once a day) This is the configuration input { jdbc { // DB and Query } } filter { ruby { //Modify the data } } output… --- ## [Logstash-output-mongodb plugin for Azure cosmos DB](https://discuss.elastic.co/t/logstash-output-mongodb-plugin-for-azure-cosmos-db/150060)
**Author:** [@jfreeman](https://discuss.elastic.co/u/jfreeman)\ **Replies:** 0\ **Last updated:** [September 26, 2018, 5:19pm UTC](https://discuss.elastic.co/t/logstash-output-mongodb-plugin-for-azure-cosmos-db/150060 "2018-09-26T17:19:29Z")
I have an Azure cosmos DB with MongoDB API support, its connection string is in the format of mongodb://username:password@URL. Logstash can push data to this DB using the community plugin logstash-output-documentdb but … --- ## [Logstash monitoring on kibana](https://discuss.elastic.co/t/logstash-monitoring-on-kibana/149901)
**Author:** [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Replies:** 3\ **Last updated:** [September 26, 2018, 4:06pm UTC](https://discuss.elastic.co/t/logstash-monitoring-on-kibana/149901 "2018-09-26T16:06:31Z")
I have following in config file but monitoring not getting display in kibana xpack.monitoring.enabled: true xpack.monitoring.elasticsearch.url: \["http://:9200", "https://:9200"\] xpack.monitoring.elasticsearch.sniffing… --- ## [Mulitple logstash to read data from single s3 bucket location](https://discuss.elastic.co/t/mulitple-logstash-to-read-data-from-single-s3-bucket-location/149678)
**Author:** [@satish\_kumar1](https://discuss.elastic.co/u/satish_kumar1)\ **Replies:** 1\ **Last updated:** [September 26, 2018, 2:10pm UTC](https://discuss.elastic.co/t/mulitple-logstash-to-read-data-from-single-s3-bucket-location/149678 "2018-09-26T14:10:55Z")
Hello All, I have a production elk cluster where i have setup a logstash node to read aws elb logs via s3-input-plugin from s3 bucket and than parse it and ingest it to Elasticsearch cluster. Last week there was a huge … --- ## [Logstash file path input get files with certain words](https://discuss.elastic.co/t/logstash-file-path-input-get-files-with-certain-words/149637)
**Author:** [@ash\_said](https://discuss.elastic.co/u/ash_said)\ **Replies:** 1\ **Last updated:** [September 26, 2018, 1:26pm UTC](https://discuss.elastic.co/t/logstash-file-path-input-get-files-with-certain-words/149637 "2018-09-26T13:26:53Z")
I am new to Logstash and am having trouble with inputting only certain files with specific file names I want Logstash to only input files with the file name for example, abc\_bigword\_1234-02.txt and not accept files like … --- ## [Resume logstash csv import](https://discuss.elastic.co/t/resume-logstash-csv-import/150002)
**Author:** [@andrew\_xoa](https://discuss.elastic.co/u/andrew_xoa)\ **Replies:** 1\ **Last updated:** [September 26, 2018, 1:17pm UTC](https://discuss.elastic.co/t/resume-logstash-csv-import/150002 "2018-09-26T13:17:58Z")
Hi, I have a csv file of about 10 GB that I'm importing into elastic via logstash. by mistake during the import the PC was restarted, I would like to know if it is possible to resume the import from the point where logs… --- ## [Parsing date format error](https://discuss.elastic.co/t/parsing-date-format-error/149446)
**Author:** [@Khaled\_Saidi](https://discuss.elastic.co/u/Khaled_Saidi)\ **Replies:** 2\ **Last updated:** [September 26, 2018, 12:48pm UTC](https://discuss.elastic.co/t/parsing-date-format-error/149446 "2018-09-26T12:48:36Z")
Hi everybody, I'm facing an issue that drive me crazy. I have a date field in my log file like : 2018-09-21T14:01:59.271245+02:00 In my logstash filter i tried a lot of pattern and the last one is: date { matc… --- ## [How do I create a DB query with variable parameters?](https://discuss.elastic.co/t/how-do-i-create-a-db-query-with-variable-parameters/149818)
**Author:** [@sversienti](https://discuss.elastic.co/u/sversienti)\ **Replies:** 4\ **Last updated:** [September 26, 2018, 12:27pm UTC](https://discuss.elastic.co/t/how-do-i-create-a-db-query-with-variable-parameters/149818 "2018-09-26T12:27:27Z")
Hi, I'd like to set up a parameterized query. I configured my logstash.conf to query my Oracle database. My table has a REF\_DATE field valued with a date CHAR(10 BYTE). I need to make the following query: \`SELECT COUN… --- ## [Mutate to concatenate two filed](https://discuss.elastic.co/t/mutate-to-concatenate-two-filed/149966)
**Author:** [@richagautam](https://discuss.elastic.co/u/richagautam)\ **Replies:** 1\ **Last updated:** [September 26, 2018, 12:18pm UTC](https://discuss.elastic.co/t/mutate-to-concatenate-two-filed/149966 "2018-09-26T12:18:52Z")
Hi Team, I am using multiple grok filter to capture specific patterns from the log file.Now i want to concatenate one of the filed of first grok filter with one filed of second grok filter. mutate { add\_field =\> … --- ## [Error: Address already in use](https://discuss.elastic.co/t/error-address-already-in-use/149985)
**Author:** [@BigAlbert](https://discuss.elastic.co/u/BigAlbert)\ **Replies:** 2\ **Last updated:** [September 26, 2018, 11:31am UTC](https://discuss.elastic.co/t/error-address-already-in-use/149985 "2018-09-26T11:31:02Z")
Hi I have installed logstash on debian container which been spunup using -link elasticsearch:elasticsearch. /config-dir/first-pipeline.conf: input { tcp { port =\> 8083 } } output { elasticsearch { hosts =\> \["… --- ## [BUG? - ISO8601 Timezone Parsing](https://discuss.elastic.co/t/bug-iso8601-timezone-parsing/149964)
**Author:** [@leeroy](https://discuss.elastic.co/u/leeroy)\ **Replies:** 0\ **Last updated:** [September 26, 2018, 9:06am UTC](https://discuss.elastic.co/t/bug-iso8601-timezone-parsing/149964 "2018-09-26T09:06:47Z")
I'm trying to parse the /var/log/install.log file on MacOS (I've checked 10.13.x and 10.14.0, they're the same syntax). Unfortunately it's lacking some consistency in date formats between various system logs, but that's … --- ## [How to use GROK pattern for JSON Object](https://discuss.elastic.co/t/how-to-use-grok-pattern-for-json-object/149959)
**Author:** [@Teja](https://discuss.elastic.co/u/Teja)\ **Replies:** 3\ **Last updated:** [September 26, 2018, 9:12am UTC](https://discuss.elastic.co/t/how-to-use-grok-pattern-for-json-object/149959 "2018-09-26T09:12:58Z")
Hi Folks, I want to apply grok for one JSON object\[field\] and extract values from the field. Sample Json Message: {"event.created":"1537941675000","isoTimeFormat":"yyyy-MM-dd'T'HH:mm:ss.SSSZ","event.payload":"dstintf=… --- ## [Logstash: parsing fields from json in array of another json](https://discuss.elastic.co/t/logstash-parsing-fields-from-json-in-array-of-another-json/149854)
**Author:** [@Jiri\_Petak](https://discuss.elastic.co/u/Jiri_Petak)\ **Replies:** 2\ **Last updated:** [September 26, 2018, 9:09am UTC](https://discuss.elastic.co/t/logstash-parsing-fields-from-json-in-array-of-another-json/149854 "2018-09-26T09:09:19Z")
Hi, i would like to parse some fields from json logs from AWS WAF. On input logstash use: s3 { bucket =\> "XXXX" access\_key\_id =\> "XXXX" secret\_access\_key =\> "XXXX" prefix =\> "waf/" region =\> "X… --- ## [The jdbc request is not executed](https://discuss.elastic.co/t/the-jdbc-request-is-not-executed/149851)
**Author:** [@Igorr](https://discuss.elastic.co/u/Igorr)\ **Replies:** 2\ **Last updated:** [September 26, 2018, 8:51am UTC](https://discuss.elastic.co/t/the-jdbc-request-is-not-executed/149851 "2018-09-26T08:51:24Z")
The jdbc request is not executed. Config in logstash: input { jdbc { jdbc\_driver\_library =\> "/etc/logstash/mysql-connector-java-5.1.46-bin.jar" jdbc\_driver\_class =\> "com.mysql.jdbc.Driver" jdbc\_connection\_string =\> … --- ## [Request log from python to build stats over user usage](https://discuss.elastic.co/t/request-log-from-python-to-build-stats-over-user-usage/148997)
**Author:** [@esseti](https://discuss.elastic.co/u/esseti)\ **Replies:** 4\ **Last updated:** [September 26, 2018, 8:34am UTC](https://discuss.elastic.co/t/request-log-from-python-to-build-stats-over-user-usage/148997 "2018-09-26T08:34:48Z")
Hi all, We have built a system that has to keep track of the number of calls a user does against our API, we also have to keep track of the logging info to build "audit" logs and give to the user the possibility to sear… --- ## [Logstash cannot parse the json data sent in filebeat](https://discuss.elastic.co/t/logstash-cannot-parse-the-json-data-sent-in-filebeat/149947)
**Author:** [@taoqin0203](https://discuss.elastic.co/u/taoqin0203)\ **Replies:** 0\ **Last updated:** [September 26, 2018, 8:03am UTC](https://discuss.elastic.co/t/logstash-cannot-parse-the-json-data-sent-in-filebeat/149947 "2018-09-26T08:03:55Z")
I went from the json log of my mobile phone in filebeat to logstash. After the data was received by logstash, I could not parse the json data into the corresponding fields. Here is my configuration and the data used. my… --- ## [Add\_field doesn't work](https://discuss.elastic.co/t/add-field-doesnt-work/149612)
**Author:** [@peterch](https://discuss.elastic.co/u/peterch)\ **Replies:** 12\ **Last updated:** [September 26, 2018, 7:39am UTC](https://discuss.elastic.co/t/add-field-doesnt-work/149612 "2018-09-26T07:39:07Z")
Dear all, I install metric beat on a window server to receive log. I tried to combine some of the field using add\_field but it cannot capture the old field value. Any idea? if "metricbeat" in \[tags\]{ mutate { add\_fi… --- ## [How can i maintain the sequence of log data while parsing it in logstash?](https://discuss.elastic.co/t/how-can-i-maintain-the-sequence-of-log-data-while-parsing-it-in-logstash/149625)
**Author:** [@richagautam](https://discuss.elastic.co/u/richagautam)\ **Replies:** 3\ **Last updated:** [September 26, 2018, 6:29am UTC](https://discuss.elastic.co/t/how-can-i-maintain-the-sequence-of-log-data-while-parsing-it-in-logstash/149625 "2018-09-26T06:29:07Z")
Hi Team, How can i maintain the sequence of log data while parsing it in logstash? My log data has a timestamp filed and some log lines have same timestamp, now while displaying data in kibana it is not coming in the … [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=678) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=680)