# Logstash **URL:** https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1098 [Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md) **Page:** 1099 --- ## [Logstash sql\_last\_start with sql server](https://discuss.elastic.co/t/logstash-sql-last-start-with-sql-server/61380)
**Author:** [@mhensen](https://discuss.elastic.co/u/mhensen)\ **Replies:** 1\ **Last updated:** [September 26, 2016, 7:55pm UTC](https://discuss.elastic.co/t/logstash-sql-last-start-with-sql-server/61380 "2016-09-26T19:55:56Z")
I am running in a little problem..I want to have logstash picking up changes of multiple datatables Therefor I have created multiple config files and did an initial filling without the :sql\_last\_start This works ok.. a… --- ## [Syslog configuration output](https://discuss.elastic.co/t/syslog-configuration-output/61522)
**Author:** [@ruzzetto](https://discuss.elastic.co/u/ruzzetto)\ **Replies:** 1\ **Last updated:** [September 26, 2016, 7:43pm UTC](https://discuss.elastic.co/t/syslog-configuration-output/61522 "2016-09-26T19:43:09Z")
Hi All, i'm a newbie of elastic world and i'm trying to grab some remote syslog logs to visualize them on kibana. I setup logstash with a configuration file like this: input { tcp { port =\> 514 type =\> syslog … --- ## [Csv file no reading?](https://discuss.elastic.co/t/csv-file-no-reading/61064)
**Author:** [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\ **Replies:** 5\ **Last updated:** [September 25, 2016, 7:17pm UTC](https://discuss.elastic.co/t/csv-file-no-reading/61064 "2016-09-25T19:17:45Z")
i am trying to read cvs file. i see similar post but see no definite answer. is it the pointer issue that cvs file does not have pointer? If so, logstash doesn't work for static cvs file? my config below: input { … --- ## [Replicate all changes from one cluster to another](https://discuss.elastic.co/t/replicate-all-changes-from-one-cluster-to-another/61519)
**Author:** [@alienmind](https://discuss.elastic.co/u/alienmind)\ **Replies:** 1\ **Last updated:** [September 26, 2016, 3:52pm UTC](https://discuss.elastic.co/t/replicate-all-changes-from-one-cluster-to-another/61519 "2016-09-26T15:52:17Z")
I intend to schedule replication from one cluster into a "backup cluster". I've considered using the snapshot API (but it's not allowed to backup from ES to another ES, going through FS, S3 or HDFS are the only options… --- ## [How to write a json values from a field in sql into elasticsearch using logstash](https://discuss.elastic.co/t/how-to-write-a-json-values-from-a-field-in-sql-into-elasticsearch-using-logstash/61514)
**Author:** [@saisn](https://discuss.elastic.co/u/saisn)\ **Replies:** 3\ **Last updated:** [September 26, 2016, 2:35pm UTC](https://discuss.elastic.co/t/how-to-write-a-json-values-from-a-field-in-sql-into-elasticsearch-using-logstash/61514 "2016-09-26T14:35:04Z")
I have a table in sql server which has json objects as a value in one of the columns. My requirement is to read those objects as it is and write into an index. I would like to know what are the best ways of doing it usin… --- ## [Unable to start logstash via conf file in windows](https://discuss.elastic.co/t/unable-to-start-logstash-via-conf-file-in-windows/61500)
**Author:** [@bhat64](https://discuss.elastic.co/u/bhat64)\ **Replies:** 3\ **Last updated:** [September 26, 2016, 1:28pm UTC](https://discuss.elastic.co/t/unable-to-start-logstash-via-conf-file-in-windows/61500 "2016-09-26T13:28:45Z")
I am new to ELK. Installed ELK on Windows 7 desktop. Ensured i set the JAVA\_HOME as per the guidelines. ELK services are running, i used nssm. I have a simple logstash.conf file to read tcp as below: input { tcp { por… --- ## [Use log timestamp](https://discuss.elastic.co/t/use-log-timestamp/61411)
**Author:** [@Roberto\_Iglesias](https://discuss.elastic.co/u/Roberto_Iglesias)\ **Replies:** 5\ **Last updated:** [September 26, 2016, 1:19pm UTC](https://discuss.elastic.co/t/use-log-timestamp/61411 "2016-09-26T13:19:55Z")
Hi I'm trying to use my log's timestamp but this is not working. My filter: date { match =\> \["realtimestamp", "MMM DD HH:mm:ss"\] target =\> "@timestamp" } And the result: "message" =\> "Sep 18 05:59:24" "@tim… --- ## [Conditional filter in logstash](https://discuss.elastic.co/t/conditional-filter-in-logstash/61120)
**Author:** [@Roberto\_Iglesias](https://discuss.elastic.co/u/Roberto_Iglesias)\ **Replies:** 3\ **Last updated:** [September 26, 2016, 12:57pm UTC](https://discuss.elastic.co/t/conditional-filter-in-logstash/61120 "2016-09-26T12:57:16Z")
Hi and thanks for reading, i really appreciate that. I continue having troubles with logstash filters. In this case, i don't know how to handle conditionals with arrays. I have this field "amavis\_action" and its content… --- ## [Request help with a grok filter pattern for date](https://discuss.elastic.co/t/request-help-with-a-grok-filter-pattern-for-date/60997)
**Author:** [@RaviSidhu](https://discuss.elastic.co/u/RaviSidhu)\ **Replies:** 2\ **Last updated:** [September 26, 2016, 10:55am UTC](https://discuss.elastic.co/t/request-help-with-a-grok-filter-pattern-for-date/60997 "2016-09-26T10:55:51Z")
Hi All, I have a csv log file with column names and a sample record as access\_date,clientIP,Col1, Col2, Col3, Col4\_uri\_path, col5\_referrer,col6\_response, col7\_hit\_count,col8\_referrer '13/Jan/2016:22:57:38','111.22… --- ## [ES and logstash randomly start throwing exception: failed to parse \[timestamp\]](https://discuss.elastic.co/t/es-and-logstash-randomly-start-throwing-exception-failed-to-parse-timestamp/44968)
**Author:** [@varh1i](https://discuss.elastic.co/u/varh1i)\ **Replies:** 1\ **Last updated:** [September 26, 2016, 10:35am UTC](https://discuss.elastic.co/t/es-and-logstash-randomly-start-throwing-exception-failed-to-parse-timestamp/44968 "2016-09-26T10:35:11Z")
I have an ELK stack running and at some point it starts throwing: "IllegalArgumentException\[Invalid format: "Mar 20 02:00:06"\];" After restarting logstash it works fine again. When I try to parse the line that gave the e… --- ## [Elapsed logstash plugin](https://discuss.elastic.co/t/elapsed-logstash-plugin/61490)
**Author:** [@il.bert](https://discuss.elastic.co/u/il.bert)\ **Replies:** 0\ **Last updated:** [September 26, 2016, 8:24am UTC](https://discuss.elastic.co/t/elapsed-logstash-plugin/61490 "2016-09-26T08:24:45Z")
Hi, I am trying to install elapsed plugin on logstash 5 but it is not working, the error message says it is not compatible with version 5 but only 2.x is it planned to do so? will it be upgraded to LS 5? thanks --- ## [Logstash 2.0 stop working when I modify system time](https://discuss.elastic.co/t/logstash-2-0-stop-working-when-i-modify-system-time/61340)
**Author:** [@xiejieyi2006](https://discuss.elastic.co/u/xiejieyi2006)\ **Replies:** 2\ **Last updated:** [September 26, 2016, 6:23am UTC](https://discuss.elastic.co/t/logstash-2-0-stop-working-when-i-modify-system-time/61340 "2016-09-26T06:23:35Z")
My logstash version is 2.0.0,working in Suse 11SP3。 when I use command 'date -s "20170923 07:51:09' change system time to year 2024, logstash agent is working fine. But if I use 'date -s "20160923 07:51:09' change time… --- ## [How to change @timestamp field in kibana with timestamp of my logs time](https://discuss.elastic.co/t/how-to-change-timestamp-field-in-kibana-with-timestamp-of-my-logs-time/61476)
**Author:** [@arunmca18](https://discuss.elastic.co/u/arunmca18)\ **Replies:** 1\ **Last updated:** [September 26, 2016, 5:53am UTC](https://discuss.elastic.co/t/how-to-change-timestamp-field-in-kibana-with-timestamp-of-my-logs-time/61476 "2016-09-26T05:53:27Z")
My Log sample is : 2016-09-26 09:27:18,091 \[\[ACTIVE\] ExecuteThread: '19' for queue: 'weblogic.kernel.Default (self-tuning)'\] AUDIT com.rockon.intranet.IRTD.resource.IRTDdoplanAPI - (Request\_Timestamp,2016-09-26 09:27… --- ## [Can logstash get the type of field?](https://discuss.elastic.co/t/can-logstash-get-the-type-of-field/61326)
**Author:** [@WangXiangUSTC](https://discuss.elastic.co/u/WangXiangUSTC)\ **Replies:** 6\ **Last updated:** [September 26, 2016, 5:27am UTC](https://discuss.elastic.co/t/can-logstash-get-the-type-of-field/61326 "2016-09-26T05:27:07Z")
I there any way can logstash get the type of field? for example, "age": 1, "name":"jone' the age's type is integer, and the jone's type is string. --- ## [Logstash 5.0.0beta1 never goes to background](https://discuss.elastic.co/t/logstash-5-0-0beta1-never-goes-to-background/61433)
**Author:** [@himbeere](https://discuss.elastic.co/u/himbeere)\ **Replies:** 5\ **Last updated:** [September 25, 2016, 8:32pm UTC](https://discuss.elastic.co/t/logstash-5-0-0beta1-never-goes-to-background/61433 "2016-09-25T20:32:04Z")
Hello. I'm trying to persuade logstash 5 to run in the background but i can't do it. My start command is: bin/logstash -f /etc/logstash/conf.d/ -l /var/log/logstash/ --log.level info -e "" Logstash starts fine but… --- ## [Host variable replaced, but not configured to be](https://discuss.elastic.co/t/host-variable-replaced-but-not-configured-to-be/61401)
**Author:** [@jdmac](https://discuss.elastic.co/u/jdmac)\ **Replies:** 3\ **Last updated:** [September 25, 2016, 7:56pm UTC](https://discuss.elastic.co/t/host-variable-replaced-but-not-configured-to-be/61401 "2016-09-25T19:56:23Z")
Hi All, TLDR; - I suspect http { } is modifying the received host field value, without being explicitly configured to. I need to stop this happening. I'm using the following configuration file. input { … --- ## [Problem processing squid logs with logstash](https://discuss.elastic.co/t/problem-processing-squid-logs-with-logstash/61137)
**Author:** [@ageldenberg](https://discuss.elastic.co/u/ageldenberg)\ **Replies:** 1\ **Last updated:** [September 25, 2016, 7:54pm UTC](https://discuss.elastic.co/t/problem-processing-squid-logs-with-logstash/61137 "2016-09-25T19:54:38Z")
Hello, everyone. I am breaking my head over this issue. Logstash configuration has the following grok statement: %{MONTH}\\s+%{MONTHDAY}\\s+%{TIME}\\s+%{IPORHOST:src\_proxy}?\\s+\\S+\\s+%{BASE16FLOAT:timestamp}\\s+%{NUMBER… --- ## [Logstash forwarder cant TLS handshake](https://discuss.elastic.co/t/logstash-forwarder-cant-tls-handshake/61312)
**Author:** [@xon](https://discuss.elastic.co/u/xon)\ **Replies:** 1\ **Last updated:** [September 25, 2016, 7:26pm UTC](https://discuss.elastic.co/t/logstash-forwarder-cant-tls-handshake/61312 "2016-09-25T19:26:28Z")
Hi there, I am new to ELK. I want to setup an OSSEC monitoring solution (potentially by adding also other sensors such as splunk/etc) but it looks it is a bit complicated... I have followed several links about this err… --- ## [Date formatting issue](https://discuss.elastic.co/t/date-formatting-issue/61166)
**Author:** [@Kyle\_Hanson](https://discuss.elastic.co/u/Kyle_Hanson)\ **Replies:** 1\ **Last updated:** [September 25, 2016, 7:25pm UTC](https://discuss.elastic.co/t/date-formatting-issue/61166 "2016-09-25T19:25:07Z")
I'm stuck trying to get my Ingres log into ElasticSearch using logstash. Specifically I want to use the date in my logs to be the @timestamp. Here is sample log data I'm trying to process: ::\[ingres … --- ## [Issue with AWS SES email communication](https://discuss.elastic.co/t/issue-with-aws-ses-email-communication/61129)
**Author:** [@111130](https://discuss.elastic.co/u/111130)\ **Replies:** 1\ **Last updated:** [September 25, 2016, 7:22pm UTC](https://discuss.elastic.co/t/issue-with-aws-ses-email-communication/61129 "2016-09-25T19:22:51Z")
Hi! Recently I've faced, that my configuration stopped to work: email{ to =\> "my@email.com" from =\> "varyfied\_email@example.com" subject =\> "Some subject" body =\> "%{message}" via =\> "smtp… --- ## [How to copy (or rename) a complex field into another field?](https://discuss.elastic.co/t/how-to-copy-or-rename-a-complex-field-into-another-field/61015)
**Author:** [@vandamo](https://discuss.elastic.co/u/vandamo)\ **Replies:** 1\ **Last updated:** [September 25, 2016, 7:18pm UTC](https://discuss.elastic.co/t/how-to-copy-or-rename-a-complex-field-into-another-field/61015 "2016-09-25T19:18:26Z")
Hi, I'd like to keep all the information from rabbitmq\_properties that are stored in \[@metadata\]\[rabbitmq\_properties\]. To achieve that, I wanted to copy the complex element into another field (not under @metadata), but… --- ## [Logstash into another different cluster?](https://discuss.elastic.co/t/logstash-into-another-different-cluster/60975)
**Author:** [@Allie\_Yang](https://discuss.elastic.co/u/Allie_Yang)\ **Replies:** 7\ **Last updated:** [September 25, 2016, 7:13pm UTC](https://discuss.elastic.co/t/logstash-into-another-different-cluster/60975 "2016-09-25T19:13:56Z")
Hi How could i export data from logstash to another ES cluster, which i never used before? It is just i need to logstash the data from my local logstash to my team's ES Cluster, rather than my own test ES cluster... I … --- ## [Logstash 5.0.0 beta: translate filter](https://discuss.elastic.co/t/logstash-5-0-0-beta-translate-filter/61439)
**Author:** [@michelmooren](https://discuss.elastic.co/u/michelmooren)\ **Replies:** 5\ **Last updated:** [September 25, 2016, 7:13pm UTC](https://discuss.elastic.co/t/logstash-5-0-0-beta-translate-filter/61439 "2016-09-25T19:13:41Z")
I'm having trouble installing the Translate filter. Since this is a community plugin: Is, and if not, will Translate be supported in Logstash 5.0.0 (beta)? --- ## [Object to string gets indexed as object in elasticsearch](https://discuss.elastic.co/t/object-to-string-gets-indexed-as-object-in-elasticsearch/61282)
**Author:** [@derEremit](https://discuss.elastic.co/u/derEremit)\ **Replies:** 2\ **Last updated:** [September 25, 2016, 6:34pm UTC](https://discuss.elastic.co/t/object-to-string-gets-indexed-as-object-in-elasticsearch/61282 "2016-09-25T18:34:17Z")
I need to convert a field to string so it does not create multiple fields and subfields in elasticsearch. Reason is I get logs from a source I can't control and they are in the format data: { 23423432\_23434343: { .… --- ## [Replacing the very last comma in a csv file](https://discuss.elastic.co/t/replacing-the-very-last-comma-in-a-csv-file/61444)
**Author:** [@Logstash-lover](https://discuss.elastic.co/u/Logstash-lover)\ **Replies:** 1\ **Last updated:** [September 25, 2016, 6:17am UTC](https://discuss.elastic.co/t/replacing-the-very-last-comma-in-a-csv-file/61444 "2016-09-25T06:17:19Z")
Im processing various .csv files thru logstash using windows. I noticed if the csv file contains a comma at the very end of the very last line, logstash seems to pause on this .csv file and not proceed to process the csv… --- ## [Logstash Filter Rspec testing](https://discuss.elastic.co/t/logstash-filter-rspec-testing/61452)
**Author:** [@ahahtyler](https://discuss.elastic.co/u/ahahtyler)\ **Replies:** 1\ **Last updated:** [September 25, 2016, 3:50am UTC](https://discuss.elastic.co/t/logstash-filter-rspec-testing/61452 "2016-09-25T03:50:08Z")
Hey all, I'm in the process of creating a logstash filter for CloudFoundry. I want to put a suite of rspec test together, however I'm having an increasingly difficult time. Here's my current issue: when I execute t… --- ## [Logstash s3 Input - Filter by date](https://discuss.elastic.co/t/logstash-s3-input-filter-by-date/61426)
**Author:** [@piyush](https://discuss.elastic.co/u/piyush)\ **Replies:** 0\ **Last updated:** [September 23, 2016, 10:29pm UTC](https://discuss.elastic.co/t/logstash-s3-input-filter-by-date/61426 "2016-09-23T22:29:58Z")
Hi Team, I made some changes in environment (nothing on logstash.conf) primarily for shield and everything is working fine, beats are sending logs and dashboard is also showing correct data. But S3 input is trying to … --- ## [Logstash is re-importing entire logfile instead of just new records](https://discuss.elastic.co/t/logstash-is-re-importing-entire-logfile-instead-of-just-new-records/61386)
**Author:** [@Kyle\_Hanson](https://discuss.elastic.co/u/Kyle_Hanson)\ **Replies:** 5\ **Last updated:** [September 23, 2016, 3:08pm UTC](https://discuss.elastic.co/t/logstash-is-re-importing-entire-logfile-instead-of-just-new-records/61386 "2016-09-23T15:08:20Z")
I have my configuration to extract my date/time out of each log record and it's going into the @timestamp successfully. I ran logstash against my log file and everything imported fine to elasticsearch. Now I've added… --- ## [Number of batch workers](https://discuss.elastic.co/t/number-of-batch-workers/60950)
**Author:** [@YuWatanabe](https://discuss.elastic.co/u/YuWatanabe)\ **Replies:** 1\ **Last updated:** [September 23, 2016, 1:50pm UTC](https://discuss.elastic.co/t/number-of-batch-workers/60950 "2016-09-23T13:50:02Z")
I have a platform where filebeat , logstash , elasticsearch , kibana is all installed in one server. In the document there is a sentence explaining about the number of workers. The number of workers may be set highe… --- ## [GeoIP error after logstash 2.3.4 -\> 2.4.0 update](https://discuss.elastic.co/t/geoip-error-after-logstash-2-3-4-2-4-0-update/61220)
**Author:** [@ppascher](https://discuss.elastic.co/u/ppascher)\ **Replies:** 7\ **Last updated:** [September 23, 2016, 1:32pm UTC](https://discuss.elastic.co/t/geoip-error-after-logstash-2-3-4-2-4-0-update/61220 "2016-09-23T13:32:52Z")
Hello, yesterday I updated Logstash from 2.3.4 to 2.4.0 (as well as kibana 4.5 -\> 4.6) and ever since I noticed GeoIP data is not added anymore. logstash.log shows the following: :message=\>"Unknown error while lookin… [Previous page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1097) [Next page](https://discuss.elastic.co/c/elastic-stack/logstash/14.md?page=1099)