Just cannot seem to make progress with rsyslog and Logstash


(Marty Hillman) #1

I even bought the book and rebuilt my test environment servers from
scratch, but I still have the same issues.

On the central server, I have redis, logstash 1.4 and elasticsearch 1.3
installed - all from apt repositories. I verified that all services are
started and I can curl results from them, telnet to them from the client
server, etc. I can get to elasticsearch, redis, et al. Netstat is showing
that port 5514 is listening as configured for rsyslog messages within
logstash. I have configured rsyslog on the client to be as noisy as
possible and send all traffic to the IP address of the central server.
However, nothing ever shows up in logstash from the remote server even when
using logger. In fact, rsyslog on the localhost is not sending any
messages to logstash or redis. LLEN even shows zero items.

I have Googled everywhere and tried a multitude of suggestions including
rebuilding the entire setup in my virtual environment numerous times (and
yes, my virtual servers can see and talk to each other just fine). There
has to be something simple I am missing. Does anyone have any suggestions?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ad5fd6f7-1343-47e3-8bee-a209099c1f04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #2

You should ask this over on the logstash list -
https://groups.google.com/forum/?hl=en-GB#!forum/logstash-users :slight_smile:

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 17 September 2014 06:04, Marty Hillman antimidas@antimidas.net wrote:

I even bought the book and rebuilt my test environment servers from
scratch, but I still have the same issues.

On the central server, I have redis, logstash 1.4 and elasticsearch 1.3
installed - all from apt repositories. I verified that all services are
started and I can curl results from them, telnet to them from the client
server, etc. I can get to elasticsearch, redis, et al. Netstat is showing
that port 5514 is listening as configured for rsyslog messages within
logstash. I have configured rsyslog on the client to be as noisy as
possible and send all traffic to the IP address of the central server.
However, nothing ever shows up in logstash from the remote server even when
using logger. In fact, rsyslog on the localhost is not sending any
messages to logstash or redis. LLEN even shows zero items.

I have Googled everywhere and tried a multitude of suggestions including
rebuilding the entire setup in my virtual environment numerous times (and
yes, my virtual servers can see and talk to each other just fine). There
has to be something simple I am missing. Does anyone have any suggestions?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ad5fd6f7-1343-47e3-8bee-a209099c1f04%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ad5fd6f7-1343-47e3-8bee-a209099c1f04%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624atXoODAxeMEqj1%3D%2BgrFCcD3VvTcvMF-L76yH9ouTy4mw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Marty Hillman) #3

Thanks Mark. Thought this was that list. :slight_smile:

On Tuesday, September 16, 2014 3:08:45 PM UTC-5, Mark Walkom wrote:

You should ask this over on the logstash list -
https://groups.google.com/forum/?hl=en-GB#!forum/logstash-users :slight_smile:

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 17 September 2014 06:04, Marty Hillman <anti...@antimidas.net
<javascript:>> wrote:

I even bought the book and rebuilt my test environment servers from
scratch, but I still have the same issues.

On the central server, I have redis, logstash 1.4 and elasticsearch 1.3
installed - all from apt repositories. I verified that all services are
started and I can curl results from them, telnet to them from the client
server, etc. I can get to elasticsearch, redis, et al. Netstat is showing
that port 5514 is listening as configured for rsyslog messages within
logstash. I have configured rsyslog on the client to be as noisy as
possible and send all traffic to the IP address of the central server.
However, nothing ever shows up in logstash from the remote server even when
using logger. In fact, rsyslog on the localhost is not sending any
messages to logstash or redis. LLEN even shows zero items.

I have Googled everywhere and tried a multitude of suggestions including
rebuilding the entire setup in my virtual environment numerous times (and
yes, my virtual servers can see and talk to each other just fine). There
has to be something simple I am missing. Does anyone have any suggestions?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ad5fd6f7-1343-47e3-8bee-a209099c1f04%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ad5fd6f7-1343-47e3-8bee-a209099c1f04%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4788daf0-a7ec-4494-9210-aa25f9eee290%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(ZillaG) #4

@Marty_Hillman, there is a separate Logstash discussion on this forum.

You can look at this: https://techpunch.co.uk/development/how-to-ship-logs-with-rsyslog-and-logstash


(system) #5