Looking at ES logging


(Ashwin Sathya) #1

Hi,

I have configured logging.yml to TRACE level messages and this level is
pretty verbose. I run a 3-node VM cluster, is there an easier way that I can
look at the logs other than manually remoting into the machine and reading
the file ?

Thanks,

Ashwin Sathya

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Lukáš Vlček) #2

http://logstash.net ?

On Fri, Sep 13, 2013 at 8:51 AM, R Ashwin Sathya
ashwin.sathya@outlook.comwrote:

Hi,****


I have configured logging.yml to TRACE level messages and this level is
pretty verbose. I run a 3-node VM cluster, is there an easier way that I
can look at the logs other than manually remoting into the machine and
reading the file ?****


Thanks,****

Ashwin Sathya****

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ashwin Sathya) #3

Logstash seems to enable output to elasticsearch. Does it mean I can parse
the elasticsearch logs using logstash and store them as an index in the same
ES cluster ? That would be awesome!!!

Thanks,

Ashwin Sathya

From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com]
On Behalf Of Lukáš Vlcek
Sent: September 13, 2013 12:26
To: elasticsearch@googlegroups.com
Subject: Re: Looking at ES logging

http://logstash.net ?

On Fri, Sep 13, 2013 at 8:51 AM, R Ashwin Sathya <ashwin.sathya@outlook.com
mailto:ashwin.sathya@outlook.com > wrote:

Hi,

I have configured logging.yml to TRACE level messages and this level is
pretty verbose. I run a 3-node VM cluster, is there an easier way that I can
look at the logs other than manually remoting into the machine and reading
the file ?

Thanks,

Ashwin Sathya

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com .
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com
mailto:elasticsearch+unsubscribe@googlegroups.com .
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #4

Yes!

I did it somehow like this while working on logs sent by one of our customers.
It was really great to build my dashboard very quick in Kibana and find easily what I was looking for.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 13 sept. 2013 à 09:07, R Ashwin Sathya ashwin.sathya@outlook.com a écrit :

Logstash seems to enable output to elasticsearch. Does it mean I can parse the elasticsearch logs using logstash and store them as an index in the same ES cluster ? That would be awesome!!!

Thanks,
Ashwin Sathya

From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On Behalf Of Lukáš Vlcek
Sent: September 13, 2013 12:26
To: elasticsearch@googlegroups.com
Subject: Re: Looking at ES logging

http://logstash.net ?

On Fri, Sep 13, 2013 at 8:51 AM, R Ashwin Sathya ashwin.sathya@outlook.com wrote:
Hi,

I have configured logging.yml to TRACE level messages and this level is pretty verbose. I run a 3-node VM cluster, is there an easier way that I can look at the logs other than manually remoting into the machine and reading the file ?

Thanks,
Ashwin Sathya

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ashwin Sathya) #5

Great!!

I will look into logstash and Kibana then. Thanks.

Thanks,

Ashwin Sathya

From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On Behalf Of David Pilato
Sent: September 13, 2013 12:56
To: elasticsearch@googlegroups.com
Subject: Re: Looking at ES logging

Yes!

I did it somehow like this while working on logs sent by one of our customers.

It was really great to build my dashboard very quick in Kibana and find easily what I was looking for.

--

David Pilato | Technical Advocate | Elasticsearch.com http://Elasticsearch.com

https://twitter.com/dadoonet @dadoonet | https://twitter.com/elasticsearchfr @elasticsearchfr | @scrutmydocs https://twitter.com/scrutmydocs

Le 13 sept. 2013 à 09:07, R Ashwin Sathya <ashwin.sathya@outlook.com mailto:ashwin.sathya@outlook.com > a écrit :

Logstash seems to enable output to elasticsearch. Does it mean I can parse the elasticsearch logs using logstash and store them as an index in the same ES cluster ? That would be awesome!!!

Thanks,

Ashwin Sathya

From: mailto:elasticsearch@googlegroups.com elasticsearch@googlegroups.com [mailto:elasticsearch@ http://googlegroups.com googlegroups.com] On Behalf Of Lukáš Vlcek
Sent: September 13, 2013 12:26
To: mailto:elasticsearch@googlegroups.com elasticsearch@googlegroups.com
Subject: Re: Looking at ES logging

http://logstash.net http://logstash.net ?

On Fri, Sep 13, 2013 at 8:51 AM, R Ashwin Sathya < mailto:ashwin.sathya@outlook.com ashwin.sathya@outlook.com> wrote:

Hi,

I have configured logging.yml to TRACE level messages and this level is pretty verbose. I run a 3-node VM cluster, is there an easier way that I can look at the logs other than manually remoting into the machine and reading the file ?

Thanks,

Ashwin Sathya

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mailto:elasticsearch%2Bunsubscribe@googlegroups.com elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mailto:elasticsearch+unsubscribe@googlegroups.com elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mailto:elasticsearch+unsubscribe@googlegroups.com elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com .
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #6

Have a look at http://three.kibana.org/ and not http://kibana.org/ BTW.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 13 sept. 2013 à 09:27, R Ashwin Sathya ashwin.sathya@outlook.com a écrit :

Great!!
I will look into logstash and Kibana then. Thanks.

Thanks,
Ashwin Sathya

From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On Behalf Of David Pilato
Sent: September 13, 2013 12:56
To: elasticsearch@googlegroups.com
Subject: Re: Looking at ES logging

Yes!

I did it somehow like this while working on logs sent by one of our customers.
It was really great to build my dashboard very quick in Kibana and find easily what I was looking for.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 13 sept. 2013 à 09:07, R Ashwin Sathya ashwin.sathya@outlook.com a écrit :

Logstash seems to enable output to elasticsearch. Does it mean I can parse the elasticsearch logs using logstash and store them as an index in the same ES cluster ? That would be awesome!!!

Thanks,
Ashwin Sathya

From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On Behalf Of Lukáš Vlcek
Sent: September 13, 2013 12:26
To: elasticsearch@googlegroups.com
Subject: Re: Looking at ES logging

http://logstash.net ?

On Fri, Sep 13, 2013 at 8:51 AM, R Ashwin Sathya ashwin.sathya@outlook.com wrote:
Hi,

I have configured logging.yml to TRACE level messages and this level is pretty verbose. I run a 3-node VM cluster, is there an easier way that I can look at the logs other than manually remoting into the machine and reading the file ?

Thanks,
Ashwin Sathya

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #7

BTW, it took me some time to build a proper logstash.conf file.
Here is what I did (not sure it's the best configuration here but at least it worked for my needs): https://gist.github.com/dadoonet/6547715

HTH

David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 13 sept. 2013 à 09:29, David Pilato david@pilato.fr a écrit :

Have a look at http://three.kibana.org/ and not http://kibana.org/ BTW.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 13 sept. 2013 à 09:27, R Ashwin Sathya ashwin.sathya@outlook.com a écrit :

Great!!
I will look into logstash and Kibana then. Thanks.

Thanks,
Ashwin Sathya

From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On Behalf Of David Pilato
Sent: September 13, 2013 12:56
To: elasticsearch@googlegroups.com
Subject: Re: Looking at ES logging

Yes!

I did it somehow like this while working on logs sent by one of our customers.
It was really great to build my dashboard very quick in Kibana and find easily what I was looking for.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr | @scrutmydocs

Le 13 sept. 2013 à 09:07, R Ashwin Sathya ashwin.sathya@outlook.com a écrit :

Logstash seems to enable output to elasticsearch. Does it mean I can parse the elasticsearch logs using logstash and store them as an index in the same ES cluster ? That would be awesome!!!

Thanks,
Ashwin Sathya

From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On Behalf Of Lukáš Vlcek
Sent: September 13, 2013 12:26
To: elasticsearch@googlegroups.com
Subject: Re: Looking at ES logging

http://logstash.net ?

On Fri, Sep 13, 2013 at 8:51 AM, R Ashwin Sathya ashwin.sathya@outlook.com wrote:
Hi,

I have configured logging.yml to TRACE level messages and this level is pretty verbose. I run a 3-node VM cluster, is there an easier way that I can look at the logs other than manually remoting into the machine and reading the file ?

Thanks,
Ashwin Sathya

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #8