Custom Log

Hello everyone, I want to know if it is possible to create customized
logs? Currently our ElasticSearch cluster is behind a load balancer
layer 4 (LVS). When error is logged, I can not find the IP of the machine.

example :

[2014-11-17 08:31:03,501][DEBUG][action.search.type ]
[node_elastic_02] All shards failed for phase: [query]

Would you have a solution?

Thank you.
Alexandre.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5469AA86.4050205%40opendoc.net.
For more options, visit https://groups.google.com/d/optout.

Alexandre,
doesn't the "node_elastic_02" already hint at the server where the
error occurred? If you use log4j for logging, you can specify a custom
format for the appender to include the IP address of the respective
node. However, at this point, it seems you just need some mapping of
node names to IP addresses as the information you want is already
contained in the logs. Am I missing something here?

Best regards,
--Jürgen

On 17.11.2014 08:57, Alexandre wrote:

Hello everyone, I want to know if it is possible to create customized
logs? Currently our Elasticsearch cluster is behind a load balancer
layer 4 (LVS). When error is logged, I can not find the IP of the
machine.

example :

[2014-11-17 08:31:03,501][DEBUG][action.search.type ]
[node_elastic_02] All shards failed for phase: [query]

Would you have a solution?

Thank you.
Alexandre.

--

Mit freundlichen Grüßen/Kind regards/Cordialement vôtre/Atentamente/С
уважением
i.A. Jürgen Wagner
Head of Competence Center "Intelligence"
& Senior Cloud Consultant

Devoteam GmbH, Industriestr. 3, 70565 Stuttgart, Germany
Phone: +49 6151 868-8725, Fax: +49 711 13353-53, Mobile: +49 171 864 1543
E-Mail: juergen.wagner@devoteam.com
mailto:juergen.wagner@devoteam.com, URL: www.devoteam.de
http://www.devoteam.de/


Managing Board: Jürgen Hatzipantelis (CEO)
Address of Record: 64331 Weiterstadt, Germany; Commercial Register:
Amtsgericht Darmstadt HRB 6450; Tax Number: DE 172 993 071

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5469AF11.8010304%40devoteam.com.
For more options, visit https://groups.google.com/d/optout.

Sorry, I expressed myself badly.

the diagram below :
http://mirror.opendoc.net/mailing-list/elasticsearch/elasticsearch.png

I would log IP of web servers in the elasticsearch log. I would bind the
web server and the Elasticsearch error.

Thank you.
Alexandre.

On 17/11/14 09:17, "Jürgen Wagner (DVT)" wrote:

Alexandre,
doesn't the "node_elastic_02" already hint at the server where the
error occurred? If you use log4j for logging, you can specify a custom
format for the appender to include the IP address of the respective
node. However, at this point, it seems you just need some mapping of
node names to IP addresses as the information you want is already
contained in the logs. Am I missing something here?

Best regards,
--Jürgen

On 17.11.2014 08:57, Alexandre wrote:

Hello everyone, I want to know if it is possible to create customized
logs? Currently our Elasticsearch cluster is behind a load balancer
layer 4 (LVS). When error is logged, I can not find the IP of the
machine.

example :

[2014-11-17 08:31:03,501][DEBUG][action.search.type ]
[node_elastic_02] All shards failed for phase: [query]

Would you have a solution?

Thank you.
Alexandre.

--

Mit freundlichen Grüßen/Kind regards/Cordialement vôtre/Atentamente/С
уважением
i.A. Jürgen Wagner
Head of Competence Center "Intelligence"
& Senior Cloud Consultant

Devoteam GmbH, Industriestr. 3, 70565 Stuttgart, Germany
Phone: +49 6151 868-8725, Fax: +49 711 13353-53, Mobile: +49 171 864 1543
E-Mail: juergen.wagner@devoteam.com
mailto:juergen.wagner@devoteam.com, URL: www.devoteam.de
http://www.devoteam.de/


Managing Board: Jürgen Hatzipantelis (CEO)
Address of Record: 64331 Weiterstadt, Germany; Commercial Register:
Amtsgericht Darmstadt HRB 6450; Tax Number: DE 172 993 071

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com
mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5469AF11.8010304%40devoteam.com
https://groups.google.com/d/msgid/elasticsearch/5469AF11.8010304%40devoteam.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5469BA13.3040009%40opendoc.net.
For more options, visit https://groups.google.com/d/optout.

If you use default Elasticsearch log4j 1.2.17, then you can modify the
conversion patterns in config/logging.yml

Example for console logging:

console:
type: console
layout:
type: consolePattern
conversionPattern: "[${hostname}][%d{ISO8601}][%-5p][%-25c] %m%n"

Then, start Elasticsearch with the system property "hostname" on command
line, e.g.

./bin/elasticsearch -Dhostname=hostname

The hostname can be converted to an IP in your favorite analysis tool
later. There may be more than one IP for your hostname so it is better to
log the hostname. Or you can find a command that prints your preferred IP,
use that instead of hostname command.

Jörg

On Mon, Nov 17, 2014 at 10:04 AM, Alexandre infos@opendoc.net wrote:

Sorry, I expressed myself badly.

the diagram below : http://mirror.opendoc.net/mailing-list/elasticsearch/
elasticsearch.png

I would log IP of web servers in the elasticsearch log. I would bind the
web server and the Elasticsearch error.

Thank you.
Alexandre.

On 17/11/14 09:17, "Jürgen Wagner (DVT)" wrote:

Alexandre,
doesn't the "node_elastic_02" already hint at the server where the
error occurred? If you use log4j for logging, you can specify a custom
format for the appender to include the IP address of the respective
node. However, at this point, it seems you just need some mapping of
node names to IP addresses as the information you want is already
contained in the logs. Am I missing something here?

Best regards,
--Jürgen

On 17.11.2014 08:57, Alexandre wrote:

Hello everyone, I want to know if it is possible to create customized
logs? Currently our Elasticsearch cluster is behind a load balancer
layer 4 (LVS). When error is logged, I can not find the IP of the
machine.

example :

[2014-11-17 08:31:03,501][DEBUG][action.search.type ]
[node_elastic_02] All shards failed for phase: [query]

Would you have a solution?

Thank you.
Alexandre.

--

Mit freundlichen Grüßen/Kind regards/Cordialement vôtre/Atentamente/С
уважением
i.A. Jürgen Wagner
Head of Competence Center "Intelligence"
& Senior Cloud Consultant

Devoteam GmbH, Industriestr. 3, 70565 Stuttgart, Germany
Phone: +49 6151 868-8725, Fax: +49 711 13353-53, Mobile: +49 171 864 1543
E-Mail: juergen.wagner@devoteam.com
mailto:juergen.wagner@devoteam.com, URL: www.devoteam.de
http://www.devoteam.de/


Managing Board: Jürgen Hatzipantelis (CEO)
Address of Record: 64331 Weiterstadt, Germany; Commercial Register:
Amtsgericht Darmstadt HRB 6450; Tax Number: DE 172 993 071

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com
mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5469AF11.
8010304%40devoteam.com
<https://groups.google.com/d/msgid/elasticsearch/5469AF11.
8010304%40devoteam.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/5469BA13.3040009%40opendoc.net.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFkkdwQFuSCdFLa2fgWOVgkrsWoz13s77kHjwtHx9kMiQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thank you Jörg.

In my scheme,
(http://mirror.opendoc.net/mailing-list/elasticsearch/elasticsearch.png)
I would log the ip of "web server 1" or 2 ...

Alexandre.

On 17/11/14 11:00, joergprante@gmail.com wrote:

If you use default Elasticsearch log4j 1.2.17, then you can modify the
conversion patterns in config/logging.yml

Example for console logging:

console:
type: console
layout:
type: consolePattern
conversionPattern: "[${hostname}][%d{ISO8601}][%-5p][%-25c] %m%n"

Then, start Elasticsearch with the system property "hostname" on command
line, e.g.

./bin/elasticsearch -Dhostname=hostname

The hostname can be converted to an IP in your favorite analysis tool
later. There may be more than one IP for your hostname so it is better
to log the hostname. Or you can find a command that prints your
preferred IP, use that instead of hostname command.

Jörg

On Mon, Nov 17, 2014 at 10:04 AM, Alexandre <infos@opendoc.net
mailto:infos@opendoc.net> wrote:

Sorry, I expressed myself badly.

the diagram below :
http://mirror.opendoc.net/__mailing-list/elasticsearch/__elasticsearch.png
<http://mirror.opendoc.net/mailing-list/elasticsearch/elasticsearch.png>

I would log IP of web servers in the elasticsearch log. I would bind
the web server and the ElasticSearch error.

Thank you.
Alexandre.


On 17/11/14 09:17, "Jürgen Wagner (DVT)" wrote:

    Alexandre,
        doesn't the "node_elastic_02" already hint at the server
    where the
    error occurred? If you use log4j for logging, you can specify a
    custom
    format for the appender to include the IP address of the respective
    node. However, at this point, it seems you just need some mapping of
    node names to IP addresses as the information you want is already
    contained in the logs. Am I missing something here?

    Best regards,
    --Jürgen

    On 17.11.2014 08:57, Alexandre wrote:

        Hello everyone, I want to know if it is possible to create
        customized
        logs? Currently our ElasticSearch cluster is behind a load
        balancer
        layer 4 (LVS). When error is logged, I can not find the IP
        of the
        machine.

        example :
        ---
        [2014-11-17 08:31:03,501][DEBUG][action.__search.type       ]
        [node_elastic_02] All shards failed for phase: [query]
        ---

        Would you have a solution?

        Thank you.
        Alexandre.



    --

    Mit freundlichen Grüßen/Kind regards/Cordialement
    vôtre/Atentamente/С
    уважением
    *i.A. Jürgen Wagner*
    Head of Competence Center "Intelligence"
    & Senior Cloud Consultant

    Devoteam GmbH, Industriestr. 3, 70565 Stuttgart, Germany
    Phone: +49 6151 868-8725 <tel:%2B49%206151%20868-8725>, Fax: +49
    711 13353-53 <tel:%2B49%20711%2013353-53>, Mobile: +49 171 864
    1543 <tel:%2B49%20171%20864%201543>
    E-Mail: juergen.wagner@devoteam.com
    <mailto:juergen.wagner@devoteam.com>
    <mailto:juergen.wagner@__devoteam.com
    <mailto:juergen.wagner@devoteam.com>>, URL: www.devoteam.de
    <http://www.devoteam.de>
    <http://www.devoteam.de/>

    ------------------------------__------------------------------__------------
    Managing Board: Jürgen Hatzipantelis (CEO)
    Address of Record: 64331 Weiterstadt, Germany; Commercial Register:
    Amtsgericht Darmstadt HRB 6450; Tax Number: DE 172 993 071


    --
    You received this message because you are subscribed to the Google
    Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from
    it, send
    an email to elasticsearch+unsubscribe@__googlegroups.com
    <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com
    <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/5469AF11.__8010304%40devoteam.com
    <https://groups.google.com/d/msgid/elasticsearch/5469AF11.8010304%40devoteam.com>
    <https://groups.google.com/d/__msgid/elasticsearch/5469AF11.__8010304%40devoteam.com?utm___medium=email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/5469AF11.8010304%40devoteam.com?utm_medium=email&utm_source=footer>__>.
    For more options, visit https://groups.google.com/d/__optout
    <https://groups.google.com/d/optout>.


--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/5469BA13.__3040009%40opendoc.net
<https://groups.google.com/d/msgid/elasticsearch/5469BA13.3040009%40opendoc.net>.

For more options, visit https://groups.google.com/d/__optout
<https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com
mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFkkdwQFuSCdFLa2fgWOVgkrsWoz13s77kHjwtHx9kMiQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFkkdwQFuSCdFLa2fgWOVgkrsWoz13s77kHjwtHx9kMiQ%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5469D65E.5060705%40opendoc.net.
For more options, visit https://groups.google.com/d/optout.