Logstash email hyperlink length limit

Elastic Stack Logstash
1

The email plug is working fine. I would like to add a hyperlink to Kibana to allow the user to display a Visualization.
All of that works, except the hyperlink is truncated in the Outlook email.
I have investigated different ways to shorten the URL. I cannot use solutions such as tinyurl for security reasons.
Do you have any suggestions?

Here is a Discovery example, which is as short as they get:

            if "_grokparsefailure" in [tags] {
                    grok {
                            match => { "message" => "%{GREEDYDATA:err_num}:%{SPACE}%{GREEDYDATA:narrative}" }
                            remove_tag => ["_grokparsefailure"]
                            add_field => [ "tags", "grok0714" ]
                    }

                    if( "grok0714" in [tags] ){

                            if( "Error" in [err_num] ){

                                    if( "No space left on device" in [narrative] ){

                                            mutate{
                                                    ### Replacing msg_level with err_num because it is blank. 2/7/2017 - Brian
                                                    replace => { "msg_level" => "%{err_num}" }
                                                    add_field => [ "[@metadata][my_body]", "

http://XXX.XXX.XXX.com:5602/app/kibana#/discover?_g=(refreshInterval:(display:'1%20minute',pause:!f,section:2,value:60000),time:(from:now-1h,mode:quick,to:now))&_a=(columns:!(component,host,err_num,narrative,tags),index:'dplr-%{ENVIRONMENT}-logstash-*',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'host:%20"%{host}"%20AND%20err_num:%20"%{err_num}"')),sort:!('@timestamp',desc))
"]
}
}
}
}
}

2

So... it's Logstash that's truncating the string?

3

I am not sure.
If I send it in an anchor tag, the text of href is truncated.

4

Here is the body of the email:

HP-UX Error: 28 from hltv0777.hydc.sbc.com:/opt/app/globalid/data/server/Release-9-0/logs/135.213.40.121_45402_exception.log
Visualization http://blpi165.bhdc.att.com:5602/app/kibana#/visualize/create?type=histogram&indexPattern=dplr-%{Test}-logstash-&_g=%28refreshInterval:%28display:%271%20minute%27,pause:!f,section:2,value:60000%29,time:%28from:now-1h,mode:quick,to:now%29%29&_a=%28filters:!%28%29,linked:!f,query:%28query_string:%28analyze_wildcard:!t,query:%27host:%20%22hltv0777.hydc.sbc.com%22%20AND%20err_num:%20%22HP-UX Error: 28%22%27%29%29,uiState:%28%29,vis:%28aggs:!%28%28id:%271%27,params:%28%29,schema:metric,type:count%29,%28id:%272%27,params:%28customInterval:%272h%27,extended_bounds:%28%29,field:%27@timestamp%27,interval:auto,min_doc_count:1%29,schema:segment,type:date_histogram%29,%28id:%273%27,params:%28field:host.raw,order:desc,orderBy:%271%27,size:5%29,schema:group,type:terms%29%29,listeners:%28%29,params:%28addLegend:!t,addTimeMarker:!f,addTooltip:!t,defaultYExtents:!f,mode:stacked,scale:linear,setYExtents:!f,shareYAxis:!t,times:!%28%29,yAxis:%28%29%29,title:%27New%20Visualization%27,type:histo! gram%29%29
Discovery http://blpi165.bhdc.att.com:5602/app/kibana#/discover?_g=(refreshInterval:(display:'1%20minute',pause:!f,section:2,value:60000),time:(from:now-1h,mode:quick,to:now))&_a=(columns:!(component,host,err_num,narrative,tags),index:'dplr-%{Test}-logstash-%27,interval:auto,query:%28query_string:%28analyze_wildcard:!t,query:%27host:%20%22hltv0777.hydc.sbc.com%22%20AND%20err_num:%20%22HP-UX Error: 28%22%27%29%29,sort:!%28%27@timestamp%27,desc%29%29
Narrative:
No space left on device
JavaClass:
%{javaclass}
Tags:
grok0714

5

Okay... but has that URL really been concatenated? It ends with "%27,desc%29%29" in your Logstash config and in your sample email body.

6

Apparently not. This may be an issue with Outlook.
Has anyone been able to email hyperlinks?

7

Surrounding the URL by angle brackets might help.

8

Thanks for your patience. Embedding the URL in a table surrounded by

tags is the secret:

dplrgid8@:/home/dplrgid8/logstash/dev/templates/$ cat example.my_body.txt

                    if( "grok0714" in [tags] ){

                            if( "Error" in [err_num] ){

                                    if( "No space left on device" in [narrative] ){

                                            mutate{
                                                    ### Replacing msg_level with err_num because it is blank. 2/7/2017 - Brian
                                                    replace => { "msg_level" => "%{err_num}" }
                                                    add_field => [ "[@metadata][my_body]", "
            <html>
            <body>
                <table style=width:\"100%\">
                    <tr>
                            <td><pre>%{err_num} from %{host}:%{path}</pre></td>
                    <tr>
                            <td><b>Copy & Paste Visualization Link to FireFox Browser:</b></td>
                    <tr>
                            <td>http://xx.xx.xx.com:5602/app/kibana#/visualize/create?embed=true&type=histogram&indexPattern=dplr-qa-logstash-*&_g=(refreshInterval:(display:'1%20minute',pause:!f,section:2,value:60000),time:(from:now-1h,mode:quick,to:now))&_a=(filters:!(),linked:!f,query:(query_string:(analyze_wildcard:!t,query:'host:%20%22hltv0777.hydc.xx.com%22%20AND%20err_num:%20%22HP-UX%20Error:%22')),uiState:(),vis:(aggs:!((id:'1',params:(),schema:metric,type:count),(id:'2',params:(customInterval:'2h',extended_bounds:(),field:'@timestamp',interval:auto,min_doc_count:1),schema:segment,type:date_histogram),(id:'3',params:(field:host.raw,order:desc,orderBy:'1',size:5),schema:group,type:terms)),listeners:(),params:(addLegend:!t,addTimeMarker:!f,addTooltip:!t,defaultYExtents:!f,mode:stacked,scale:linear,setYExtents:!f,shareYAxis:!t,times:!(),yAxis:()),title:'New%20Visualization',type:histogram))</td>
                    <tr>
                            <td><b>Copy & Paste Discovery Link to FireFox Browser:</b></td>
                    <tr>
                            <td>http://xx.xx.xxcom:5602/app/kibana#/discover?_g=(refreshInterval:(display:'1%20minute',pause:!f,section:2,value:60000),time:(from:now-1h,mode:quick,to:now))&_a=(columns:!(err_num,type,narrative),index:'dplr-qa-logstash-*',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'err_num:%20%22HP-UX%20Error:%2028%22%20AND%20type:%20%22habitat.exception%22%20AND%20%20host:%20%22hltv0777.hydc.xx.com%22')),sort:!('@timestamp',desc))</td>
                    <tr>
                            <td><b>Narrative:</b></tb>
                    <tr>
                            <td><pre>%{narrative}</pre></td>
                    <tr>
                            <td><b>Tags:</b></tb>
                    <tr>
                            <td><pre>%{tags}</pre></td>
                </table>
            </pre>
            </body>
            </html> " ]
                                            }

                                            throttle {
                                                    before_count =>-1 
                                                    after_count => 1
                                                    period => "3600"
                                                    key => "%{narrative}"
                                                    add_field => [ "tags", "throttled" ]
                                            }
                                    }
                            }
                    }
            }

Results in:

HP-UX Error: 28 from hltv0777.hydc.sbc.com:/opt/app/globalid/data/server/Release-9-0/logs/135.213.40.121_45402_exception.log
Copy & Paste Visualization Link to FireFox Browser:
http://xx.xx.xxcom:5602/app/kibana#/visualize/create?embed=true&type=histogram&indexPattern=dplr-qa-logstash-*&_g=(refreshInterval:(display:'1%20minute',pause:!f,section:2,value:60000),time:(from:now-1h,mode:quick,to:now))&_a=(filters:!(),linked:!f,query:(query_string:(analyze_wildcard:!t,query:'host:%20%22hltv0777.hydc.xx.com%22%20AND%20err_num:%20%22HP-UX%20Error:%22')),uiState:(),vis:(aggs:!((id:'1',params:(),schema:metric,type:count),(id:'2',params:(customInterval:'2h',extended_bounds:(),field:'@timestamp',interval:auto,min_doc_count:1),schema:segment,type:date_histogram),(id:'3',params:(field:host.raw,order:desc,orderBy:'1',size:5),schema:group,type:terms)),listeners:(),params:(addLegend:!t,addTimeMarker:!f,addTooltip:!t,defaultYExtents:!f,mode:stacked,scale:linear,setYExtents:!f,shareYAxis:!t,times:!(),yAxis:()),title:'New%20Visualization',type:histogram))

Copy & Paste Discovery Link to FireFox Browser:
http://xx.xx.xx.com:5602/app/kibana#/discover?_g=(refreshInterval:(display:'1%20minute',pause:!f,section:2,value:60000),time:(from:now-1h,mode:quick,to:now))&_a=(columns:!(err_num,type,narrative),index:'dplr-qa-logstash-*',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'err_num:%20%22HP-UX%20Error:%2028%22%20AND%20type:%20%22habitat.exception%22%20AND%20%20host:%20%22hltv0777.hydc.xx.com%22')),sort:!('@timestamp',desc))

Narrative:
No space left on device

Tags:
grok0714

9

Also, some of the credit goes to using the "Share Link" feature of Kibana. It seems to do some magic, but the table is still required.

10

This still does not work dynamically, i.e., replacing field names with text.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.