Delete Docs Based on Field with Multiple Lines Inside

Elastic Stack Elasticsearch
1

Hi all,
I have a bunch of docs in my index which are stack traces, and therefore have multiple lines. Here is an example:

{
  "_index": "web-jsp-2017.10.04",
  "_type": "logs",
    "_id": "AV7zFQERh_tiaf86dz-0",
    "_version": 1,
    "_score": null,
    "_source": {
      "container": "px-web-jsp",
      "source_host": "bos1px-app1",
      "level": "ERROR",
      "exception_class": "java.net.URISyntaxException",
      "message": "UT005001: An exception occurred processing the request",
      "mdc": {},
      "exception_message": "Relative path in absolute URI: https://s.4cdn.org:443http://s.4cdn.org/image/title/24.jpg",
      "path": "/es_data1/prodcburke_data/app1",
      "@timestamp": "2017-10-04T08:47:20.389Z",
      "stacktrace": "java.net.URISyntaxException: Relative path in absolute URI: https://s.4cdn.org:443http://s.4cdn.org/image/title/24.jpg\n\tat java.net.URI.checkPath(URI.java:1804)\n\tat java.net.URI.<init>(URI.java:679)\n\tat io.undertow.security.handlers.SinglePortConfidentialityHandler.getRedirectURI(SinglePortConfidentialityHandler.java:51)\n\tat io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.getRedirectURI(ServletConfidentialityConstraintHandler.java:83)\n\tat io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:49)\n\tat io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)\n\tat io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:56)\n\tat io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)\n\tat io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)\n\tat io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)\n\tat io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)\n\tat io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)\n\tat io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat com.paytronix.server.container.ParseRequestURLHandler.handleRequest(ParseRequestURLHandler.java:230)\n\tat io.undertow.server.handlers.PathHandler.handleRequest(PathHandler.java:91)\n\tat io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)\n\tat io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)\n\tat io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)\n\tat io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)\n\tat io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)\n\tat io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java:745)\n",
"thread_name": "default task-29",
"@version": "1",
"host": "bos1-es1",
"logger_name": "io.undertow.request",
"timestamp": "2017-10-04 04:47:20.389"
        },
        "fields": {
"@timestamp": [
  1507106840389
]
        },
        "highlight": {
          "message": [
            "@kibana-highlighted-field@UT005001@/kibana-highlighted-field@: @kibana-highlighted-field@An@/kibana-highlighted-field@ @kibana-highlighted-field@exception@/kibana-highlighted-field@ @kibana-highlighted-field@occurred@/kibana-highlighted-field@ @kibana-highlighted-field@processing@/kibana-highlighted-field@ @kibana-highlighted-field@the@/kibana-highlighted-field@ @kibana-highlighted-field@request@/kibana-highlighted-field@"
          ]
        },
        "sort": [
          1507106840389
        ]
      }

I have an existing list of regexes that I apply to these stacktraces, and would like to use it to delete docs from my cluster.

I have been trying to use a command like this to delete docs based on another field -

POST web-jsp-2017.10.04/_delete_by_query
{
"query": {
"regexp" : {
  "logger_name.keyword": "stdout"
           }
         }
}

does anyone have any ideas about how to delete fields like my stacktraces? Ideally using regex.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.