Delete Docs Based on Field with Multiple Lines Inside

Hi all,
I have a bunch of docs in my index which are stack traces, and therefore have multiple lines. Here is an example:

{
  "_index": "web-jsp-2017.10.04",
  "_type": "logs",
    "_id": "AV7zFQERh_tiaf86dz-0",
    "_version": 1,
    "_score": null,
    "_source": {
      "container": "px-web-jsp",
      "source_host": "bos1px-app1",
      "level": "ERROR",
      "exception_class": "java.net.URISyntaxException",
      "message": "UT005001: An exception occurred processing the request",
      "mdc": {},
      "exception_message": "Relative path in absolute URI: https://s.4cdn.org:443http://s.4cdn.org/image/title/24.jpg",
      "path": "/es_data1/prodcburke_data/app1",
      "@timestamp": "2017-10-04T08:47:20.389Z",
      "stacktrace": "java.net.URISyntaxException: Relative path in absolute URI: https://s.4cdn.org:443http://s.4cdn.org/image/title/24.jpg\n\tat java.net.URI.checkPath(URI.java:1804)\n\tat java.net.URI.<init>(URI.java:679)\n\tat io.undertow.security.handlers.SinglePortConfidentialityHandler.getRedirectURI(SinglePortConfidentialityHandler.java:51)\n\tat io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.getRedirectURI(ServletConfidentialityConstraintHandler.java:83)\n\tat io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:49)\n\tat io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)\n\tat io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:56)\n\tat io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)\n\tat io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)\n\tat io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)\n\tat io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)\n\tat io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)\n\tat io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat com.paytronix.server.container.ParseRequestURLHandler.handleRequest(ParseRequestURLHandler.java:230)\n\tat io.undertow.server.handlers.PathHandler.handleRequest(PathHandler.java:91)\n\tat io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)\n\tat io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)\n\tat io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)\n\tat io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)\n\tat io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)\n\tat io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)\n\tat io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java:745)\n",
"thread_name": "default task-29",
"@version": "1",
"host": "bos1-es1",
"logger_name": "io.undertow.request",
"timestamp": "2017-10-04 04:47:20.389"
        },
        "fields": {
"@timestamp": [
  1507106840389
]
        },
        "highlight": {
          "message": [
            "@kibana-highlighted-field@UT005001@/kibana-highlighted-field@: @kibana-highlighted-field@An@/kibana-highlighted-field@ @kibana-highlighted-field@exception@/kibana-highlighted-field@ @kibana-highlighted-field@occurred@/kibana-highlighted-field@ @kibana-highlighted-field@processing@/kibana-highlighted-field@ @kibana-highlighted-field@the@/kibana-highlighted-field@ @kibana-highlighted-field@request@/kibana-highlighted-field@"
          ]
        },
        "sort": [
          1507106840389
        ]
      }

I have an existing list of regexes that I apply to these stacktraces, and would like to use it to delete docs from my cluster.

I have been trying to use a command like this to delete docs based on another field -

POST web-jsp-2017.10.04/_delete_by_query
{
"query": {
"regexp" : {
  "logger_name.keyword": "stdout"
           }
         }
}

does anyone have any ideas about how to delete fields like my stacktraces? Ideally using regex.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.