NEST 6.2 using wildcard in indices

Elastic Stack Elasticsearch
1

Hi, is there a way to use a wildcard query in indices using NEST?

We have indices like:
index-2018.08
index-2018.09

and we only want to use those indices in one query in c#.
We used index-* but it shows invalid response.

2

Can you provide an example with NEST of what you're doing, and how the response is invalid?

3

Here's the sample NEST

var searchResponse = client.Search<Doc>(s => s

            .Index("myindex-*")

                .Size(1000)

                .Query(q => q

                    .MatchAll()

                )

            );

and here's the error message I get

{Invalid NEST response built from a unsuccessful low level call on POST: /6.2/elasticsearch/myindex-%2A/doc/_search?typed_keys=true}

4

It looks like you got some HTML encoding issues in the source code block of your example.

If you disable direct streaming for the request and capture the request and response bytes, can you see more details on what the error is?

var searchResponse = client.Search<Doc>(s => s
	.Index("myindex-*")
	.Size(1000)
	.Query(q => q
		.MatchAll()
	)
	.RequestConfiguration(r => r
		.DisableDirectStreaming() // capture request and response bytes, for this request only
	)
);

Console.WriteLine(searchResponse.DebugInformation);

Would you also be able to provide some details on your environment:

  1. NEST version e.g. 6.2.0
  2. Elasticsearch version e.g. 6.0.0
  3. Runtime e.g. .NET Framework 4.5, .NET Core 2.0, etc.
  4. Operating system e.g. Windows 10, MacOS High Sierra, etc.
5

Hi @forloop

I was able to execute the request, and below are the details captured:

 Invalid NEST response built from a unsuccessful low level call on POST: /6.2/elasticsearch/myindex-%2A/doc/_search?typed_keys=true
# Audit trail of this API call:
 - [1] BadResponse: Node: https://***********.com/6.2/elasticsearch/ Took: 00:00:01.5121155
# OriginalException: Elasticsearch.Net.ElasticsearchClientException: The remote server returned an error: (400) Bad Request.. Call: Status code 400 from: POST /6.2/elasticsearch/myindex-%2A/doc/_search?typed_keys=true ---> System.Net.WebException: The remote server returned an error: (400) Bad Request.
   at System.Net.HttpWebRequest.GetResponse()
   at Elasticsearch.Net.HttpConnection.Request[TResponse](RequestData requestData)
   --- End of inner exception stack trace ---
# Request:
{"size":1000,"query":{"match_all":{}}}

Please see also the environment details,

  1. NEST version - 6.2.0
  2. Elasticsearch version - 6.2.3
  3. Runtime - .NET Framework 4.6.1
  4. Operating system - Windows 10

Thanks!

6

Thanks @_kyllr. Would it be possible to also post the response body, so we can see what the 400 Bad Request relates to.

You can get the response body with

var searchResponse = client.Search<Doc>(s => s
	.Index("myindex-*")
	.Size(1000)
	.Query(q => q
		.MatchAll()
	)
	.RequestConfiguration(r => r
		.DisableDirectStreaming() // capture request and response bytes, for this request only
	)
);

Console.WriteLine(Encoding.UTF8.GetString(searchResponse.ApiCall.ResponseBodyInBytes));
7

Hi @forloop,

Here's the response body,

<!DOCTYPE html>
<html>
    <head>
        <title>Runtime Error</title>
        <meta name="viewport" content="width=device-width" />
        <style>
         body {font-family:"Verdana";font-weight:normal;font-size: .7em;color:black;}
         p {font-family:"Verdana";font-weight:normal;color:black;margin-top: -5px}
         b {font-family:"Verdana";font-weight:bold;color:black;margin-top: -5px}
         H1 { font-family:"Verdana";font-weight:normal;font-size:18pt;color:red }
         H2 { font-family:"Verdana";font-weight:normal;font-size:14pt;color:maroon }
         pre {font-family:"Consolas","Lucida Console",Monospace;font-size:11pt;margin:0;padding:0.5em;line-height:14pt}
         .marker {font-weight: bold; color: black;text-decoration: none;}
         .version {color: gray;}
         .error {margin-bottom: 10px;}
         .expandable { text-decoration:underline; font-weight:bold; color:navy; cursor:hand; }
         @media screen and (max-width: 639px) {
          pre { width: 440px; overflow: auto; white-space: pre-wrap; word-wrap: break-word; }
         }
         @media screen and (max-width: 479px) {
          pre { width: 280px; }
         }
        </style>
    </head>

    <body bgcolor="white">

            <span><H1>Server Error in '/6.2/elasticsearch' Application.<hr width=100% size=1 color=silver></H1>

            <h2> <i>Runtime Error</i> </h2></span>

            <font face="Arial, Helvetica, Geneva, SunSans-Regular, sans-serif ">

            <b> Description: </b>An application error occurred on the server. The current custom error settings for this application prevent the details of the application error from being viewed remotely (for security reasons). It could, however, be viewed by browsers running on the local server machine.
            <br><br>

            <b>Details:</b> To enable the details of this specific error message to be viewable on remote machines, please create a &lt;customErrors&gt; tag within a &quot;web.config&quot; configuration file located in the root directory of the current web application. This &lt;customErrors&gt; tag should then have its &quot;mode&quot; attribute set to &quot;Off&quot;.<br><br>

            <table width=100% bgcolor="#ffffcc">
               <tr>
                  <td>
                      <code><pre>

&lt;!-- Web.Config Configuration File --&gt;

&lt;configuration&gt;
    &lt;system.web&gt;
        &lt;customErrors mode=&quot;Off&quot;/&gt;
    &lt;/system.web&gt;
&lt;/configuration&gt;</pre></code>

                  </td>
               </tr>
            </table>

            <br>

            <b>Notes:</b> The current error page you are seeing can be replaced by a custom error page by modifying the &quot;defaultRedirect&quot; attribute of the application&#39;s &lt;customErrors&gt; configuration tag to point to a custom error page URL.<br><br>

            <table width=100% bgcolor="#ffffcc">
               <tr>
                  <td>
                      <code><pre>

&lt;!-- Web.Config Configuration File --&gt;

&lt;configuration&gt;
    &lt;system.web&gt;
        &lt;customErrors mode=&quot;RemoteOnly&quot; defaultRedirect=&quot;mycustompage.htm&quot;/&gt;
    &lt;/system.web&gt;
&lt;/configuration&gt;</pre></code>

                  </td>
               </tr>
            </table>

            <br>

    </body>
</html>
8

Following this thread..

9

Is this the response body from the NEST call to Elasticsearch? If it is, it looks like there is a proxy in front of Elasticsearch (perhaps IIS, based on the content)?

10

Yes, we are using aws-es-proxy.
What should we do to in this case? Do we have any workarounds regarding on this one?

11

this aws-es-proxy?

The issue is somewhere from the proxy layer and back so there isn't an appropriate workaround that can be offered for the client in this case, as the issue doesn't appear to be with the client.

You'll need to find out what that issue is. One thing that may help is to turn on customErrors on, as per the error page, and see if it provides some additional insight. If you log exceptions behind the proxy, then you may be able to see the exception stack trace there too.

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.