Ruby, App Search, filters & my sanity

Elastic Search
1

Hello! I've been losing my mind trying to figure out how to use multiple filters on a query using the Ruby gem for App Search (elastic-enterprise-search).

Reading the docs here:

I'm not seeing any examples using multiple filters.

So let's head here:

Great! Now, I need to do is something like this (JSON):

"filters": {
  "all": [
    { "states": ["Washington", "Idaho"] },
    { "world_heritage_site": ["true"] }
  ]
}

And then convert it to Ruby:

filters: {
  all: [
    { state: ['Washington', 'Idaho'] },
    { world_heritage_site: ['true'] }
  ]
}

But I keep running into an error:
"Filters contains an invalid set of keys for object inside of field: all; can only have clauses or a single field name"

I've tried other variations of the syntax as well:

filters: {
  all: {
    state: ['Washington', 'Idaho'],
    world_heritage_site: ['true']
  }
}

No dice. I can just remove either one of the OR's and it'll work:

filters: {
  all: {
    state: ['Washington', 'Idaho'],
  }
}

Or with the other syntax will also work:

filters: {
  all: [
    { world_heritage_site: ['true'] }
  ]
}

But I'll keep getting the above error if I add more than one OR.

I feel like I'm missing something super obvious here and any pointers would be greatly appreciated. Thank you!

2

Heya Brian, thanks for posting! I'm not a Ruby expert so I'm going to grab one of my coworkers who is, but let me see what I can help with in the meanwhile.

To quickly start - the 3rd example you gave should definitely not work / should absolutely trigger the error you're seeing. From your example, all either wants an array of objects with a single key, or it wants a single object with a single key. The below should definitely fail:

  all: {
    state: ['Washington', 'Idaho'],
    world_heritage_site: ['true']
  }

I'm testing against my local national parks sample engine and I'm a little puzzled. Just to make sure, are you definitely giving us the code you're actually running (presumably against a sample engine), or are you just giving us a dummy/example? If you're actually sending this search, then the main issue I see is that state should be states, and you should be getting a Filters contains invalid key: state error.

If it's just an example - would it be possible for you to give us the exact code/syntax you're sending? Again, it sounds like the issue is your all array potentially has an object with multiple keys in it instead of just a single key, but it's hard to say without the actual code.

3

Hi Constance,
You are correct in that this was all dummy code. You bring up a good point and I should try to create a full, focused example.

I loaded that parks data back in as a new engine to test against. My data is a little busted but I'm not worrying about results at this point, just the query going through.

Here's a failing example:

client = Elastic::EnterpriseSearch::AppSearch::Client.new(host: url, http_auth: auth_key)
client.search('parks', {
  query: "olympic",
  filters: {
    all: {
      states: ["Washington", "Idaho"],
      world_heritage_site: ["true"]
    }
  }
})

Like you said, the above does fail and all should be an array:

client.search('parks', {
  query: "olympic",
  filters: {
    all: [
      { states: ['Washington', 'Idaho'] },
      { world_heritage_site: ['true'] }
    ]
  }
})

This actually returns a Ruby error:
TypeError (no implicit conversion of String into Integer)

Which makes me think the gem isn't ready for those arrays, because if I run this:

client.search('parks', {
  query: "olympic",
  filters: {
    all: [
      { states: "Washington" },
      { world_heritage_site: "true" }
    ]
  }
})

The query is run but I get the "Filters contains an invalid..." error again.

And as before, if I remove one of the filters it works:

client.search('parks', {
  query: "california",
  filters: {
    all: [
      { states: "Washington" },
    ]
  }
})

My gut says I'm doing something goofy syntax-wise and missing something obvious...

4

Hey Brian! No worries at all, I went back and looked more closely at the gem source code and I think I have super wild hunch as to what's going wrong. Can you try adding a body key to the 2nd arg, like this:

client.search('parks', {
  body: {
    query: "olympic",
    filters: {
      all: [
        { states: "Washington" },
        { world_heritage_site: "true" }
      ]
    }
  }
})

Let me know if that produces a non-error response (fingers crossed in advance, and deep apologies if this is a wild goose chase and doesn't work)

5

That works! I never thought to add that because I thought the gem stripped away some of the "lower level" Elastic stuff.

Good deal, thank you very much for letting me borrow your knowledge for a bit.

6

Not on you at all, seriously! We definitely need to update the documentation for that gem to show an example with a body, as well as as an example with filters. I'll see what I can do here about opening a PR to add that. Thanks for helping us find a gap in our docs!

Also now that I know it works (thanks for being my guinea pig, haha!) I'll dive more into what was going for anyone curious (feel free to skip if you're not). Here's the source code in specific that clued me in:

A quick explanation of why this was going wrong: outside of the body key, arguments are forwarded as basic query params. So your original call was getting translated to something like /search?query=foo&filters=bar, which does sorta work since apparently our search endpoint accepts both query params and JSON bodies.

However, since query strings don't tend to handle complex/nested objects very well, it was failing when you tried to go into an array. Switching over to a JSON body fixes that and allows for more complex filters.

Hope that all makes sense, and super glad I could help! :tada:

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.