# Searching multiple fields(x,y) in Elasticsearch index "A" for entries in present in a field(z) in different index "B"

**URL:** <https://discuss.elastic.co/t/searching-multiple-fields-x-y-in-elasticsearch-index-a-for-entries-in-present-in-a-field-z-in-different-index-b/41753>\
**Category:** Elasticsearch\
**Created:** [February 15, 2016, 9:23am UTC](https://discuss.elastic.co/t/searching-multiple-fields-x-y-in-elasticsearch-index-a-for-entries-in-present-in-a-field-z-in-different-index-b/41753 "2016-02-15T09:23:22Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Natarajan\_venkataram](https://avatars.discourse-cdn.com/v4/letter/n/f14d63/32.png) [@Natarajan\_venkataram](https://discuss.elastic.co/u/Natarajan_venkataram)\
**Post date:** [February 15, 2016, 9:23am UTC](https://discuss.elastic.co/t/searching-multiple-fields-x-y-in-elasticsearch-index-a-for-entries-in-present-in-a-field-z-in-different-index-b/41753/1 "2016-02-15T09:23:22Z")

</div>

Hi,

I have a index which captures logs from various security devices at the perimeter. I want to check for possible connections from/to blacklisted IPs.

I have a index, which has the logs of the device and another index which contains blacklisted IPs ( updated daily).

How do I filter device logs for connections to the blacklisted IP addresses.

I'm a newbie to ELK stack. Can you please help me out with this.

Thanks  
Natarajan

---

<div class="post-metadata">

**Author:** ![Camilo\_Sierra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camilo_sierra/32/14397_2.png) [@Camilo\_Sierra](https://discuss.elastic.co/u/Camilo_Sierra)\
**Post date:** [February 15, 2016, 9:26am UTC](https://discuss.elastic.co/t/searching-multiple-fields-x-y-in-elasticsearch-index-a-for-entries-in-present-in-a-field-z-in-different-index-b/41753/2 "2016-02-15T09:26:23Z")

</div>

Hello i already had this use case, and for the query i used [https://www.elastic.co/guide/en/elasticsearch/reference/1.4/query-dsl-terms-filter.html#\_terms\_lookup\_twitter\_example](https://www.elastic.co/guide/en/elasticsearch/reference/1.4/query-dsl-terms-filter.html#_terms_lookup_twitter_example)

---

<div class="post-metadata">

**Author:** ![Natarajan\_venkataram](https://avatars.discourse-cdn.com/v4/letter/n/f14d63/32.png) [@Natarajan\_venkataram](https://discuss.elastic.co/u/Natarajan_venkataram)\
**Post date:** [February 15, 2016, 11:57am UTC](https://discuss.elastic.co/t/searching-multiple-fields-x-y-in-elasticsearch-index-a-for-entries-in-present-in-a-field-z-in-different-index-b/41753/3 "2016-02-15T11:57:17Z")

</div>

Hi Camilo,

Should the field names in both the indices be same ?  
The query I tried based on the term filter is not working as expected. Please let me know if I have made any mistake in the query.

Device logs -  
Index name : device\_logs  
Fields to be filtered : source\_address, destination\_address

Blacklisted IP -  
Index name : blacklisted\_ip  
Field to be used for filtering - ip\_address

Please find the query below -  
I have used "id" : "\*" for filtering on all the documents in the field ip\_address

curl -XGET localhost:9200/device\_logs/\_search -d '{  
{  
"query": {  
"filtered": {  
"filter": {  
"terms": {  
"logs": {  
"index": "blacklisted\_ip",  
"type": "logs",  
"id": "\*",  
"path": "ip\_address"  
},  
"\_cache\_key": "blacklst\_ip\_match"  
}  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![Camilo\_Sierra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camilo_sierra/32/14397_2.png) [@Camilo\_Sierra](https://discuss.elastic.co/u/Camilo_Sierra)\
**Post date:** [February 15, 2016, 1:00pm UTC](https://discuss.elastic.co/t/searching-multiple-fields-x-y-in-elasticsearch-index-a-for-entries-in-present-in-a-field-z-in-different-index-b/41753/4 "2016-02-15T13:00:59Z")

</div>

what i made is in your index blacklisted\_ip i keep a inner object with all the blacklist ip\_address, and after in id i give the id of my inner object.

i'm not sure that the "\*" in the Id works, and if it works not sure that is a good idea.

---

<div class="post-metadata">

**Author:** ![Natarajan\_venkataram](https://avatars.discourse-cdn.com/v4/letter/n/f14d63/32.png) [@Natarajan\_venkataram](https://discuss.elastic.co/u/Natarajan_venkataram)\
**Post date:** [February 15, 2016, 2:30pm UTC](https://discuss.elastic.co/t/searching-multiple-fields-x-y-in-elasticsearch-index-a-for-entries-in-present-in-a-field-z-in-different-index-b/41753/5 "2016-02-15T14:30:45Z")

</div>

Thanks for the inputs Camilo ... I'm a newbie and I have not worked with inner objects in ES..Is there any ES guide for that where I can familiarise with the concepts... I'm not able to find any examples on inner objects for this use case. Please let me know..

And, I'm currently using logstash for loading data to ES.. Is there any mechanism to index these entries from Logstash to ES index inner objects ??

PS : I'm currently using ES 2.2 where terms filter has been replaced by terms query.

---

<div class="post-metadata">

**Author:** ![Camilo\_Sierra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camilo_sierra/32/14397_2.png) [@Camilo\_Sierra](https://discuss.elastic.co/u/Camilo_Sierra)\
**Post date:** [February 15, 2016, 4:02pm UTC](https://discuss.elastic.co/t/searching-multiple-fields-x-y-in-elasticsearch-index-a-for-entries-in-present-in-a-field-z-in-different-index-b/41753/6 "2016-02-15T16:02:35Z")

</div>

hear can help you to understand, [https://www.elastic.co/blog/managing-relations-inside-elasticsearch](https://www.elastic.co/blog/managing-relations-inside-elasticsearch)

and for your use case hear using Array (string in the mapping ES):

```
curl -XPUT localhost:9200/ip_list/blacklisted/1 -d '{
   "ip_address" : ["127.0.0.0", "127.0.0.1"]
}'

curl -XPUT localhost:9200/device_logs/logs -d '{
   "text" : "test test test",
   "ip" : "127.0.0.0"
}'

curl -XGET localhost:9200/device_logs/logs/_search -d '{
  "query" : {
    "filtered" : {
      "filter" : {
        "terms" : {
          "ip" : {
            "index" : "ip_list",
            "type" : "blacklisted",
            "id" : "1",
            "path" : "ip_address"
          }
        }
      }
    }
  }
}'
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:16pm UTC](https://discuss.elastic.co/t/searching-multiple-fields-x-y-in-elasticsearch-index-a-for-entries-in-present-in-a-field-z-in-different-index-b/41753/7 "2017-07-05T23:16:22Z")

</div>


