# Compare two different indexes and create new ones

**URL:** <https://discuss.elastic.co/t/compare-two-different-indexes-and-create-new-ones/152696>\
**Category:** Elasticsearch\
**Created:** [October 16, 2018, 3:57pm UTC](https://discuss.elastic.co/t/compare-two-different-indexes-and-create-new-ones/152696 "2018-10-16T15:57:51Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![JohnOllhorn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johnollhorn/32/36697_2.png) [@JohnOllhorn](https://discuss.elastic.co/u/JohnOllhorn)\
**Post date:** [October 16, 2018, 3:57pm UTC](https://discuss.elastic.co/t/compare-two-different-indexes-and-create-new-ones/152696/1 "2018-10-16T15:57:52Z")

</div>

HI people,  
I'm just starting to analyze my data with Elasticsearch, so I have practically no idea -\_-

I would like to try text mining based on all the data I have stored.

Is the following possible, and if so where do I have to google to?

I have two indexes "Newsfeeds" and "Stopswords".  
In the first step, I'd like to:

- Count how many stopwords and which ones appear in the newsfeeds.
- Save the remaining words from the newsfeeds ( plus the stopswords ) in a new index.

Is this generally possible? Can we do this alone with ES, Kibana and Logstash?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 16, 2018, 10:46pm UTC](https://discuss.elastic.co/t/compare-two-different-indexes-and-create-new-ones/152696/2 "2018-10-16T22:46:56Z")

</div>

You way want to look at using percolator here, it'd be suitable to matching the stopwords.

However you would need to do a bit of work in your own code to take the feeds, pass them through the percolation and then create the other index. That part is not something the stack can automatically do for you.

---

<div class="post-metadata">

**Author:** ![JohnOllhorn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johnollhorn/32/36697_2.png) [@JohnOllhorn](https://discuss.elastic.co/u/JohnOllhorn)\
**Post date:** [October 18, 2018, 8:11pm UTC](https://discuss.elastic.co/t/compare-two-different-indexes-and-create-new-ones/152696/4 "2018-10-18T20:11:58Z")

</div>

Hello, folks,

Can I set the type of a field to Percolate when creating an index via mapping? If so, how? 🙂

```auto
try {
      await client.indices.create({
        index: 'test',
        mapping: {
          properties: { query: { type: "percolator" } }
        },
        ignore: [400]
      })
    } catch (e) {
      throw e
    }

```

That doesn't seem to work.

```json
GET /test/_search
{
    "query" : {
        "percolate" : {
            "field" : "query",
            "document" : {
                "description" : "test"
            }
        }
    }
}

```

return

```json
{
  "error": {
    "root_cause": [
      {
        "type": "query_shard_exception",
        "reason": "field [query] does not exist",
        "index_uuid": "EvFmjhdYQPiBD4cEhoYkiw",
        "index": "test"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "test",
        "node": "QCctHTucSGaVxmsobvKmzQ",
        "reason": {
          "type": "query_shard_exception",
          "reason": "field [query] does not exist",
          "index_uuid": "EvFmjhdYQPiBD4cEhoYkiw",
          "index": "test"
        }
      }
    ]
  },
  "status": 400
}

```

I always have to manually over the Dev tools:

```json
PUT /test/_mapping/feed
{
  "properties": {
    "query": { "type": "percolator" }
  }
}

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 18, 2018, 10:27pm UTC](https://discuss.elastic.co/t/compare-two-different-indexes-and-create-new-ones/152696/5 "2018-10-18T22:27:07Z")

</div>

What does the document look like?

---

<div class="post-metadata">

**Author:** ![JohnOllhorn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johnollhorn/32/36697_2.png) [@JohnOllhorn](https://discuss.elastic.co/u/JohnOllhorn)\
**Post date:** [October 19, 2018, 6:46am UTC](https://discuss.elastic.co/t/compare-two-different-indexes-and-create-new-ones/152696/6 "2018-10-19T06:46:03Z")

</div>

```
{
  "_index": "test",
  "_type": "feed",
  "_id": "25419144fd9fec74ab26ce5b86e4fa0059a8c123",
  "_version": 1,
  "_score": null,
  "_source": {
    "title": "Cross-Channel, Data und Mediamix: Das sind die zentralen Herausforderungen für Marketingentscheider",
    "description": "<div class=\"alignleft detailm\"> <img src=\"https://www.horizont.net/news/media/26/Manager-Marketingchef-CMO-253003-detailm.jpeg\" alt=\"Manager Marketingchef CMO\" title=\"Manager Marketingchef CMO\"/> <br/><span class=\"imgsubtitle\"></span> <div class='imgcreditbg'><span class='imgcredit'>© Fotolia / kasto</span></div> </div>In seiner jährlichen Studie \"Getting Media Right\" hat Kantar Millward Brown die zentralen Branchen-Herausforderungen für Marketingspezialisten analysiert. Eines der Ergebnisse: 45 Prozent halten den Mediamix in ihrem Unternehmen für nicht optimal.",
    "summary": "<div class=\"alignleft detailm\"> <img src=\"https://www.horizont.net/news/media/26/Manager-Marketingchef-CMO-253003-detailm.jpeg\" alt=\"Manager Marketingchef CMO\" title=\"Manager Marketingchef CMO\"/> <br/><span class=\"imgsubtitle\"></span> <div class='imgcreditbg'><span class='imgcredit'>© Fotolia / kasto</span></div> </div>In seiner jährlichen Studie \"Getting Media Right\" hat Kantar Millward Brown die zentralen Branchen-Herausforderungen für Marketingspezialisten analysiert. Eines der Ergebnisse: 45 Prozent halten den Mediamix in ihrem Unternehmen für nicht optimal.",
    "date": "2018-10-18T20:06:00.000Z",
    "pubdate": "2018-10-18T20:06:00.000Z",
    "pubDate": "2018-10-18T20:06:00.000Z",
    "link": "http://www.horizont.net/marketing/nachrichten/cross-channel-data-und-mediamix-das-sind-die-zentralen-herausforderungen-fuer-marketingentscheider-170469?utm_source=rss&utm_medium=referral&utm_campaign=news-marketing&utm_term=%7Butm_term%7D",
    "guid": "http://www.horizont.net/marketing/nachrichten/cross-channel-data-und-mediamix-das-sind-die-zentralen-herausforderungen-fuer-marketingentscheider-170469?utm_source=rss&utm_medium=referral&utm_campaign=news-marketing&utm_term={utm_term}",
    "author": "Redaktion",
    "comments": null,
    "origlink": null,
    "image": {},
    "source": {},
    "categories": [
      "Nachrichten"
    ],
    "enclosures": [
      {
        "url": "https://www.horizont.net/news/media/26/Manager-Marketingchef-CMO-253003-detailp.jpeg",
        "type": "image/jpeg",
        "length": ""
      }
    ],
    "rss:@": {},
    "rss:title": {
      "@": {},
      "#": "Cross-Channel, Data und Mediamix: Das sind die zentralen Herausforderungen für Marketingentscheider"
    },
    "rss:link": {
      "@": {},
      "#": "http://www.horizont.net/marketing/nachrichten/cross-channel-data-und-mediamix-das-sind-die-zentralen-herausforderungen-fuer-marketingentscheider-170469?utm_source=rss&utm_medium=referral&utm_campaign=news-marketing&utm_term=%7Butm_term%7D"
    },
    "rss:description": {
      "@": {},
      "#": "<div class=\"alignleft detailm\"> <img src=\"https://www.horizont.net/news/media/26/Manager-Marketingchef-CMO-253003-detailm.jpeg\" alt=\"Manager Marketingchef CMO\" title=\"Manager Marketingchef CMO\"/> <br/><span class=\"imgsubtitle\"></span> <div class='imgcreditbg'><span class='imgcredit'>© Fotolia / kasto</span></div> </div>In seiner jährlichen Studie \"Getting Media Right\" hat Kantar Millward Brown die zentralen Branchen-Herausforderungen für Marketingspezialisten analysiert. Eines der Ergebnisse: 45 Prozent halten den Mediamix in ihrem Unternehmen für nicht optimal."
    },
    "rss:author": {
      "@": {}
    },
    "rss:category": {
      "@": {},
      "#": "Nachrichten"
    },
    "rss:enclosure": {
      "@": {
        "url": "https://www.horizont.net/news/media/26/Manager-Marketingchef-CMO-253003-detailp.jpeg",
        "length": "",
        "type": "image/jpeg"
      }
    },
    "permalink": "http://www.horizont.net/marketing/nachrichten/cross-channel-data-und-mediamix-das-sind-die-zentralen-herausforderungen-fuer-marketingentscheider-170469?utm_source=rss&utm_medium=referral&utm_campaign=news-marketing&utm_term={utm_term}",
    "rss:guid": {
      "@": {},
      "#": "http://www.horizont.net/marketing/nachrichten/cross-channel-data-und-mediamix-das-sind-die-zentralen-herausforderungen-fuer-marketingentscheider-170469?utm_source=rss&utm_medium=referral&utm_campaign=news-marketing&utm_term={utm_term}"
    },
    "rss:pubdate": {
      "@": {},
      "#": "Thu, 18 Oct 2018 22:06:00 +0200"
    },
    "dc:creator": {
      "@": {}
    },
    "meta": {
        "version": "1.0",
        "encoding": "utf-8"
      },
      "atom:link": {
        "@": {
          "href": "https://www.horizont.net/news/feed/",
          "rel": "self",
          "type": "application/rss+xml"
        }
      }
    }
  },
  "fields": {
    "date": [
      "2018-10-18T20:06:00.000Z"
    ],
    "meta.pubdate": [
      "2018-10-18T20:17:44.000Z"
    ],
    "meta.date": [
      "2018-10-18T20:17:43.000Z"
    ],
    "meta.pubDate": [
      "2018-10-18T20:17:44.000Z"
    ],
    "pubDate": [
      "2018-10-18T20:06:00.000Z"
    ],
    "pubdate": [
      "2018-10-18T20:06:00.000Z"
    ]
  },
  "sort": [
    1539893160000
  ]
}

```

It's an RSS feed.  
I'm not using the RSS input plugin (it's just outdated).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 16, 2018, 6:46am UTC](https://discuss.elastic.co/t/compare-two-different-indexes-and-create-new-ones/152696/7 "2018-11-16T06:46:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.