Using existing field for mapping


(Ayush) #1

I am new to elastic search, I have created an index "cmn" with a type
"mention". I am trying to import data from my existing solr to
elasticsearch, so I want to map an existing field to the _id field.

I have created the following file under /config/mappings/cmn/,

{
    "mappings": {
"mentions":{

"_id" : {
"path" : "docKey"
}
}
}
}

But this doesn't seem to be working, every time I index a record the
following _id is created,

        "_index": "cmn",
        "_type": "mentions",
        "_id": "k4E0dJr6Re2Z39HAIjYMmg",
        "_score": 1

Also, the mapping is not reflects. I have also tried the following option,

{
    "mappings": {

"_id" : {
"path" : "docKey"
}
}
}

SAMPLE DOCUMENT: Basically a tweet.

 {
           "usrCreatedDate": "2012-01-24 21:34:47",
           "sex": "U",
           "listedCnt": 2,
           "follCnt": 432,
           "state": "Southampton",
           "classified": 0,
           "favCnt": 468,
           "timeZone": "Casablanca",
           "twitterId": 473333038,
           "lang": "en",
           "stnostem": "#ootd #ootw #fashion #styling #photography 

#white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"sourceId": "tw",
"timestamp": "2014-04-09T22:58:00.396Z",
"sentiment": 0,
"updatedOnGMTDate": "2014-04-09T22:56:57.000Z",
"userLocation": "Southampton",
"age": 0,
"priorityScore": 57.4700012207031,
"statusCnt": 14612,
"name": "YazzyK",
"profilePicUrl":
"http://pbs.twimg.com/profile_images/453578494556270594/orsA0pKi_normal.jpeg",
"mentions": "",
"sourceStripped": "Instagram",
"collectionName": "STREAMING",
"tags": "557/161/193/197",
"msgid": 1397084280396.33,
"version": 1464949081784713200,
"url2":
"{"urls":[{"url":"http://t.co/YbPFrXlpuh","expandedURL":"http://instagram.com/p/mliZbgxVZm/","displayURL":"instagram.com/p/mliZbgxVZm/","start":88,"end":110}]}",
"links": "http://t.co/YbPFrXlpuh",
"retweetedStatus": "",
"twtScreenName": "YazKader",
"postId": "454030232501358592",
"country": "Bermuda",
"message": "#ootd #ootw #fashion #styling #photography
#white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"source": "<a href="http://instagram.com"
rel="nofollow">Instagram",
"parentStatusId": -1,
"bio": "Live and breathe Fashion. Persian and proud-
Instagram: @Yazkader",
"createdOnGMTDate": "2014-04-09T22:56:57.000Z",
"searchText": "#ootd #ootw #fashion #styling #photography
#white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"isFavorited": "False",
"frenCnt": 214,
"docKey": "tw_454030232501358592"
}
Also, how can we create unique mapping for each "TYPE" and not just the
index.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/15c33940-d95c-48b4-afcd-a3cccdabafaf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Tihomir Lichev) #2

What API are you using to communicate with ES ?

03 август 2014, неделя, 11:14:59 UTC+3, Ayush написа:

I am new to elastic search, I have created an index "cmn" with a type
"mention". I am trying to import data from my existing solr to
elasticsearch, so I want to map an existing field to the _id field.

I have created the following file under /config/mappings/cmn/,

{
    "mappings": {
"mentions":{

"_id" : {
"path" : "docKey"
}
}
}
}

But this doesn't seem to be working, every time I index a record the
following _id is created,

        "_index": "cmn",
        "_type": "mentions",
        "_id": "k4E0dJr6Re2Z39HAIjYMmg",
        "_score": 1

Also, the mapping is not reflects. I have also tried the following option,

{
    "mappings": {

"_id" : {
"path" : "docKey"
}
}
}

SAMPLE DOCUMENT: Basically a tweet.

 {
           "usrCreatedDate": "2012-01-24 21:34:47",
           "sex": "U",
           "listedCnt": 2,
           "follCnt": 432,
           "state": "Southampton",
           "classified": 0,
           "favCnt": 468,
           "timeZone": "Casablanca",
           "twitterId": 473333038,
           "lang": "en",
           "stnostem": "#ootd #ootw #fashion #styling #photography 

#white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"sourceId": "tw",
"timestamp": "2014-04-09T22:58:00.396Z",
"sentiment": 0,
"updatedOnGMTDate": "2014-04-09T22:56:57.000Z",
"userLocation": "Southampton",
"age": 0,
"priorityScore": 57.4700012207031,
"statusCnt": 14612,
"name": "YazzyK",
"profilePicUrl": "
http://pbs.twimg.com/profile_images/453578494556270594/orsA0pKi_normal.jpeg
",
"mentions": "",
"sourceStripped": "Instagram",
"collectionName": "STREAMING",
"tags": "557/161/193/197",
"msgid": 1397084280396.33,
"version": 1464949081784713200,
"url2": "{"urls":[{"url":"http://t.co/YbPFrXlpuh
","expandedURL":"http://instagram.com/p/mliZbgxVZm/","displayURL":"
instagram.com/p/mliZbgxVZm/\ http://instagram.com/p/mliZbgxVZm/\
","start":88,"end":110}]}",
"links": "http://t.co/YbPFrXlpuh",
"retweetedStatus": "",
"twtScreenName": "YazKader",
"postId": "454030232501358592",
"country": "Bermuda",
"message": "#ootd #ootw #fashion #styling #photography
#white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"source": "<a href="http://instagram.com"
rel="nofollow">Instagram",
"parentStatusId": -1,
"bio": "Live and breathe Fashion. Persian and proud-
Instagram: @Yazkader",
"createdOnGMTDate": "2014-04-09T22:56:57.000Z",
"searchText": "#ootd #ootw #fashion #styling #photography
#white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"isFavorited": "False",
"frenCnt": 214,
"docKey": "tw_454030232501358592"
}
Also, how can we create unique mapping for each "TYPE" and not just the
index.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9f442b15-f956-4829-8436-c4ab10fd429e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Ayush) #3

I am using Java API, but have tried doing this using,

  1. putting a mapping file in /config/mapping etc.
  2. Using curl

On Monday, August 4, 2014 1:35:00 PM UTC+5:30, Tihomir Lichev wrote:

What API are you using to communicate with ES ?

03 август 2014, неделя, 11:14:59 UTC+3, Ayush написа:

I am new to elastic search, I have created an index "cmn" with a type
"mention". I am trying to import data from my existing solr to
elasticsearch, so I want to map an existing field to the _id field.

I have created the following file under /config/mappings/cmn/,

{
    "mappings": {
"mentions":{

"_id" : {
"path" : "docKey"
}
}
}
}

But this doesn't seem to be working, every time I index a record the
following _id is created,

        "_index": "cmn",
        "_type": "mentions",
        "_id": "k4E0dJr6Re2Z39HAIjYMmg",
        "_score": 1

Also, the mapping is not reflects. I have also tried the following option,

{
    "mappings": {

"_id" : {
"path" : "docKey"
}
}
}

SAMPLE DOCUMENT: Basically a tweet.

 {
           "usrCreatedDate": "2012-01-24 21:34:47",
           "sex": "U",
           "listedCnt": 2,
           "follCnt": 432,
           "state": "Southampton",
           "classified": 0,
           "favCnt": 468,
           "timeZone": "Casablanca",
           "twitterId": 473333038,
           "lang": "en",
           "stnostem": "#ootd #ootw #fashion #styling #photography 

#white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"sourceId": "tw",
"timestamp": "2014-04-09T22:58:00.396Z",
"sentiment": 0,
"updatedOnGMTDate": "2014-04-09T22:56:57.000Z",
"userLocation": "Southampton",
"age": 0,
"priorityScore": 57.4700012207031,
"statusCnt": 14612,
"name": "YazzyK",
"profilePicUrl": "
http://pbs.twimg.com/profile_images/453578494556270594/orsA0pKi_normal.jpeg
",
"mentions": "",
"sourceStripped": "Instagram",
"collectionName": "STREAMING",
"tags": "557/161/193/197",
"msgid": 1397084280396.33,
"version": 1464949081784713200,
"url2": "{"urls":[{"url":"http://t.co/YbPFrXlpuh
","expandedURL":"http://instagram.com/p/mliZbgxVZm/
","displayURL":"instagram.com/p/mliZbgxVZm/\
http://instagram.com/p/mliZbgxVZm/\","start":88,"end":110}]}",
"links": "http://t.co/YbPFrXlpuh",
"retweetedStatus": "",
"twtScreenName": "YazKader",
"postId": "454030232501358592",
"country": "Bermuda",
"message": "#ootd #ootw #fashion #styling #photography
#white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"source": "<a href="http://instagram.com"
rel="nofollow">Instagram",
"parentStatusId": -1,
"bio": "Live and breathe Fashion. Persian and proud-
Instagram: @Yazkader",
"createdOnGMTDate": "2014-04-09T22:56:57.000Z",
"searchText": "#ootd #ootw #fashion #styling #photography
#white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"isFavorited": "False",
"frenCnt": 214,
"docKey": "tw_454030232501358592"
}
Also, how can we create unique mapping for each "TYPE" and not just the
index.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aaa67013-17d7-4c25-a697-65b7cbc4240d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Tihomir Lichev) #4
  1. Try go execute GET http://yourhost:9200/_mapping and check if the
    mapping mentions is properly created with correct _id path
  2. When indexing the documents you can construct your index request this
    way:

IndexRequestBuilder indexRequestBuilder = getClient().prepareIndex(index,
type).setSource(data);

where data is your Map or JSON you want to index

I tested with curl (-XPUT for mapping and -XPOST for indexing ! ) and seems
like it works :slight_smile:
I'm also using _id path and Java API in my own project and have no problems
with it so far

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d8e367d3-e6f7-4b54-9f3a-30154eb66a69%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #5