# Common lines for multiple files

**URL:** <https://discuss.elastic.co/t/common-lines-for-multiple-files/18784>\
**Category:** Elasticsearch\
**Created:** [July 22, 2014, 8:52am UTC](https://discuss.elastic.co/t/common-lines-for-multiple-files/18784 "2014-07-22T08:52:36Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Effash](https://avatars.discourse-cdn.com/v4/letter/e/ecc23a/32.png) [@Effash](https://discuss.elastic.co/u/Effash)\
**Post date:** [July 22, 2014, 8:52am UTC](https://discuss.elastic.co/t/common-lines-for-multiple-files/18784/1 "2014-07-22T08:52:36Z")

</div>

Hello everyone,

I have a lot of files with a lot of short lines (~45000 per file). The  
lines consist in a keyword and some additional data  
I store each file and its metadata as objects in {index: "default", \_type:  
"file", id: filename, \_source: {various metadata} }  
I store each line as children of my files:  
body\_mapping = {"line": {  
"\_parent" :{  
"type" :"file"  
}  
}  
}

{"\_index": "default",  
"\_type": "line",  
"\_id": line\_number,  
"\_parent": filename  
"\_source": {"keyword": keyword,  
"metadata"}  
}

My goal is to search accross all my files by keyword {"query":

{"query\_string":

{"query" : keyword,

"fields" : ["keyword"]

}

}  
But there is more to it: I want to search a bunch of keywords from a given  
file (all lines from an existing file or a new one) and aggregate the  
results by filename.  
For example, the result would be:  
{filename1: [{keyword: my\_search\_keyword,  
metadata\_for\_this\_keyword\_in\_file1, \_id: line\_number},  
{keyword: my\_search\_keyword, metadata\_for\_this\_keyword\_in\_file1, \_id:  
line\_number}, ...],  
filename2: [{keyword: my\_search\_keyword,  
metadata\_for\_this\_keyword\_in\_file2, \_id:  
line\_number}, keyword: {my\_search\_keyword,  
metadata\_for\_this\_keyword\_in\_file2, \_id: line\_number}, ...],  
filename5: [{keyword: my\_search\_keyword,  
metadata\_for\_this\_keyword\_in\_file5, \_id: line\_number},  
{keyword: my\_search\_keyword, metadata\_for\_this\_keyword\_in\_file5, \_id:  
line\_number}, ...],  
}

Important point: There are a lot of collisions, keyword-wise.

At the moment I am using elasticsearch-py with the es.msearch function. My  
query is mentioned above. However this is quite slow, so I suspect that  
either my object design, mapping, or search strategy are wrong.

Would you have an insight to give? Thanks a lot!

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/39410826-61e2-4b31-8e17-72358c5a6ed6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/39410826-61e2-4b31-8e17-72358c5a6ed6%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:14am UTC](https://discuss.elastic.co/t/common-lines-for-multiple-files/18784/2 "2017-07-06T01:14:18Z")

</div>

Closed on July 6, 2017, 1:14am UTC