I'm getting an error while using IndexMany for indexing documents as attachments. Yesterday, When I published the application on server it worked fine. But when I try to run it from visual studio it is throwing this error. Can someone help me in solving this.
string path = "D:\ElasticShare\";
List<string> filesList = new List<string>();
try
{
string[] filePath = Directory.GetFiles(path, "*", SearchOption.AllDirectories);
foreach (string f in filePath)
{
filesList.Add(f);
}
}
catch (Exception ex)
{
Response.Write(ex.Message.ToString());
}
try
{
foreach (string file in filesList)
{
Attachment attach = new Attachment
{
Name = Path.GetFileNameWithoutExtension(file),
Content = Convert.ToBase64String(File.ReadAllBytes(file)),
ContentType = Path.GetExtension(file)
};
var doc = new Document()
{
Id = counter,
Title = Path.GetFileNameWithoutExtension(file),
FilePath = Path.GetFullPath(file), //added to get the path of the file
File = attach
};
list.Add(doc);
counter++;
}
}
catch (Exception e)
{
Response.Write(e.Message.ToString());
}
try
{
var response = client.IndexMany(list, "reader"); //throwing an error at this place.
}
catch (Exception ex)
{
Response.Write(ex.Message.ToString());
}
No, I'm adding all the documents to a list and using IndexMany, I'm indexing the list.
Each documents refers to an attachment.
In this example I have 20 documents.
I was able to index 100 documents yesterday and also even now using the application that is published on server which is the same code as now. But all of sudden, it started throwing this error.
I think the error is happening on the elasticsearch side. I have attached a small snap of the error that is thrown. You can please have a look at it once.
I think to get log details, I need to enable slow logging right? (correct me if I'm wrong. I have never tried logging. so not sure if I'm thinking in the right direction or not).
But I can see elasticsearch.log and elasticsearch_index_indexing_slowlog.log and elasticsearch_index_search_slowlog text files. The last two files are empty where as the file is showing something like this. (Not sure if you are looking for the same or not. Sorry)
elasticsearch.log is the right one. But please prefer copying the logs then doing some screenshots. </> helps to format.
So here, elasticsearch does not send any out of memory exception.
Definitely this is something which is happening on the client side.
Either add more memory to your process/machine (I don't know how you do this in .Net world) or reduce the size of the document you are sending to elasticsearch.
Sorry for that. I had to post the screenshot because of the limitation of words in the reply body.
Either add more memory to your process/machine (I don't know how you do this in .Net world) or reduce the size of the document you are sending to elasticsearch.
Yeah will try to run the same code on different machine and check if it is working or not.
But when we are indexing individual document , will reducing the size help?
Just started working on it. Your application scope seems to be very big than my requirement. So I'm just taking hints from your application scope and creating a small one. And moreover I didnt understand how /from where to start with from your project . Any pointers on how to use it in the existing project? TIA
The same code is working perfectly fine on a console application.
I even compared the content using text compare tool and everything is same except the index name.
This is totally confusing now.
Oh okok. Thanks @dadoonet Will check on some other device then.
And one more thing, Will the performance of the query depend on the platform we are working on? Because I think console application is taking more time than the normal web application to fetch results. Anything to work on to improve the performance? (Will the number of shards and replicas affect the performance? because I'm going with default number 5 for each which I dont use at all.)
I think as you said its the problem with the IDE.
Because I was able to index 20 documents. But when i have some 50 documents the error was thrown near client.IndexMany(list,"indexName").
Still facing this issue. Don't understand how to solve the problem.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
. Any pointers on how to use it in the existing project? TIA
