Nest ingest a list of attachments

[gutospinelli]

Hi everyone!

I am trying to index an object called "News" who has a list of "NewsFiles" using NEST/C#

The simplified code for my news class follows:

public class News
{
(...)
public List<NewsFiles> Files { get; set; }
}    

and the simplified code of my NewsFiles class also follows:

public class NewsFiles 
{
        public int Id { get; set; }
        public string Content { get; set; } 
        public Attachment File{ get; set; }
}

*the attachment type above refers to the Nest.Attachment class.

I can't index an instance of News class containing one or more file attachments, using the pipeline I've created (possibly the source of the error)

The code I use to index the object follows:

var response = nest.ElasticClient.Index(news, p => p
                        .Index("intranet")
                        .Pipeline("attachmentsnewsfiles")
                    );

The pipeline used:

ElasticClient.Ingest.PutPipeline("attachmentsnewsfiles", p => p
              .Description("news files attachment pipeline")
              .Processors(pr => pr
                .Foreach<News>(fe => fe
                        .Field(f => f.Files)
                        .Processor(fep => fep
                            .Attachment<NewsFiles>(a => a
                                .Field(f => f.Content)
                                .TargetField(f => f.File)
                            )
                        )
                    )
                )
            );

and the error I get : "field [content] not present as part of path [content]"

Can someone give me a hand? Thanks a lot!

[dadoonet (David Pilato)]

Welcome!

I'm not a .Net dev so I'm not sure my answer will be accurate or not. If not, it would be great if you could translate your code to a recreation script that we can run in the Kibana dev console.

But, I believe that you are putting the "binary" content in File field and you want to extract the text to Content field.
If so, I believe you should write:

.Field(f => f.File)
.TargetField(f => f.Content)

[gutospinelli]

Hi friend!

First of all, thank you for your time and your response.

Unfortunatlly it didn't worked. The error only changed
from:
"field [content] not present as part of path [content]"
to:
"field [attachment] not present as part of path [attachment]"

In fact I believe that the following relation is correct, since it works when I map in a 1:1 relation (one Attachment attribute directly in one News class).

.Field(f => f.Content)
.TargetField(f => f.File)

My guess is that I am missing something in the use of the Foreach, since aparently the request is trying to find a path to "attachment" and the Field can't be found. Since there's a field called attachment in the NewsFiles class, maybe the processor is looking on the News class, not on the NewsFiles.

Hope my poor explanation is clear enough.

[dadoonet (David Pilato)]

Could you provide a full recreation script as described in About the Elasticsearch category. It will help to better understand what you are doing. Please, try to keep the example as simple as possible.

A full reproduction script will help readers to understand, reproduce and if needed fix your problem. Then we can try to translate the solution to the .Net world.

You can use the _simulate API for that.

[gutospinelli]

Problem Solved!

ElasticClient.Ingest.PutPipeline("attachments", p => p
              .Description("Document attachments pipeline")
              .Processors(pp => pp
                .Foreach<News>(fe => fe
                    .Field(f => f.Files)
                    .Processor(fep => fep
                        .Attachment<Attachment>(a => a
                            .Field("_ingest._value.content")
                            .TargetField("_ingest._value.file")
                        )
                    )
                )
              )
            );