XML and .NET

Elastic Stack Elasticsearch
1

I am trying to send data to Elasticsearch using NEST and ASP.NET Core that contains mainly text and XML data.

My class is defined as follow:

public class TestData
 {
        public string text1{ get; set; }
        public string text2{ get; set; }
        public string text3{ get; set; }
        public Object obj { get; set; }
  }

Here's the quick sample to populate data:

public class myXML
{
     public string field1 { get; set;}
     public string field2 { get; set;}
}

public static List<TestData> SampleData()
{
           XmlSerializer xmlobj = new XmlSerializer(typeof(myXML));
            myXML data = new myXML();
            data.field1= "field1";
            data.field2 = "field2";

            var xmlString = new StringWriter();
            xmlobj .Serialize(xmlString, data);

            return new List<TestData>
            {
                new TestData { text1="text1", text2 = "text2", text3 = "text3", obj= xmlString }
            }
}

I am inserting the data as follow

var Samples = SampleData();
var result = client.IndexManyAsync(Samples );

I do have the index created just fine with all the fields in TestData mapped correctly. However, when i create an index pattern with Kibana, I do not see the data being inserted.

I was hoping that ElasticSearch would also parse the XML in separate fields.

Are there any examples of inserting XML data to Elasticsearch using .NET with either NEST or Elasticsearch.net low level API ?

2

Elasticsearch won't parse a string field containing XML on an incoming JSON document into separate document fields. You might be able to do this using a community XML ingest processor plugin. I'd advise adding the fields that are part of the XML as fields on the document instead though.

3

Thanks for your response. Does the plugin only parses XML files only ? I have xml data coming in as a stream so does the plug requires dumping the data to disk first before being sent to elasticsearch ?

By adding the fields that are part of the XML, do you mean parsing the XML and creating the fields ahead as part of the index ?

4

The plugin is a community plugin and I've not used it personally, so can't really speak to its configurability. From reading the README on the repository though, it extracts fields and values from attributes and contents of the XML, which are passed to Elasticsearch as a field in JSON. The typical usage with the .NET client would be to construct a POCO representing the document to send to Elasticsearch, where one of the fields contains an XML string for the XML ingest processor to target on ingest.

What I mean is that it looks like you have complete control over the data you'd like to send to Elasticsearch - you could parse the fields you'd like from XML and include these as fields in a JSON document to send to Elasticsearch.