Architecting Connected Systems Blog
I ran into this the other day. I thought possibly something was broken on the OWA servers, but after reviewing, I didn't find anything wrong with OWA.
Amazingly, what I found was that the site url had been changed and the link to the site notebook stays the same! Attempting to click on the link will cause OWA to attempt to open the site notebook, but it will fail miserably. You will need to update your navigation node to point to your new site url.
Chris
There are some pretty sweet posts on how to do this here:
This biggest piece of it is that everything is done via the property bag of a term set and terms. You just need to know what the property names are, which include:
Chris
There are several questions about how to create and assign device channels programmatically. I just attempted to do the same and found the following:
To create a device channel, you can do this:
using (SPSite site = new SPSite("http://intranet.contoso.com"))
using (SPWeb web = site.RootWeb)
{
SPList list = web.Lists.TryGetList("Device Channels");
SPListItem li = list.AddItem();
li["Name"] = "Windows Phone";
li["ContentType"] = "Device Channel";
li["Active"] = true;
//alias can contain no spaces
li["Alias"] = "WindowsPhone";
li["Description"] = "The windows phone mobile channel";
li["Device Inclusion Rules"] = "Windows Phone";
li.Update();
}
If you look at how the master page settings page is laid out, it shows you all the device channels and any master pages that are tied to them. When you look at the code, you will find that the settings are converted into a MasterPageMappingsFile object (in the Microsoft.SharePoint.Publishing.Mobile namespace). It inherits from a base class called MappingsFile<T>, both of which are marked as internal and thus you cannot use them. When you review how it builds the list of mappings, it does so using a file called __DeviceChannelMappings.aspx that is stored in the "/_catalogs/masterpage/__DeviceChannelMappings.aspx". It looks like this:
<%@ Reference VirtualPath="~CustomMasterUrlForMapping0" %><%@ Reference VirtualPath="~CustomMasterUrlForMapping1" %><%@ Page Language="C#" Inherits="Microsoft.SharePoint.Publishing.Internal.WebControls.MappingsFileBasePage" %><html xmlns:mso="urn:schemas-microsoft-com:office:office" xmlns:msdt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882"><%@ Register Tagprefix="SharePoint" Namespace="Microsoft.SharePoint.WebControls" Assembly="Microsoft.SharePoint, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" %>
<head>
<meta name="WebPartPageExpansion" content="full" />
<!–[if gte mso 9]>
<SharePoint:CTFieldRefs runat=server Prefix="mso:" FieldList="FileLeafRef"><xml>
<mso:CustomDocumentProperties>
<mso:ContentTypeId msdt:dt="string">0x010100FDA260FD09A244B183A666F2AE2475A6</mso:ContentTypeId>
</mso:CustomDocumentProperties>
</xml></SharePoint:CTFieldRefs><![endif]–>
</head><body><mappings>
<mapping>
<channelAlias>WindowsPhone</channelAlias>
<masterUrl href="/_catalogs/masterpage/windowsphone.intranet.master" token="~sitecollection/_catalogs/masterpage/windowsphone.intranet.master" />
</mapping>
<defaultChannelMapping>
<siteMasterUrl token="~sitecollection/_catalogs/masterpage/seattle.master" href="/_catalogs/masterpage/seattle.master" />
<systemMasterUrl token="~sitecollection/_catalogs/masterpage/seattle.master" href="/_catalogs/masterpage/seattle.master" />
<alternateCssUrl token="" href="" />
<themedCssFolderUrl token="" href="" isthemeshared="false" />
</defaultChannelMapping>
</mappings></body></html>
Now that you know where the values are stored, you can programmatically modify the file using XML tools by downloading the files, changing it and then uploading it. It should be noted that the file format may change in the future and its most likely why they have locked it down from an object model code standpoint.
Enjoy!
Chris
So what the heck does that title mean? Well, for those of you that are not familiar with Search (which is a majority of you out there). The actual engine is called "Ceres". As in the dwarf planet in our solar system (Wikipedia). Keeping with universe terms, there is also a constellation of nodes in the search engine that make up the universe of bodies in the engine. If you take a minute, you will find several references to Constellation in the various classes inside the assemblies, but enough about the universe, what about extending the Ceres engine?
When it comes to search, many of you are already familiar with the various nodes types that make up the system. This includes:
But that's the easy part. and so are the architecture design aspects! This post will take you into a rabbit hole that you may never come out of! For the purpose of this post, we are interested in the Content Processing component AND the Query component. If you dive into the core of the Content Processing component you will find that it is made up of a series of flows. You can find the registered flows in the "C:Program FilesMicrosoft Office Servers15.0SearchResourcesBundles" directory, I will describe what these dlls are and how they get generated later in the post. Here is the full list (in the future I will update this post with what each of these flows purpose is):
The most important flow is the Microsoft.Crawlerflow. This flow is the master flow and defines the order of how all the other flows will be executed. A flow is simply an xml document that defines the flows and operators that should be executed on an item that is processed in the engine. The xml makes up an OperatorGraph. Each Operator has a name and a type attribute. The type attribute is made up of the namespace where the class lives that contains the code for the flow and then the name property of a special attribute added to the class. Each operator is deserialized into an instance of a class as the flow is "parsed". As you review the xml, you should see that the flow of the flow is determined by the "operatorMonkier" that has the name of the next operator that should be executed. The first part of this file looks like the following:
|
<?xml version="1.0" encoding="utf-8"?> <OperatorGraph dslVersion="1.0.0.0" name="" xmlns="http://schemas.microsoft.com/ceres/studio/2009/10/flow"> <Operators> <Operator name="FlowInput" type="Microsoft.Ceres.Evaluation.Operators.Core.Input"> <Operator name="Init" type="Microsoft.Ceres.ContentEngine.Operators.BuiltIn.Mapper"> |
As you can see the first operator that is executed is of the type "Microsoft.Ceres.Evaluation.Operators.Core.Input". This means that if you look in the "Microsoft.Ceres.Evaluation.Operators" namespace, you will find a class that is decorated like the following:
|
[Serializable, Operator("Input", MinInputCount=0, MaxInputCount=0)] public class InputOperator : TypedOperatorBase<InputOperator>, IMemoryUsingOperator, IOutputTypeConfigurableOperator { … |
You should note that the class is marked as serializable and that the Operator attribute has been added with the name "Input". Again, the combination of the namespace of the class and the name of the attribute are used to find the operator when the flow is executed.
One of the flows that I am most interested in is the Microsoft.CrawlerContentEnrichmentSubFlow. As some of you are aware, you can "extend", really don't like that word used in context of Content Enrichment now that I know how to do flow insertion, using a web service to add your own logic to create new crawled properties on items that pass through the engine. You can find more information about content enrichment and examples of using it at http://msdn.microsoft.com/en-us/library/jj163968.aspx. Now, Microsoft is going to tell you that this is the only supported way to extend the Ceres engine. And that is correct. What I am about to show you has never been done outside of Microsoft and if you venture down this path, you do so on your own. Anyway, the problem with the CES is that it is not flexible and it uses stupid old technology called web services. that means it is sending this big ugly xml around on the wire…not JSON. Bummer. That's not the only thing. When you look at the pipeline and all the things you are indexing, if you do not put a trigger on the CES, EVERY single item will be passed to your service. You would then need to have all kinds of logic to determine the type of the item, what properties exist on it by looping through them all and so many other weird bad things it just makes me cringe. Now, if you do put a trigger on it, you are now limiting yourself to implementing a very targeted set of logic. You have no ability to add more than one CES with different triggers with different logic. Huh? Big feature gap here. I'm not a fan. So for people that just don't want to multiple the time it takes to do a crawl by 100 to 1000x over because you implemented CES, you need a better option. A faster option. A more reliable and performant option. One that lives in the engine, not outside of it. If you want to know how to do this…keep reading!
Ok, so this is all simple so far. But how does one add a new flow to the Ceres engine and then implement your own Operators? Well, this is much more difficult than you think!
The first step is to create an operator class that inherits from TypedOperatorBase<T>. Where T is the class name. This is an abstract class and you must implement the method called ValidateAndType. You can see most of this in the operator example above. The next step is to add the Serializable and Operator attributes to the class. Ok, fair enough, now what do we do? If you look at the XML of an operator, you will see that you can implement properties and that those properties are simply deseriabled to the properties in the class. Ok, so add some properties. In my example, I create a class with one property:
|
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using Microsoft.Ceres.Evaluation; namespace CustomOperator public CustomOperator() [Property(Name="custom")] protected override void ValidateAndType(OperatorStatus status, IList<Microsoft.Ceres.Evaluation.Operators.Graphs.Edge> inputEdges) |
Ok, great. So now what do we do? Well, I wasn't sure if the system would just pick up the assembly from the GAC dynamically so I figured, let's just deploy the solution and try to add a flow with the operator in it. Here's how you do that:
Deploy the assembly to the GAC…easy, right-click the project, select "Deploy"
Next, create a new flow (xml file) that uses the operator:
|
<?xml version="1.0" encoding="utf-8" ?> <OperatorGraph dslVersion="1.0.0.0" name="CustomFlow" xmlns=" http://schemas.microsoft.com/ceres/studio/2009/10/flow"> <Operators> <Operator name="SubFlowInput" type="Microsoft.Ceres.ContentEngine.Operators.BuiltIn.SubFlow.SubFlowInput"> |
Connect to the ceres engine and try to deploy the flow:
|
Add-PsSnapin Microsoft.SharePoint.Powershell & "C:Program FilesMicrosoft Office Servers15.0SearchScriptsceresshell.ps1" Connect-System -Uri (Get-SPEnterpriseSearchServiceApplication).SystemManagerLocations[0] -ServiceIdentity contososp_farm Connect-Engine -NodeTypes InterActionEngine $flowname = "CustomFlow" Remove-Flow $flowname Get-Content C:CustomOperatorCustomOperator$flowname.xml | Out-String | Add-Flow $flowname Stop-Flow –FlowName $flowname –ForceAll |
You will get the following error that the system cannot find the Operator called CustomOperator.CustomOperator. Bummer. So that didn't work. So how do I "register" my operator with the engine? Well, it turns out that their is so much more that needs to be done than simply creating an operator class. You also need to create several other classes with special attributes attached to them. Sooo…here we go!
First off, you will need to create a Producer class. This producer is really the class that does all the work. The operator is really just a way to get some parameters into the producer. As you can see the Producer inherits from SingleOutputProducer<T>, where T is your operator class. Here is an example of the producer:
|
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using Microsoft.Ceres;
namespace CustomOperator private IUpdateableRecord holder; public override void ProcessRecord(IRecord record) base.SetNextRecord(record); |
Next up is to create a NamedPlugInSource. Operators are also called "PlugIns". These plugins must be registered with the system in order for you to use them. If you review all the operator assembiles, you will see that there is always some kind of *PlugInSource class that has the role of adding plugins to the Ceres core system. For my pluginsource, I only have one operator and that is my CustomOperator:
using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Ceres.CoreServices.Services;
using Microsoft.Ceres.CoreServices.Services.Node;
using Microsoft.Ceres.Evaluation.Operators;
using Microsoft.Ceres.Evaluation.Operators.PlugIns;
namespace CustomOperator
{
[DynamicComponent]
public class CustomPlugInSource : NamedPlugInSource<OperatorBase>
{
public static OperatorBase PlugIn1()
{
File.AppendAllText(@"c: empsearch.txt", "PlugIn1");
return new CustomOperator();
}
protected override void AddPlugIns()
{
File.AppendAllText(@"c: empsearch.txt", "AddPlugIns");
Func<OperatorBase> f = PlugIn1;
base.Add(typeof(CustomOperator),f);
}
}
}
Now that you have the plugin built. You will notice that is has been decorated with the "DynamicComponent" attribute. This is where the "Ah-ha" moment kicks in. By adding this attribute to the assembly, Ceres knows that is must start this as a managed component in the system. However, just simply deploying this to the GAC, will not get Ceres to recognize the assembly and load the components. We'll get to that soon, we still have lots more to talk about!
Next up is an Evaluator. An Evaluator is responsible for actually making the call to the producer. In my example I create a class that inherits from ProductEvaluator<T> where T is my CustomOperator. ProductEvaluator is again an abstract class with one method called GetProducer. You must instatiate your producer here and return it. There are many types of producers, but I have not had the time to document all of them as of yet. Soon though!
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Ceres.CoreServices.Services;
using Microsoft.Ceres.CoreServices.Services.Node;
using Microsoft.Ceres.CoreServices.Services.Container;
using Microsoft.Ceres.Evaluation.Processing;
using Microsoft.Ceres.Evaluation.Processing.Producers;
using Microsoft.Ceres.Evaluation.Operators;
using Microsoft.Ceres.Evaluation.DataModel;
using Microsoft.Ceres.Evaluation.Operators.Graphs;
namespace CustomOperator
{
public class CustomEvaluator : ProducerEvaluator<CustomOperator>
{
/*
protected override IRecordSet SetupOutput(Edge outputEdge, IList<IRecordSet> inputs)
{
CustomProducer cp = new CustomProducer();
return null ;
}
*/
protected override IRecordProducer GetProducer(CustomOperator op, Microsoft.Ceres.Evaluation.DataModel.Types.IRecordSetTypeDescriptor type, IEvaluationContext context)
{
return new CustomProducer(op, type, context);
}
}
}
Next on the list is an EvaluatorBinder. The evaluator binder is responsible for registering an operator with an evaluator. This class will inherit from AbstractEvaluatorBinder and need to implement the AddBoundOperators and BindEvaluator methods:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Ceres.Evaluation.Processing;
using Microsoft.Ceres.Evaluation.Operators;
namespace CustomOperator
{
public class CustomEvaluatorBinder : AbstractEvaluatorBinder
{
protected override void AddBoundOperators()
{
base.Add(typeof(CustomOperator));
}
public override Evaluator BindEvaluator(OperatorBase op, IEvaluationContext context)
{
if (op is CustomOperator)
{
return new CustomEvaluator();
}
return null;
}
}
}
Last on the list is the EvaluatorBinderSource. Similar to a PlugInSource, this will also be decorated with the DynamicComponent attribute which will instantiate and register the evaluators. Here is the binder source:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Ceres.CoreServices.Services;
using Microsoft.Ceres.CoreServices.Services.DependencyInjection;
using Microsoft.Ceres.CoreServices.Services.Node;
using Microsoft.Ceres.CoreServices.Services.Container;
using Microsoft.Ceres.Evaluation.Processing;
using Microsoft.Ceres.Evaluation.Operators;
namespace CustomOperator
{
[DynamicComponent]
public class CustomEvaluatorBinderSource : AbstractContainerManaged
{
[Exposed]
public IEvaluatorBinder CieEvaluatorBinder
{
get
{
this.exampleBinder = new CustomEvaluatorBinder();
return this.exampleBinder;
}
}
private CustomEvaluatorBinder exampleBinder;
}
}
You now have everything you need to add a new flow and operator to the Ceres engine! Kinda. If you deploy the code at this point, you will notice if you try to run the above install script, you will still get the same error! This is because the assemblies only get loaded when you restart the Host Controller service. NOTE: You can read more about the Host controller service in Randy Williams and I's MSPress book on SharePoint due out very soon. Ok, so re-start the service. Try the commands…NO GO…bummer. But I did everything you said Chris! Why doesn't it recognize my operator? Well…going back to my previous statement, Ceres nodes don't look at the entire GAC and analyze every class. That would be WAAAY to expensive. So it only does the one that it is told to do. This was the final magic step that I stumbled upon very luckily.
For each node that is started (via the NodeRunner.exe process), each one is fed its own configuration file that drives the WCF configuration. This file is stored in C:Program FilesMicrosoft Office Servers15.0SearchRuntime1.0
oderunner.exe.config. It is a very generic file, not much going on here. As part of the NodeController code, it will look for another file and feed some special values into the process in addition to the regular app.config file. These files are stored in the Ceres node directory which is in C:Program FilesMicrosoft Office Servers15.0DataOffice ServerApplicationsSearchNodes<RandomNodeID>. Each role that has been assigned to the server will get a directory under this path. Since most of what we are doing is related to the ContentProcessingComponent, let's look there first. If you open and explore this directory, what you will find is a nodeprofile.xml file. It looks like this…tell me if you notice anything interesting:
<?xml version="1.0" encoding="utf-8"?>
<NodeProfile xmlns="http://schemas.microsoft.com/ceres/hostcontroller/2011/08/nodeprofile">
<AutoStart xmlns="">true</AutoStart>
<Stopped xmlns="">false</Stopped>
<Modules xmlns="" />
<Properties xmlns="">
<Property Key="Managed.Node.Name" Type="string" Value="ContentProcessingComponent1" />
<Property Key="Managed.SystemManager.ConstellationName" Type="string" Value="A99B1A" />
<Property Key="Managed.Node.SystemName" Type="string" Value="A99B1A" />
<Property Key="Managed.SystemManager.ConstellationVersion" Type="int" Value="-1" />
<Property Key="Managed.Runtime.Version" Type="string" Value="1.0" />
<Property Key="Managed.Node.LocalSystemManager" Type="bool" Value="False" />
<Property Key="Managed.Node.ShutdownOnComponentFailed" Type="bool" Value="True" />
<Property Key="Managed.Node.ProcessPriorityClass" Type="string" Value="BelowNormal" />
<Property Key="Managed.Node.DynamicAssemblies" Type="string" Value="Microsoft.Ceres.ContentEngine.AnnotationPrimitives, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Bundles, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Component, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.DataModel.RecordSerializer, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Fields, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.LiveEvaluators, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.NlpEvaluators, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.NlpOperators, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Operators, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Operators.BuiltIn, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Parsing.Component, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Parsing.Evaluators, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Parsing.Operators, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Processing, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Processing.BuiltIn, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Properties, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.AliasLookup, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.RecordCache, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.RecordType, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Repository, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Services, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.SubmitterComponent, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Types, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Util, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.ContentEngine.Processing.Mars, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.DataModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.DataModel.Types, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.Engine, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.Engine.WcfTransport, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.Operators, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.Operators.BuiltIn, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.Operators.Core, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.Operators.Parsing, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.Processing, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.Processing.BuiltIn, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.Evaluation.Services, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.SearchCore.DocumentModel, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.SearchCore.Admin, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.SearchCore.ContentRouter, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.SearchCore.Services, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.SearchCore.Utils, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.SearchCore.Schema.SchemaCatalogProxy, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.SearchCore.Query.MarsLookupComponent, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.SearchCore.FastServerMessages, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.SearchCore.Schema.SchemaCatalog, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.NlpBase.AnnotationStore, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.NlpBase.Automata, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.NlpBase.Dictionaries, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.NlpBase.DictionaryInterface, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.NlpBase.Ese.Interop, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.NlpBase.RichFields, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.NlpBase.RichTypes, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.NlpBase.StringDistance, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.NlpBase.Transformers, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.NlpBase.IndexTokenizer, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.AnalysisEngine.Operators, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.SearchAnalytics.Operators, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;Microsoft.Ceres.UsageAnalytics.Operators, Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c;CustomOperator, Version=1.0.0.0, Culture=neutral, PublicKeyToken=7d300eac1b9f50c2" />
<Property Key="Managed.Node.SearchServiceApplicationName" Type="string" Value="14087e61-67e2-4245-b23d-0e52c6dcf704" />
<Property Key="Managed.Node.SystemDisplayName" Type="string" Value="0a1ee46f-59f2-49b7-bfca-bb4d20adaf1a" />
<Property Key="Managed.Node.BasePort" Type="int" Value="17042" />
<Property Key="Managed.Node.BasePort.4" Type="int" Value="17046" />
<Property Key="PortShared" Type="bool" Value="True" />
</Properties>
</NodeProfile>
If you guessed the "Managed.Node.DynamicAssemblies" property…then you are very smart! [:D] Yep…that is what we are looking for. Those are the only assemblies that will be loaded into the AppDomain. Only these assemblies will be interrogated for the DynamicComponent attribute. Great! So as you can see, I have added my CustomOperater assembly to the list. Let's try again and run the script. Dang it! NO GO! It still doesn't like my CustomOperator.CustomOperator operator! Grrr….so at this point, I'm really wondering if my assembly is getting laoded…after a browse in the ULS logs…I see these:
08/20/2013 22:32:32.18 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x0784 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomPlugInSource) : CustomPlugInSource moved from [Inactive] to [Configuring and eventSent=False]
08/20/2013 22:32:32.18 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x39E4 Search Search Platform Services aiyyo Verbose Microsoft.Ceres.CoreServices.Management.ManagementServer : Registered agent CustomOperator.CustomEvaluatorBinderSource.ComponentManager of type Microsoft.Ceres.CoreServices.Services.Container.IComponentManagerManagementAgent
08/20/2013 22:32:32.18 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x0784 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomPlugInSource) : CustomPlugInSource moved from [Configuring] to [Configured and eventSent=False]
08/20/2013 22:32:32.18 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x0784 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomPlugInSource) : CustomPlugInSource moved from [Configured] to [Resolving and eventSent=True]
08/20/2013 22:32:32.18 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x0784 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomPlugInSource) : CustomPlugInSource moved from [Resolving] to [Readying and eventSent=False]
08/20/2013 22:32:32.18 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x0784 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomPlugInSource) : CustomPlugInSource moved from [Readying] to [Ready and eventSent=True]
08/20/2013 22:32:32.18 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x0784 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomPlugInSource) : CustomPlugInSource moved from [Ready] to [Activating and eventSent=False]
08/20/2013 22:32:32.18 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x0784 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomPlugInSource) : CustomPlugInSource moved from [Activating] to [Active and eventSent=True]
08/20/2013 22:32:32.18 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x0784 Search Search Platform Services aiywj Medium ComponentManager(CustomOperator.CustomPlugInSource) : CustomOperator.CustomPlugInSource [Active] started
08/20/2013 22:32:32.18 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x0784 Search Search Platform Services aiyv9 Verbose ComponentManager(CustomOperator.CustomPlugInSource) : ***** QUEUESENTINEL finished task for CustomOperator.CustomPlugInSource: CustomOperator.CustomPlugInSource[Active]state Active
08/20/2013 22:32:32.20 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x39E4 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomEvaluatorBinderSource) : CustomEvaluatorBinderSource moved from [Inactive] to [Configuring and eventSent=False]
08/20/2013 22:32:32.20 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x39E4 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomEvaluatorBinderSource) : CustomEvaluatorBinderSource moved from [Configuring] to [Configured and eventSent=False]
08/20/2013 22:32:32.20 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x39E4 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomEvaluatorBinderSource) : CustomEvaluatorBinderSource moved from [Configured] to [Resolving and eventSent=True]
08/20/2013 22:32:32.20 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x39E4 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomEvaluatorBinderSource) : CustomEvaluatorBinderSource moved from [Resolving] to [Readying and eventSent=False]
08/20/2013 22:32:32.20 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x39E4 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomEvaluatorBinderSource) : CustomEvaluatorBinderSource moved from [Readying] to [Ready and eventSent=True]
08/20/2013 22:32:32.20 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x39E4 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomEvaluatorBinderSource) : CustomEvaluatorBinderSource moved from [Ready] to [Activating and eventSent=False]
08/20/2013 22:32:32.20 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x39E4 Search Search Platform Services aiywq High ComponentManager(CustomOperator.CustomEvaluatorBinderSource) : CustomEvaluatorBinderSource moved from [Activating] to [Active and eventSent=True]
08/20/2013 22:32:32.20 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x39E4 Search Search Platform Services aiywj Medium ComponentManager(CustomOperator.CustomEvaluatorBinderSource) : CustomOperator.CustomEvaluatorBinderSource [Active] started
08/20/2013 22:32:32.20 NodeRunnerContent1-0a1ee46f-59f (0x3994) 0x39E4 Search Search Platform Services aiyv9 Verbose ComponentManager(CustomOperator.CustomEvaluatorBinderSource) : ***** QUEUESENTINEL finished task for CustomOperator.CustomEvaluatorBinderSource: CustomOperator.CustomEvaluatorBinderSource[Active]state Active
Ok…they ARE being loaded. So what the hell is going on? Well…the clue WAS in the logs files. After running the PowerShell to attempt to add the flow, I noticed something. It was the name of the component that is actually being used to register a flow. Its called QueryProcessingComponent1. Well ok, so it seems that even though the content processing node does all the work, the query processing component manages all the registration of the plugins and operators. After going back to the node directory, I find the QueryProcessingComponent1 directory and find that it too has a NodeProfile.xml file. Bingo. Adding the assembly to the property and restarting the host controller one more time, I again attempt to add a custom flow, with a custom operator.
YYYYYEEEESSSS!!!! NO ERROR…………..I successfully inserted my flow and operator into the Ceres engine! Now, what part is missing? Well, even though the flow is now installed and working, it is not a part of the main flow ("Microsoft.CrawlerFlow"). I would need to insert the flow into that main file and then redeploy it. The main issue with that, is that not all of the operators are recognized by the system. Yeah, weird I know. This is part of the installationdeployment of the search service application and is there by default. if you ever want to make changes, you would need to add all the possible assemblies to the query processing component and then update the main flow.
In terms of debugging, you can attach to the NodeRunner.exe processes and debug your operator and evaluators. Easy.
Now for some clean up. All those bundles of flows at the top of this post. How did they get there? Well, what happens is each time you upload a flow, it will generate a new assembly with the flow added to it as a resource. If you were to reflect on any of the assemblies above, you can get the flow xml out of the assembly. But this is also easily done using the Windows PowerShell commands above.
I will be posted all the code for this project on code.msdn.microsoft.com. You can use it as a starting point for implementing your own flows and operators. But you are probably asking, why would I do something that is not supported. Well, its the same reason you want to keep your job. The customer wants high performance and needs to implement way more than the Content Enrichment Service can provide and saying no will stop any chance you have of completing an incredibly cool and awesome project. Now, why is this not supported if you CAN do it? Well, as you can see, it is VERY complex. Only a few people in the world are going to be able to build these, deploy them and successfully use them. So you are still asking yourself…why did you post this if we can't really do it…great question!
BECAUSE I WANT IT SUPPORTED. If we band together and find various use cases for doing this, the product team will have no choice but to train the Level I, II, and III support people on how to troubleshoot these. As of right now, it is simply unsupported from the fact that the support people don't even know what a flow and operator is when it comes to supporting SharePoint Search (update: met with the search team and the COE support team *is* familiar with flows so were one step closer to support). It would be my goal to get some ISVs to start playing around with creating custom flows and operators to make SharePoint Search a BEAST (not that is already isn't cuz its the best on the market right now, sorry Google Appliance but you suck big time)! So…there you have it. Do with it what you will, have fun, be smart and as always…enjoy!
Chris
So what have I been working on for the past two weeks? Well, other than consulting clients, books and working on my garden, I have also been involved with the Microsoft Learning SharePoint 2013 Advanced Development Microsoft Official Curriculum (MOC) course 20489 that will be available later in the year (sorry no link just yet but it will be here when it is released). I was able to finish up two chapters on Search quickly as that is one of my main fortes, but then decided to take what I though was the middle of two Business Connectivity Services (BCS) chapters. For those of you not familiar with BCS, you can find a great overview here. It turns out, the module was the hardest one! Why? Because it covers things that no one has ever done before (outside of the product team that is).
So what is the big deal about what I worked on? You are probably saying to yourself…BCS has been around for a while right? Well, yes this is very true, and there are several great posts about how to expose external data using SharePoint Designer and Visual Studio using the various BDC model types (Database, WCF, .NET Connectivity and Custom). You can also find how to implement stereotyped methods that support CRUD methods and search indexing (link here). Given all that content, there were a game changing set of features that were added to BCS in SharePoint 2013 that add a whole new level of complexity. These features include:
There are plenty of posts on OData in general (this one from MSDN is pretty awesome if you are just getting started) and a few posts on how to setup a BDC OData model. And although my fellow SharePoint MVP Scot Hillier did a presentation on the subscriber model at the last SharePoint Conference it was only in context of a database model. When it comes to integrating the two features (OData and the subscriber methods) together, that is where a massive black hole exists and is the focus of this blog post.
The first step to getting this whole thing to work is to create an OData service. This is very simple with the tools provided by Visual Studio and steps to do this are provided in this MSDN post
The next step is to build your basic BCS model using the new item template wizard provided in Visual Studio 2012. This has also been nicely blogged about by several of my colleagues and does have an article on MSDN. The important thing to note about the MSDN article I reference is that it is using an OData feed that is hosted by http://services.odata.org. Since you do not own this service, you will not be able to extend it to implement the subscribe and unsubscribe methods that I discuss later in this post. Therefore, you can follow the steps in the article, but use a local instance of your OData service.
Once the service has been generated, you must add some supporting methods to your OData service to accept data from SharePoint when a subscription occurs. There are some gotchas to this. Currently there is no real guidance on how to set this up properly. The little that does exist will point you to mixed signals as to how to successfully setup the communication layers. In my example below, you will see that I am using a GET for the http method. This was the only successful way that I was able to get the method parameters to populate in the web method in the OData service. As you will see later, there are also some very important BDC method properties that must be set in order for all of this to work:
|
[WebGet] // Generate a new Guid that will function as the subscriptionId. if (DeliveryURL == null || EventType == null || EntityName == null || SelectColumns == null) // This sproc will be used to create the subscription in the database. string sqlConn = "Data Source=.;Initial Catalog=Northwind;uid=sa;pwd=Pa$$w0rd"; // Create connection to database. cmd.Parameters.Add(new SqlParameter("@SubscriptionId", subscriptionId)); try return subscriptionId; [WebGet] string sqlConn = "Data Source=.;Initial Catalog=Northwind;uid=sa;pwd=Pa$$w0rd"; // Create connection to database. cmd.Parameters.Add(new SqlParameter("@SubscriptionId", subscriptionId)); try |
On the BCS side, you need to add the stereotyped methods that will send the data to the web methods you just created in the last step. This includes the EntitySubscriber and EntityUnsubscriber methods. First the let's review the EntitySubscriber method. In the table below, you will notice that I am sending the OData web method parameters in the querystring. You can use the '@' parameter notation just like in regular BDC Models to token replace the values. You should also use HTML encoded '&' to signify the ampersand (this was one of the things that took me a while to figure out). Notice the various method parameters. They include:
Here is the working method XML for the Subscribe method:
| <Method Name="SubscribeCustomer" DefaultDisplayName="Customer Subscribe" IsStatic="true"> <Properties> <Property Name="ODataEntityUrl" Type="System.String">/Subscribe?DeliveryURL='@DeliveryURL'&EventType=@EventType&EntityName='@EntityName'&SelectColumns='@SelectColumns'</Property> <Property Name="ODataHttpMethod" Type="System.String">GET</Property> <Property Name="ODataPayloadKind" Type="System.String">Property</Property> <Property Name="ODataFormat" Type="System.String">application/json;odata=verbose</Property> <Property Name="ODataServiceOperation" Type="System.Boolean">false</Property> </Properties> <AccessControlList> <AccessControlEntry Principal="NT AuthorityAuthenticated Users"> <Right BdcRight="Edit" /> <Right BdcRight="Execute" /> <Right BdcRight="SetPermissions" /> <Right BdcRight="SelectableInClients" /> </AccessControlEntry> </AccessControlList> <Parameters> <Parameter Direction="In" Name="@DeliveryURL"> <TypeDescriptor TypeName="System.String" Name="DeliveryURL" > <Properties> <Property Name="IsDeliveryAddress" Type="System.Boolean">true</Property> </Properties> </TypeDescriptor> </Parameter> <Parameter Direction="In" Name="@EventType"> <TypeDescriptor TypeName="System.Int32" Name="EventType" > <Properties> <Property Name="IsEventType" Type="System.Boolean">true</Property> </Properties> </TypeDescriptor> </Parameter> <Parameter Direction="In" Name="@EntityName"> <TypeDescriptor TypeName="System.String" Name="EntityName" > <DefaultValues> <DefaultValue MethodInstanceName="SubscribeCustomer" Type="System.String">Customers</DefaultValue> </DefaultValues> </TypeDescriptor> </Parameter> <Parameter Direction="In" Name="@SelectColumns"> <TypeDescriptor TypeName="System.String" Name="SelectColumns" > <DefaultValues> <DefaultValue MethodInstanceName="SubscribeCustomer" Type="System.String">*</DefaultValue> </DefaultValues> </TypeDescriptor> </Parameter> <Parameter Direction="Return" Name="SubscribeReturn"> <TypeDescriptor Name="SubscriptionId" TypeName="System.String" > <Properties> <Property Name="SubscriptionIdName" Type="System.String">SubscriptionId</Property> </Properties> </TypeDescriptor> </Parameter> </Parameters> <MethodInstances> <MethodInstance Type="EventSubscriber" ReturnParameterName="SubscribeReturn" ReturnTypeDescriptorPath="SubscriptionId" Default="true" Name="SubscribeCustomer" DefaultDisplayName="Customer Subscribe"> <AccessControlList> <AccessControlEntry Principal="NT AuthorityAuthenticated Users"> <Right BdcRight="Edit" /> <Right BdcRight="Execute" /> <Right BdcRight="SetPermissions" /> <Right BdcRight="SelectableInClients" /> </AccessControlEntry> </AccessControlList> </MethodInstance> </MethodInstances> </Method> |
Next is the unsubscribe method, notice how SharePoint must pass back the subscription id that lives in the external system. The name of the SubscriptionIdName property will always be SubscriptionId. This subscription id must be saved somewhere, but the question is…where?:
| <Method Name="UnSubscribeCustomer" DefaultDisplayName="Customer Unsubscribe"> <Properties> <Property Name="ODataEntityUrl" Type="System.String">/UnSubscribe?SubscriptionId='@SubscriptionId'</Property> <Property Name="ODataHttpMethod" Type="System.String">GET</Property> <Property Name="ODataPayloadKind" Type="System.String">Property</Property> <Property Name="ODataServiceOperation" Type="System.Boolean">false</Property> </Properties> <AccessControlList> <AccessControlEntry Principal="NT AuthorityAuthenticated Users"> <Right BdcRight="Edit" /> <Right BdcRight="Execute" /> <Right BdcRight="SetPermissions" /> <Right BdcRight="SelectableInClients" /> </AccessControlEntry> <AccessControlEntry Principal="Contosodomain users"> <Right BdcRight="Edit" /> <Right BdcRight="Execute" /> <Right BdcRight="SetPermissions" /> <Right BdcRight="SelectableInClients" /> </AccessControlEntry> </AccessControlList> <Parameters> <Parameter Name="@SubscriptionId" Direction="In"> <TypeDescriptor Name="SubscriptionId" TypeName="System.String"> <Properties> <Property Name="SubscriptionIdName" Type="System.String">SubscriptionId</Property> </Properties> </TypeDescriptor> </Parameter> </Parameters> <MethodInstances> <MethodInstance Name="UnSubscribeCustomer" DefaultDisplayName="Customer Unsubscribe" Type="EventUnsubscriber" Default="true"> <AccessControlList> <AccessControlEntry Principal="NT AuthorityAuthenticated Users"> <Right BdcRight="Edit" /> <Right BdcRight="Execute" /> <Right BdcRight="SetPermissions" /> <Right BdcRight="SelectableInClients" /> </AccessControlEntry> </AccessControlList> </MethodInstance> </MethodInstances> </Method> |
Now that those items are setup, you need to deploy your BCS model and set permissions. This is very common activity so I'll skip the details in this blog post, however I will say that it is annoying that the user that uploads the model is not automatically added (or have an option somewhere to add them on the import page) as a admin with permissions to the model and methods [:(]
Now that the model is deployed, the next step is to enable a feature that enables subscription support, which brings us back to the question brought up before…where does SharePoint store the subscription id of the external system? A list of course! To create this list, there are two features of which you can enable. One is called BCSEvents, the other is called ExternalSubscription. The funny thing about these two features and their relationship is that the BCSEvents feature is made up of a feature activation receiver. That receiver has only one goal: To activate the ExternalSubscription feature. In addition to this interesting design, you will find that the BCSEvents is a hidden feature whereas the ExternalSubscription feature is actually visible in the web features settings page. What does the ExternalSubscription feature do? It creates our list of course! This list is called "External Subscriptions Store". This is a hidden list and can be unhidden using PowerShell, but it exists in the "_private/ExtSubs" folder and has no views from which you can view the data, so again Windows PowerShell is the way to go if you want to see what lives in the list. Here is a screen shot of the columns of the list:
Next you need to create a subscription. This can be done explicitly or implicitly. The explicit way is to make a call to the entity's subscribe method as shown here (as previously pointed out above, the notification callback url is ignored in an OData Model):
|
function SubscribeEntity() { //these are the helper methods and variables var context; // This code runs when the DOM is ready and creates a context object which is needed to use the SharePoint object model entity = web.getAppBdcCatalog().getEntity("NorthwindModel", "Customers"); lob = entity.getLobSystem(); lobSystemInstances = lob.getLobSystemInstances(); context.executeQueryAsync(GetLobSubscribesystemInstance, failmethod); // Initialize the LobSystemInstance. |
Subscriptions can be one of three types (you can learn more about event types here):
Note that there is no event type that supports ItemAdding, ItemUpdating or ItemDeleting. This means you cannot cancel the insertion, update or deletion in the external source, you can only expect to receive notification after the event has occurred.
The implicit way is to create an alert or to setup an event receiver. This means you should setup an external list pointing to your OData model. You can then use the ribbon to create an alert which will in turn execute the web method call to create the subscription. Note that you must setup your outgoing email settings on your farm, or the alert ribbon button will not display! If an error occurs when creating an event receiver, you will be passed the web method exception from your OData service. This can be very helpful for troubleshooting.
NOTE: When you create an external list, there are several items that seems to get cached in the external list's properties that will require you to delete the list and then re-create it. This means that as you are testing your solution, you should create a Windows PowerShell script that will remove your BDC model, re-deploy it, remove the external list and then add it back.
Once this has all been completed, you can now start telling SharePoint that things have changed. As much work as we have done to this point, it is really rather simple compared to the amount of work needed for this component of the eco-system. There are several approaches you could take to do this:
Each of these have advantages and drawbacks in terms of time and complexity. No matter what, you need some component that will tell SharePoint that something has changed. In the code samples I provide, you have a simple console application that will allow you to send the notification to SharePoint for testing purposes.
So now that you have something that can send a message to SharePoint, what does that message look like? This process of communication is un-documented anywhere, until now, and is the real meat of this post! It turns out that there are two message parsers that come out of the box with SharePoint. These include an IdentityParser and an ODataEntryContentNotificationParser. The difference between the two is that one only tells SharePoint that a set of identities has changed and the other actually can pass the changed properties of the item to SharePoint. Both requires a completely different style of ATOM message to be sent.
In the case of the IdentityParser, it is looking for a message that looks like the code snippet below. This particular piece of XML must have a valid XPath to "/a:feed/a:entry/a:content/m:properties/b:BcsItemIdentity". If it does not, then any call to "retrieve the item" in your event receiver will fail. The message will be received and the event receiver will execute as long as you don't make calls to the various properties that will not be available without the id. You should also be aware that none of the other items that live outside of the XPath are ever looked at and can be anything you like as they are not validated or used:
| <?xml version="1.0" encoding="utf-8" standalone="yes"?> <feed xml:base="http://services.odata.org/OData/OData.svc/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns:b="http://schemas.microsoft.com/bcs/2012/" xmlns="http://www.w3.org/2005/Atom"> <entry> <title type="text">Customers</title> <id>http://www.northwind.com/customers</id> <author> <name>External System</name> </author> <content type="application/xml"> <m:properties> <b:BcsItemIdentity m:type="Edm.String"><CustomerID>ALFKI</CustomerID></b:BcsItemIdentity> <d:Name>Customer</d:Name> </m:properties> </content> </entry> </feed> |
In the case of the ODataEntryContentNotificationParser, you must pass an XML message that has a valid XPath to "/a:entry/a:link/m:inline/a:entry". The XML node in this XPath must itself be a valid ATOM message. Again, everything that is outside the XPath seems to be ignored and only the embedded ATOM Message is used:
| <?xml version="1.0" encoding="utf-8" standalone="yes"?> <atom:entry xml:base="http://sphvm-92723:90/WcfDataService2.svc" xmlns="http://www.w3.org/2005/Atom" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns:atom="http://www.w3.org/2005/Atom"> <atom:category term="NorthwindModel.EntitySubscribe" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme"/> <content type="application/xml"> <m:properties> <d:SubscriptionId m:type="Edm.Int32">1</d:SubscriptionId> <d:EntityName>Customers</d:EntityName> <d:DeliveryURL>{11}</d:DeliveryURL> <d:EventType m:type="Edm.Int32">{12}</d:EventType> <d:UserId m:null="true" /> <d:SubscribeTime m:type="Edm.Binary">AAAAAAAABE4=</d:SubscribeTime> <d:SelectColumns>*</d:SelectColumns> </m:properties> </content> <id>OuterId</id> <atom:id>http://sphvm-92723:90/WcfDataService2.svc/EntitySubscribes(1)</atom:id> <atom:link href="EntitySubscribe(1)" rel="self" title="EntitySubscribe"/> <atom:link href="Customers(2147483647)" rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/customers" type="application/atom+xml;type=entry"> <m:inline> <entry xml:base="http://sphvm-92723:90/WcfDataService2.svc/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom"> <id>http://sphvm-92723:90/WcfDataService2.svc/Customers('57849')</id> <title type="text" /> <updated>2012-04-30T11:50:20Z</updated> <author> <name /> </author> <link rel="edit" title="Customer" href="Customers('57849')" /> <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Orders" type="application/atom+xml;type=feed" title="Orders" href="Customers('57849')/Orders" /> <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/CustomerDemographics" type="application/atom+xml;type=feed" title="CustomerDemographics" href="Customers('57849')/CustomerDemographics" /> <category term="NorthwindModel.Customer" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" /> <content type="application/xml"> <m:properties> <d:CustomerID>{0}</d:CustomerID> <d:CompanyName>{1}</d:CompanyName> <d:ContactName>{2}</d:ContactName> <d:ContactTitle>{3}</d:ContactTitle> <d:Address>{4}</d:Address> <d:City>{5}</d:City> <d:Region>{6}</d:Region> <d:PostalCode>{7}</d:PostalCode> <d:Country>{8}</d:Country> <d:Phone>{9}</d:Phone> <d:Fax>{10}</d:Fax> </m:properties> </content> </entry> </m:inline> </atom:link> <title>New Customer entry is added</title> <updated>2011-07-12T09:21:53Z</updated> </atom:entry> |
In addition to the two out of the box parsers, there is a setting that specifies "Custom". By implementing our own NotificationParser, we can format the message in a much more simple and efficient way such as JSON. The main method to implement is the ChangedEntityInstance method. As part of this parser, you will be passed the message byte array in the initialization and it would be your responsibility to parse the message and pass back the entity instance.
| public abstract class NotificationParser { // Methods protected NotificationParser() { } public void Initialize(NameValueCollection headers, byte[] message, IEntity entity, ILobSystemInstance lobSystemInstance) // Properties public abstract Identity ChangedItemIdentity { get; } |
Summary:
Now that you have all the pieces, you can download the code I have placed on the code.msdn.microsoft.com site here. This code has a BCS OData model fully working with the subscriber methods. As code generation techniques have become more common place, OData layers generated via Visual Studio are more common as well. It will be well worth implementing these new BDC method stereotypes in your OData model and in your OData services to provide the ability to be notified when data changes in your remote systems!