SharePoint 2013 REST endpoints and more…Microsoft "REST SOA"

I'm yet again, building what will surely be the best development course on SharePoint 2013 (until someone copies it) in the world.  As part of the course, their is a module on making calls to SharePoint REST-ful services.  REST was available in SharePoint 2010 and could be made via calls to the various "svc" end points.  This wasn't ideal at all.  The reality is that no one in the SharePoint team likely knew much about how Facebook built their api(s).  Single end point, multiple paths in that single end point.  So let by-gones be by-gones…enter SharePoint 2013 and Microsoft with "ODATA".

The first thing to note.  The old REST end points (listdata.svc and excelrest.aspx, et all) are still there (with a few rare exceptions, backwards compatibility has always been a major strength of the SharePoint platform).  But in 2013, these are obsolete.  You should now use the new "/_api/" end point.  This end point will have many sub paths off of it to do what you need to do.  These include (from the public Microsoft TechNet Documentation):

  • site
  • web
  • search
  • contextinfo
  • lists

Some of the pages on technet allude to others (such as on this page which eludes to a BCS end point, but that really is BS…it don't exists ladies and germs).  Unfortunately, this documentation was based on earlier builds and is incorrect.  The end points have changed in the Beta release.  As part of building the course, I want a COMPLETE coverage of all the end points.  So I wondered where these end points might be defined.  Since _api seems to be a virtual directory, you would think it would be in IIS or the web.config files.  Nope:

So as an experienced developer, I knew the answer would be in the HttpModules.  Looking at the modules, only one really does everything and that is the SPRequestModule:


 

In reviewing the SPRequestModule code with Reflector.  You will find that deep down the "_api" directory actually gets mapped to client.svc.  An AH-HA moment!  In reviewing the code of the client.svc, you will find that there are two methods in this class:

 

The one we are interested in is ProcessRestQuery(), it contains a call to the ProcessQuery method of the RestService class which makes a call to ClientServiceHost.CreateServiceHost():

You will see a call to GeServiceHostTypes():

 

This is where things get interesting.  You will now see a call to the "microsoft.sharepoint.client/serverRuntime" entry in the web.config file:

Looking at this section in the web.config file, it points to a class called Microsoft.SharePoint.Client.SPClientServiceHost.  This class actually exists in the Microsoft.SharePoint.dll in SP15.  Looking at that class you will find a couple of interesting methods.  Mainly the GetRegisteredClientCallableProxyLibrariesFromSetupDirectory and its overrides.  It should really only have one method, but because of the beta, it has two.  If you're smart enough, you will be able to figure out why.  Looking at the non parameter based method, you see a call to "configclientcallable" directory.  It is rounding up all the "proxylibrary.*.xml" files.  This directory exists in {SPRoot}ConfigClientCallable.  In looking at this directory, you hit the JACKPOT!

It has several xml files.  If you open the files you will find a set of xml files that define pointers to ServerStubs:

 

Looking at one of the files (ProxyLibrary.stsom), you find a pointer to an assembly:

 

So let's find this ServerStub assembly…oh wait…dotNET 4.0 keeps the GAC in a new location (C:windowsmicrosoft.netassembly).  This is where you need to look.  You will only find three stubs…but wait…there are 14 files?!?  Yeah…There are 11 missing ServerStub files in Beta!!  Hint hint hint…you can figure out why their were two method calls above and what we will have at RTM when it comes to RESTful services.   Some good stuff for SURE, but here are two biggies:

  • Taxonomy
  • Translation

Now…is that where this post stops?  Hell no.  How does the API know about the actual URLS off of "_api"?  This is where Microsoft SharePoint Team did a great job!  I have used these patterns many times but this is the first time I have seen Microsoft implement them.  As such is the pattern with Microsoft…wait to see if everyone likes it, then implement it.  Of course, that doesn't win you market share (look at the fight with Apple), nor does it give you a marketing brand that is deemed "Innovative", but rather "Follower".  I digress. How does the SPClientServiceHost know what to respond too?  As part of building a ServiceHost, you have to build a ProxyMap.  This knows to map the path to the actual class (and its methods and properties of which you can call).  In order to do this, it actually looks at custom attributes of each of the loaded assemblies.  It looks for three assembly attributes:

  • UrlSegmentAliasMap
  • ClientNamespaceMap
  • MetadataProviderType

The most important and one you see most documentation about on TechNet (even though most of it is wrong) is the "UrlSegmentAliasMap".  This is not defined on the ServerStubs!  It is defined across multiple assemblies of SharePoint.  If you look at the Microsoft.SharePoint.dll, you will find these custom attributes:

 

  • web
  • site
  • lists
  • contextinfo
  • navigation
  • events

You will also see some "ClientNamespaceMap" attributes:

  • Microsoft.ShraePoint.BusinessData.MetadataModel.ClientOM
  • Microsoft.ShraePoint.BusinessData.MetadataModel
  • Microsoft.SharePoint.ApplicationPages.ClientPickerQuery

Each of these will have a mapping to a particular class in some assembly.  The clientnamespacemap ones are mainly to the ServerStub files.  The regular ones map to Object Model classes.  So then begs the question…now that I know ALL the REST api paths…how to I get the methods?  Looking at each of the pointed too classes, you will find an attribute on each method called "ClientCallable":

 

This means you can call this method in your REST api calls!  The parameters can be passed with the matching method parameter names in the REST service call!

So what does all this mean?  It means that the SharePoint team actually built something that can be dynamically scalable!  They can simply add these proxy xml files to point to stub assemblies, and/or add the custom attributes to methods that are ready to be exposed to our client side code with complete EASE.&nbs
p; This has to be one of the best flexible implementations of REST SOA (I am going to coin that phrase by the way).

You now have the power…do with it what you will.

Enjoy,
Chris

SharePoint 2010 REST-like Services – Getting them to work!

Given that it is only beta, I still have to give SharePoint 2010 some leaway yet, until RTM comes out, who knows if
these things will be the same or not. 

Anyone that has programmed Facebook API, knows they have one of the most intense and best documented REST-ful
service layers on the planet!  I have many applications that use their API and the OpenID login IP-STS.  If you were to
look at Facebook and their implementation, you would know it is the BEST way to implement REST services.  I give them
full credit for the revolution that has started!

Given that, let's look at SharePoint 2010 and its REST-like services.  Here's my issues:

  1. Performance sucks
    1. Implementation should have been with HttpHandler vs "Accept" header in the HTTP request with the requested
      format you want returned.  This adds 23 bytes to every request, oh and wait, it gets better, for some reason you
      need a "if-match" header too, another 15 bytes.  Facebook simply does this in the request URL listdata.atom or
      listdata.xml – wow, how easy is that?
    2. My favorite tweet about this REST Implementation: "Oh my god, they killed REST, you bastards!"
  2. Documentation so far is poor
    1. Take for instance the "POST" for adding a new item, the documentation is wrong, it says to do this:

POST /_vti_bin/ListData.svc/Employees HTTP/1.1
Accept: application/atom+xml
Content-Type: application/atom+xml
Host: www.contoso.com
Content-Length: ###
Expect: 100-continue
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"
xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
xmlns="http://www.w3.org/2005/Atom">
<category scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" term="Microsoft.SharePoint.Linq.EmployeesItem" />
<title />
<author>
<name />
</author>
<updated>2009-04-30T02:15:21.1353156Z</updated>
<id />
<link href="http://www.contoso.com/_vti_bin/ListData.svc/Projects(2)"
rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Projects"
type="application/atom+xml;type=feed" />
<link href="http://www.contoso.com/_vti_bin/ListData.svc/Projects(3)"
rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Projects"
type="application/atom+xml;type=feed" />
<content type="application/xml">
<m:properties>
<d:Created m:type="Edm.DateTime" m:null="true" />
<d:FullName>James Earl Jones</d:FullName>
<d:HireDate m:type="Edm.DateTime">1987-04-29T19:15:14.7861156-07:00</d:HireDate>
<d:ID m:type="Edm.Int32">0</d:ID>
<d:Modified m:type="Edm.DateTime" m:null="true" />
<d:Path m:null="true" />
<d:Salary m:type="Edm.Double">195000</d:Salary>
<d:Version m:null="true" />
<d:Owshiddenversion m:type="Edm.Int32" m:null="true" />
</m:properties>
</content>
</entry>

The error is in the "category" element, you are not using "Linq", you should be using "DataService".

Changing the term attribute will fix your issues:

<category scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme"
term="Microsoft.SharePoint.Linq.EmployeesItem" />

TO:

<category scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme"
term="Microsoft.SharePoint.DataService.EmployeesItem" />

You should also wrap your WebRequest calls in an try/catch handling a WebException.
You can then interrogate the ResponseStream and see what error you are actually getting back and
fix any other issues you might run into!

As always, Enjoy!
Chris

Follow me on twitter!