Month: January 2013

Posted on

CellStorage.svc – Intelligent Updating with Office Clients

In one of my previous posts, I came to an unlikely conclusion that Office Clients don't support delta updates as has been so widely marketed.  This was because on every single client I had tested it on, never did it make such an intelligent call.  Since I have two of four chapters done for our upcoming book, I had some time and decided to install windows 8 on a 120GB SSD drive on my laptop.  After doing so, I also decided to install Office 2013 to see what the experience would be like. Not bad, a lot of re-learning to do, but hey, I'm an MCT, we love learning new stuff.  Sooo, I thought, let's try my previous tests that have subsequently failed so many times on this freshly installed OS and Office Client. To my surprise, something different happened when saving and editing SharePoint stored Office XML documents. 

The reason the previous Office clients I was working didn't do the delta updates is because they failed to make the proper calls to cellstorage.svc.  The HTTP request headers appear to be correct, but it doesn't send a body, which is the most important part of the request.  When this happens, it falls back to "FULL" update mode and every time you make a change, the full file is sent using an HTTP PUT request (no call to cellstorage.svc is made after the first failure).  Delta updates only support Office XML documents. The reason they are only Office XML is that deep inside cellstorage it makes a call to create a pointer to an XmlReader object.  Obviously, the older files are not based on this file format and an attempt to read them is futile, therefore, you won't have any calls to cellstorage.svc, but simply the regular HTTP PUT calls.

Again, until today I was not able to get it to work. But now is a different story.  As far as what happens when it does work, this POST will help you.  I am working with several others to figure out why my other clients did not work (more to come later).

CellStorage supports various different command types (not sure the entire set, but this is a lot of them):

  • GetDocMetaInfo
  • WhoAmI
  • ServerTime
  • Cell (get and set)
  • Coauth
  • SchemaLock
  • ReleaseLock

There is a standard process to this:

  • First step is to send a request for the Document MetaInfo
  • Second step is to actually get the parts of the document that are being viewededited at that moment (cell get)
  • Third step is to request to start editing the document (requesting a schema lock)
  • Fourth step is send back any changes that a person makes (cell set)
  • Last step is to tell the server you are done (release the schema lock)

During this entire process, the client will ping the sharedaccess.asmx web service to ensure that it is the only one editing the document.  This is done about every 20 seconds.  As part of the request, it is looking for the ETag to change.  If it has changed, that means someone updated the document and the version you have is now old and you will need to refresh your copy, or overwrite what they did.  This scenario should never happen, but it looks like they attempt to check that it somehow does (it is MicrosoftSharePoint right?).

Some interesting facts:

  • When you press Ctrl-S and you haven't made any changes, it doesn't save the file, but it does do a call for GetDocMetaInfo.  So although it is not as expensive as previous versions were, there is still a cost to this (the DocProps are HUGE)
  • More to come…

The following APPENDIX is a series of CellStorage calls for each of the steps above:

APPENDIX: 

The first request (Doc MetaInfo) looks like this:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<RequestVersion Version="2" MinorVersion="2" xmlns="http://schemas.microsoft.com/sharepoint/soap/"/>
<RequestCollection CorrelationId="{5E1AF699-7B07-4FF3-ADCA-16E1E40201CC}" xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<Request Url="http://www.sanspug.org/Presentations/Safety%20Plan%20Template.docx" UserAgent="{1984108C-4B93-4EEB-B320-919432D6E593}" UserAgentClient="msword" UserAgentPlatform="win" Build="15.0.4420.1017" MetaData="1031" RequestToken="1">
<SubRequest Type="GetDocMetaInfo" SubRequestToken="1"/>
<SubRequest Type="WhoAmI" SubRequestToken="2"/>
</Request>
</RequestCollection>
</s:Body>
</s:Envelope>

The response is this:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<ResponseVersion Version="2" MinorVersion="2" xmlns="http://schemas.microsoft.com/sharepoint/soap/"/>
<ResponseCollection WebUrl="http://www.sanspug.org" xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<Response Url="http://www.sanspug.org/Presentations/Safety%20Plan%20Template.docx" RequestToken="1" HealthScore="0">
<SubResponse SubRequestToken="1" ErrorCode="Success" HResult="0">
<SubResponseData>
<DocProps>
<Property Key="vti_internalversion" Value="513"/>
…
</DocProps>
<FolderProps>
<Property Key="vti_hassubdirs" Value="true"/>
…</FolderProps>
</SubResponseData>
</SubResponse>
<SubResponse SubRequestToken="2" ErrorCode="Success" HResult="0">
<SubResponseData UserName="Chris Givens" UserLogin="i:0#.w|chrisgivensadministrator" UserEmailAddress="givenscj@hotmail.com" UserSIPAddress=""/>
</SubResponse>
</Response>
</ResponseCollection>
</s:Body>
</s:Envelope>

The second request (get doc parts) is like this:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<RequestVersion Version="2" MinorVersion="2" xmlns="http://schemas.microsoft.com/sharepoint/soap/"/>
<RequestCollection CorrelationId="{459EA7C7-7B07-4FF3-ADCA-16E1E40201CC}" xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<Request Url="http://www.sanspug.org/Presentations/Safety%20Plan%20Template.docx" UserAgent="{1984108C-4B93-4EEB-B320-919432D6E593}" UserAgentClient="msword" UserAgentPlatform="win" Build="15.0.4420.1017" MetaData="1031" RequestToken="1">
<SubRequest Type="ServerTime" SubRequestToken="1"/>
<SubRequest Type="Cell" SubRequestToken="2">
<SubRequestData PartitionID="383adc0b-e66e-4438-95e6-e39ef9720122" BinaryDataSize="103">DQALAJzPKfM5lAabBgIAAO4CAACqAiAAjBCEGZNL606zIJGUMtblk1oEFgANbXN3b3JkB3dpbnoCCABFFOUPdwEWAgYAAwUAigICAADaAgYAAwAAygIIAAgAgAOEAEELAawCAFUDAQ==</SubRequestData>
</SubRequest>
<SubRequest Type="Cell" SubRequestToken="3">
<SubRequestData GetFileProps="true" BinaryDataSize="103">DQALAJzPKfM5lAabBgIAAO4CAACqAiAAjBCEGZNL606zIJGUMtblk1oEFgANbXN3b3JkB3dpbnoCCABFFOUPdwEWAgYAAwUAigICAADaAgYAAwAAygIIAAgAgAOEAEELAawCAFUDAQ==</SubRequestData>
</SubRequest>
<SubRequest Type="Cell&quo
t; SubRequestToken="4">
<SubRequestData PartitionID="7808f4dd-2385-49d6-b7ce-37aca5e43602" BinaryDataSize="103">DQALAJzPKfM5lAabBgIAAO4CAACqAiAAjBCEGZNL606zIJGUMtblk1oEFgANbXN3b3JkB3dpbnoCCABFFOUPdwEWAgYAAwUAigICAADaAgYAAwAAygIIAAgAgAOEAEELAawCAFUDAQ==</SubRequestData>
</SubRequest>
<SubRequest Type="Cell" SubRequestToken="5">
<SubRequestData BinaryDataSize="88">DQALAJzPKfM5lAabBgIAAO4CAACqAiAAjBCEGZNL606zIJGUMtblk1oEFgANbXN3b3JkB3dpbnoCCABFFOUPdwEWAgYAAxcAAgQIAAQ1DAALAawCAFUDAQ==</SubRequestData>
</SubRequest>
<SubRequest Type="GetVersions" SubRequestToken="6"/>
</Request>
</RequestCollection>
</s:Body>
</s:Envelope>

The response is this (but also included the binary that was requested in the request after the XML part of the response):

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<ResponseVersion Version="2" MinorVersion="2" xmlns="http://schemas.microsoft.com/sharepoint/soap/"/>
<ResponseCollection WebUrl="http://www.sanspug.org" xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<Response Url="http://www.sanspug.org/Presentations/Safety%20Plan%20Template.docx" RequestToken="1" HealthScore="0">
<SubResponse SubRequestToken="1" ErrorCode="Success" HResult="0">
<SubResponseData ServerTime="634934067280000000"/>
</SubResponse>
<SubResponse SubRequestToken="2" ErrorCode="Success" HResult="0">
<SubResponseData>DAALAJ3PKfM5lAabFgMCAACsAgAMWkCfInndLk8SpahJoDsL3N5J72OAed0uTxKlqEmgOwvc3knvYz9FAAAAAAAAA4gEAAAFVQ4CBgADBQD6AigAQJ8ied0uTxKlqEmgOwvc3knvYwCEACYCIAATHwkQgsj7QJiGZTP5NMIdbAFwWQwcuPLQK0r+TVOn9MDyqn7BNTEAEFAFZRBVAGgpfUBUooP7XAJQAFAAUAAAcLkMWcPYYK6I7UOVZMRDNVJSnZUxADtQMYDEgGKDDF05g8hQM1MOXDSAwFADUwxcMHzIZDZmzFw4iMSEYoONXTF90Ggmc01bYYWEXdZSzWhico1lY1ZNaTh1wVAAALUTASYCIAAO6XY6MoAMTbnd88ZQKUM+TAEgJgwcuPLQK0r+TVOn9MDyqn7BAwClEwFBBwGLAQ==</SubResponseData></SubResponse><SubResponse SubRequestToken="3" ErrorCode="Success" HResult="0">
<SubResponseData Etag="&quot;{2EC13881-6761-4F26-A5AC-4CBBC5F4E8E0},2&quot;" CreateTime="129677043240000000" HaveOnlyDemotionChanges="False" LastModifiedTime="129677044800000000" ModifiedBy="System Account">
<xop:Include href="cid:http://tempuri.org/1/634933491321512406" xmlns:xop="http://www.w3.org/2004/08/xop/include"/>
</SubResponseData>
</SubResponse>
<SubResponse SubRequestToken="4" ErrorCode="Success" HResult="0">
<SubResponseData>DAALAJ3PKfM5lAabFgMCAACsAgAMWkCjInndLk8SpahJoDsL3N5J72OAed0uTxKlqEmgOwvc3knvY0dFAAAAAAAAA4gEAAAFVQ4CBgADBQD6AigAQKMied0uTxKlqEmgOwvc3knvYwCEACYCIAAO6XY6MoAMTbnd88ZQKUM+TAEgJgwcuPLQK0r+TVOn9MDyqn7BAwAgJgyZIgShWw0pQpGd0oV6Kr7gAAClEwEmAiAAEx8JEILI+0CYhmUz+TTCHWwBcFkMHLjy0CtK/k1Tp/TA8qp+wTUxABBQBWUQVQBoKX1AVKKD+1wCUABQAFAAAHC5DFnD2GCuiO1DlWTEQzVSUp2VMQA7UDGAxIBigwxdOYPIUDNTDlw0gMBQA1MMXDB8yGQ2ZsxcOIjEhGKDjV0xfdBoJnNNW2GFhF3WUs1oYnKNZWNWTWk4dcFQAAC1EwFBBwGLAQ==</SubResponseData></SubResponse><SubResponse SubRequestToken="5" ErrorCode="Success" HResult="0">
<SubResponseData HaveOnlyDemotionChanges="False">DAALAJ3PKfM5lAabFgMCAACsAgBVDgIGAAMXAAoEKgBUqbSHwoLARZ4MOEP3qCkZ6gQENQwHAYsB</SubResponseData></SubResponse><SubResponse SubRequestToken="6" ErrorCode="Success" HResult="0">
<GetVersionsResponse>
<GetVersionsResult>
<results>
<list id="{713BA73C-EFCD-49C9-B23D-8A2106868919}"/>
<versioning enabled="0"/>
<settings url="http://www.sanspug.org/_layouts/15/LstSetng.aspx?List={713BA73C-EFCD-49C9-B23D-8A2106868919}"/>
<result version="@1.0" url="http://www.sanspug.org/Presentations/Safety Plan Template.docx" created="12/6/2011 8:08 PM" createdRaw="2011-12-07T04:08:00Z" createdBy="SHAREPOINTsystem" createdByName="System Account" size="30106" comments=""/>
</results>
</GetVersionsResult>
</GetVersionsResponse>
</SubResponse>
</Response>
</ResponseCollection>
</s:Body>
</s:Envelope>

The request that drives the saving of a part back is (also included is a multi part binary that has the changes):

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<RequestVersion Version="2" MinorVersion="2" xmlns="http://schemas.microsoft.com/sharepoint/soap/"/>
<RequestCollection CorrelationId="{71E42242-7B07-4FF3-ADCA-16E1E40201CC}" xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<Request Url="http://www.sanspug.org/Presentations/Safety%20Plan%20Template.docx" UserAgent="{1984108C-4B93-4EEB-B320-919432D6E593}" UserAgentClient="msword" UserAgentPlatform="win" Build="15.0.4420.1017" MetaData="7" RequestToken="1">
<SubRequest Type="Coauth" SubRequestToken="1">
<SubRequestData CoauthRequestType="RefreshCoauthoring" SchemaLockID="29358EC1-E813-4793-8E70-ED0344E7B73C" ClientID="{7D99133D-52B2-44E4-BCFC-A56DBD9BE639}" Timeout="3600"/>
</SubRequest>
<SubRequest Type="SchemaLock" SubRequestToken="2" DependsOn="1" DependencyType="OnNotSupported">
<SubRequestData SchemaLockRequestType="RefreshLock" SchemaLockID="29358EC1-E813-4793-8E70-ED0344E7B73C" ClientID="{7D99133D-52B2-44E4-BCFC-A56DBD9BE639}" Timeout="3600"/>
</SubRequest>
<SubRequest Type="Cell" SubRequestToken="3" DependsOn="2" DependencyType="OnSuccessOrNotSupported">
<SubRequestData Coalesce="true" CoauthVersioning="true" GetFileProps="true" BypassLockID="29358EC1-E813-4793-8E70-ED0344E7B73C" SchemaLockID="29358EC1-E813-4793-8E70-ED0344E7B73C" BinaryDataSize="32021">
<i:Include xmlns:i="http://www.w3.org/2004/08/xop/include" href="cid:820fe252-7a34-4b40-802f-e666a88a8941-0@tempuri.org"/>
</SubRequestData>
</SubRequest>
</Request>
</RequestCollection>
</s:Body>
</s:Envelope>

The file update response is:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<ResponseVersion Version="2" MinorVersion="2" xmlns="http://schemas.microsoft.com/sharepoint/soap/"/>
<ResponseCollection WebUrl="http://www.sanspug.org" xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<Response Url="http://www.sanspug.org/Presentations/Safety%20Plan%20Template.docx" RequestToken="1" HealthScore="0">
<SubResponse SubRequestToken="1" ErrorCode="Success" HResult="0">
<SubResponseData LockType="SchemaLock" CoauthStatus="Alone"/>
</SubResponse>
<SubResponse SubRequestToken="2" ErrorCode="DependentOnlyOnNotSupportedRe
questGetSupported" HResult="2147500037">
<SubResponseData/>
</SubResponse>
<SubResponse SubRequestToken="3" ErrorCode="Success" HResult="0">
<SubResponseData Etag="&quot;{2EC13881-6761-4F26-A5AC-4CBBC5F4E8E0},3&quot;" HaveOnlyDemotionChanges="False">DAALAJ3PKfM5lAabFgMCAACsAgBVDgIGAAMLADoEBAAAAIQAJgIgAPY1ejJhBxREloZR6QBmek2kAHgmYSrmvvwoD0GvMdRy1rRs+wDyCHgmntUZQQPX8L5QziuNKUuTBADyCFETASYCIAAO6XY6MoAMTbnd88ZQKUM+TAEgKAx+xz7RnpjZsFpTs0Q6CxcfpgwAICgUfsc+0Z6Y2bBaU7NEOgsXH6YMAKUTASYCIAATHwkQgsj7QJiGZTP5NMIdbAFwSQykwL9CeUV1ACf0G+m6m5XCJTEACVAFdAJw13VWUwNQAFAAALUTAUFKBAIAAAcBiwE=
</SubResponseData>
</SubResponse>
</Response>
</ResponseCollection>
</s:Body>
</s:Envelope>

 Closing the session request is:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<RequestVersion Version="2" MinorVersion="2" xmlns="http://schemas.microsoft.com/sharepoint/soap/"/>
<RequestCollection CorrelationId="{C2ADFD14-7B07-4FF3-ADCA-16E1E40201CC}" xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<Request Url="http://www.sanspug.org/Presentations/Safety%20Plan%20Template.docx" UserAgent="{1984108C-4B93-4EEB-B320-919432D6E593}" UserAgentClient="msword" UserAgentPlatform="win" Build="15.0.4420.1017" MetaData="1031" RequestToken="1">
<SubRequest Type="Coauth" SubRequestToken="1">
<SubRequestData CoauthRequestType="ExitCoauthoring" SchemaLockID="29358EC1-E813-4793-8E70-ED0344E7B73C" ClientID="{7D99133D-52B2-44E4-BCFC-A56DBD9BE639}"/>
</SubRequest>
<SubRequest Type="SchemaLock" SubRequestToken="2" DependsOn="1" DependencyType="OnNotSupported">
<SubRequestData SchemaLockRequestType="ReleaseLock" SchemaLockID="29358EC1-E813-4793-8E70-ED0344E7B73C" ClientID="{7D99133D-52B2-44E4-BCFC-A56DBD9BE639}"/>
</SubRequest>
</Request>
</RequestCollection>
</s:Body>
</s:Envelope>

 The response is:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body><ResponseVersion Version="2" MinorVersion="2" xmlns="http://schemas.microsoft.com/sharepoint/soap/"/>
<ResponseCollection WebUrl="http://www.sanspug.org" xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<Response Url="http://www.sanspug.org/Presentations/Safety%20Plan%20Template.docx" RequestToken="1" HealthScore="0">
<SubResponse SubRequestToken="1" ErrorCode="Success" HResult="0">
<SubResponseData/>
</SubResponse>
<SubResponse SubRequestToken="2" ErrorCode="DependentOnlyOnNotSupportedRequestGetSupported" HResult="2147500037">
<SubResponseData/>
</SubResponse>
</Response>
</ResponseCollection>
</s:Body>
</s:Envelope>

 

Posted on

Office Web Apps and WOPI Binding at Site Collection Level

If you haven't made it too far into OWA land, then let me forewarn you about some things.  There are two version of Excel Services.  The one that comes with SharePoint and the one that comes with OWA.  The truth is:

You can have one and only one!  – (update:  I should have added here, "in certain areas" to get the point across)

UPDATE: Evidentally, some think it ok and acceptable, to view excel reports through the  excel services web part (with the limited viewing area) rather than the more straight forward way of just viewing it directly in the browser.  I'm not one of those people.

You must decide which one you want.  In our upcoming book, I point out the various difference between the two versions.  The net net is if you want advanced Business intelligence features, you must use Excel Services.  If you want advanced viewing and editing, you must use OWA.  You cannot target web apps, content databases or site collections for the decision.  It is a global farm setting decision.

I find this unacceptable to the community as a whole and I can see a service pack coming that enables us to configure this at least on a web application level and if possible on a site collection basis too.

That being said, it is true, as pointed out by Bryan Hart on my Facebook page, that you can disable the "Viewing" action to allow Excel Services to view the files, but allow editing to be done by OWA.  This is done with the New-SPWOPISupressionSetting cmdlet.

At some point, this will all change with new features and versions and hopefully work much better than it does now.

Chris

Posted on

Installing Project Server 2013 – Previous version of product exists

This error is not very specific.  It could mean project server 2010 or sharepoint 2010 exists.  If your uninstall doens't complete successfully (or it does but you don't realize it left some residual), then you may run into this problem.  The key to figuring this out is to run ProcMon.exe on the server when you attempt to do the install, then work backwards on all the registry keys that it tries to access.

In my case, because the HKLMSoftwareMicrosoftShared ToolsWeb Server Extensions14.0 registry key still existed, it thought that SharePoint 2010 was installed.  However, I had removed it.  Yes this residual registry key was preventing the install from proceeding.  Simply delete this registry key and your install will continue as desired!

Enjoy!
Chris

Posted on

How Request Management Works in SharePoint 2013

Spence Harbar has a great 3-part blog here on how to configure request management.  It however does not go into the deep level of what is occurring in the code.  This blog will take you a bit deeper into how Request Management works.

 What is Request Management?  It is a Reverse Proxy implemented in SharePoint 2013.  It is defined by the SPRoutingReverseProxy class.  As Spence points out, Request Management is implemented in the Http Module of SharePoint (Microsoft.SharePoint.ApplicationRuntime.SPRequestModule).  Internally it works like this:

  • An HTTP request is received by a SharePoint Web
    Application
  • As part of the ASP.NET pipeline, the SPRequestModule
    is passed the HTTP request for processing
  • The module will check if Request Management
    settings are present for the respective web application
  • If settings are present, the module will then
    check to see if Routing or Throttling has been enabled and if the request management
    service is started.  The module will also
    check to see if the request routing has already been cached as part of any previous requests for the client by
    an internal reverse proxy called the SPRoutingReverseProxy
  • Request Management will then check to see if the
    client has been assigned an affinity to a particular machine via the SPRoutingAffinity
    logic
  • If the request does have affinity, Request
    Management will check that the target server is available, if it doesn’t, then
    the next Routing Target will be selected
  • Once a target has been selected, the HTTP
    context will have a “SPRequestManagementRouted” item added to it for future
    routing
  • Lastly, the request will then be routed to the
    target, the response will be proxied back to the client as long as it responds
    before the Request Execution Timeout in the Request Management Settings
    • The request is sent using basic HttpRequest classes and it is basically copying the request from the SharePoint server to the target server, so as long as the target server is a part of the SharePoint farm, technically any applications running on the SharePoint server can be routed too (not just SharePoint ones).

I'm actually very excited about having this reverse proxy in SharePoint.  There have been a number of things I have been wanting to do on the ACS network but just couldn't do with both SharePoint and my other apps running.  This gives me the ability to route the http requests to other applications on my internal network, not just SharePoint ones!

Even though Request Management is implemented as a reverse
proxy, it is lacking some common features of a full reverse proxy such as:

  • URL Rewriting
  • Static and Dynamic Content Caching
  • Response manipulation (adding or removing JavaScript,
    etc)

The process of determining the final target host is as follows:

 

  • The request is passed to the SP Routing Rule
    Evaluator
  • The routing rules are evaluated and a pool of
    targets will be returned.  The first routing
    rule to have targets wins.
  • Each target is pinged to determine if they are
    available, only targets that are available are considered
  • For each available target, the throttling rules
    are applied, if a target fails it is removed from the routing list
  • The remaining targets, if more than one, are
    evaluated based on their weighting, the machine with the lowest weighting is
    returned
  • Lastly, and not performant-ly by the way, if there is an affinity machine, it is
    selected and returned (this should have been done first before analyzing the weighting, this is a design flaw in the code)

The last point about this new feature is the APIs are all set as internal.  This is LAME.  It keeps people like me and the real MVPs(the ones that could build these tools whether Microsoft MVP or not) from building GUI tools for it.  Maybe they will open it up in the future?

To learn even more about how Request management works, buy our book when it comes out!

Enjoy!
Chris