Author: givenscj

Posted on

How to remove blank comments on Append Changes to Existing Text

This is a rather simple issue, but an issue none the less.  It has been covered a few times before:

The issue is that it creates a blank comment every time someone edits a versioned list item.  The reason is that it *IS* actually adding text but it is an empty DIV tag:

<DIV></DIV>

When you set the column to be plain text, it doesn't add this empty DIV tag.  Ok, so just change to plain text and everyone is happy right?  Not quite.  When users connect to the list via the Outlook option, you get a very nasty set of HTML saved to the column (even in Plain text mode):

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META NAME="Generator" CONTENT="MS Exchange Server version 08.01.0240.003">
<TITLE></TITLE>
</HEAD>
<BODY>
<!– Converted from text/plain format –>
</BODY>
</HTML>

Here's what you do in the event receiver to get rid of all this mess and have it work wether it is Plain Text or Rich Text:

class ContosoEventReceiverAppendChanges : SPItemEventReceiver
{
public override void ItemUpdating(SPItemEventProperties properties)
{
base.ItemUpdating(properties);
Hashtable ht = new Hashtable();
foreach (DictionaryEntry o in properties.AfterProperties)
{
if (o.Value.ToString() == "<DIV></DIV>" || o.Value.ToString().Contains("<BODY> <!– Converted from text/plain format –> </BODY>"))
{
ht.Add(o.Key, o.Key);|
}
}
foreach (string key in ht.Keys)
{
properties.AfterProperties[key] = "";
}}}

Powershell to deploy it:

$web = get-spweb http://sharepoint.contoso.com
$list = $web.lists["TaskTest"]
$def = $list.EventReceivers.add()
$def.Assembly = "Contoso.Common, Version=1.0.0.0, Culture=Neutral, PublicKeyToken=a61cba8dcde3eaaa";
$def.Class = "Contoso.Common.ConotosEventReceiverAppendChanges";
$def.Name = "ItemUpdated Event";
$def.Type = [Microsoft.SharePoint.SPEventReceiverTy
pe]::ItemUpdating;
$def.SequenceNumber = 1000;
$def.Synchronization = [Microsoft.SharePoint.SPEventReceiverSynchronization]::Synchronous;
$def.Update();
$web.dispose()

Enjoy,
Chris

Follow me on twitter: @givenscj

Check out the previous blog post:
https://blogs.architectingconnectedsystems.com/blogs/cjg/archive/2012/05/30/Project-Server-2010-_2D00_-Moving-Project-Sites-and-deleting-the-old-PWA-causes-issues.aspx

Posted on

SharePoint Farm Full Password Reset

At a customer of mine we recently had to do a full password reset of ALL of Active Directory.  This included all user and service accounts, even the SharePoint ones.  If you have EVER had to do this, you know that if you don't change the password in SharePoint a few minutes after the change in AD, you will be forced to manually change them across soo many places its not even funny.  It turns out, we had to do a second password reset not long after the first one.  It will become a commonplace thing that ALL passwords will be reset every couple of months and therefore, changing the passwords across all the farm servers (SharePoint, FAST and SQL Server) just ain't going to cut it.

Therefore, I wrote a PowerShell script to do it!  It worked really well!  It changes every single place one could imagine to have a username and password in your farm.  I was going to post it, but decided that I would make a product out of it.  If you would like to try out this gem, send me an email and I can send you a trial:

chris@architectingconnectedsystems.com

Thanks,
Chris

Follow me on twitter: @givenscj

Check out the previous blog post:
https://blogs.architectingconnectedsystems.com/blogs/cjg/archive/2012/05/23/Office-Web-Apps-_2D00_-Word-Viewing-Service-Error.aspx

Posted on

Project Server 2010 – Moving Project Sites and deleting the old PWA causes issues

Recently we had to deconstruct the entire Project Server environment and rebuild it.  This involved moving the sharepoint sites related to the projects and then re-attaching the old project server databases to a new PWA (ie, the old PWA was http://projects.contoso.com, the new PWA is http://projects.contoso.com/PWA).  Then we had to relink the PWA projects to their old but moved sites urls.

This all went fine until the moment we had to view or edit the properties of the documents in the team sites.  We would get a lovely correlation error when trying to view or edit.  The generic error was:

05/30/2012 10:36:10.02  w3wp.exe (0x207C)                        0x1D80 Project Server                 General                        g7ls Exception  System.ServiceModel.FaultException: The server was unable to process the request due to an internal error.  For more information about the error, either turn on IncludeExceptionDetailInFaults (either from ServiceBehaviorAttribute or from the <serviceDebug> configuration behavior) on the server in order to send the exception information back to the client, or turn on tracing as per the Microsoft .NET Framework 3.0 SDK documentation and inspect the server trace logs.    Server stack trace:      at System.ServiceModel.Channels.ServiceChannel.ThrowIfFaultUnderstood(Message reply, MessageFault fault, String action, MessageVersion version, FaultConverter faultConverter)     at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)     at System.S… 22df46a2-3f76-4c8c-b317-d2076882b490
05/30/2012 10:36:10.02* w3wp.exe (0x207C)                        0x1D80 Project Server                 General                        g7ls Exception …erviceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)     at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)     at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)    Exception rethrown at [0]:      at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)     at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)     at Microsoft.Office.Project.Server.Interfaces.IAuthentication.GetUserByName(String username)     at Microsoft.Office.Project.Server.AuthenticationHelper.AuthenticateUserByName(String userName, Guid … 22df46a2-3f76-4c8c-b317-d2076882b490
05/30/2012 10:36:10.02* w3wp.exe (0x207C)                        0x1D80 Project Server                 General                        g7ls Exception …siteId) 22df46a2-3f76-4c8c-b317-d2076882b490

Changing the includeExceptionDetailInFaults property of the D:Program FilesMicrosoft Office Servers14.0WebServicesSharedProjectServerPSIweb.config file to "True", we get a more meaningful error:

05/30/2012 10:36:12.68  w3wp.exe (0x207C)                        0x282C Project Server                 General                        g7ls Exception  System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: No site found (Fault Detail is equal to An ExceptionDetail, likely created by IncludeExceptionDetailInFaults=true, whose value is: System.InvalidOperationException: No site found    at Microsoft.Office.Project.Server.DataAccessLayer.DAL.GetConnectionStrings(PlatformContext context)     at Microsoft.Office.Project.Server.DataAccessLayer.DAL..ctor(PlatformContext context)     at Microsoft.Office.Project.Server.Base.PlatformContext.get_Dal()     at Microsoft.Office.Project.Server.Base.PlatformContext.InitSqlCommandTimeout(CachedContextProperties cachedProperties)     at Microsoft.Office.Project.Server.Base.PlatformContext.PreInitialize(Guid trackingId, Guid siteId, SPUrlZone urlZone, CultureInfo languageCulture, Cultu… a73494b6-2bff-4b2f-9faf-67ad55d9f7d9
05/30/2012 10:36:12.68* w3wp.exe (0x207C)                        0x282C Project Server                 General                        g7ls Exception …reInfo localeCulture)     at Microsoft.Office.Project.Server.Base.PlatformContext..ctor(Boolean isWindowsUser, String username, Guid userId, Guid trackingId, Guid siteId, SPUrlZone urlZone, CultureInfo languageCulture, CultureInfo localeCulture, Guid delegateUid, String delegateName)     at Microsoft.Office.Project.Server.Base.Platform…). a73494b6-2bff-4b2f-9faf-67ad55d9f7d9

After reviewing the possibilites of what could cause this, I found a few things:

  • Project sites have special content types (Project Site Document)
  • The Project Site Document has a special field type (Microsoft.Office.Project.PWA.CustomFieldTypes.SPFieldPWALink)
  • Project site lists have event receivers (Microsoft.Office.Project.PWA.WSSEventReceivers.PSDBUpdater)

I tried several different things before I found the one item that was really causing the issues with viewing and editing.  They included:

  • Trying to remove the content type and set them to "Document" – Didn't work
  • Trying to remove the event receivers (removing them was successful, but it didn't fix the problem)
  • Removing the "Links" column values from all the items (the values removed, but it didn't fix the problem)
  • Removing the "Links" field from the list – this was the answer

Turns out that the custom field type had recorded the old PWA url and when the display forms were trying to render, the call to the PWA authentication service would fail because it couldn't find the old PWA url. After removing the "Links" field from the "Content Type/List" and changing the content types to "Document", we were able to viewedit the properties of the items onc
e again:

$f = $list.fields["Links"]
$f.delete()

foreach ($li in $list.items)
{
if ($li["Content Type"] -eq "Project Site Document")
{
$li.file.checkout()
$li["Content Type"] = "Document"
$li.update()
$li.file.checkin("")
}
}

Enjoy!
Chris

Follow me on twitter: @givenscj

Check out the previous blog post:
https://blogs.architectingconnectedsystems.com/blogs/cjg/archive/2012/05/30/SharePoint-Farm-Full-Password-Reset.aspx

Posted on

Office Web Apps – Word Viewing Service Error

All of a sudden, we started having issues with our Office Web Apps install for a set of users.  No reason, no changes, just some users can't use OWA to VIEW files (they can however edit the files in the browser).  The service account for the application pool and the word viewing app pool both have dbo on the content database in question.

Here's the errors:

User:

Word Web App cannot open this document for viewing because of an unexpected error.  To view this document, open it in Microsoft Word.

WFE:

 05/23/2012 07:56:55.17  w3wp.exe (0x32CC)                        0x310C Office Web Apps                Office Viewing Architecture    dg2s Medium   RequestDispatcher: response received for document F4813f0a0691d4d61a109e9ab86a17815m4d776f4fd83844328210cf1c84e667cbm06ebaf2e1684459c9c00819543a2a08em from http://servername:32843/69c32f311e164e8184803a179b32d194/Conversion.svc. d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.19  w3wp.exe (0x32CC)                        0x310C Office Web Apps                Office Viewing Architecture    b4vn Medium   Conversion response received: CantFindSourceDocument d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.19  w3wp.exe (0x32CC)                        0x310C Office Web Apps                Office Viewing Architecture    b4vi Verbose  DocumentInfoCache.RemoveDocumentInfo: docId=d=F4813f0a0691d4d61a109e9ab86a17815m4d776f4fd83844328210cf1c84e667cbm06ebaf2e1684459c9c00819543a2a08em&z=06EBAF2E-1684-459C-9C00-819543A2A08E1, outputFormat=_v00000000-0000-0000-0000-000000000202 d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.19  w3wp.exe (0x32CC)                        0x310C Office Web Apps                Office Viewing Architecture    vjza Medium   Librarian.OnConversionResponseReceived(F4813f0a0691d4d61a109e9ab86a17815m4d776f4fd83844328210cf1c84e667cbm06ebaf2e1684459c9c00819543a2a08em, Png, docdata.xml) – Response=CantFindSourceDocument, Data=NO d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.20  w3wp.exe (0x32CC)                        0x310C Office Web Apps                Office Viewing Architecture    vjzo Medium   Librarian.SetCompleted(F4813f0a0691d4d61a109e9ab86a17815m4d776f4fd83844328210cf1c84e667cbm06ebaf2e1684459c9c00819543a2a08em, Png, docdata.xml) – status = ConversionError d3e88a2a-977b-48df-a70a-4765c4ff1ac9

APP:

05/23/2012 07:56:55.57  w3wp.exe (0x1330)                        0x0B6C Office Web Apps                Office Viewing Architecture    bv6l Medium   DownloadManager: Starting download for document F4813f0a0691d4d61a109e9ab86a17815m4d776f4fd83844328210cf1c84e667cbm06ebaf2e1684459c9c00819543a2a08em. d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.57  w3wp.exe (0x1330)                        0x0B6C Office Web Apps                Office Performance Tracing     dcw3 Verbose  PerfCounterAverageTimer started: WssStorageReadTime d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.57  w3wp.exe (0x1330)                        0x0B6C SharePoint Foundation          General                        6t8j Verbose  Site lookup by id found the typical site /training. d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.57  w3wp.exe (0x1330)                        0x0B6C SharePoint Foundation          General                        8kh7 High     Cannot complete this action.  Please try again. d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.58  w3wp.exe (0x1330)                        0x0B6C SharePoint Foundation          General                        8kh7 High     Stack trace: onetutil.dll: (unresolved symbol, module offset=00000000000A2551) at 0x000007FEF0142551 onetutil.dll: (unresolved symbol, module offset=00000000000A3711) at 0x000007FEF0143711 owssvr.dll: (unresolved symbol, module offset=0000000000009212) at 0x000007FEF0399212 owssvr.dll: (unresolved symbol, module offset=0000000000044378) at 0x000007FEF03D4378 mscorwks.dll: (unresolved symbol, module offset=00000000002CB6D7) at 0x000007FEF9BFB6D7 Microsoft.SharePoint.Library.ni.dll: (unresolved symbol, module offset=00000000000E5128) at 0x000007FEF2D85128 Microsoft.SharePoint.ni.dll: (unresolved symbol, module offset=0000000001AB43D0) at 0x000007FEE0AA43D0 Microsoft.SharePoint.ni.dll: (unresolved symbol, module offset=0000000001AA811D) at 0x000007FEE0A9811D Microsoft.SharePoint.ni.dll: (unre… d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.58* w3wp.exe (0x1330)                        0x0B6C SharePoint Foundation          General                        8kh7 High     …solved symbol, module offset=0000000001AADC71) at 0x000007FEE0A9DC71 Microsoft.SharePoint.ni.dll: (unresolved symbol, module offset=0000000001AA2A25) at 0x000007FEE0A92A25 Microsoft.SharePoint.ni.dll: (unresolved symbol, module offset=0000000001AA4717) at 0x000007FEE0A94717 Microsoft.SharePoint.ni.dll: (unresolved symbol, module offset=0000000001AA46A2) at 0x000007FEE0A946A2  d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.58  w3wp.exe (0x1330)            &nbs
p;           0x0B6C Office Web Apps                Office Viewing Architecture    ekp1 Medium   No SPFile found for Id 06ebaf2e-1684-459c-9c00-819543a2a08e d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.58  w3wp.exe (0x1330)                        0x0B6C Office Web Apps                Office Viewing Architecture    ekp3 Medium   SharepointReaderAsync: No SPFile provided d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.58  w3wp.exe (0x1330)                        0x0B6C Office Web Apps                Office Performance Tracing     a495 Verbose  PerfCounterAverageTimer stopped: WssStorageReadTime, elapsed time = 20 milliseconds d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.58  w3wp.exe (0x1330)                        0x0B6C Office Web Apps                Office Viewing Architecture    bv6p Unexpected Exception thrown downloading file C:WindowsTEMPwaccache69c32f31-1e16-4e81-8480-3a179b32d194domainuserca171873-6de3-4dd4-abef-09d23996d26coutput.docx: System.ArgumentException: SharepointReaderAsync: no SPFile     at Microsoft.Office.Web.Environment.Sharepoint.SharepointReaderAsync.End()     at Microsoft.Office.Web.Conversion.Framework.DownloadManager.OnFileReadComplete(IAsyncResult ar) d3e88a2a-977b-48df-a70a-4765c4ff1ac9
05/23/2012 07:56:55.58  w3wp.exe (0x1330)                        0x0B6C Office Web Apps                Office Viewing Architecture    fvj9 Unexpected DownloadManager: Download FAILED for document F4813f0a0691d4d61a109e9ab86a17815m4d776f4fd83844328210cf1c84e667cbm06ebaf2e1684459c9c00819543a2a08em; Time spent: 24ms. d3e88a2a-977b-48df-a70a-4765c4ff1ac9

The only possible hints lie in the users access (this says it is request view and edit access but the user doesn't have it):

05/23/2012 07:56:54.20  w3wp.exe (0x32CC)                        0x2B94 SharePoint Foundation          General                        ewn8 Verbose  OriginalPermissionMask check failed. asking for 0x00000005, have 0xB008431041 9806faaf-ad37-406f-9164-5705a9aca938

You'll also find several errors about configuration settings missing for the Word Viewing settings:

05/23/2012 07:56:54.30  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbw9 Verbose  Trying to get setting C2ROFFERENABLED 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.30  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbx0 Verbose  Tried to obtain setting C2ROFFERENABLED but it does not exist in the list of known settings for this environment. 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.30  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbw9 Verbose  Trying to get setting WLSIAOFFERENABLED 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.30  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbx0 Verbose  Tried to obtain setting WLSIAOFFERENABLED but it does not exist in the list of known settings for this environment. 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.31  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbw9 Verbose  Trying to get setting USEPARENTHELPFUNCTION 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.31  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbx0 Verbose  Tried to obtain setting USEPARENTHELPFUNCTION but it does not exist in the list of known settings for this environment. 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.31  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbw9 Verbose  Trying to get setting SHOWFEEDBACKLINK 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.31  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbx0 Verbose  Tried to obtain setting SHOWFEEDBACKLINK but it does not exist in the list of known settings for this environment. 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.33  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbw9 Verbose  Trying to get setting HIDETITLE 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.33  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps     
           Office Web Apps Configuration  bbx0 Verbose  Tried to obtain setting HIDETITLE but it does not exist in the list of known settings for this environment. 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.33  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbw9 Verbose  Trying to get setting MACOFFICETRIALOFFERENABLED 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.33  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbx0 Verbose  Tried to obtain setting MACOFFICETRIALOFFERENABLED but it does not exist in the list of known settings for this environment. 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.33  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbw9 Verbose  Trying to get setting TRIMOICBUTTONSIFUNAVAILABLE 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.33  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbx0 Verbose  Tried to obtain setting TRIMOICBUTTONSIFUNAVAILABLE but it does not exist in the list of known settings for this environment. 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.33  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbw9 Verbose  Trying to get setting WORDWEBAPPSHORTRESOURCENAME 9806faaf-ad37-406f-9164-5705a9aca938
05/23/2012 07:56:54.33  w3wp.exe (0x32CC)                        0x2B94 Office Web Apps                Office Web Apps Configuration  bbw9 Verbose  Trying to get setting WORDWEBAPPFULLRESOURCENAME 9806faaf-ad37-406f-9164-5705a9aca938

What have I done so far?  Here's the things I have tried: 

  1. I created a new Word Viewing Service and set it as the default service application.  Had the user retry…the documents still didn't display in OWA and I still get all the wonderful OWA settings are missing messages (so the messages can be ignored) – NO GO
  2. I gave the user contribute access, had them re-try…still didn't display in OWA, but at least a new button showed up on the error that lets them "Open in Word" (realized this happens after the second attempt to open the file) – NO GO
  3. Last ditch effort, added the app pool for the OWA conversion service to the local WSS_ADMIN_WPG…NO GO.
  4. Made the user a site collection admin – NO GO  

Weird that it only applies to a small set of users.  It shouldn't matter that they just have View Only rights to the file, it should open in OWA…again editing works just fine…

Sooo…how did I fix this?  I went to the handy SQL Profiler tool and watched the SQL traffic.  I re-run all the queries and looked at all the results sets that were returned.  I opened the OWA dlls in Reflecter and explored every method call.  I found the "no SPFile" returned set of code, everything looks file, but, it is the hint that I was looking for.  Stepping back, one should ask…why is the file not being returned?  It is there, but…it isn't?  In looking at the result sets, I found something interesting.  Once upon a time, users had enabled the DocumentID feature of sharepoint.  This assigned document ids to some of the documents.  They also are using this tool called "CopyPaste".  When copying a record from one site collection to another, the document ID values were being replicated to the new location and metadata of the files.  When OWA makes the call to get the file, it is going through the SharePoint APIs and in this case, the document ID is wrong and the site that used to hold the file is gone (or in some cases not accessible by the user because we removed their permissions)!  Document IDs suck and this is just another reason to use my SharePoint Durable Urls tool (http://www.sharepointdurableurls.com). That means…if you hit the document id link for the file, you get simple html that says "File Not Found".  BINGO!  Now…how to remove the DocumentID residual?  Here's what I did:

foreach($li in $list.items)
{
$docid = $li["Document ID"]

if ($docid)
{
"Found doc id in " + $li["Name"]
"ID is: $docid"

$li.file.checkout()
$li["Document ID"] = ""
$li.update()
$li.file.checkin("Removed Document ID")
}
}

Yet another case of the code making one call (View is using the DocumentID redirect) in one instance and another in another instance (Editing does not use the DocumentID redirect).  Very annoying.

Enjoy!
Chris

Follow me on twitter: @givenscj

Check out the previous blog post:
https://blogs.architectingconnectedsystems.com/blogs/cjg/archive/2012/05/18/Announcing-_2D00_-ACS-Rebate-and-Charity-Event.aspx

Posted on

Announcing – ACS Rebate and Charity Event

For those of you that are ACS Courseware customers, if you didn't get our email today, here it is:

Dear ACS Courseware customers, As I have mentioned in the past, we have been wanting to do a promotional campaign for ACS courseware sales through the Courseware Library.  And today I’m pleased to announce, the time has come!  For the next three months (now until July 31st, 2012), if you send us your courseware library receipt, we will refund you 10% of your purchase price!

As an alternative, if instead of receiving a rebate you would like your 10% to go to a charity, we have selected three charities that we would like to support with the promotion:

Susan F Koman
Action against
Hunger Mercy Corps

To take advantage of this promotion, simply send your receipts to rebate@architectingconnectedsystems.com and specify if you want a check or for the rebate to go to one of the above charities.  Please allow 45 days to process after program ends for rebates to be maileddeposited.

Thanks again for being an ACS customer and we look forward to what the rest of 2012 will bring and beyond (SharePoint 2013 of course)!

Thanks again,
Chris Givens
CEO, Architecting Connected Systems

Posted on

WebDAV, Options and PropFind…Oh My…

I have been struggling with a particular problem at my current contract.  One where windows 7 machines could connect to a webdav path to sharepoint sites and libraries, but older windows XP machines could not.

Originally I thought it might be something I configured on the server (like permissive versus strict), but it wasn't anything to do with that.  You see alot of posts on server admins turning on the IIS webdav, but that wasn't it either.  

For this particular environment, it just so turned out that StoragePoint was installed on the very same web app that we were trying to connect too, but it was simple chance that it was installed on this particular web app and thusly it wasn't StoragePoint that was causing the problem (but I won't old my breath that it won't cause issues later, there Librarian product has so many bugs it isn't funny).

So what was it?  I was able to track down the issue using WireShark.  I would start the trace and then follow the HTTP calls (which on the working mapped drive was PROPFINDs, and the broken web app, OPTIONS calls).  It turns out that once you have a successful OPTIONS response, windows will cache the information in your registry.  After that, all you will see are PROPFINDs.  This is documented here:

http://support.microsoft.com/kb/2019105

After watching the TCP stream, I noticed the last call to the OPTIONS after all the authentication was a 404 error.  That's when I was able to make some nice google searches and stumbled upon this blog by sudeepg:

http://blogs.msdn.com/b/sudeepg/archive/2010/04/02/webdav-folders-no-full-functionality.aspx

He found that you have to have a root site collection in order to respond to the OPTIONS http request.  Sure enough, after adding the root site collection (in my dev environment which doens't need a root site collection), the WEBDAV calls started working.

Talk about a shot in the head…its the simple things…
Chris

Follow me on twitter: @givenscj

Check out the previous blog post:
https://blogs.architectingconnectedsystems.com/blogs/cjg/archive/2012/05/14/Awesome-ACS-SharePoint-Tools-_2D00_-On-Sale-Now_2100_–SharePoint-Filter_2C00_-SharePoint-Desinger-LockDown-and-SharePoint-Durable-Urls.aspx

Posted on

Awesome ACS SharePoint Tools – On Sale Now! SharePoint Filter, SharePoint Designer LockDown and SharePoint Durable Urls

I have been a busy SharePoint programmer as of late and I'm proud to announce that all my work has led to some new and innovative solutions for SharePoint customers!  I'll be promoting these heavily over the next few months as they have been needed for some time now!

Here are the three tools:

You can get all three of them for $3595!!!

Here's what they do and why I built them:

SharePoint Designer LockDown –  customers are fed up with being limited to making users Site Collection admins just so they can use SharePoint Designer.  The other permissions like "Use Remote APIs" is also lacking and turning it off pretty much isn't an option when trying to keep regular non-super users from getting access to Designer (it is free ya know, and it doesn't help they put the link to download it on the ribbon menus).  This tool uses a SharePoint list where you place the domainusername of the person, and the site collection url that they should have access too.  If they aren't in the list, they don't get access!  The tool itself is scoped at a web application level.   Check out the video demo here

SharePoint Durable Urls – This is a feature that is in Oracle Portal and has been lacking in SharePoint for…oh yeah…forever.  This is similar to a bit.ly type of system, but rather than just providing shorten urls, it provides "durable" urls.  If the item is ever moved, renamed or the main URL of the site is changed.  The durable URL WILL STILL WORK!  Novel right?  And no, this is not the same as the DocumentID feature in SharePoint.  DocIDs in SharePoint are bloated ugly creatures that will kill you in the middle of the night! Check out the video demo here

SharePoint Filter – Needless to say, I'm pretty sure everyone has gotten bitten by legal and HR on those nice "Social" features in SharePoint.  Namely…User Profile properties (like your status message), social tagging and the note board.  I have heard horror stories about mis-behaving employees posting very naughty messages on noteboards and tagging items with inappropriate terms.  SharePoint Filter will prevent this from occurring and it will allow you to clean out anything you might already have in your Farm's social databases!  Check out the video demo here

To see these tools in action via a live demo, send me an email and I'll give you a username and password to hit the site and see how they work!

I will have a referral program where you can earn up to 15% of the sales.  Email me about potential customers and I'll get you tagged to them!  First come first served!

Enjoy!
Chris

Posted on

HUGE Permissions bug with Import-SPWeb -includeusersecurity

The Export-SPWeb and Import-SPWeb is a tool for moving sites around, but with all the changes that have occured with SharePoint 2010 over the past two years, it seems some of the plumbing isn't connected correctly.

One of the issues we were seeing is that after importing a site, users are able to get to the site, but not to libraries underneath the site.  You would see ULS logs like:

OriginalPermissionMask check failed. asking for 0x00020000, have 0x00000000 

Nice list of the perm masks for reference:
http://social.msdn.microsoft.com/Forums/en/sharepointdevelopment/thread/c32bc150-7249-423d-8018-d2f23afc1f3b

Exception: System.UnauthorizedAccessException: Attempted to perform an unauthorized operation.. 

Access Denied for SITEURL. StackTrace:    at Microsoft.SharePoint.Utilities.SPUtility.HandleAccessDenied(HttpContext context)     at Microsoft.SharePoint.Utilities.SPUtility.HandleAccessDenied(Exception ex)     at Microsoft.SharePoint.SPWeb.GetWebPartPageContent(Uri pageUrl, Int32 pageVersion, PageView requestedView, HttpContext context, Boolean forRender, Boolean includeHidden, Boolean mainFileRequest, Boolean fetchDependencyInformation, Boolean& ghostedPage, Byte& verGhostedPage, String& siteRoot, Guid& siteId, Int64& bytes, Guid& docId, UInt32& docVersion, String& timeLastModified, Byte& level, Object& buildDependencySetData, UInt32& dependencyCount, Object& buildDependencies, SPWebPartCollectionInitialState& initialState, Object& oMultipleMeetingDoclibRootFolders, String& redirectUrl, Boolean& ObjectIsList, Guid& listId)     at Microsoft.SharePoint.ApplicationRuntime.SPRequestModuleData.FetchWebPartPageInformationForInit(HttpContext context, SPWeb spweb, Boolean mainFileRequest, String path, Boolean impersonate, Boolean& fGhostedPage, Byte& verGhostedPage, Guid& docId, UInt32& docVersion, String& timeLastModified, SPFileLevel& spLevel, String& masterPageUrl, String& customMasterPageUrl, String& webUrl, String& siteUrl, Guid& siteId, Object& buildDependencySetData, SPWebPartCollectionInitialState& initialState, String& siteRoot, String& redirectUrl, Object& oMultipleMeetingDoclibRootFolders, Boolean& objectIsList, Guid& listId, Int64& bytes)     at Microsoft.SharePoint.ApplicationRuntime.SPRequestModuleData.GetFileForRequest(HttpContext context, SPWeb web, Boolean exclusion, String virtualPath)     at Microsoft.SharePoint.ApplicationRuntime.SPRequestModule.InitContextWeb(HttpContext context, SPWeb web)     at Microsoft.SharePoint.WebControls.SPControl.SPWebEnsureSPControl(HttpContext context)     at Microsoft.SharePoint.ApplicationRuntime.SPRequestModule.GetContextWeb(HttpContext context)     at Microsoft.SharePoint.ApplicationRuntime.SPRequestModule.PostResolveRequestCacheHandler(Object oSender, EventArgs ea)     at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()     at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)     at System.Web.HttpApplication.PipelineStepManager.ResumeSteps(Exception error)     at System.Web.HttpApplication.BeginProcessRequestNotification(HttpContext context, AsyncCallback cb)     at System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr managedHttpContext, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr managedHttpContext, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr managedHttpContext, IntPtr nativeRequestContext, IntPtr moduleData, Int32 flags)     at System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr managedHttpContext, IntPtr nativeRequestContext, IntPtr moduleDa…

Looking at the permissions for the library (Site Actions->Site Permissions OR Library->Library Permissions), you WOULD see what you are supposed to see as far as permissons are concerned (even the CheckPermissions tool shows what we want).  However, looks are DECEIVEING!  Also, when we try to break permissions, it would continue to show that the list was inheriting from the parent.  No errors would be thrown in the UI.  When using powershell and tracing the verbose calls in ULSViewer you would actually see some more revealing details:

05/10/2012 09:23:21.29 PowerShell.exe (0x1180) 0x0D3C SharePoint Foundation General bj8s Medium setSecurityScopeUnique: start 
05/10/2012 09:23:21.29 PowerShell.exe (0x1180) 0x0D3C SharePoint Foundation General bj8t Medium setSecurityScopeUnique: already unique 
05/10/2012 09:23:21.29 PowerShell.exe (0x1180) 0x0D3C SharePoint Foundation General bj8w Medium setSecurityScopeUnique: finished 

Notice the second line:  "Already unique"…WTF…no its not!  The UI shows it is inheriting permissions!  So what is going on here?

Turning on the handy SQL Profiler, I did the following powershell on a "working" list:

$web = get-spweb http://blah
$list = $web.lists["Project Documents"]
$list.BreakInheritance($true, $false)
$list.Update()

What you see in profiler is a call to:

declare @p15 uniqueidentifier
set @p15='4EE7D0C0-0133-4600-A96E-53BD0D133DEF'
exec proc_SecChangeToUniqueScope @SiteId='EFAF8E96-FA9F-4816-A965-CE7A8C6ED8AC',@WebId='E504097A-2BA1-400F-BCC2-5DE6B32E8615',@OldScopeId='3FB9F8DA-A957-4B82-B4AA-FECB3B1B4551',@CopyFromScopeId='3FB9F8DA-A957-4B82-B4AA-FECB3B1B4551',@Url=N'asi/Shared Documents',@DocId='00000000-0000-0000-0000-000000000000',@bIsWeb=0,@UserId=1073741823,@CopyAnonymousMask=1,@CopyRoleAssignments=1,@ClearSubScopes=0,@bBreakBySiteOwner=0,@ReturnAuditMask=1,@MaxScopeInList=50000,@NewScopeId=@p15 output,@RequestGuid='00000000-0000-0000-0000-000000000000'
select @p15

When performing the same operation on a "broken" list (remember, from an imported web), you DO NOT see this call made.  So I decided, lets look at that stored procedure and see what it does.  The SP will actually look at the "Perms" table for a "ScopeId", then it will replicate what it needs in all the related tables to create a copy of a parent scope (actually doesn't have to be a parent, but the SharePoint team doesn't let us do this through the UIs – wouldn't that be handy!).

So what is the problem?  Well…the problem centers on that pesky "ScopeId".  What is happening is that SharePoint has TWO ways to get permissions information:

  1. Get the scopeid from the AllLists table (using urls or ids)
  2. Get the scopeid from the perms table (using urls or ids)

UPDATE:  There is also a scopeid on the AllDocs table that also gets out of whack!

In some places the object model and the "obfuscated" SPRequest COM objects are making two different calls!  One will look at the AllLists table, the other will look
at the Perms table.  This is the BUG.  ALL access APIs are using the "Perms" table, all UI APIs (including the CheckPermissions tool) are using the "AllLists" table.  They should look at the same place!  What is happening is the scopeID of imported objects is being set to the parent webs scopeID!  So even though the UI shows you what you WANT to see, the internals think that the permissions are that of the web's parent web!  THAT'S NOT GOOD!

How do you fix this?  Your not going to like the answer…none of the object model calls would reset the scopeid of the lists or webs.  I had to go into the database and reset the mapping of the scopeIds.  Once I did this, BAM!  Everything started working again…

Second big bug I have found in SharePoint 2010 in the last two weeks.

SQL Queries to help you find things:

Find List Scopes:

SELECT tp_Title, tp_ScopeId
FROM [AllLists]
where
tp_WebId = 'GUID'

Find Doc Scopes:

SELECT WebId, DirName, LeafName, scopeid
FROM [AllDocs]
where DirName like '%blah%'

Find Web Scopes:

SELECT ID, FullUrl, scopeid
FROM [AllWebs]
where fullurl like '%blah%'

Find Scope Permissions:

declare @scopeid as varchar(max)
set @scopeid = 'A3A5D2C8-5624-4E2A-BE7A-7D51C26DB081'

SELECT *
FROM [Perms]
where scopeid in (@scopeid)

select *
FROM RoleAssignment ra, Roles r
where ScopeId = @scopeid
and ra.RoleId = r.RoleId

Grrr….
Chris

Follow me on twitter: @givenscj

Check out the previous blog post:
https://blogs.architectingconnectedsystems.com/blogs/cjg/archive/2012/04/20/ATTENTION_3A00_-Database-Leak-in-SharePoint-2010_210021002100_.aspx

Posted on

Project Server PSI WCF bug – WSEC_CAT_UID and WSEC_GRP_UID

At one of my customers we are automating the project and project site creation for Project Server 2010.  As part of that, we wanted the proper permissions to be setup for each resource/user.  This includes making them in the "Project Manager" group and the "My Projects" and "My Resources" categories.  The code wasn't very straight forward, but I got most of it working (creating the resource and making a resource a user).  The part that doesn't work is the group and categories assignment.  Here is the code:

SvcSecurity.SecurityGroupsDataSet sgDataSet = new SvcSecurity.SecurityGroupsDataSet();
//Get the security group
SvcSecurity.SecurityGroupsDataSet sglist = securityClient.ReadGroupList();
SvcSecurity.SecurityGroupsDataSet.SecurityGroupsRow groupDs = null;
foreach (SvcSecurity.SecurityGroupsDataSet.SecurityGroupsRow sg in sglist.SecurityGroups)
{
if (sg.WSEC_GRP_NAME == groupName)
{
groupDs = sg;|
}
}

SvcSecurity.SecurityGroupsDataSet ds = new SvcSecurity.SecurityGroupsDataSet();
// Specify which users belong to the new group.

SvcSecurity.SecurityGroupsDataSet ds = new SvcSecurity.SecurityGroupsDataSet();
// Specify which users belong to the new group.

SvcSecurity.SecurityGroupsDataSet.GroupMembersRow groupMembersRow = ds.GroupMembers.NewGroupMembersRow();
groupMembersRow.WSEC_GRP_UID = groupDs.WSEC_GRP_UID;
// Add the GUID of the resource to the group.
groupMembersRow.RES_UID = NewResGuid;
ds.GroupMembers.AddGroupMembersRow(groupMembersRow);
securityClient.SetGroups(ds);

SecurityGroupsDataSet.GroupMembersRow groupMembersRow = ds.GroupMembers.NewGroupMembersRow();
groupMembersRow.WSEC_GRP_UID = groupDs.WSEC_GRP_UID;
// Add the GUID of the resource to the group.
groupMembersRow.RES_UID = NewResGuid;
ds.GroupMembers.AddGroupMembersRow(groupMembersRow);
securityClient.SetGroups(ds);

The problem lies in the finding of the group.  The DataSet that is returned returns the WSEC_GRP_UID.  This particular column is NOT what the following AddGroupMembersRow call wants!  It actually is looking for a second GUID in the database called WSEC_GRP_GUID.  This is not returned in the PSI call and therefore, you have ZERO chance of successfully going through the PSI apis to get the id.  You end up having to query the database directly…yuk.

Anyone that develops PSI out there?  You really need to fix this…

Chris

Posted on

ATTENTION: Database Leak in SharePoint 2010!!!

I have been doing a massive Oracle Portal migration, 3000+ pages of content, with a targeted subset of that in my dev envrionment at just over 3GB of content.  I noticed the other day that my development content database kept getting bigger and bigger.  Running a "dbcc checkdb" pointed me to the largest tables.  After clearing the Audit logs, EventLogs and everything else I could think of, then shrinking the database files, the databse size was still WAY bigger than what it was supposed to be (it should be 35GB, but it is currently 72GB).  The dbcc command pointed to the AllDocStreams table as the source of the problem.  I thought it possible the new web recycle bin may be the culprit, so I set out to find the root cause and prove to the community that "Redmond, we have a problem"

My first test was to create a web and then delete it over and over again.  That didn't change the row sizes in the database.  Next test was to do the same thing, only add a document to the doc library.  That didn't cause any leaks.  The last step was to enable versioning and to upload a document a few times, then delete the webs…VIOLA!  Found the leak!  Want to try this on your own…here is the PowerShell that proves that we have a big issue here:

function CheckRows($table, $conn)
{
$sql = "select count(*) as count from $table";
$cmd = new-object system.data.sqlclient.sqlcommand($sql, $conn)

$reader = $cmd.executereader()

while($reader.read())
{
$line = $reader["count"].tostring()
}

$reader.close()

"$table has " + $line + " rows"
}

function ReportRowCounts($cdb)
{

$connString = $cdb.LegacyDatabaseConnectionString

$conn = new-object system.data.sqlclient.sqlconnection
$conn.connectionstring = $connstring
$conn.open()

CheckRows "AllDocs" $conn
CheckRows "AllDocStreams" $conn
CheckRows "AllDocVersions" $conn

$conn.close()

}

function UploadFile($web)
{

$list = $web.lists["Shared Documents"]
$list.EnableModeration = $true
$list.EnableMinorVersions = $true
$list.contenttypesenabled = $true
$list.update()

$spFolder = $list.rootfolder
$spFileCollection = $spFolder.Files
$file = get-item "c:scriptsexport.zip"
$spfile = $spFileCollection.Add($file.name,$file.OpenRead(),$true)
$spfile.checkout()
$spfile.checkin("checkin")
$spfile.approve("approved")

}

$cdb = get-spcontentdatabase "WSS_Content_Contoso"
$web = get-spweb http://www.contoso.com

ReportRowCounts($cdb)

#create a site
$subweb = $web.webs.add("Test1", "Test1", "", 1033, "STS#0", $false, $false);

ReportRowCounts($cdb)

$subweb.delete()

ReportRowCounts($cdb)

$subweb = $web.webs.add("Test1", "Test1", "", 1033, "STS#0", $false, $false);
UploadFile $subweb
UploadFile $subweb
UploadFile $subweb
UploadFile $subweb
UploadFile $subweb

ReportRowCounts($cdb)

$subweb.delete()

ReportRowCounts($cdb)

You will see the AllDocStreams table continues to get larger and larger.  This is the table that has your BLOBS!  The worst table to have a database leak!  From a high level, what does this mean?  It means if you turn on versioning, and say upload a 1Gb file and then make 5 changes to it, you will have 5GB of storage used in your database.  If you then delete the web, you will lose file pointers in the database and that 5GB will remain in the database forever and never get cleaned up.  Now, how likely is it that you are creating and deleting webs in production?  Not too many people will have a fancy process setup like this, but in Development and QA, you will.  I can't afford to keep restorting a "clean" copy of my development databases everytime it gets too big.

NOTE:  This bug shows up in all builds after SP1 (the web recycle build), my environment is the latest release build of 14.6112.5000.

Here's how to determine if you are having a problem and how BIG it is:

$global:count = 0
$global:size = 0

function CheckDatabase($cdb)
{
$connString = $cdb.LegacyDatabaseConnectionString
$conn = new-object system.data.sqlclient.sqlconnection
$conn.connectionstring = $connstring
$conn.open()

$sql = "select count(*) as count, sum(datalength(content)) as size from alldocstreams where id not in (select id from alldocs)";
$cmd = new-object system.data.sqlclient.sqlcommand($sql, $conn)
$reader = $cmd.executereader()

while($reader.read())
{
$count = [double]::parse($reader["count"].tostring())
$size = [double]::parse($reader["size"].tostring())

}

$reader.close()
$conn.close()

$global:Count += $count
$global:size += $size

}

$cdbs = get-spcontentdatabase

foreach ($cdb in $cdbs)
{
CheckDatabase $cdb
}

"Orphaned rows: " + $global:count
"Orphaned rows size: " + $global:size + " bytes"

Microsoft…you need to get us a cumulative updatehot fix RIGHT NOW.

UPDATE:  AUGUST 2012 Cumulative Update fixes this bug:
http://technet.microsoft.com/en-us/sharepoint/ff800847.aspx

Not one to say enjoy too, but at least I know what is going on now and so do you!
Chris

Follow me on twitter: @givenscj

Check out the previous blog post:
https://blogs.architectingconnectedsystems.com/blogs/cjg/archive/2012/04/20/Project-Server-PSI-WCF-bug-_2D00_-WSEC_5F00_CAT_5F00_UID-and-WSEC_5F00_GRP_5F00_UID.aspx