Updated 3/18/2015:
Turns out when you migrate from one domain to another, this error pops up yet again (even after running the migrate user commands)! Another option of fixing this is to open *each* site collection and delete the workflows. Then deactive and active the feature. Then you can create sites again!
Yet again, another weird bug in SharePoint 2010 that I had to track down and fix this week! What is this? Here's what happened:
My customer has a few workflows created to approve important documents. All of a sudden, these workflows stopped working with an access denied error message. In looking at the log files, you would see this (it's rather long):
06/27/2012 06:40:02.96 w3wp.exe (0x1314) 0x2A00 SharePoint Foundation General ftd0 Verbose Access Denied. Exception: System.UnauthorizedAccessException: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)) at Microsoft.SharePoint.Library.SPRequestInternalClass.OpenWeb(String bstrUrl, String& pbstrServerRelativeUrl, String& pbstrTitle, String& pbstrDescription, String& pbstrTitleResourceId, String& pbstrDescriptionResourceId, Guid& pguidID, String& pbstrRequestAccessEmail, UInt32& pwebVersion, Guid& pguidScopeId, UInt32& pnAuthorID, UInt32& pnLanguage, UInt32& pnLocale, UInt16& pnTimeZone, Boolean& bTime24, Int16& pnCollation, UInt32& pnCollationLCID, Int16& pnCalendarType, Int16& pnAdjustHijriDays, Int16& pnAltCalendarType, Boolean& pbShowWeeks, Int16& pnFirstWeekOfYear, UInt32& pnFirstDayOfWeek, Int16& pnWorkDays, Int16& pnWorkDayStartHour, Int16& pnWorkD… aed86901-9e96-4e1d-acac-32f5d8bba3d8
06/27/2012 06:40:02.96* w3wp.exe (0x1314) 0x2A00 SharePoint Foundation General ftd0 Verbose …ayEndHour, Int16& pnMeetingCount, Int32& plFlags, Boolean& bConnectedToPortal, String& pbstrPortalUrl, String& pbstrPortalName, Int32& plWebTemplateId, Int16& pnProvisionConfig, String& pbstrDefaultTheme, String& pbstrDefaultThemeCSSUrl, String& pbstrThemedCssFolderUrl, String& pbstrAlternateCSSUrl, String& pbstrCustomizedCssFileList, String& pbstrCustomJSUrl, String& pbstrAlternateHeaderUrl, String& pbstrMasterUrl, String& pbstrCustomMasterUrl, String& pbstrSiteLogoUrl, String& pbstrSiteLogoDescription, Object& pvarUser, Boolean& pvarIsAuditor, UInt64& ppermMask, Boolean& bUserIsSiteAdmin, Boolean& bHasUniquePerm, Guid& pguidUserInfoListID, Guid& pguidUniqueNavParent, Int32& plSiteFlags, DateTime& pdtLastContentChange, DateTime& pdtLastSecurityChange, String& pbstrWelcomePage, Boolean& pb… aed86901-9e96-4e1d-acac-32f5d8bba3d8
06/27/2012 06:40:02.96* w3wp.exe (0x1314) 0x2A00 SharePoint Foundation General ftd0 Verbose …OverwriteMUICultures, Boolean& pbMUIEnabled, String& pbstrAlternateMUICultures, Int32& puiVersion, Int16& pnClientTag) at Microsoft.SharePoint.Library.SPRequest.OpenWeb(String bstrUrl, String& pbstrServerRelativeUrl, String& pbstrTitle, String& pbstrDescription, String& pbstrTitleResourceId, String& pbstrDescriptionResourceId, Guid& pguidID, String& pbstrRequestAccessEmail, UInt32& pwebVersion, Guid& pguidScopeId, UInt32& pnAuthorID, UInt32& pnLanguage, UInt32& pnLocale, UInt16& pnTimeZone, Boolean& bTime24, Int16& pnCollation, UInt32& pnCollationLCID, Int16& pnCalendarType, Int16& pnAdjustHijriDays, Int16& pnAltCalendarType, Boolean& pbShowWeeks, Int16& pnFirstWeekOfYear, UInt32& pnFirstDayOfWeek, Int16& pnWorkDays, Int16& pnWorkDayStartHour, Int16& pnWorkDayEndHour, Int16& pnMeetingC… aed86901-9e96-4e1d-acac-32f5d8bba3d8
06/27/2012 06:40:02.96* w3wp.exe (0x1314) 0x2A00 SharePoint Foundation General ftd0 Verbose …ount, Int32& plFlags, Boolean& bConnectedToPortal, String& pbstrPortalUrl, String& pbstrPortalName, Int32& plWebTemplateId, Int16& pnProvisionConfig, String& pbstrDefaultTheme, String& pbstrDefaultThemeCSSUrl, String& pbstrThemedCssFolderUrl, String& pbstrAlternateCSSUrl, String& pbstrCustomizedCssFileList, String& pbstrCustomJSUrl, String& pbstrAlternateHeaderUrl, String& pbstrMasterUrl, String& pbstrCustomMasterUrl, String& pbstrSiteLogoUrl, String& pbstrSiteLogoDescription, Object& pvarUser, Boolean& pvarIsAuditor, UInt64& ppermMask, Boolean& bUserIsSiteAdmin, Boolean& bHasUniquePerm, Guid& pguidUserInfoListID, Guid& pguidUniqueNavParent, Int32& plSiteFlags, DateTime& pdtLastContentChange, DateTime& pdtLastSecurityChange, String& pbstrWelcomePage, Boolean& pbOverwriteMUICultures, Boolea… aed86901-9e96-4e1d-acac-32f5d8bba3d8
06/27/2012 06:40:02.96* w3wp.exe (0x1314) 0x2A00 SharePoint Foundation General ftd0 Verbose …n& pbMUIEnabled, String& pbstrAlternateMUICultures, Int32& puiVersion, Int16& pnClientTag). aed86901-9e96-4e1d-acac-32f5d8bba3d8
06/27/2012 06:40:02.96 w3wp.exe (0x1314) 0x2A00 SharePoint Foundation General 8gs1 Verbose Access Deni
ed for /contoso/_layouts/IniWrkflIP.aspx?List={b713a4ad-747e-4980-ad1c-c420d3f588b7}&ID=1&TemplateID={3cf9d212-c83d-4486-9fce-ee8d00efacbc}&Source=blah StackTrace: at Microsoft.SharePoint.Utilities.SPUtility.HandleAccessDenied(HttpContext context) at Microsoft.SharePoint.Utilities.SPUtility.HandleAccessDenied(Exception ex) at Microsoft.SharePoint.Library.SPRequest.OpenWeb(String bstrUrl, String& pbstrServerRelativeUrl, String& pbstrTitle, String& pbstrDescription, String& pbstrTitleResourceId, String& pbstrDescriptionResourceId, Guid& pguidID, String& pbstrRequestAccessEmail, UInt32& pwebVersion, Guid& pguidScopeId, UInt32& pnA… aed86901-9e96-4e1d-acac-32f5d8bba3d8
06/27/2012 06:40:02.96* w3wp.exe (0x1314) 0x2A00 SharePoint Foundation General 8gs1 Verbose …uthorID, UInt32& pnLanguage, UInt32& pnLocale, UInt16& pnTimeZone, Boolean& bTime24, Int16& pnCollation, UInt32& pnCollationLCID, Int16& pnCalendarType, Int16& pnAdjustHijriDays, Int16& pnAltCalendarType, Boolean& pbShowWeeks, Int16& pnFirstWeekOfYear, UInt32& pnFirstDayOfWeek, Int16& pnWorkDays, Int16& pnWorkDayStartHour, Int16& pnWorkDayEndHour, Int16& pnMeetingCount, Int32& plFlags, Boolean& bConnectedToPortal, String& pbstrPortalUrl, String& pbstrPortalName, Int32& plWebTemplateId, Int16& pnProvisionConfig, String& pbstrDefaultTheme, String& pbstrDefaultThemeCSSUrl, String& pbstrThemedCssFolderUrl, String& pbstrAlternateCSSUrl, String& pbstrCustomizedCssFileList, String& pbstrCustomJSUrl, String& pbstrAlternateHeaderUrl, String& pbstrMasterUrl, String& pbstrCustomMasterUrl, String& pbs… aed86901-9e96-4e1d-acac-32f5d8bba3d8
06/27/2012 06:40:02.96* w3wp.exe (0x1314) 0x2A00 SharePoint Foundation General 8gs1 Verbose …trSiteLogoUrl, String& pbstrSiteLogoDescription, Object& pvarUser, Boolean& pvarIsAuditor, UInt64& ppermMask, Boolean& bUserIsSiteAdmin, Boolean& bHasUniquePerm, Guid& pguidUserInfoListID, Guid& pguidUniqueNavParent, Int32& plSiteFlags, DateTime& pdtLastContentChange, DateTime& pdtLastSecurityChange, String& pbstrWelcomePage, Boolean& pbOverwriteMUICultures, Boolean& pbMUIEnabled, String& pbstrAlternateMUICultures, Int32& puiVersion, Int16& pnClientTag) at Microsoft.SharePoint.SPWeb.InitWeb() at Microsoft.SharePoint.SPWeb.get_Title() at Microsoft.SharePoint.SPSite.OpenWeb(Guid gWebId, Int32 mondoHint) at Microsoft.SharePoint.Workflow.SPWinOEWSSService.GetWebForWorkflow(SPWorkflow wf, SPWorkflowUserContext runAsUser) at Microsoft.SharePoint.Workflow.SPWinOEWSSService.get… aed86901-9e96-4e1d-acac-32f5d8bba3d8
06/27/2012 06:40:02.96* w3wp.exe (0x1314) 0x2A00 SharePoint Foundation General 8gs1 Verbose …_Web() at Microsoft.SharePoint.Workflow.SPWinOEWSSService.GetWebForListItemService() at Microsoft.SharePoint.Workflow.SPWinOEWSSService.UpdateModerationStatus(Guid id, Guid listId, SPItemKey itemKey, SPModerationStatusType newModerationStatus, String comments) at Microsoft.Office.Workflow.Actions.SetTaskProcessItemModerationStatus.DoUpdate(ActivityExecutionContext context) at Microsoft.SharePoint.WorkflowActions.WaitForDocumentUnlockActivity.Execute(ActivityExecutionContext executionContext) at Microsoft.Office.Workflow.Actions.SetTaskProcessItemModerationStatus.Execute(ActivityExecutionContext context) at System.Workflow.ComponentModel.ActivityExecutor`1.Execute(T activity, ActivityExecutionContext executionContext) at System.Workflow.ComponentModel.ActivityExe… aed86901-9e96-4e1d-acac-32f5d8bba3d8
Looking at the stack trace, you can see that the code is making a call to the GetWebForWorkflow method of the SPWinOEWSSService. In reflecting this method, you will see that it can take one of three paths:
- Execute as the system account
- Execute as the workflow template author
- Execute as the workflow template owner
The system account always has access, so it was obvious that this wasn't the path it was taking. The owner seems to always be set to "-1" and no where in the massive amount of code that I reviewed did it ever get set to anything else. That left the author as the most probably entry point. When reviewing where the author comes from, you find a tangled set of code that really doesn't make much sense at all (hmm…SharePoint in all its glory). After decoding the entanglement, I found that the author comes from some set of properties that end up in a object array for the SPWebWorkflow. This array is built from a call to SPRequest.GetListItemWorkflowAsSafeArray. This of course means that you can't go any farther into the code unless you get really fancy building a SPRequest wrapper (which can be done, but its really hard). Because the properties are built in a somewhat OnDemand fashion, you don't really have any way of knowing when that code will be executed…and because I'm in production, I can't sit here and do IISRESET's all day long watching for some workflow related Stored Procedure via SQL Profiler.
So how did I figure out what was up with this? Well, the hint was in the SPRequest method call name. GetListItemWorkflowAsSafeArray. ListItem? Which implies the metadata for the workflow is in the list. Which means the author info is also coming from the listlistitem. So where do workflows get stored in a siteweb? They get saved to the \_catlogswfpub library at the site collection root web.
In looking at this library through SharePoint Designer, you will see that when you first activate the workflow features, the files are created by THE USER who actually made the feature activation call (whether through the UI or stsadm or PowerShell). Hmm, very poor design.
Why is it poor design? Well, the author comes from the created field for the list item that has the workflow template defined. In our case, this account was recently DISABLED. When an account becomes disabled, you can't do JACK in SharePoint. This includes executing the workflows that were
built off the template in the first place. So what are your options?
- Delete all the files, reactivate the workflows – YOU MUST DELETE THE FILES, if you do not, the files will stay with the created field still set to the old userId (If you have ever taken any of my course, you know rule number one is "SharePoint never deleted anything"). This means you'll still have the ACCESS DENIED error even after you deactivate and re-activate the workflows
- Try to modify the CREATED and MODIFIED columns in the list to change the SPUser (I didn't try this, but it seems like it should work)
So, have you seen this? Probably not. Will every SharePoint farm on the planet eventually have this issue – YES! You won't keep accounts around forever (unless you have a very poor security policy). So it is worthwhile to review all your site collections and check to see who the workflow files are owned by. Hopefully you have an SPAdmin or SPFarm account that you can use to enable the workflows with. SO…NET NET:
Never enable workflow features as a regular user, always use some SharePoint account that will never be disabled
Yuk, yuk…yuk|
Chris