ACS Blogs

Oracle Portal 10g to SharePoint Migration

cjg — Fri, 27 Jan 2012 17:08:00 GMT

Do you have Oracle Portal 10g? Want to move to SharePoint with full fidelity? I have scripts that will do exactly that! This would include:

Portal Groups
All pages with all "things" (URLs, content, files)
Files with versions and comments
Workflows
Permissions

Contact me at:
chris@architectingconnectedsystems.com

I'd be happy to chat with you about moving to SharePoint 2010!

Oh, and here are some interesting facts about Oracle Portal versus SharePoint:

Workflows have steps for approvals (whereas SharePoint an approval is just a single step)
Groups are global in Oracle Portal (SharePoint groups are limited to site collections)
Oracle uses a concept of "things". Everything is one and everything can have other "things". This is *super* powerful notion and means that a single page can contain "things", where a thing can be a URL, a file, content, etc. SharePoint pages can only have web parts on them.
Each "thing" on the page can have security for it, SharePoint doesn't let you do this
Each "thing" has a version (no matter what it is), SharePoint doesn't have this
Each "thing" on a page can have a workflow. SharePoint workflows are tied to list items, you can't workflow web parts
Oracel has a built in notion of shortcut links (a GUID that takes you to the page you want to hit), SharePoint doesn't

So what saves SharePoint at the end of the day?

Office Web Apps is the savior, you can't do fancy Office Document manipulation in Oracle
Oracle Portal is *much* more expensive than SharePoint

Chris

1 Year Anniversary Tribute - eBay upgrade video

cjg — Fri, 20 Jan 2012 01:40:00 GMT

As a tribute to the 1-year anniversary of the start of the eBay upgrade project, I thought I would share the video I created to show at our celebration party in San Jose. It is a great video, and shows how much fun we had (and you should too) when we did the upgrade. The team deserves a tremendous amout of credit for the work and effort they put in...so here it is:

http://youtu.be/EnK2c_lntOY

As a run down, here is what occurs in the video:

The SharePoint odessey - the 2001 to 2010 evolution. When the monkey discovers the monolith and evolves. Simiarly, eBay's discovery of SharePoint 2010 starts with a monolithic-white board (its only appropriate that we use Richard Struass's Also sprach Zarathustra to show this)
You see that it really is a love of "SharePoint" that progresses the joining of all parties
As we drive through the initial requirements (found and un-found), you will realize that this project was done in 3.5 months (when similar quotes were at 14 months at 4x the cost). This drives the selection of Queen's..."Under Pressure"
At one point you will see Ramin and I on the ground with our hands in the air...it was a fun moment as we were not really knowing what was going to happen...
You will see photos of our countdown timer of the night we went live (we had an upgrade window of the weekend to finish the upgrade successfully). It slowly winds down until the "GO-NOGO Moment"
As the pictures are displayed, you will see various moments over the 3.5 months of us working intent-ly getting the job done
Not everything went as planned and hence you see the "GO-NOGO" which occured at least 4-5 times (some elements include NDR64 discoveries and the infamous Search ResultsProvider that has claimed countless develpers...
Katy Perry's song of "Firework" is perfect for our dispay of the team members that worked so very hard to make the upgrade happen and thusly are amazingly bright and wonderful people that randomly came together to accomplish something amazing (NOTE: there are moments in the song lyrics that relate to hidden moments)
Let me be the first to say that eBay has an amazing NetOps team that just kicks butt (Kenny Cheng and Ken MacIntosh), they saved our a$$ a few times...
You will see a snapshot of a fortune cookie that "I" got...it was the week of the upgrade and it said "Your hard work is about to pay off", it was the best fortune EVER!
There were some times when we had some pretty "interesting" dinners...hence the snapshot of the menu...
The video ends with 00:00:00 and pictures of the eBay intranet running SharePoint 2010 and our SharePoint Conference 2011 session - "How eBay successfully upgrade their intranet to 2010"

Enjoy!
Chris

Fixing: Action 4.0.23.0 of Microsoft.SharePoint.Upgrade.SPSiteWssSequence failed.

cjg — Thu, 19 Jan 2012 02:11:00 GMT

This was a tough error, THAT IS AN UPGRADE SHOWSTOPPER, that I ran into a few months ago. Just now getting around to blogging it. Here's the full error:

[powershell] [SPSiteWssSequence] [ERROR] [9/29/2011 12:02:27 PM]: Action 4.0.23.0 of Microsoft.SharePoint.Upgrade.SPSiteWssSequence failed.
[powershell] [SPSiteWssSequence] [ERROR] [9/29/2011 12:02:27 PM]: Inner Exception: Field name already exists.

The name used for this field is already used by another field in the list. Select another name and try again.<nativehr>0x81020013</nativehr><nativestack></nativestack>
[powershell] [SPSiteWssSequence] [ERROR] [9/29/2011 12:02:27 PM]: at Microsoft.SharePoint.Library.SPRequestInternalClass.UpdateField(String bstrUrl, String bstrListName, String bstrXML)
at Microsoft.SharePoint.Library.SPRequest.UpdateField(String bstrUrl, String bstrListName, String bstrXML)
[powershell] [SPSiteWssSequence] [ERROR] [9/29/2011 12:02:27 PM]: Exception: Field name already exists.

The name used for this field is already used by another field in the list. Select another name and try again.
[powershell] [SPSiteWssSequence] [ERROR] [9/29/2011 12:02:27 PM]:    at Microsoft.SharePoint.SPGlobal.HandleComException(COMException comEx)
   at Microsoft.SharePoint.Library.SPRequest.UpdateField(String bstrUrl, String bstrListName, String bstrXML)
   at Microsoft.SharePoint.SPField.UpdateCore(Boolean bToggleSealed)
   at Microsoft.SharePoint.SPField.Update()
   at Microsoft.SharePoint.SPFieldIndexCollection.IndexOneField(Object field, Boolean bIndexed)
   at Microsoft.SharePoint.SPFieldIndexCollection.Add(SPField field)
   at Microsoft.SharePoint.Upgrade.RestoreWorkflowAndDatasourceLibraryPermissions.Upgrade()
   at Microsoft.SharePoint.Upgrade.SPActionSequence.Upgrade()

What causes this? Diving into the upgrade code I find that it is obviously trying to add a column to a list. But which one? The hint was in the name of one of the methods - "RestoreWorkflowAndDatasourceLibraryPermissions". Looking at this, it is checking for all task lists in each site that are setup for workflow.

I reran the code in powershell and eventually found the offending list:

foreach($web in $site.allwebs)

{

$ht = new-object system.collections.hashtable

foreach($list in $web.lists)

{

foreach($ct in $list.ContentTypes)

{

if ($ct.Id.IsCHildof([Microsoft.SHarePoint.SPBuiltInContentTYpeId]::WorkflowTask))

{

$ht.add($list.id, $list.id)

"Web:" + $web.TItle

"List: " + $list.Title

"Content Type:" + $ct.Name

} #end if

} #end ct

} #end list

foreach($id in $ht.values)

{

$l1 = $web.lists[$id]

#$g = new-object system.guid ([Microsoft.SHarePoint.SPBuiltInFieldId]::WorkflowInstanceID.tostring())

$g = new-object system.guid ("de8beacf-5505-47cd-80a6-aa44e7ffe2f4")

#$l1.FieldIndexes.delete($g);

$l1.FieldIndexes.add($list.fields[[Microsoft.SHarePoint.SPBuiltInFieldId]::WorkflowInstanceID]);

$l1.update()

#$g = new-object system.guid([Microsoft.SHarePoint.SPBuiltInFieldId]::Guid.tostring())

$g = new-object system.guid("ae069f25-3ac2-4256-b9c3-15dbc15da0e0")

#$l1.FieldIndexes.delete($g);

$l1.FieldIndexes.add($list.fields[[Microsoft.SHarePoint.SPBuiltInFieldId]::Guid]);

$l1.update()

} #end hashtable

} #end web

What was the cause? When you create a workflow, it asks for a task list. You can point the workflow at an existing task list or create a new one. It seems that not only did the users create a workflow that pointed to an existing list, but that list in fact was the task list that the workflow was using for task tracking!! This causes a loop in the upgrade code and hence the failure.

Fixing it requires deleting some of the fields that were added by the workflow.

Enjoy!

Chris

Adding "Add folder" to SharePoint 2010 document library views

cjg — Tue, 17 Jan 2012 17:08:00 GMT

A customer asked me if they could add a "Add Folder" to the document library list type in SharePoint 2010. I said, "probably", but wasn't sure exactly how one might do it. I dug in and found it! The secret lies in the vwstyles.xsl file. It is located in:

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\LAYOUTS\XSL

You will want to open the file and find the template called:

<xsl:template name="Freeform">

From there you should see some XSLT that is responsible for creating the "Add document" link. Right above the variable declaraton"<xsl:variable name="Url">", you need to add a new one:

<xsl:variable name="FolderUrl">
<xsl:value-of select="$HttpVDir"/>/<xsl:value-of select="$XmlDefinition/List/@title"/>/Forms/Upload.aspx?Type=1&IsDlg=1&List=<xsl:value-of select="$List"/>&RootFolder=<xsl:value-of select="$XmlDefinition/List/@RootFolder"/>
</xsl:variable>

Then add a new table row below the current one that defines the "Add Document" link:

        <tr>
          <td class="ms-addnew" style='padding-bottom: 3px'>
          <span style='height:10px;width:10px;position:relative;display:inline-block;overflow:hidden;' class='s4-clust'><img src='/_layouts/images/fgimg.png' alt='' style='left:-0px !important;top:-128px !important;position:absolute;' /></span>
          <xsl:text disable-output-escaping="yes" ddwrt:nbsp-preserve="yes">&nbsp;</xsl:text>
          <xsl:choose>
            <xsl:when test="List/@TemplateType = '115'">
              <a class="ms-addnew" id="{$ID}-{$WPQ}"
                 href="{$Url}"
                 onclick="BLOCKED SCRIPTNewItem2(event, "{$Url}");BLOCKED SCRIPTreturn false;"
                 target="_self">
                <xsl:value-of select="$AddNewText" />
              </a>
            </xsl:when>
            <xsl:otherwise>
              <a class="ms-addnew" id="{$ID}Folder"
                 href="{$FolderUrl}"
                 onclick="BLOCKED SCRIPTNewItem2(event, "{$FolderUrl}");BLOCKED SCRIPTreturn false;"
                 target="_self">Add folder
              </a>
            </xsl:otherwise>
          </xsl:choose>
          </td>
        </tr>

BAM! You now have a handly "Add folder" link at the bottom of the page instead of using the ribbon. NOTE: This change would need to be applied to all WFEs in the farm and you would need to re-check after every CU or Service Pack deployed. Also, you will need to add some other logic to keep it from showing up in places you don't want it...I'll have to add this in a later update to this blog.

NOTE: Also keep in mind that this file is cached on its first load and an IISRESET is required in order to see any changes.

Enjoy!
Chris

CanUpgrade [Microsoft.SharePoint.Administration.SPIisWebSite] failed.

cjg — Mon, 09 Jan 2012 17:40:00 GMT

How do I always seem to be the guy finding this stuff is beyond me, but I'm always excited when I figure this stuff out...ok, was getting this error after trying to run the PSConfig after an upgrade to SP1 and December 2011 CU.

[OWSTIMER] [SPUpgradeSession] [INFO] [1/8/2012 1:48:44 PM]: SPWebApplication Name=SharePoint - xxx
[OWSTIMER] [SPUpgradeSession] [ERROR] [1/8/2012 1:48:44 PM]: CanUpgrade [Microsoft.SharePoint.Administration.SPIisWebSite] failed.
[OWSTIMER] [SPUpgradeSession] [INFO] [1/8/2012 1:48:44 PM]: SPWebApplication Name=SharePoint - xxx
[OWSTIMER] [SPUpgradeSession] [ERROR] [1/8/2012 1:48:44 PM]: Exception: The system cannot find the path specified.
[OWSTIMER] [SPUpgradeSession] [INFO] [1/8/2012 1:48:44 PM]: SPWebApplication Name=SharePoint - xxx
[OWSTIMER] [SPUpgradeSession] [ERROR] [1/8/2012 1:48:44 PM]:    at System.DirectoryServices.DirectoryEntry.Bind(Boolean throwIfFail)
   at System.DirectoryServices.DirectoryEntry.Bind()
   at System.DirectoryServices.DirectoryEntry.get_AdsObject()
   at System.DirectoryServices.PropertyValueCollection.PopulateList()
   at System.DirectoryServices.PropertyValueCollection..ctor(DirectoryEntry entry, String propertyName)
   at System.DirectoryServices.PropertyCollection.get_Item(String propertyName)
   at Microsoft.SharePoint.Administration.SPIisVirtualDirectory.get_Path()
   at Microsoft.SharePoint.Administration.SPProvisioningAssistant.GetBuildVersionOnIisWebSite(Int32 iisInstanceId)
   at Microsoft.SharePoint.Upgrade.SPSequence.get_CanUpgrade()
   at Microsoft.SharePoint.Upgrade.SPUpgradeSession.CanUpgrade(Object o)

So what is causing this? Using the handy reflector tool I went into the upgrade code and found the sequence that upgrades SPWebApplication objects. This creates a list of all children objects that must be upgraded and then upgrades those objects one by one. One of the children objects is the created or extended IIS web sites. In order to check if these "CanUpgrade", it must look at the file in the IIS virtual directory "_vti_pvt\buildversion.cnf". If it can't find this file (the virtual path is deleted or missing), then the upgrade will stop for that web application and your upgrade is stuck in a weird state with this crappy error "Product / patch installation or server upgrade required"!

How do you fix? You can try to add the directory back, but how do you know what the directory is that you need to add? Funny thing, you can't find it unless you open up the SharePoint_Config database. I will spare you the details and jump to the "supported" way of fixing this.  In Central administration you won't see the offending missing zone to delete it, therefore you must resort to powershell:

$webapp = get-spwebapplication "http://yourfailedurl"

foreach($setting in $webapp.iissettings.values)
{
$setting.path
}

foreach($zone in $webapp.iissettings.keys)
{
$z = $zone
}

$webapp.IisSettings.Remove("missingzone")
$webapp.update()

Rerun your PSconfig command, viola, you are back in business!!!

Enjoy!
Chris

Managed Metadata and SharePoint Search - Error from SharePoint site: *** Index was outside the bounds of the array.

cjg — Tue, 27 Dec 2011 19:57:00 GMT

There are a lot of posts out there saying that this is caused by a space in the name of the column. I'm sure that is one issue, but their is a deeper one than this. If you are like me and you want to tag lots of content in your site using various tools or your own, most of them do not know that they need to update the hidden note field to have a specific value too.

If you create a TaxonomyField and then set its value:

-1#TermName|TermGuid

This will add the value to the local HiddenTaxonomyList properly and you will get resolution when you look at the value in the editing of the item, but you missed a step! Simply doing the first step will only set the value to "|TermGuid" for the note field. SP Search wants to look up the default value via the TermName, not the Guid. Major fail if you ask me (it should be able to do both).

You must also set the hidden note field (its name will be "FieldName_0") to a value:

TermName|TermGuid

After doing this, FAST\Sharepoint will index your column and you are off an running with awesome term searching in your search center!

Some example code in PowerShell:

function CreateField($list, $name, $fieldname)
{
#setup the MMS
$site = get-spsite $global:site
$session = get-sptaxonomysession -site $site
$termstore = $session.termstores["Managed Metadata Service"]
$termGroup = $termstore.groups["GroupName"]
$termSet = $termGroup.TermSets[$name]

$field = $list.Fields.CreateNewField("TaxonomyFieldType",$fieldname);
$field.sspid = $termstore.id
$field.termsetid = $termSet.id

$list.fields.add($field)
$list.update()
}

function UpdateMMSNote($item, $name)
{
$text = $item[$name].Label + "|" + $item[$name].TermGuid
$name = $name + "_0"
$item[$name] = $text
}

function SetField($item, $termsetname, $fieldname, $value)
{

#setup the MMS
$site = get-spsite $global:site
$session = get-sptaxonomysession -site $site
$termstore = $session.termstores["Managed Metadata Service"]
$termGroup = $termstore.groups["GroupName"]
$termSet = $termGroup.TermSets[$termsetname]

$term = $termset.terms[$value];
$item[$fieldname] = "-1;#" + $term.labels[0].value + "|" + $term.id
$item.update()
}

CreateField $list "Technology" "Technology"
SetField $item "Technology" "Technology" "SharePoint"
UpdateMMSNote $item "Technology"

Enjoy!
Chris

Fixing GroupBy List Views in SharePoint 2010 with SharePoint Designer

cjg — Mon, 26 Dec 2011 21:02:00 GMT

You may run into a problem with getting your list views to work in SharePoint 2010 if you keep your old 2007 look and feel. Their are several articles out there that show you how to fix it programmatically, but not as an end user.

The problem is with list views that have a "Group by" added and the default XML in the CAML query is set to "TRUE" for the Collapsed attribute. Here are the steps to fix this:

You can open the view page you created for the list in SharePoint designer
Do a search for "GroupBy" in the page ASP.NET code.
Set the Collapse property to "FALSE"
Add the following Javascript to the page to "Click" the links to collapse the elements:

<script language="javascript" type="text/javascript">

ExpLinkFormer = function()
{
this._init();
}

ExpLinkFormer.prototype = {
_init: function() {

var links = document.links;

for (var i = 0; i < links.length; i++) {
if (links.href == "BLOCKED SCRIPT" && links.onclick.toString().indexOf("ExpCollGroup") > -1) {
links.click();
}
}
}
}

var expLnk = new ExpLinkFormer();

</script>

Save the page - problem solved!
Repeat for all the views that have the problem

Enjoy!

Chris

SQL Azure supported tables/views

cjg — Mon, 26 Dec 2011 20:52:00 GMT

Here is a complete list of supported tables\views in SQL Azure (11.0.1814.30):

sys.all_columns;
sys.all_objects;
sys.all_parameters;
sys.all_sql_modules;
sys.all_views;
sys.assemblies;
sys.assembly_types;
sys.CHECK_CONSTRAINTS;
sys.column_type_usages;
sys.columns;
sys.COLUMNS;
sys.computed_columns;
sys.database_permissions;
sys.database_role_members;
sys.databases;
sys.default_constraints;
sys.dm_exec_requests;
sys.dm_exec_sessions;
sys.dm_tran_active_transactions;
sys.dm_tran_database_transactions;
sys.dm_tran_locks;
sys.dm_tran_session_transactions;
sys.event_notification_event_types;
sys.event_notifications;
sys.events;
sys.foreign_key_columns;
sys.foreign_keys;
sys.identity_columns;
sys.index_columns;
sys.indexes;
sys.key_constraints;
sys.numbered_procedure_parameters;
sys.numbered_procedures;
sys.objects;
sys.parameters;
sys.plan_guides;
sys.procedures;
sys.schemas;
sys.spatial_index_tessellations;
sys.spatial_indexes;
sys.spatial_reference_systems;
sys.sql_expression_dependencies;
sys.sql_logins;
sys.sql_modules;
sys.stats;
sys.stats_columns;
sys.synonyms;
sys.syscharsets;
sys.syscolumns;
sys.syscursorrefs;
sys.syscursors;
sys.sysdatabases;
sys.syslanguages;
sys.sysobjects;
sys.sysreferences;
sys.system_columns;
sys.system_objects;
sys.system_parameters;
sys.system_sql_modules;
sys.system_views;
sys.systypes;
sys.sysusers;
sys.table_types;
sys.tables;
sys.trigger_event_types;
sys.trigger_events;
sys.triggers;
sys.types;
sys.views;

Unsupported tables:

sys.allocation_units;
sys.assembly_modules;
sys.assembly_references;
sys.asymmetric_keys;
sys.backup_devices;
sys.certificates;
sys.change_tracking_databases;
sys.change_tracking_tables;
sys.COLUMN_DOMAIN_USAGE;
sys.COLUMN_PRIVILEGES;
sys.column_xml_schema_collection_usages;
sys.configurations;
sys.CONSTRAINT_COLUMN_USAGE;
sys.CONSTRAINT_TABLE_USAGE;
sys.conversation_endpoints;
sys.conversation_groups;
sys.conversation_priorities;
sys.credentials;
sys.crypt_properties;
sys.cryptographic_providers;
sys.data_spaces;
sys.database_audit_specification_details;
sys.database_audit_specifications;
sys.database_files;
sys.database_mirroring;
sys.database_mirroring_endpoints;
sys.database_mirroring_witnesses;
sys.database_principal_aliases;
sys.database_principals;
sys.database_recovery_status;
sys.destination_data_spaces;
sys.dm_audit_actions;
sys.dm_audit_class_type_map;
sys.dm_broker_activated_tasks;
sys.dm_broker_connections;
sys.dm_broker_forwarded_messages;
sys.dm_broker_queue_monitors;
sys.dm_cdc_errors;
sys.dm_cdc_log_scan_sessions;
sys.dm_clr_appdomains;
sys.dm_clr_loaded_assemblies;
sys.dm_clr_properties;
sys.dm_clr_tasks;
sys.dm_cryptographic_provider_properties;
sys.dm_database_encryption_keys;
sys.dm_db_file_space_usage;
sys.dm_db_index_usage_stats;
sys.dm_db_mirroring_auto_page_repair;
sys.dm_db_mirroring_connections;
sys.dm_db_mirroring_past_actions;
sys.dm_db_missing_index_details;
sys.dm_db_missing_index_group_stats;
sys.dm_db_missing_index_groups;
sys.dm_db_partition_stats;
sys.dm_db_persisted_sku_features;
sys.dm_db_script_level;
sys.dm_db_session_space_usage;
sys.dm_db_task_space_usage;
sys.dm_exec_background_job_queue;
sys.dm_exec_background_job_queue_stats;
sys.dm_exec_cached_plans;
sys.dm_exec_connections;
sys.dm_exec_procedure_stats;
sys.dm_exec_query_memory_grants;
sys.dm_exec_query_optimizer_info;
sys.dm_exec_query_resource_semaphores;
sys.dm_exec_query_stats;
sys.dm_exec_query_transformation_stats;
sys.dm_exec_trigger_stats;
sys.dm_filestream_file_io_handles;
sys.dm_filestream_file_io_requests;
sys.dm_fts_active_catalogs;
sys.dm_fts_fdhosts;
sys.dm_fts_index_population;
sys.dm_fts_memory_buffers;
sys.dm_fts_memory_pools;
sys.dm_fts_outstanding_batches;
sys.dm_fts_population_ranges;
sys.dm_io_backup_tapes;
sys.dm_io_cluster_shared_drives;
sys.dm_io_pending_io_requests;
sys.dm_os_buffer_descriptors;
sys.dm_os_child_instances;
sys.dm_os_cluster_nodes;
sys.dm_os_dispatcher_pools;
sys.dm_os_dispatchers;
sys.dm_os_hosts;
sys.dm_os_latch_stats;
sys.dm_os_loaded_modules;
sys.dm_os_memory_allocations;
sys.dm_os_memory_brokers;
sys.dm_os_memory_cache_clock_hands;
sys.dm_os_memory_cache_counters;
sys.dm_os_memory_cache_entries;
sys.dm_os_memory_cache_hash_tables;
sys.dm_os_memory_clerks;
sys.dm_os_memory_node_access_stats;
sys.dm_os_memory_nodes;
sys.dm_os_memory_objects;
sys.dm_os_memory_pools;
sys.dm_os_nodes;
sys.dm_os_performance_counters;
sys.dm_os_process_memory;
sys.dm_os_ring_buffers;
sys.dm_os_schedulers;
sys.dm_os_spinlock_stats;
sys.dm_os_stacks;
sys.dm_os_sublatches;
sys.dm_os_sys_info;
sys.dm_os_sys_memory;
sys.dm_os_tasks;
sys.dm_os_threads;
sys.dm_os_virtual_address_dump;
sys.dm_os_wait_stats;
sys.dm_os_waiting_tasks;
sys.dm_os_worker_local_storage;
sys.dm_os_workers;
sys.dm_qn_subscriptions;
sys.dm_repl_articles;
sys.dm_repl_schemas;
sys.dm_repl_tranhash;
sys.dm_repl_traninfo;
sys.dm_resource_governor_configuration;
sys.dm_resource_governor_resource_pools;
sys.dm_resource_governor_workload_groups;
sys.dm_server_audit_status;
sys.dm_tran_active_snapshot_database_transactions;
sys.dm_tran_commit_table;
sys.dm_tran_current_snapshot;
sys.dm_tran_current_transaction;
sys.dm_tran_top_version_generators;
sys.dm_tran_transactions_snapshot;
sys.dm_tran_version_store;
sys.dm_xe_map_values;
sys.dm_xe_object_columns;
sys.dm_xe_objects;
sys.dm_xe_packages;
sys.dm_xe_session_event_actions;
sys.dm_xe_session_events;
sys.dm_xe_session_object_columns;
sys.dm_xe_session_targets;
sys.dm_xe_sessions;
sys.DOMAIN_CONSTRAINTS;
sys.DOMAINS;
sys.endpoint_webmethods;
sys.endpoints;
sys.extended_procedures;
sys.extended_properties;
sys.filegroups;
sys.fulltext_catalogs;
sys.fulltext_document_types;
sys.fulltext_index_catalog_usages;
sys.fulltext_index_columns;
sys.fulltext_index_fragments;
sys.fulltext_indexes;
sys.fulltext_languages;
sys.fulltext_stoplists;
sys.fulltext_stopwords;
sys.fulltext_system_stopwords;
sys.function_order_columns;
sys.http_endpoints;
sys.internal_tables;
sys.KEY_COLUMN_USAGE;
sys.key_encryptions;
sys.linked_logins;
sys.login_token;
sys.master_files;
sys.master_key_passwords;
sys.message_type_xml_schema_collection_usages;
sys.messages;
sys.module_assembly_usages;
sys.openkeys;
sys.parameter_type_usages;
sys.parameter_xml_schema_collection_usages;
sys.partition_functions;
sys.partition_parameters;
sys.partition_range_values;
sys.partition_schemes;
sys.partitions;
sys.REFERENTIAL_CONSTRAINTS;
sys.remote_logins;
sys.remote_service_bindings;
sys.resource_governor_configuration;
sys.resource_governor_resource_pools;
sys.resource_governor_workload_groups;
sys.routes;
sys.ROUTINE_COLUMNS;
sys.ROUTINES;
sys.SCHEMATA;
sys.securable_classes;
sys.server_assembly_modules;
sys.server_audit_specification_details;
sys.server_audit_specifications;
sys.server_audits;
sys.server_event_notifications;
sys.server_event_session_actions;
sys.server_event_session_events;
sys.server_event_session_fields;
sys.server_event_session_targets;
sys.server_event_sessions;
sys.server_events;
sys.server_file_audits;
sys.server_permissions;
sys.server_principal_credentials;
sys.server_principals;
sys.server_role_members;
sys.server_sql_modules;
sys.server_trigger_events;
sys.server_triggers;
sys.servers;
sys.service_broker_endpoints;
sys.service_contract_message_usages;
sys.service_contract_usages;
sys.service_contracts;
sys.service_message_types;
sys.service_queue_usages;
sys.service_queues;
sys.services;
sys.soap_endpoints;
sys.sql_dependencies;
sys.symmetric_keys;
sys.sysaltfiles;
sys.syscacheobjects;
sys.syscomments;
sys.sysconfigures;
sys.sysconstraints;
sys.syscurconfigs;
sys.syscursorcolumns;
sys.syscursortables;
sys.sysdepends;
sys.sysdevices;
sys.sysfilegroups;
sys.sysfiles;
sys.sysforeignkeys;
sys.sysfulltextcatalogs;
sys.sysindexes;
sys.sysindexkeys;
sys.syslockinfo;
sys.syslogins;
sys.sysmembers;
sys.sysmessages;
sys.sysoledbusers;
sys.sysopentapes;
sys.sysperfinfo;
sys.syspermissions;
sys.sysprocesses;
sys.sysprotects;
sys.sysremotelogins;
sys.sysservers;
sys.system_components_surface_area_configuration;
sys.system_internals_allocation_units;
sys.system_internals_partition_columns;
sys.system_internals_partitions;
sys.TABLE_CONSTRAINTS;
sys.TABLE_PRIVILEGES;
sys.tcp_endpoints;
sys.trace_categories;
sys.trace_columns;
sys.trace_event_bindings;
sys.trace_events;
sys.trace_subclass_values;
sys.traces;
sys.transmission_queue;
sys.type_assembly_usages;
sys.user_token;
sys.via_endpoints;
sys.VIEW_COLUMN_USAGE;
sys.VIEW_TABLE_USAGE;
sys.xml_indexes;
sys.xml_schema_attributes;
sys.xml_schema_collections;
sys.xml_schema_component_placements;
sys.xml_schema_components;
sys.xml_schema_elements;
sys.xml_schema_facets;
sys.xml_schema_model_groups;
sys.xml_schema_namespaces;
sys.xml_schema_types;
sys.xml_schema_wildcard_namespaces;
sys.xml_schema_wildcards;

Top SharePoint Conference 2011 Sessions and Stats

cjg — Wed, 26 Oct 2011 20:07:00 GMT

They didn't say I couldn't post this data, so here it is. If you wanted to know what sessions you should review\watch, this should help you out. Also, if you are thinking about being a speaker, these are the type of eval scores you should expect to get:

Interesting Facts:

- Over 11,000 evals submitted,

- Core Conference Average: 4.28 (Partner, Customer, Analyst tracks excluded) (up from 4.24 in 2009!),

- Customer track avg. 4.1,

- Partner track avg. 3.76,

- Analyst track avg. 3.79

- Keynote avg. 3.83 (only 415 evals)

(the following stats have sessions with <10 evals removed)

Core Track Ranking:

Track Name	Evals	Average
IT Professional	2303	4.41
Search	931	4.29
Communities/Sites	646	4.27
Insights	1217	4.25
Planning, Deployment and Adoption	500	4.25
Internet Business	1064	4.25
Developer	2190	4.25
Content	760	4.19
Office Client	310	4.17
Project	188	4.13
Grand Total	10109	4.28

Top 20 Sessions Overall

Session	Evals	Average
SPC373: Performance Tuning SharePoint 2010 (Eric Shupps)	188	4.84
SPC402: Deep Dive on SharePoint Ribbon Development & Extensibility (Chris O'Brien)	74	4.82
SPC393: Step-by-step: Building Search Driven Applications That Matter (Scot Hillier)	72	4.82
SPC315: Beyond the Basics: An Advanced Conversation on FAST Search for SharePoint 2010 (Thomas Molbach)	58	4.79
SPC270: SharePoint 2010 Solutions for Public Sector (Ken Mallit)	27	4.78
SPC224: Architecting and Automating SharePoint Governance (Dan Holme)	142	4.77
SPC392: Solving Enterprise Search Challenges with SharePoint 2010 (Matthew McDermott)	41	4.76
SPC404: Deep Dive: Implementing Kerberos for your BI Applications (Tom Wisnowski)	34	4.74
SPC407: Enterprise Deployment Considerations for the User Profile Service Application (Spencer Harbar)	80	4.74
SPC370: Multi-Tenancy with SharePoint 2010 (Spencer Harbar)	40	4.74
SPC202: Attractive Business Intelligence: Dashboards, Pivots, Scorecards, KPIs, and Reports Using Microsoft SharePoint 2010, Office 2010, PowerPivot, and SQL Server 2008 R2 (Rafal Lukawiecki)	192	4.72
SPC385: Service Application Federation with SharePoint 2010 (Shannon Bray)	15	4.72
SPC245: Making SharePoint 2010 Collaboration Rock by Increasing Findability (Scott Jamison)	73	4.70
SPC276: Developing Cloud-Based Applications for SharePoint Online using Windows Azure (Steve Fox)	50	4.70
SPC319: Application Lifecycle Management: Automated builds and testing for SharePoint projects (Chris O'Brien)	98	4.70
SPC312: Best Practices for Creating Publishing Page Layouts (Geoffrey Edge)	35	4.69
SPC338: Developing SharePoint applications with HTML5 and JQuery (Ted Pattison)	207	4.68
SPC212: More Than My: How Microsoft is Driving Social Adoption and Intranet Transformation (Chris Slemp)	35	4.67
SPC304: Advanced Scorecarding and Dashboards with Excel, Visio and PerformancePoint Services (Peter Myers)	71	4.66
SPC331: Creating a FAST Search Driven Windows Phone 7 Application for a SharePoint Internet Sites (Shad Phillips)	17	4.66

Overall Conference Top Speakers

MS Speakers

1^st SPC315: Beyond the Basics: An Advanced Conversation on FAST Search for SharePoint 2010 (Thomas Molbach & Thomas Svensen)

2^ndSPC270: SharePoint 2010 Solutions for Public Sector (Ken Mallit)

3^rd SPC404: Deep Dive: Implementing Kerberos for your BI Applications (Tom Wisnowski)

Non MS Speakers

1^st SPC373: Performance Tuning SharePoint 2010 (Eric Shupps)

2^nd SPC402: Deep Dive on SharePoint Ribbon Development & Extensibility (Chris O'Brien & Andrew Connell)

3^rd SPC393: Step-by-step: Building Search Driven Applications That Matter (Scot Hillier)

SharePoint Conference PPTs - Download them via PowerShell!

cjg — Mon, 10 Oct 2011 22:48:00 GMT

Here's the script to download the PPTs via PowerShell - the link to the file is below in case of copy paste errors!

function DownloadFile([string]$auth, [string] $file)
{
"Using " + $auth + " for file " + $file

$url = "http://www.mssharepointconference.com/sessionpresentations/" + $file
$file = "c:\SPC11\" + $file

$uri = new-object uri($url);
$httpReq = [system.net.HttpWebRequest]::Create($uri)
$httpReq.headers.add("Cookie", $auth)
$httpReq.Timeout = 75000
$httpReq.method = "GET"

$res = $httpReq.GetResponse()
$rs = $res.GetResponseStream();

$fs = new-object system.io.filestream($file, [System.IO.FileMode]::OpenOrCreate)
$read = new-object byte[] 1024

$count = $rs.Read($read, 0, $read.length)

while($count -ne 0)
{
$fs.write($read,0,$count)
$count = $rs.Read($read, 0, $read.length)
}

$fs.close()
$rs.Close()
$res.close()

}

function GetAuth([string] $username, [string]$password)
{
[system.net.servicepointmanager]::Expect100Continue = $false

#####################################
# Start session on SPConf site
####################################

$uri = new-object uri("http://www.mssharepointconference.com/Pages/Default.aspx");
$httpReq = [system.net.HttpWebRequest]::Create($uri)
$httpReq.Accept = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, */*"
$httpReq.ContentType = "application/x-www-form-urlencoded"
$httpReq.headers.Add("Accept-Language", "en-US")
$httpReq.method = "GET"

$res = $httpReq.GetResponse()
$rs = $res.GetResponseStream();
[System.IO.StreamReader]$sr = New-Object System.IO.StreamReader -argumentList $rs;
[string]$results = $sr.ReadToEnd();

#get the cookies
$strCookies = $res.Headers["set-cookie"].toString();
$res.Close();

$vnlehn = $strcookies

####################################
# Do init post to live.com
####################################

$cookies = new-object system.net.CookieContainer
$uri = new-object uri("https://login.live.com/wlogin.srf?appid=000000004C03DD21&alg=wsignin1.0&appctx=ConferenceVue");
$httpReq = [system.net.HttpWebRequest]::Create($uri)
$httpReq.Accept = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, */*"
$httpReq.Referer = "http://www.mssharepointconference.com/Pages/Login.aspx?ReturnUrl=%2f_layouts%2fauthenticate.aspx%3fSource%3d%252FPages%252FDefault%252Easpx&Source=%2FPages%2FDefault%2Easpx"
$httpReq.ContentType = "application/x-www-form-urlencoded"
$httpReq.method = "GET"

$res = $httpReq.GetResponse()
$rs = $res.GetResponseStream();
[System.IO.StreamReader]$sr = New-Object System.IO.StreamReader -argumentList $rs;
[string]$results = $sr.ReadToEnd();

#get the cookies
$strCookies = $res.Headers["set-cookie"].toString();
$res.Close();

$mspr = $strCookies.substring(0, $strCookies.indexof(";")+1)
$mspok = $strCookies.substring($strCookies.indexof("MSPOK"))
$mspok = $mspok.substring(0, $mspok.indexof(";"))

$strCookies = $mspr + $mspok

#get random html values
$ppft = $results.remove(0, $results.indexof("PPFT"))
$ppft = $ppft.remove(0, $ppft.indexof("value") + 7)
$ppft = $ppft.substring(0, $ppft.indexof("`""))

$posturl = $results.remove(0, $results.indexof("g_QS=")+6)
$posturl = $posturl.substring(0, $posturl.indexof("`""))

####################################
# Do the login
####################################

$uri = new-object uri("https://login.live.com/ppsecure/post.srf?" + $posturl)
$post = "login=" + $username + "&passwd=" + $password +"&type=11&LoginOptions=3&NewUser=1&MEST=&PPSX=Passport&PPFT=" + $ppft + "&idsbho=1&PwdPad=&sso=&i1=&i2=1&i3=4778&i4=&i12=1&i13=&i14=333&i15=334"
$encoding = new-object system.text.asciiencoding
$buf = $encoding.GetBytes($post)

$httpReq = [system.net.HttpWebRequest]::Create($uri)
$httpReq.method = "POST"
$httpReq.Referer = "https://login.live.com/wlogin.srf?appid=000000004C03DD21&alg=wsignin1.0&appctx=ConferenceVue"
$httpReq.contentlength = $buf.length
$httpReq.Headers.add("Cookie", $strCookies)
$httpReq.ContentType = "application/x-www-form-urlencoded"
$stream = $httpReq.GetRequestStream()

[void]$stream.write($buf, 0, $buf.length)
$stream.close()

$res = $httpReq.GetResponse()
$rs = $res.GetResponseStream();
[System.IO.StreamReader]$sr = New-Object System.IO.StreamReader -argumentList $rs;
[string]$results = $sr.ReadToEnd();

$stoken = $results.remove(0, $results.indexof("id=`"stoken")+19)
$stoken = $stoken.substring(0, $stoken.indexof("`""))

####################################
# Return token back to spconf site
####################################

$uri = new-object uri("http://www.mssharepointconference.com/_layouts/ConferenceVue/WindowsLive.aspx")
$post = "stoken=" + $stoken + "&appctx=ConferenceVue&action=login"

$encoding = new-object system.text.asciiencoding
$buf = $encoding.GetBytes($post)

$httpReq = [system.net.HttpWebRequest]::Create($uri)
$httpReq.method = "POST"
$httpReq.contentlength = $buf.length
$httpReq.ContentType = "application/x-www-form-urlencoded"
$httpReq.Accept = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, */*"
$httpReq.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET4.0C; .NET4.0E; MS-RTC LM 8; LEN2)"
$httpReq.headers.Add("Cookie", $vnlehn + "; webauthtoken=")
$httpReq.headers.Add("Accept-Language", "en-US")
$httpReq.KeepAlive = $true
$httpReq.AllowAutoRedirect = $false
$stream = $httpReq.GetRequestStream()

[void]$stream.write($buf, 0, $buf.length)
$stream.close()

$res = $httpReq.GetResponse()
$rs = $res.GetResponseStream();
[System.IO.StreamReader]$sr = New-Object System.IO.StreamReader -argumentList $rs;
[string]$results = $sr.ReadToEnd();

#get the cookie
$strCookies = $res.Headers["Set-Cookie"].toString();

#get the webauthtoken
if ($strcookies.contains("webauthtoken"))
{
$webauthtoken = $strCookies.substring($strCookies.indexof("webauthtoken"))
$webauthtoken = $webauthtoken.substring(0, $webauthtoken.indexof(";"))
}

#get the fedauth
if ($strcookies.contains("FedAuth"))
{
$fedauth = $strCookies.substring($strCookies.indexof("FedAuth"))
$fedauth = $fedauth.substring(0, $fedauth.indexof(";"))
}

return $vnlehn + ";" + $fedauth + ";" + $webauthtoken + ";"
}

######################################
# Parse the HTML of the SPConf site
#####################################

#$results = get-content AllHtml.txt

#parse out the pptx links
#while($results.contains("SessionPresentations"))
#{
#$results = $results.remove(0, $results.indexof("SessionPresentations/")+21)
#$filename = $results.substring(0, $results.indexof("pptx") + 4)
#$results = $results.remove(0, $results.indexof("pptx")+4)

#$filename

#add-content "c:\temp\ppts.txt" $filename
#}

##############################
# Start program
###############################

[System.Reflection.Assembly]::LoadWithPartialName( "System.Web" ) | out-null
[System.Reflection.Assembly]::LoadWithPartialName( "System.Net" ) | out-null

$username = read-host "Enter your LiveID username: "
$password = read-host "Enter your LiveID password: "

$auth = GetAuth $username $password

while(-not $auth.tostring().contains("webauthtoken"))
{
$auth = GetAuth $username $password
}

############################
# Download all the PPTs
###############################

#use for testing in case you NEVER get the webauthtoken back...
#$auth = ""

$ppts = get-content "c:\ppts.txt"
mkdir "c:\spc11"

foreach($line in $ppts)
{
DownloadFile $auth $line

}

Here's all the PPTX files on the site (Place in a files called c:\ppts.txt):

SPC216.pptx
SPC340.pptx
SPC102.pptx
SPC2991.pptx
SPC214.pptx
SPC240.pptx
SPC367.pptx
SPC2994.pptx
SPC249.pptx
SPC263.pptx
SPC253.pptx
SPC256.pptx
SPC274.pptx
SPC285.pptx
SPC202.pptx
SPC225.pptx
SPC345.pptx
SPC228.pptx
SPC350.pptx
SPC104.pptx
SPC229_Mercaldo.pptx
SPC387.pptx
SPC2992.pptx
SPC410.pptx
SPC376.pptx
SPC258.pptx
SPC265.pptx
SPC273.pptx
SPC201.pptx
SPC311.pptx
SPC403.pptx
SPC335.pptx
SPC406.pptx
SPC226.pptx
SPC358.pptx
SPC359.pptx
SPC245.pptx
SPC246.pptx
SPC262.pptx
SPC221.pptx
SPC200.pptx
SPC330.pptx
SPC346.pptx
SPC347.pptx
SPC351.pptx
SPC239.pptx
SPC242.pptx
SPC371.pptx
SPC252.pptx
SPC109.pptx
SPC254.pptx
SPC255.pptx
SPC382.pptx
SPC266.pptx
SPC271.pptx
SPC3995.pptx
SPC224.pptx
SPC310.pptx
SPC223.pptx
SPC298.pptx
SPC244.pptx
SPC366.pptx
SPC212.pptx
SPC264.pptx
SPC108_Barnard.pptx
SPC391.pptx
SPC280.pptx
SPC3992.pptx
SPC269.pptx
SPC289.pptx
SPC292.pptx
SPC215.pptx
SPC219.pptx
SPC338.pptx
SPC343.pptx
SPC234.pptx
SPC352.pptx
SPC103.pptx
SPC357.pptx
SPC248.pptx
SPC398.pptx
SPC288.pptx
SPC251.pptx
SPC306.pptx
SPC307.pptx
SPC204.pptx
SPC322.pptx
SPC206.pptx
SPC207.pptx
SPC2993.pptx
SPC348.pptx
SPC409_Kogan.pptx
SPC243.pptx
SPC368.pptx
SPC375.pptx
SPC383.pptx
SPC388.pptx
SPC275-1.pptx
SPC281.pptx
SPC3996.pptx
SPC293.pptx
SPC314.pptx
SPC213.pptx
SPC331.pptx
SPC401.pptx
SPC220.pptx
SPC222.pptx
SPC336.pptx
SPC3982.pptx
SPC397.pptx
SPC260.pptx
SPC261.pptx
SPC279.pptx
SPC291.pptx
SPC400_1.pptx
SPC203.pptx
SPC315_1.pptx
SPC342.pptx
SPC257.pptx
SPC259.pptx
SPC299.pptx
SPC365_1.pptx
SPC373_1.pptx
SPC377.pptx
SPC272.pptx
SPC389_1.pptx
SPC250.pptx
SPC278.pptx
SPC290.pptx
SPC304_A.pptx
SPC319_A.pptx
SPC287.pptx
SPC321_1.pptx
SPC296.pptx
SPC356_1.pptx
SPC374_1.pptx
SPC386_1.pptx
SPC270.pptx
SPC392_f.pptx
SPC414.pptx
SPC3999.pptx
SPC309_1.pptx
SPC328.pptx
SPC333_1.pptx
SPC339_1.pptx
SPC341.pptx
SPC407.pptx
SPC349_1.pptx
SPC237.pptx
SPC355_1.pptx
SPC360_1.pptx
SPC354_1.pptx
SPC379_1.pptx
SPC101.pptx
SPC396.pptx
SPC282.pptx
SPC410r.pptx
SPC308.pptx
SPC313_1.pptx
SPC205.pptx
SPC318.pptx
SPC210.pptx
SPC230.pptx
SPC233.pptx
SPC295.pptx
SPC236.pptx
SPC268.pptx
SPC369_1.pptx
SPC380_1.pptx
SPC384_1.pptx
SPC412_1.pptx
SPC266r.pptx
SPC241.ppt
SPC326.pptx
SPC329_1.pptx
SPC217.pptx
SPC218.pptx
SPC402_1.pptx
SPC404_1.pptx
SPC337_1.pptx
SPC408_1.pptx
SPC232_1.pptx
SPC378_1.pptx
SPC385_1.pptx
SPC300.pptx
SPC302.pptx
SPC344.pptx
SPC320.pptx
SPC362.pptx
SPC324.pptx
SPC327.pptx
SPC332.pptx
SPC334.pptx
SPC227.pptx
SPC277.pptx
SPC283.pptx
SPC413.pptx
SPC294.pptx
SPC3983.pptx
SPC267.pptx
SPC247.pptx
SPC361.pptx
SPC364.pptx
SPC393.pptx
SPC394.pptx
SPC395.pptx
SPC3993.pptx
SPC297.pptx
SPC303.pptx
SPC353.pptx
SPC2995.pptx
SPC372.pptx
SPC323.pptx
SPC411.pptx
SPC399.pptx
SPC3994.pptx

SPC221.pptx
SPC235.pptx
SPC276.pptx
SPC286.pptx
SPC312_1.pptx
SPC316_1.pptx
SPC363.pptx
SPC370.pptx
SPC381_a.pptx
SPC3991_1.pptx
SPC3997_Maio.pptx

Enjoy!
Chris

Case Study: How to Use Managed Metadata in Site Creation to drive Searchability

cjg — Mon, 10 Oct 2011 17:12:00 GMT

As part of our eBay presentation at the SharePoint Conference 2011, we showed how we performed the upgrade in 3 months and utilized Managed Metadata (MMS) in site creation to drive searchability across the SharePoint Farm. The biggest question I have received so far from the presentation is about the MMS subject. So I'm going to outline it in detail here. For those of you that are not familiar with MMS, I like to describe it using the 3rd rule in my three rules of development. The first rule is "Don't code the same line of code twice", if you see the same line of code twice, you did something wrong (no polymorphism, missing base method calls, inheritance, etc). The second is don't put the same thing in memory more than once (this really needs no clarification). The third rule says don't put the same thing on disk twice. By doing this, you now must implement synchronization between the multiple instances. The only product I worked with over the years that ever implemented this pattern perfectly was Lotus Notes. It is for this reason, we have MMS in 2010.

In SharePoint 2007, imagine you have created a list in SharePoint that uses a choice column. You would have 50 states, plus the other various items the US owns (Examples: Guam, Virgin Islands...can you name them all?). If you use the state column in other places in the Farm, you pretty much have to copy and paste all the items over and over again. This is somewhat of a problem. For instance, if Texas ever successfully succeeds from the union, you gotta remove them from all the choice columns! What about Iraq and Afghanistan? Maybe those should be added to the list? Who knows what might happen in the future, but you'd have to update each and every source in order for the data to be meaningful. Therefore, rule #3 is a good one...don't put data on disk more than twice. MMS helps us do this by centrally stored our list data in one place.

So how did we use MMS at eBay? The summary of the process is that when users create their own site, the content inside the site should be pre-tagged with MMS data so as to allow for simple searching of their content via FAST Search. Here's a breakout of the parts:

Site Creation:

The first step is to create a site creation form. This can be anything you are comfortable with (an aspx page, an infopath form, web parts, etc). In the case of eBay we used two web parts. One is the site creation landing page web part. Previously the values used to "tag" the sites was hardcoded in the code. As part of the upgrade I changed this to pull dynamically from MMS. This web part allows the entry of the following information:

Business Unit (MMS)
Organization (MMS)
Office Location (MMS)

Site Name
Description
Keywords (terms that will allow users to find the site later)
Template (teamsite, blog, etc)
Security (allow inheritance/everyone to view the site)

When clicking submit, several things happen:

Create a new list item with all the details of the site creation request
Submit to the site creation processing webpart with the site creation request id
Start a long running transaction
Impersonate the user that made the request
Get a site number from a database for the site URL (example: 12345)
Determine what site collection to add the site to (allows for load balancing the sites across content databases)
Create the new web
Add to the custom site directory list (this list is used for later look up on a site lookup web part)
Setup navigation for the site
Set the web property bag with the Organization, BusinessUnit and Location properties
Create the metadata columns (outlined below)
Add the site keywords to FAST Search

All throughout this process if any errors occur another list is setup to keep track of the errors and steps of the long running process. This allows OPS to debug the process and work with development in case something needs to be tweaked.

Site Directory:

You will need to ensure you have a list setup to keep track of your sites. In the case of a pre-existing site directory it is likely that you have already created a choice column with your values. This doesn't move very easily to MMS. You will have to migrate the values to new MMS columns (create new column with different name, move value over, deleted old column, create column with old name, copy values again) . If you already have this site directory, it is simple to run back through all the sites and add the web properties and then add the MMS columns to all the existing lists in the already existing sites (so not only new sites have MMS, but old ones too)!

MMS Setup:

We used the following PowerShell script to pre-populate the MMS:

$session = get-sptaxonomysession -site http://contoso.com
$termstore = $session.termstores["Contoso Corporate MMS"]

$termGroup = $termstore.groups["Contoso"]

if (-not $termGroup)
{
$termGroup = $termstore.creategroup("Contoso")
}

$termSet = $termGroup.TermSets["Organization"]

if (-not $termSet)
{
$termSet = $termGroup.CreateTermSet("Organization")
}

foreach($t in $termset.terms)
{
$t.delete()
}

$termSet.createterm("Finance",1033)
$termSet.createterm("Human Resources",1033)
$termSet.createterm("Information Technology",1033)
$termSet.createterm("Legal",1033)
$termSet.createterm("Marketing and Sales",1033)

$termSet = $termGroup.termsets["BusinessUnit"]

if (-not $termSet)
{
$termSet = $termGroup.CreateTermSet("BusinessUnit")
}

foreach($t in $termset.terms)
{
$t.delete()
}

$termSet.createterm("Corporate",1033)
$termSet.createterm("PayPal",1033)
$termSet.createterm("StubHub",1033)
$termSet.createterm("Bill Me Later",1033)

$termSet = $termGroup.termsets["Location"]

if (-not $termSet)
{
$termSet = $termGroup.CreateTermSet("Location")
}

foreach($t in $termset.terms)
{
$t.delete()
}

$termSet.createterm("APAC - Australia",1033)
$termSet.createterm("APAC - China",1033)
$termSet.createterm("APAC - Hong Kong",1033)
$termSet.createterm("APAC - India",1033)
$termSet.createterm("APAC - Israel",1033)
$termSet.createterm("APAC - Korea",1033)
$termSet.createterm("APAC - Taiwan",1033)
$termSet.createterm("EU - Austria",1033)
$termSet.createterm("EU - Belgium",1033)
$termSet.createterm("EU - Czech Replubic",1033)
$termSet.createterm("EU - Estonia",1033)
$termSet.createterm("EU - France",1033)
$termSet.createterm("EU - Germany",1033)
$termSet.createterm("EU - Ireland",1033)
$termSet.createterm("EU - Italy",1033)
$termSet.createterm("EU - Luxembourg",1033)
$termSet.createterm("EU - Netherlands",1033)
$termSet.createterm("EU - Poland",1033)
$termSet.createterm("EU - Spain",1033)
$termSet.createterm("EU - Sweden",1033)
$termSet.createterm("EU - Switzerland",1033)
$termSet.createterm("EU - UK",1033)
$termSet.createterm("NA - Canada",1033)
$termSet.createterm("NA - U.S.",1033)
$termSet.createterm("Global",1033)
$termStore.CommitAll()

Note that once it is created, you should not delete the values. This will cause the linkage between any previously created columns to be eliminated. Not fun to look at and fix later. So be sure to tell the ops team not to run the create more than once...EVER...

MMS Columns:

The hardest part of the entire process is the MMS column creation. It works like this:

Check to see if the web prop bag has been set, if not then set the web props to the values in the intranet site directory list, otherwise set to some default values
For each list, add the metadata columns

You should not add metatdata to lists that are "hidden" or that are type "Survey". It will break those list types
Great example is the HiddenTaxonomyList, kinda weird adding a MMS column to that one as it keeps track of MMS usage across the site, creates a bit of a loop...not good!

You should also avoid everything but the following site types, if you don't you will get some VERY ugly messages from SharePoint 2010 (try adding MMS to an Access Services site, it doesn't work!):

STS
MPS
BLANKINTERNET
CMSPUBLISHING
PUBLISHING
WIKI
BLOG
SPS

Adding the metadata column involves the following process:

Detemine if the column already exists (choose your column name wisely), in this case, we preappended with "Geo" to avoid any clashes with already existing column names and internal SP names
Provision a note field to support your MMS field (yup, each MMS column is actually two columns)
Create a new taxonomy field using the Guid of the note field in the TextField property
Set the default value of the field using the "WSSID;#VALUE|TERMID", hint: use -1 if WSSID has not been defined yet

The result is every viable list has MMS columns added:

Each MMS column has a default value set:

When a new item is created, the user doesn't have to do anything! The values are pre-populated with the values they selected at site creation time!

Search:

As part of the site creation there is a keyword text box. Users can type keywords delimited by semicolon. Their keywords are then added to FAST Search. Initially we thought adding to both SharePoint Search and FAST Search was the answer, but that didn't work as it actually would show two keywords in the results and not just one. You can also have the keywords added as search suggestions. What this means is that when they start typing in the search box, you will see the keyword displayed as a suggestion:

Then when the search is executed, you will see the site as a keyword and the results for that keyword displayed as business as usual:

Once the columns have been added to all the list items in all the sites, you have setup FAST Content SSA and indexed the content you will get some pretty cool results. Because of the TaxonomyFilter refiners that is preconfigured in the FAST Search Center, you will see the pre-defined MMS column values on the left side of the screen:

Users can now find content specific to their geography, organization and functional group. Results are now more meaningful and can be found much faster than with SharePoint 2007!

Enjoy!
Chris

SharePoint Conference 2011 Recap

cjg — Fri, 07 Oct 2011 21:31:00 GMT

Here is a short recap of all the cool stuff that happened this week.

Over 7,500 people showed up in Anaheim, CA on Sunday to enjoy networking and learning about SharePoint! The event started off with the exhibit hall reception where you got your first sneak peak at the vendors.

Exhibit Hall

The first big booths you would get to see was Quest and HP in the center. HP didn't have much that interested me, but the Quest folks sure made things entertaining with their "Bonk" game. You can find lots of pictures of SharePoint superstars on facebook and twitter, it was quite humorous and they gave away lots of cold hard cash to answer SharePoint questions! In the following pictures you'll see @joelolsen @michaelnoel @buckleyplanet and then @ZlatanDzinic and @eusp

To the left of HP was the the great guys from AvePoint (@danholme, @tweetraw, @jthake)! As they have done for the past few SPConferences, they had a killer Ducati motorcycle to giveaway to some lucky attendee:

AvePoint also hosted the party of all parties, The Red Party, at a local bar called Heat Ultra Lounge. Everyone was dressed in Red and the inside has all red lights, it was amazingly awesome! AvePoint announced their new product versions including a Governance tool that has been under the wing of Jeremy Thake.

On the right side of Quest were the very cool K2 guys showing off their awesome BlackPearl product and other workflow tools. They were giving away a Segway for some lucky attendee!

Behind Quest and K2 were the good ole boys from fpweb.net ("fp" by the way means "Front Page" which was from an acquisition from a company called VTI which is the name of the web services directory VTI_BIN). fpweb pretty much has the coolest booth I had ever seen. It has two stories with a full living room on the top floor. It was of course roped off to only special fpweb MVPs and customers. After a long day of sessions and walking the exhibit hall it was great to relax and drink a glass of wine on their very comfy couches.

fpweb.net had a couple of events during the week, including one that I master minded called the "Move like Jagger Contest". There were several people at the conference that brought wives and kiddos and need extra passes. It just turned out that three people gave me their passes and we decided to make people work for them by moving like Mick Jagger. Rob LaMear, CEO of fpweb, put in some extra incentive by placing $100 bills on each of the passes, that got us about 10 people to do some Mick Jagger moves, one such person is @grumpytech.

I'll be posting the video very soon! The other event was a "fpweb.net is flexbile", where several young ladies came out and did some pretty amazing moves:

As far as new comers to the SharePoint software space, I though these two vendors had a pretty rocking demo:

#1 - ESRI - a long time ruling company in geocoding and mapping software came out with a Solution Package with a web part, custom field type with wizards to pull SharePoint list data and backend data to do geo-coding and mapping with heat maps. This stuff even knows how to use the SQL Server SpatialTypes to generate the data - AMAZINGLY COOL!
#2 - SouthLabs - A very nice looking iPad and iPhone app for connecting to SharePoint with many authentication method support. They were some very cool and bright guys from Uruguay and I think they will be able to win quite a lot of business from SharePoint customers with their mobile apps.
#3 - Pingar- They have an auto tagging solution designed by several computer science artificial intelligence people which used wikipedia as an input source to determine how to auto tag content in your SharePoint Farm. It is by far the most advanced auto-tagging solution I have seen

And of course the hottest booth babe was my wifey who was hanging out at the @sharesquared booth:

Keynote:

The keynote area was HUGE! As you can imagine, fitting over 7500 people into a room is quite a task. During the keynote we had SharePoint Team member @thekameleon dj'ing.

The SharePoint trio, Jared Spararo, Jeff Teper and Kurt Delbene spoke and in-between they played some "cute" videos of celebrities, but unfortunately they didn't have anyone actually "present" for us to look at.

The real highlight for me of the keynote was when the customer video played. The very opening of the video had my customer eBay's intranet called "TheHUB". The eBay mobile app also made an appearance with my name displayed and then immediately followed by me making a cameo on screen!

I'll post the video link later so you can watch it! One of the other neat demos they did was have a SQL Server Denali instance setup in a cluster with some pretty crazy numbers around data corpus. They pulled the network cable on the active SQL Server and had the cluster failover to the passive SQL Server node. The hardware was in the range of $1.5 million and as much as it was a fun demo to see, it is not likely that many customers would be purchasing hardware of that magnitude. Especially when you need have to build out the disaster recovery component at a similar cost in a different geographically located data center:

Sessions:

There were lots and lots of sessions! One thing I noticed was that FAST Search made an appearance in each of the session sets. It was great for me as I had just finished my FAST Search course and was able to validate that everything was actually covered! One of the coolest sessions on FAST was the one held by Joshua at the International Monetary Fund (IMF). They had done some pretty amazing stuff with FAST Search Center. The other impressive FAST Search session was done by Paul Branson, he was very knowledgeable and impressed me quite a bit with his demos and movement around the FAST directories.

Our eBay upgrade session was on Wednesday at 3:15, so we were able to party the night before and not have a early morning session where everyone was hungover and sleeping from the previous night! We packed the room (with standing room only) with several Microsoft folks and saw several SharePoint colleagues in the audience. I guess we did a pretty good job as our scores are right around 4.6 out of 5.0. That's pretty outstanding survey results! After the session I had several people ask me to blog about the MMS implementation and my upgrade tool. I'll be doing that next week for everyone!

I'll be posting the slides to the SanSpug.org site for our members to check out as soon as I am able to meaningfully categorize them all.

Parties:

There were a ton of parties this week. As I already mentioned, the best party was the AvePoint "Red Party" party, having an entire dance club to ourselves is pretty darn awesome!

The next best party would have to be the Speaker Party on Thursday night. They brought in two in-and-out burger trucks and cooked us up fresh burgers, fries and shakes...the best part however was their was a DJ. And not only were the speakers invited, but the event staff too. The speakers and MS folks kind sat around chatted, but the staff got down! Two of the staff folks were just flat out awesome break dancers and started a competition...it was killer! The DJ started to play the last song and then I yelled "Move like Jagger", he pointed at me and said "That's a better one..." and mixed/faded into it perfectly from Sister Sledge. We danced for a good hour at that party and I had my heart going and sweated it up...

The next best party was the Disneyland event hosted by Nuedesic. They managed to rent out the entire park just for the SharePoint conference, and what was even more cool, they had bars setup all over the park. The staff said they had never seen alcohol served park-wide before, so I'm guessing they spent a heck of a lot of money getting that event setup for us:

The next best party of the week was the Octoberfiesta party, although we didn't make it over, I heard they did some pretty cool stuff and everyone enjoyed it:

The next best drinking event was the karaoke late into the night on Wednesday. We had the likes of Andrew Connell, Dux Raymond, AvePoint folks, Dave Milner and several others singing it up. I did the Sister Sledge, "We Are 'SharePoint' Family",but I'd have to say the best singer ended up being Dave Miler! We (Randy Williams and I) tried to get a mosh pit going with some body banging, but the karaoke lady stopped us right away...darn it...

Summary:

Even though we didn't learn anything new about v.next (other than that they have hired alot of UI developers), the event was a great chance for the SharePoint community to come together, "new kids on the block" vendors to show off their products and have one heck of a set of parties!

Enjoy!
Chris

SharePoint Printing Services, UrlAction, STSNavigate2 - Fixing "Invalid page URL:" in SharePoint 2010

cjg — Sat, 01 Oct 2011 18:38:00 GMT

After the eBay upgrade, I have decided that it is really easy to do SharePoint upgrades and thusly I'm doing another upgrade here in San Diego. As part of the upgrade, you have to go through and analyze the 3rd party components that are installed and make sure they are going to work in 2010. As I have learned, most are not SP2010 ready (OutlookPowerTools, older CorasWorks versions to name a few). Some of them work with some tweaking. That's what this blog post is about.

This particular customer has the PrintingServices.wsp that allows a one click approach to printing pages and items. What this product does is adds custom actions with a javascript url as the target of the click. This works great in SharePoint 2007, but unfortunately, does not in SP2010. The reason it does not work in SP2010 is that they do not simply pass the URL to the page anymore. It is now wrapped by another javascript method called "STSNavigate2":

function STSNavigate2(evt, url)
{ULSxSy:;
STSNavigate(url);
}

As you can see, it in turn calls function STSNavigate:

STSNavigate in turn calls STSPageUrlvalidation:

function STSPageUrlValidation(url)
{ULSxSy:;
return PageUrlValidation(url);
}

It then calls PageUrlValidation - THIS IS THE KEY PART:

Notice that the only valid URLS start with http and contain a colon!!!! Not sure the reasoning behind this "validation", but it breaks any custom actions that had a simple "javascript" call in them.

You can fix this by changing the C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\LAYOUTS\1033\Init.js file. Note that this file has been minified (variable names reduced and spaces removed) and does not match the init.debug.js file. Find the following text in the init.js file:

function PageUrlValidation(a){ULSA13:;if(a.substr(0,4)=="http"||a.substr(0,1)=="/"||a.indexOf(":")==-1)return a;else{var L_InvalidPageUrl_Text="Invalid page URL: ";alert(L_InvalidPageUrl_Text);return ""}}

Change it too:

function PageUrlValidation(a){ULSA13:;if(a.toLowerCase().indexOf("javascript")>-1) return a; if(a.substr(0,4)=="http"||a.substr(0,1)=="/"||a.indexOf(":")==-1)return a;else{var L_InvalidPageUrl_Text="Invalid page URL: ";alert(L_InvalidPageUrl_Text);return ""}}

This will then allow the older javascript links to work. be sure to document the change as it would get overwrite with any service packs later.

Enjoy!
Chris

New 5-day Course - FAST Search for SharePoint!

cjg — Sat, 01 Oct 2011 18:31:00 GMT

After finally having some time to do some course development, I have finished our FAST Search for SharePoint course. We implemented FAST at eBay and we learned quite a few lessons after doing it, all this is rolled up into the course! I'll be giving away a couple copies of the course during the SharePoint Conference. You'll have to join us at our SoCal party hosted by ACS, ShareSquared and New Horizons SoCal to win! Here is the outline:

Designing and Deploying FAST Search
- Install FAST ESP For SharePoint
- Configure FAST ESP
- Configure Claims Authentication
- Index Content into FAST Databases
- Setup FAST Search Center
- Exploring Nctrl
- Disabling unnecessary FAST services
- Reconfiguring FAST Services
- Explore Log Files
- Explore Performance Counters
- Explore WMI
- InfoToo.exe

Feeding Content
- Utilize DocPush.exe to Feed documents
- Utilize DocLog.exe to get pipeline information
- Enable the Spy Stage
- Create a custom Pipeline Extension (Console Application)
- Create a custom Pipeline Extension (PowerShell)
- Configure FAST Search database connector
- Perform a crawl using the FAST Search database connector
- Clearing the FAST Search index

Processing Content
- Managing Property Extractions
- Creating a custom Property Extractor
- Explore optionalprocessing.xml
- Enabling Offensive Content Filtering
- Enable Advanced Filter Pack
- Enable XML Format Detection
- Enable XML Mapping
- Create a XML Mapper Configuration File
- Configuring StopWordThresholds
- Configuring Managed Property Levels
- Utilize the DocVector managed property to generate a Tag Cloud

Searching Content and Retrieving Results
- Simple Searches via SharePoint Search
- RSS and Windows Explorer Search
- AND clause
- ANDNOT clause
- OR clause
- COUNT clause
- XRANK clause
- NEAR clause
- ONEAR clause
- Wildcard query
- Random query
- Testing Search Result Security
- Adding Keywords
- Adding Best Bets
- Adding Visual Best Bets
- Adding Crawl Properties
- Adding Managed Properties
- Modifying Search Results XSLT
- Modifying Refiners (Managed Property)
- Modifying Refiners (Managed Metadata)
- Managed User Profile Properties
- Creating User Context Properties
- Creating User Contexts
- Implementing Site Promotions
- Add Query Suggestion (PowerShell)
- Modify the Summary Length in Results
- Explore SharePoint Search Scopes
- Create Search Scopes utilizing Extended Search Filters
- Add Query Suggestion (PowerShell)
- Explore SharePoint Search Scopes
- Create Search Scopes utilizing Extended Search Filters

Implementing Relevance and Linguistics
- Configuring Query time Lemmatization
- Explore FAST Dictionaries
- Configuring Query time Lemmatization
- Exploring Dictionaries
- Explore Refiners
- Add new Refiners
- Explore Ranking Profiles
- Create new Ranking Profiles
- Change default ranking profile
- Managed Property Boosts
- Configuring RankLog Analysis
- Explore Current Spell checking suggestions
- Add a Spell Check Exception
- Reset the Spell Checking Dictionary
- Create a Site Promotion
- Create a Site Demotion
- Create a Document Promotion
- Create a Document Demotion
- Character Normalization
- Antiphrasing
- Synonyms
- Configure FAST Tokenization (Word Breaker)
- Enable substring tokenization
- Configure substring and linguistic tokenization properties

See everyone at the conference!
Chris

SharePoint Conference 2011 Session - How eBay Successfully Upgraded their Intranet to SharePoint 2010

cjg — Fri, 26 Aug 2011 17:11:00 GMT

Dear SharePoint Colleagues,

Join the eBay team and myself as we discuss the eBay upgrade to SharePoint 2010 and how we did it in 3.5 months. The Conference session link is here:

How eBay Successfully Upgraded their Intranet to SharePoint 2010

Here is an outline of what we plan on speaking about:

Why eBay moved to SharePoint 2010
The environments (2007 and 2010)
eBay Governance
eBay Project Plan - how we managed the project and the methodologies we used
How we documented the farm
How we approached the technical side of the upgrade
How we increased performane by 30%
How eBay implemented disaster recovery
And alot more!

We will answer questions after the session, but to have a one on one with the eBay team, join us Wednesday night at our SoCal SharePint. We will have a limited number of drink tickets and details on how to get the tickets are here:

http://blogs.architectingconnectedsystems.com/blogs/cjg/archive/2011/08/24/SharePoint-Conference-2011-_2D00_-SoCal-SharePint.aspx

Hope to see you there!
Chris

SharePoint Conference 2011 - SoCal SharePint

cjg — Wed, 24 Aug 2011 04:36:00 GMT

Dear SharePoint Colleagues,

Get ready for a great evening at the 2011 SharePoint Conference! On Wednesday night, Oct 5th, after the exhibit hall reception, we will be having a SoCal SharePint party. Join us at Bar Louie's outside patio reserved specifically for us (directly across the street from the Convention Center) for a quick drink from 7 to 9pm. Meet the eBay team, one of a few reference customers of the SPC11 conference, and several SharePoint MVPs and superstars. After the SoCal SharePint, we will head over to enjoy another SharePint in the ESPNZone in Downtown Disney!

This event is hosted by the local Southern California companies that want to make sure you have a great time while in Anaheim, CA:

Architecting Connected Systems (San Diego,CA) - eBay Upgrade Architect and Preferred Microsoft Certified SharePoint Courseware Company
New Horizons of Southern California (Anahiem, CA) - A Premier Microsoft Certified SharePoint Training Company
ShareSquared (Pasadena, CA) - A Premier Microsoft Certified SharePoint Consulting and Software Company

Get your free drink tickets! There are several ways to get a free drink ticket:

Find any of the eBay team (they will be wearing eBay shirts and badges as well as have a session during the conferece) and ask them about their upgrade, and you get a free drink ticket!
Stop by the ShareSquared booth and ask them about their great SharePoint offerings, and you get a free drink ticket!
Call your local SoCal New Horizons Account Executive, 888-825-6684, and/or stop by the New Horizons office in Anaheim, and you get a free drink ticket!

Looking forward to seeing you there!
Chris

SharePoint 2010 Upgrade Project and Goverance Plan Samples

cjg — Wed, 24 Aug 2011 04:29:00 GMT

Dear SharePoint Colleagues,

We are pleased to announce that we are making available generic MS Project Plan and Governance plans and the SP Build Guide and MS Project Deployment Project based on all the upgrade projects we have been involved with over the years (including the latest eBay project). I will post the PayPal links for these later this week in this blog post and will setup a seperate site for them. Licensing will be based on user and will be non-transferrable except for projects you work on at least 25%. I'll post some pictures of the generic outline of the sections for both so you see what you are getting. You will get a full stampled PDF version and/or a MS Project template of each as part of your purchase. These items have been and will continue to be included in ACS SharePoint courses at no additional charge.

OR get all four for a discount!

SharePoint 2010 Documentation Package - $599

Each of these are based on large SharePoint projects and years of working with customers around the globle implementing SharePoint 2007 and 2010. The most recent being the large eBay upgrade project.

Enjoy!
Chris

What the hell is NDR64 and why should you care?

cjg — Wed, 24 Aug 2011 03:57:00 GMT

Sooo, I mentioned that we did a few *firsts* at eBay. One of them was to put Windows Server 2008 R2 in the DMZ and try to get SharePoint 2010 to work. Needless to say, after setting everything up properly, it didn't. After enabling 500 failed request tracing in the web servers and doing some crazy port analyzer stuff with good ole WireShark, I determined that there was some kind of firewall issue occuring between IIS and the domain controller in the DMZ.

Everything was setup properly in the firewall, with IIS and with SharePoint, so we all were very confused by what was occuring. It got so bad, that we made a preimer support call to see if someone could help us figure out what the hell was going on. Turns out we got one of the top IIS guys in Redmond on the phone and he was able to very quickly tell us we were having an RPC problem. We were all very confused, why are we having an RPC problem? He quickly told us about NDR64. NDR64 is a new protocol for RPC communicaiton in windows server 2008 and windows server 2008 R2. This protocol was not exactly written by Microsoft nor did Microsoft want to implement it (hint it was a MS partner that drove them to do it, but NDA keeps me from saying who and why). here's the details of NDR64:

http://bit.ly/nN1YOt

So what does that mean? Well, let me explain it...turns out that today's firewalls are able to recognize RPC packets and conversations and can dynamically open the ports between servers based on these conversations. The new NDR64 is a 64bit version of the older 32bit version of the RPC protocol and MOST firewalls today, DO NOT know how to handle or interrogate the traffic. SO what does that mean? It means that you better be ready to open a whole bunch of high level ports to ensure that your authentication traffic will work when running SharePoint 2010 in your DMZ on Server 2008 and Server 2008 R2.

Until all the firewall vendors write a new bios/software to support his new RPC protocol, you will have to keep from implementing a DMZ 2010 environment, or open a lot of ports until your firewall vendor figures it out.

I'd start calling your firewall vendor NOW and see when you can get the latest firmware/software version.

Enjoy,
Chris

FAST Seach - Admin OM Exception occurred. Message: Failed to communicate with the WCF service - The trust relationship between the primary domain and the trusted domain failed.

cjg — Sat, 20 Aug 2011 22:09:00 GMT

I was getting this error:

Admin OM Exception occurred. Message: Failed to communicate with the WCF service. , Error Code:e107, Stacktrace:Microsoft.SharePoint.Search.Extended.Administration.Common.AdminException: Failed to communicate with the WCF service. —> System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: The trust relationship between the primary domain and the trusted domain failed.

This was a great blog to point me to the issue. Turns out I had two domains in my Active Directory that had lost the trust setup but nothing was broken with my Active Directory or FAST setup. I had to delete the trusts and add them back, then it started working again.

http://blog.isaacblum.com/tag/the-trust-relationship-between-the-primary-domain-and-the-trusted-domain-failed/

Enjoy!
Chris

Just another Don't Do this when Developing SharePoint Blog Post - IT IS NEEDED!

cjg — Tue, 09 Aug 2011 18:38:00 GMT

Ok, you hear over and over and over again, don't do this, don't do that when developing SharePoint. But it seems that not too many people listen to all that wonderful best practice guidance. As part of the eBay project, we inherited A LOT of code from previous developers and consultants (which will remain unnamed, but ask me at SPC and I'll tell you). This code has so many bad components to it it was unbelieveable. Every SharePoint rule was broken. There were so many broken things that we couldn't fix them all, so we focused on the most important and visable items.

Maarten has done a great job of summarizing all the things we had to do to fix this code. Check out his blog post here:

http://withinsharepoint.com/archives/186

Enjoy!
Chris

Output Caching in SharePoint 2010 - It doesn't work, err it does...

cjg — Fri, 29 Jul 2011 20:56:00 GMT

Blog #7 in How I Successfully Upgraded eBay to SharePoint 2010 - See Previous Blog In Series

In my last blog post I made a comment that Output Caching does not work in 2010. Well, it works, but that's the problem. SharePoint 2010 has elements that are very poorly designed...but we all knew that didn't we?

What is object caching? It's an ASP.NET feature that allows the HTML of the page to be cached in the web server's memory and very quickly be presented to the clients (albeit slowly with high latency). Output caching can be turned on on your publishing sites in the Site Actions, Site Settings page for the site collection. Again, once turned on, the HTML is cached....this is bad. Yes, I said it is bad. Why?

There is this wonderful control called "PersonalSiteActions". This control emits javascript directly to the page. This javascript is specifc to a user. Wha???? Specifc to a user? But that means....yep....you got it. The first person to hit the page will cause the HTML to render and THAT HTML WILL BE CACHED. What does the rendered HTML look like? Here it is:

function ctl00_ctl39_SocialNavControl_insertMyProfileMenu() {
var menus = document.getElementsByTagName('menu');
if (menus == null) return;
var menu = null;
for (var i=0, len=menus.length; i<len; i++) {
if (menus.id.lastIndexOf('PersonalActionMenu') != -1) {
menu = menus; break;        }    }    if (menu == null)   return;
var elm = document.createElement('ie:menuitem');
elm.setAttribute('menugroupid', '50');
elm.setAttribute('description', 'View and manage your profile.');
elm.setAttribute('text', 'My Profile');
elm.setAttribute('onmenuclick', 'STSNavigate2(event,\'http:\u002f\u002fwww.blah.org:80\u002fsites\u002fmy\u002fPerson.aspx?accountname=CONTOSO\GIVENSCJ)');
elm.setAttribute('id', 'ID_MySiteLinksMenu');

var elm2 = document.createElement('ie:menuitem');
elm2.setAttribute('menugroupid', '50');
elm2.setAttribute('description', 'Open your personal homepage');
elm2.setAttribute('text', 'My Site');
elm2.setAttribute('onmenuclick', 'STSNavigate2(event,\'http:\u002f\u002fwww.blah.org:80\u002fsites\u002fmy\u002f\')');
elm2.setAttribute('id', 'ID_MySiteMenu');
...

What happens when the next user comes along? The javascript is CACHED. When they click on the "My Profile" link, they will go to the GIVENSCJ personal site NOT their personal site. This is what is know as a non-cache safe control. One might think that you could just go into the ASPX page and edit this, but NO. This is inside the PersonalSiteActions control. One might think, I'll create my own control...well, that's a start, but upon further investigation via reflector, you see that is IMPOSSIBLE. The control has several permission based checks for if a page is in Shared mode or Personal mode and all kinds of internal methods and properties. Dead end.

This same problem happens for any control you have written that outputs user specific content. You must ensure that ALL of your controls are cache safe for the basic out of the box output caching to work. If you don't want these to be cached, you are goign to have to tell the output cache not to cache those sections. Now we are customizing the out of the box master pages...YUK.

NOTE: One other issue with the PersonalSiteActions is the "My Site" link. Notice how it takes you to the default.aspx page that has the newsfeeds. How is that your "My Site"??? Really? The SharePoint team could have kept the branding similar as it just confused the hell out of users when they are navigating in SharePoint 2010. So what can you do with this?

Change the link where the javascript points
Update the page to have a redirect

The only way to change the link for the PersonalSiteActions is to do HTML capture and rewrite (yes we even tried javascript and fancy JQuery - no go!). You can write an HTTP Handler for this. What happens if you do this? Our friend Output Caching is not a friend to HTML rewrite. It forces the output cache to disable itself. Output cache minus the quirks in SharePoint 2010, can cause a huge increase in application performance. Implmenting HTTP redirect forces this off, but it worked as I was able to replace the link to the proper link for the my site

The easiest way to get the "branding" right, was to put on the default.aspx page, a redirect to person.aspx. Problem solved, output cache saved.

Enjoy,
Chris

Increasing SharePoint Performance - CSS, Javascript and Pictures

cjg — Fri, 29 Jul 2011 19:44:00 GMT

Blog #6 in How I Successfully Upgraded eBay to SharePoint 2010 - See Previous Blog In Series

One of the requirements of our upgrade was that the upgraded system would perform better than the current 2007 production environment. This included the latency to Europe and other non-US countries. Their are many options that one would have to do this, these include the following:

Increased bandwidth
Decrease latency
Page Weight
Caching (Server, F5, Proxy, CDN)

Increased bandwidth:

In our case, bandwidth was not an issue. In your circumstances, that may or may not be the case. If you have a small pipe, you won't be able to get too many requests going through it. In the end, this really isn't a big deal.

Decreased Latancy:

SUPER SUPER important. Latency is the time is takes for the html packets to get to the client. If you have high latencies, your users are going to hate your intranet/SharePoint and call it slow. Even though you spent millions on hardware and it is able to generate the HTML faster than any other system, it can't get it to them fast enough.

Decrease Page Weight:

If you were to fire up Fiddler and watch the typcial traffic flow for a request to SharePoint 2010. You will see several requests are being made to the server. Because there is an order to the madness, their is a multiplier effect to the time it will take for a user to fully load a page (even with async requests occuring). YOUR NUMBER ONE GOAL IS TO MINIMIZE THE HTTP REQUESTS FOR A PAGE. How do you do this? You can do this in several ways:

Compress CSS
Minify Javascript
Evalute Picture Usage

Compress CSS:

The CSS classes between 2007 and 2010 changed A LOT! If you had custom CSS that overrode the out of the box CSS, it very likely will not have any effect in 2010. It should go without saying....you should remove this CSS. You should also attempt to determine how many different CSS files you have being downloaded from your pages. If you can consolidate those into one CSS, then you are going to be in a very good position moving foward. Steps:

Find all CSS classes used in your pages
Evaluate if they can be combined into one CSS file
Determine what CSS is used
Remove un-used CSS from your CSS files
Compress the CSS (remove all white space)

How does one find out is CSS is used or not? Here's the steps:

Get all your CSS classes used in your CSS files
Export all ASPX files from SharePoint to the SharePoint Root
Run a search on all the files for each CSS
Any CSS that is not found can be deleted

1. How does one export all the CSS? Heres a handy script:

$templateDir = "c:\program files\common files\microsoft shared\web server extensions\14\template"

del "$templateDir\ASPX" -recurse -force
mkdir "$templateDir\ASPX"

#export all aspx files to temp directory
$cdbs = get-spcontentdatabase
$count = 0

foreach($cdb in $cdbs)
{
"Processing " + $cdb.Name

$conn = new-object system.data.sqlclient.sqlconnection $cdb.legacydatabaseconnectionstring.replace("Timeout=15","")
$conn.open()
$cmd = $conn.CreateCommand()
$cmd.CommandTimeout = 4000
$cmd.commandtext = "select dirname, leafname, content from alldocs ad, alldocstreams ads where ad.Id = ads.id and ads.id in (select id from alldocs ad where extensionforfile in ('aspx')) and ad.internalversion = ads.InternalVersion";
$reader = $cmd.executereader()

while ($reader.read())
{
$filename = $reader["leafname"].replace("/","_")
$fileContent = $reader["content"]
$dirname = $reader["dirname"]

$temp = $count.tostring() + "`t" + $cdb.Name +"`t" + $dirname +"`t" + $filename
add-content "Files_ASPX.txt" $temp

[System.IO.File]::Writeallbytes($templateDir + "\ASPX\" + $count.tostring() + "_" + $filename, $fileContent)
$count++
}

$reader.close()
$conn.close()
}

2. How does one search for the CSS? Here's another handy script (note it will take a while for this to finish):

$classes = get-content "Css.txt"
cd "c:\program files\common files\microsoft shared\web server extensions\14"

foreach($_ in $classes)
{
dir -r -i *.aspx | select-string $_ | select filename, linenumber >> $_.txt
dir -r -i *.ascx | select-string $_ | select filename, linenumber >> $_.txt
dir -r -i *.css | select-string $_ | select filename, linenumber >> $_.txt
}

3. Create a list of all the CSS classes used in your CSS,

4. Compare the output of the last script and your CSS list, determine what is not used. I'll leave it to you to write the script for that. Just use a hashtable :)

5. Compress your CSS, tools for that:

http://www.csscompressor.com/

Minify BLOCKED SCRIPT

SharePoint 2010 will cause your page weight to go up. This is because of its heavy use of JavaScript. They did think ahead and minify the javascript files, so that ensure you don't take a huge hit on the files. If you have your own custom JavaScript and you haven't minified it, you are causing a higher page weight and your remote users will not have an optimal experience.

Tools to Minify are here:

http://www.minifyjavascript.com/

Evaluate Picture Usage:

Does your site have a lot of pictures? Doesn't look like SharePoint anymore? That's both a good and bad thing. The more pictures you have, the large the page weight. Here's what you can do to decrease the page weight:

Ensure that buffer images are width 1 and heigth 1 - I can't tell you how many times I have seen an image created that is 10K and is used as a buffer image. It only needs to be a few bytes and a 1x1 image!!!
Use sprites - make lots of small images into one to reduce your HTTP requests

Caching:

There are lots of ways to do caching. And there is a ranking of most benefical and easy caching methods...here they are:

Proxy Server
Content Delivery Network (CDN)
Hardware compression
F5 Load Balancer
Web Server

Proxy Server - The proxy server method *really* is the best one. You can put a proxy server in each of your geo locations. The job of the proxy server is to cache common elements like CSS, Images and Javascript and do it on that side of the "pond". This makes such a HUGE difference its not even funny.

Content Delivery Network (CDN) - very similar to the Proxy server in that it keeps a local copy of the content, but you have to factor that into your web page design/HTML. I'm not a fan from an easy of manageability standpoint, but it would match the proxy server in the performance increase you would see.

Hardware Compression - By deploying devices on your network edges, you can compress the WAN traffic. This tends to be a very costly solution and requires a lot of testing. In larger organizations, good luck with getting the network guys to mess with anything sitting next to the routers!

F5 Load balancer - an F5 can act like a proxy server by caching files, but typcially the F5 sites in front of the web servers on the Farm side of the network. It is very cost prohibative to have an F5 in each data center in each geo location acting as a proxy server. That being said, it can still site on the Farm side and cache the CSS, Images and Javascript. It still is exposed to the latency of your network, but the files will be in memory and be served up faster to the clients because they won't have to be fetched from the web server file system or content database.

Web Server - You can have the web server cache HTML from the execution of the ASPX pages. This does NOT help with the latency issues, but it does help with the actual execution performance. As we will see in the next blog post, Object Caching KILLS SharePoint 2010 so you have to be careful where you implement it. You can also implement Blob caching at the web server level. This is very beneficial for publishing sites as it will keep SharePoitn from hitting the content database every time to load images, CSS and Javascript.

In the end, we were able to increase the visible and technical user performance for Eurpoean users by over 30%. They have made alot of comments of how much faster it is, even if their are weird quirks with the sites, they are still amazed at how fast it is! Chalk one up for the CJG-mister!

Enjoy!
Chris

See the next blog post in this series HERE

Rapid Solution Deployment - SharePoint 2010

cjg — Fri, 29 Jul 2011 19:32:00 GMT

Blog #5 in How I Successfully Upgraded eBay to SharePoint 2010 - See Previous Blog In Series

As part of the eBay upgrade, we had several code changes that had to be made and tested. This included Solution Deployment and all its glory. One of the things I really got tired of doing was going into Central Adminsitration and undeploying solutions. It is a very painful process. You may ask, why didn't you do it through Visual Studio...yeah right...it errors out half the time and never really gives you a great message on how to resolve it. Therefore, I wrote a script for that! I'm going to be posting several of the scripts we utilized to very common processes during the upgrade. Here is the script that we built to deploy the solutions:

$minutes = $args[0]

if (-not $minutes)
{
$minutes = 15
}

$di = new-object system.io.directoryinfo("c:\WSPDirectory")

foreach ($fi in $di.getfiles("*.wsp"))
{
$date = get-date
$date = $date.addminutes($minutes*-1)

if ($date -lt $fi.lastwritetime)
{

$fi.name
$sol = $null

try
{
$sol = get-spsolution $fi.name -erroraction silentlycontinue
}
catch{}

if ($sol)
{
"Retracting $fi"
uninstall-spsolution $fi.name -confirm:$false -allwebapplications -erroraction silentlycontinue
uninstall-spsolution $fi.name -confirm:$false -erroraction silentlycontinue

while($sol.jobexists -eq $true)
{
start-sleep 3
$sol.jobstatus
$sol = get-spsolution $fi.name
}

"Deleting $fi"
$sol.delete()
}

"Adding $fi"
add-spsolution $fi.fullname

$sol = get-spsolution $fi.name

"Installing $fi"
install-spsolution $fi.name -allwebapplications -gacdeployment -erroraction silentlycontinue -force
install-spsolution $fi.name -gacdeployment -erroraction silentlycontinue -force

while($sol.jobexists -eq $true)
{
start-sleep 3
$sol.jobstatus
$sol = get-spsolution $fi.name
}

"Deployed $fi"
}
}

Note that one thing that can be added to this script is an interrogation of the solution ID. This is better than going off the name of the solution (which actually changes in production deployment using YYYY-MM-DD notation) and this script would not work for production. Only in Development and QA was this handy.

Enjoy!
Chris

See the next blog post in this series HERE

Upgrading to SharePoint 2010 - Excel Workbooks And InfoPath Forms With Data Connections

cjg — Thu, 21 Jul 2011 22:34:00 GMT

Blog #4 in How I Successfully Upgraded eBay to SharePoint 2010 - See Previous Blog In Series

So you decided to move your content databases around eh? The urls for the site collections and webs are different? Did you ever think about all those lonely lost users that created InfoPath Forms and Excel Workbooks with Data Connections to lists? Hmm...probably not!

So what would be the steps to start this massive endeavour? Well, the first would be to identify all the Excel and InfoPath workbooks in your farm, then you should download all of them, check if they have a data connection, then update them if they do. Wha? What if you have 1000s of them? Damn...you will spend months doing that...good luck!

Wait...Microsoft built us a commandlet for it. It is called "Update-SPInfoPathUserFileUrl". What does it do you ask? Well, it will iterate through all the infopath forms and data connection libraries in your farm and update similar to the Url Updater I talked about in the previous post. The only problem with this commandlet...IT DOESN"T WORK! Yes, you heard it right, it doesn't work. After serveral calls with the product support team, they finally conceded to us this fact. What is wrong with it you ask, here's a run down:

If the data connection library has anything else other than an ODC file, it will fail
If the ODC file has a CDATA tag in it (or is not a standard everyday XML file), the tool fails when it tries to create an XMLDocument of the file
For no reason at all, it won't update all the files

They submitted some internal bug requests and told us they would try to get us a hotfix (but it would take several weeks). Unfortunately, that didn't fly very well with us as we had to do the upgrade in two weeks! So, I wrote our own tool to do it programmatically! I'll summarize the steps:

Find all the InfoPath and Data Connection files in the Farm (across all content databases)
Run the tool passing in the same input file of replacement URLs
Update the InfoPath Files (which are CAB files by the way) in place in their respective libraries
Update the Data Connection (ODC) files, which are simply XML in place

So finding the files is the easy part, here's the script:

#export all aspx files to temp directory
$cdbs = get-spcontentdatabase

$count=0
foreach($cdb in $cdbs)
{
"Exporting file list from " + $cdb.Name

$conn = new-object system.data.sqlclient.sqlconnection $cdb.legacydatabaseconnectionstring.replace("Timeout=15","")
$conn.open()
$cmd = $conn.CreateCommand()
$cmd.CommandTimeout = 4000
$cmd.commandtext = "select DirName, leafname from AllDocs where extensionforfile in ('xsn', 'odc')";
$reader = $cmd.executereader()
$i = 0

while ($reader.read())
{
$line = $cdb.WebApplication.Url + $reader["dirname"] + "/" + $reader["leafname"]
add-content "InfoPathFilesToUpDate.txt" $line
$i++
}

$reader.CLose()
$conn.CLose()
}

Once you have the list of the files you need to update, the rest is easy....relatively speaking . So what's the next step? You gotta loop through all these files and determine if they have the old URLs in them. For an xml file, that is pretty easy. It's just a text file, but most ODC files are not valid XML files...YUK. But that's beside the point, its easy to replace the ODC file connection info, just look for the <odc:Connection> element. Inside of it will be a connection string with the url you are looking for.

Now on to the InfoPath files. How does one update an InfoPath file without opening it in InfoPath and changing the Data Connection? You could download it and use the "extract.exe" tool to output the file contents, then rebuild it using "makecab.exe", but wow, that is just too much work. And yes, I tried it, and it sucked. Basically Windows still has the ability to have a folder to be "tagged". Did you ever have that problem with hackers hitting your FTP server and "tagging" it? You couldn't delete their tag without wipping the entire drive. Guess what...when extracting certain InfoPath files, you can "tag" your directory. When making a scrip with a "delete all" files in a folder command errors out more RED than you ever care to see. So how do you successfully update an InfoPath CAB file?

Use the awesome Reflector tool to get the CabinetExtractor class out of the SharePoint dlls for the CommandLet.

This is an entire set of classes specifcally designed to update CAB files in memory. PERFECT!

Once I had the code working, I simply passed in the list of files, checked the manifest.xml file in the InfoPath CAB package for any offending URLs and if they were found, updated the file and then streamed it back into the CAB file. Then I just upload to the library. Everything worked like a charm!

NOTE: Site and List templates are CAB files...internally, they have a version called "3" in them. Magically, if you change this value to "4", a majority of them will work in 2010...HINT HINT...

Enjoy,
Chris

See next blog post in this series here

Upgrading SharePoint 2010 - Changing Your Host Headers - What It Means To Your Consultant

cjg — Thu, 21 Jul 2011 20:51:00 GMT

Part 3 of How I Successfully Upgraded eBay - Previous Blog Post

So what does it mean for you to "retire" an older web application for branding purposes? It is a simple change in Central Administration, but a nasty proposition when it comes to content in the site and most consultants will take the easy way out and say, no you can't do that (because they know how much work it is and don't really know what will break). What am I talking about? Let me give you an example:

In 2007, the web application host header was http://abc.contoso.com
In 2010, you want it to be http://xyz.contoso.com
Moving a content database from one web applicaiton to another with a different name (for service delivery reasons)
Moving a site collection from one content database to another content database in a different web application

Again, it is easy to do via Central Administration. It is easy to do the DNS and add an AAM, but did you think about content? Huh, what about the content...here's what you didn't think about:

Non relative links used in 3rd Party applications (Corasworks)
ASPX pages with non-relative links
Content Editor web parts with non-relative links
Update custom web part properties with non-relative links
Navigation nodes with existing non-relative links
Excel and InfoPath with Data Connections

The problem is non-relative links. It was a big problem in 2007 and in most vendors products that they didn't check the URL of where the code was running and convert the saved content into relative links! This is a major pain in the ass! Now you will see in 2010 and in most vendor products, they have learned a VERY valuable lesson. ALWAYS USE RELATIVE LINKS!

How does one upgrade 10s of 1000s of pages, 1000s of content editor web parts and navigation nodes with the new URLs? You build a tool of course! The tool is simple in its requirements:

Take an input file of old url to new url
Update all aspx pages with the new links - CHECK
Update Content Editor web parts with the new links - CHECK
Update custom web parts with links - CHECK
Update Navigation nodes with new links - CHECK
Update....wait...how to you update Excel and InfoPath? - OH MY...(the topic of the next blog post)

The tool is easy to build, but add in 1000s of sites, 1000s of pages and web parts...you have some problems:

It will take too long for it to update all your resources before the upgrade is finished. So what do you do to meet your upgrade window?
You have external links that still point to these older urls

The problem has a simple solution. HttpModules. All you have to do is intercept the HTTP request and redirect to the proper place. How do you know where to redirect them? Same as the tool, use the input file for the tool. Create a SharePoint list that the HttpModule pulls from that will redirect the request to the proper place. Here is a coule of examples:

Redirect http://abc to http://xyz
Redirect http://abc/HelloWorld to http://def/HelloWorld

There is an issue here. You would have two entries in the configuration list:

In this case, the order of replacement matters. If the first entry is applied, they will incorrectly get redirected to http://xyz/HelloWorld. This is not the desired outcome. You must redirect in order of most specific first, then to the most generic. You must also be careful with generic entries. Suppose you have two web applications:

If you have the following in your configuration list:

http://abc->http://xyz

You can expect that the requests to http://abc will be redirected properly, but the http://abcdef, will also get redirected and not the way you want. Also in the case of http://abc->http://abcdef, you would be in a continuous loop of adding http://abc to the redirect and of course, you will eventually overrun the address line in the browser!

Enjoy,
Chris

See next blog post in this series