Author: givenscj

Posted on

Case Study: How to Use Managed Metadata in Site Creation to drive Searchability

As part of our eBay presentation at the SharePoint Conference 2011, we showed how we performed the upgrade in 3 months and utilized Managed Metadata (MMS) in site creation to drive searchability across the SharePoint Farm.  The biggest question I have received so far from the presentation is about the MMS subject.  So I'm going to outline it in detail here. For those of you that are not familiar with MMS, I like to describe it using the 3rd rule in my three rules of development.  The first rule is "Don't code the same line of code twice", if you see the same line of code twice, you did something wrong (no polymorphism, missing base method calls, inheritance, etc).  The second is don't put the same thing in memory more than once (this really needs no clarification).  The third rule says don't put the same thing on disk twice.  By doing this, you now must implement synchronization between the multiple instances.  The only product I worked with over the years that ever implemented this pattern perfectly was Lotus Notes.  It is for this reason, we have MMS in 2010.

In SharePoint 2007, imagine you have created a list in SharePoint that uses a choice column. You would have 50 states, plus the other various items the US owns (Examples: Guam, Virgin Islands…can you name them all?).  If you use the state column in other places in the Farm, you pretty much have to copy and paste all the items over and over again.  This is somewhat of a problem.  For instance, if Texas ever successfully succeeds from the union, you gotta remove them from all the choice columns!  What about Iraq and Afghanistan?  Maybe those should be added to the list?   Who knows what might happen in the future, but you'd have to update each and every source in order for the data to be meaningful.  Therefore, rule #3 is a good one…don't put data on disk more than twice.  MMS helps us do this by centrally stored our list data in one place. 

So how did we use MMS at eBay?  The summary of the process is that when users create their own site, the content inside the site should be pre-tagged with MMS data so as to allow for simple searching of their content via FAST Search.  Here's a breakout of the parts:

Site Creation:

The first step is to create a site creation form.  This can be anything you are comfortable with (an aspx page, an infopath form, web parts, etc).  In the case of eBay we used two web parts.  One is the site creation landing page web part.  Previously the values used to "tag" the sites was hardcoded in the code.  As part of the upgrade I changed this to pull dynamically from MMS.  This web part allows the entry of the following information:

  • Business Unit (MMS)
  • Organization (MMS)
  • Office Location (MMS)

  • Site Name
  • Description
  • Keywords (terms that will allow users to find the site later)
  • Template (teamsite, blog, etc)
  • Security (allow inheritance/everyone to view the site)

When clicking submit, several things happen:

  • Create a new list item with all the details of the site creation request
  • Submit to the site creation processing webpart with the site creation request id
  • Start a long running transaction
  • Impersonate the user that made the request
  • Get a site number from a database for the site URL (example: 12345)
  • Determine what site collection to add the site to (allows for load balancing the sites across content databases)
  • Create the new web
  • Add to the custom site directory list (this list is used for later look up on a site lookup web part)
  • Setup navigation for the site
  • Set the web property bag with the Organization, BusinessUnit and Location properties
  • Create the metadata columns (outlined below)
  • Add the site keywords to FAST Search

All throughout this process if any errors occur another list is setup to keep track of the errors and steps of the long running process.  This allows OPS to debug the process and work with development in case something needs to be tweaked.

Site Directory:

You will need to ensure you have a list setup to keep track of your sites.  In the case of a pre-existing site directory it is likely that you have already created a choice column with your values.  This doesn't move very easily to MMS.  You will have to migrate the values to new MMS columns (create new column with different name, move value over, deleted old column, create column with old name, copy values again) .  If you already have this site directory, it is simple to run back through all the sites and add the web properties and then add the MMS columns to all the existing lists in the already existing sites (so not only new sites have MMS, but old ones too)!

MMS Setup:

We used the following PowerShell script to pre-populate the MMS:

$session = get-sptaxonomysession -site http://contoso.com
$termstore = $session.termstores["Contoso Corporate MMS"]

$termGroup = $termstore.groups["Contoso"]

if (-not $termGroup)
{
$termGroup = $termstore.creategroup("Contoso")
}

$termSet = $termGroup.TermSets["Organization"]

if (-not $termSet)
{
$termSet = $termGroup.CreateTermSet("Organization")
}

foreach($t in $termset.terms)
{
$t.delete()
}

$termSet.createterm("Finance",1033)
$termSet.createterm("Human Resources",1033)
$termSet.createterm("Information Technology",1033)
$termSet.createterm("Legal",1033)
$termSet.createterm("Marketing and Sales",1033)

$termSet = $termGroup.termsets["BusinessUnit"]

if (-not $termSet)
{
$termSet = $termGroup.CreateTermSet("BusinessUnit")
}

foreach($t in $termset.terms)
{
$t.delete()
}

$termSet.createterm("Corporate",1033)
$termSet.createterm("PayPal",1033)
$termSet.createterm("StubHub",1033)
$termSet.createterm("Bill Me Later",1033)

$termSet = $termGroup.termsets["Location"]

if (-not $termSet)
{
$termSet = $termGroup.CreateTermSet("Location")
}

foreach($t in $termset.terms)
{
$t.delete()
}

$termSet.createterm("APAC – Australia",1033)
$termSet.createterm("APAC – China",1033)
$termSet.createterm("APAC – Hong Kong",1033)
$termSet.createterm("APAC – India",1033)
$termSet.createterm("APAC – Israel",1033)
$termSet.createterm("APAC – Korea",1033)
$termSet.createterm("APAC – Taiwan",1033)
$termSet.createterm("EU – Austria",1033)
$termSet.createterm("EU – Belgium",1033)
$termSet.createterm("EU – Czech Replubic",1033)
$termSet.createterm("EU – Estonia",1033)
$termSet.createterm("EU – France",1033)
$termSet.createterm("EU – Germany",1033)
$termSet.createterm("EU – Ireland",1033)
$termSet.createterm("EU – Italy",1033)
$termSet.createterm("EU – Luxembourg",1033)
$termSet.createterm("EU – Netherlands",1033)
$termSet.createterm("EU – Poland",1033)
$termSet.createterm("EU – Spain",1033)
$termSet.createterm("EU – Sweden",1033)
$termSet.createterm("EU – Switzerland",1033)
$ter
mSet.createterm("EU – UK",1033)
$termSet.createterm("NA – Canada",1033)
$termSet.createterm("NA – U.S.",1033)
$termSet.createterm("Global",1033)
$termStore.CommitAll()

Note that once it is created, you should not delete the values.  This will cause the linkage between any previously created columns to be eliminated.  Not fun to look at and fix later.  So be sure to tell the ops team not to run the create more than once…EVER…

MMS Columns:

The hardest part of the entire process is the MMS column creation.  It works like this:

  • Check to see if the web prop bag has been set, if not then set the web props to the values in the intranet site directory list, otherwise set to some default values
  • For each list, add the metadata columns
    • You should not add metatdata to lists that are "hidden" or that are type "Survey".  It will break those list types
    • Great example is the HiddenTaxonomyList, kinda weird adding a MMS column to that one as it keeps track of MMS usage across the site, creates a bit of a loop…not good!
  • You should also avoid everything but the following site types, if you don't you will get some VERY ugly messages from SharePoint 2010 (try adding MMS to an Access Services site, it doesn't work!):
    • STS
    • MPS
    • BLANKINTERNET
    • CMSPUBLISHING
    • PUBLISHING
    • WIKI
    • BLOG
    • SPS
  • Adding the metadata column involves the following process:
    • Detemine if the column already exists (choose your column name wisely), in this case, we preappended with "Geo" to avoid any clashes with already existing column names and internal SP names
    • Provision a note field to support your MMS field (yup, each MMS column is actually two columns)
    • Create a new taxonomy field using the Guid of the note field in the TextField property
    • Set the default value of the field using the "WSSID;#VALUE|TERMID", hint: use -1 if WSSID has not been defined yet

The result is every viable list has MMS columns added:

Each MMS column has a default value set:

When a new item is created, the user doesn't have to do anything!  The values are pre-populated with the values they selected at site creation time!

Search:

As part of the site creation there is a keyword text box.  Users can type keywords delimited by semicolon.  Their keywords are then added to FAST Search.  Initially we thought adding to both SharePoint Search and FAST Search was the answer, but that didn't work as it actually would show two keywords in the results and not just one.  You can also have the keywords added as search suggestions.  What this means is that when they start typing in the search box, you will see the keyword displayed as a suggestion:

Then when the search is executed, you will see the site as a keyword and the results for that keyword displayed as business as usual:

 

Once the columns have been added to all the list items in all the sites, you have setup FAST Content SSA and indexed the content you will get some pretty cool results.  Because of the TaxonomyFilter refiners that is preconfigured in the FAST Search Center, you will see the pre-defined MMS column values on the left side of the screen:

Users can now find content specific to their geography, organization and functional group.  Results are now more meaningful and can be found much faster than with SharePoint 2007!

Enjoy!
Chris

Posted on

SharePoint Conference PPTs – Download them via PowerShell!

Here's the script to download the PPTs via PowerShell – the link to the file is below in case of copy paste errors!

function DownloadFile([string]$auth, [string] $file)
{
"Using " + $auth + " for file " + $file

$url = "http://www.mssharepointconference.com/sessionpresentations/" + $file
$file = "c:SPC11" + $file

$uri = new-object uri($url);
$httpReq = [system.net.HttpWebRequest]::Create($uri)
$httpReq.headers.add("Cookie", $auth)
$httpReq.Timeout = 75000
$httpReq.method = "GET"

$res = $httpReq.GetResponse()
$rs = $res.GetResponseStream();

$fs = new-object system.io.filestream($file, [System.IO.FileMode]::OpenOrCreate)
$read = new-object byte[] 1024

$count = $rs.Read($read, 0, $read.length)

while($count -ne 0)
{
$fs.write($read,0,$count)
$count = $rs.Read($read, 0, $read.length)
}

$fs.close()
$rs.Close()
$res.close()

}

function GetAuth([string] $username, [string]$password)
{
[system.net.servicepointmanager]::Expect100Continue = $false

#####################################
# Start session on SPConf site
####################################

$uri = new-object uri("http://www.mssharepointconference.com/Pages/Default.aspx");
$httpReq = [system.net.HttpWebRequest]::Create($uri)
$httpReq.Accept = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, */*"
$httpReq.ContentType = "application/x-www-form-urlencoded"
$httpReq.headers.Add("Accept-Language", "en-US")
$httpReq.method = "GET"

$res = $httpReq.GetResponse()
$rs = $res.GetResponseStream();
[System.IO.StreamReader]$sr = New-Object System.IO.StreamReader -argumentList $rs;
[string]$results = $sr.ReadToEnd();

#get the cookies
$strCookies = $res.Headers["set-cookie"].toString();                   
$res.Close();

$vnlehn = $strcookies

####################################
#  Do init post to live.com
####################################

$cookies = new-object system.net.CookieContainer
$uri = new-object uri("https://login.live.com/wlogin.srf?appid=000000004C03DD21&alg=wsignin1.0&appctx=ConferenceVue");
$httpReq = [system.net.HttpWebRequest]::Create($uri)
$httpReq.Accept = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, */*"
$httpReq.Referer = "http://www.mssharepointconference.com/Pages/Login.aspx?ReturnUrl=%2f_layouts%2fauthenticate.aspx%3fSource%3d%252FPages%252FDefault%252Easpx&Source=%2FPages%2FDefault%2Easpx"
$httpReq.ContentType = "application/x-www-form-urlencoded"
$httpReq.method = "GET"

$res = $httpReq.GetResponse()
$rs = $res.GetResponseStream();
[System.IO.StreamReader]$sr = New-Object System.IO.StreamReader -argumentList $rs;
[string]$results = $sr.ReadToEnd();

#get the cookies
$strCookies = $res.Headers["set-cookie"].toString();                   
$res.Close();

$mspr = $strCookies.substring(0, $strCookies.indexof(";")+1)
$mspok = $strCookies.substring($strCookies.indexof("MSPOK"))
$mspok = $mspok.substring(0, $mspok.indexof(";"))

$strCookies = $mspr + $mspok

#get random html values
$ppft = $results.remove(0, $results.indexof("PPFT"))
$ppft = $ppft.remove(0, $ppft.indexof("value") + 7)
$ppft = $ppft.substring(0, $ppft.indexof("`""))

$posturl = $results.remove(0, $results.indexof("g_QS=")+6)
$posturl = $posturl.substring(0, $posturl.indexof("`""))

####################################
#  Do the login
####################################

$uri = new-object uri("https://login.live.com/ppsecure/post.srf?" + $posturl)
$post = "login=" + $username + "&passwd=" + $password +"&type=11&LoginOptions=3&NewUser=1&MEST=&PPSX=Passport&PPFT=" + $ppft + "&idsbho=1&PwdPad=&sso=&i1=&i2=1&i3=4778&i4=&i12=1&i13=&i14=333&i15=334"
$encoding = new-object system.text.asciiencoding
$buf = $encoding.GetBytes($post)

$httpReq = [system.net.HttpWebRequest]::Create($uri)
$httpReq.method = "POST"
$httpReq.Referer = "https://login.live.com/wlogin.srf?appid=000000004C03DD21&alg=wsignin1.0&appctx=ConferenceVue"
$httpReq.contentlength = $buf.length
$httpReq.Headers.add("Cookie", $strCookies)
$httpReq.ContentType = "application/x-www-form-urlencoded"
$stream = $httpReq.GetRequestStream()

[void]$stream.write($buf, 0, $buf.length)
$stream.close()

$res = $httpReq.GetResponse()
$rs = $res.GetResponseStream();
[System.IO.StreamReader]$sr = New-Object System.IO.StreamReader -argumentList $rs;
[string]$results = $sr.ReadToEnd();

$stoken = $results.remove(0, $results.indexof("id=`"stoken")+19)
$stoken = $stoken.substring(0, $stoken.indexof("`""))

####################################
#  Return token back to spconf site
####################################

$uri = new-object uri("http://www.mssharepointconference.com/_layouts/ConferenceVue/WindowsLive.aspx")
$post = "stoken=" + $stoken + "&appctx=ConferenceVue&action=login"

$encoding = new-object system.text.asciiencoding
$buf = $encoding.GetBytes($post)

$httpReq = [system.net.HttpWebRequest]::Create($uri)
$httpReq.method = "POST"
$httpReq.contentlength = $buf.length
$httpReq.ContentType = "application/x-www-form-urlencoded"
$httpReq.Accept = "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, */*"
$httpReq.UserAgent = "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3; .NET4.0C; .NET4.0E; MS-RTC LM 8; LEN2)"
$httpReq.headers.Add("Cookie", $vnlehn + "; webauthtoken=")
$httpReq.headers.Add("Accept-Language", "en-US")
$httpReq.KeepAlive = $true
$httpReq.AllowAut
oRedirect = $false
$stream = $httpReq.GetRequestStream()

[void]$stream.write($buf, 0, $buf.length)
$stream.close()

$res = $httpReq.GetResponse()
$rs = $res.GetResponseStream();
[System.IO.StreamReader]$sr = New-Object System.IO.StreamReader -argumentList $rs;
[string]$results = $sr.ReadToEnd();

#get the cookie
$strCookies = $res.Headers["Set-Cookie"].toString();                   

#get the webauthtoken
if ($strcookies.contains("webauthtoken"))
{
$webauthtoken = $strCookies.substring($strCookies.indexof("webauthtoken"))
$webauthtoken = $webauthtoken.substring(0, $webauthtoken.indexof(";"))
}

#get the fedauth
if ($strcookies.contains("FedAuth"))
{
$fedauth = $strCookies.substring($strCookies.indexof("FedAuth"))
$fedauth = $fedauth.substring(0, $fedauth.indexof(";"))
}

return $vnlehn + ";" + $fedauth + ";" + $webauthtoken + ";"
}

######################################
#  Parse the HTML of the SPConf site
#####################################

#$results = get-content AllHtml.txt

#parse out the pptx links
#while($results.contains("SessionPresentations"))
#{
#$results = $results.remove(0, $results.indexof("SessionPresentations/")+21)
#$filename = $results.substring(0, $results.indexof("pptx") + 4)
#$results = $results.remove(0, $results.indexof("pptx")+4)

#$filename

#add-content "c: empppts.txt" $filename
#}

##############################
#  Start program
###############################

[System.Reflection.Assembly]::LoadWithPartialName( "System.Web" ) | out-null
[System.Reflection.Assembly]::LoadWithPartialName( "System.Net" ) | out-null

$username = read-host "Enter your LiveID username: "
$password = read-host "Enter your LiveID password: "

$auth = GetAuth $username $password

while(-not $auth.tostring().contains("webauthtoken"))
{
$auth = GetAuth $username $password
}

############################
# Download all the PPTs
###############################

#use for testing in case you NEVER get the webauthtoken back…
#$auth = ""

$ppts = get-content "c:ppts.txt"
mkdir "c:spc11"

foreach($line in $ppts)
{
DownloadFile $auth $line

}

Here's all the PPTX files on the site (Place in a files called c:ppts.txt):

SPC216.pptx
SPC340.pptx
SPC102.pptx
SPC2991.pptx
SPC214.pptx
SPC240.pptx
SPC367.pptx
SPC2994.pptx
SPC249.pptx
SPC263.pptx
SPC253.pptx
SPC256.pptx
SPC274.pptx
SPC285.pptx
SPC202.pptx
SPC225.pptx
SPC345.pptx
SPC228.pptx
SPC350.pptx
SPC104.pptx
SPC229_Mercaldo.pptx
SPC387.pptx
SPC2992.pptx
SPC410.pptx
SPC376.pptx
SPC258.pptx
SPC265.pptx
SPC273.pptx
SPC201.pptx
SPC311.pptx
SPC403.pptx
SPC335.pptx
SPC406.pptx
SPC226.pptx
SPC358.pptx
SPC359.pptx
SPC245.pptx
SPC246.pptx
SPC262.pptx
SPC221.pptx
SPC200.pptx
SPC330.pptx
SPC346.pptx
SPC347.pptx
SPC351.pptx
SPC239.pptx
SPC242.pptx
SPC371.pptx
SPC252.pptx
SPC109.pptx
SPC254.pptx
SPC255.pptx
SPC382.pptx
SPC266.pptx
SPC271.pptx
SPC3995.pptx
SPC224.pptx
SPC310.pptx
SPC223.pptx
SPC298.pptx
SPC244.pptx
SPC366.pptx
SPC212.pptx
SPC264.pptx
SPC108_Barnard.pptx
SPC391.pptx
SPC280.pptx
SPC3992.pptx
SPC269.pptx
SPC289.pptx
SPC292.pptx
SPC215.pptx
SPC219.pptx
SPC338.pptx
SPC343.pptx
SPC234.pptx
SPC352.pptx
SPC103.pptx
SPC357.pptx
SPC248.pptx
SPC398.pptx
SPC288.pptx
SPC251.pptx
SPC306.pptx
SPC307.pptx
SPC204.pptx
SPC322.pptx
SPC206.pptx
SPC207.pptx
SPC2993.pptx
SPC348.pptx
SPC409_Kogan.pptx
SPC243.pptx
SPC368.pptx
SPC375.pptx
SPC383.pptx
SPC388.pptx
SPC275-1.pptx
SPC281.pptx
SPC3996.pptx
SPC293.pptx
SPC314.pptx
SPC213.pptx
SPC331.pptx
SPC401.pptx
SPC220.pptx
SPC222.pptx
SPC336.pptx
SPC3982.pptx
SPC397.pptx
SPC260.pptx
SPC261.pptx
SPC279.pptx
SPC291.pptx
SPC400_1.pptx
SPC203.pptx
SPC315_1.pptx
SPC342.pptx
SPC257.pptx
SPC259.pptx
SPC299.pptx
SPC365_1.pptx
SPC373_1.pptx
SPC377.pptx
SPC272.pptx
SPC389_1.pptx
SPC250.pptx
SPC278.pptx
SPC290.pptx
SPC304_A.pptx
SPC319_A.pptx
SPC287.pptx
SPC321_1.pptx
SPC296.pptx
SPC356_1.pptx
SPC374_1.pptx
SPC386_1.pptx
SPC270.pptx
SPC392_f.pptx
SPC414.pptx
SPC3999.pptx
SPC309_1.pptx
SPC328.pptx
SPC333_1.pptx
SPC339_1.pptx
SPC341.pptx
SPC407.pptx
SPC349_1.pptx
SPC237.pptx
SPC355_1.pptx
SPC360_1.pptx
SPC354_1.pptx
SPC379_1.pptx
SPC101.pptx
SPC396.pptx
SPC282.pptx
SPC410r.pptx
SPC308.pptx
SPC313_1.pptx
SPC205.pptx
SPC318.pptx
SPC210.pptx
SPC230.pptx
SPC233.pptx
SPC295.pptx
SPC236.pptx
SPC268.pptx
SPC369_1.pptx
SPC380_1.pptx
SPC384_1.pptx
SPC412_1.pptx
SPC266r.pptx
SPC241.ppt
SPC326.pptx
SPC329_1.pptx
SPC217.pptx
SPC218.pptx
SPC402_1.pptx
SPC404_1.pptx
SPC337_1.pptx
SPC408_1.pptx
SPC232_1.pptx
SPC378_1.pptx
SPC385_1.pptx
SPC300.pptx
SPC302.pptx
SPC344.pptx
SPC320.pptx
SPC362.pptx
SPC324.pptx
SPC327.pptx
SPC332.pptx
SPC334.pptx
SPC227.pptx
SPC277.pptx
SPC283.pptx
SPC413.pptx
SPC294.pptx
SPC3983.pptx
SPC267.pptx
SPC247.pptx
SPC361.pptx
SPC364.pptx
SPC393.pptx
SPC394.pptx
SPC395.pptx
SPC3993.pptx
SPC297.pptx
SPC303.pptx
SPC353.pptx
SPC2995.pptx
SPC372.pptx
SPC323.pptx
SPC411.pptx
SPC399.pptx
SPC3994.pptx

SPC221.pptx
SPC235.pptx
SPC276.pptx
SPC286.pptx
SPC312_1.pptx
SPC316_1.pptx
SPC363.pptx
SPC370.pptx
SPC381_a.pptx
SPC3991_1.pptx
SPC3997_Maio.pptx

Enjoy!
Chris

Posted on

SharePoint Conference 2011 Recap

Here is a short recap of all the cool stuff that happened this week.

Over 7,500 people showed up in Anaheim, CA on Sunday to enjoy networking and learning about SharePoint!  The event started off with the exhibit hall reception where you got your first sneak peak at the vendors. 

Exhibit Hall

The first big booths you would get to see was Quest and HP in the center.  HP didn't have much that interested me, but the Quest folks sure made things entertaining with their "Bonk" game.  You can find lots of pictures of SharePoint superstars on facebook and twitter, it was quite humorous and they gave away lots of cold hard cash to answer SharePoint questions!  In the following pictures you'll see @joelolsen @michaelnoel @buckleyplanet and then @ZlatanDzinic and @eusp

 

 To the left of HP was the the great guys from AvePoint (@danholme, @tweetraw, @jthake)!  As they have done for the past few SPConferences, they had a killer Ducati motorcycle to giveaway to some lucky attendee:

 

AvePoint also hosted the party of all parties, The Red Party, at a local bar called Heat Ultra Lounge.  Everyone was dressed in Red and the inside has all red lights, it was amazingly awesome!  AvePoint announced their new product versions including a Governance tool that has been under the wing of Jeremy Thake. 

On the right side of Quest were the very cool K2 guys showing off their awesome BlackPearl product and other workflow tools.  They were giving away a Segway for some lucky attendee!

 

Behind Quest and K2 were the good ole boys from fpweb.net ("fp" by the way means "Front Page" which was from an acquisition from a company called VTI which is the name of the web services directory VTI_BIN). fpweb pretty much has the coolest booth I had ever seen.  It has two stories with a full living room on the top floor.  It was of course roped off to only special fpweb MVPs and customers.  After a long day of sessions and walking the exhibit hall it was great to relax and drink a glass of wine on their very comfy couches.

 

fpweb.net had a couple of events during the week, including one that I master minded called the "Move like Jagger Contest".  There were several people at the conference that brought wives and kiddos and need extra passes.  It just turned out that three people gave me their passes and we decided to make people work for them by moving like Mick Jagger.  Rob LaMear, CEO of fpweb, put in some extra incentive by placing $100 bills on each of the passes, that got us about 10 people to do some Mick Jagger moves, one such person is @grumpytech. 

 

I'll be posting the video very soon!  The other event was a "fpweb.net is flexbile", where several young ladies came out and did some pretty amazing moves:

 

As far as new comers to the SharePoint software space, I though these two vendors had a pretty rocking demo:

  • #1 – ESRI – a long time ruling company in geocoding and mapping software came out with a Solution Package with a web part, custom field type with wizards to pull SharePoint list data and backend data to do geo-coding and mapping with heat maps. This stuff even knows how to use the SQL Server SpatialTypes to generate the data – AMAZINGLY COOL!  
  • #2 – SouthLabs – A very nice looking iPad and iPhone app for connecting to SharePoint with many authentication method support.  They were some very cool and bright guys from Uruguay and I think they will be able to win quite a lot of business from SharePoint customers with their mobile apps.
  • #3 – Pingar– They have an auto tagging solution designed by several computer science artificial intelligence people which used wikipedia as an input source to determine how to auto tag content in your SharePoint Farm.  It is by far the most advanced auto-tagging solution I have seen

And of course the hottest booth babe was my wifey who was hanging out at the @sharesquared booth:

 

Keynote:

The keynote area was HUGE!  As you can imagine, fitting over 7500 people into a room is quite a task.  During the keynote we had SharePoint Team member @thekameleon dj'ing.

 

The SharePoint trio, Jared Spararo, Jeff Teper and Kurt Delbene spoke and in-between they played some "cute" videos of celebrities, but unfortunately they didn't have anyone actually "present" for us to look at. 

 

The real highlight for me of the keynote was when the customer video played.  The very opening of the video had my customer eBay's intranet called "TheHUB".  The eBay mobile app also made an appearance with my name displayed and then immediately followed by me making a cameo on screen! 

 

I'll post the video link later so you can watch it!  One of the other neat demos they did was have a SQL Serve
r Denali instance setup in a cluster with some pretty crazy numbers around data corpus.  They pulled the network cable on the active SQL Server and had the cluster failover to the passive SQL Server node.  The hardware was in the range of $1.5 million and as much as it was a fun demo to see, it is not likely that many customers would be purchasing hardware of that magnitude.  Especially when you need have to build out the disaster recovery component at a similar cost in a different geographically located data center:

 

Sessions:

There were lots and lots of sessions!  One thing I noticed was that FAST Search made an appearance in each of the session sets.  It was great for me as I had just finished my FAST Search course and was able to validate that everything was actually covered!  One of the coolest sessions on FAST was the one held by Joshua at the International Monetary Fund (IMF).  They had done some pretty amazing stuff with FAST Search Center. The other impressive FAST Search session was done by Paul Branson, he was very knowledgeable and impressed me quite a bit with his demos and movement around the FAST directories.

Our eBay upgrade session was on Wednesday at 3:15, so we were able to party the night before and not have a early morning session where everyone was hungover and sleeping from the previous night!   We packed the room (with standing room only) with several Microsoft folks and saw several SharePoint colleagues in the audience.  I guess we did a pretty good job as our scores are right around 4.6 out of 5.0.  That's pretty outstanding survey results!  After the session I had several people ask me to blog about the MMS implementation and my upgrade tool.  I'll be doing that next week for everyone!

I'll be posting the slides to the SanSpug.org site for our members to check out as soon as I am able to meaningfully categorize them all. 

Parties:

There were a ton of parties this week. As I already mentioned, the best party was the AvePoint "Red Party" party, having an entire dance club to ourselves is pretty darn awesome! 

 

The next best party would have to be the Speaker Party on Thursday night.  They brought in two in-and-out burger trucks and cooked us up fresh burgers, fries and shakes…the best part however was their was a DJ.  And not only were the speakers invited, but the event staff too.  The speakers and MS folks kind sat around chatted, but the staff got down!  Two of the staff folks were just flat out awesome break dancers and started a competition…it was killer!  The DJ started to play the last song and then I yelled "Move like Jagger", he pointed at me and said "That's a better one…" and mixed/faded into it perfectly from Sister Sledge.  We danced for a good hour at that party and I had my heart going and sweated it up…

 

The next best party was the Disneyland event hosted by Nuedesic.  They managed to rent out the entire park just for the SharePoint conference, and what was even more cool, they had bars setup all over the park.  The staff said they had never seen alcohol served park-wide before, so I'm guessing they spent a heck of a lot of money getting that event setup for us:

 

The next best party of the week was the Octoberfiesta party, although we didn't make it over, I heard they did some pretty cool stuff and everyone enjoyed it:

 

The next best drinking event was the karaoke late into the night on Wednesday.  We had the likes of Andrew Connell, Dux Raymond, AvePoint folks, Dave Milner and several others singing it up.  I did the Sister Sledge, "We Are 'SharePoint' Family",but I'd have to say the best singer ended up being Dave Miler!  We (Randy Williams and I) tried to get a mosh pit going with some body banging, but the karaoke lady stopped us right away…darn it…

 

Summary:

Even though we didn't learn anything new about v.next (other than that they have hired alot of UI developers), the event was a great chance for the SharePoint community to come together, "new kids on the block" vendors to show off their products and have one heck of a set of parties!

Enjoy!
Chris

Posted on

New 5-day Course – FAST Search for SharePoint!

After finally having some time to do some course development, I have finished our FAST Search for SharePoint course.  We implemented FAST at eBay and we learned quite a few lessons after doing it, all this is rolled up into the course!  I'll be giving away a couple copies of the course during the SharePoint Conference.  You'll have to join us at our SoCal party hosted by ACS, ShareSquared and New Horizons SoCal to win!  Here is the outline:

  • Designing and Deploying FAST Search
    • Install FAST ESP For SharePoint
    • Configure FAST ESP
    • Configure Claims Authentication
    • Index Content into FAST Databases
    • Setup FAST Search Center
    • Exploring Nctrl
    • Disabling unnecessary FAST services
    • Reconfiguring FAST Services
    • Explore Log Files
    • Explore Performance Counters
    • Explore WMI
    • InfoToo.exe
  • Feeding Content
    • Utilize DocPush.exe to Feed documents
    • Utilize DocLog.exe to get pipeline information
    • Enable the Spy Stage
    • Create a custom Pipeline Extension (Console Application)
    • Create a custom Pipeline Extension (PowerShell)
    • Configure FAST Search database connector
    • Perform a crawl using the FAST Search database connector
    • Clearing the FAST Search index
  • Processing Content
    • Managing Property Extractions
    • Creating a custom Property Extractor
    • Explore optionalprocessing.xml
    • Enabling Offensive Content Filtering
    • Enable Advanced Filter Pack
    • Enable XML Format Detection
    • Enable XML Mapping
    • Create a XML Mapper Configuration File
    • Configuring StopWordThresholds
    • Configuring Managed Property Levels
    • Utilize the DocVector managed property to generate a Tag Cloud
  • Searching Content and Retrieving Results
    • Simple Searches via SharePoint Search
    • RSS and Windows Explorer Search
    • AND clause
    • ANDNOT clause
    • OR clause
    • COUNT clause
    • XRANK clause
    • NEAR clause
    • ONEAR clause
    • Wildcard query
    • Random query
    • Testing Search Result Security
    • Adding Keywords
    • Adding Best Bets
    • Adding Visual Best Bets
    • Adding Crawl Properties
    • Adding Managed Properties
    • Modifying Search Results XSLT
    • Modifying Refiners (Managed Property)
    • Modifying Refiners (Managed Metadata)
    • Managed User Profile Properties
    • Creating User Context Properties
    • Creating User Contexts
    • Implementing Site Promotions
    • Add Query Suggestion (PowerShell)
    • Modify the Summary Length in Results
    • Explore SharePoint Search Scopes
    • Create Search Scopes utilizing Extended Search Filters
    • Add Query Suggestion (PowerShell)
    • Explore SharePoint Search Scopes
    • Create Search Scopes utilizing Extended Search Filters
  • Implementing Relevance and Linguistics
    • Configuring Query time Lemmatization
    • Explore FAST Dictionaries
    • Configuring Query time Lemmatization
    • Exploring Dictionaries
    • Explore Refiners
    • Add new Refiners
    • Explore Ranking Profiles
    • Create new Ranking Profiles
    • Change default ranking profile
    • Managed Property Boosts
    • Configuring RankLog Analysis
    • Explore Current Spell checking suggestions
    • Add a Spell Check Exception
    • Reset the Spell Checking Dictionary
    • Create a Site Promotion
    • Create a Site Demotion
    • Create a Document Promotion
    • Create a Document Demotion
    • Character Normalization
    • Antiphrasing
    • Synonyms
    • Configure FAST Tokenization (Word Breaker)
    • Enable substring tokenization
    • Configure substring and linguistic tokenization properties

 See everyone at the conference!
Chris

Posted on

SharePoint Printing Services, UrlAction, STSNavigate2 – Fixing "Invalid page URL:" in SharePoint 2010

After the eBay upgrade, I have decided that it is really easy to do SharePoint upgrades and thusly I'm doing another upgrade here in San Diego.  As part of the upgrade, you have to go through and analyze the 3rd party components that are installed and make sure they are going to work in 2010.  As I have learned, most are not SP2010 ready (OutlookPowerTools, older CorasWorks versions to name a few).  Some of them work with some tweaking.  That's what this blog post is about. 

This particular customer has the PrintingServices.wsp that allows a one click approach to printing pages and items.  What this product does is adds custom actions with a javascript url as the target of the click.  This works great in SharePoint 2007, but unfortunately, does not in SP2010.  The reason it does not work in SP2010 is that they do not simply pass the URL to the page anymore.  It is now wrapped by another javascript method called "STSNavigate2":

function STSNavigate2(evt, url)
{ULSxSy:;
    STSNavigate(url);
}

As you can see, it in turn calls function STSNavigate:

function STSNavigate(Url)
{ULSxSy:;
    if (window.location.search.indexOf("IsDlg=1") !=-1)
    {
        if (Url.indexOf("?") !=-1)
        {
            if (Url.match("&$") !="&")
            {
                Url=Url+"&IsDlg=1";
            }
            else
            {
                Url=Url+"IsDlg=1";
            }
        }
        else
        {
            Url=Url+"?IsDlg=1";
        }
    }
    if (isPortalTemplatePage(Url))
        window.top.location=STSPageUrlValidation(Url);
    else
        window.location=STSPageUrlValidation(Url);
}

STSNavigate in turn calls STSPageUrlvalidation:

function STSPageUrlValidation(url)
{ULSxSy:;
    return PageUrlValidation(url);
}

It then calls PageUrlValidation – THIS IS THE KEY PART:

function PageUrlValidation(url)
{ULSxSy:;
    if ((url.substr(0, 4)=="http") ||
        (url.substr(0, 1)=="/")     ||
        (url.indexOf(":")==-1))
    {
        return url;
    }
    else
    {
        var L_InvalidPageUrl_Text="Invalid page URL: ";
        alert(L_InvalidPageUrl_Text);
        return "";
    }
}

Notice that the only valid URLS start with http and contain a colon!!!!  Not sure the reasoning behind this "validation", but it breaks any custom actions that had a simple "javascript" call in them. 

You can fix this by changing the C:Program FilesCommon FilesMicrosoft SharedWeb Server Extensions14TEMPLATELAYOUTS1033Init.js file.  Note that this file has been minified (variable names reduced and spaces removed) and does not match the init.debug.js file.  Find the following text in the init.js file:

function PageUrlValidation(a){ULSA13:;if(a.substr(0,4)=="http"||a.substr(0,1)=="/"||a.indexOf(":")==-1)return a;else{var L_InvalidPageUrl_Text="Invalid page URL: ";alert(L_InvalidPageUrl_Text);return ""}}

Change it too:

function
PageUrlValidation(a){ULSA13:;if(a.toLowerCase().indexOf("javascript")>-1) return a; if(a.substr(0,4)=="http"||a.substr(0,1)=="/"||a.indexOf(":")==-1)return
a;else{var L_InvalidPageUrl_Text="Invalid page URL:
";alert(L_InvalidPageUrl_Text);return ""}}

This will then allow the older javascript links to work.  be sure to document the change as it would get overwrite with any service packs later.

Enjoy!
Chris

Posted on

SharePoint Conference 2011 Session – How eBay Successfully Upgraded their Intranet to SharePoint 2010

Dear SharePoint Colleagues, 

Join the eBay team and myself as we discuss the eBay upgrade to SharePoint 2010 and how we did it in 3.5 months.  The Conference session link is here:

How eBay Successfully Upgraded their Intranet to SharePoint 2010  

Here is an outline of what we plan on speaking about:

  • Why eBay moved to SharePoint 2010
  • The environments (2007 and 2010)
  • eBay Governance
  • eBay Project Plan – how we managed the project and the methodologies we used
  • How we documented the farm
  • How we approached the technical side of the upgrade
  • How we increased performane by 30%
  • How eBay implemented disaster recovery
  • And alot more!

We will answer questions after the session, but to have a one on one with the eBay team, join us Wednesday night at our SoCal SharePint.  We will have a limited number of drink tickets and details on how to get the tickets are here:

https://blogs.architectingconnectedsystems.com/blogs/cjg/archive/2011/08/24/SharePoint-Conference-2011-_2D00_-SoCal-SharePint.aspx 

Hope to see you there!
Chris

 

Posted on

What the hell is NDR64 and why should you care?

Sooo, I mentioned that we did a few *firsts* at eBay.  One of them was to put Windows Server 2008 R2 in the DMZ and try to get SharePoint 2010 to work.  Needless to say, after setting everything up properly, it didn't.  After enabling 500 failed request tracing in the web servers and doing some crazy port analyzer stuff with good ole WireShark, I determined that there was some kind of firewall issue occuring between IIS and the domain controller in the DMZ. 

Everything was setup properly in the firewall, with IIS and with SharePoint, so we all were very confused by what was occuring.  It got so bad, that we made a preimer support call to see if someone could help us figure out what the hell was going on.  Turns out we got one of the top IIS guys in Redmond on the phone and he was able to very quickly tell us we were having an RPC problem.  We were all very confused, why are we having an RPC problem?  He quickly told us about NDR64.   NDR64 is a new protocol for RPC communicaiton in windows server 2008 and windows server 2008 R2.  This protocol was not exactly written by Microsoft nor did Microsoft want to implement it (hint it was a MS partner that drove them to do it, but NDA keeps me from saying who and why).  here's the details of NDR64:

 http://bit.ly/nN1YOt

So what does that mean?  Well, let me explain it…turns out that today's firewalls are able to recognize RPC packets and conversations and can dynamically open the ports between servers based on these conversations.  The new NDR64 is a 64bit version of the older 32bit version of the RPC protocol and MOST firewalls today, DO NOT know how to handle or interrogate the traffic.  SO what does that mean?  It means that you better be ready to open a whole bunch of high level ports to ensure that your authentication traffic will work when running SharePoint 2010 in your DMZ on Server 2008 and Server 2008 R2.

Until all the firewall vendors write a new bios/software to support his new RPC protocol, you will have to keep from implementing a DMZ 2010 environment, or open a lot of ports until your firewall vendor figures it out.

I'd start calling your firewall vendor NOW and see when you can get the latest firmware/software version.

Enjoy,
Chris

Posted on

SharePoint 2010 Upgrade Project and Goverance Plan Samples

Dear SharePoint Colleagues, 

We are pleased to announce that we are making available generic MS Project Plan and Governance plans and the SP Build Guide and MS Project Deployment Project based on all the upgrade projects we have been involved with over the years (including the latest eBay project).  I will post the PayPal links for these later this week in this blog post and will setup a seperate site for them.  Licensing will be based on user and will be non-transferrable except for projects you work on at least 25%.  I'll post some pictures of the generic outline of the sections for both so you see what you are getting.  You will get a full stampled PDF version and/or a MS Project template of each as part of your purchase.  These items have been and will continue to be included in ACS SharePoint courses at no additional charge.

OR get all four for a discount!

  • SharePoint 2010 Documentation Package – $599
    		•

    Each of these are based on large SharePoint projects and years of working with customers around the globle implementing SharePoint 2007 and 2010. The most recent being the large eBay upgrade project.

    Enjoy!
    Chris

    Posted on

    SharePoint Conference 2011 – SoCal SharePint

    Dear SharePoint Colleagues,

     

    Get ready for a great evening at the 2011 SharePoint Conference!  On Wednesday night, Oct 5th, after the exhibit hall reception, we will be having a SoCal SharePint party.  Join us at Bar Louie's outside patio reserved specifically for us (directly across the street from the Convention Center) for a quick drink from 7 to 9pm. Meet the eBay team, one of a few reference customers of the SPC11 conference, and several SharePoint MVPs and superstars.  After the SoCal SharePint, we will head over to enjoy another SharePint in the ESPNZone in Downtown Disney!

    This event is hosted by the local Southern California companies that want to make sure you have a great time while in Anaheim, CA:

    Get your free drink tickets!  There are several ways to get a free drink ticket:

    • Find any of the eBay team (they will be wearing eBay shirts and badges as well as have a session during the conferece) and ask them about their upgrade, and you get a free drink ticket!
    • Stop by the ShareSquared booth and ask them about their great SharePoint offerings, and you get a free drink ticket!
    • Call your local SoCal New Horizons Account Executive, 888-825-6684, and/or stop by the New Horizons office in Anaheim, and you get a free drink ticket!

    Looking forward to seeing you there!
    Chris

    Posted on

    FAST Seach – Admin OM Exception occurred. Message: Failed to communicate with the WCF service – The trust relationship between the primary domain and the trusted domain failed.

    I was getting this error:

    Admin OM Exception occurred. Message: Failed to communicate with the WCF service. , Error Code:e107, Stacktrace:Microsoft.SharePoint.Search.Extended.Administration.Common.AdminException: Failed to communicate with the WCF service. —> System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: The trust relationship between the primary domain and the trusted domain failed. 

    This was a great blog to point me to the issue.  Turns out I had two domains in my Active Directory that had lost the trust setup but nothing was broken with my Active Directory or FAST setup.  I had to delete the trusts and add them back, then it started working again.

    http://blog.isaacblum.com/tag/the-trust-relationship-between-the-primary-domain-and-the-trusted-domain-failed/

    Enjoy!
    Chris