March 2010

March 31, 2010

SharePoint Tool Ideas For YOU to Build!

Looking for the next thing to build for SharePoint? Here's a quick list from notes over the years:

Claims filter web part (2010)

This would present a set of registered claims to be used in BI solutions (Excel Services, Report Services, Performance Point, etc)

Velocity Sandbox Solution disable (2010) – as suggested by @sahilmalik

When in an Sandbox environment, if one Sandbox Solution goes crazy based on X rate, you deactivate it

Generic Sandbox Solution Validator – I BUILT THIS ONE ALREADY!
Timer Job Disabler – I BUILT THIS ONE ALREADY!
A feature that inserts a status message when the Site has not been used or is about to be deleted (2010)
Column Updater for field names

This tool would update all column names for a specific field type
across an entire farm in the event the name of the field changes.

Custom event handler to call IFilters and then parse files for bad words (similar to ForeFront feature)
Create email alias in active directory/exchange from the email settings page (if site collection admin)
Copy file permissions to a document library from NTFS
HttpModule to intercept and encrypt documents on "View"
Wiki linking based on different word other than page name
Extend web application policy permissions (override site collection level permissons)
Find and replace web part instances and migrate audience settings tool
Solution query tool (show what features are part of the solution wsp on central admin)
Page to add Multiple columns at once for content types (especially page layouts)

Ok, you guys and gals get to work!
Chris

March 31, 2010

Scripting Sharepoint 2010, Zero to C (as in E=MC^2)

So you wanna script your entire SharePoint 2010 Farm install eh? Want me to show you how to do it? I'm sure you do! Here's the steps:

Setup the pre-reqs. Download all the pre-req files, name them per the script and place them in the PreReqs share

set share=”//servername/PreReqs”

prerequisiteinstaller
/unattended /SQLNCli:%share%/sqlncli.msi /ChartControl:%share%/MSChart.exe
/IDFXR2:%share%/MicrosoftGenevaFramework.amd64.msi
/Sync:%share%/Synchronization.msi /filterpack:%share%/filterpack.msi
/ADOMD:%share%/SQLSERVER2008_ASADOMD10.msi

Create a config.xml file

Run the following command pointing to your config.xml file to install sharepoint

setup /config <pathto>config.xml

Configure SharePoint 2010

set s="C:Program FilesCommon
FilesMicrosoft Sharedweb server extensions14BINstsadm.exe"
set ps="c:program
filescommon filesmicrosoft sharedweb server
extensions14inpsconfig.exe"
set farmadmin=CONTOSOSP_Farm
set sql=DBNAME
set p=Pa$$w0rd

%ps% -cmd configdb -create -server
%sql% -database SharePoint_Config -user %farmadmin% -password %p% -passphrase
%p% -admincontentdatabase SharePoint_AdminContent
%ps% -cmd adminvs -provision -port
9999 -windowsauthprovider onlyusentlm
%ps% -cmd services install
%ps% -cmd secureresources
%ps% -cmd installfeatures

Start all Services

Get-SPServiceInstance | foreach-object {Start-SPServiceInstance
-identity $_.Id }

Create ALL the service applications

$DbServerAddress = "DBNAME"

$farmPassPhrase = ’Pa$$w0rd’

$svcPwd = ’Pa$$w0rd’

$username =
"CONTOSOSP_Service"

$password = ConvertTo-SecureString
$svcPwd -asplaintext -force

$credential = New-Object
System.Management.Automation.PSCredential $Username, $Password

$managedAccount =
new-SPManagedAccount -credential $credential

$app = New-SPIisWebServiceApplicationPool "All Services" -account $managedAccount

#New-SPUsageApplication -name
"Usage and Health Service Application"

New-SPAccessServiceapplication
-applicationpool $app -name "Access Services"

New-SPBusinessDataCatalogserviceapplication
-applicationpool $app -name "Business Connectivity Service Services"

New-SPExcelServiceApplication
-applicationpool $app -name "Excel Services Application"

$md = New-SPMetadataServiceApplication
-applicationpool $app -name "Managed Metadata Service"

New-SPMetadataServiceApplicationProxy
-name "Managed Metadata Service" -serviceapplication $md

$pps =
New-SPPerformancePointServiceApplication -applicationpool $app -name
"PerformancePoint Service"

New-SPPerformancePointServiceApplicationProxy
-name "PerformancePoint Service" -serviceapplication $pps

New-SPStateServiceApplication -name
"State Service"

$ps =
New-SPProfileServiceApplication -applicationpool $app -name "User Profile
Service"

New-SPProfileServiceApplicationProxy
-name "User Profile Service" -serviceapplication $ps

$vgs =
New-SPVisioServiceApplication -applicationpool $app -name "Visio Graphics
Service"

New-SPVisioServiceApplicationProxy
-serviceapplication $vgs -name "Visio Graphics Service"

$was =
New-SPWebAnalyticsServiceApplication -applicationpool $app -name "Web
Analytics Service Application"

New-SPWebAnalyticsServiceApplicationProxy
-serviceapplication $was -name "Web Analytics Service Application"

New-SPWordConversionServiceApplication
-applicationpool $app -name "Word Service"

$serviceapp =
New-SPSecureStoreServiceApplication -Name "Secure Store Service"
-partitionmode:$false -sharing:$false -databaseserver $DbServerAddress
-databasename "SSO" -applicationpool $app -auditingEnabled:$true
-auditlogmaxsize 30

$proxy = $serviceapp |
New-SPSecureStoreServiceApplicationProxy -defaultproxygroup:$true -name
"Secure Store Service Proxy"

Update-SPSecureStoreMasterKey
-ServiceApplicationProxy $proxy -Passphrase $farmPassPhrase

Start-Sleep -s 5

Update-SPSecureStoreApplicationServerKey
-ServiceApplicationProxy $proxy -Passphrase $farmPassPhrase

$searchapp =
New-SPEnterpriseSearchServiceApplication -name "Search Service
Application" -applicationpool $app

$proxy = New-SPEnterpriseSearchServiceApplicationProxy
-name "Search Service Application Proxy" -searchapplication
$searchapp

$si =
Get-SPEnterpriseSearchServiceInstance -local

Set-SPEnterpriseSearchAdministrationComponent
-searchapplication $searchapp
-searchserviceinstance $si

$ct = $searchapp |
New-SPEnterpriseSearchCrawlTopology

$crawlStore =
$searchApp.CrawlStores | where {$_.Name -eq
"Search_Service_Application_CrawlStore"}

New-SPEnterpriseSearchCrawlComponent
-searchapplication $searchapp -crawltopology $ct -searchserviceinstance $si
-crawldatabase $crawlstore

$ct |
Set-SPEnterpriseSearchCrawlTopology -active

Write-Host -ForegroundColor Yellow
"Waiting on Crawl Components to provision…"

while ($true) {

$ct =
Get-SPEnterpriseSearchCrawlTopology -Identity $ct -SearchApplication $searchApp

$state = $ct.CrawlComponents |
where {$_.State -ne "Ready"}

if ($ct.State -eq
"Active" -and $state -eq $null) {

break

}

Write-Host -ForegroundColor Yellow
"Waiting on Crawl Components to provision…"

Start-Sleep 2

}

$qt = $searchapp | New-SPEnterpriseSearchQueryTopology
-partitions 1

$p1 = ($qt |
Get-SPEnterpriseSearchIndexPartition)

New-SPEnterpriseSearchQueryComponent
-indexpartition $p1 -querytopology $qt -searchserviceinstance $si

$p1 |
Set-SPEnterpriseSearchIndexPartition

$propertyStore =
$searchApp.PropertyStores | where {$_.Name -eq
"Search_Service_Application_PropertyStore"}

$p1 |
Set-SPEnterpriseSearchIndexPartition -PropertyDatabase
$propertyStore.Id.ToString()

$qt
| Set-SPEnterpriseSearchQueryTopology –active

Your done, enjoy!
Chris Givens aka CJG

March 29, 2010

SharePoint 2010 Password Change Policy

SharePoint 2010 has a great new feature for setting your service account passwords for when your AD team has locked down the password policies. In Central Administration you can click Security->Manage service accounts links, and after selecting the account, you can then set the passwords for these accounts from central administration. You may get an Access Denied error via the Microsoft.SharePoint.Win32.SPNetApi32.NetUserChangePassword method. This is very simply calling the C++ Netapi32.dll to change the password. Reference this for more information:

http://msdn.microsoft.com/en-us/library/aa370650%28VS.85%29.aspx

In doing more research, any accounts that have "Do not allow user to change password" will fail to update the password. The actual changing of the password is done as if it is the user changing it. You should also watch out for the "Accounts can only change once per day setting", this would keep you from changing your password more than once in a single day.

Chris

March 29, 2010

SharePoint 2010 install accounts

This has been throwing everyone for a crazy loop and its time to clear up what's going on. So far most of my environments have been setup with accounts called SP_Admin, SP_Farm and SP_Service. the SP_Admin is the install account, SP_Farm is the Database access account (weird naming right?), and the service applications get SP_Service.

I encourage you to do this and see what happens on the other side of the install and Farm configuration wizard. You'll find out real quick what type of things you need to do to get SharePoint 2010 working using this best practice approach of service accounts. But, rather than just leave you hanging, one big help for you in your quest for least privileged SharePoint Installs is:

SP_Admin and SP_Farm must be Local Administrators on the SharePoint server.

It ends up that that "Database access account" really turns out to be the main Farm account. When you get to user profile synchronization, if this account is not a local admin, the setup of the ForeFront Identity Manager will fail miserably and you will need to kill off the User Profile Synch services (using powershell) and start all over after adding in the local admin rights. I'm not sure if that particular screen will get updated in RTM or not, but it would make a hell of a lot more sense if it had "Farm Account" rather than "Database Access Account" on the setup screen:

Another note, when you go to add a new server to the farm (unless you do some SQL permission setting), you will have to be logged in as SP_Farm, not SP_admin.

Chris

March 28, 2010

Becoming the Fastest SharePoint Developer in the World!

Who is it? I'm not sure, but I'm somewhere up there in the top 1% and that lets me charge $200/hr for my time. So why do I say I'm in the top 1%? Because I have this:

What is it you ask? It's MY code gen tool. It is the culmination of 10 years of development projects combined into a massive code generation platform. Some of you know I am one of the architects behind the Quiznos, Boston Market, Panda Express…etc, etc, etc gift and loyalty platform. It processes millions of transactions every day and has some very cool architectural components very similar to Visa and Discover networks (but a lot better). Some of you might also know that I worked at Avanade when they FULLY owned Enterprise Library (no MS patterns and practices didn't exist before Avanade created it) and had built the first TRUE SOA framework for .NET called ACA.NET. Through large projects like these, I have evolved the tool to generate the many different layers that one would ever want to have in a data-driven platform.

So back to SharePoint, what have I done with the tool and SharePoint? Being a top selling author of SharePoint courseware, I kinda know what people are doing day in and day out. Taking that information, I have automated those tasks in my code gen tool. Let me demonstrate!

Many of you have seen my previous blog post on BCS/BDC Report Card. The problem with BCS/BDC is that tables that have relationships don't work very well when WRITING data. So what are our options?

Custom Web Application in layouts directory
Custom Web Parts

The first option isn't very flexible and you end up having to implement you own security, so option #2 becomes a more preferred method when taking advantage of SharePoint's simple web part page setup and security standpoint.

Let's take the table example from my last post:

CREATE TABLE [dbo].[Contact](
   [ContactId] [int] NOT NULL,
   [FirstName] [varchar](50) NOT NULL,
   [LastName] [varchar](50) NOT NULL,
   [StatusId] [int] NOT NULL,
   [ModifyDate] [datetime] NOT NULL,
   [CreateDate] [datetime] NOT NULL
)

and the related table:

CREATE TABLE [dbo].[Status](
    [StatusId] [int] NOT NULL,
    [ShortName] [varchar](50) NOT NULL,
    [LongName] [varchar](50) NOT NULL,
    [ModifyDate] [datetime] NOT NULL,
    [CreateDate] [datetime] NOT NULL
)

For now, I'm not going to implement the relationship in the database and mimic where BCS functionality maxes out at. First step is to point the tool at the database. Note that I can support the following data sources:

For now, I'm simply using SQL Server. But using any of the above I could generate the same set of code base. Other things I need to do is set the output path for my generated code and the namespace of the code that will come out the other side. I will discuss some of the other things that happen automatically when a new configuration is created (which are important for SharePoint steps later on).

The next step is to generate the table and relationship configurations. It's a super advanced topic (likely the discussion of a Computer Science thesis paper in massively scalable system generation techniques) to go into what really is happening and all the possible overrides, just know that I have solved many of the typical code generation problems that typical tools can't overcome.

Notice that I too, like BDCMetaman, generate my own BDC app def files. For the lasts step I simply had to generate these (it did it automatically) so the actual core knows what type of code to generate. It would give me an error if it finds a table that doesn't have a configuration during runtime.

As with any good codegen tool, it has to generate your data access layer. So, let's do that, note all the options I can choose from:

Enterprise library is the MOST flexible (RUNTIME switching of the back end data source), so we'll use that. We have all the basic entered. Time to generate some code! First step is to generate the Base classes and class managers. With a single of a button, I will generate several millions of lines of code. And after the 5 year of building this tool, I realized that all the things I have built take a LONG time to generate. So I built a nice configuration selector:

So, selecting "Classes Only", I can generate the class manager and base classes. But also note that this step sets up the massive directory structure of possibilites for code that could have been generated:

Now simply generating the .cs files really doesn't help me much, so I needed the codegen tool to generate the Visual Studio project files too. So clicking the "All (Solution)" configuration option, I am able to generate these wrapper projects (only 2005 and 2008, haven't updated for 2010 until it RTMs), after clicking the upgrade in 2010, I have a non-compiling project will all files included in the project:

Why doesn't it compile? I didn't generate the Data Access layer. I have to generate the Stored procedures for the tables to support the class manager. Going back into the tool I change the config to Stored procedures, push a button and get a full set of SPs for multipe different platforms:

Executing the SP script I get a set of CRUD+4 SPs generated that is supported by my class manager (and every other architectural layer that I generate). Clicking the Database Interface button I'm able to generate the Database interface class that allows me to do all kinds of fancy things around transactional support. Now my project will compile.

Amazingly enough, all that is pretty basic for a codegen tool and I have kept this as simple as possible for the reader base. There are several other things that I didn't go into and won't go into (to keep my competitive advantage over my competition) that you'll never find in other tools.

At this point, your asking, "So, what about SharePoint?". Ok, let's do it. Now that I have the basics in place. Let's look at what gets generated for SharePoint. I don't have a configuration menu item for SharePoint so it's a manual setup in the configuration utility to enable all the codegen points, but luckily its only a couple of properties. Regenerating, I will get the following created in the SharePoint directory:

These are exactly what you think. Autogenerated eventreceivers for lists and web parts. By why two web parts? Webparts and webparts secure? WebParts are designed to be admin based web parts, no security around what you can do to the base data, whereas the secure ones, care about who YOU are and limit you to what you can see in the database.

In the SharePoint2007 folder, I get this:

< p>What is all the stuff? This is the yummy SharePoint part of the tool, these are all Features and do the following:

Content Types = a content type that matches exactly the attribute in the "data source"
GlobalList/GlobalListSecure = a list with a page that maps to each web part (eliminates the manual adding to lists, which sucks when you have 100's of tables)
Lists = a list that uses the content type to create a new instance of an exact copy of the data source. This is used to migrating back end data to SharePoint
WebParts/WebPartsSecure = a set of DWP files of all web parts generated to allow you to add to any pages
bdc.xml = a similar file you would get from BDCMetaman or the SharePoint SDK tool

So let's put it all together shall we? I compile all the necessary projects, add the assemblies to the GAC, add the safe controls and install and activate the the Global List feature. This is what I will get in SharePoint:

Clicking on the contact_Add.aspx page gives us the following:

Clicking Add gives us this:

What does that mean? It means by default the web part saved the data to the database. If I had so chosen, I could also have told it to save to SharePoint (using the list feature generated previously) or to XML. On success I could even have it redirect to some page:

Now, the web part itself has two ways to be generated. With everything in childcontrols or using a UserControl. Since most customers don't really like my ugly basic table (with no CSS elements for customization), they choose the user controls feature. So, with a couple of changes, I can tell it to skip any table building and load up a user control from the controltemplates directory. This then allows the customer to change the way the web parts looks however they want!

So, to quote a couple of movies, "whoop de do Basal, what does it all mean?". Back to the problem at hand, relational tables inside a database. The thing BCS can't handle currently. Let's check it out.

First thing to do is add the reference in the database. From there I then have to refresh a Relationship Property table in the tool. This basically say how I want things to be generated from a relationship standpoint (both incoming and outgoing). I simply set some configuration settings (one to start using the user control rather than internal child control magic) regenerate the code with a couple clicks of a button, redeploy my web parts and I now have this:

Notice that I have a drop down for my relationship. If I had set the contactid to an identity key, it would not be in this web part and therefore the user wouldn't have to type it in. Clicking "Add contact" will again add a contact to the database. Of course, being that the items in the database are set to "not null", if I had not filled in anything, I would get this:

To keep this short, all the web parts work. If security was needed, I would use the Secure version of the web parts and then make calls using the current user's identity. All generated from pretty much any data source you could imagine.

Even though it took a lot longer to build (and read) this blog, if I had skipped the blog, this all took about 10 minutes to setup and deploy (no matter how many entities was in the data source). Which by the way, Microsoft Entity Framework == Boring…there is so much missing from that framework and hence why I built mine years before MS did.

If I get lots of questions and request to see more, I might do another more in depth blog on some of the rules around the generation.

Chris

March 26, 2010

BCS/BDC Report Card

First and foremost, I LOVE SHAREPOINT. It's awesome, it pays my bills and IT ROCKS. I give credit where credit is due, but I'm also fair in that I will point out flaws too. As some of you surely remember, I tweeted a little rant about BCS and how annoyed I was that customer after customer keeps asking me to make BCS do something it just can't do (point proven by the tweet I got from Paul Andrew "BCS is not a silver bullet", of which I replied, "It's a marketing problem", driven by conference presenters saying its the save all solution of the century). A lot of people feel that because it can now both READ and WRITE data that you can DO ANYTHING with it. WRONG. So very wrong. In an effort to prove that point, very simply here's my BCS Report Card (2007 scores in parenthesis):

Read Data A+ (B)
Search Data B- (B-)
Write Data D (I)

Why you say? Why not A's across the board? Let me explain it too you. Let's start with a very simple table:

CREATE TABLE [dbo].[Contact](
    [FirstName] [varchar](50) NOT NULL,
    [LastName] [varchar](50) NOT NULL,
    [ModifyDate] [datetime] NOT NULL,
    [CreateDate] [datetime] NOT NULL
)

Using BCS, I can point to this data, read it, NOT search it and write it. Why can't I search it? I didn't give it a identifier/primary key. That means I can't point BCS at repeated non-unique data for searching and indexing, and hence why it gets a B-. Adding the following will make it searchable (and now usable in SharePoint Designer 2010):

ContactId int primary key

NOTE* it doesn't have to be a primary key, just need a column to be an identifier

Perfect. A few more clicks in Designer and I have an external content type with a list using it. Another couple of clicks in the Ribbon and I can even have the data syncing with my outlook (of which Fabian Williams first blogged an example of this). A few more clicks in my Search Service Application and its indexed. AWESOME! Let me repeat that…AWESOME! The problem comes into play when I add another table with a relationship:

–NEW TABLE–
StatusId int primary key
ShortName varchar(50)
LongName varchar(50)
ModifyDate datetime
CreateDate datetime

–ADD REF COLUMN TO CONTACT TABLE–
StatusId int

Regenerating my BCS/BDC application (which is not very friendly in Designer, but that's for another time) generates the same basic view of the contact data. However, notice that THE USER IS RESPONSIBLE for determining what the value of the StatusId column is…REALLY? REALLY? Of course if they type in something bad they get this:

Wait, what was that checkbox on the List generation? Generate InfoPath form?

That sounds promising, but:

The point is that BCS/BDC really only works with a single non-relational entity when it comes to WRITING of data. Even given the fact that with SharePoint Designer (or manually if you are apt too because you get finally get so frustrated that you want to throw your laptop when doing it with Designer) you can create "Associations" to build relationships between similar or disparate data sources (another awesome feature), it really only helps with the BDC web parts for sending of filtering values. What does this mean?

BDC ONLY WORKS WITH DE-NORMALIZED MEANINGFUL TABLES WHEN YOU WANT FULL* FUNCTIONALITY

The asterisk on "Full" simply means, the functionality offered by BCS (ref scorecard above), not what you would REALLY want.

What do I think would be ideal? The best options is to allow us to edit the InfoPath Form!!! Then we can map the control to a drop down populated from a data source! One would think that we should be able to edit the list column and point to corresponding column in the target association table to generate a selectable drop down (or even select the column in the BCS wizard or schema), but those are locked:

OR even better, when *generating* the BCS/BDC application definition, make it possible to create Metadata "Pointers" in the metadata service to these "distinct" items and have the column use the metadata service.

But even if they (SP Product Team) go to the extreme to implement all this later (which seems inevitable as soon as they see this), "distinct" items brings in a whole different set of problems like caching and large list like issues (being they have already addressed this pattern of problem in large lists, seems trivial for them).

Why did I asterisk out "generating". BCS is a code gen tool. It generates the BDC application definition file (and some other medium difficulty plumbing) for you with a fancy wizard. Let me add, its a very BASIC codegen tool. In the post that will follow this one, I will demonstrate for the first time my CodeGen tool (with its SharePoint extensions). Watch for that post!

Some of the comments I received when I tweeted my original rant where:

Fabian & Chakkaradeep – use a .NET type : this is why I give an A+ to READ, but still doesn't resolve the WRITE issues
Some said use BDCMetaman, sorry it doesn't solve these issues either, its just another codegen tool like BCS

In summary, to quote a really great movie "Choose, but choose wisely" your integration points with SharePoint. In the end you may fall over dead and old before you get it working (or realize it just won't work) with BCS (at least in this version #2).

Chris

March 25, 2010

MCT Virtual Summit 2010

I'm proud to announce I'll be presenting FOUR sessions at the MCT Virtual Summit this year! If your an MCT, you can become a Fan and find registration links on the MSL Facebook page: http://bit.ly/cxtXxt

Here's a summary of my sessions:

Train the Trainer(s) – 10174A: Administering and Managing SharePoint 2010 (TS) – together with my co-author and SharePoint MVP, Dan Holme, we will be outlining the modules of this course and how we envisioned it to be delivered. We'll answer questions about the course and possibly (time permitting) do some demos from some of the labs.
Courseware library success story – Join me as I share with you my latest chapters in my courseware library adventures and where it has led me!
SharePoint 2010 MOC and CWLibrary course overview – I'll review all the upcoming MSL SharePoint 2010 MOC courses, certifications and the currently available SharePoint 2010 courses in the CWLibrary.
SharePoint 2010 BI – I'll review and demo many of the new features of SharePoint 2010 Business Intelligence!

Look forward to seeing all of you wonderful MCTs there!
Chris

March 18, 2010

SharePoint 2010 Claims based Forms Authentication

Forms based authentication in SharePoint 2010 has changed. It is now Claims based Forms Authentication, which means all the forms configuration stuff you see all over the web doesn't quite work in the same way. Even if you set the membership and role providers in your web
applications and central administration you forms auth still won't work!

In SharePoint 2010, several new WCF services will be handleing the mapping of the claims to the backend forms auth identities. Out of the box, claims based forms auth doesn't work! You gotta remove the <clear/> element from the Membership and Role provider sections in the web.config of the {SharePoint Root}WebServicesRoot.

Chris

March 17, 2010

WCF Timeouts on small memory’d SharePoint 2010 machine

In writing labs for an upcoming Microsoft Course, I found that the limitations on my environment was less than satisfactory to run all of the services of SharePoint 2010 at the same time. Even though all Web Application and app pools were running, I was getting WCF timeouts mainly for the User Profile Service. The default timeout value in SharePoint for the services is pretty much set to 20 sec. If you have a machine that doesn't have the full 8GB of memory needed, then you may find your services are taking a while to spin up. This can be devistating to some of the setup/install things you may be doing (because there are still many things that are NOT transnational).

I found that by increasing the WCF timeout, I could avoid a lot of setup/install problems (mainly around service applications like User Profile). To change the WCF timeout, open the C:Program FilesCommon FilesMicrosoft SharedWeb Server Extensions14WebClientsProfileclient.config file, update the Bindings to double the timeout from "00:00:20" to "00:00:40". Bam! Your user profile won't timeout all the time and the service management page will display!

Chris