Migrating User Forms Based Permissions in 2010 Upgrade

So not everything gets upgraded in the RTM build of 2010.  Seems the usernames and permissions end up getting lost when you upgrade.  There are several things you have to do to get your upgraded forms based authentication to work:

First error you will get is "Forms Based Authentication on classic Web applications has been deprecated."  this is solvable by opening SharePoint 2010 Management Shell to upgrade the web application to use claims based authentication

$w = Get-SPWebApplication "http://webappurl/"
$w.UseClaimsAuthentication = $true;
$w.Update()
$w.ProvisionGlobally()

Second thing is to enable your forms based auth in the SecurityToken Service as in my other blog post

Next you need to ensure that your settings are valid in the Framework64 directory if you are using the machine.config to propagate your sql server settings for Aspnetsqlmembership.  This changed as SharePoint is now 64bit, and not 32bit.

Once this is done, you will NOT be able to login!  Why?  Because the internal naming of the user you have setup as the site collection administrator was tied to the old aspnetsqlmembershipprovider:username naming scheme in the databases.  The new naming convention is  i:0#.f|aspnetsqlmembershipprovider|username

How do you update this?  well, other than going into the database and updating it directly, you really have only one other choice, remove and add each user!  If you have 100's of users, if not 1000's, well your in for a long day!  Luckily, I wrote a nice little powershell script that goes in and adds/removes all the users.  Here it is (as with anything, use at your own discretion):

$spweb = get-spweb "webappurl"
foreach($spgroup in $spweb.groups)
{
write-host $spgroup.name

foreach($spuser in $spgroup.users)
{
write-host "Migrating: " $spuser.name
$newuser = $spweb.ensureuser($spuser.name)
write-host "SPUser=" $spuser
write-host "NewUser=" $newuser
write-host "Adding user:" $newuser.name " to group: " $spgroup.name
$spgroup.adduser($newuser)
if ( $newuser.userlogin -ne $spuser.userlogin)
{
write-host "Removing old user:" $spuser.name " from group: " $spgroup.name
$spgroup.removeuser($spuser)
}
}
}

SharePoint MVP MSDN Rankings

I have had quite a few people ask me how I am able to say when I'm passing someone in the forums.  It's simple really, I built a nice parser that pulls down all the SharePoint MVPs with their stats.  With the latest MVP announcements (New and re-news – congrats to everyone by the way), I'd figure it would be good to promote a sense of community and a challenge to those MVPs that may not be pulling their full weight in the forums community as well as encourage some of you non-MVPs to set some targets for how much you want to participate.  Let me be clear though, forum participation is not the only thing that drives an MVP award.  There are several people on this list that I know personally that are awesome and deserve their MVP even without forum participation.  Things that factor in include:

  • The company you work for
  • Your customers (Fortune 100)
  • Your project references
  • Your code and tool contributions (Codeplex)
  • Products you have invented and market
  • Your community participation (Books, courses, forums, blogs, etc)
  • Your involvement with the product team

So, where does your favorite SharePoint MVP stack up in the forums, you will probably be surprised!?! Check out this list (let me know if I missed anyone or if someone has a separate id that I may have missed).  I really do consider the top 10 the best of the best, anyone that participates that much deserves an MVP simply for their dedication and helpfulness.  FYI, I'm at (1624 | 336 | 083) so I have a way to go.  This is updated as of 3/28/2010.

Participate in the Forum Jam 2010 : http://bit.ly/9PQSRJ

Name | Points | Posts | Answers

MVPs

1) Mike Walsh      | 48357 | 15438 | 1657
2) Paul Galvin      | 15895 | 3304 | 890
3) Fabrice Romelard| 9585 | 1264| 683
4) Mike Oryszak    | 8139 | 1326| 500
5) Moonis Tahir    | 7054 | 1186| 468
6) John D. Ross    | 6919 | 1495| 348
7) Waldek Mastykarz| 6437 | 1110| 396
8) Gary Lapointe   | 4972 | 807 | 318
9) Ishai Sagi      | 3830 | 596 | 255
10) Randy Drisgill  | 3382 | 623 | 206

Michael Nemtsev | 3324 | 717 | 198
John Timney     | 3091 | 436 | 207
Wictor Wilen    | 2683 | 428 | 164
Ayman El-Hattab | 2308 | 268 | 113
Becky Bertram   | 2061 | 273 | 134
MatthewMcDermott| 2043 | 619 | 089
Andrew Woodward | 1570 | 386 | 085
Mirjam Van Olst | 1504 | 234 | 089
Scot Hillier    | 1299 | 212 | 082
Stephane Eyskens| 1252 | 227 | 074
Karine Bosch    | 1228 | 222 | 074
Jan Tielens     | 1198 | 197 | 080
Paul Stork      | 0977 | 147 | 062
James Milne     | 0774 | 169 | 042
Philippe Sentaen| 0994 | 184 | 058
Andrew Connell  | 0731 | 158 | 043
Eric Shupps     | 0721 | 113 | 048
Renaud Comte    | 0656 | 170 | 032
Liam Cleary     | 0604 | 073 | 040
Asif Rehmani    | 0538 | 084 | 031
Nick Swan       | 0484 | 131 | 022
Spencer Harbar  | 0450 | 062 | 028
Nicolas Georgeau| 0422 | 121 | 016
Juan Valenzuela | 0369 | 047 | 022
Marwan Tarek    | 0368 | 049 | 023
Steven Van Craen| 0330 | 090 | 015
Vivekthangaswamy| 0306 | 093 | 020
Paul Schaeflein | 0318 | 053 | 022
Jerry Yasir     | 0284 | 052 | 016
Darrin Bishop   | 0282 | 072 | 016
Wes Preston     | 0268 | 045 | 015
Ivan Wilson     | 0246 | 105 | 011
Mohanad Omar    | 0233 | 034 | 016
John Holliday   | 0226 | 073 | 012
Zac Smith       | 0226 | 036 | 014
Ben Robb        | 0219 | 032 | 012
Pierre Vivier-Me| 0216 | 055 | 011
Panagiotis Kanav| 0214 | 047 | 011
Todd Klindt     | 0210 | 042 | 012
GabrieleDelGiovi| 0185 | 037 | 010
Sahil Malik     | 0178 | 054 | 008
Michael Greth   | 0147 | 036 | 008
Agnes Molnar    | 0134 | 034 | 012
Romeo Pruno     | 0134 | 024 | 009
Bryan Phillips  | 0129 | 023 | 007
Robin Meure     | 0129 | 037 | 005
Steve Sofian    | 0126 | 028 | 006
Didier Danse    | 0124 | 062 | 003
Thiago Soares   | 0114 | 041 | 008
Walter Van Vugt | 0108 | 041 | 010
Ed Musters      | 0102 | 021 | 005
Ton Stegeman    | 0092 | 014 | 007
Pierre Erol Gira| 0086 | 027 | 004
Chandima Kulathi| 0086 | 023 | 003
Kevin Laahs     | 0076 | 028 | 002
Kathy Hughes    | 0073 | 015 | 004
Benjamin Curry  | 0071 | 013 | 002
BrendonSchwartz | 0070 | 010 | 005
Fumio Mizobata  | 0065 | 010 | 004
Sampathperera   | 0062 | 011 | 004
Ricardo Munoz   | 0061 | 013 | 002
Kanwal khipple  | 0050 | 015 | 002
Dan Holme       | 0039 | 009 | 002
Sebastian Wilcze| 0037 | 001 | 001
Bil Simser      | 0034 | 014 | 001
Daniel Webster  | 0032 | 024 | 000
Carsten Keutmann| 0032 | 006 | 001
Reza Alirezaei  | 0030 | 005 | 002
Shane Perran    | 0028 | 004 | 002
Igor Macori     | 0028 | 007 | 002
Brian Farnhill  | 0016 | 003 | 001
Aleksandr Chervy| 0013 | 013 | 000
Christoph Muller| 0012 | 026 | 003
Eli Robillard   | 0010 | 008 | 000
Rehman Gul      | 0006 | 003 | 000
Amanda Murphy   | 0006 | 003 | 000
Sarbjit Singh Gi| 0006 | 008 | 000
Matt Ranlett    | 0004 | 003 | 000
Penny Coventry  | 0002 | 001 | 000
Loke Kit Kai    | 0002 | 006 | 000
Danial Larson   | 0002 | 002 | 000
Todd Bleeker    | 0002 | 001 | 000
Rob Foster      | 0002 | 001 | 000
Stacy Draper    | 0002 | 001 | 000
Arno Nel        | 0000 | 008 | 000
Joy Rathnayake  | 0000 | 006 | 000
Shane Young     | 0000 | 002 | 000
Juan Herrera    | 0000 | 001 | 000
Bill English    | 0000 | 000 | 000
Box Fox            | 0000 | 001 | 000
Ted Pattison    | 0000 | 000 | 000
Orin Thomas     | 0000 | 000 | 000
Mohamed Zaki    | 0000 | 000 | 000
Adam Buenz      | 0000 | 000 | 000
Bob Mixon       | 0000 | 000 | 000
Robert Bogue    | 0000 | 000 | 000
Colin Spence    | 0000 | 000 | 000
Todd Baginski   | 0000 | 000 | 000
Daniel Wessels  | 0000 | 000 | 000
Daniel Seara    | 0000 | 000 | 000
Stephen Cummins | 0000 | 000 | 000
Christian Hougar| 0000 | 000 | 000
Carlos Sanz     | 0000 | 001 | 000
Mohamed Zaki    | 0000 | 000 | 000
Shady Khorshed  | 0000 | 000 | 000
Bill Brockbank  | 0000 | 000 | 000
Alex Pearce     | 0000 | 000 | 000
Serge Tremblay  | 0000 | 000 | 000
Valy Greavu     | 0000 | 000 | 000
Alexander Romano| 0000 | 000 | 000

SharePoint Tool Ideas For YOU to Build!

Looking for the next thing to build for SharePoint?  Here's a quick list from notes over the years:

  • Claims filter web part (2010)
    • This would present a set of registered claims to be used in BI solutions (Excel Services, Report Services, Performance Point, etc)
  • Velocity Sandbox Solution disable (2010) – as suggested by @sahilmalik
    • When in an Sandbox environment, if one Sandbox Solution goes crazy based on X rate, you deactivate it
  • Generic Sandbox Solution Validator – I BUILT THIS ONE ALREADY!
  • Timer Job Disabler – I BUILT THIS ONE ALREADY!
  • A feature that inserts a status message when the Site has not been used or is about to be deleted (2010)
  • Column Updater for field names
    • This tool would update all column names for a specific field type
      across an entire farm in the event the name of the field changes.
  • Custom event handler to call IFilters and then parse files for bad words (similar to ForeFront feature)
  • Create email alias in active directory/exchange from the email settings page (if site collection admin)
  • Copy file permissions to a document library from NTFS
  • HttpModule to intercept and encrypt documents on "View"
  • Wiki linking based on different word other than page name
  • Extend web application policy permissions (override site collection level permissons)
  • Find and replace web part instances and migrate audience settings tool
  • Solution query tool (show what features are part of the solution wsp on central admin)
  • Page to add Multiple columns at once for content types (especially page layouts)

Ok, you guys and gals get to work!
Chris

SharePoint 2010 Password Change Policy

SharePoint 2010 has a great new feature for setting your service account passwords for when your AD team has locked down the password policies.  In Central Administration you can click Security->Manage service accounts links, and after selecting the account, you can then set the passwords for these accounts from central administration.  You may get an Access Denied error via the Microsoft.SharePoint.Win32.SPNetApi32.NetUserChangePassword method.   This is very simply calling the C++ Netapi32.dll to change the password.  Reference this for more information:

http://msdn.microsoft.com/en-us/library/aa370650%28VS.85%29.aspx

In doing more research, any accounts that have "Do not allow user to change password" will fail to update the password.  The actual changing of the password is done as if it is the user changing it.  You should also watch out for the "Accounts can only change once per day setting", this would keep you from changing your password more than once in a single day.

Chris

SharePoint 2010 install accounts

This has been throwing everyone for a crazy loop and its time to clear up what's going on.  So far most of my environments have been setup with accounts called SP_Admin, SP_Farm and SP_Service.  the SP_Admin is the install account, SP_Farm is the Database access account (weird naming right?), and the service applications get SP_Service.

I encourage you to do this and see what happens on the other side of the install and Farm configuration wizard.  You'll find out real quick what type of things you need to do to get SharePoint 2010 working using this best practice approach of service accounts.  But, rather than just leave you hanging, one big help for you in your quest for least privileged SharePoint Installs is:

SP_Admin and SP_Farm must be Local Administrators on the SharePoint server. 

It ends up that that "Database access account" really turns out to be the main Farm account.  When you get to user profile synchronization, if this account is not a local admin, the setup of the ForeFront Identity Manager will fail miserably and you will need to kill off the User Profile Synch services (using powershell) and start all over after adding in the local admin rights.  I'm not sure if that particular screen will get updated in RTM or not, but it would make a hell of a lot more sense if it had "Farm Account" rather than "Database Access Account" on the setup screen:

 

Another note, when you go to add a new server to the farm (unless you do some SQL permission setting), you will have to be logged in as SP_Farm, not SP_admin.

Chris

Becoming the Fastest SharePoint Developer in the World!

Who is it?  I'm not sure, but I'm somewhere up there in the top 1% and that lets me charge $200/hr for my time.  So why do I say I'm in the top 1%?  Because I have this:

 

What is it you ask?  It's MY code gen tool.  It is the culmination of 10 years of development projects combined into a massive code generation platform.  Some of you know I am one of the architects behind the Quiznos, Boston Market, Panda Express…etc, etc, etc gift and loyalty platform.  It processes millions of transactions every day and has some very cool architectural components very similar to Visa and Discover networks (but a lot better).  Some of you might also know that I worked at Avanade when they FULLY owned Enterprise Library (no MS patterns and practices didn't exist before Avanade created it) and had built the first TRUE SOA framework for .NET called ACA.NET. Through large projects like these, I have evolved the tool to generate the many different layers that one would ever want to have in a data-driven platform.

So back to SharePoint, what have I done with the tool and SharePoint?  Being a top selling author of SharePoint courseware, I kinda know what people are doing day in and day out.  Taking that information, I have automated those tasks in my code gen tool.  Let me demonstrate!

Many of you have seen my previous blog post on BCS/BDC Report Card.  The problem with BCS/BDC is that tables that have relationships don't work very well when WRITING data.  So what are our options?

  • Custom Web Application in layouts directory
  • Custom Web Parts 

The first option isn't very flexible and you end up having to implement you own security, so option #2 becomes a more preferred method when taking advantage of SharePoint's simple web part page setup and security standpoint.

Let's take the table example from my last post:

CREATE TABLE [dbo].[Contact](
    [ContactId] [int] NOT NULL,
    [FirstName] [varchar](50) NOT NULL,
    [LastName] [varchar](50) NOT NULL,
    [StatusId] [int] NOT NULL,
    [ModifyDate] [datetime] NOT NULL,
    [CreateDate] [datetime] NOT NULL
)

and the related table:

CREATE TABLE [dbo].[Status](
    [StatusId] [int] NOT NULL,
    [ShortName] [varchar](50) NOT NULL,
    [LongName] [varchar](50) NOT NULL,
    [ModifyDate] [datetime] NOT NULL,
    [CreateDate] [datetime] NOT NULL
)

For now, I'm not going to implement the relationship in the database and mimic where BCS functionality maxes out at.  First step is to point the tool at the database.  Note that I can support the following data sources:

 For now, I'm simply using SQL Server.  But using any of the above I could generate the same set of code base.  Other things I need to do is set the output path for my generated code and the namespace of the code that will come out the other side.  I will discuss some of the other things that happen automatically when a new configuration is created (which are important for SharePoint steps later on).

The next step is to generate the table and relationship configurations.  It's a super advanced topic (likely the discussion of a Computer Science thesis paper in massively scalable system generation techniques) to go into what really is happening and all the possible overrides, just know that I have solved many of the typical code generation problems that typical tools can't overcome.

Notice that I too, like BDCMetaman, generate my own BDC app def files. For the lasts step I simply had to generate these (it did it automatically) so the actual core knows what type of code to generate.  It would give me an error if it finds a table that doesn't have a configuration during runtime.

As with any good codegen tool, it has to generate your data access layer.  So, let's do that, note all the options I can choose from:

 

Enterprise library is the MOST flexible (RUNTIME switching of the back end data source), so we'll use that.  We have all the basic entered. Time to generate some code!  First step is to generate the Base classes and class managers.  With a single of a button, I will generate several millions of lines of code.  And after the 5 year of building this tool, I realized that all the things I have built take a LONG time to generate.  So I built a nice configuration selector:

So, selecting "Classes Only", I can generate the class manager and base classes.  But also note that this step sets up the massive directory structure of possibilites for code that could have been generated:

 

Now simply generating the .cs files really doesn't help me much, so I needed the codegen tool to generate the Visual Studio project files too.  So clicking the "All (Solution)" configuration option, I am able to generate these wrapper projects (only 2005 and 2008, haven't updated for 2010 until it RTMs), after clicking the upgrade in 2010, I have a non-compiling project will all files included in the project:

 

Why doesn't it compile?  I didn't generate the Data Access layer.  I have to generate the Stored procedures for the tables to support the class manager.  Going back into the tool I change the config to Stored procedures, push a button and get a full set of SPs for multipe different platforms:

Executing the SP script I get a set of CRUD+4 SPs generated that is supported by my class manager (and every other architectural layer that I generate).  Clicking the Database Interface button I'm able to generate the Database interface class that allows me to do all kinds of fancy things around transactional support.  Now my project will compile.

Amazingly enough, all that is pretty basic for a codegen tool and I have kept this as simple as possible for the reader base.  There are several other things that I didn't go into and won't go into (to keep my competitive advantage over my competition) that you'll never find in other tools. 

At this point, your asking, "So, what about SharePoint?".  Ok, let's do it.  Now that I have the basics in place.  Let's look at what gets generated for SharePoint.  I don't have a configuration menu item for SharePoint so it's a manual setup in the configuration utility to enable all the codegen points, but luckily its only a couple of properties. Regenerating, I will get the following created in the SharePoint directory:

 

These are exactly what you think.  Autogenerated eventreceivers for lists and web parts.  By why two web parts?  Webparts and webparts secure?  WebParts are designed to be admin based web parts, no security around what you can do to the base data, whereas the secure ones, care about who YOU are and limit you to what you can see in the database.

In the SharePoint2007 folder, I get this:

 

What is all the stuff?  This is the yummy SharePoint part of the tool, these are all Features and do the following:

  • Content Types = a content type that matches exactly the attribute in the  "data source"
  • GlobalList/GlobalListSecure = a list with a page that maps to each web part (eliminates the manual adding to lists, which sucks when you have 100's of tables)
  • Lists = a list that uses the content type to create a new instance of an exact copy of the data source.  This is used to migrating back end data to SharePoint
  • WebParts/WebPartsSecure = a set of DWP files of all web parts generated to allow you to add to any pages
  • bdc.xml = a similar file you would get from BDCMetaman or the SharePoint SDK tool

So let's put it all together shall we?  I compile all the necessary projects, add the assemblies to the GAC, add the safe controls and install and activate the the Global List feature.  This is what I will get in SharePoint:

 

Clicking on the contact_Add.aspx page gives us the following:

Clicking Add gives us this:

 

What does that mean?  It means by default the web part saved the data to the database.  If I had so chosen, I could also have told it to save to SharePoint (using the list feature generated previously) or to XML.  On success I could even have it redirect to some page:

 Now, the web part itself has two ways to be generated.  With everything in childcontrols or using a UserControl.  Since most customers don't really like my ugly basic table (with no CSS elements for customization), they choose the user controls feature.  So, with a couple of changes, I can tell it to skip any table building and load up a user control from the controltemplates directory.  This then allows the customer to change the way the web parts looks however they want!

So, to quote a couple of movies, "whoop de do Basal, what does it all mean?".  Back to the problem at hand, relational tables inside a database.  The thing BCS can't handle currently.  Let's check it out.

First thing to do is add the reference in the database.  From there I then have to refresh a Relationship Property table in the tool.  This basically say how I want things to be generated from a relationship standpoint (both incoming and outgoing).  I simply set some configuration settings (one to start using the user control rather than internal child control magic) regenerate the code with a couple clicks of a button, redeploy my web parts and I now have this:

 

Notice that I have a drop down for my relationship.  If I had set the contactid to an identity key, it would not be in this web part and therefore the user wouldn't have to type it in.  Clicking "Add contact" will again add a contact to the database.  Of course, being that the items in the database are set to "not null", if I had not filled in anything, I would get this:

 

To keep this short, all the web parts work.  If security was needed, I would use the Secure version of the web parts and then make calls using the current user's identity. All generated from pretty much any data source you could imagine.

Even though it took a lot longer to build (and read) this blog, if I had skipped the blog, this all took about 10 minutes to setup and deploy (no matter how many entities was in the data source). Which by the way, Microsoft Entity Framework == Boring…there is so much missing from that framework and hence why I built mine years before MS did.

If I get lots of questions and request to see more, I might do another more in depth blog on some of the rules around the generation.

Chris

MCT Virtual Summit 2010

I'm proud to announce I'll be presenting FOUR sessions at the MCT Virtual Summit this year!  If your an MCT, you can become a Fan and find registration links on the MSL Facebook page:  http://bit.ly/cxtXxt

Here's a summary of my sessions:

  1. Train the Trainer(s) – 10174A: Administering and Managing SharePoint 2010 (TS) – together with my co-author and SharePoint MVP, Dan Holme, we will be outlining the modules of this course and how we envisioned it to be delivered.  We'll answer questions about the course and possibly (time permitting) do some demos from some of the labs.
  2. Courseware library success story – Join me as I share with you my latest chapters in my courseware library adventures and where it has led me!
  3. SharePoint 2010 MOC and CWLibrary course overview – I'll review all the upcoming MSL SharePoint 2010 MOC courses, certifications and the currently available SharePoint 2010 courses in the CWLibrary.
  4. SharePoint 2010 BI – I'll review and demo many of the new features of SharePoint 2010 Business Intelligence!

Look forward to seeing all of you wonderful MCTs there!
Chris

SharePoint Timer Jobs

Have I mentioned how much I don't like Timer Jobs?  Let me rephrase that, timer jobs that other people write that aren't designed correctly?  Yeah, irks me.  So we can't say that all the Timer Jobs in the Beta are perfect, some have some issues, but its beta and we should come to expect not everything is going to work right now. 

For those that really cause us trouble though, how do we debug them?  We can attach to the owstimer.exe service I suppose, but then you are causing more problems with the other jobs waiting to get some of the precious CPU cycles while your debugging.

So how about we try a different approach?  I like using SQL Profiler, it is one of my favorite tools. It comes in really handy to figure out what really is going on on a SharePoint page (the Developer Dashboard is nice, but don't let it fool you, there is more going on than what it tells you) and…Timer Jobs.  problem is, sql profiler requires you to get really specific about what you want to see.  Not much you can do when everything coming in is tagged as a ".NET Application"…except for…turn everything OFF!

Turn off all the services, turn off all the timer jobs you don't want to look at or care about.  Turning off services is easy enough to do…timer jobs…ugg…little more difficult.  What about searching for a timer job?  Yeah, good luck with that one with just Out of the Box.

So I built a neat little tool and posted it to codeplex.  It lets you turn all the timer jobs off, now two of them cannot be turned off, but they don't end up causing much traffic, so no worries there.  With this tool you can search for timer jobs by Name, Title, ID, Schedule and Lock Type.  You can update, updateable properties and click "Update".  You can disable, enable and "Run Now" with just a click of a button!

Sweet!  Download the source code here:  http://bit.ly/4ydtNr

I plan on doing some heavy reverse engineering on some of these bad boys…I encourage you to do the same!

Chris