O365 Search – Deleting Tenant Level Result Types

I have a really big client on the east coast.  We are automating the deployment of many of the search components in O365, very cool stuff.  However, not everything is hugs and kisses.  Turns out when you import a search configuration at the tenant level, you may import some Result Types.  Unfortunately, there is no link on the main Search page for the tenant admin that will let you delete these result types.

That does not mean it doesn't exist.  It does…you just have to guess where it is:

https://{tenant}-admin.sharepoint.com/_layouts/15/manageresulttypes.aspx?level=tenant

Bam…you can now delete any result types you imported… weird that the link is not available on the main search admin page.

Enjoy,
Chris

Transitioning from O365 Preview to O365 V1

Back in April I started writing a course for developing on Office 365.  Mainly focused on the O365 APIs (Mail, Calendar, Contacts and Files).  As part of building the course, I was able to provide quite a bit of feedback to engineering (check out my uservoice and the items that have since been implemented) on the various gaps that I found as compared to other similar platforms (both Microsoft and non-Microsoft).  The list was incredibly long and I wasn't particularly convinced that all of those items would be engineered anytime soon.  However as I was redoing my course, to my pleasant surprise, many of those suggestions had been implemented in a very short time!

Tools, Tools and more Tools

In the preview, you have to install two different sets of tools.  Those were rolled into a single installer package that points to a Web Installer that will always grab the latest for you.  That particular package as of 11/2014 is in Update 3 status.  You can download the tool here (forgive the fact that the .exe may be named "OfficeToolsForVS2013Update1.exe", again it is a web installer package and grabs the latest).  You can also follow Chakkaradeep's blog for all the latest on the tools as they continue to evolve (http://chakkaradeep.com/):

http://aka.ms/officedevtoolsforvs2013

Note that if you try to "Add Connected" service and end up with a set of *ApiSample files added, then you still have the old tools installed (this happened to me as I have many different Visual Studio clients on various machines).  This of course will cause problems if you try to open the new *working* samples mentioned below.  You will need to uninstall those old tools and add the new tools via the link referenced above.  This sample code is now part of the tools via "Client" classes that you can now choose to use or you can simply create your own.

NOTE:  In some cases you may find you have some older MVC project templates, if you create a new one and switch it to use the "Organizational Accounts" for its auth, it will add a "UserProfile" action that has older ADAL code in it (this has nothing to do with the O365 tools that you have installed and is part of the ASP.NET Identity team's code).  Any time you try to load the newer samples, the ADAL package will get upgraded using NuGet and the UserProfile action will not compile.  It is best just to delete it or update your Visual Studio to have the latest templates and recreate the project.

Samples, Samples and more Samples

The original page for all the O365 API preview stuff has been deprecated (http://msdn.microsoft.com/en-us/library/office/discovery-service-rest-operations%28v=office.15%29.aspx) and it now lives here:

http://msdn.microsoft.com/en-us/office/office365/howto/platform-development-overview

You can get O365 API samples here:

http://dev.office.com/code-samples

And they are copied directly to GitHub here:

https://github.com/OfficeDev

This is in contrast to the ones you will find on MSDN that are probably not up-to-date and do not compile with the new tools (think – the old Authenticator class or broken auth flows):

 https://code.msdn.microsoft.com/windowsapps/Office-365-APIs-How-to-use-609102ea

The core Azure AD samples (nothing to do with O365) that you should be using are here:

https://github.com/AzureADSamples

and a description of each is here:

http://msdn.microsoft.com/en-us/library/azure/dn646737.aspx

Note that the sample code on these pages use some hidden tricks for the Auth.  Notably, it is using OWIN on MVC app startup that create and initalize the cache in the AuthenticationContext class (NOTE, this code was developed by Andrew Connell and is pretty freakin awesome, be sure to thank him next time you see him as I can only imagine the time he put into this).  This context is then used to request an access token for you using the refresh token from the first "hidden" OWIN request.   If you are creating an application from scratch and not using these samples, you will not have this OWIN code and therefore are missing the first leg of the auth.  Even though the samples compile and Azure AD gives you back a token, you will not be able to use that token!  You must have a user context in order to be able to make calls to the O365 API endpoints. Simply sending the clientId and clientSecret will not establish a user context (although the platform will give you a useless/stupid access token). This is because Azure AD is used for access control with external custom applications (by extending the consent framework) that have nothing to do with O365 and may not even need such an advanced user context created and thusly, it is simply used for OAuth (reference the Azure ToDoService example).

Compiling samples (hints and tricks):

The samples utilize several different packages.  These include the ones on this page (http://msdn.microsoft.com/en-us/office/office365/howto/adding-service-to-your-Visual-Studio-project#O365NuGets), and a couple of extras:

  • Azure Active Directory Auth Library (ADAL) – note current stable is 2.12, with 3.x in pre-release.  Some of the samples you may come across are using code in previous and later releases.  I have found it best to just update all the way, then figure out how to get the older code to work (which in some cases will take a really long time).  As an example, you may see that the Sync and Async methods for getting an access token may or may not exist and the various overrides for them may not be present.  It is very painful trying to figure out what changes were made from v1->v2->v3 as the code has changed so much.  This has nothing to do with the O365 API/SDK aspect (they have to deal with it just like we do), and more on the Azure tools team's side.

You should be aware that if you try to load a sample and it has any reference to the "Authenticator" class, that sample is now out of date and has a net worth of just about zero.  

Using the discovery service

  • When asking for permission for the discovery service, be sure to use "https://api.office.com/discovery/" and not "https://api.office.com/discovery" – note the extra slash at the end
  • You should also ensure that your API calls go to "https://api.office.com/discovery/v1.0/me" and not "https://api.office.com/discovery/me" – note the version in the URL – some of the O365 API page examples have not been updated to show this
  • You should definitely use the discovery service now to get your service endpoints.  The reason to do this is you may not know the O365 endpoint for a user's SharePoint ahead of time and the now defunc resource string "Microsoft.SharePoint" does not work anymore.  You must specify the authority/resource such as https://domain.sharepoint.com or https://domain-my.sharepoint.com, otherwise your request will fail (but will still work with the old preview APIs).

Note that all APIs are now returning ODATA v4.0 responses for JSON (no more result.d.results, it is result.value):

http://www.odata.org/documentation/odata-version-4-0/

Various things:

  • You should also note that your "Accept" HTTP header should be set to "application/json" and not "application/json; odata=verbose" or you will get an error. 
  • The structure of the JSON posts have changed.  You will notice the addition of the "EmailAddress" parent for email addresses now
  • Mail endpoint changed from "/Messages/('id')" to "/Messages/{MessageId}"
  • For new events, you cannot pass the "Status" the way you did before, it has changed its schema as well
  • JSON Error messages are now results.error.message rather than results.error.innermessage.message
  • Other than the endpoint for the O365 APIs, nothing really changed much in the "Contacts" api, your code should still work after making the endpoint change
  • Files API now has the "me" endpoint!  This was one of the big asks that I had when I first looked at the O356 APIs in respect to the OneDrive for Consumer API offering (ref this blog post).  The consumer team has moved their API to the OneDrive for business and it is just plan AWESOME!
  • Files API no longer requires SharePoint like RequestDigests in the headers!
  • Uploading a file is done via a PUT, not a POST (you still have to put the binaryStringRequestBody header in the request
  • The older way of telling the API that you want to overwrite a file has changed to have the parameter "nameConflict=overwrite"
  • .JSON files are blocked by SharePoint, so don't try with the API or you get a 404 error
  • When deleting a file, you must provide the "if-match" header with "*" or a matching tag, or the call will fail (an empty header will fail)
  • I'd guess that the "/me/drive" endpoint will go away in V2 as we will have unlimited storage soon

Mobile and Cordova

  • Samples on the O365 Dev site are out of date with the new tools (https://github.com/OfficeDev/TrainingContent/tree/master/O3654-2%20Deep%20dive%20into%20Mobile%20Development%20with%20Office%20365%20and%20Cordova)
  • As of 11/2014, the latest tool version is 0.3.  It can be downloaded here link
  • The location of the service scripts files has changed to be /services/office365/scripts
  • The settings.js does not have the references to all needed files and it has been moved to another file called "o365loader.js".  You should add a reference to this on your html pages.
  • You will need internet access to compile your first project (npm downloads and such)
  • The Exchange namespace is gone, it is now Microsoft.OutlookServices
  • If you get an error in your Ripple Emulator, I found that if you had a previous version, it doesn't work with 0.3 – ref this post.
    • Try running "npm install -g ripple-emulator" to get the latest
  • It is also possible you will get a "windows.open" error like "gap:["App","show","App590841629"]".  This simply means you have a loading error in one of you JS files, fix it and the window.open event will be handled properly
  • The Ripple Emulator is set to use a dynamic port when your application starts.  This means the default port of 4400 that the VS code is expecting will not work.  You will need to change the Settings.js file to have the proper port and then update your Azure AD to have the matching port in the redirectUri
  • In the Ripple Emulater, be sure to change the cross domain from "Local" to "Remote" or you will get errors when making O365 API calls
  • There are some other issues with the current build (notably many related to Windows Phone development) – ref this post
  • If you get an error around the inappbrowser plugin, you have to stop debugging and then try try again…eventually it should come around (you may find you have to do this in several occasions as it seems there may be some race conditions occurring somewhere)
  • The official Android samples from the O365 team are here. They use Eclipse and are much easier to get up and running with as they have setup instructions.
  • MS Open Tech Group Android samples have been updated to the new V1 API and are now using Android Studio rather than Eclipse, but you are on your own to get them up and running.

Now, once your environment is up and running, you may run into some interesting challenges. 

The First is with the consent framework.  If you ever give your application the "Have full access to users' mailbox", your mail token will be worthless with "Access Denied".  Simply remove that from your app permission set and viola, your requests will work again.  This seems to be a bug in V1.0

Secondly is that you cannot simply call the code on this page without the first leg of the auth (getting an access code first):

http://msdn.microsoft.com/en-us/office/office365/howto/common-file-tasks-client-library

Third, you cannot use a service token generated from the "common" authority.  It must be from "your" tenant authority…ie…

https://login.windows.net/Common vs https://login.windows.net/tenantID

If you do this, the request will fail.  NOTE:  Andrew C. mentioned that this should work, but I think their are some outlier cases that this does not work (such as what I am seeing in some of my course labs).

Fourth, a token is only good for the resource that you requested. It cannot be used for any other resources and there is no way (unfortunately), to put in more than one resource (would be nice to do "https://graph.windows.net;https://acs579.sharepoint.com" with the scopes union-ed). This directly leads to the fifth point…

Fifth, Azure AD cannot handle double hops very well.  Consider the following situation (User->Client->WebAPI->O365) described as:

  • Create a WebAPI project with a web app on the Azure side (it is created
    with custom permissions and given O365 SharePoint permissions).
  • Add the Azure auth code into the OWIN part to validate a bearer token from client apps
  • Add some method calls to make a call to O365 apis
  • Create a windows 8 client app with a native client app on the Azure side (give it the custom permissions)
  • Run the windows 8 app, login as a user to that app (code flow is initiated with resource target)
  • Code flow is initiated, tokens generated (access and bearer) – notice how only the scope of the target resource is sent back
  • Send the access token in the authentication header, add the refresh token to the basic headers
  • Analyze the token claims looking for the custom permission, continue on to access O365 SharePoint if present
  • Attempt to use the webapi client id and secret with the refresh token to get a new access token for the "https://domain.sharepoint.com" resource
  • Attempt to use the token…get slapped in the face with a "client_secret is invalid message", but I know 100% it is valid…*in all likeliness this is a bug in the current Azure AD version*

I have yet found a way to support this type of user flow with Azure AD (User->Client->WebAPI->O365).  Obviously the user will never be able to respond to any prompts from the webapi as it is headless and hidden away on some network far far away.  It would be great if I could pass the first access token in an auth flow to Azure with valid clientid and secret to simply generate a new access token for a second resource.  I have no idea why I have to keep passing a refresh token around (one that doesn't even seem to work when used with a client id that it was not generated with in the first place) in all my apps just to get a new token.  IMHO, It's incredibly wasteful and not very well designed.

So there you go, just a few helpful hints for your O365 API journey if you came from preview days.  I'll keep updating the post as I go…

Chris

 

Developing for Office 365

Here is the outline of the latest ACS Course.  If you are a consulting company and want to get your team up to speed, let me know and we can setup a remote training engagement!

  • Introduction to Office 365
    • Sign up for Office 365 Developer
    • Explore Office 365 Admin Center
    • Explore Office 365 Services (Outlook, Calendar, People, Newsfeed, OneDrive, Sites)
    • Provision an O365 Developer Site
    • Install Office Dev Tools
    • Install the NAPA App
    • Install O365 API Tools
    • Configure Visual Studio for Office 365 Development
    • Exploring Azure AD PowerShell Cmdlets
    • Exploring SharePoint Online PowerShell Cmdlets
    • Exploring Exchange Online PowerShell Cmdlets
    • Exploring Lync Online PowerShell Cmdlets
  • Authentication
    • Generating FedAuth Cookies (Windows App)
    • Generating FedAuth Cookies (Windows PowerShell)
    • Generating Bearer Tokens (Provider Hosted Apps)
    • Registering SharePoint Apps Manually
    • Requesting On-The-Fly Permissions
    • Using Basic Auth against O365 APIs
    • Setup Azure Active Directory
    • Configure Azure Active Directory
    • Synchronize local AD with Azure AD
    • Configure Single Sign-On
    • Using the Microsoft Remote Connectivity Analyzer
    • Registering Azure AD Applications
    • Manual Authentication to Azure AD
    • Using the Azure Active Directory SDK
    • Authorization using Consent Framework
    • Registering Applications with Azure AD
    • Building Apps that utilize Consent Framework
    • Managing Tokens
    • Extending the Azure Consent Framework
    • Implementing On Behalf API
    • End User Telemetry (MyApps)
    • Tenant Admin Telemetry (Cloud App Discovery)
  • Working with REST and CSOM
    • Using the Discovery Service
    • Working with Mail REST APIs
    • Working with Calendar REST APIs
    • Working with Contact REST APIs
    • Working with OneDrive REST APIs
    • Explore REST Services
    • REST List Service Calls (PUT, GET, MERGE)
    • REST UserProfile Service
    • REST Social Service
    • REST Search Service
    • Explore Client Object Model
    • Utilize Client Object Model (.NET)
    • UserProfilesSocial CSOM
    • Taxonomy CSOM
    • Publishing CSOM
    • BCS CSOM
    • Explore JavaScript Object Model
    • Working with Workflows
    • Working with REST
    • Using LINQ with OData
    • Working with OData Limits
    • Working with OData Filters
    • Creating Projects
    • Creating Resources
    • Creating Tasks
    • Creating Custom Fields
    • REST Authentication (Windows, Forms, O365)
  • Programming Apps
    • Explore SharePoint Store Apps
    • Install a SharePoint Store App
    • Create a SharePoint hosted App
    • Deploy a SharePoint hosted App (App Catalog)
    • Deploy a SharePoint hosted App (Side Loading)
    • Create and deploy a Provider hosted App
    • Inviting External Users
    • Exploring App and User Permissions
    • Exploring Context Tokens
    • Exploring Access Tokens
    • Exploring Refresh Tokens
    • Download the Office AMS samples
    • Explore Office AMS
    • Using the SharePoint Web Proxy
    • Exploring Office Apps Store
    • Installing Office Store Apps
    • Browsing and Removing Installed Apps
    • Create a simple mail app with Napa Tools
    • Create a simple mail app with Visual Studio
    • Review sample O365 Mail Apps (EWS Calls, Attachments, Identity Tokens, YouTube integration)
    • Creating a Seller Dashboard account
    • Create and Publish AppStore SharePoint App
    • Use the Cloud Business App template
    • Creating Data Tables and Screens
    • Register a Yammer App
    • Create a Yammer App
    • Make Yammer API Calls
  • Workflows
    • Create basic SharePoint Workflow
    • Create workflow that calls O365 APIs
    • Deploy and configure Workflow Apps using Install Events
    • Utilize JSONP
  • Mobile Development
    • Explore O365 Mobile Features
    • Install Apache Cordova
    • Create a multi device targeted project
    • Add connected services to multi device projects
    • Run a multi-device project (Android and Ripple)
    • Install the O365 Android SDK
    • Work with Android O365 Sample Apps
    • Download required referenced SDKs (GSON, Azure AD)
    • Create an Android App that calls O365 APIs
    • Create a Windows 8.1 App that calls O365 APIs
    • Create a Windows Phone 8.1 App that calls O365 APIs
    • Create a Windows Phone 8.1 App that calls OAuth services
  • OneNote APIs
    • Registering a LiveID App
    • Download and install Live SDK
    • Retrieve LiveID Tokens
    • Retrieving Notebooks, Sections and Section Groups
    • Creating Pages
    • Creating Pages in Sections
    • Importing PDFs
    • Importing Images
    • Exploring apigee.com
  • OneDrive APIs
    • Explore OneDrive APIs
    • Downloading and uploading files
    • Getting user data
    • File and Folder properties
    • Move, copy, delete files

O365 and Non-ADFS Federation – It’s not O365’s fault

I'm not a fan of ADFS.  It sits right below Azure Access Control Services (ACS).  ACS used to be my favorite identity provider aggregation platform, but how times change (ref this post).  Several new players are entering the market and all of them realize the missing gaps in ADFS and ACS.  Therefore, the drive to make a better authentication toaster is ON!

The goal of this post is to make you realize that you are having the wool pulled over your eyes in terms of the bigger picture around Identity Management when it comes to O365.  To make this point, let's take a look at a typical on-premises solution:

  • A SharePoint customer wants to setup multiple site collections (yes, all host header site collections) to support different customers
  • Each customer has different requirements around supporting federated external customers
  • Some customers prefer Google as their auth, others Yahoo and others Facebook, etc

On-premises

Solving this problem with on-premises environments is a piece of cake.  You have complete control over adding trusted Identity Providers (IdP) all day long. To solve the problem above, its likely you are going to think that adding a Trusted Identity Provider for each required auth mechanism is the answer.  That would be false.  As tempted as you may be to do this, its a really bad idea to have more than one. You should ultimately have a single Identity Provider with that provider supporting multiple "Apps/Replaying Parties (in this case site collections)" that would then show different options when a user needs to auth.  This allows you the most flexibility in that you simply set the IdP once, then configure the apps in the IdP.  Totally seamless, perfect.

Office 365 and Single Sign On (ADFS)

If you had this requirement in O365, you are out of luck.  Do not pass GO, do not collect $200USD.  In terms of O365, you are by default forced to use this one IdP mentality.  For each verified O365 domain, you can only setup a single Federated Identity Provider.  In most cases, everyone I have seen has used ADFS to integrate their on-premises Active Directory to sync accounts and/or configure Single-Sign On to redirect to their ADFS proxy.  This unfortunately, is the Microsoft promoted way of achieving Single-Sign-On nirvana.  I'm not a baby swimming in a pool, so let's take a look at what lives outside the koolaid-y-ish sugar water.  If you do this (which unfortunately, you probably already have), you are locking yourself into the current limited features of ADFS and hoping that ADFS gets some much needed development attention in the future (such as supporting OAuth 2.0). That being true, ADFS is a Microsoft product, you'll find a lot of people that can help you (support and consultant resources) to get it setup within a few hours.  You could also do it yourself by using TechNet/MSDN et all. 

Office 365 and Something else

If you want to use something other than ADFS, well, you better dig in for doing a ton of research on how WS-Fed and WS-Trust works and similarly, OAuth.  If you don't have time for that, then you'll need to request the services of people that do know that space. Unfortunately, having the identity management skills matched with an understanding of the future of App Models across vendors is a rare breed of consultant.  You also have to contend with a cold hard fact:

  • Currently, Office 365 only supports users of your verified domains (such as contoso.com)

Say what?  It means that even if you wanted to use Twitter, Facebook, whatever, the people that signed up have to have used the email address from their verified domain!  Uh…say that again?  Yeah, that means you can't login with your user@hotmail.com account using Facebook.  If you signed up with your contoso.com account on Facebook, then it will likely map your email over in the claims and let you in, but that's up to how fancy your Identity Provider is.  Azure AD is expecting a very specific format of token with specific claims.  If it doesn't get them, then your login will fail.

So if you can't login using something other than a verified account, what's the point?  

I feel this is a major fundamental loss of functionality between on-premises and O365.  If I want to create a site collection with a specific relaying party app to my IP (which then provides a set of auth services), I should be able to do it. It is such a common pattern in on-premises SharePoint. I don't think the band aid of "external access" fits the bill here.  Those links that are generated can be freely sent all over the web punching small barely noticeable holes in your O365 tenant. I'd like to see the governance page in O365 that shows me every link that has been generated across the entire tenant (you can see on a site by site basis using this link – https://domain.sharepoint.com/_layouts/15/start.aspx#/Access%20Requests/pendingreq.aspx).  There is not global admin UI to tell you who has been added or what has been shared – reference notes at bottom of this article. With an IdP in the picture, you simply disable their access and now they will never gain access to anything in your environment. This would require the removal of this "external sharing" feature that bypasses Azure AD in O356.  Wow, isn't that innovative?

Can I even use something other than ADFS in O365?

Heck yeah you can!  I recommend it!  By adoptingchoosing an Identity Provider solution (whether one you install on-premises or cloud hosted) that allows for you to expand
out past basic AD accounts, you will be ready for when Azure and O365 wakes up one day to
realize they should have allowed it in the first place.  You can set this up by removing the ADFS federation and replacing it with new Ws-Fed endpoints of your IdP.  This is enabled using the following cmdlets (this one is using Auth0.com):

  • $cred = Get-Credential
  • Connect-MsolService –Credential $cred
  • Set-MsolDomainAuthentication -DomainName "chrisgivens.com" -FederationBrandName "chrisgivens.com" -Authentication Federated -PassiveLogOnUri "https://chrisgivens.auth0.com/wsfed/ID" -ActiveLogonUri "https://chrisgivens.auth0.com/ID/trust/usernamemixed?connection=yoyo.chrisgivens.com" -MetadataExchangeUri "https://chrisgivens.auth0.com/wsfed/ID/FederationMetadata/2007-06/FederationMetadata.xml?connection=fs.chrisgivens.com" -SigningCertificate "Cert" -IssuerUri "urn:chrisgivens" -LogOffUri "https://chrisgivens.auth0.com/logout" -PreferredAuthenticationProtocol WsFed 

NOTE:  If you have already setup ADFS, you'll need to disconnect from that.  I have yet to test out all the paths, but the one I choose was really painful.  I used the Remove-MsolFederatedDomain cmdlet.  This forced me to remove all users (but did not require me to remove my groups).  Once I removed all users the command succeeded, but proceeded to delete the domain completely from my tenant!  I then had to re-add it, re-verify it, then fire off a DirSync.  Once that once done, I could run the above commands and it redirected properly to my IdP.

After configuring the federation when you go to "login.microsoftonline.com" and type your verified email, it will redirect to your IdP login page.  From there, you will see all the options that you have enabled for that relaying party.  Even though you can enable all the OAuth 2.0 providers you can imagine, an attempt to login will fail miserably.  There are some current technology issues that Azure AD must resolve:

  1. Azure expects a UPN/Email that matches a user in your Azure AD (at a minimum, you would have needed to sign up for a service using that verified domain email)
  2. Azure AD is expecting an ObjectID
  3. O365 Services have to support these accounts (even if that means simply ignoring them)

For the first issue.  Azure AD would have to allow the addition of users other than your verified domain.  Or at your IP, you map a user from external to internal.  For the second issue, again, this can resolved by either Azure, or by the Identity Provider.  IPs can choose to implement this today by integrated with Azure AD to query for the object id based on a valid UPN.

Until this these items are resolved, you can certainly fall back to the "forced" way by Microsoft to use Active Directory users from a verified domain (and yes, you should feel angry about this) by setting up a pass through from your IP to ADFS/Active Directory.  The preferred way is to utilize a client on your Active Directory server (pretty much every IdP has this). As only a few of you know or will understand, if you use ADFS, you will currently lose Office Client features. But don't be worried, later this year, Office will release an update that will support SAML-P.  This will let you utilize many different types of auth (basically anything you can do via browser) with Office clients.  That will be awesome, even though users will never fully understand what just happened.

Take It Up Another Level

Home realm discovery is not extensible in Azure AD.  What is home realm discovery?  Its the login drop page that takes your context, in this case, your email domain, and then redirects you to the proper IdP.  What does it mean to not be extensible?  It means that IdP's cannot create a portal of Apps page.  What does that mean?  Think Universal Access Gateway (UAG).  Imagine UAG-like portals, only hosted by your IdP, such that an external party can login to your IdP, and are then presented with the applications (such as O365), that they have access too.  When they select it, it will immediately redirect you to your O365 site (no need to go to login.microsoftonline.com).  As it stands right now, O365 does not support external home realm discovery pages.  This is also a serious limitation that needs to be added to the road map.

Summary

If enough people push the O365 and the Azure teams to fill the gap of this major difference between the products, I feel it would fulfill many of the requirements we have with customers today and make O365 an even more popular platform!  Here is your call to action:

  • Quit using ADFS, find an IdP with an eye towards the future
  • Highly "Suggest" to Azure and O365 that they support external identities (Vote for it here on UserVoice)
  • "Suggest" to Azure that they allow tenants to have a custom home realm discovery page

Stay classy internet,
CJG

 

 

O365 is not secure

I used to think it was.  After all the amazing things you read on the O365 Trust Center, you all but think it is Fort Knox (well, at least one part of it is right Bill Baer?) 

Let me go back to that title. "O365 is not secure…once you start adding users".  Last week I blogged about a security oversight in O365's partner sharing feature.  Basically what this means is that once you accept an invitation from a partner (aka a "user"), you become just another customer in their "customer management" portal. You can read more about this feature here:

http://blogs.office.com/2014/01/28/introducing-the-new-office-365-partner-admin-center/

Do you know who else they have as customers and who is accessing your data and when?  No. They don't really have any logs to tell you when a partner is accessing your portal for accountability.  Nor do you know who on the backend is working on your O365 deployment project.  Just too many unknowns to say it is secure.

O365/Azure Apps

But that's not the focus of this post.  A few weeks ago, O365 released the O365 APIs for Exchange (Mail, Contacts, Events and Files).  All of this piggybacks on Azure Active Directory for authorization utilizing the Consent Framework.  The Azure AD platform allows you to build an application that can make calls to Azure Active Directory or your O365 data. The permissions of the consent framework can allow an application to read the current mail item, all your mails items, your contacts, your calendar…your files. 

As a simple example, one of the more impressive abilities is to create a Mail App.  You can do this using Visual Studio 2013 and creating an App for Office project.  This isn't going to be a developer post so I'm not going to show you any code, but simply talk about the importance of "Mail Apps" and your users.

Once you have a Mail app created, you can deploy it to your O365 instance, or even post it in the Office store.   You can find a small set of mail apps in the following link (these have been publicly deployed to the office store, but you can have rogue apps that simply are registered somewhere in Azure AD and users give them ad hoc permissions):

http://office.microsoft.com/en-us/store/apps-for-outlook-FX102804983.aspx?app=outlook.exe

Many of these are poorly designed (for example, the LinkedIn app is a total disaster, the cert is locally generated and not even a public CA).  In some cases, you may find the deployment endpoint isn't even available so the app install fails.  If you were to install one of these apps, you will find that they simply request access to the "current" mail item.  This is the preferred scenario if you were to use these apps.  In some cases however, you may find that the app is given "carte blanche" to your entire mail inbox.

So back to users and mail apps.  When you are in your Outlook, your users will ultimately find the settings icon and this nice menu item called "Manage Apps":

 

Clicking on this, they will be taken to the "Exchange" app center.  This is where you see administrator installed apps and user installed apps:

 

This is JUST for Exchange.  This has nothing to do with the bigger picture of Azure AD Apps and the consent framework.  There are some governance controls implemented in the Exchange app architecture, however, when it comes to Azure AD and your tenant, the story hasn't been written yet. This goes back to the premise of this blog post.  All it takes to make your highly secure O365 environment to be unsecure is:

  • A user
  • An App
  • A click 

The permissions given to that app can now comprise the integrity of your entire system.  When on-premise, you at least had the ability to prevent incoming requests using your router and DMZ.  In this instance, O365 doesn't have any of that, so once permissions have been given, that's all folks.  So what hacking scenario might come up?

  • Hacker builds an Azure app, puts your company name and logo on it
  • Hacker does some social engineering to find out your name, title, work phone number and other information
  • Hacker asks you to install the app and because it has your logo and company info, your user thinks nothing of it and clicks "trust"
  • Hacker can now gain access to an access token that can be used to download all your email and your my site files

If you are in defense or government, you know hacking enemy number one is China.  For those of you that think that insensitive to say, you haven't watched the traffic logs with the FBI and CIA as it happens, almost on a daily basis.  But the US is not a good guy when it comes to that stuff either, so all is fair in love and war.

O365/Azure App Governance

It is my suggestion, that you disable the ability for users to add apps.  It simply creates security holes that equate to users writing company checks that can't be cashed.  So, with that recommendation, can I provide you the steps to deny a user the ability for users to add apps" in your tenant, *sure*, thanks to some recently surfaced info (see below) [:D] (thanks to @richdizz, @jthake and @Matthias1o1) In terms of Exchange, there are some user roles in Exchange that you can "remove" to prevent users from deploying apps, these roles are enabled by default:

 

————————
[UPDATE 6-5-2014]
After this post the following was brought to my attention from the O365 team.  It is a non-UI based way to disable all user consent, thanks to the O365 team for pushing this content out!:

http://blogs.msdn.com/b/exchangedev/archive/2014/06/05/managing-user-consent-for-applications-using-office-365-apis.aspx

————————

Even with the above update, what is still not available to us is an Azure AD UI (to make it more apparent of the risk) that allows me to
disable apps, and/or designate a "safe/white list" of apps that my users *can* utilize in Exchange or Azure AD.

Just for kicks, I have created a UserVoice suggestion that more advanced App Governance features (such as a UI and white list) be added to Azure AD and O365, you can vote for it here:

http://officespdev.uservoice.com/forums/224641-general/suggestions/6010008-create-app-governance-in-o365-azure-ad

My God, What Have You Done?

So, this brings up an interesting point.  Last night at the SanSpug.org meeting I announced that we have moved to Auth0.com as our auth mechanism as it provides the most flexibility and telemetry data.  One person asked, what is telemetry data?  I said every time you login, I get data about you (first name, last name, email, blah blah).  It freaked out half the group.  Only one person was educated enough to know which of the supported auth systems had the least amount of data being sent.  So again, this brings up that interesting point I started this paragraph with…do you know what apps you have simply just clicked on the "authorize" button just so you could use the app?  I bet you don't, my god, what have you done?  Think about all those apps across all your services (facebook, twitter, liveid, google, etc).  So in an effort to help you understand just how bad this is going to get with O365, based on how bad it has gotten in a general sense, here is the links to where you can check what you have authorized for each of the major platforms:

Facebook:

https://www.facebook.com/settings?tab=applications

Twitter:

https://twitter.com/settings/applications

Google:

https://security.google.com/settings/security/permissions?pli=1

LiveID:

https://account.live.com/consent/Manage

eBay:

http://cgi6.ebay.com/ws/eBayISAPI.dll?ManageESubscriptions

Yammer:

https://www.yammer.com/YOURDOMAIN/account/applications

SharePoint:

I suppose you could say that this page (http://youweburl/_layouts/15/viewlsts.aspx) is as close as it gets?  At a farm level, in O365:

https://yourdomain-admin.sharepoint.com/_layouts/15/TA_AllAppPrincipals.aspx

On-premises:

http://centraladminurl/_admin/AllAppPrincipals.aspx

Azure AD:

https://myapps.microsoft.com

Back to the question…"My god, what have you done?"  After clicking on all those links, I'm sure you have an idea of just how bad the problem has gotten.  Some of you might be surprised to find out that your Hotmail/Outlook.com email and OneDrive (consumer) files have probably already been downloaded and analyzed by that cool app you gave permissions too.  Most of you that read my blogs are the cream of the intellectual crop.  You understand what that "Trust" dialog means.  For the rest of the population, not so much.  The meaning of the word "Trust" has been eroded by fancy graphics and the promise of a simple functionality at the expensive of selling your data soul to the app provider.  That all being said,  its time for enterprise solutions to have a "white list" to keep users from eroding the meaning of "Trust", to anyone's dog or cat.  One last thing on O365, no matter how secure you make the back end, its the front end that matters.

CJG

Apache Cordova, Apache Ripple, Visual Studio 2013 and O365 APIs

"Mobile first" has become a common theme these days and building code and applications that you can write once and deploy to multiple target platforms is a key to achieving reach and use for your backend platform.  With the O365 APIs being release a few weeks ago, you can now do REST development against O365 exchange services (Mail, Contacts and Calendar, however, no Tasks yet).  A lot of focus has been on putting out samples for the various mobile platforms (Windows Phone and Android as the top two).  That being said, let's take a look at these Android examples and then the new Apache Cordova project templates. 

Android:

The Android SDKs that you can download here need some serious help in order to get them loaded into a Java IDE (such as Eclipse Kepler).  For example, not only do you need the Active Directory library, you also need several Google SDKs (yeah, I said Google).  In order to get all the examples working, you need to download the following:

Once you have all these, you can extract the O365 SDK and you will notice several projects that you can import into Eclipse (you will notice that all of them have errors, this is because you need to add the referenced JAR files to the project build path):

 

Each project will need the "com.microsoft.adal", "office365-base-sdk", json-2.2.4" and "guava-16.0.1" JARs.  Once you have these added, you will notice that all the projects now show no errors.  You are now free to select a project and run it. The first app is called the "Asset Management" app:

 

Opening the application allows you to start configuring it:

The authentication options can be of three types:

Azure Active Directory (AAD) is the OAuth method. Selecting it will allow you to utilize redirection to O365 login page and then back to your application.  Once you have authenticated, you can click the "Show Cars" button:

It simply makes a call to a SharePoint picture library called "Cars" (that you have to create and add items too) to show you the picture of the car and its description:

In addition, there is a "File Discovery" application.  This can be configured by modifying the "Constants.java" file (which is a common thing across the apps) by adding your client id and secret.  Once you have updated the files, running the app will show the following icon:

Once open, you will see that it queries the discovery service and shows all your services (via the Discovery API):

Clicking on the Files service will show you your OneDrive for Business Files:

You may find that you keep getting an exception around the dialog.dismiss method.   This is a bug in the code, all you have to do is wrap it in a try catch block and the code will run ok.

There is also a mail app (but it doesn't have a .project file so it won't import as a project).  Unfortunately I was never able to get it to work because it kept throwing memory errors, but supposedly you can also get it to show you your mail. You will also need a couple of extra JAR files to get it to compile:

Apache Cordova:

Not into building platform specific Apps like the Android ones above?  That's where the Apache Cordova platform comes into play!  If you do a quick Google search, you will find this site:

http://cordova.apache.org/

This site simply describes itself as "A platform for building native mobile applications using HTML, CSS and JavaScript".  Basically what that means is that it is a translation middleware of common web standard elements as common inputs and then transform your code to a target such as Android or Windows Phone.  The neat part is that Apache Cordova is now integrated into Visual Studio 2013 Update 2.  You can download the Apache Cordova project extension here:

http://www.microsoft.com/en-us/download/details.aspx?id=42675

This will give you the ability to create Hybrid Apps:

 Once you have created a Hybrid App, you will see the following in your Solution Explorer:

In Visual Studio, you will have a set of target devices and also the ability to specific your debug target:

 

Notice here, the "Ripple" debugging target.  Before you start debugging, you can modify the supported features of the target device via the App Manifest's "Plugins" tab:

 

 Now this is where things get interesting.  You can right-click the project and select "Add->Connected Service".  From here you can add either "Windows Azure Mobile" or "Office 365" services. 

 

The view is similar to if you were doing an MVC or Web Application Project, but the output is VERY different.  What you will notice is several folders and JS files are added:

 

The above "settings.js" does some initialization of the O365 SDK. The remainder of the files have some pretty powerful code in it that all but builds your application logic!

 

Once you have all of these files in your project, you have to add them to your "index.html" page:

 <body>
    <p>Hello, your application is
ready!</p>

    <!-- Cordova reference, this is
added to your app when it's built. -->
    <script src="cordova.js"></script>

    <!-- cjg699 references -->
    <script src="./services/mobile services/cjg699/service.js"></script>

    <script src="scripts/jquery-2.1.1.js"></script>
    <script src="services/office365/settings.js"></script>
    <script src="scripts/aadgraph.js"></script>
    <script src="scripts/exchange.js"></script>   
    <script src="scripts/o365adal.js"></script>
    <script src="scripts/utility.js"></script>   
    <script src="scripts/index.js"></script>
    <script src="scripts/o365.js"></script>
   
    <input type="button"
id="btnGetEvents" name="btnGetEvents"
onclick="GetEvents();" value="Get Events" />
    <br />
    <br />
    <div id="Events">
    </div>
</body>

Once that is done, you can add your own O365.js file with your logic.  In this case, I am simply making a call to get the events:

function
GetEvents() {
    var authContext = new
O365Auth.Context();
   
   
authContext.getIdToken('https://outlook.office365.com/')
    .then((function (token) {

        // authentication succeeded
        var client = new Exchange.Client('https://outlook.office365.com/ews/odata',
token.getAccessTokenFn('https://outlook.office365.com'));

       
client.me.events.getEvents().fetch()       
        .then(function (events) {
            // get currentPage of
calendar events
            var myevents =
events.currentPage;

           
$("#Events").empty();

            for (i = 0; i <
myevents.length; i++)
            {
               
$("#Events").append("Subject: " +
myevents[i]._Subject)
            }           

        }, function (reason) {
            console.log(reason);
        });
    }).bind(this), function (reason) {
        console.log(reason);
    });

}

 Pressing "F5", Google Chrome will start with the Ripple emulator:

 

You can now click on the "Get Events" button which will attempt to execute the code, but first direct you to provide consent:

Clicking on "Get Events", you will see your mail:

 

One important note, the JavaScript code will cache your access token.  If you didn't get your permissions setup properly, then you'll have a bad token cached.  You will need to call the "SignOut" method to clear the CacheManager and the bad token as they don't expose the CacheManager directly.

If you wanted to test on Android, you would simply change the drop down target and press "F5" again.  It would then attempt to start the current Android SDK Emulator and upload the APK file, you would see exactly the same UI and button and the same results!

Unfortunately, at the moment, the JavaScript API for O365 is only provided in the Hybrid Apps project and is not really compatible with the other project types (such as MVC).  Apache Cordova, the Ripple emulator and project templates such as Hybrid and Cloud Business Apps are the future of application development!