Design a security trimmed navigation in SharePoint Online

Design a security trimmed navigation in SharePoint Online

...or SharePoint 2013, SharePoint 2016, SharePoint 2010...you get the picture by now.

What users want...

Users want to navigate quickly to the required information...ow gods if this was the only thing they wanted. Let's asume we have a user that wants just that :-)

...users get!

When working with SharePoint you have to take the boundaries and limits into account. One of the biggest challanges with navigation in SharePoint is how to get information accross site collections? Your biggest asset here is the use of search. Search is able to look beyond site collection boundaries. Another mayor benefit is that it is security trimmed. This means that you only see what you are allowed to see.

For more information on security trimming in SharePoint read the following:
https://technet.microsoft.com/en-us/library/dn167721.aspx

This makes search one of the best candidates for navigation.

My Enterprise Search Center is up and running now what?

In order to really use search you can (or must?) plan your information architecture.

Let's say we have created a site collection:
- Projects

We have created 1 (for simplicity) content type called: Base with a single site column: Customer. Customer is of the type managed metadata.

When we create a new Project site we add a list with the content type Base. After that we add a record to the list, defining the Title and Customer.

We created the following structure:
/sites/projects/Project1 | Base.Customer:Contoso 
/sites/projects/Project2 | Base.Customer:Microsoft
/sites/projects/Project3 | Base.Customer:Contoso

Because we added records to the lists and thus the site columns Search creates crawled properties. When this is done we can create managed properties.

Let's say we created a managed property called: Customer.
We can now search for:
Customer:Contoso.
This should result in 2 hits Project1 and Project3.

You can use the above to create the following wireframe:


The recipe:
- 1 web part page
- 1 search refiner webpart
- 1 search results webpart

Configure the refiner webpart to be able to refine on Customer (division or status).
Configure the search result webpart with the following config:
path:https://<tenant>.sharepoint.com/sites/projects    (contentclass:STS_List) ContentType:Base

Hope you have fun with this!

For background reading

https://support.office.com/en-us/article/Manage-the-search-schema-in-SharePoint-Online-d4fab46d-ba41-4c03-9d4c-32b5b33198b6#__toc351360841

https://support.office.com/en-us/article/Manage-the-search-schema-in-SharePoint-Online-d4fab46d-ba41-4c03-9d4c-32b5b33198b6?ui=en-US&rs=en-US&ad=US

https://support.office.com/en-us/article/Manually-request-crawling-and-re-indexing-of-a-site-a-library-or-a-list-9afa977d-39de-4321-b4ca-8c7c7e6d264e?ui=en-US&rs=en-US&ad=US

http://en.share-gate.com/blog/understand-sharepoint-crawled-and-managed-properties-for-search

Troubleshooting search in SharePoint Online.
https://blogs.perficient.com/microsoft/2014/11/troubleshooting-search-in-sharepoint-online-o365/

SharePoint Online, Windows Intune, Information Rights Management and a cherry pie

SharePoint Online, Windows Intune, Information Rights Management and a cherry pie

A short story on how Windows Intune, Information Rights Management SharePoint Online can work together while eating a delicious cherry pie.

First things first. What's what?

SharePoint Online 

Organizations use SharePoint to create websites. You can use it as a secure place to store, organize, share, and access information from almost any device. All you need is a web browser, such as Internet Explorer, Chrome, or Firefox.
https://support.office.com/en-us/article/What-is-SharePoint-97b915e6-651b-43b2-827d-fb25777f446f

Windows Intune

Microsoft Intune offers features for the management of mobile devices and applications, and pc management from the cloud. With Intune enables organizations to their employees virtually anywhere and on virtually any device access to enterprise applications, data and business information sources and at the same time secure.
https://www.microsoft.com/nl-nl/server-cloud/products/microsoft-intune/overview.aspx

Information Rights Management

Within SharePoint IRM protection is applied to files on the list and library level. Before your organization can make use of IRM protection, you must first set up Rights Management. IRM SharePoint Online is based on the Active Directory Rights Management service Microsoft Azure (Microsoft Azure AD RM) to encrypt and assign usage restrictions.
https://support.office.com/nl-nl/article/IRM-Information-Rights-Management-instellen-in-het-SharePoint-beheercentrum-239ce6eb-4e81-42db-bf86-a01362fed65c#

How can Windows Intune, IRM and SharePoint Online benefit from each other?

Really short: Windows Intune enables organizations to mark devices as trustworthy, or not. This makes it possible to say (for instance) : hey! Only devices I trust are allowed to access to SharePoint Online or corporate email.

A typical flow for conditional access might look as follows:

Use conditional access to manage access to Microsoft Exchange On-premises, Exchange Online, Exchange Online Dedicated, and SharePoint Online.

https://technet.microsoft.com/en-us/library/dn818907.aspx

This shows how Microsoft Windows Intune and SharePoint Online work seamlessly together.

How does IRM help in this picture?
IRM helps secure confidential content in the following ways:
- prevent an authorized user to copy content for unauthorized use, customize, print, fax or copy and paste.
- prevent an authorized user to copy the content with the print screen feature of Microsoft Windows.
- prevent an unauthorized user to display the content that is sent in an e-mail message after being downloaded from the server.
- Access to content is limited to a specified period, after which users must confirm their credentials and need to download the content again.
- Assists in the implementation of corporate policy for use and dissemination of content in your organization.
https://support.office.com/nl-nl/article/IRM-Information-Rights-Management-op-een-lijst-of-bibliotheek-toepassen-3bdb5c4e-94fc-4741-b02f-4e7cc3c54aa1?ui=nl-NL&rs=nl-NL&ad=NL

Conclusion

Microsoft has done a great job in advanced security and protection of your intellectual property. Please use your own intellect by properly following the plan-do-act circle!

Grand Theft Data (Loss Prevention) in SharePoint 2016

Grand Theft Data (Loss Prevention) in SharePoint 2016 or Online

Sorry, no new version of GTA is forthcoming…this blog is about Data Loss Prevention (DLP) in SharePoint 2016 public beta 2 release.

DLP what is it not? DLP has nothing to do with backup or recovery. In general it is a combination of technology and processes that safeguard sensitive information from (un)intentional loss.

This blog is a summary from a nice blog on the MVP Award Program site.

What is DLP? It is a complementary set of technologies to aid your business strategy to handle and protect sensitive business data that restricts sensitive data being put into SharePoint. 

Examples are:

• Credit card numbers
• Passport numbers
• Etc.

DLP is consists of 2 main elements:

• Discovery
• Policy

An important point to mention here is that both of these options do apply to both items stored in SharePoint 2016 On Prem, SharePoint Online and Items stored in OneDrive.

 

Data Loss Prevention: Discovery

Having the ability to perform a DLP query based on a DLP template. A.k.a check if credit card numbers and the like are in SharePoint! The outcome relies 100% on search having crawled all content.

 

In other words the Discovery section let you FIND and REPORT information that holds sensitive information.

 

You will be needing one (or more) eDiscovery sites for this.

So if you have a document with a creditcard number in it, the eDiscovery site will report this document.

 

Data Loss Prevention: Policy

Having the ability to enforce and actually restrict viewing of sensitive information! To do this you need to create a (set of) policy(s).

 

After you have set the policies and you have linked it to your site collection(s) the document with the creditcard number will be blocked!

 

You will need to create one (or more) compliance site(s) for this.

 

Please read Steve's blog for full details! It is really a very good document (and read)!

 

Sources:

https://blogs.msdn.microsoft.com/mvpawardprogram/2016/01/13/data-loss-prevention-dlp-in-sharepoint-2016-and-sharepoint-online/?utm_content=buffer8ff4b&utm_medium=social&utm_source=facebook.com&utm_campaign=buffer

http://www.mcafee.com/us/products/total-protection-for-data-loss-prevention.aspx
https://www.checkpoint.com/products/dlp-software-blade/
http://www.microsoft.com/en-us/download/details.aspx?id=49961

Update 29th januari 2016
I just came across some extra information from Microsoft Netherlands, written by Hans van der Meer, that might be usefull to you as well:
https://blogs.microsoft.nl/microsoftvoorwerk/technologie-helpt-voorkom-datalekken/

eDiscovery:
https://blogs.office.com/2015/06/17/introducing-compliance-search-in-office-365/?fromblog=997321

Think before you act

Think before you act

 

 

SharePoint is just an enabler. The people that use your precious intranet, collaboration or whatever solution are the ones you need to keep happy. Mostly they don't give a…great deal about the technology below, in this case: SharePoint.

 

They just need to do their job and preferably as fast as possible! 

That is one of the major reasons why you need to think before you act. Ask yourself:

• What are we trying to accomplish with this solution?
• Who will be using this solution?
• What is it that they need to do?

You need to be able to answer the questions above before even thinking about creating site collections, libraries or other objects. Please do! Plan for SharePoint 2013

 

Once you have the answers you will need to document them. Again Microsoft helps us by providing templates for documenting your solutions Planning worksheets. 

 

The planning worksheets help you build your logical architecture. Perhaps you already notice we are working from the business down! Please remember that we are building a solution for real people.

 

In my future posts we will be diving head first in the information architecture to find out that different persona's work with your solution, requiring their own specifics that need to be facilitated!

Usefull links:

Plan for SharePoint 2013

Planning worksheets

lPlan logical architectures

SharePoint Search - What cannot be found?

SharePoint 2010 has awesome search capabilities! That being said, it still is SharePoint. You have got to remember that SharePoint is a platform. Platforms are build with greatness in mind.... so it is not a custom solution for your specific problem!

Like all platforms the search capabilities have their limits. If you know them you can respond to them. This blog is about what you can do with search and what you cannot do with search.

Consider the following scenario: Document Management
First a little story so we can pin point requirements... and some cannot's!
So you have your SharePoint environment all configured to work as Document Management System (DMS). How awesome is that!

So how does this DMS work?

You upload your documents to the drop-off library. You configured a content organizer rule (or 500 of them, not counting your Record Center) that makes sure that your document automagicly moves to the correct document library.

Your DMS also supports Document-ID's. So the document you created in your Dropp-Off Library gets a unique and one of kind ID.

Now, some business-joker wants to locate 'his' document. Geeees the nerve of some people. So, where is it? You uploaded one document? Well, give me 5 minutes and we'll have it located! But what if we are working in a reasonable sized company and we are processing 5.000 or more documents per month? That's about 170 documents per day.

SharePoint Search to the rescue! You have content organiser rules in place so you know your content (otherwise you cannot create good content organizer rules). Let's say the document that the user wants to locate is a complaint.

Try using the CQWP in such an environment :-) look at this blog about that hell! So you configured search to display all non-handled complaints and.... nothing?! WTF?

So why is that? Dunno :-) but perhaps it is because one of the following limits, boundries or other functionalities:

• Documents in the drop-off library aren't crawled
• Checked out documents aren't crawled
• Search can only find what it crawled via an incremental or full crawl.

So what does this mean in real life scenario's:

• The document was uploaded to the drop-off library but wasn't (yet) indexed so SharePoint does a check-out and the complaint will not be processed!
• The document wasn't indexed correct at the drop-off library so SharePoint does a check-out and the complaint will not be processed! Documents in a drop-off library aren't visible for search and thus the complaint cannot be found by the end user!
• The document was uploaded via an automated process with incorrect metadata and thus it is again checked out.
• The document was uploaded via an automated process with correct metadata but the content organizer rules didn't run yet (scheduled daily by default).
• The document was processed correct but

I mentioned the document-id feature earlier. When you create a document center there is a little box in wich you can enter a specific document-id. This box uses search capabilities so all the issues/functionalities mentioned above count for this little gem also!

Sources:

http://blogs.technet.com/b/speschka/archive/2009/10/30/sharepoint-2010-content-organizer-part-1-a-cool-new-feature-for-managing-your-content.aspx
http://blogs.technet.com/b/speschka/archive/2009/10/30/sharepoint-2010-content-organizer-part-3-the-rules-engine.aspx
http://groundhog667.blogspot.nl/2013/08/sharepoint-2010-content-query-web-part.html
http://roykimsharepoint.wordpress.com/2011/05/28/content-organizer-feature-for-large-site-hierarchies/

Information management - Deployment

This blog is all about Deployment of your solution!

First thing first. Where are we in the process? When we are ready for deployment we already covered (in one form or another):
- Information architecture - Done!
- Logical architecture - Done!
- Physical architecture - Done!
- Installation - Done!

Woohoo! So now we can implement our solution, finally!

You have several options for a SharePoint deployement:
- Manual
- Automated
- Mixed

Manual deployment
A manual deployment is exactly what it is, a manual deployment. You ask somebody to manually, a.k.a using the SharePoint user interface, to create the required objects, like:

• Web apps
• Site Collections
• Sites
• Content Types
• Site Columns
• Libraries
• SharePoint Groups
• Activate features
• Configure (SharePoint) settings
• etc.

Requirements for a manual deployment:

• Accurate information architecture
• Extremely organized and punctual consultant(s)
• Test environment
• Time!

One of the key take aways here is having a specific persona in your team. One that is able to replicate exactly what is written in the information architecture and is able to spot mistakes. I have been working with SharePoint more then 7 years and haven't seen a 100% accurate information architecture with matching logical architecture.

Another key aspect if this type of deployment is time. Manually creating and testing all objects can take days or even weeks! This, off course, depends on your implementation. Think about a DMS solution with 1-5 site collections, 40 content types, 20 or so document libraries and content organizer rules that move documents through your system.

Automated deployment
An automated deployment is a deployment where you use pre-configured scripts, compiled code to create the objects mentioned above.

You could create SharePoint features using Visual Studio that deploy webpart, content types, libraries, etc. Another way is by using Powershell scripts.

Requirements for an automated deployment:

• Accurate information architecture
• Development team
• Development environment
• Test environment

There is a new requirement here Development environment. I personally do not think it it neccesary in a manual deployment but this is very arguable, I agree!

The requirement time isn't on the list. You've been using that during the developement process. Building the features or scripts. Deployment should be a breeze....

Mixed deployment
Off course it is really feasible to use a mixed deployment, e.g.:
Creation of web applications, site collections, content databases, content types, site columns, libraries etc is done using powershell scripts.
Creation of custom webparts is done via custom developed features.
Content organizer rules are created manually.

Requirements for an mixed deployment:

• Accurate information architecture
• Development team
• Development environment
• Test environment
• Extremely organized and punctual consultant(s)
• (Time)

With a mixed deployment you need, off course, all the requirements above!

SharePoint 2010 Content Query Web Part demystified

This blog is more about architecture & information management in a real world SharePoint environment. Since content aggregation is part of architecture & information management I decided to share my findings on the elusive Content Query Web Part a.k.a. CQWP.

The story
According to Microsoft the CQWP can be used to show aggregate content:

• over a single list
• over a site and all subsites
• over a site collection

The story demystified
The above mentioned is accurate...although not complete.
Imagine this real world scenario. Contoso is a medium sized company with approx 25.000 documents. These documents reside in a single Site Collection and are spread over multiple document libraries. Off course we implemented a thorough information architecture and this all documents are neatly classified in terms of content type and metadata.

Now we have a business requirement: "I want to see all contracts assigned to me.".
Further, an architectural principle states only OOB functionality is allowed.

And we are off.... Let's skip a couple of steps to the point the CQWP is the way to go for this requirement (OOB it is... really... trust me... no you cannot use search).

The CQWP: What does it do and where can I find it?
The CQWP is an out of the box webpart that, as stated above, enables you to aggregate data. We need aggregation over multiple libraries and we need filtering. Before this little 'gem' makes it appearance you need to enable it. This is done via the activation of the publishing infrastructure feature.

Analysis
The contracts are stored in several different libraries. Perhaps due to shere numbers, authorization, etc.

Implementation
So we create a new page, add the CQWP.
Next part is the configuration. The 3 elements we are going to focus on here are:

• Source
• Filter
• Sorting

First off the source. Since we are testing the solution we are setting the source to aggregate over a single list.
Content Type = Contract
The filter is DocumentManager (people picker field type) and it's value is set to '[Me]'.

Run the page and voila we have a result of 2 contracts (just an example). As a wise man once said: WOOHO! We have a(n almost) working solution!

Now, change the source of the filter to aggregate everything over the site collection and:
CRASH!

What just happened? Where is my nicely aggregated list of contracts?

CQWP internals
At this point we need to start debugging the solution. Review filter settings, (if you didn't already) enable trace logging, etc. etc.

As you perhaps already know the CQWP runs a CAML query that you probably want to analyze. Glyn Clough has written a nice blog on just how to do that! Off course we are using the ULS Viewer to analyze  the huge amounts of ULS logs created. Somewhere in there you will find something like:
"xxxxxxx* w3wp.exe (0x0C98) 0x16F0 Web Content Management Publishing 7352 Warning ...entTypeId" Nullable="True" Type="ContentTypeId"/><Value Type="ContentTypeId">0x0101</Value></BeginsWith><Eq><FieldRef ID="{f366697d-21a6-493e-af7d-9b3cf5410ea4}" Nullable="True" Type="User"/><Value Type="User"><UserID/></Value></Eq></And></Where><OrderBy></OrderBy></Query>' generated the following error:The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator. at the following url: XXXXX. Web Part title: Content Query b8210847-657f-40f3-80af-22bf444ad8f8"

The good, the bad and the ugly
The good part about this log is that it is extremely clear and straightforward! The CQWP is prohibited from displaying the results because it exceeds the set limitation of the list view threshold (LVT). As you can read in SharePoint 2010 capacity management the default value of the lvt is 5.000 items.

The bad part about this is that this setting isn't there because it looks really cool but it has a real purpose. Try cranking it up to, let's say, 30.000 and let half a dozen users try to access the page. You can actually see the performance penalty it causes.

The ugly part... now what? The CQWP should only return a couple of contracts, not even close to 5.000.

Indexed columns
As it turns out the columns you use with filtering AND sorting need to be indexed columns, according to Microsoft.

The ultimate catch
Try and add more than 5.000 items in a single document library, filter on a single column (using the CQWP) and give it a shot! When you use an indexed column as a filter it will return results. When you remove the indexed column from that specific list it will not return results and spit back the log mentioned above.

So, the solution is adding the filter column on all document libraries in the site collection and....CRASH! WTF? We did everything correct and still nothing.

"Statement" by Microsoft
(This is not the actual statement but a, valid, free interpretation)
The CQWP only uses the indexed columns when selecting a single list as source. The CQWP ignores the indexed columns when the source is anything else than a single list!

And that my friends is how the cookie crumbles!

Sources:
Designing large lists and maximizing list performance (SharePoint Server 2010)
Data in SharePoint 2010 – Part 2 – Content Query Web Part
SharePoint Server 2010 capacity management: Software boundaries and limits
Inspecting the caml of a content query web part