SharePoint Server 2016 New Features

As many SharePoint Developers/Users are very much eager to know what's new in SharePoint Server 2016, I tried to list out all the newly available features (information is gathered from various Microsoft & Related blogs/forums).

Though there are various new features, the below list covers most of the major/significant features.

Please note that the features are available in SharePoint Server 2016 Beta 2 with Release Candidate.

• Central Administration is no longer provisioned on all servers by default

• Document Library accessibility

• Fast Site Collection Creation

• File names - expanded support for special characters

• Hybrid in SharePoint Server

• Large file support

• MinRole farm topology

• Mobile experience

• Open Document Format (ODF) available for document libraries

• Sharing

• SharePoint Search Service application

• Zero Downtime Patching

• Removed 5,000 View Threshold – sort of

Central Administration is no longer provisioned on all servers by default

SharePoint Server 2016 Central Administration is now provisioned on the first server in a farm by default when using the SharePoint Products Configuration Wizard. Central Administration is not provisioned on additional servers in a farm by default.

Note: You can provision or unprovision Central Administration on individual servers in a farm no matter what the server role.

Document Library accessibility

• Keyboard shortcuts are provided for the following document tasks:
  • Alt + N - New
  • Alt + E - Edit
  • Alt + U - Upload
  • Alt + M - Manage
  • Alt + S - Share
  • Alt + Y - Synchronization
• Focus improvements, such as keeping focus on prior elements and focus trapping.
• Announcements for upload progress.
• Announcements for file name and file types when browsing folder and file lists.

Fast Site Collection Creation

By using a template, a Site Collections can be created in 1 second. This compares well to SharePoint 2013 that takes up almost over 40 seconds sometimes.

This new feature provides templates that work at same level as SQL Server, which reduces the round trips required between the SharePoint and SQL servers. Use the SPSiteMaster Windows PowerShell cmdlets to create sites and site collections quickly.

File names - expanded support for special characters

SharePoint has historically blocked file names that included the &, ~, {, and } characters, file names that contained a GUID, file names with leading dots, and file names longer than 128 characters. These restrictions are removed in SharePoint Server 2016.

Note: Restricted characters such as % and # are still not allowed in file names.

Hybrid in SharePoint Server 2016

Hybrid sites features allows your users to have an integrated experience while using SharePoint Server and SharePoint Online sites:

• Users can follow SharePoint Server and SharePoint Online sites, and see them consolidated in a single list.
• Users have a single profile in Office 365, where all of their profile information is stored.

Hybrid OneDrive for Business

Hybrid sites features are used in concert with Hybrid OneDrive for Business (introduced in SharePoint Server 2013 with Service Pack 1 (SP1)):

• Users can sync files with Office 365 and share them with others.
• Users can access their files directly through Office 365 from any device.

Cloud hybrid search

Cloud hybrid search is a new hybrid search solution alternative. With cloud hybrid search:

• You index all of your crawled content, including on-premises content, to your search index in Office 365. You can set up the crawler in SharePoint Server 2016 Release Candidate to crawl the same content sources and use the same search connectors in Office SharePoint Server 2007, SharePoint Server 2010, and SharePoint Server 2013.
• When users query your search index in Office 365, they get unified search results from both on-premises and Office 365 content.

Large file support

Though It is still not recommend storing large files in SharePoint, you can now go way beyond the previous 2GB limit for files. Though there's no real limit, Microsoft has strongly recommended it stays at 10GB. Otherwise, end users will very likely get disconnected or get a time out while uploading large files.

Note: You can configure the desired maximum file-size limit on a per-web application basis in your SharePoint farm.

MinRole farm topology

You can now install just the role that you want on particular SharePoint 2016 servers. This will only install what’s required there, but even better, it'll make sure that all servers that belong to each role are compliant. You’ll also be able to convert servers to run new roles if needed.

Mobile experience

Considering the usage of mobiles and tablets noway days, SharePoint 2016 comes with a touch friendly interface.

When you use a mobile device to access the home page for a SharePoint Server 2016 team site, you can tap tiles or links on the screen to navigate the site. You can also switch from the mobile view to PC view, which displays site pages as they are seen on a client computer.

Open Document Format (ODF) available for document libraries

The Open Document Format (ODF) enables you to create new files in a document library and save as ODF files so that users can edit the new file with a program they choose.

Sharing

• Create and Share folder
• Sharing Hint
• See who the folder is shared with when viewing a folder
• Members can share
• Improved invitation mail
• One-click email to approve or deny a request for access

SharePoint Search Service application

SharePoint Search supports indexing of up to 500 million items per Search Server application.

Zero Downtime Patching

Now this will surely please many of you managing the SharePoint servers, the size and number of the packages are immensely reduced. They’ve also removed the downtime previously required to update SharePoint servers.

Removed 5,000 View Threshold – virtually

A Document Library/List can have 30,000,000 documents, that’s never been an issue. However, many of you know that 5,000 seems to be the actual limit for many end users that don't know they had to index their columns.

Actually the 5,000 view threshold is actually necessary, or your entire SharePoint would be slown down. It prevents SQL to lock the entire database really.
Instead of removing this unpopular threshold, they automated the creation of Indexed Columns. This means that technically the limit is still there, but you won’t have to worry about it.

Note: Allowing automatic management of indices is configurable (on/off) through list settings.

New Compliance Center

Not only can you leverage a lot of the compliance features in Office 365 with your On-Premises SharePoint 2016, new sites have been introduced to help you stay in control.

The In-Place Policy Hold Center and the Compliance Center allow you to build your own policies and apply them against your environment. New basic policies allow you to delete data in OneDrive for Business sites after an x amount of years for example, not unlike “Retention Policies” if you think about it.

SharePoint Server 2016 Release Candidate Available

After release of SharePoint Server 2016 Beta 2 in November 2015, Recently Microsoft announced the availability of SharePoint Server 2016 Release Candidate (RC). SharePoint Server 2016 RC is mostly feature complete and represents an important milestone for customers and partners looking to deploy and evaluate SharePoint Server 2016 before general availability in Spring 2016.

You can download SharePoint Server 2016 RC via the Microsoft Download Center.

Release Candidate is an update to SharePoint Server 2016 Beta 2 and can be installed over Beta 2 only.

You can download SharePoint Server 2016 Beta 2 via the Microsoft Download Center.

Note that SharePoint Server 2016 Beta 2 is only an development evaluation version and it is not recommended for production use.

SharePoint Custom ULS Logging

Here is a working code snippet (C# class) to write custom errors or logs to ULS log.

using Microsoft.SharePoint;
using Microsoft.SharePoint.Administration;
using System.Collections.Generic;

namespace CustomLogging
{
    class LoggingService : SPDiagnosticsServiceBase
    {
        public static string diagnostcAreaName = "CustomLogging";
        public static string errorPrefix = "CustomLogging: ";

        private LoggingService()
            : base("CustomLogging Logging Service", SPFarm.Local)
        {
        }

        protected override IEnumerable<SPDiagnosticsArea> ProvideAreas()
        {
            List<SPDiagnosticsArea> areas = new List<SPDiagnosticsArea>
            {
                new SPDiagnosticsArea(diagnostcAreaName, new List<SPDiagnosticsCategory>
                {
                    new SPDiagnosticsCategory("CustomLogging", TraceSeverity.Unexpected, EventSeverity.Error)
                })
            };

            return areas;
        }

        public static void LogError(string message, EventSeverity eventSeverity)
        {
            SPSecurity.RunWithElevatedPrivileges(delegate ()
            {
                SPDiagnosticsService diagnosticService = SPDiagnosticsService.Local;
                diagnosticService.WriteTrace(0, new SPDiagnosticsCategory(diagnostcAreaName, TraceSeverity.Monitorable, eventSeverity),
                   TraceSeverity.Monitorable, errorPrefix + "{0}", new object[] { message });
            });

        }
    }
}

An entry to ULS logs can be written using the below snippet with make use of the above class.

try
{
    //Business logic
}
catch (Exception ex)
{
    LoggingService.LogError(ex.Message + " " + ex.StackTrace + "\r" + ex.Source, (EventSeverity)EventSeverity.Information);
}

SharePoint Cross Site Lookup Field

Can we create a cross site lookup field?
Yes of course, it can be. Thought it is not possible OTB, a simple trick will do this.

Scenario: Creating a cross site lookup field between the two peer web sites or sub web sites under a same site collection.
Example: Create a lookup field between two sites http:/sitecollection/webA & http://sitecollection/webB

The trick is just to set the lookup field webId with the target list webId.
Code snippet to create cross site lookup field programatically:

private bool CreateCrossSiteLookupField(SPWeb sourceWeb, SPWeb targetWeb, string sourceListTitle, string targetListtitle, string lookUpFieldName, string lookupValueColumn, bool isRequiredField)
{
    ////Get the source and target lists 
    SPList sourceList = sourceWeb.Lists.TryGetList(sourceListTitle);
    SPList targetList = targetWeb.Lists.TryGetList(targetListtitle);

    if (null != sourceList && null != targetList)
    {
        ////Add a lookup field to source list with target list as lookup list
        sourceList.Fields.AddLookup(lookUpFieldName, targetList.ID, isRequiredField);

        ////Get the created lookup field from source list 
        SPFieldLookup lookupField = (SPFieldLookup)sourceList.Fields[lookUpFieldName];

        ////Set the lookup field's webID with target list webId
        lookupField.LookupWebId = targetList.ParentWeb.ID;

        ////Set the lookup field's display field 
        lookupField.LookupField = targetList.Fields[lookupValueColumn].InternalName;

        ////Finally update the lookup field
        lookupField.Update();
    }
    return true;
}

Code snippet to create cross site lookup field with PowerShell:

Add-PSSnapIn "Microsoft.SharePoint.Powershell"

#Get the webs and lists
$webA = Get-SPWeb http://siteurl/webA/
$webB = Get-SPWeb http://siteurl/webB/
$sourceList = $webA.Lists.item("SourceList")
$targetList = $webB.Lists.item("TargetList")

#Add a lookup field to source list with target list as lookup list
$sourceList.fields.AddLookup("LookupFieldName", $targetList.id, "true")

#Get the created lookup field from source list
$lookupField = $sourceList.Fields["LookupFieldName"]

#Set the lookup field's webID with target list webId
$lookupField.LookupWebId = $targetList.ParentWeb.ID

#Set the lookup field's display field
$lookupField.LookupField = $targetList.Fields["Title"].InternalName
$lookupField.Update();

How to Get Outgoing E-Mail Settings Programatically

Sometimes it might be needed to get the Web Application outgoing emails settings programatically.
The below code snippet gives all the settings including Outbound SMTP server, From address, Reply-to address and Character set information.

using (SPSite site = new SPSite("http://siteurl"))
{
    SPWebApplication webApp = site.WebApplication;
    string outboundSMTPServer = webApp.OutboundMailServiceInstance.Parent.Name;
    string fromAddress = webApp.OutboundMailSenderAddress;
    string replyToAddress = webApp.OutboundMailReplyToAddress;
    int characterSet = webApp.OutboundMailCodePage;
}

Web application outgoing email settings can also be updated programatically as below.

using (SPSite site = new SPSite("http://siteurl"))
{
    string outboundSMTPServer = "Outbound SMTP server";
    string fromAddress = "From address";
    string replyToAddress = "Reply-to address";
    int characterSet = 65001; //Character set code

    SPWebApplication webApp = site.WebApplication;
    webApp.UpdateMailSettings(outboundSMTPServer,fromAddress,replyToAddress,characterSet);
}

Similarly we can get & set the Global outgoing email settings as below.
GET:

var globalAdmin = SPAdministrationWebApplication.Local;
var outboundSMTPServer = globalAdmin.OutboundMailServiceInstance.Parent.Name;
var fromAddress = globalAdmin.OutboundMailSenderAddress;
var replyToAddress = globalAdmin.OutboundMailReplyToAddress;
var characterSet = globalAdmin.OutboundMailCodePage;

SET:

string outboundSMTPServer = "Outbound SMTP server";
string fromAddress = "From address";
string replyToAddress = "Reply-to address";
int characterSet = 65001; //Character set code
var globalAdmin = SPAdministrationWebApplication.Local;
globalAdmin.UpdateMailSettings(outboundSMTPServer, fromAddress, replyToAddress, characterSet); 

SharePoint AutoComplete Lookup Field with JQuery

Created a Custom SharePoint Lookup field that offers new functionalities to default SharePoint lookup field by allowing Auto Complete or Type Ahead search functionality with Contains CAML query.

Below is a few of the features offered by AutoComplete Lookup field over standard SharePoint Lookup field:

• Auto Complete or Type Ahead lookup search (queries after each typing)
• OTB SharePoint Lookup Dropdown look and feel for the results 
• Proper validations
• Searches the lookup keyword with Contains CAML query
• Configurable for No.of results to be displayed
• Configurable for No.of characters to start the auto complete search

Limitations:

• It cannot be multi-valuated
• Only one AutoComplete Lookup field is allowed per list
• The target lookup field is frozen and always the Title field

The project is packaged as a Windows installer using the SharePoint Solution installer; to install, simply run setup.exe. The software is available on CodePlex here.

Please do let me know if you find any issues or bugs or useful enhancement that could be added.

SharePoint ListItem Update vs SystemUpdate vs UpdateOverwriteVersion

Here is a simple table format to explain the differences among Update vs SystemUpdate vs UpdateOverwriteVersion vs SystemUpdate(true). 

This information is pretty much useful to look at the differences between the following SPListItem methods when working with event receivers or workflows.

	Update()	SystemUpdate()	SystemUpdate(true)	UpdateOverwriteVersion()
Updates the item in the database	ü	ü	ü	ü
Creates a new version	ü	û	ü	û
Updates the Modified and Modified By values	ü	û	û	ü

How to update Modified By field in a SharePoint list item programatically

While adding/updating a list item using SPSecurity.RunWithElevatedPrivileges with administrator privileges, Created By / Modified By fields are set to System Account as the piece of code is executed with Application Pool account context.

But in an ideal case Created By / Modified By fields should have the original user who actually did the modification. In order to achieve this Created By / Modified By fields needs to be updated at the time of inserting / updating list item.

The catchy point here is to get the Current User out of the SPSecurity.RunWithElevatedPrivileges scope because within the scope it is always System Account as mentioned.

The below code snippet works as as explained above:

////Get the current user out of the SPSecurity.RunWithElevatedPrivileges scope
SPUser currentUser = SPContext.Current.Web.CurrentUser;

SPSecurity.RunWithElevatedPrivileges(delegate()
{
    using (SPSite newSite = new SPSite("http://siteurl"))
    {
        using (SPWeb newWeb = newSite.OpenWeb())
        {
            SPList list = newWeb.Lists.TryGetList("List Title");
            if (null != list)
            {
                bool isNewItem = true; //Adding or Updating item
                SPListItem item = list.GetItemById(1);
                if (isNewItem)
                {
                    item["Author"] = currentUser;
                }
                item["Editor"] = currentUser;

                item.Update();
            }
        }
    }
});

Reference:
How to update Modified By field programatically
How to update Created By field programatically
SharePoint List Item update Modified By field programatically
SharePoint List Item update Created By field programatically

Unable to Create a Search Managed Property

While i was trying to create a new Search Managed Property against a crawled property using either PowerShell or UI I am gettin an error "There was an internal problem connecting to or querying the database".

I spent enough time in figuring out the problem. At last I have identified that the Search Service Application doesn't have a valid Administrator who has access to SSA database.
Note: The mappings will be created in DB MSSManagedProperties table.

Administrator can be added by going to Central Administration --> Manage Service Applications --> Select Search Service Application --> Click on Administrators (Ribbon) --> Add an user with Full control (who has proper privileges to DB).

This process will provide the needful privileges to SSA to add the new search managed properties.

Check User exists in SharePoint Group

I was looking for direct API method to check whether specified user exists in SharePoint group. Unfortunately there is no direct API check.

I found below single statement is efficient for this check. I made it a generic for re-usability.

SPGroup spGroup = web.Groups["Group Name"];
if (null != spGroup)
{
  SPUser spUser = web.EnsureUser("Login Account");
  if (null != spUser)
  {
    bool userExsists = spUser.Groups.Cast<SPGroup>().Any(g => g.Name.ToLower() == spGroup.Name.ToLower());
     if (!userExsists)
     {
        spGroup.AddUser(spUser);
     }
  }
}

Migration SharePoint 2007 to SharePoint 2010

As we know SharePoint 2010 has significant user experience, performance, and other improvements, most of the organizations with existing SharePoint 2007 applications are planning to upgrade to SharePoint 2010. New hardware and software requirements, architectural changes, and UI changes in the product will require solid migration and testing plans to ensure these upgrades proceed smoothly.

My goal of this migration article is to illustrate step by step migration from SharePoint 2007 to SharePoint 2010.

Here I am going to describe the migration process steps for the below scenario which exists in SharePoint 2007 environment.

Scenario: Migrating Team Site with below content/functionalities/custom solutions

• Ø  Regular Team Site
• Ø  Team Site with sub-sites
• Ø  Team Site with list template
• Ø  Team Site with sub sites using Site Template
• Ø  Team Site with sub sites using Site Template + Custom WP solution
• Ø  Team Site with sub sites using Site Template + Custom WP solution + Custom Content Types

Upgrade and Migration for SharePoint Server 2010

Before you run any process to upgrade from Microsoft Office SharePoint Server 2007 to Microsoft SharePoint Server 2010, you have to determine which upgrade approach to take from the below two major approaches. The Article Upgrade process overview (SharePoint Server 2010) gives the more information possible approaches for migration.

• 1.      In-place
• 2.      Database Attach

The article Determine upgrade approach (SharePoint Server 2010) helps in comparing the pros and cons for each approach and to review information about special cases that might influence your approach. In addition to this information, be sure to read Review supported and unsupported upgrade paths (SharePoint Server 2010) to understand exactly which upgrade situations are valid and lead to successful upgrades.

Here in my case I am using the Database Attach Approach. The Source and Destination servers are different. The below describes the Step-by-Step Migration Process.

1.      Check Pre-Requisites for upgrading from SharePoint 2007 to 2010

1.       Verify the Supporting Topology

2.       SharePoint 2007 with SP2 environment

3.       A web application with above scenario in SharePoint 2007 environment

4.       SharePoint Server 2010 environment

5.       A web application with no site collection in SharePoint 2010 environment

2.      Run the pre-upgrade check

Note: To perform an upgrade, you must have installed Office SharePoint Server 2007 with Service Pack 2 (SP2).

The pre-upgrade checker is a STSADM operation that you run in a Microsoft Office SharePoint Server 2007 environment to find any potential issues for upgrade and to review recommendations and best practices.

STSADM -o preupgradecheck

In my environment, I got the following issues after running the preupgradecheck.

Issue 1: This server machine in the farm does not have a 64 bit version of Windows Server 2008 SP2 or higher installed.

Description: This issue is related to hardware, software, pre-requisites of SharePoint 2010 server.

Issue 2: Orphaned site collections

An orphaned site collection is a site collection exists in the content database, but it is not available in the configuration site map(CA/UI). Such site collections are not accessible and will not be upgraded properly. The following orphaned site collections were found.

/sites/MyTest (Data Source=WS2003\OfficeServers; Initial Catalog=WSS_Content_80; Integrated Security=True; Enlist=False; Connect Timeout=15)

Description: This issue is related to MyTest Site is corrupted in SharePoint 2007 server.

Issue 3: Missing server file or server configuration issues

Description: This issue is related to custom web parts, event handlers, and features etc.

Note: The generated report is available in LOG files.  

3.      Clean up the Environment:

Resolution of Issue 1:

Resolve all the environment issues mentioned in Issue 1.

Resolution of Issue 2:

·         Verify whether the reported orphan site collection exists in the specified web application through CA (UI).

Then verify whether the same orphaned site collection exists in content database using the below STSADM command

STSADM -o enumallwebs -databasename  <<name of the content database>>

It gives the count of the site collections in content database and lists all the site collections and webs inside with IDs. The orphaned site collection should be found here.

·         The orphaned site collection Sites/MyTest is deleted in content database using below STSADM command

STSADM -o deletesite -force -siteid <<site guid>> -databaseserver <<database server>> -databasename <<name of the content database>>

·         RECOMMENDATION: After deleting the orphaned site collection, it is recommended to repair the content database for any schema related issues using below STSADM command.

STSADM -o databaserepair -url <<site collection url>> -databasename  <<name of the content database>>

Note: The siteid is copied from the STSADM command prompt after running enumallwebs command.

Resolution of Issue 3:

This issue is related to custom solution. Lists all the missing web parts, features, content types etc., in content database; Make sure all the custom solutions are deployed to the farm.

Note: Run the preupgradecheck command again after resolving all the listed issues to see if there are any issues still.

4.      Backup Content Database -SharePoint 2007 Environment 

1.      Set the previous version  content database to be read-only

In any type of database attach upgrade, you can set the database to read-only temporarily to ensure that you capture all the data in the backup so that you are restoring and upgrading the current state of the environment. If the database are set to read-only, users can continue to view content, but they will be unable to add or change content.

2.      Back up the previous version databases by using SQL Server tools

5.      Restore Content Database - SharePoint 2010 Environment 

1.      Restore a backup copy of the database (database attach with read-only databases)

After you configure the new server farm (SharePoint 2010), you can restore the backup copies of the database new environment.

2.      Set the databases to read-write (database attach with read-only databases)

You must also set the database back to read-write before you attach and upgrade them.

6.      Verify Custom Components 

Before you attach the content database to the Web application, use the 

Test –SPContentDatabase Windows PowerShell cmdlet to verify that you have all the custom components that you need for that database against the web application.

Test-SPContentDatabase -Name <<restored content database name>> -WebApplication <URL>

Ref: Verify Custom components.

7.      Attach a Content Database to a Web application 

You can use either the Mount -SPContentDatabase cmdlet in Windows PowerShell or the addcontentdb Stsadm command to attach a content database to a Web application.

Note: Using the SharePoint Central Administration pages to attach a content database is not supported for upgrading.

PowerShell Cmdlet

Mount-SPContentDatabase -Name <<restored content database name>> -DatabaseServer <<database server>> -WebApplication <<URL>> [-Updateuserexperience]

 STSADM command

STSADM -o addcontentdb -url <<web application url>> -databasename <<content database name>> [-preserveolduserexperience]

Note: Updateuserexperience & preserveolduserexperience are optional and used for visual upgrade respectively, which applies the 2010 look and feel for 2007 pages.

After you have attached a database, you can use the Upgrade Status page in Central Administration to check the status of upgrade on your site collections. After the upgrade process is complete, you can review the upgrade log file to see whether there were any issues during upgrade. Also, you can review each upgraded site to find and address any issues with how the content is displayed. For more information, see Verify upgrade and review upgraded sites (SharePoint Server 2010).

To view the Upgrade Status page:

In Central Administration, click Upgrade and Migration, and then click Check upgrade status.

8.      Visual Upgrade 

Once the content database is restored, attached to a web application and the site is accessed through browser, the site holds the 2007 look and feel if user experience operation is not used in PowerShell/STSADM while attaching content database.

To apply the 2010 look and feel to the restored sites follow the below simple approach.

Logon to SharePoint 2010 team site; click on Site Settings->Visual Upgrade, select the update the user interface radio button, which will update the SharePoint 2010 interface.

9.      Verify Successful Upgrade 

Run the below PowerShell command to check whether the upgrade/migration is successful

Test-SPContentDatabase –Name <name of the content database > -WebApplication <web application url>

It generates a report of missing custom features, web parts and event receivers….etc.

10. Deploy Custom Solution 

1.      Add & deploy the custom solution to SharePoint 2010 restored site using PowerShell or STSADM.

2.      Activate the content type and webpart features in SharePoint 2010 restored site.

11. Again Verify Successful Upgrade 

Run the PowerShell command as follows to check for successful upgrade

Test-SPContentDatabase –Name <database name> -WebApplication <URL>

      This time it should be successful, no warning/errors.

I hope this post is useful in manual migration.