SharePoint 2010 vs SP2010 + FAST vs SP 2013 Search Capabilities Comparison

I put together this article to highlight the enterprise search capabilities that are provided in SharePoint Server 2010, FAST Search Server 2010 for SharePoint, and what’s new in SharePoint 2013. Info presented is tailored towards the search experience desired in typical environments, focusing on key elements and functionality.

O = available † = improved in 2013

Features and Capabilities SharePoint Server 2010 FAST Search Server 2010 for SharePoint SharePoint Server 2013 Enterprise
Keyword Query Language
Keyword Query Language (KQL) KQL is the default query language for building search queries. Using KQL, you specify the search terms or property restrictions that are passed to the SharePoint search service.
O O O
FAST Query Language (FQL)
FAST Query Language (FQL) syntax and Keyword Query Language (KQL) syntax for advanced search queries.
O O
Directly Integrated into SharePoint
Meet the scale and performance needs of your entire organization or the specialized needs of individual departments.
O
Search Analytics
The search system determines the relevance of search results in part by how content is connected, how often an item appears in search results, and which search results people click.
O
List/Library Inline Search
Search input box that focuses on current SharePoint List/Library, negating the need for end users to “Ctrl-F” search via their web browser – search all items in current list through the convenient additional search box.
O
Search from Mobile Devices & Tablets
Search beyond the search center. Conduct searches from the Windows desktop and on your mobile device.
O O O
Continuous Crawling
Setting this option eliminates the need to schedule incremental crawls and automatically starts crawls as necessary to keep the search index fresh.
O
Display Templates
Create completely customizable templates for display of search results,
Taxonomy tag integration
Bring the power of taxonomy into search. Tag metadata is shown in results, and users can refine by taxonomy-based tags.
O O O
Metadata-driven refinement panel
With the refinement, users can narrow the results of their searches and navigate to the right content faster.
O O O
Discover structure and entities in unstructured content
You can store these entities in your search index as separate managed properties and use those properties later — for example, in search refiners.
O
Phonetic and nickname search
Confidently search for a person’s name as it sounds – without worrying about the exact spelling.
O O O
Search Diagnostics
You can access and analyze several query and crawl health reports, logs and usage reports from the Search service application in the SharePoint 2013 Central Administration to monitor the health of the search system.
O
Contextual search
Tailor different results and refinement options based on the profile of the user or audience.
O O
Thumbnails and previews
Thumbnails and previews make the results of a search query visual, allowing users to recognize the right content quickly.
O O
> 500m content volume with sub-second query response time
Scale to extremes with FAST Search Server 2010 for SharePoint & Integrated FAST in SharePoint 2013 while maintaining sub-second query response times.
O O

I have also attached several documents which provide in-depth detail on the products described;

Getting Started with Enterprise Search in SharePoint 2010 Products : General Enterprise Search Guidance for SharePoint 2010
oit2013-model-sharepoint-search-architecture: Search Architectures for SharePoint 2013 diagram
sp-2013-enterprise-search-model: Enterprise Search Architectures for SharePoint 2013, includes hardware specs.

SharePoint Search Licensing Overview

SharePoint licensing is a complex subject. As well as there being multiple product configuration options, license prices often depend on a number of factors such as the type of organization, the relationship with Microsoft and/or licensing retailer etc.

As a general rule, if you are interested in giving all internal users access to FAST search via SharePoint 2010 Enterprise, you are looking at roughly twice the licensing costs of an installation using the standard features. In SharePoint 2013 Enterprise, FAST functionality is included in the Enterprise CAL’s. FAST Search for SharePoint 2010 is licensed separately so to take advantage of all the enterprise search features you will need additional server licenses for servers running the FAST Search software.

It is also possible to use a single server for any of the above server licenses. For example if you wanted to run an Intranet and a public Internet website on the same server you could apply the SharePoint 2010 Server license as well as the SharePoint 2010 Internet Sites license to a single server. A FAST license can also be applied to a SharePoint 2010 server if required (saving hardware costs but still requiring SharePoint Server and FAST licenses).

itgroove can offer specific license recommendations when the SharePoint version platform is decided.

SharePoint 2013 Search Boundary Key Changes
As perhaps a trade-off for the tight integration of FAST functionality into SharePoint 2013, there are some significant new restrictions in topology that should be known, as they affect search architecture planning:

Limit SP 2010 SP 2013
Crawl Databases 10 per Search Service Application 5 per Search Service Application
Crawl Components 16 per Search Service Application 2 per Search Service Application
Index Partitions 20 per Search Service Application 20 per Search Service Application
Link DB N/A 2 per Search Service Application
Query Processing Component N/A 1 per server
Content Processing Component N/A 1 per server
Analytics Processing Component N/A 6 per Search SA

Although most organizations should be able to meet their immediate requirements for Search with a single Search Service Application (ideally on a single, dedicated SharePoint web application), these limitations should be noted for future expansion. This design sample shows the logical architecture and interaction of search components, and an example of a fault-tolerant server farm that provides enterprise search for a content corpus that contains approximately 40 million items:http://go.microsoft.com/fwlink/p/?LinkId=259181

Topology Changes from SharePoint 2010

When designing your topology, it is important to understand what has changed since SharePoint 2010 Search and the performance of the components. Due to the technical changes some of them need to be ideally separated onto separate servers when scaling up or if you have a large search user base. In a small to medium farm of 2 to 4 SharePoint servers in SharePoint 2010, the recommendation was to place the query component on the web front end servers. In a medium sized search orientated farm it was recommended to have three physical tiers consisting of separate web front end servers, query servers and crawl servers.

In SharePoint 2013, even with a small farm of two SharePoint servers it is recommended to place the query services on a separate server from the web front end server. The query and crawl services can run together on a separate server. This is applicable for up to four SharePoint servers but once you increase the number of servers, it is recommended to split out the query and crawl components onto their own servers and leave the remaining service applications together.

Here’s the official guidance on modifying your SharePoint Server 2013 Search Topology:
http://technet.microsoft.com/en-us/library/jj862356(v=office.15).aspx

Here’s a couple more great posts from two stand-up guys on how to modify your search topology:
Changing the SharePoint 2013 Search Topology
New-SPEnterpriseSearchIndexComponent could not find part of the path error

How you structure your architecture depends on what you intend to use it for, be it enterprise or for the internet. There are a number of considerations to take into account such as high availability, fault tolerance, the amount of content and the number of queries you will receive.

The query processing component takes most of the load off the SQL server including disk space load and CPU, it therefore requires more local resources than in SharePoint 2010. The query processing component ideally, therefore, should be on its own dedicated server. The index component is equally resource hungry, because it performs two roles in communicating to the query processing component and the content processing component.

The index is the main store of information from the crawled content and can be split into multiple partitions to scale up and provide fault tolerance. Another thing to consider, if you are crawling a large amount of data, it is advisable to have a dedicated web server for crawling. The diagram below is an extract from a TechNet article on topologies, which shows the optimum environment for search oriented environment:
search index

Technical Diagrams for SharePoint 2013 Search

Title Description
SharePoint Server 2013 SearchIC663131
Zoom into the model in full detail with Zoom.it from Microsoft (best on desktop or laptop computers)Visio version (best for users with Visio)PDF version (best for mobile devices or tablet computers)
This design sample shows the logical architecture and interaction of search components, and an example of a fault-tolerant server farm that provides enterprise search for a content corpus that contains approximately 40 million items. For more information, see the following articles:

Enterprise Search Architectures for SharePoint Server 2013IC673360
Zoom into the model in full detail with Zoom.it from Microsoft (best on desktop or laptop computers)Visio version (best for users with Visio)PDF version (best for mobile devices or tablet computers)
This design sample illustrates small, medium, and large-size farm architectures for enterprise search. It contains search and farm topology examples, scaling guidance, and hardware requirements. For more information, see Overview of search in SharePoint Server 2013.
Internet Sites Search Architectures for SharePoint Server 2013IC663133
Zoom into the model in full detail with Zoom.it from Microsoft (best on desktop or laptop computers)Visio version (best for users with Visio)PDF version (best for mobile devices or tablet computers)
This design sample provides guidance on the hardware requirements and performance considerations for Internet Sites search topologies. In addition, the model contains an example medium-sized Internet Sites farm. For more information, see Plan for Internet, intranet, and extranet publishing sites in SharePoint Server 2013.

New-SPEnterpriseSearchIndexComponent could not find part of the path error

When attempting to reorganize your SharePoint 2013 topology you may also want to specify a new path for your search Index. The reason for this might be that you want to have a dedicated drive for a large or busy search index. You can find the general technique for modifying the topology here: http://mmman.itgroove.net/2013/03/changing-the-sharepoint-2013-search-topology/

Looking up the involved New-SPEnterpriseSearchIndexComponent at http://technet.microsoft.com/en-us/library/ff607766(v=office.14).aspx we can see the new file path is designated by the -IndexLocation switch. An example would be:

$serv = "MYSERVERNAME"
$NewIndexPath = "E:SharePointIndex"
New-SPEnterpriseSearchIndexComponent -SearchTopology $newTopology -SearchServiceInstance $serv -RootDirectory $NewIndexPath

But wait- if the destination path does not exist on the source server, it will error out with “Could not find part of the path”. At first I thought it was permissions, but having checked that the new destination folder had appropriate permissions (WSS_WPG with modify permissions), I was still scratching my head. It became clear this was a bug, as described on http://office.microsoft.com/en-ca/help/sharepoint-server-2013-known-issues-HA102919021.aspx :

“New-SPEnterpriseSearchIndexComponent checks the existence of RootDirectory in the wrong server

You want to add a new index component to the search topology, and want to specify a non-default root directory for the index files. For example:

**Keith Note: Do not use this PowerShell, it has typos quoted as-is from TechNet. Provided for reference only:

New-SPEnterpriseSearchIndexComponent -SearchTopology $t -SearchServiceInstance $ssi -IndexPartition 1 -RootDirectory ""d:index4"

The cmdlet might fail with the following error message because it incorrectly checks if the indicated root directory exists on the server the cmdlet is run on:

Cannot bind parameter ‘RootDirectory’ to the target. Exception setting “”RootDirectory””: “”Could not find a part of the path ‘d:index4’.”

Workaround You can create the new index component using the following procedure instead:”

$host02 = (Get-SPServer "<Name of server>").Name
$ssa = Get-SPEnterpriseSearchserviceApplication
$t = $ssa.ActiveTopology.Clone()
$ic = (New-Object Microsoft.Office.Server.Search.Administration.Topology.IndexComponent $host02,1);
$ic.RootDirectory ="d:index4"
$t.AddComponent($ic)

Note: In the code snippet on the MS bug report, there are three typos (fixed in the PowerShell shown above)

1. missing space between SP-GetServerName and parameter
$host02 = (Get-SPServer””).Name
should be
$host02 = (Get-SPServer “”).Name

2. double quotes in beginning of the path declaration:
$ic.RootDirectory =””d:index4″
should be:
$ic.RootDirectory =”d:index4″

3. extra quote at the end:
$t.AddComponent($ic)”
should be:
$t.AddComponent($ic)

Note 2: You must permit TCP port 808 through your Windows Firewall on the source and destination servers or the topology move in general won’t work!

Roll Up SharePoint 2013 Tasks with Content Search Web Part

It’s exciting to see MS Project integration in SharePoint 2013 has been taken to a whole new level – Project is now treated as a first class citizen, and this means a lot for a shop like us who manage all our issues and projects via SharePoint.

I recently was tasked to roll up Project tasks from across our portal into one view- and was pretty fired up to learn that this requirement (something that would have previously required custom code) can now be accomplished with the new out of the box SharePoint 2013 Content Search Web Part.

About the Content Search Web Part

From the description of the Content Search Web Part in the SharePoint ribbon bar:

“Content Search Web Part will allow you to show items that are results of a search query you specify.
When you add it to the page, this Web Part will show recently modified items from the current site. You can change this setting to show items from another site or list by editing the Web Part and changing its search criteria.
As new content is discovered by search, this Web Part will display an updated list of items each time the page is viewed.”

Actually, the declaration “this Web Part will show recently modified items from the current site.” is not untrue but it’s not revealing the whole truth: with this web part you can actually leverage content from anywhere your search crawler goes – it’s not just limited to content on the current site where the web part resides. With the tight integration of SharePoint Search and FAST for SharePoint Search in SharePoint 2013, Microsoft added the Content Search Web Part to address the issue of not being able to cross site collections for the content aggregation. In fact, the Content Search Web Part will display all content available in the search index, regardless if it is SharePoint content or not! (As long as viewers have read rights to the content)

The Content Search Web Part can be found in the Content Roll-up category of the Web Part Gallery.

Our Requirements

In this use case of the Content Search Web Part, our goal is to show each users all their Project Tasks from across the SharePoint 2013 farm, in a list-style view. We do not want to show them Tasks that are currently in a “Completed” status.

Configure the Content Search Web Part

1. While on a SharePoint 2013 page using an account with edit privileges, click Edit Page > Insert > Web Part > Content Rollup > Content Search:

1
2
3
2. This will add a stock Content Search Web Part to the page. Click Edit Page > Edit Web Part to get the Web Parts Configuration Panel to display:4
5
3. Change the value of Select a query to Items matching a content type (System). Select the Task content type.
4. Change the value of the Restrict By App dropdown from the default “Current Site” to “Specify a URL” and then choose the root URL of your SharePoint web app (in our case, it’s https://go.itgroove.ca):
15
5. Click on “Switch to Advanced Mode“. On the Advanced Mode screen we can see the guts of our query – our query will look like this at this point (with our itgroove URL added):
path:”https://go.itgroove.ca”   -Status:Completed path:”https://go.itgroove.ca”  ContentTypeId:0x0108*
6. In the Keyword filter dropdown, select “Name of the user who runs the query” and click the “Add keyword filter” button, so you get the {User.Name} token added to the query text:
8
7. Move your cursor to the end of the current query text (with an extra space on the end). In the Property Filter dropdown, click “Show All Managed Properties”. This one is a bit sneaky as the dropdown will retract. Open the dropdown again and you will have a whole pile of Managed Properties to work from. Find and select the “Status” managed property.
8. Change the Operator dropdown (by default set to “Equals”) to Not Equals and the Select Value dropdown to “Manual“. Type in the word Completed into the input. When all this is done your query should look like this:
18
9. That’s it for the Build Your Query window – you can optionally tweak some of the other settings like sorting and ranking but that’s up to you and beyond the scope of this tutorial. Press OK to be returned to the web part’s configuration panel.
10. Under Number of Items to show, change the value to a max of 50.  This is not your overall record result count to show, but rather the number of items to display on one paginated page, if you choose the Control “List with Paging”. So if you choose 5 for example, this will show 5 records on the page, for which users can use the pagination arrows to navigate to the next 5 with. I’m not aware of a theoretical limit on maximum results you can have, it’s likely based on overall search limits.
11. Under the Display Templates section of the Web Part Configuration panel, change the Item dropdown to the value “Two Lines”. This will give is the simple two line classic list-style presentation of the results as opposed to the default image template. You can of course customize these templates visual styles and structure at will, with simple HTML and JavaScript. You can find some more on how to do that here.
12. Your Web Part output will now (assuming you of course have some SharePoint Task Lists populated with data) look like this:
21

Configure Status and AssignedTo in Search Service Application Managed Properties

13. But wait! We ain’t done yet – out of the box, it’s not going to filter the Status property filter, meaning that our current filter of -Status:Completed is having no effect. This is because by default the Task Status field is not enabled to be included in the search results (which are what, after all, the data for this web part is derived from). What we need to do is go to the Central Administration > Application Management > Manage Service Applications > Search Service Application > Queries and Results > Search Schema  page. Type in “Status” into the Filter input and click the green arrow button:

22
14. Click on the dropdown on the “Status” property and click “Edit/Map Property”. Ensure that the Searchable and also Allow Multiple Values checkbox is on – this will let you return the results from a Choice field. You’ll notice in our case the Mappings to crawled properties are mapped to the internal field names ows_Issue_x0020_Status and ows_Task_x0020_Status:
24
The output will look as so:
27
15. Save this and return to the main Managed Properties page. Enter “AssignedTo” in the filter and locate that Managed Property. Again, make sure Searchable and also Allow Multiple Values are checked:
26
16. Well, it took a bit to get to this point, but once you have the hang of this it’s not so painful. On the next search crawl update (you can kick it off manually if you just can’t wait), your Task rollup list will now show up, as desired, minus the Tasks that are Completed.

A special shout out is in order to Yaroslav Pentsarskyy for getting me going with the Content Search Web Part at http://www.youtube.com/watch?v=_7hVH2Z6p_k&feature=youtu.be