SharePoint Online Search Word Breaking

Recently I was asked if search in SharePoint Online would return two results for words separated by hyphens, dashes, or other characters. For example, we’d like to know if  entering the search phrase “committal” will return both “noncommittal” and “non-committal”

The answer is, it depends on the specific Search scenario and configuration. It’s indeed an under-documented topic so I thought I’d try and give you good context here.

  • SharePoint Online search “verticals” (Result Sources) determines the source of the search and characteristics of the query logic. We can also create Custom Result Sources.
  • Out of the box Result Sources, are mapped to Managed Properties, which are in essence “columns” of search data. As with Result Sources, there are out of the box managed properties, and we also can create our own.
  • The Search Query component tokenization process splits the stream of text retrieved from the managed properties into individual words (tokens) at the time of a query which includes word breaking, stemming, query spellchecking and the native thesaurus capabilities.
    • This tokenization will only take place if the following setting on the specific Managed Property being searched is turned off:
    • Since we cannot directly modify the out of the box Managed Properties, we are bound to whatever those properties have set for Complete Matching, as the first factor in Word Breaking being applied or not.

Here’s the inventory of the special characters used for tokenization in the context of using SharePoint Document Libraries as the Search Result source:

Character Is Wordbreaker?
– hyphen Yes
_ underscore Yes
. period Yes
& ampersand Invalid
% percent Invalid
+ plus Yes
# pound Invalid

There are other possible root content source types as well: File Share, Exchange and Open Search, and they have their own idiosyncrasies regarding word breaking. For example, ampersand would work as a word breaker in the context of a File Share search.

So, in conclusion, yes, a search of OOTB SP Document Libraries for “committal” will return both “noncommittal” and “non-committal”, with higher relevance given to an item with both portions of the string present.

Microsoft Ignite Conference: Day 4 Round-Up

Last day of the MS Ignite conference for me. Let’s take it to the bridge:

Another metaphor? You betcha:

– Cloud vs on-premise
– Microsoft as Service Provider vs Shrink Wrapper
– Unified experiences vs Diversified devices
– People-driven Intranets vs Intranet-driven people
– Windows OS as a true personal & biz life and productivity platform vs Windows OS as a 900 pound gorilla

My top picks for Day 4 in the SharePoint/Office 365 realm (with a little .NET 5 in there to maintain my Dev street cred):

My last-day sessions:

JEA: A PowerShell Toolkit to Secure a Post-Snowden World

When asked what to do about corporate hacking, Ex NSA Director Michael Hayden replied, “Man up and defend yourselves.” Edward Snowden then rocked the world by disclosing privileged NSA information. JitJea stands for “Just In Time, Just Enough Admin.” It’s a Windows PowerShell toolkit that admins use to perform functions without giving them admin privileges.

MVP Panel: SharePoint On-Premises, Online and Everything in Between
Imagine five great minds coming together to talk about Microsoft SharePoint across the board, be it within Microsoft Office 365, in Microsoft Azure, on-premises and certainly hybrid. Via a panel Q&A format, these MVP experts expose how online and hybrid improvements increase both deployment scenarios and value. This session is designed to help ITIs and ITDMs find the right cloud formula to deploy based on practical business and technical considerations. This is a must-not-miss session for any IT pro!

How to Decide When to Use SharePoint and Yammer and Office 365 Groups and Outlook and Skype
Your users may struggle with these questions: Should I share a message via Skype for Business instead of Yammer, Office 365 Groups, or Exchange? Should I collaborate on data using an Excel sheet or a SharePoint list? Should I share a file in Outlook, in a meeting, from OneDrive for Business, on Yammer, in a Group, or in a SharePoint site? This session is the ‘How To’ user’s guide What happens when your users can’t decide what technology or feature to use? They use what they know, or what’s easy; even if better options exist. In this session, Richard and Kanwal help you maximize the value of your Office 365 investment by providing the guidance you need to help your users make better, more effective decisions on how they get work done.

Experts Unplugged: Office 365 Security

OneNote for OneLife: From Notes to Productivity and Platform
OneNote is awesome. Really. Knowledge, learning and info on the bleeding edge benefits from structure – but not too much structure. We all need a place to inscribe understanding, without having to go overboard on the word-processing end of things.  I’m always thrilled to hang out with the OneNote crew as I sometimes wonder if they know they have the future of Education and Wikis in their hands.

Microsoft OneNote gives you one place for your notes and other content with you, anywhere now that OneNote is across all platforms and devices. Write by hand, type, record, snap a picture, clip from the web, or use a growing number of other partner apps and devices and OneNote saves it. Organized or not, you can easily find your notes in any form (text, writing, picture, or audio) with OneNote search. This session demonstrates end-user productivity scenarios at work to give you a clear understanding of how OneNote can help drive adoption of Office 365 with cross-platform, real-time collaboration, and extensibility with OneNote API. You’ll walk away wanting to use, evangelize, and build on OneNote personally or for your organization.

Microsoft Ignite Conference: Day 3 Round-Up

I’ll kick off Day 3’s post with another imprint of pure experience. Scale- On-premise, Cloud, Global, Local. The rush/distraction/tunnel vision of being one of 23,000+ people moving through the Microsoft Ignite conference, contrasted with simple but essential logistics like bio-breaks and food, are a big parallel for me to the distinct juncture we are at in technology:  empower everyone at a mass scale, but make sure the human details are taken care of, and that everyone has a voice.

The entrance hall may resemble a slightly above average shopping mall scene for most, but in context, on the ground, it was more like the entrance to a spaceship waiting to take off:

Existential experiences aside, I waited a full 5 minutes for cell phone guy to abandon his hostile takeover of Microsoft (pleading eye contact included), and finally realized that was 5 minutes I would never get back, so I snapped my obligatory “largest Expo Hall ever” pic:
Really, words don’t help much with describing the scale of this event. 23,000 of the world’s finest Microsoft-oriented IT professionals in not one, but two Conference centers daisy-chained together. Being from Canada, the SCALE of business in America is always impressive-  this time it was the hammer of Thor (axe of Abe Lincoln?). There were numerous, well-attended core educational/interactive groups with all the best of the Microsoft team providing direct interaction with attendees. These were no tradeshow stunt doubles, but really the actual program leads and people who make things move at Microsoft. Super high quality interactions all over the floor.

Office 365

TechNet is my bible, which would make Joanne & KC here (Senior Content Writers for Microsoft), pretty high up in the toga-wearing department:
TechNet Rocks

Aside from the separate, colossal pool of core Microsoft and Partner & Vendor talent present in the Expo Hall, here’s the top sessions from day 3, on the SharePoint/Office 365 tip (with some guest appearances from OneNote & Visio, as I love both):

Whats New for IT Professionals in SharePoint Server 2016
What's New for IT Professionals in SharePoint Server 2016

“Engineering paths directly influenced by SP Uservoice” See:
“Durable Links- permalinks based on resource ID. Move Docs freely, URL stays the same”
“No downtime CU patching”
“OneDrive integration big priority for Engineering team”


This article describes initial investments made in installation and deployment of SharePoint Server 2016:

MinRole for the win!!

Embrace the BYOD Revolution: Effectively Manage a Multi-Device, Multi-Generational Workforce

A major business transformation is brewing in the enterprise today. Mobile technologies, business velocity, geographically dispersed and multi-generational workforce are converging to deliver the promise of responsive organizations. Organizations that miss this paradigm shift will face dire consequences. How can you effectively manage this shift, ensure that it will be sustainable and reap the benefits of being a responsive organization? In this session, learn how to apply practical steps and effective techniques to manage your multi-device and multi-generational workforce.

MVP Panel: Sample Apps and Intelligent Solutions Showcasing Office Graph and Delve Extensibility

Preparing for a meeting, but not sure what documents are relevant? Writing a proposal and looking for similar documents to help you out? Interested in what your colleagues are working on to stay updated? With the new Office Graph, answers to those questions are within your reach. In this demo-packed session, we show you how the Office Graph works and how it can be used when building custom apps and enriching existing solutions and portals. All scenarios are backed up by real-life solutions that you could use in your organization.

SharePoint 2010 vs SP2010 + FAST vs SP 2013 Search Capabilities Comparison

I put together this article to highlight the enterprise search capabilities that are provided in SharePoint Server 2010, FAST Search Server 2010 for SharePoint, and what’s new in SharePoint 2013. Info presented is tailored towards the search experience desired in typical environments, focusing on key elements and functionality.

O = available † = improved in 2013

Features and Capabilities SharePoint Server 2010 FAST Search Server 2010 for SharePoint SharePoint Server 2013 Enterprise
Keyword Query Language
Keyword Query Language (KQL) KQL is the default query language for building search queries. Using KQL, you specify the search terms or property restrictions that are passed to the SharePoint search service.
FAST Query Language (FQL)
FAST Query Language (FQL) syntax and Keyword Query Language (KQL) syntax for advanced search queries.
Directly Integrated into SharePoint
Meet the scale and performance needs of your entire organization or the specialized needs of individual departments.
Search Analytics
The search system determines the relevance of search results in part by how content is connected, how often an item appears in search results, and which search results people click.
List/Library Inline Search
Search input box that focuses on current SharePoint List/Library, negating the need for end users to “Ctrl-F” search via their web browser – search all items in current list through the convenient additional search box.
Search from Mobile Devices & Tablets
Search beyond the search center. Conduct searches from the Windows desktop and on your mobile device.
Continuous Crawling
Setting this option eliminates the need to schedule incremental crawls and automatically starts crawls as necessary to keep the search index fresh.
Display Templates
Create completely customizable templates for display of search results,
Taxonomy tag integration
Bring the power of taxonomy into search. Tag metadata is shown in results, and users can refine by taxonomy-based tags.
Metadata-driven refinement panel
With the refinement, users can narrow the results of their searches and navigate to the right content faster.
Discover structure and entities in unstructured content
You can store these entities in your search index as separate managed properties and use those properties later — for example, in search refiners.
Phonetic and nickname search
Confidently search for a person’s name as it sounds – without worrying about the exact spelling.
Search Diagnostics
You can access and analyze several query and crawl health reports, logs and usage reports from the Search service application in the SharePoint 2013 Central Administration to monitor the health of the search system.
Contextual search
Tailor different results and refinement options based on the profile of the user or audience.
Thumbnails and previews
Thumbnails and previews make the results of a search query visual, allowing users to recognize the right content quickly.
> 500m content volume with sub-second query response time
Scale to extremes with FAST Search Server 2010 for SharePoint & Integrated FAST in SharePoint 2013 while maintaining sub-second query response times.

I have also attached several documents which provide in-depth detail on the products described;

Getting Started with Enterprise Search in SharePoint 2010 Products : General Enterprise Search Guidance for SharePoint 2010
oit2013-model-sharepoint-search-architecture: Search Architectures for SharePoint 2013 diagram
sp-2013-enterprise-search-model: Enterprise Search Architectures for SharePoint 2013, includes hardware specs.

SharePoint Search Licensing Overview

SharePoint licensing is a complex subject. As well as there being multiple product configuration options, license prices often depend on a number of factors such as the type of organization, the relationship with Microsoft and/or licensing retailer etc.

As a general rule, if you are interested in giving all internal users access to FAST search via SharePoint 2010 Enterprise, you are looking at roughly twice the licensing costs of an installation using the standard features. In SharePoint 2013 Enterprise, FAST functionality is included in the Enterprise CAL’s. FAST Search for SharePoint 2010 is licensed separately so to take advantage of all the enterprise search features you will need additional server licenses for servers running the FAST Search software.

It is also possible to use a single server for any of the above server licenses. For example if you wanted to run an Intranet and a public Internet website on the same server you could apply the SharePoint 2010 Server license as well as the SharePoint 2010 Internet Sites license to a single server. A FAST license can also be applied to a SharePoint 2010 server if required (saving hardware costs but still requiring SharePoint Server and FAST licenses).

itgroove can offer specific license recommendations when the SharePoint version platform is decided.

SharePoint 2013 Search Boundary Key Changes
As perhaps a trade-off for the tight integration of FAST functionality into SharePoint 2013, there are some significant new restrictions in topology that should be known, as they affect search architecture planning:

Limit SP 2010 SP 2013
Crawl Databases 10 per Search Service Application 5 per Search Service Application
Crawl Components 16 per Search Service Application 2 per Search Service Application
Index Partitions 20 per Search Service Application 20 per Search Service Application
Link DB N/A 2 per Search Service Application
Query Processing Component N/A 1 per server
Content Processing Component N/A 1 per server
Analytics Processing Component N/A 6 per Search SA

Although most organizations should be able to meet their immediate requirements for Search with a single Search Service Application (ideally on a single, dedicated SharePoint web application), these limitations should be noted for future expansion. This design sample shows the logical architecture and interaction of search components, and an example of a fault-tolerant server farm that provides enterprise search for a content corpus that contains approximately 40 million items:

Topology Changes from SharePoint 2010

When designing your topology, it is important to understand what has changed since SharePoint 2010 Search and the performance of the components. Due to the technical changes some of them need to be ideally separated onto separate servers when scaling up or if you have a large search user base. In a small to medium farm of 2 to 4 SharePoint servers in SharePoint 2010, the recommendation was to place the query component on the web front end servers. In a medium sized search orientated farm it was recommended to have three physical tiers consisting of separate web front end servers, query servers and crawl servers.

In SharePoint 2013, even with a small farm of two SharePoint servers it is recommended to place the query services on a separate server from the web front end server. The query and crawl services can run together on a separate server. This is applicable for up to four SharePoint servers but once you increase the number of servers, it is recommended to split out the query and crawl components onto their own servers and leave the remaining service applications together.

Here’s the official guidance on modifying your SharePoint Server 2013 Search Topology:

Here’s a couple more great posts from two stand-up guys on how to modify your search topology:
Changing the SharePoint 2013 Search Topology
New-SPEnterpriseSearchIndexComponent could not find part of the path error

How you structure your architecture depends on what you intend to use it for, be it enterprise or for the internet. There are a number of considerations to take into account such as high availability, fault tolerance, the amount of content and the number of queries you will receive.

The query processing component takes most of the load off the SQL server including disk space load and CPU, it therefore requires more local resources than in SharePoint 2010. The query processing component ideally, therefore, should be on its own dedicated server. The index component is equally resource hungry, because it performs two roles in communicating to the query processing component and the content processing component.

The index is the main store of information from the crawled content and can be split into multiple partitions to scale up and provide fault tolerance. Another thing to consider, if you are crawling a large amount of data, it is advisable to have a dedicated web server for crawling. The diagram below is an extract from a TechNet article on topologies, which shows the optimum environment for search oriented environment:
search index

Technical Diagrams for SharePoint 2013 Search

Title Description
SharePoint Server 2013 SearchIC663131
Zoom into the model in full detail with from Microsoft (best on desktop or laptop computers)Visio version (best for users with Visio)PDF version (best for mobile devices or tablet computers)
This design sample shows the logical architecture and interaction of search components, and an example of a fault-tolerant server farm that provides enterprise search for a content corpus that contains approximately 40 million items. For more information, see the following articles:

Enterprise Search Architectures for SharePoint Server 2013IC673360
Zoom into the model in full detail with from Microsoft (best on desktop or laptop computers)Visio version (best for users with Visio)PDF version (best for mobile devices or tablet computers)
This design sample illustrates small, medium, and large-size farm architectures for enterprise search. It contains search and farm topology examples, scaling guidance, and hardware requirements. For more information, see Overview of search in SharePoint Server 2013.
Internet Sites Search Architectures for SharePoint Server 2013IC663133
Zoom into the model in full detail with from Microsoft (best on desktop or laptop computers)Visio version (best for users with Visio)PDF version (best for mobile devices or tablet computers)
This design sample provides guidance on the hardware requirements and performance considerations for Internet Sites search topologies. In addition, the model contains an example medium-sized Internet Sites farm. For more information, see Plan for Internet, intranet, and extranet publishing sites in SharePoint Server 2013.

SharePoint 2013 Search Query Tool

Found a nice little GUI for visualizing, assembling and debugging SharePoint 2013 Search API queries:

Use this tool to test out and debug search queries against the SharePoint 2013 Search REST API.

Learn how to build an HTTP POST query, and how the different parameters should be formatted.

After running the query, you can view all types of result sets returned; Primary Results, Refinement Results, Query Rules Results, Query Suggestions, in addition to the actual raw response received from the Search service.

Can be used to query both SharePoint 2013 On-Premise, and SharePoint Online.

More about the Search REST API in SharePoint 2013 on the authors blog: and on TechNet.

1-8-2014 8-30-28 AM

Search Crawl Error 8007042d

You may encounter the following Search Crawl log error:

Central Administration -> Application Management -> Manage Service application -> Search Service Application -> Crawling (left side navigation) -> Crawl Log ->

Retrieving the COM class factory for component with CLSID {0FF1CE14-0005-0000-0000-000000000000} failed due to the following error: 8007042d

Go to services.msc from command prompt. Start (or restart) the service “office sharepoint server search” . Give IIS an IISReset for good measure and see if the error persists.

If that fails, then the issue could be caused by lack of permission on one of the service accounts. To see the real error, run the Windows Event Viewer. Click on Run and then type Eventvwr. This particular error fell under the System log. The error was described as:

DCOM had the error “The service did not start due to a logon failur attempting to start the service OSearch with arguments”.

The fix was to start Component Services (Under Administration Tools). Under DCOM config, you will find the OSearch component. First find out what the credentials are by using the Identity tab. If the Service is using a system account, you can either elevate the system account or supply a user with significant rights.

Quick troubleshooting tips:
1. Verify the username and password in the Central Administration App Pool (right click->Properties->Identity[tab])
2. Verify that any services that are running relating to IIS or WSS/MOSS are using valid usernames and passwords.
3. Verify that any services running for SQL Server are using the right username and password.
4. Verify that the account in question has the proper access to the sharepoint databases.
The moral of the story is changing service account passwords wreaks havoc on your SharePoint environment. If you are going to change passwords, there is really only one way to do it.

Synonyms not working in SharePoint Search

The custom thesaurus functionality of SharePoint search should be old hat to those of us experienced with MS Full Text Search, which is a mature technology that’s been around for years. What may trip you up in trying to get custom synonyms working with SharePoint’s implementation is the nuances of the file paths and deployment across multiple physical servers in a farm.

A basic example of the thesaurus format is as follows. This one would match instances of “it” to “information technology”

<XML ID="Microsoft Search Thesaurus">
    <thesaurus xmlns="x-schema:tsSchema.xml">
            <sub>information technology</sub>

First, does your thesaurus file validate as proper XML? Sometimes a forgotten –> closing tag etc. can be a showstopper.

Thesaurus files are stored in several locations on a SharePoint Server:

  Program Files  Microsoft Office Servers  14.0  Data  Config  

Updates to the thesaurus files here will ensure that any new Search Service Applications that you create will have the updated thesaurus definitions when they spin up.

  Program Files  Microsoft Office Servers  14.0  Data  Applications  Config  

GUID is the guid of your Search Service Application, which you can get by using Get-SPServiceApplication. That powershell command will list your service applications and their corresponding GUID’s.

When you’ve verified your updated tsneu.xml files are in those locations, you’ll need to restart the search service with these commands:

net stop osearch
net start osearch

Once the service is restarted, go to your search center and plug in one of the newly added expansion/replacement terms to see your results.

If you’re running search on multiple servers, you will need to perform these steps on each server running search. If there are multiple search applications running, you have to copy your thesaurus files to each config directory under the GUID folder for each search service application.

You can run into the following situation when trying to get a custom thesaurus to kick in for the SharePoint search service:

1. You set up the thesaurus file(s), in our example we’ll use the basic tsneu.xml, which is the language-neutral default thesauraus.
2. You try putting the thesaurus XML file into

%ProgramFiles%  Microsoft Office Servers  14.0  Data  Applications  Config 

and in

%ProgramFiles%  Microsoft Office Servers  14.0  Data  office servers  applications  data  

but with no luck – you don’t see the search results.
3. You try creating a folder on each query/crawl server with the GUID of the search application.
4. Restart the services.

Still no synonyms? It may be that you have partitioned the index, in which case the path changes, and it´s not the default one but the one you set up into the Central Administration > Manage search topology.

Chang the files there, restart the search service and voila!