SharePoint 2013 web crawler doesn't extract links from webpages

Symptom

You discover that the Microsoft SharePoint 2013 web crawler does not extract or process links from certain webpages.

Cause

By default the SharePoint 2013 crawler extracts links only from the following file name extensions: 
.ascx
.asp
.aspx
.htm
.html
.jhtml
.jsp
.mht
.php

Resolution

To check the current settings for filtering extensions, run the following command lines: 

$ssa = Get-SPEnterpriseSearchServiceApplication "Search Service Application"
$ssa.GetProperty("ExtensionsToFilter")
Here's the default output that you'll receive:
;ascx;asp;aspx;htm;html;jhtml;jsp;mht;php;
To add the .shtml extension to the property, run the following commands:

$ssa.SetProperty("ExtensionsToFilter", ";ascx;asp;aspx;htm;html;jhtml;jsp;mht;php;shtml;")
$ssa.Update()

 To restart the search functionality on a crawler when no crawling is occurring, run the following commands:

net stop osearch15
net start osearch15
Properties

Article ID: 2953907 - Last Review: Apr 5, 2014 - Revision: 1

Feedback