How Does Exchange Autodiscover Work?

Autodiscover is one of the more annoying features of Exchange since Microsoft reworked the way their Email solution worked in Exchange 2007. All versions since have implemented it and Microsoft may eventually require its use in versions following Exchange 2016. So what is Autodiscover and how does it work?

Some Background

Prior to Exchange 2007, Outlook clients had to be configured manually. In order to do that, you had to know the name of the Exchange server and use it to configure Outlook. Further, if you wanted to use some of the features introduced in Exchange 2003 SP2 and Outlook 2003 (and newer), you had to manually configure a lot of settings that didn’t really make sense. In particular, Outlook Anywhere requires configuration settings that might be a little confusing to the uninitiated. This got even more complicated in larger environments that had numerous Exchange servers but could not yet afford the expense of a load balancer.

The need to manually configure email clients resulted in a lot of administrative overhead, since Exchange admins and Help Desk staff were often required to configure Outlook for users or provide a detailed list of instructions for people to do it themselves. As most IT people are well aware, even the best set of instructions can be broken by some people, and an IT guy was almost always required to spend a lot of time configuring Outlook to talk to Exchange.

Microsoft was not deaf to the cries of the overworked IT people out there, and with Exchange 2007 and Outlook 2007 introduced Autodiscover.

Automation Salvation!

Autodiscover greatly simplifies the process of configuring Outlook to communicate with an Exchange server by automatically determining which Exchange server the user’s Mailbox is on and configuring Outlook to communicate with that server. This makes it much easier for end users to configure Outlook, since the only things they need to know are their email address, AD user name, and password.

Not Complete Salvation, Though

Unfortunately, Autodiscover didn’t completely dispense with the need to get things configured properly. It really only shifted the configuration burden from Users over to the Exchange administrator, since the Exchange environment has to be properly configured to work with Autodiscover. If things aren’t set up properly, Autodiscover will fail annoyingly.

How it Works

In order to make Autodiscover work without user interaction, Microsoft developed a method for telling Outlook where it needed to look for the configuration info it needed. They decided this was most easily accomplished with a few DNS lookups based on the one piece of information that everyone had to put in regardless of their technical know how, the email address. Since they could only rely on getting an email address from users, they knew they’d need to have a default pattern for the lookups, otherwise the client machines would need at least a little configuration before working right. Here’s the pattern they decided on:

  1. Look in Active Directory to see if there is information about Exchange
  2. Look at the root domain of the user’s email Address for configuration info
  3. Look at autodiscover.emaildomain.com for configuration info
  4. Look at the domain’s root DNS to see if any SRV records exist that point to a host that holds configuration info.

Note here that Outlook will only move from one step to the next if it doesn’t find configuration information.

For each step above, Outlook is looking for a specific file or a URL that points it to that file. The file in question is autodiscover.xml. By default, this is kept at https://<exchangeservername>/autodiscover/autodiscover.xml. Each step in the check process will try to find that file and if it’s not there, it moves on. If, by the end of step 4, Outlook finds nothing, you’ll get an error saying that an Encrypted Connection was unavailable, and you’ll probably start tearing your hair out in frustration.

What’s in the File?

Autodiscover.xml is a dynamically generated file written in XML that contains the information Outlook needs to access the mailbox that was entered in the configuration wizard. When Outlook makes a request to Exchange Autodiscover, the following things will happen:

  1. Exchange requests credentials to access the mailbox.
  2. If the credentials are valid, Exchange checks the AD attributes on the mailbox that has the requested Email address.
  3. Exchange determines which server the Mailbox is located on. This information is usually stored in the msExchangeHomeServer attribute on the associated AD account.
  4. Exchange examines its Topology data to determine the best Client Access Server (CAS) to use for access to the mailbox. The Best CAS is determined using the following checks:
    1. Determine AD Site the Mailbox’s Server is located in
    2. Determine if there is a CAS assigned to that AD site
    3. If no CAS is in the site, use Site Topology to determine next closest AD Site.
    4. Step 3 is repeated until a CAS is found.
  5. Exchange returns all necessary configuration data stored in AD for the specific server. The configuration data returned is:
    1. CAS server name
    2. Exchange Web Services URL
    3. Outlook Anywhere Configuration Data, if enabled.
    4. Unified Communications Server info
    5. Mapi over HTTPS Proxy server address (if that is enabled)
  6. Outlook will take the returned information and punch it into the necessary spots in the user’s profile information.

Necessary Configuration

Because all of this is done automatically, it is imperative that the Exchange server is configured to return the right information. If the information returned to Autodiscover is incorrect, either the mailbox connection will fail or you’ll get a certificate error. To get Autodiscover configured right, parts 5.1, 5.2, 5.3, and 5.5 of the above process must be set. This can be done with a script, in the Exchange Management Shell, and in the Exchange Management UI (EMC for 2007 and 2010, ECP/EAP for 2013/2016).

Importance of Autodiscover

With the release of Outlook 2016, it is no longer possible to configure server settings manually in Outlook. You must use Autodiscover. Earlier versions can avoid using it by manually configuring each outlook client. However, before doing that, consider the cost of having to touch each and every computer to properly configure Outlook. It can take 5 minutes or more to configure Outlook on one computer using the manual method, and with Exchange 2013 it can take longer as you also are required to input Outlook Anywhere configuration settings, which are more complex than just entering a server name, username, and password. If you multiply that by the number of computers you might have in your environment and add in the time it takes to actually get to the computers, boot them up, and get to the Outlook settings, the time spent configuring Outlook manually starts to add up very quickly. Imagine how much work you’d be stuck with configuring 100 systems!

In contrast, it usually only takes 10 to 20 minutes to configure Autodiscover. When Autodiscover is working properly, all you have to do is tell your users what their email address is and Outlook will do all the work for you. With a little more configuration or some GPO work, you don’t even have to tell them that!

When you start to look at the vast differences in the amount of time you have to spend configuring Outlook, whether or not to use Autodiscover stops being a question of preference and starts being an absolutely necessary part of any efficient Exchange-based IT environment. Learning to configure it properly is, therefore, one of the most important jobs of an Exchange administrator.

Advertisements

Removing Addresses from an Exchange 2007/2010/2013 Server

This is probably a rare issue, but something I’ve come across in my work. Occasionally an Exchange Administrator may need to remove an Email address domain (The part of the email address that comes after the @ sign). For instance, you may be in a situation where a portion of the users in an Exchange environment are migrated to a Cloud based email solution. This can be a little tricky because even if you remove the email address domain from your list of Accepted Domains in Exchange, the addresses may remain on users’ mailboxes. In this post, I’ll explain the process of removing email domains from an Exchange Server in the proper order.

Step 1 – Remove Address Policies that Use the Domain

Before you can actually remove an accepted domain from Exchange, you have to make sure there are no Address Policies that assign email addresses to users that utilize that accepted domain. In Exchange 2007 and 2010, you can do this by opening EMC (Exchange Management Console) and navigating to Organization Configuration>Hub Transport. Clicking the Address Policies tab will allow you to view the address policies in place. You should then remove any policies that define addresses based on the Email Address Domain you want to remove.

In Exchange 2013, you would open the Exchange Admin Center and navigate to Mail Flow>Email address policies, then modify or remove any policies that include the offending Email Address Domain.

Step 2 – Remove the Domain from the list of Accepted Domains

This step is pretty self-explanatory. In this situation we just remove the domain from the list of accepted domains on the Exchange server. This will tell the Exchange server not to accept emails destined for that domain. This can be done from the same location in EMC for Exchange 2007/2010, and from the Mail Flow system in Exchange 2013 by clicking on Accepted Domains, and then right clicking on the domain you want to remove. Selecting delete will remove that domain.

Step 3 – Remove Email Addresses

This part can be a little tricky. Removing the email address policies won’t necessarily remove the email addresses that users have from their accounts, and if those addresses remain you could still end up having mail go places you don’t want it to. Resolving this issue requires some work with PowerShell in the Exchange Management Shell (EMS).

After the Email Domain is removed, open EMS and run the following command:

get-mailbox | where {$_.emailaddresses -like “*domain.com”}

Replace domain.com with whatever domain you’ve removed. This will give you a list of all the users that have one or more email addresses attached to their domain that match the domain you’ve removed. If there are none, you’re done. If there are some mailboxes with the domain attached, you’ll want to run the following script to remove them:

$users = get-mailbox | where{$_.emailaddresses -like “*domain.com”}
foreach ($user in $users)
{
$addresses = (get-mailbox $user.alias).emailaddresses
$fixedaddresses = $addresses | where {$_.proxyaddressstring -notlike “*domain.com”}
set-mailbox $user.alias -emailaddresses $fixedaddresses
}

This will reset the email addresses on the account.

Office 365 Hybrid Configuration Failures

This is just a quick post that is meant to help people out who are having some issues with creating a Hybrid Configuration with Office 365 and Exchange 2010 SP3. There are some serious bumps in the road that you can come across when setting this up that may cause you to spend countless hours troubleshooting without any real success. I’ll elaborate on a couple of the problems that I’ve run into here, and follow up with the solution that worked for me with these issues at the end of the post.

AutoDiscover Failures and Free/Busy Issues

One of the things that you may run into after completing is AutoDiscover failures. You’ll know you have this problem when a cloud (or on-prem) user can log into OWA, but cannot set up their mailbox in Outlook or through Activesync. This can also present in an unusual fashion when you attempt to look up cross-premises Calendar information. Cross-premises calendar sharing utilizes the Exchange Federated Sharing features of Exchange, and this in turn utilizes Autodiscover to work properly. If you can’t view calendars in either direction (from On-Prem to Cloud or Cloud to On-Prem), and you get an error that the Free/Busy information couldn’t be read, look into Autodiscover first.

Generally, there isn’t a whole lot you can do to resolve Autodiscover errors, since Autodiscover is something that you have some pretty limited control over. Microsoft Recommends that the Autodiscover.company.com record that you publish in your Public DNS, so you shouldn’t have to change your Autodiscover record when introducing a Hybrid configuration. Unfortunately, there isn’t much more you can actually do once the Records are configured.

There is, however, a tool you can use to help you troubleshoot some issues with Autodiscover and Office 365 in a hybrid environment. Since Autodiscover is required for Free/Busy exchange to function, it may actually be possible to resolve your error by using Microsoft’s Free/Busy error troubleshooting tool. It’s available here: http://support.microsoft.com/kb/2555008

If you aren’t experiencing Free/Busy errors, the tool may not be as handy, but I suggest trying to go through it a bit anyway, since it can give you some tips for resolving Autodiscover errors. If you have on-prem users that are having trouble configuring clients with autodiscover, tell the tool you have on-prem users that can’t see free-busy for Cloud users. If you have cloud users that are having trouble, do the opposite. If neither are working, use the other option available in the tool.

What Solved My Problem

Interestingly, it took me about 2 or 3 days of digging before I finally found the solution to my autodiscover and free/busy issues. It turned out that my problems were caused by some information that Microsoft failed to let anyone know about.

When you run the Hybrid Configuration tool, it will make some major changes to each of the CAS and HUB servers that you add as Hybrid Endpoints. However, because the hybrid configuration wizard actually makes these changes remotely and on demand, it does not actually complete the setup for you. Once you complete the Hybrid Configuration Wizard and add *any* CAS or HUB servers as hybrid endpoints (All your CAS and HUB servers should be hybrid endpoints for optimum functionality), *make sure to reboot those servers*. The changes that are made by the Hybrid Configuration wizard *will not* apply fully until the World Wide Web Publishing Services and IIS services are restarted. You can achieve the same goal by running IISRESET on your CAS/HUB servers like I did if you are in a situation where rebooting will create unnecessary downtime, but a full reset is a good idea.

Internal DNS and Exchange Autodiscover

The Issue

By now, anyone who has managed, deployed, or worked with an Exchange 2007 or later environment should be familiar with Autodiscover. If you aren’t yet, I’ll give a short Explanation of what it is and how it works.

Autodiscover is a feature that allows any Mail Client that supports Autodiscover to configure the appropriate server settings for communication so you don’t have to input everything manually. It’s very handy. Unfortunately, you can end up with a lot of headaches related to Autodiscover when you start having to deal with Certificates. The issues you may run into are specifically limited to Exchange Organizations that have a Domain Name that uses a non-public TLD like domain.local, or a public domain name that they don’t actually own and can’t use externally as well. On an unrelated note, this is one of the reasons that Microsoft has started recommending the use of Public domain names for Active Directory domains.

If you have a domain that isn’t publicly useable on your Exchange AD environment, you will run into certificate errors when mail clients use Autodiscover. This becomes particularly problematic when you use Exchange 2013 and try to use HTTPS for Outlook Anywhere. This is because Microsoft is now enforcing certificate validity with Exchange 2013’s Autodiscover features (Note, though, that Outlook Anywhere will be configured to use HTTP only when your Exchange Server certificate is determined to be invalid in Exchange 2013). With Exchange 2007 and 2010, you will get a Certificate error every time you open Outlook. Generally, this error will state that the name on the certificate is not valid.

The Cause

To solve the issue with certificates, you need to configure your environment so it enforces the appropriate action with Autodiscover. By default, Autodiscover will attempt to communicate with a number of URLs based on the Client’s email address (for external users) or domain name (for internal users). It will take the following pattern when checking for Autodiscover services:

1. Autodiscover will attempt to find the Autodiscover configuration XML file at the domain name of the SMTP address used in configuration (because internal domain computers configure themselves automatically by default, this matches the Internal Domain. For example, the first place autodiscover looks is https://domain.com/Autodiscover/autodiscover.xml for external addresses. Change domain.com with domain.local for what Exchange looks for on Internal clients.

2. If the autodiscover record is not found at domain.com/domain.local, the server will attempt to connect to https://autodiscover.domain.com/Autodiscover/Autodiscover.xml (replace domain.com with domain.local for internal). This is why the typical recommendation for having an A Record for Autodiscover in your DNS that points to the mail server exists. In addition, you would need to have autodiscover.domain.com as a SAN on the SSL certificate installed on the Exchange server for it to be valid when attempting to connect to autodiscover using this step.

3. If autodiscover information cannot be found in either of the first two steps, Exchange will attempt to use a Service Locator record in DNS to determine the appropriate location of the configuration files. This record points the Autodiscover service to a specific location for getting the configuration it needs.

Because of the way this works, there is some configuration necessary to get Autodiscover working correctly. This usually involves adding Subject Alternate Names to the SSL certificate you use for your Exchange Server to allow the many host names used to be authenticated with the certificate.

The problem lately, though, is that many Third Party Certificate Authorities that provide SSL certificates are beginning to deny requests for Subject Alternate Names that aren’t publicly available (There are valid security reasons for this that I won’t go in to in this post, but maybe later). As a result, you won’t be able to get a valid SSL certificate that allows domain.local as a SAN. This means that the automated steps Exchange uses for Autodiscover configuration will always fail on an Internal domain with a name that is not publicly accessible or not owned.

The Solution

IMPORTANT NOTE: This particular solution only applies to computers on your network that are *not* added to the domain. Domain-joined computers have a different solution to work with. Please read my article on resetting the Active Directory SCP for resolving Autodiscover issues like this on domain-joined computers.

There are actually two ways to solve the certificate issues, here. The first would be to prevent Outlook from automatically entering a user’s information when they create their profile. This will result in more work for you and your users, so I don’t recommend it. The other solution is to leverage that last step of the Autodiscover configuration search to force it to look at a host name that is listed on the certificate. This is actually fairly simple to do. Follow these steps to configure the Service Locator record in your internal domain.

  1. Open the DNS manager on one of your Domain Controllers.
  2. Expand out the management tree until you can see your Internal Domain’s Forward Lookup Zone. Click on it, and make sure there are no A records for autodiscover.domain.local in the zone.
  3. Once no autodiscover A records exist, right click the Zone name and select Other New Records.
  4. Select Service Location (SRV) from the list.
  5. Enter the settings as shown below:Image
  6. Hit OK to finish adding the record.

Once the SRV record is added to the internal DNS zone, Outlook and other autodiscover clients that attempt to configure themselves with a domain.local SMTP address will work properly without the Certificate errors on all versions of Exchange.

Other Nifty Stuff

There are some additional benefits to utilizing the Service Locator record for Autodiscover rather than an Autodiscover A record, even in your public domain. When you use a SRV record, you can also point public clients to communicate with mail.domain.com or outlook.domain.com, or whatever you have configured your external server name to be. This means you can get away with having a single host name on your SSL certificate, since you wouldn’t need autodiscover.domain.com to get autodiscover working. Since most Third Party CAs charge a bit more for SANs than they do for Single Name SSL certs, you can save a bit of money (for this to work, though, you may need to change your Internal and External Web Services URLs in Exchange to match the name you have configured).

Another Problem the SRV record Fixes

There are also some other issues you may run into that are easily fixed by adding a SRV record. One of the most common is the use of multiple Email Domains in a single Exchange Environment. If you have users that are not assigned a Primary or secondary SMTP address that matches the domain name listed on your SSL certificate, you’ll discover that those users and the rest of your users will not be able to share calendar data between their mailboxes. You can fix this by adding an Autodiscover SRV record to the DNS zone that manages the additional mail domains. For example, you have domain1.com and domain2.com on the same Exchange Server. user@domain1.com can’t see user@domain2.com’s calendar. The fix for this is to add the SRV record to the domain2.com DNS zone and point it to the public host name for domain1.com’s mail server. Once that’s done the services that operate the calendar sharing functions will be properly configured and both users will be able to share calendars.