Do you find the Support WebCast transcripts helpful?
Let us know!

Note This document is based on the original spoken WebCast transcript. It has been edited for clarity.

Mohammad Nadeem: This is Mohammad Nadeem, and I will be the presenter for this WebCast. The topic we are discussing today is DNS troubleshooting in transports in the Microsoft^® Exchange 2000 Server product.

For the contents of this presentation, we'll have a brief introduction. We'll discuss some basic configuration, the way the Exchange and the DNS server should be configured for mail to flow in your Exchange 2000 organization. But the core of this presentation will be about troubleshooting and how to solve DNS problems specific to transports in Exchange 2000 Server. We'll take a systematic and logical approach to finding, verifying, troubleshooting, and resolving DNS issues in Exchange 2000, as they relate to mail flow problems. Along the way, we'll discuss some tools that are very useful in troubleshooting DNS problems in Exchange. Obviously, we'll share some great resources with you for your reading pleasure and for reference.

Going on to the introduction (slide 3), in Exchange 2000, especially with the transports component within Exchange 2000, there can be a lot of mail flow problems. If you are an Exchange admin, you may have encountered these problems. Quite honestly, a fairly large number of these problems are due to DNS, or are DNS-related problems. As a matter of fact, DNS problems are one of the highest call generators within PSS, in the Exchange 2000 transports specialty.

This presentation was designed to provide training and insight for the Microsoft employees who support the transports specialty. The internal version of this presentation also has live demos of all of the troubleshooting tools that we use in troubleshooting and resolving these issues. Unfortunately, in this WebCast, I will not be able to show the live demos, but we'll definitely talk about these tools in as much detail as possible.

While we are talking about the internal DNS server, let me just remind you that during this WebCast, all we are talking about is DNS, and not SMTP. The reason I want to make this distinction is because they are very interrelated, and a lot of times people confuse DNS with SMTP - both components and both processes are used while we are basically processing mail in Exchange 2000 servers. But these are two separate and different processes.

Just to briefly differentiate these, DNS is a mechanism that is used for name resolution for your domain names, or fully qualified domain names (FQDN) of the servers. We normally look for an IP address, and in most cases, you get it from the DNS server. After you get the IP address from the name resolution mechanism, like the DNS server, the SMTP protocol engine then takes over and uses that IP address, makes the connection directly, and then delivers data over that connection. Again, we are not talking about SMTP problems in this WebCast today, but only DNS issues specific to transport.

On the next slide (slide 4), we are talking about configuration, and I think the next few slides will continue to talk about configuration. The first slide I'm looking at is the Exchange 2000 Server configuration. This basically talks about how your Exchange 2000 Server should be set up. There are two things you can do here. The first thing, which is highly recommended, is to point your Exchange 2000 Server to one of your internal DNS servers. The way you do that is to go to My Network Places, and then select Properties of the local area connection, and then go in to TCP/IP properties and configure the DNS server, entering the IP address statically of the DNS server.

Another thing you can do, which is optional, is you can configure your SMTP Virtual Servers to resolve names using the external DNS servers. By external DNS servers, I mean servers that are not on your network or within your organization, or that you don't have control over. You are basically just using servers on the Internet. They are commonly known as external DNS servers, and also known as a DMZ resolver. I may use either one of these terminologies interchangeably, DNS servers or DMZ resolvers. They both mean the same thing.

Another thing to keep in mind while you're talking about the integration of the Exchange 2000 server is that Microsoft Windows^® 2000-based computers normally have a primary DNS domain name, which is basically a computer name and a primary DNS suffix, and which you can easily find in System Properties, clicking the Identification tab, and then clicking the Properties button. This is very important, because this entire string, computer name plus the primary DNS suffix, constitutes the fully qualified domain name, which is registered with your server.

Each adapter - for example, your network adapters - may also have a DNS domain name from its IP configuration, including its own DNS servers. If you have two NIC cards, each one of them can use a different DNS server. Just keep that in mind when we are talking about configuration.

The DNS client services also keep track of all of the transient devices and their IP configuration. For example, if you have wireless modems, or a second NIC and devices like that, DNS client services will keep track of all of that.

Moving on to the DNS server configuration on the next slide (slide 5), this basically talks about, briefly, how your DNS server should be set up. We recommend you set up your DNS server in one of two different ways. Your DNS server should be set up as a caching server, or you should enable forwarders on your DNS servers. Caching servers are configured during the installation of the DNS server itself. The caching servers do not host any zones; and as such, they're not targeted for any domains.

The way to configure a forwarder is to go to your DNS server properties, and there is a Forwarders tab and an Enable forwarders check box. You select that check box, and then you enter the IP addresses of the servers that you want to use as forwarders for your DNS server. Also, there are a couple of KB articles mentioned on the slide (Q300202 [http://support.microsoft.com/support/misc/kblookup.asp?ID=300202] and Q277693 [http://support.microsoft.com/support/misc/kblookup.asp?ID=277693]). These are good KB articles, especially the first one. It's fairly detailed and shows how to configure your DNS server for Internet access, and it shows all of the steps in an easy and user-friendly matter.

Also, I want to make a distinction about how the Exchange 5.5 Internet Mail Connector (IMC) used to work in Exchange 5.5 Server. In Exchange 5.5 IMC, when you are talking to the Internet, you just point a DNS server there, and you really don't have to point your IMC or Exchange 5.5 to an internal DNS server. Because in Exchange 5.5, most likely the internal names are resolved from the WINS servers. You only needed DNS for external name resolution. In Exchange 2000 and Windows 2000, we get away from WINS and we are relying on DNS. If we just point our Exchange 2000 server just to an external server, chances are that you will not be able to resolve your internal domain name. That's why we have to point the Exchange 2000 servers to one of the internal DNS servers, so that we can resolve all of the internal domain names and, at the same time, configure the DNS server in one of the two ways mentioned on the slide, either as caching servers or enabling forwarders, to resolve external names that we need to resolve outside of our organization.

Moving on to the next slide (slide 6), talking about the IP address, the entire objective and mission of doing name resolution and using DNS servers is basically very simple - to get the IP address. That's all we want from the DNS or the name resolution mechanism, the literal IP address. We should be able to obtain the IP address from the DNS server using one of these chains. For example, asking for an MX > A > IP; or MX > CNAME > A > IP; or maybe CNAME > A > IP; or MX > Literal-IP.

One other point I want to make here is for the SMTP DNS resolver only. Remember there are two components in Windows and Exchange 2000 that can perform the name resolutions or talk to the DNS servers. The first one is the core SMTP resolver, which is built into the operating system; and the second one is the external DNS server that's configured on your SMTP Virtual Server, Properties. If the resolution is done by the core SMTP resolver, we can also get the IP address using WINS or the HOSTS file. But if you're using the DMZ resolver, the external resolver, that does not use WINS or HOSTS files or cache or things like that.

At the end of the configuration section, there is one slide where I will bring up some of the most common configuration problems that customers run into. We'll discuss those possibilities some time later during this presentation.

That will bring us to the troubleshooting section. This is the slide titled "Symptoms" (slide 7). So far, we've just talked about the configuration of Exchange Server and the DNS server. Troubleshooting, like I mentioned before, is a major portion of this presentation, so from this slide on, we'll be just talking about troubleshooting, for the most part.

The very first step in troubleshooting DNS issues should be to identify the symptoms. Each problem has a symptom; so do DNS problems. They are mentioned on the slide. For example, one of the most common symptoms of a DNS problem in your Exchange 2000 environment is that you'll have messages sitting in a queue and, in most cases, they will be sitting in just one specific remote queue, or maybe a few specific remote queues, but not all of them. Even though the latter possibility exists, it's highly unlikely. We do get customer situations where it's only backing up or queued for one or two specific domains. Most likely they will be in a retry mode. That's one of the most common symptoms for DNS-related problems in Exchange 2000. Also, the queue diagnostics may indicate it is a DNS problem.

One of the other symptoms of a DNS issue is you might be getting an NDR with a DSN code 5.4.0 on Exchange 2000 Service Pack 1 or SP2, or you may be getting a 5.0.0 NDR before Service Pack 1 - that is, with the Exchange 2000 RTM product.

One other symptom of a DNS-related problem might be you may see an Event 4000 in the application log, which normally says "message delivery to the remote domain…" and then gives the name of the domain, "…failed". Keep in mind that Event 4000 can also be logged against SMTP protocol session failure. It's not always DNS, it could be SMTP protocol session failures. That you should keep in mind.

If you see any one of the symptoms that we talked about on this slide, the chances are that your e-mail flow problems are attributable to DNS. The next slide (slide 8) also talks about one of the queue diagnostics, which says the remote server did not respond to a connection attempt. It could very well be a DNS problem. One precaution here is that in installations before Exchange 2000 Service Pack 1 and Windows 2000 Service Pack 2, if you're receiving this error message or a message in the queue diagnostics, it's very possible that the DMZ resolver or the external resolver did not resolve the target domain. Just keep that in mind.

We have discussed the common symptoms of DNS or DNS-related issues in mail flow in Exchange 2000. The next logical step is to verify whether we are really having a DNS problem here or not. Quite honestly, verification (slide 9) is not very difficult when we encounter a DNS problem. There are three different things mentioned on this slide. Each one of them could possibly be used as a workaround to implement relief to a problem. These are not just verifications. They may also be workarounds or even solutions in some customer scenarios.

The first thing we can do on an Exchange 2000 server is, if you have queues backed up in retry mode for one or maybe several specific remote domains, we can create an SMTP connector going to that remote domain. On the General tab, in the Smart Host entry, we need to enter the literal IP address of the remote domain server in brackets. That tells the system to bypass the name resolution process and to directly use the IP to connect to that remote domain. After you do that, you will have to stop and restart the SMTP and the routing engine services. Detailed steps are mentioned in KB article Q285863 (http://support.microsoft.com/support/misc/kblookup.asp?ID=285863).

Another thing you can do is to verify whether you are encountering a DNS problem in Exchange 2000 Server is point your problematic Exchange 2000 server, the server that has the masters backed up, to a known good DNS server, a DNS server that you know for sure is a good working DNS server has no problems. For example, it could be a DNS server of your ISP, or it could be any other known good DNS server, on the Internet. That would be another test to verify if you are having DNS problems.

One other thing you can do is add the FQDN entry in the HOSTS files, but only in the scenario where you're using the core SMTP resolver and not the external DMZ resolver.

If you have used one of these mechanisms and you are able to resolve your mail flow issue - for example, if you used the first method, you configured the SMTP connector and now your problem is resolved - it definitely verifies that something was wrong in the name resolution process of getting an IP address from the DNS server. That pretty much is proof that you're having a DNS problem.

The next slide (slide 10) talks about the core SMTP or DMZ resolver. So far we have looked at the symptoms and we have verified that, yes, this is a DNS problem. The next systematic, logical step, now that we know we have a DNS problem, is to resolve it, pinpoint it, and get to the root cause, get to the bottom of it. Here are the next steps that we should be performing to find that out. There are two ways Exchange 2000 and Windows 2000 can perform name resolution. One of them is called core SMTP resolver and the second one is the external resolver. We need to figure out which component is doing the name resolution here.

By definition, resolution is done by the external resolver only if a couple of conditions are true. Number one, that you have configured your virtual server instance as a DMZ. As I mentioned before, go to the SMTP Virtual Server Properties, click the Delivery tab, click the Advanced button, click Configure External Servers, and then enter the literal IP address of your external server. Number two, the target domain that we are trying to resolve should be an FQDN and not an FQDN of your internal server, because external resolvers only resolve external FQDNs, and do so only using external DNS servers. This is very important to know because each behaves differently. Each has a slightly different algorithm or logic to resolve names, so this situation is to find out which component in Exchange 2000 server is really performing the name resolution.

The next step, which is on the next slide (slide 11), is to find out what servers are being used for name resolution. By what servers, I mean the actual DNS servers. Again, depending on which component is performing the name resolution, you can find it very easily. For example, if you know for sure that the name resolution is done by the core SMTP resolver, then simply using ipconfig /all is going to give you the IP address of the DNS server that is being used for name resolution. Also, check if you have configured an external resolver, and that should be easily identifiable by looking into the SMTP Virtual Server Properties, clicking Delivery, clicking Advanced, and clicking Configure External Servers - this is in Exchange System Manager (ESM), by the way. We know by now what component is performing the name resolution and also which DNS server they're using, trying to talk to, and get the IP address from it.

Also, it is important at this stage to figure out the problematic domains. As I mentioned, in most cases there is only one or a few selected domains that we are having problems trying to send e-mail to. It's highly unlikely that DNS problems can cause all of the queues to back up or cause no e-mail flow. Although it is possible, and we have seen those few cases where it happened.

The way you find your problematic domain is to get a copy of the NDR and it will tell you for sure which domains you're having problems with. You can look in the Exchange System Manager, under the SMTP queues, and try to see which queues are backed up or in retry mode. Also, there is a utility called Aqadmcli.exe, which can be very useful in identifying what domains we are having problems with.

The next slide (slide 12) talks about NSLookup. I'm pretty sure a lot of people are familiar with NSLookup and, quite honestly, this is an extremely powerful tool and ideal choice to troubleshoot DNS problems in Exchange 2000. By default, this utility is part of the operating system itself and it can be found in the Winnt\System32 folder. The way NSLookup can be helpful is, for example, we can point the NSLookup tool to a specific DNS server, and this is the DNS server that we identified in one of the previous tabs that you're using for name resolution. We can point it to a specific DNS server and then we can use the NSLookup tool to query that DNS server for specific types of resource records, what we are interested in - for example, the MX Record, A Record, CNAME, or any record. It will let you see what's in the DNS database.

With NSLookup, we are really interested in two things. First of all, we want to verify whether the records exist in the DNS database or not. That's a crucial step, because if they're not there, chances are we will not be able to send mail. Second, if they exist in the DNS database, are they correct? If they're not correct, we'll have problems again. This tool will let you verify the existence of the records, and of course you can then verify whether they are correct or not. This is a very crucial step and very common. Most of the common problems are due to missing or incorrect records in the DNS database.

I'll move on to the next slide (slide 13), about the Reverse DNS Lookup failures. Quite often customers run into problems where they receive a 5.5.0 NDR. In most cases a 5.5.0 NDR is generated because of an SMTP protocol session error. But the only time you will get a 5.5.0 NDR attributable to a DNS problem, it will be caused by a Reverse DNS Lookup failure. Basically, the easiest way to identify a reverse DNS failure is by using a Telnet session. Telnet is a great tool, by the way, for troubleshooting a lot of SMTP problems, but it comes in handy with this specific reverse DNS failure problem also. The KB article (Q153119; http://support.microsoft.com/support/misc/kblookup.asp?ID=153119) mentions how you use Telnet to test communication. If you use Telnet and if you are running into a reverse DNS failure when you issued the MAIL FROM command, for example, if you type MAIL FROM: user@domain.tld, if you're getting a response that says "5.5.0: Requested Action Not Taken, DNS Failure," that tells you that the Reverse DNS Lookup is failing. If you're trying to send mail to a remote domain, they probably are performing a Reverse DNS Lookup and they are failing that Reverse DNS Lookup, and as such, you are getting a 5.5.0 NDR.

SMTP protocol log is one of the .dll files that's not available externally. You will have to ask for it from PSS. Basically, we've registered the Protolog.dll regsvr32 it, and it logs events to the application log and records the whole SMTP conversation. Again, if you enable this log and you see the whole conversation, and when issuing the MAIL FROM command you receive a 5.5.0 DNS failure response, that again verifies that this is due to a Reverse DNS Lookup failure.

While we are on the subject, I think we should mention the way the vrfy command is implemented in Exchange 2000. When will we support the vrfy command? It's not implemented in its true sense. If anyone is really interested in getting true vrfy command functionality, they will have to write a custom protocol event sink. There are design reasons for not supporting the vrfy command fully, it's primarily because of security and performance reasons.

Slow DNS (slide 14) is another problem that customers may run into. It's not as common as a Reverse DNS, but occasionally our customers run into it. Customers will only run into this in cases where the resolution is performed by the external DMZ resolver for the external domains. The symptoms you will see will be slightly different here, and were not mentioned on the symptoms slide in the beginning of this presentation. You will see basically that e-mail is accumulating in the queues and it's being delivered, but it's very slow, extremely slow. It's happening because you're using the external DNS resolver. Slowness or slow LAN links, or if the DNS server is slow in responding, that shows that thread is pretty much tied up doing resolutions. The messages behind that one message will back up in queue. As soon as the thread is done resolving one name, it will be done with this message, pass it on to SMTP, and it will take on the next message. That's why the delivery will be slow.

The workaround to resolve that issue is to open MetaEdit 2.0, which is the metabase editor. You enter a value in there, a metabase key, under /SmtpSvc/1/MaxRemQThreads; that is the key that controls this. The default value is 1. You can change it to a higher number to work around this issue.

Let's talk about DNS NDR error codes (slide 15). These are some of the error codes that are attributable to DNS problems. One of the most common ones when Exchange 2000 was released was the 5.0.0 NDR. It was a generic NDR caused by a number of issues, including a bad header, a missing address in the Active Directory^®, DNS problems, and other issues. If, for example, you are running Exchange 2000 Server without Service Pack 1, and you're getting this NDR 5.0.0, my first recommendation is to upgrade the server to at least Service Pack 1. I would actually recommend going on to Service Pack 2; but at least go to Service Pack 1, because Service Pack 1 has code that has a breakdown of this 5.0.0 NDR.

One of the breakdowns of the 5.0.0 NDR is 5.4.0 in Exchange 2000 SP1. This is authoritative host not found fully on the target domain. Basically, they are unable to contact the authoritative server for the domain. We'll just get a specific 5.4.0 NDR because this is more common, and we'll discuss it in more detail on the next slide.

Another type of NDR that you may receive because of DNS problems is a 5.5.0 NDR, which we discussed on the Reverse DNS Lookup Failure slide. One point I want to make here is something I mentioned previously, that generally speaking, the 5.5.0 NDR occurs due to SMTP protocol session failures. This is the only instance where it may happen because of a DNS problem, and that is that reverse DNS is failing.

When talking about the 5.4.0 NDRs (slide 16), as I mentioned, these are authoritative host not found NDRs, and the bottom line is we are unable to reach the authoritative server for the domain name that we are trying to resolve. That will cause this NDR. There are a variety of reasons for this. For example, your primary DNS suffix is incorrect, or the search order may be incorrect. For example, someone has put an incorrect entry in the SmartHost, an incorrect IP address or the IP address of the server does not exist, or the IP address does not exist; or, for example, the SMTP Virtual Server itself does not have a valid FQDN; or the lookup of your virtual server FQDN failed. It can also happen when a contact's domain does not resolve to any SMTP addresses spaces. These are a variety of reasons for the 5.4.0 NDR.

On our next slide (slide 17), we have a tool that may be useful in troubleshooting DNS problems. It's called NetDiag. NetDiag is a resource kit utility. It may be useful in troubleshooting DNS problems. It may not be as useful as NSLookup, but still, depending on the scenario, it can be very useful. It is a command-line utility. Basically what you do is type netdiag /test:dns and it gives you an output on the DNS server, mostly in the form of whether it passed or failed the DNS test, along with other information - for example, information on your NIC card. If you run this command and you are seeing failures, that will hint to you what's wrong on your servers.

Also, the netdiag /fix switch verifies that all service records that are in the Netlogon.dns file are registered with the primary DNS server. If they're not, your servers will encounter problems trying to identify the services in the Active Directory. The article mentioned here, Q219289 (http://support.microsoft.com/support/misc/kblookup.asp?ID=219289), talks about this in a little more detail.

That brings us to the configuration slide (slide 18). As I pointed out earlier, there are a lot of problems that we see attributable to configuration issues. Most commonly, some admin or somebody in the organization has changed the FQDN name of the server, and it can be changed in two ways: for example, someone has changed the name of the server itself, or the primary DNS suffix. That is under System Properties, clicking the Identification tab, clicking the Properties button, and there is the name of the computer. If you click the More button, you will see the primary DNS suffix. It can be changed by changing either of those. This is what is registered in your DNS database. If you change that and your registration in DNS changes, the other servers probably will not be able to talk to you, because they're trying to use a different IP or some other information that they had previously in a Exchange database, a totally different registration for you in the DNS database. Also, if someone changes the virtual server's fully qualified domain name on your SMTP Virtual Server Properties, clicking Delivery and clicking the Advanced tab, that would cause also the same problems.

Other common configuration issues are incorrect entries in the .hosts files, incorrect entries in the DNS database, or missing records in the DNS database. All of these are common, simple issues, but trust me, they can generate a lot of huge problems as far as e-mail flow issues are concerned in Exchange 2000 servers.

Another very useful and very ignored tool is IPConfig (slide 19). It's a built-in utility in the operating system and it can be very useful in troubleshooting more common DNS problems. For example, if you use the ipconfig /all switch, it prints out the entire IP configuration, including the information on multiple NICs, plus your primary DNS suffix name, your DNS suffix search order, and things like that, which can be crucial in determining and identifying where exactly the problem is.

Other switches that are useful in troubleshooting are, for example, /flushdns, which flushes the client resolver cache. By the way, it does not flush those entries that are in the HOSTS file. If you want to flush the entire cache, you basically would have to disable the DNS client service to do that. /registerdns forces a refresh of the client name registration of the DNS server. It could be a useful tool if you're troubleshooting, or having problems with DNS, and you want to reregister a client with DNS. This might be the tool to do this. Basically, /displaydns displays the client resolver cache. It will let you see exactly what is in your cache, including the entries in the HOSTS files.

Depending on the scenario that you are troubleshooting, IPConfig can be very useful in troubleshooting, identifying, or pinpointing where the problem lies. Honestly, I have resolved several of the issues just by using this tool alone.

The next slide (slide 20) talks about NetMon. NetMon is the ultimate tool in DNS troubleshooting and resolving problems in Exchange that relate to DNS in Exchange transports generally. It shows the conversation for the DNS server and remote SMTP server. So if you filter on the DNS protocol and filter on your Exchange 2000 server and the DNS server, you can see the conversation between the DNS client, which is the Exchange server, and your DNS server. You can physically see what we are querying for from the DNS server and exactly what the DNS server is returning to us. So you can see the IP address that is being returned, and then you can verify that it is correct; or you may find that there is nothing that is being returned to us; or you may see that the conversation has just been dropped or canceled or does not complete. You will see what exactly happens while the Exchange server is trying to talk to the DNS server to get an IP address.

It can be very, very useful in determining problems related to DNS or mail flow. For example, you may rule out that this is not really a DNS problem, because you may see the correct IP address returned, and because you know you are getting the IP address, you should be opening up a session and trying to deliver the data over that session. You may want to, after you have verified it's really not DNS, just filter on SMTP and then see the conversation between your Exchange 2000 server and the remote domain's SMTP server. You may find that this is really not DNS. It's actually an SMTP problem. Anyway, NetMon is the ultimate tool in DNS troubleshooting in Exchange 2000 server.

The next slide brings us to Regtrace (slide 21). This is a highly advanced tool; actually, it's a debug tool. If you are at this point in troubleshooting an Exchange 2000 problem, you are working with Microsoft PSS. You can collect or get the Regtrace file, but you will not be able to view it. The tool that views the Regtrace output is a Microsoft internal tool. You may not be able to get it outside of Microsoft. If you are at this point, you are collecting data and going to provide it to a Microsoft support professional to help you identify where the problem lies in DNS.

The majority of the DNS problems can be quickly figured out by using the tools and techniques that we have talked about so far in this presentation. On the other hand, a fact of life is that Exchange 2000 Server and DNS servers are extremely complex products, so there can be a very difficult problem or complex scenario, and you really have to work with Microsoft to resolve that. This is one of the tools that helps in debugging and trying to resolve these advanced and complex problems in Exchange 2000, which relate to mail flow issues. There is a KB article (Q238614; http://support.microsoft.com/support/misc/kblookup.asp?ID=238614) that actually gives detailed, step-by-step instructions on how to set up a Regtrace in Exchange 2000.

The next slide talks about some of the known DNS issues (slide 22). I will not go over all of these KB articles. Basically, I referenced these for your reading, and you may want to just review these. Most of them, by the way, are issues fixed in Service Pack 1, but they are some of the most common issues that customers run into in Exchange 2000 RTM and Service Pack 1. These are some very interesting articles. You may want to go over them later when you have the time to do so.

The next slide (slide 23) talks about the Event Viewer, DNS logs. If you are troubleshooting a DNS problem and, based on the troubleshooting that we have done so far, you have ruled out that the problem really is not the Exchange 2000 server, but actually the DNS server itself, trying to provide us a response or an IP address, a good way to start troubleshooting that particular DNS server is to start looking to the DNS logs. It's part of the Event Viewer, so if you go on a DNS server and open Event Viewer, there's a log called DNS server and this logs all informational warnings and errors on that particular DNS server, so it could be very useful in pinpointing. You may find an error in it or a warning that can guide you or give you a clue of what's wrong with the DNS server, trying to provide us an answer.

Based on this presentation, chances are you should be able to resolve a lot of your DNS problems related to mail flow issues in Exchange 2000 server. As I mentioned before, it's possible that you may not be able to resolve it and you need to work with Microsoft at this point (slide 24). When you need to open an incident or a case with Microsoft PSS, it's very useful, very time saving if you have the relevant data already captured to provide it to a Microsoft professional. It saves time. It will help you resolve your issue faster, because it's highly likely that when you talk to a support professional he is going to ask you to provide this data anyway. So why not capture it in advance, and then when you call in, you already have the data. Just give it to the support professional and hopefully he or she will provide you an answer rather quickly.

I want to emphasize that this data should be captured while the problem is being reproduced, and data is being taken directly - because if the data is gathered, but not with the problem reproduced and not with a problem occurring on the server, that data is pretty much useless. Please make sure, to save time, that it's captured or gathered while the problem is being reproduced.

The most common data logs and tracings that a support professional would be interested in looking at, while trying to resolve a mail flow problem due to DNS in an Exchange 2000 server, would be a NetMon trace, a Regtrace (that's we talked about in the last couple of slides were NetMon and Regtrace), and an application log, but with logging levels turned up to level 7 in the registry for all the Exchange Transport categories. Also, a WinRoute snapshot would be very useful in trying to see the whole topology and to see if there are any problems in routing in reference to DNS problems in Exchange 2000. This is most of the data and logs that you should provide while opening a ticket with Microsoft on troubleshooting. It's possible that the support professional may come back and ask for maybe one or two more logs, but that will depend on the specific issue that you are running into.

That brings us to the resources section. I think there are a few slides left after this. I'm not really going into very much detail, because these slides are for your reference. For example, this resource slide talks about Internet gateways (slide 25). I've put a few of them up here. By no means am I or Microsoft endorsing or advertising these gateways. I just happened to find these useful, and I use them on a regular basis. These can help you find domain names to NSLookup and use several other commands and tools on the Internet, which help you in troubleshooting DNS problems. For example, we can perform an NSLookup or use the WHOIS command, use Traceroute, perform a ping, and things like that. This is very handy in troubleshooting a lot of different kinds of issues.

The next slide (slide 26) gives you a couple more of these Internet gateways. For example, this is specific to the WHOIS command. Keep in mind that the NSI Registrar database contains only nonmilitary and non-U.S. government domains and contacts. You will not find the government and military domain names in there.

Another good reference that I wanted to point out to you guys is the DNS server Help file (slide 27). This is a very useful resource and reference. Unfortunately, it's also the most ignored reference. It's built into the server, and it has some great topics, for example how to install and deploy the DNS servers, and the configuration and optimization of DNS servers. There are a lot of how-tos and a lot of different concepts described in the help file. It also talks about maintenance, troubleshooting, and best practices. You will find a great deal of information about DNS server in the help file. It's a very useful reference.

The next slide (slide 28) talks about some of the recommended reading for DNS systems. Most of them are white papers. For example, the namespace design paper, the Active Directory technical summary paper, the general paper on Windows 2000 DNS, and the WINS overview as it relates to the name resolution. Also there is a book mentioned here as a reference and recommended reading. There are several good RFCs that are related to DNS mentioned here also.

The next couple of slides (slide 29), I think, are the related RFCs for Windows 2000 DNS: RFC 1034 is Concept and Facilities, RFC 1035 is Implementation and Specification, these are good ones as far as running the concept and implementation is concerned. Other RFCs talk about, for example, application and support, DNS extensions for IP version 6.0, and incremental zone transfers in DNS. You can find all of these RFCs by going to http://www.ietf.org.

The next slide (slide 30) also gives some more RFCs related to Windows 2000 DNS; these are RFCs 1996, 2136, 2181, 2308. Again, these are referenced for your reading purposes.

The next slide (slide 31) talks about the Internet drafts related to Windows 2000 DNS. Again, these are drafts. These are not complete RFCs, but they are in the works, so you may see them as RFCs at some point in the future. There are three mentioned up here, "DNS Resource Records for Specifying the Location of Services," "Using the UTF-8 Character Set…," and "Interaction between DHCP and DNS."

The next slide (slide 32) mentions a couple of more of these Internet drafts related to Windows 2000 DNS.

That pretty much brings us to the end of this presentation. I think we'll start the Q&A session now.

Otto Cate: Great. Thank you very much for the presentation.

Before we move on to the Q&A portion of the Support WebCast today, we have a couple of quick program notes to share with you: The Q&A portion of the Support WebCast is intended to encourage further discussion of today's Support WebCast topic. One-on-one product support issues are really outside of the scope of the Support WebCast. If you need technical assistance, please submit an incident on the Web, or contact Product Support Services and speak to a support professional.

The first question here: Where can the Aqadmcli.exe utility be found?

Mohammad: I probably did not mention this. It's a good question. It's an internal utility, but our customers can access it. Generally, if you are working with Microsoft, we may provide you this utility to gather data and send the output back to the Microsoft support professional. It's not really an internal-only Microsoft utility; it can be given to customers. We are allowed to do so, but only in circumstances where they really need it. If you are working with us on a DNS solution or any related issue, you can ask us to provide this tool and we can send that to you in an e-mail. It's a small utility, but yet very, very helpful.

Otto: Is the error 5.4.0 the same on SP2?

Mohammad: Yes, it is, exactly. As I mentioned in the presentation, before Service Pack 1, the 5.0.0 was the generic NDR. Starting in Service Pack 1, they changed the code and actually split the 5.0.0 NDR into a number of different NDRs. One of the breakouts of that NDR is a 5.4.0 NDR, and it stays the same in Service Pack 1 and Service Pack 2. It will stay the same in Service Pack 3 as well.

Otto: We are currently using a tunnel to get replication between our site and the organization. We're running Exchange 5.5. It looks like it's kind of a two-part question here. Should we have a DNS record for the remote system or a HOSTS or LMHOSTS record? Then as further follow-up, it looks like the replication is according to Q300129.

Mohammad: I'm not sure. I'll probably need more detail to answer this. This is more like a support incident type of a question. I don't know, based on this question, if these are two different organizations, or just one organization in two different sites.

If they are two different organizations, basically each of them needs to have an MX record on the Internet so the other people outside, on the Internet, can talk to them. If, for example, they are part of the same organization in two different sites, and one of them is Exchange 2000 and one of them is 5.5, again, you should be able to resolve the names. You can use FQDN names or, for example, a NetBIOS name on the 5.5 end.

The bottom line is that you have to either have records in DNS or WINS to resolve it, or you have to have a HOSTS file or LMHOSTS file for each one of the remote domains or sites in your organization.

Otto: Great. Thank you. We have two domain names. One is a .com and one is a .ca. Some of our clients cannot send mails to the .com, but can send them to the .ca domain. Is that a DNS issue? If so, what can we do to fix this issue?

Mohammad: Again, this is a very broad question. I will try to answer to see if I can help with this question. Basically, it may be a DNS problem. Again, you have to look at the symptoms, and if you go through the steps and troubleshooting approach that we covered in this WebCast, that will help you, first of all, identify whether it's a DNS problem or not. For example, I know that you cannot send to .com, but I don't know what the symptoms are. If the symptoms are some of the symptoms that we mentioned in this WebCast, chances are it very well could be a DNS problem. I would suggest looking at the symptoms, and trying to use the troubleshooting steps and logic covered in this WebCast to narrow it down. If you think it's a DNS problem, you can verify it easily by following one of the three steps that we talked about in the beginning of this presentation, which are to bypass DNS name resolution either using an SMTP connector, for example from the Exchange 2000 server, or using a good, known DNS server, or by creating entries in the HOSTS file. You can use some of those methods to verify if it's a DNS problem. Upon verification, you can take the next steps mentioned in this WebCast to try to pinpoint exactly where the problem is. From their broad statement, it may be a DNS problem.

Otto: When you mentioned the ipconfig /all switch, you mentioned "dumped." Does that mean all of the configuration information would be erased or simply displayed?

Mohammad: We talked about the IPConfig switch, and there were several switches that I talked about. I think the customer may be talking about when I used the word "flush" for the client resolver cache. It's not erased or deleted. It's there.

The /flushdns switch basically will erase the cache, so that would be gone. You would have to build a new cache after you use a /flushdns switch on the IPConfig.

The /displaydns switch, on the other hand, is only going to display. It's going to display the entire client resolver cache, including the entries in the HOSTS files. Basically it will not change the cache or the HOSTS files; it will just display it.

The /flushdns is going to flush the cache entries in the DNS resolver cache. It will not flush the entries in the HOSTS file. The HOSTS file entries will still be there. What I mentioned in the presentation was - for example, you do not want to have the HOSTS file entries in the cache. You will literally have to disable the DNS client service.

The only command that's going to really remove the entries in the cache will be the /flushdns command.

Otto: The next question: Did I understand correctly that using the SMTP protocol logs requires a special .dll that you must request from Microsoft? This is not available out-of-the-box?

Mohammad: Yes, that is correct. You've got it right. It's an internal .dll and it's, again, like the Aqadmcli.exe tool that we talked about. It's an internal tool, it's not available externally, although if you are working on a support incident with Microsoft, we can send you this .dll to register with your server and then get the troubleshooting data out of your application log. But I want to make a quick point that you should not leave the .dll registered with your server after you are finished troubleshooting, because there is a performance hit, and we do not want this .dll running on production server for a long period of time, unless we are troubleshooting a specific issue.

Otto: Next question here: Where do the DNS NDR error codes appear?

Mohammad: Basically, when you receive the NDR, or when the end users receive the copy of the NDR, the last line of the NDR will give you a three-digit code. It could be one of those we talked about, a 5.0.0, 5.4.0. Actually, there are a variety of these DSN codes. If you want more information on these DSN codes, I think if you search on TechNet for "DSN codes" and "NDR in Exchange 2000" you will find a KB article that talks about all of these different types of NDRs.

To answer the question, you will find that three-digit code at the end of the NDR, in the last line.

Otto: The URL for TechNet is http://www.microsoft.com/technet/. Is that correct?

Mohammad: Yes.

Otto: Great. The next question here, this looks like it's a future release question, but I wanted to address it just in case there may be some public information about it: When will SP3 be released for Exchange 2000? Is there any public information about that yet?

Mohammad: No. There is no official release date on Service Pack 3, so I cannot give you any dates here, but I can tell you they are definitely working on it, and it should be out as soon as it's ready; that's the proper time to release it. There is no official release date yet.

Otto: Great. If you happen to lose your Internet connection or it becomes unavailable, is it possible for MX records and DNS information to disappear as well?

Mohammad: No. That's not true. If your Internet connection is gone, your DNS information will be there. One thing would be, for example, if you have internal DNS servers, of course, your DNS information will be there, but if you are going over the WAN link to access the DNS server, of course, you will not be able to find the MX record or any other record that you're looking for. That does not mean that information will disappear from the DNS server. In no circumstances, when information is not available, does DNS information disappear.

Otto: Great. Thanks for the clarification. The next question: Can you please review again how to figure out whether the resolution is being completed by SMTP, DNS, or in DNS sink's DMZ resolver?

Mohammad: Yes. It's very simple. There are two different accounts we talked about, the core SMTP resolver and the external DMZ resolver. In 90 percent of the cases, it's done by the core SMTP resolver. In the remaining 10 percent of the cases, where it is done by the external DMZ resolver, that is when you have actually configured an external DNS server in your SMTP Virtual Server. So the easiest way to verify and correctly answer your question is, go to your SMTP Virtual Server Properties and go to the Delivery tab, which is the last tab, click the Advanced button on the bottom right, and there is a button there that says Configure. On the left side it will say Configure External Servers. If you click that Configure button, and if you don't have any IP addresses in there, that means you are not using the external DMZ resolver, which means you are using the core SMTP resolver.

Otto: Thank you. Can you discuss the likely cause for the "remote server did not respond to a connection attempt: retry reason" code?

Mohammad: Yes. Definitely. Like I said before, in Service Pack 1 installations and Windows 2000 Service Pack 2, it could be because the external DMZ resolver was not able to resolve the target domain. That's fixed in Service Pack 1 of Exchange 2000. The most likely cause of the "remote server did not respond to a connection attempt" is that we are trying to connect to the remote server, and for whatever reason, we are not getting a response from the remote server. Starting in Service Pack 1, this is not a DNS problem. Before Service Pack 1, as I mentioned, you can see this because the external DMZ resolver is unable to resolve the target domain, which is a name resolution problem that our component, the external DMZ resolver, was not able to resolve correctly. That was a problem in the Exchange component before Service Pack 1, but that's fixed in Service Pack 1. If you're seeing this message after Service Pack 1 or in Service Pack 2, this is quite possibly due to an SMTP problem, or a problem with the remote server on the other end.

Otto: I'm not sure if this is off-topic, but: Is there any public information concerning plans to support RBL? Is that something that we're planning on at all?

Mohammad: I have no information on that. Honestly, I don't know if we have plans to do it or not. There is no public information on that, at this point.

Otto: In Exchange 5.5, the diagnostic logging included the ability to archive all in and out SMTP messages. Is this something that can be done with Exchange 2000 as well?

Mohammad: Yes, absolutely. It can be done with Exchange 2000 server, but to do that, you need a sink, and it's called Archivesink.dll. Again, if you search in the Knowledge Base and TechNet, search for "Archivesink.dll" and "Exchange 2000 and Service Pack 2", for example, you will find the KB article that talks about all of the different detailed steps and instructions of how to archive messages in Exchange 2000.

To answer, yes, it is possible, but you will need Archivesink.dll. Again, it may be available in the resource kit or in the support tools. I'm not quite sure. I think it's probably available out there. If not, for any reason, we'll help you get it from PSS.

Otto: As far as the error message 5.0.0, is that something that may continue after upgrading to SP2?

Mohammad: Yes. It will continue; we talked about this particular error DSN code, it was a generic DSN code in Exchange 2000 RTM. It happened because of a variety of reasons. In Service Pack 1, it was a breakdown of this particular DSN and it produced a some DSN codes, again specific to each different problem. Yes, it will continue, later, like in Service Pack 2 and Service Pack 3. In most cases, there are issues with Active Directory or some unknown issue that we are unable to identify. Microsoft developers keep identifying issues. There are more NDRs they need to root out of 5.0.0. They will do it, and as I understand it, it will continue.

Otto: Do we have a URL we can point people to that serves as a reference for the NDR numbers, like the 5.4.0 error?

Mohammad: Yes. Definitely. This is a good question. It's very helpful, actually. I think there is a Microsoft Knowledge Base article. If you hang on I can give you that number. It basically talks about a lot of these different DSNs and NDR codes. Actually, it goes all the way to explain briefly what they are and why they are caused. Let me give you that KB article number. It's a public Knowledge Base article, and its number is Q284204 (http://support.microsoft.com/support/misc/kblookup.asp?ID=284204). The title is "Delivery Status Notifications in Exchange 2000 Server."

Otto: Great. Thank you. Is there a Knowledge Base article that outlines the process of message routing in the categorizer?

Mohammad: There is a KB article on that. I can try to find it. The other thing I wanted to mention was there is also a white paper that I wrote internally on the categorization, how exactly the messages are processed through categorization. That white paper is in the works right now. Because that was an internal version that I wrote, it's being converted into a public version. Hopefully it should be available to Microsoft customers pretty soon. I don't have a date, but it will be available soon. Just check on the http://www.microsoft.com/exchange/ Web site and when it's available, it should be posted there.

Otto: Great. Thank you. This question might require one-on-one support, but I also wanted to throw this out just to see if you had something off the top of your head here. We have Exchange 2000 configured to have @companyname.tld. Some of our staff are not members of that server, but they still use @companyname.tld. Thus, when we send them e-mail, our Exchange 2000 rejects the e-mail. Is that a common issue?

Mohammad: I think it's a common issue, but you are right, it's more of a support incident issue. I can just take an educated guess here, but it will require support. It could be due to a variety of reasons: how you are configured, the recipients, the typology, where are we sending from, the data we look at when we are rejected, what kind of NDRs we are getting, and exactly what the symptoms are when it happens. Things like that. It will require some troubleshooting to answer the question.

Otto: It sounds like, based on the possible number of factors, it might be best handled through support, either online or through phone call to a support professional.

The next question: What is the recommended configuration of an external (Internet accessible) DNS server? Should it be Active Directory integrated or should it be a regular, primary zone? Can you discuss the pros and cons of this type of scenario?

Mohammad: Yes. It can be either one of those. It could be Active Directory or it could be standard primary. The benefit that you have with Active Directory integrated is that the DNS zone files become part of Active Directory. Their replication is done through the Active Directory replication, and you don't have to have zone transfers in the DNS replication and things like that to worry about.

Standard primary, on the other hand, was more commonly used before Active Directory was introduced. You can use either one of those. The advantage of using the Active Directory integrated is that zone transfer and replication is done via Active Directory. Of course, your files are more secure, because they're a part of the Active Directory.

Otto: Great. We have experienced a number of message flow problems. Our DNS servers and Exchange servers are managed by two different individuals. In utilizing your troubleshooting steps, what type of information should I provide to the individual, stating that it's a DNS problem?

Mohammad: It's a broad question. The answer pretty much is on those two or three different slides that we talked about. I think the question being asked is, when we try to troubleshoot these e-mail flow issues, what information should I provide to the DNS administrator or the Exchange administrator? I'm not sure which employee you need to provide that information to. In either case, we need to provide symptoms, tell them this is exactly the behavior that is happening on our server. For example, e-mail is queuing up in retry mode, or we're getting a specific kind of NDR that we discussed, although that's definitely a basic sentence.

Obviously, we would like to note bypass name resolution, and I'll mention again those three steps that we talked about, putting the entries in the HOSTS file or pointing your Exchange server to a good, known DNS server, or creating an SMTP connector directly to the remote domain using the IP address. You can use any one of those methods to bypass DNS. If you did follow one of those methods and e-mail flow is established, that will be one more thing you want to take back to your Exchange or DNS admins.

Plus, a very good thing you can take to them is a NetMon trace. For example, if the Exchange Server is querying DNS server for a record and we are not getting it, we should take that NetMon trace to the DNS or Exchange admin. One more thing you could do is when you're performing NSLookup and you are finding problems, that either the records are missing or incorrect or you're not getting a return, or anything back from the DNS server, you should also take the output of the NSLookup to the Exchange or DNS admin, whomever you are working with.

Pretty much all of the troubleshooting that we have done, whichever tool or whichever step, helps your point, to prove this is a DNS problem. You should take the output of that tool and repeat the same test in front of the Exchange admin or the DNS admin, whomever you are working with. Try to make them understand this is what the problem is, and this is what you're finding wrong about DNS, and hopefully, if they're knowledgeable, they should be able to resolve it easily.

Otto: Is there a risk of a performance hit when running PerfMon against the Exchange 2000 box? For instance, it looks like they may be doing this for queue sizes.

Mohammad: There is, depending on, again, how powerful your boxes are, the amount of memory they have, the speed of the CPUs, and how busy these servers are. A lot of times in large organizations, the Exchange 2000 bridgehead servers stay very busy. We do not recommend running PerfMon for an extended period of time, only and until you want to troubleshoot something. There is no problem in running PerfMon. It's safe to run it, but you will have to determine your baseline during the PerfMon. What's the CPU usage? What's the memory usage of the server? How busy is the server? Can it handle running PerfMon on it for extended periods of time?

It also depends how many objects, counters, and instances you want to monitor on the specific server. If you want to monitor everything, like all objects, counters, and instances, of course the performance hit will be slightly larger, compared to when you only monitor a specific object or a few specific objects and a few counters and a few instances. There's definitely a slight cost to it, but it needs to be determined based on the individual scenario and typology and the server. And again, the CPU, the RAM, and how busy the server is, all of those factors should be considered.

Otto: Does Exchange 2000 use single-label server names? Does it need WINS to resolve those single-label names, if that's the case?

Mohammad: Yes. Like we talked about in one of the slides, the Core SMTP resolver has a built-in component called GetHostByName. That component is going to search the local cache, including your host file name, and you're going to talk to the DNS server. It may talk to the WINS server, if the server is configured to do so. It may even broadcast, if necessary. Yes, it may go in and talk to the WINS server, but you know, again, that's almost like the second-to-last step. The GHBN, the GetHostByName component, pretty much drives every built-in mechanism to try resolve the name. Again, it will only be used with the core SMTP resolver and not with the external resolver.

Otto: Can you discuss the likely causes for the 5.1.1 NDR?

Mohammad: Yes, 5.1.1 NDRs are mostly encountered with Exchange 2000 RTM. It's still possible that you can still get a 5.1. NDR. The generic causes of the 5.1. NDRs are again mentioned in that KB article that I referenced (Q284204; http://support.microsoft.com/support/misc/kblookup.asp?ID=284204), and I will also mention it here. These are generic NDRs to resolve problems like missing or bad addresses. 5.1.0 and 5.1.1 NDRs are pretty much the same. One of the reasons is general categorizer-based failures. If you are trying to categorize a message and that contact - for example, if the entries do not have a target address attribute - this is a misconfiguration. These kinds of loose configurations will call a 5.1.0 or a 5.1.1 NDR and, like I was saying, most likely they are related to a problem with the Active Directory, like attributes missing, objects not there, object not found. These will cause 5.1.0 and 5.1.1 NDRs.

Otto: Great. It looks like we have a few more questions in the queue, but I wanted to take some time to solicit some feedback from our audience.

If you happen to have any suggestions for future Support WebCast topics or some general comments about today's show or even the WebCast program as a whole, we'd love to hear your feedback. We definitely appreciate it. If you'd like to send us some feedback, you can submit e-mail to us at supweb@microsoft.com. Simply type the word "feedback" in the subject line and I'll be able to separate those out for you.

We're also interested in your feedback concerning show times. Right now we are doing an 8:00 A.M. Pacific time show. We normally do some at 10:00 A.M. Pacific, and also we've been considering doing some at 4:00 P.M. Pacific. If you'd like to provide some feedback to us concerning what time would work best for you and why, that would definitely help us as well. We'd love to hear your feedback.

Moving on to the next question: Can you refer us to a KB article that outlines the permissions for Exchange and its administrators?

Mohammad: I don't have it handy, off the top of my head, but I think that a KB article exists on this. If I'm not mistaken, there is a paper on this, if you go to http://www.microsoft.com/exchange/ and search for "permissions," I believe you will hit that white paper or document, which talks about all of the permissions on Exchange 2000 Server. I'm pretty sure there's a KB article that exists on this as well, I just don't have them on top of my head right now.

[Follow-up answer: http://www.microsoft.com/technet/treeview/default.asp?url=/TechNet/prodtechnol/exchange/deploy/depovg/exchperm.asp.]

Otto: I'm trying to troubleshoot an intermittent communication problem between Exchange Server and Windows 2000 Server. Would DNS problems possibly be an issue here? I know that's kind of a general question, but is that something that would be a possibility?

Mohammad: Just looking at the symptoms, again, it's a broad question, and I'm just basically answering this question based on an educated guess. Intermittent problems are most likely caused by issues on the network. In almost all circumstances, if there is an intermittent problem going on - for example, queues back up, and without any user intervention the queue is cleared and e-mail is delivered, and we haven't changed anything on the Exchange or the DNS or anywhere else, it just flows on its own, then chances are it's a network problem, a network latency or network bandwidth issue.

That being said, it's very possible that it could be a DNS problem. For example, if you have two DNS servers and you're using both for your name resolutions, let's say one of the DNS servers has good records or correct records and then you hit that server, the e-mail flows and it works fine. When we hit the server with a bad entry or no entry, the problem happens. It's possible, but again, it requires some troubleshooting to get to the bottom of it.

Otto: Right. I have clients who are trying to access an Exchange server that is no longer on the domain when they try to access public folders. Those who use XP receive the "Outlook retrieving data from the Microsoft Exchange server" message. The public folder store finally shows, but it may take them about a minute or so. There are no entries in DNS for the old server, both on the server and client. Is that potentially a DNS issue?

Mohammad: It could be a DNS issue. It also may be an Active Directory issue, if you are trying to contact the global catalog servers and there is some delay trying to get to the global catalog server. Hence, we see that pop-up box. That delay may be due to the fact that we are trying to find that in the DNS. It's possible. Again, it would require some troubleshooting.

Otto: Great. It appears that we've answered all of the questions that were submitted today, so that's going to wrap up our session. I really wanted to thank all of you for joining us and hope that this information was useful to you.

I also wanted to thank Mohammad for coming out today and giving us a great presentation.

We hope that you all have the opportunity to tune in again in the near future. Thanks. Have a great day.