This article is a step-by-step instruction guide to enable
advanced users to configure Internet Protocol security (IPSec) so that they can
secure the communications between two host computers.
Ensure that you know what the following terms mean before you
perform the following instructions:
- Authentication: The process to determine if the identity of
a computer is legitimate. Windows 2000 IPSec supports three kinds of
authentication: Kerberos, certificates, and preshared keys. Kerberos
authentication can work only if both endpoints (computers) are in the same
Windows 2000 domain. This type of authentication is the preferred method. If
the computers are in different domains, or at least one of them is not in a
domain, you must use either certificates or preshared keys. Certificates can
work only if each endpoint contains a certificate that is signed by an
authority that the other endpoint trusts. Preshared keys have the same problems
that passwords do: They do not remain secret for a very long period of time. If
the endpoints are not in the same domain and you cannot obtain certificates,
preshared keys are your only authentication option.
- Encryption: The process of making data indistinct in
preparation for transmission between two endpoints. By using well-tested
algorithms, each endpoint constructs and exchanges cryptographic keys. The
process ensures that only the endpoints know the keys; and if any key-exchange
sequences are intercepted, the interceptor obtains nothing of value.
- Filter: A description of the Internet Protocol (IP)
addresses and protocols that can trigger the establishment of an IPSec security
- Filter action: The security requirements that can be
enabled when the traffic matches the filters in a filter list.
- Filter list: A collection of filters.
- Internet Protocol security policy: The collection of rules
that describe how communications between computers are secured.
- Rule: The link between a filter list and a filter action.
When the traffic matches a filter list, the corresponding filter action can be
triggered. An IPSec policy can contain multiple rules.
- Security association: The collection of authentication and
encryption methods that the endpoints negotiate to establish a secure
Find IPSec in Microsoft Management Console
You configure IPSec by using Microsoft Management Console (MMC).
Windows 2000 creates an MMC with the IPSec snap-in during the installation
process. To locate IPSec, click Start
, point to Programs
, click Administrative Tools
, and then click Local Security Policy
. In the MMC that opens, click IP security policies on
in the left pane. Then, MMC displays the existing
default policies in the right pane.
Change the IP Address, Computer Names, and User Names
For the purposes of this example, Alice is a user that has a
computer named "Alicepc" with IP address 172.16.98.231 and Bob has a computer
named "Bobslap" with IP address 172.31.67.244. They connect their computers by
using the Abczz program.
Alice and Bob must ensure that the traffic
is encrypted when they directly connect to each other by using the Abczz
program. When Abczz makes its connection, the initiator uses a random high port
on itself and connects (for the purposes of this example) to the destination on
port 6667/TCP or 6668/TCP (where TCP is the abbreviation for Transmission
Control Protocol). Typically, these ports are used for Internet Relay Chat
(IRC). Because either Alice or Bob can initiate connections, the policy must
exist on both ends.
Create the Filter List
The menus for creating IPSec policies are accessible if you
right-click IP Security Policies
in the MMC console. The first
menu item is "Create IP security policy." Even though this location may seem to
be the place to begin, it is not the correct location. Before you can create a
policy and its associated rules, you need to define filter lists and filter
actions, which are necessary components of any IPSec policy. Begin your work by
clicking Manage IP filter lists and filter actions
The dialog box that is displayed has two tabs: One for filter lists and the
other for filter actions. First, the Manage IP filter lists
tab opens. There are already two predefined filter lists that you do not use.
Instead, you can create a specific filter list that corresponds to the other
computer that you want to connect to.
Assume that you create the
policy on the computer that belongs to Alice:
- Click Add to create a new filter list. Name the list "Abczz to Bob's
- Click Add to add a new filter. A wizard starts.
- Click My IP address as the source.
- Click a specific IP address as the destination, and then
enter the IP address (172.31.67.244) of the computer that belongs to Bob.
Alternatively, if the computer that belongs to Bob is registered in the Domain
Name System (DNS) or the Windows Internet Name Service (WINS), you can select a
specific DNS name, and then enter the name of the computer that belongs to Bob
instead, which is "Bobslap".
- Abczz uses TCP for its communication, so click TCP as the protocol type.
- For the IP protocol ports, click From any port. Click To this port, type: 6667, and then click Finish to complete the wizard.
- Repeat the preceding steps, except this time type:
6668 as the port number, and then click Close.
Your filter list contains two filters: One for communications
from Alice to Bob on port 6667 (which belongs to Bob) and one on port 6668
(which belongs to Bob). (Bob has both port 6667 and 6668 set up on his
computer: One port is for outgoing communication and the other for incoming
communication.) These filters are mirrored, which is generally necessary
anytime you create an IPSec filter. For every filter that is mirrored, the list
can contain (but not display) an exact opposite filter where the source and
destination addresses are reversed. Without mirrored filters, IPSec
communications is usually unsuccessful.
Create the Filter Action
You have defined the kind of traffic that must be secured. Now
you must specify the security mechanism. Click the Manage filter actions
tab. There are three defaults that are listed. Rather than using
the Require security
action, you must create a new action that is more stringent.
To create the new action:
- Click Add to create a new filter action. A wizard starts. Name the action
- For the General option, click Negotiate security, and then click Do not communicate with
computers that do not support IPSec.
- Click the High for the IP Traffic Security
option, and then click Finish to close the wizard.
- Double-click the new filter action (which you previously
named "Encrypt Abczz").
- Click to clear the Accept unsecured communication,
but always respond using IPSec check box. This step ensures that the
computers must negotiate IPSec before an Abczz packet is sent.
- Click Session key perfect forward secrecy
to ensure that key material is not reused, click OK, and then click Close.
Create the IPSec Policy
You have obtained the policy elements. Now you can create the
policy itself. Right-click the right pane of the MMC, and then click Create IP security policy
. When the wizard starts:
- Name the policy "Alice's IPSec".
- Click to clear the Activate the default response
rule check box.
- Click Edit properties if it is not selected, and then finish the wizard. The Properties dialog box of the policy opens.
For an IPSec policy to work, it must contain at least one rule
that links a filter list to a filter action.
To specify rules in the
- Click Add to create a new rule. When the wizard starts, click This
rule does not specify a tunnel.
- Click Local area network (LAN) for the
- Click Windows 2000 default (Kerberos V5
protocol) for the authentication method if both the computers of Alice
and Bob are in the same Windows 2000 domain. If not, click Use this
string to protect the key exchange (preshared key), and then enter a
string (use a long string that you can remember and type without making
- Select the filter list that you created earlier. In this
example, the filter list is "Abczz to Bob's PC". Then, select the filter action
that you created earlier. In this example, the filter action is "Encrypt
- Finish the wizard, and then click Close.
Configure the Other Endpoints
Repeat on the computer that belongs to Bob all of the preceding
procedures that had been applied to the computer that belongs to Alice. The
necessary changes are obvious, for example, "Abczz to Bob's PC" must be changed
to "Abczz to Alice's PC".
Assign the Policies
You have defined the policies on both ends. Now you must assign
- In the Local Security Settings MMC,
right-click the policy (Abczz in this example).
- Click Assign.
Only one IPSec policy can be assigned at one time, but a single
policy can have as many rules as you need. For example, if Alice also needs
secure communications with Eve by using a different protocol, you have to
create the appropriate filter lists and actions, and then add a rule to the
IPSec (which belongs to Alice) that links together that specific filter list
and filter action. Click Use a different shared key for this
. The policy for Alice now has two rules: One for Abczz
communications with Bob and another for the communications with Eve. Because
Bob and Eve do not need to communicate securely to each other, the policy for
Bob does not have anything added to it, and the policy for Eve contains a
single rule for communications with Alice.
Use IPSecMon to Test Your Policy
Windows 2000 includes a utility (IPSecMon.exe) that you can use
to test whether an IPSec security association is successfully established. To
- Click Start, and then click Run.
- Type: ipsecmon, and then press
- Click Options.
- Change the refresh interval to 1.
You must establish communications from one endpoint to the
other. There can be a delay because it takes a few seconds for the endpoints to
exchange cryptographic information and complete the security association. You
can observe this behavior in IPSecMon. When the endpoints each build their
security associations, you can observe an entry in IPSecMon that displays this
If you expect a security association to be built, but
nothing happens, go back and review the filter lists on each endpoint. Ensure
that you have received the correct definitions for the protocols that you use
as you can easily reverse the source and destination addresses or reverse the
ports. You may want to consider the creation of a new filter list that
specifies all traffic. Also, you can add a new rule to the policy that uses
this filter list, and then disable the existing rule. Perform these steps on
both endpoints. Then, you can use the ping
command to test connectivity: The ping
command can display "Negotiating IP security" during the security
association phase, and then display its normal results when the security
association is established.
NAT and IPSec Are Incompatible
If there is any Network Address Translation (NAT) between the two
endpoints, IPSec does not work. IPSec embeds endpoint addresses as part of the
payload. IPSec also uses source addresses when it computes packet checksums
before depositing the packets on the wire. NAT can change the source address of
outbound packets, and the destination uses the address in the header when it
computes its own checksums. The original source-computed checksums, carried in
the packets, do not match the destination-computed checksums, and the
destination can drop the packets. You cannot use IPSec with any type of NAT