This article was previously published under Q234580
This article has been archived. It is offered "as is" and will no longer be updated.
This article describes how to allow or prevent communication between secure and unsecured computers.
The IP Security (IPSec) Policy's negotiation policy can either allow or disallow unsecured traffic. When two computers that support IPSec communicate with each other, they establish security associations that are determined through negotiation of the Internet Security Association and Key Management Protocol (ISAKMP) and IPSec Policy rules. If either of two computers does not support IPSec, they must communicate without security, which is called a "soft association". Downlevel clients do not support IPSec communication and cannot communicate unless soft associations are permitted.
IPSec-Aware Computers Trying to Communicate with Non-IPSec-Aware Computers
The initiating computer sends an ISAKMP request to the non-IPSec-aware computer and receives a "destination unreachable" response back from the responding computer. This is because the non-IPSec-aware computer cannot send or receive ISAKMP messages using UDP port 500. The ISAKMP message gets to the client, but the client does not "understand" the packet.
The initiating computer attempts this four times.
The IPSec policy can allow unsecured communications by making a soft association with the remote computer. A soft association tells the IPSec driver to use no security between the two addresses and sets the security operation for that security association to None, which allows unsecured packets to be transmitted across the wire.
Non-IPSec-Aware Computer Trying to Communicate with IPSec-Aware Computer
The non-IPSec computer initiates communication by sending an unsecured packet to the IPSec-aware computer. The IPSec-aware computer contains rules for responding as well as initiating as stated above. If the inbound packets match an IP filter, the associated filter actions are followed. The IPSec computer sends an ISAKMP request for security association negotiation to the non-IPSec-aware computer. The IPSec computer can raise the security level if possible, but the non-IPSec-aware computer cannot do so.
The responding IPSec-aware computer attempts to negotiate four times, but the initiator cannot reply to any of the attempts.
The IPSec-aware computer then checks its security policy to set up a soft association if allowed, or it ignores the incoming insecure packets. If the policy does not allow unsecured communication, the IPSec computer can only communicate with other IPSec computers. The more tightly secured the communications are, the more resource intensive the communication becomes. If the policy does allow unsecured communication, soft associations are allowable. This is less secure and less resource intensive.