RADIUS Authentication Protocols
If you enforce FIPS compliance on your systems, there is currently no supported authentication protocol for communicating with a RADIUS server. PAP and CHAP use the MD5 algorithm to encode their responses, and one step in the construction of the MS-CHAPv2 response requires using the MD4 algorithm to match how NT systems hash their passwords. Neither of these algorithms are permitted by FIPS-compliant mode and Password Server will not let you enable RADIUS if FIPS-compliant mode is enabled.
PAP, or Password Authentication Protocol, is the least secure option available for RADIUS. RADIUS servers expect any password sent via PAP to be encrypted in a particular way that is not considered secure.
CHAP, or Challenge-Handshake Authentication Protocol, is also considered insecure. It constructs the message for the server using an MD5 hash, the security of which has been severely compromised by various attacks:
It is, however, more secure than PAP and is the recommended option that is guaranteed to be supported by all RADIUS servers.
MS-CHAPv2 is the most secure option available for Password Server to use in communications with RADIUS and is the recommended protocol if your server supports it. Despite this, it is still vulnerable to attacks in some environments: