HelpMaster v10.x.x and above.
After upgrading to HelpMaster v10 or above, Active Directory authentication on the Web Module elicits a Challenge authentication response and no matter how many times you enter your Windows Domain credentials the challenge dialogue remains. This is due to security enhancements starting with Microsoft's Dot.Net Framework v3.5 SP1 as outlined in detail in the Microsoft MSDN article below. In short this is because the default domain DNS resolved machine name or FQDN is not being used in the URL, e.g. Http://[MachineName].[DomainName].com/[VirtualDirectory]/Winlogin.aspx, but you are using a host name, machine IP address, machine name only, a registered domain name, or an alias rather than the resolved name returned by DNS on your domain, so Windows Authentication fails as a result.
After upgrading to HelpMaster v10 or above, Active Directory authentication on the Web Module elicits a Challenge authentication response and no matter how many times your Windows Domain credentials are entered the challenge dialogue repeats.
NOTE: If your credentials are accepted the first time you enter them at the Windows Security challenge dialogue, then your configuration is not suffering from this issue but your problem is that Windows Authentication is not enabled on your browser or the host name hasn't been entered as a Trusted Site. In this case see the Knowledge Base article linked above under "See Also Link".
To cater for the security enhancement introduced in the Dot.Net Framework v3.5 SP1 and above versions, your IIS host name, IP address, registered domain name, machine name, FQDN, and/or alias needs to be added to a registry key on the IIS server hosting the HelpMaster Web Module as follows;
1. Click Start, click Run, type 'regedit', and then click OK,
2. In Registry Editor, navigate to the following registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0,
3. Right-click MSV1_0, point to New, and then click Multi-String Value,
4. Type BackConnectionHostNames, and then press ENTER,
5. Right-click BackConnectionHostNames, and then click Modify,
6. In the Value data box, type the host name or the host names for the sites (all host names that may be used in the request URL) of the local computer, each on a new line and then click OK,
7. Quit Registry Editor, and then restart the workstation or server. The problem will not be resolved until the machine has been restarted.
PRD Software Knowledge Base article - https://support.prdsoftware.com/knowledgebase/viewkbarticle/107
Internet Explorer settings - http://www.helpmasterpro.com/Community/Discussion-Board/aft/126.aspx#126
Microsoft MSDN article - http://msdn.microsoft.com/en-us/library/cc982052%28v=vs.90%29.aspx
Microsoft Knowledge Base article - http://support.microsoft.com/kb/896861