Error encountered while initializing(Initialization routine) the shared library (ARERR 9755)

At AR Server startup the arerror.log shows an authentication failure while loading
the bmc.cmdb.cmdbengine.
To reproduce the error the consultant performed the following installations:
Install ARS 7.0.1 p6
Install Approval server 7.0.1 and Assignment Engine 7.0.1
Install ARS license and BMC Atrium CMDB Application license
Install CMDB 2.0.1 p6
Restart ARS
The arerror.log shows:
Fri Jul 18 14:40:22 2008 390600 : Error encountered while initializing(Initialization routine) the shared library
(ARERR 9755)
Fri Jul 18 14:40:22 2008 bmc.cmdb.cmdbEngine
Fri Jul 18 14:40:22 2008 390600 : Authentication failed (ARERR 623)

BMC Atrium CMDB 2.x

Solaris

At AR Server startup the Remedy-App-Service-Password is validated from the ar.conf. To confirm the exact
failure ARAPILOGGING=88 was set and the AR Server service restarted. The arapires.log shows:
#
# ARVerifyUser
#
ver
#
# ARGetServerInfo
#
gsi
8
2
23
41
55
140
139
4
246
ARGetServerInfo results
Return Code: OK
Server Info List: 8 items
Server Info Struct: SERVER LICENSE
Value: (char) Server
Server Info Struct: AR SYSTEM SERVER ID
Value: (char) WWW000000028564
Server Info Struct: UNIQUE SERVER IDENTIFIER
Value: (char) ac15a921
Server Info Struct: SERVER LANGUAGE
Value: (char) en_US;WESTERN
Server Info Struct: LOCALIZED SERVER
Value: (integer) 0
Server Info Struct: ALERT EVENTS SCHEMA
Value: (char) Alert Events
Server Info Struct: VERSION
Value: (char) 7.0.01 Patch 006 200712062121
Server Info Struct: RPC CLIENT XDR SIZE LIMIT
Value: (integer) 0
Status List: 0 items
Start Time: 1217265955.464214 (Mon Jul 28 2008 10:25:55)
Finish Time: 1217265955.485241 (Mon Jul 28 2008 10:25:55)
Elapsed Time: 0.021027
ARGetServerInfo results
Return Code: ERROR
Status List: 1 items
Status Struct:
Message type: ERROR
Message number: 623
Message: Authentication failed
Appended:
Start Time: 1217265955.322327 (Mon Jul 28 2008 10:25:55)
Finish Time: 1217265955.486861 (Mon Jul 28 2008 10:25:55)
Elapsed Time: 0.164534
#
# ARTermination
#
term
The consultant executed the ARS 7.0.1 p6 installer again in upgrade mode with no change in behavior. The
customer next restored from db backup to just after the ARServer installation, commented out the unwanted
armonitor.conf entries and restarted AR Server. The arerror.log now shows a generic authentication error rather
than the detailed message specific to the bmc.cmdb.cmdbengine. The armonitor.log also shows that arsvcdsp is
crashing repeatedly.
Fri Jul 19 13:30:30 2008 390600 : Authentication failed (ARERR 623)
The consultant noted that there was an environment variable for ARCONFIGDIR pointing to an incorrect path.
The environment variable path was corrected from /arapps/remedy/arconfig to /arapps/remedy/conf.
Set the ARCONFIGDIR environment variable to /arapps/remedy/conf instead of /arapps/remedy/arconfig.
Removed /etc/arsystem/.multilicense
Moved the directory /etc/arsystem/apptest1 to /etc/arsystem/apptest1.bak
Deleted the /arapps/remedy directory (ARS Install directory)
Downloaded a new ARS 7.0.1 patch 6 installation, decompressed and installed it.
The install was performed using the overwrite option and resulted with no errors on startup and the arsvcdsp is
running. The ARCONFIGDIR was incorrectly referening a directory that did not contain the ar.conf. Instead of
seeing the authentication error it would have been more helpful to see ar.conf not found.