Can’t start Agile Service – Error with incorrect NT password
Hi,
We had a power outage and our dev system went down now services won’t start. We get the following error.
We’re not sure what password for ‘superadmin’ got changed as we have new IT folks at the company. Is there a file that it is stored in where we can see what user is starting the service?
Thank you,
Laura
<Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: Server installed as Windows NT service with incorrect password for user superadmin; The password may have been changed since the server was installed as a Windows NT Service. Contact the Windows NT system administrator.
weblogic.security.SecurityInitializationException:
<Feb 20, 2019 12:55:09 PM PST> <Error> <Socket> <BEA-000438> <Unable to load performance pack. Using Java I/O instead. Please ensure that wlntio.dll is in: ‘D:OracleMiddlewarewlserver_12.1serverbin;C:WindowsSunJavabin;C:Windowssystem32;C:Windows;D:OracleMiddlewarewlserver_12.1serverbin;D:OracleJavajdk1.7.0_67bin;C:Program Files (x86)Javajre7bin;C:Windowssystem32;C:Windows;C:WindowsSystem32Wbem;C:WindowsSystem32WindowsPowerShellv1.0;;.’
>
AgileAuthenticationProviderImpl.initialize
<Feb 20, 2019 12:55:15 PM PST> <Critical> <Security> <BEA-090052> <Server installed as Windows NT service with incorrect password for user superadmin; The password may have been changed since the server was installed as a Windows NT Service. Contact the Windows NT system administrator.>
<Feb 20, 2019 12:55:15 PM PST> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: Server installed as Windows NT service with incorrect password for user superadmin; The password may have been changed since the server was installed as a Windows NT Service. Contact the Windows NT system administrator.
weblogic.security.SecurityInitializationException: Server installed as Windows NT service with incorrect password for user superadmin; The password may have been changed since the server was installed as a Windows NT Service. Contact the Windows NT system administrator.
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:945)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1054)
at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
at weblogic.security.SecurityService.start(SecurityService.java:148)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
Truncated. see log file for complete stacktrace
Caused By: javax.security.auth.login.LoginException: java.lang.ExceptionInInitializerError
at com.agile.admin.security.weblogic.WLSLoginModule.login(WLSLoginModule.java:193)
at com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
at java.security.AccessController.doPrivileged(Native Method)
at com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:106)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
at com.bea.common.security.internal.service.JAASLoginServiceImpl.login(JAASLoginServiceImpl.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.bea.common.security.internal.utils.Delegator$ProxyInvocationHandler.invoke(Delegator.java:57)
at com.sun.proxy.$Proxy13.login(Unknown Source)
at weblogic.security.service.internal.WLSJAASLoginServiceImpl$ServiceImpl.login(WLSJAASLoginServiceImpl.java:89)
at com.bea.common.security.internal.service.JAASAuthenticationServiceImpl.authenticate(JAASAuthenticationServiceImpl.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.bea.common.security.internal.utils.Delegator$ProxyInvocationHandler.invoke(Delegator.java:57)
at com.sun.proxy.$Proxy34.authenticate(Unknown Source)
at weblogic.security.service.WLSJAASAuthenticationServiceWrapper.authenticate(WLSJAASAuthenticationServiceWrapper.java:40)
at weblogic.security.service.PrincipalAuthenticator.authenticate(PrincipalAuthenticator.java:338)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:930)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1054)
at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
at weblogic.security.SecurityService.start(SecurityService.java:148)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused by: java.lang.RuntimeException: java.sql.SQLRecoverableException: IO Error: The Network Adapter could not establish the connection
at com.agile.util.sql.LocalConnectionFactory.getConnectionCache(LocalConnectionFactory.java:115)
at com.agile.util.sql.LocalConnectionFactory.getJDBCConnection(LocalConnectionFactory.java:74)
at com.agile.util.sql.ConnectionFactory.getConnection(ConnectionFactory.java:37)
at com.agile.util.crypto.ContainerCryptoUtil.getKeyStorePassword(ContainerCryptoUtil.java:103)
at com.agile.util.crypto.ContainerCryptoUtil.<clinit>(ContainerCryptoUtil.java:72)
… 39 more
Caused by: java.sql.SQLRecoverableException: IO Error: The Network Adapter could not establish the connection
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:458)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:546)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:236)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
at oracle.jdbc.pool.OracleDataSource.getPhysicalConnection(OracleDataSource.java:280)
at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:207)
at oracle.jdbc.pool.OracleConnectionPoolDataSource.getPhysicalConnection(OracleConnectionPoolDataSource.java:139)
at oracle.jdbc.pool.OracleConnectionPoolDataSource.getPooledConnection(OracleConnectionPoolDataSource.java:88)
at oracle.jdbc.pool.OracleImplicitConnectionCache.makeCacheConnection(OracleImplicitConnectionCache.java:1598)
at oracle.jdbc.pool.OracleImplicitConnectionCache.makeOneConnection(OracleImplicitConnectionCache.java:515)
at oracle.jdbc.pool.OracleImplicitConnectionCache.defaultUserPrePopulateCache(OracleImplicitConnectionCache.java:222)
at oracle.jdbc.pool.OracleImplicitConnectionCache.<init>(OracleImplicitConnectionCache.java:198)
at oracle.jdbc.pool.OracleConnectionCacheManager.createCache(OracleConnectionCacheManager.java:306)
at com.agile.util.sql.OracleConnectionImpl.initialize(OracleConnectionImpl.java:286)
at com.agile.util.sql.OracleConnectionImpl.init(OracleConnectionImpl.java:227)
at com.agile.util.sql.LocalConnectionFactory.getConnectionCache(LocalConnectionFactory.java:112)
… 43 more
Caused by: oracle.net.ns.NetException: The Network Adapter could not establish the connection
at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:392)
at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:434)
at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:687)
at oracle.net.ns.NSProtocol.connect(NSProtocol.java:247)
at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1102)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:320)
… 59 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at oracle.net.nt.TcpNTAdapter.connect(TcpNTAdapter.java:150)
at oracle.net.nt.ConnOption.connect(ConnOption.java:133)
at oracle.net.nt.ConnStrategy.execute(ConnStrategy.java:370)
… 64 more
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:864)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
at java.security.AccessController.doPrivileged(Native Method)
Truncated. see log file for complete stacktrace
>
<Feb 20, 2019 12:55:15 PM PST> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down.>
Hi Laura,
The superadmin password is typically never changed, however I wonder if someone copied the dmp file from another environment and hasn’t restored the keystore and special passwords.
Hi Keith,
This is the development system so when it was created I’m assuming the db was exported from production and imported to dev system. But the dev system has been working fine until this power outage.
Is there some file that may have been corrupted in the power outage?
Do we need to ‘restore the keystore and special passwords’?
How would we do that?
Is there a way to see what the username is for superadmin or is it really ‘superadmin’?
Sorry, lots of questions, thanks for your help.
Laura
Hi Laura,
I’ve seen this before on our system and the issue was an expired database user account. DO you know how to check this? We usually login to the database via SQL*Plus and rune the following script to see if any important agile accounts are expired and locked. Throw the following into a .sql file and run via SQL*Plus if you can
column username format a20;
column account_status format a20;
column expiry_date format a20;
column lock_date format a20;
set line 120;
select username, account_status, expiry_date, lock_date from dba_users;
I don’t believe any of the accounts starting the “AGILE” should be expired/locked so you’d want to reset them
dmcdonald
Hi ,
Has anyone found a solution as I have same the issue with similar errors after refreshing the dev environment with Prod backup. I have updated the boot.properties file with username and password in plain text and restarted but still I get the same errors. Please let me know if anyone has any solution.
Thank You
S
Hopefully, there is a backup of the original dmp file in dev somewhere that can be loaded back into the database so the application can start. The keystore is stored in the database and is environment specific. Attached is a script that I use to extract these details whenever an instance is created, allowing me to run the generated script to restore the keystore and passwords.
Hi Keith and dmcdonald!
Thank you for your quick advice. As it turns out, it was a simple issue – the listener service on the db was still down. It didn’t start automatically so the agileplm service couldn’t connect. Once that service was started Agile PLM service started fine!
I will save this info for the future, and in my hunting around the servers I did find the restorekeystore scripts so if we need them in the future we do have them. 🙂
Thanks again!
Laura