How to troubleshoot the bootup of GlassFish + MySQL image on OpenSolaris

I've been using an Amazon image with GlassFish+MySQL+OpenSolaris. As I'm no sysadmin, I relied on a prebuilt image furnished by Sun as a base for my development.

Still I've been experiencing  various issues at bootup, when the app server wouldn't start. This is especially true when you reboot the instance with Amazon EC2 tools. It behaves better if you terminate and restart the instance.

So I devised a set of steps to ensure that the thing gets working. 

Check if the services are running:

svcs  | grep mysql

svcs  | grep glassfish

If not, start them:

svcadm restart svc:/application/mysql:default

svcadm restart svc:/application/glassfish/domain1:default

Keep an eye on the server log

e.g. tail -f /opt/gf21/glassfish/domains/domain1/logs/server.log

If that reports failure, you have work to do.

Sometimes the app server can't start due to a previous process not having released the essentials TCP ports. Do a

netstat -a | grep 3700 

if it shows some daemons listening do a

ps -ef

look for usr/jdk/instances/jdk1.6.0/jre/../bin/java -Dcom.sun.aas.instanceRoot=/opt/gf2

That may be a stale instance. Try to kill -9 and look for the process above to be vanished and the port 3700 to be free.

Try again 

svcadm restart svc:/application/glassfish/domain1:default