installing/upgrading Python + MySQLdb on Mac OS X

I've been working on a project that required the use of the urllib2.urlopen that takes a 3rd argument to indicate the guard timeout to break the blocking on an unresponsive socket. Alas this handy feature is only available in the versions of Python starting with 2.6. As Leopard comes with 2.5, I was faced with the prospect of upgrading to 2.6, hopefully without breaking existent functionality.

First step is to see where Python is installed by default:
which python
yields
/usr/bin/python
and
ls -l /usr/bin/python
yields
lrwxr-xr-x 1 root wheel 72 21 Feb 2008 /usr/bin/python -> ../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/python

So there are the goodies. Good to know.


Next download Python for Mac from here.
Unpack and run
python setup.py build:
Traceback (most recent call last)
File "setup.py", line 5, in
from setuptools import setup, Extension
ImportError: No module named setuptools

BUMMER !

Go to http://pypi.python.org/packages/2.6/s/setuptools/ and download the current .egg version. I got a setuptools-0.6c9-py2.6.egg.sh.

Ok, now I do:
chmod u+x ./setuptools-0.6c9-py2.6.egg.sh
./setuptools-0.6c9-py2.6.egg.sh
to get greeted by
./setuptools-0.6c9-py2.6.egg.sh is not the correct name for this egg file.
Please rename it back to setuptools-0.6c9-py2.6.egg and try again.

Fine,
mv ./setuptools-0.6c9-py2.6.egg.sh ./setuptools-0.6c9-py2.6.egg
./setuptools-0.6c9-py2.6.egg

Processing setuptools-0.6c9-py2.6.egg
Copying setuptools-0.6c9-py2.6.egg to /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages
Adding setuptools 0.6c9 to easy-install.pth file
Installing easy_install script to /Library/Frameworks/Python.framework/Versions/2.6/bin
Installing easy_install-2.6 script to /Library/Frameworks/Python.framework/Versions/2.6/bin

Installed /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg
Processing dependencies for setuptools==0.6c9
Finished processing dependencies for setuptools==0.6c9

And we give it another try:

python setup.py build

yields

running build
running build_py
creating build
creating build/lib.macosx-10.3-fat-2.6
copying _mysql_exceptions.py -> build/lib.macosx-10.3-fat-2.6
creating build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/__init__.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/converters.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/connections.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/cursors.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/release.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/times.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
creating build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/__init__.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/CR.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/FIELD_TYPE.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/ER.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/FLAG.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/REFRESH.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/CLIENT.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
running build_ext
building '_mysql' extension
creating build/temp.macosx-10.3-fat-2.6
gcc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -Dversion_info=(1,2,3,'gamma',1) -D__version__=1.2.3c1 -I/usr/local/mysql/include -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c _mysql.c -o build/temp.macosx-10.3-fat-2.6/_mysql.o -Os -arch i386 -fno-common
gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -bundle -undefined dynamic_lookup build/temp.macosx-10.3-fat-2.6/_mysql.o -L/usr/local/mysql/lib -lmysqlclient_r -lz -lm -o build/lib.macosx-10.3-fat-2.6/_mysql.so
ld warning: in build/temp.macosx-10.3-fat-2.6/_mysql.o, file is not of required architecture
ld warning: in /usr/local/mysql/lib/libmysqlclient_r.dylib, file is not of required architecture

I think the 2 warnings have to do with the "-arch ppc" in the 2nd invocation of gcc, but given that we also have the "-arch i386" there we're probably OK.

Next

sudo python setup.py install

Password:
running install
running bdist_egg
running egg_info
writing MySQL_python.egg-info/PKG-INFO
writing top-level names to MySQL_python.egg-info/top_level.txt
writing dependency_links to MySQL_python.egg-info/dependency_links.txt
reading manifest file 'MySQL_python.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'MySQL_python.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.3-fat/egg
running install_lib
running build_py
copying MySQLdb/release.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
running build_ext
creating build/bdist.macosx-10.3-fat
creating build/bdist.macosx-10.3-fat/egg
copying build/lib.macosx-10.3-fat-2.6/_mysql.so -> build/bdist.macosx-10.3-fat/egg
copying build/lib.macosx-10.3-fat-2.6/_mysql_exceptions.py -> build/bdist.macosx-10.3-fat/egg
creating build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/__init__.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/connections.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
creating build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/__init__.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/CLIENT.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/CR.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/ER.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/FIELD_TYPE.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/FLAG.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/REFRESH.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/converters.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/cursors.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/release.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/times.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
byte-compiling build/bdist.macosx-10.3-fat/egg/_mysql_exceptions.py to _mysql_exceptions.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/connections.py to connections.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/CLIENT.py to CLIENT.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/CR.py to CR.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/ER.py to ER.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/FIELD_TYPE.py to FIELD_TYPE.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/FLAG.py to FLAG.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/REFRESH.py to REFRESH.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/converters.py to converters.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/cursors.py to cursors.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/release.py to release.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/times.py to times.pyc
creating stub loader for _mysql.so
byte-compiling build/bdist.macosx-10.3-fat/egg/_mysql.py to _mysql.pyc
creating build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/PKG-INFO -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/SOURCES.txt -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/dependency_links.txt -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/top_level.txt -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
writing build/bdist.macosx-10.3-fat/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg' and adding 'build/bdist.macosx-10.3-fat/egg' to it
removing 'build/bdist.macosx-10.3-fat/egg' (and everything under it)
Processing MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg
Copying MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg to /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages
Adding MySQL-python 1.2.3c1 to easy-install.pth file

Installed /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg
Processing dependencies for MySQL-python==1.2.3c1
Finished processing dependencies for MySQL-python==1.2.3c1

Looks good. Let's try it:

python
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg/_mysql.py:3: UserWarning: Module _mysql was already imported from /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg/_mysql.pyc, but /Users/bhapca/Downloads/MySQL-python-1.2.3c1 is being added to sys.path

huh ?

>>> dir(MySQLdb)
['BINARY', 'Binary', 'Connect', 'Connection', 'DATE', 'DATETIME', 'DBAPISet', 'DataError', 'DatabaseError', 'Date', 'DateFromTicks', 'Error', 'FIELD_TYPE', 'IntegrityError', 'InterfaceError', 'InternalError', 'MySQLError', 'NULL', 'NUMBER', 'NotSupportedError', 'OperationalError', 'ProgrammingError', 'ROWID', 'STRING', 'TIME', 'TIMESTAMP', 'Time', 'TimeFromTicks', 'Timestamp', 'TimestampFromTicks', 'Warning', '__all__', '__author__', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__revision__', '__version__', '_mysql', 'apilevel', 'connect', 'connection', 'constants', 'debug', 'escape', 'escape_dict', 'escape_sequence', 'escape_string', 'get_client_info', 'paramstyle', 'release', 'result', 'server_end', 'server_init', 'string_literal', 'test_DBAPISet_set_equality', 'test_DBAPISet_set_equality_membership', 'test_DBAPISet_set_inequality', 'test_DBAPISet_set_inequality_membership', 'thread_safe', 'threadsafety', 'times', 'version_info']
>>>

Seems OK I guess.

Bottom line. The steps above seem to result in 2 different versions of Python coexisting side by side, one that came from Apple in

/System/Library/Frameworks/Python.framework/Versions/2.5/bin/python

and the one I installed in

/Library/Frameworks/Python.framework/Versions/2.6/bin/python

cron jobs terminate prematurely on Godaddy accounts

I've been using a large tarball file to upload my online store updates to godaddy and a cron job to unpack the tarball.

As the size of that tarball increased, due to adding more products, I noticed that the tar process would terminate prematurely, causing big issues, with the store left half-updated, in an inconsistent state.

I've been suspecting the folks at godaddy running some sort of watchdog timer and killing off processes that take longer to run.

Well, according to some reports, that indeed seems to be the case.

One option to go around this would be to break the large tarball in several smaller ones, that won't keep tar humming for too long. Also I should perhaps consider sending only actual delta changes instead overwriting the whole store in one go.

connection pools time-outs in GlassFish

I've been using JDBC connection pools and JMS connection factories for a recent project and upon deployment on GlassFish I did a sanity test to confirm that everything is OK. It passed the test.

So off I go, leaving it running on an Amazon EC2 instance. When I check on it about 3 days later I notice that it doesn't work anymore ... "server error", the UI informs me (which is a bad error message for the end users, whom might not know what a server (those working in restaurants) has to do with it). Anyway, by looking at the logs I see a few of these:

Exception: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

and

Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after statement closed.

Now the way I implemented this was in the "init" method of the servlet I would create my instances of javax.jms.Connection and java.sql.Connection as member fields of the servlet, hoping to reuse them for the lifetime of the servlet.

The errors in the logs seem to indicate that those connections get yanked out while my servlet is running, albeit idle. Not cool.

Looking at the admin console I see an "Idle Timeout:" configuration setting which defaults to 300 sec for both my JDBC connection pool and JMS connection factory. I changed that to 300000000. I have the feeling that this is going to fix the issue, but I need to run it for a while to confirm.

I'm not sure if this is the right thing of approaching this, so if you know any better please add a comment.

UPDATE:
It seems that changing the configuration settings for GlassFish is not enough to fix the problem.

Likely the wait_timeout configuration setting of MySQL has something to do with this too. By default mysqld will close idle connections after 8 hours, so we'd need to increase that to up to the max value of 31536000. One way to do that is through a configuration file (e.g. /etc/my.cnf, where we'd add a line with 'wait_timeout= 31536000' under the [mysqld] section.

Yet a better approach would be to catch the exceptions mentioned above in the servlet code and recreate the connection to the DB.

ANOTHER UPDATE:
It seems that catching the exception and re-opening the connection in the servlet code doesn't work very well. I get the following exception when trying to re-establish the connection:

Exception:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

Last packet sent to the server was 0 ms ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
...

Other people have experienced this too. One workaround indicated there, like I mentioned above, is to tinker with wait_timeout configuration of MySQL.

Another avenue to explore would be the "Connection Validation" capabilities of the JDBC connection pool. That's easily available through GlassFish admin console. According to the docs "Optionally, the application server can validate connections before they are passed to applications. This validation allows the application server to automatically reestablish database connections if the database becomes unavailable due to network failure or database server crash. Validation of connections incurs additional overhead and slightly reduces performance."

It'd be interested to see if the "table" validation method fixes this issue.

java hangs after installing "Java for Mac OS X 10.5 Update 4"

I had been experiencing some issues with Java on my MacBook after an update was pushed by Apple.

Had to jump through some hoops, but I came up with a workaround. For more details see this post.

How to download and install Nokia .wgz widgets OTA

According to Nokia we should be using an "AddType x-nokia-widget .wgz" directive to change the settings of web server to add a new mapping from the filename to a new MIME type.

Tough some people don't have access to server settings, let alone that Nokia's instructions don't include the complete MIME type (application/x-nokia-widget) so I'm not sure that what they suggested would work well.

Another way is to include the appropriate HTTP response headers through a script running on the server side.

This one, writteh in PHP, seems to be doing the job just fine. Pointing the S60 browser to its URL downloads the widget and starts its installation.

$filename = 'example.wgz';
 
header('Content-type: application/x-nokia-widget');
header('Content-Disposition: attachment; filename="'.$filename.'"');
readfile($filename);

java.rmi.server.UID is not random

Well, here's an unfortunate result of having different ways of doing the same thing in Java.

I needed a way to generate a random unique ID to be used as an authentication token. So I thought hmmm a sort of UID ought to do it. So the first candidate that Google suggested was

java.rmi.server.UID

Running a quick test gave me the following results in 2 subsequent runs: '-31a4aa0:121f7402438:-7f1f' and '-31a4aa0:121f7402438:-7f1e'. Epic fail. So much for randomness.

Looking for alternatives,

java.util.UUID

seems to be a better choice. In particular its

randomUUID()

factory method seems to claim cryptographic strength.

How to run GlassFish on the default HTTP port 80 on Solaris

If GlassFish is run as non-root process, with reduced privileges, as recommended, it won't be able to bind to ports < 1024. You'll get an exception:

java.net.BindException: Permission denied: 80 at com.sun.enterprise.web.connector.grizzly.SelectorThread.initEndpoint

To get around this you could set up a domain that has the ability to bind to standard Unix ports, including the port 80. For this you use the --serviceproperties net_privaddr flag of asadmin create-service command.

You can use the following steps. Replace the italic portion with your specific settings.

/opt/gf21/glassfish/bin# ./asadmin create-domain --adminport 4848 --instanceport 80 --profile cluster --user admin --domaindir /opt/gf21/glassfish/domains ptmp

Please enter the admin password>

Please enter the admin password again>

Please enter the master password [Enter to accept the default]:>

Please enter the master password again [Enter to accept the default]:>

Using port 4848 for Admin.

Using port 80 for HTTP Instance.

Using default port 7676 for JMS.

Using default port 3700 for IIOP.

Using default port 8181 for HTTP_SSL.

Using default port 3820 for IIOP_SSL.

Using default port 3920 for IIOP_MUTUALAUTH.

Using default port 8686 for JMX_ADMIN.

On Unix platform, port numbers below 1024 may require special privileges.

Domain being created with profile:cluster, as specified on command line or environment.

Security Store uses: JKS

Domain ptmp created.

You need to create a file with the admin credentials:

cat > /opt/gf21/glassfish/domains/ptmp/.aspass

AS_ADMIN_USER=admin

AS_ADMIN_PASSWORD=something

AS_ADMIN_MASTERPASSWORD=something

^D

/opt/gf21/glassfish/bin# ./asadmin create-service --passwordfile /opt/gf21/glassfish/domains/ptmp/.aspass --type das --serviceproperties net_privaddr /opt/gf21/glassfish/domains/ptmp

The Service was created successfully. Here are the details:

Name of the service:application/SUNWappserver/ptmp

Type of the service:Domain

Configuration location of the service:/opt/gf21/glassfish/domains

Manifest file location on the system:/var/svc/manifest/application/SUNWappserver/ptmp_opt_gf21_glassfish_domains/Domain-service-smf.xml.

The service could be enabled using svcadm command.

Command create-service executed successfully.

Ckeck that the newly created service is online. 

# svcs -a | grep ptmp

online         14:26:14 svc:/application/SUNWappserver/ptmp:default

If not make it so.

# svcadm enable svc:/application/SUNWappserver/ptmp:default

An extra sanity check:

# netstat -a | grep 80

Time-out when calling file_get_contents on PHP scripts hosted on Godaddy accounts

I noticed that if you're using a Godaddy shared web hosting plan you're limited to calling file_get_contents only on URLs that use the standard HTTP port 80. 

It may be a firewall rule that they have in place.

So if you have an url like http://app.example.com:8080/do?param=value, 

the call to file_get_contents will hang and eventually timeout, 

even if the same url is accessible directly from within a browser.

If you control the called web component you may be able to change it 

so that it would listen on the standard port 80.

Class name is wrong or classpath is not set for : com.mysql.jdbc.jdbc2.optional.MysqlDataSource

If you get this error when trying to ping a newly created JDBC connection pool from within GlassFish admin console, it's likely because you're missing the JDBC driver mysql-connector-java-5.1.*-bin.jar under the lib/ext folder on your domain. 


To fix it copy that file there (e.g. ~/glassfish-v2ur2/domains/domain1/lib/ext/mysql-connector-java-5.1.5-bin.jar ), restart the domain and try again.

You can get the latest copy of the driver from here.


Update: There was at least a comment below, indicating that this method may not work for GlassFish version 3.1.1 and the more recent JDBC connection drivers. Your best bet may be to use the official docs, for the particular version of GlassFish you're running. For 3.1.1 one doc and another doc indicates that mysql-connector-java-5.1.14-bin.jar should be placed under the lib folder, not under lib/ext folder. 

How to troubleshoot the bootup of GlassFish + MySQL image on OpenSolaris

I've been using an Amazon image with GlassFish+MySQL+OpenSolaris. As I'm no sysadmin, I relied on a prebuilt image furnished by Sun as a base for my development.

Still I've been experiencing  various issues at bootup, when the app server wouldn't start. This is especially true when you reboot the instance with Amazon EC2 tools. It behaves better if you terminate and restart the instance.

So I devised a set of steps to ensure that the thing gets working. 

Check if the services are running:

svcs  | grep mysql

svcs  | grep glassfish

If not, start them:

svcadm restart svc:/application/mysql:default

svcadm restart svc:/application/glassfish/domain1:default

Keep an eye on the server log

e.g. tail -f /opt/gf21/glassfish/domains/domain1/logs/server.log

If that reports failure, you have work to do.

Sometimes the app server can't start due to a previous process not having released the essentials TCP ports. Do a

netstat -a | grep 3700 

if it shows some daemons listening do a

ps -ef

look for usr/jdk/instances/jdk1.6.0/jre/../bin/java -Dcom.sun.aas.instanceRoot=/opt/gf2

That may be a stale instance. Try to kill -9 and look for the process above to be vanished and the port 3700 to be free.

Try again 

svcadm restart svc:/application/glassfish/domain1:default

transient error: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)

While experimenting with an Amazon OpenSolaris+GlassFish+MySQL AMI. I have been getting some weird errors when launching mysql. 

-bash-3.2$ mysql -u root -p

Enter password: 

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)

I don't know if it has to do with this particular image (install/config issue) or if it's a side effect of the fact that's run on a virtual server, but a quick workaround seems to be:

svcadm restart mysql ran as root 

error when bundling OpenSolaris - based AMI on Amazon EC2

I needed a server in the cloud for a project, with Glassfish and MySQL installed. Having found a pre-built AMI (Amazon Machine Image) based on OpenSolaris it seemed to be fit for my purpose.

After kicking its tires I decided to place my customizations and re-bundle an image that would have them on upon reboot.

So I followed along the instruction from the Getting Started guide (http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/). Well, no good. The bundling failed with the following error reported by ec2-bundle-vol:

Cloning / into image file /mnt/ptmpimage...
Excluding:
     /export/home
     /rpool
     /export
     /
     /mnt
     /mnt
     /mnt/ptmpimage
     /mnt/ec2-bundle-workspace
Creating flash archive of file system...
Time: 0.086058s
Time: 0.023259s
Time: 0.229003s
Time: 0.023268s
Time: 0.113336s
Time: 0.023153s
Time: 0.036836s
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
devfsadm: disk_link: invalid disk device number (2051)
Time: 0.400378s
ERROR: Failed to evaluate 'flar create -n ec2.archive -S -R / -x /export/home/gf21 -x /export/home/mysql -x /rpool/boot -x /rpool/etc -x /export/home -x /bin -x /boot -x /dev -x /devices -x /etc -x /export -x /home -x /kernel -x /lib -x /lost+found -x /media -x /mnt -x /net -x /opt -x /platform -x /proc -x /.profile -x /root -x /rpool -x /sbin -x /sysbench -x /system -x /tmp -x /usr -x /var -x /mnt/cert-SGB3LC7W3STUADHK6YLJGLH45V6NGQQU.pem -x /mnt/ec2-bundle-workspace -x /mnt/pk-SGB3LC7W3STUADHK6YLJGLH45V6NGQQU.pem -x /mnt/cert-SGB3LC7W3STUADHK6YLJGLH45V6NGQQU.pem -x /mnt/ec2-bundle-workspace -x /mnt/pk-SGB3LC7W3STUADHK6YLJGLH45V6NGQQU.pem -x /mnt/ptmpimage -x /mnt/ec2-bundle-workspace /mnt/ec2-bundle-workspace/archive 2> /dev/null'. Reason: .

It turns out that Solaris AMIs have a different bundling procedure, so the instructions provided by Amazon in their guide don't apply (I think they're for Fedora (or other RedHat derrivatives) images).

For instructions specific to Solaris see this.

"The directory or file cannot be created (0x52)" error while installing S60 5th Edition SDK

While installing some of the newer SDK from Forum Nokia I'm getting greeted by the following error:



Nice, after looking at that bloated folder it's clearly that my FAT32 partition can't cope with the multitude of files that Nokia chose to dump there.

A quick look at FN reveals that others have the same problem (see here) and a suggestion for a fix, by deleting the resources for languages other than English.

So I went ahead an wrote the following script that does the trick:

import fnmatch
import os
import time

path = 'C:\\S60\\devices\\S60_5th_Edition_SDK_v1.0\\epoc32\\release\\winscw\\udeb\\z\\resource'

while True:

   try:
       for file in os.listdir(path):
           if (fnmatch.fnmatch(file, '*.r0[023456789]') or
               fnmatch.fnmatch(file, '*.r[123456789][0123456789]') or
               fnmatch.fnmatch(file, '*.r[123456789][0123456789][0123456789]')):
               print file
               os.remove(path + '\\' + file)
         
   except IOError as (errno, strerror):
       print "I/O error({0}): {1}".format(errno, strerror)
   except WindowsError as (errno, strerror):
       print "Windows error({0}): {1}".format(errno, strerror)
   except:
       print "Unexpected error:", sys.exc_info()[0]
       raise

   time.sleep(10)

Update the path variable with your installation path and run this script after you hit the first error and chose retry. When you're done with the install kill the script.

missing WRT widget components when packaging with Aptana Studio

I've been using the Aptana Studio Community Edition and its plugin for creating S60 WRT widgets. It has a few good features that increase productivity, like a decent editor with code formatting, integrated contextual help, packaging and deploying functioning.

One annoying issue I found was that after adding new components to the widget, e.g. new .js, .png files, they are not picked up for packaging, unless you do a project refresh (right-click on the project in the project pane on the left side of the screen and chose "Refresh").

Failing to do this may result in wasting time debugging the unexpected behavior. One such step in debugging could be renaming the .wgz package to .zip and exploring that zip's content to check for the newly added components. If they are missing, surely a refresh is needed.

Unneeded white space in textarea

I noticed an annoying behavior of textarea elements that have white space between the opening and closing tags. That space shows up as the initial value of that element in the browser. 

I've been using an edit control built on top of a textarea for a recent project and I noticed that upon clicking on the control to get focus, the caret cursor would not land in the top let corner of the control, instead it would have a few characters of left padding. 

That's because my Komodo Edit IDE was formatting the code by putting the end tag on a different line and would add sufficient white space for a balanced indentation. That was causing the padding to appear. 

The workaround is to keep the textarea element with a truly empty value like this:

<textarea></textarea>

How to install GlassFish on CentOS and other Redhat derivatives

First you need to have Java installed, I prefer OpenJDK, which is available from EPEL , a community-run project which makes Fedora packages available to users of Red Hat Enterprise Linux 5, CentOS 5, and other RHEL 5 derivatives. First install the package that enables the EPEL repository:

su -c "rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-2.noarch.rpm"

Then fetch the packages needed:

su -c "yum install java-1.6.0-openjdk"

su -c "yum install java-1.6.0-openjdk-devel"

Set the JAVA_HOME environment variable to point to the location of the JDK installation. To find that location you can do a which java and repeated  ls -l to follow the links to the concrete location.

Do the installation, e.g.:

java -Xmx256m -jar glassfish-installer-v2.1-b60e-linux.jar

Add the as-install/bin/ directory to the PATH environment variable.

To connect to the admin console, open a web browser to on http://hostname:4848

For login, the default admin user name is admin and the password is adminadmin. 

  

How to enable access logging in GlassFish

For debugging, monitoring and audit purposes it's sometimes useful to analyze access logs. 

By default Sun's GlassFish server has such logging disabled. It can be enabled by editing the domain.xml file (e.g. domains/domain1/config/domain.xml) and changing the accessLoggingEnabled attribute of the http-service element to true.

Following the restart of GlassFish the logs are available under the logs/access folder, (e.g. domains/domain1/logs/access/).

Driving Recorder at 5000 downloads

Following the posting of my Driving Recorder application on GetJar at the beginning of the year I just went through the milestone of passing the 5000th download. 

So far the feedback from these people trying the app has been mute. The popularity seem to be holding steady, but the lack of feedback, and purchases :), makes me think that either their reporting system is broken or those numbers are mostly the result of site duplication robots.

How to install VMWare tools on CentOS 5.0

VMWare tools are needed for a smooth integration between the guest and host OS. I'm currently using VMWare Fusion for Mac, and I needed CentOS for a project. Here are the steps I went through:

In CentOS do yum install gcc kernel-devel

Compare the version of OS, reported by the uname -r with that of kernel headers, reported by rpm -q kernel-devel. If there's a mismatch, update the headers with yum -y upgrade kernel kernel-devel and reboot.

In WMware Fusion go to the Virtual Machine menu -> Install VMware Tools. 

In CentOS do a mount /dev/cdrom /mnt. Now you should have a gzipped tar with the tools in /mnt. Extract them into the /tmp folder with tar -C /tmp -zxvf /mnt/VMwareTools*.tar.gz

cd /tmp/vmware-tools-distrib

./vmware-install.pl and follow the prompts.

How to hide the .php extension in your URLs

Sometime you could implement web components that expose public REST-ful interfaces implemented in PHP. In those cases it may be useful to expose end points that are technology-agnostic. 

So, for example, instead of http://example.com/handler.php?param1=val1&param2=val2 you'd have http://example.com/handler?param1=val1&param2=val. Thus, you could later on decide to switch the underlying technology to something like servlets and reuse the old format for the URLs.

An easy way to accomplish this is using Apache's ability to do URL rewriting. Simply edit the .htaccess file in the root folder of your web directory and place the following directives:

RewriteEngine on 

RewriteRule handler handler.php

How to install MySQL on CentOS and other RedHat derivatives

One simple solution is to rely on yum and do:

sudo yum install mysql-server.i386

To ensure that the daemon starts automatically at boot-up you can do:

sudo chkconfig mysqld on

Where is httpd.conf located on Ubuntu ?

/etc/apache2

"Update error" when installing SIS or SISX bundles

I came across a really useful service provided by Nokia, which gives you access to a pool of devices (real hardware) for testing.

Although I intended to use that for a different project, I thought I might try an app that I developed earlier, to see how it looks on different screen resolutions. To my surprise, the app failed to install on some phones, with the insightful error message "Update error". This is as bad as it gets in terms of usability on Nokia's side. It would have been nice to get a more detailed explanation to speed up the progress.

After a bit of digging it turns out was because my package had a file that already existed on the phone, that special c:\resource\errrd which, ironically, enables more detailed error messages. 

One lesson learned would be to ensure that all files installed by the app have a unique id, usually the app UID, somewhere in their full name (path included), to avoid collisions. Also, in hindsight I should have made a separate "enableDetailedErrMessages.sis" that would install c:\resource\errrd.

The other important lesson is to be more specific in the error messages. If there are more details that could assist the user, by all means be forthcoming.

JSM messages left in queue, not consumed

I've been working on a project involving a servlet that had to consume messages from a JMS queue and dole them out to the requesting client as they arrived. For this I chose the asynchronous message consumption, to avoid blocking the servlet thread, to allow for other processing that had to be done while waiting for messages to arrive.

When testing I noticed that the servet would pick up the messages as they arrived in the queue, but for some reason they would be left there. At the same time the following exception would appear in the server log:

DirectConsumer:Caught Exception delivering messagecom.sun.messaging.jmq.io.Packet cannot be cast to com.sun.messaging.jms.ra.DirectPacket.

After doing some investigations I found this notice in the J2EE tutorial: "The Java EE platform specification does not impose strict constraints on how web components should use the JMS API. In the Application Server, a web component can send messages and consume them synchronously but cannot consume them asynchronously".

Switching to synchronous consumption, using a timed receive took care of the problem.

Reformatting the Economist web page to fit small screens

I've been a reader of the Economist magazine, using the Opera browser on my Nokia 770 Internet tablet. The recent changes in the layout of the page introduced the need for horizontal scrolling when zooming the view to make the fonts larger. This affects the usability quite severely.

I implemented a workaround for that, as a bookmaklet which increases the font size of the main pane without increasing its width, so the content reflows nicely within the screen bounds.

Here's the code; add it as the "url" field in your bookmark, then simply load the bookmark once you're on the page you want magnified.

javascript:function k34(){c = document.getElementById('content'); d=c.getElementsByTagName('div');for(i=0;i<d.length;i++){
if(d[i].className=='col-left'){d[i].style.fontSize='x-large';break;}}}; k34();


Note: this should be a single line, no breaks

The need for technology

I've been thinking on how technology, particularly software, has been satisfying the basic human needs and what areas hold potential for further developments.

1. The need for certainty and security - usually involves basic surviving needs (food, shelter, clothes, etc). Since lots of people drive their security from a steady job or income, anything that can help them do their job better would satisfy this need. No wonder that business software has such success. In the same category would also go tools for managing personal finances and finding investment opportunities.

2. The need for uncertainty, adventure, novelty. This area is mostly satisfied by entertainment applications (games, media distribution/consumption) and e-commerce promoting travel and adventure.

3. The need for uniqueness, appreciation and social proof. This can explain the success of services that allow users to create content and claim ownership and enhance status (blogs, media sites, discussion groups).

4. The need for community, belongging and intimacy is covered by participating in social networks, dating services, etc.

5. The need for growth and learning is met by online education and news feeds. This is probably the area that's currently best served by the avalanche of information that can be accessed online. The challenge is how to filter that based on individual interests.

6. The need to contribute and make the world a better place. This could explain the widespread of open-source projects and people contributing their expertise without immediate payoff. The web also helps spread awareness about worthy innitiatives in need of resources.

privacy with Google

Many people are using Google applications like Gmail, Docs and Calendar to store and transmit confidential information. However many are not aware that using such apps over networks that are routinely monitored (e.g. your employer's network) may expose that information.

That's because by default people use the plain "http" URI scheme when accessing Google's apps. As a result the information between your browser and Google's server flows in clear.

A straightforward solution is to use the "https" scheme, which results in the same information flowing encrypted using SSL/TLS, providing much better security.

I tried some of the apps I use and it works well (e.g. https://www.gmail.com, https://docs.google.com, https://www.google.com/calendar)

iPhone mania

With a few days away from the official launch, it's interesting to see how iPhone will fare in the marketplace. I'm expecting to see a well-thought usable UI and seamless integration with other applications like iTunes. Following the good tradition of Apple, everything is just going to work, right off the box.

One thing I'm not sure is the overall robustness of the operating system. Retrofitting OSs that were designed to run on a desktop for use in an embedded environment, with scarce resources, so far produced mediocre results. Contrast that with an OS like Symbian, built from ground-up with the ability to gracefully handle exceptions common in the embedded environments. One interesting metric to look for will be the average time between reboots.

Will the users care about reboots ? As the line between the phone (thus far perceived as a robust device) and the computer blurs, it remains to be seen.