Error when running Xcode sample projects

I've been through some sample code downloaded from the net and after opening a project and attempting to build it I was greeted with the following error:

"There is no SDK with specified name or path 'Unknown Path'". This is likely because those samples were built with a different version of the SDK than those I had installed.

A simple fix is to go to Project->Edit Project Settings, then go to "Base SDK for All Configurations" drop-down list and pick a version that you have installed on your system instead of the grayed (missing) one that's displayed.

installing/upgrading Python + MySQLdb on Mac OS X

I've been working on a project that required the use of the urllib2.urlopen that takes a 3rd argument to indicate the guard timeout to break the blocking on an unresponsive socket. Alas this handy feature is only available in the versions of Python starting with 2.6. As Leopard comes with 2.5, I was faced with the prospect of upgrading to 2.6, hopefully without breaking existent functionality.

First step is to see where Python is installed by default:
which python
yields
/usr/bin/python
and
ls -l /usr/bin/python
yields
lrwxr-xr-x 1 root wheel 72 21 Feb 2008 /usr/bin/python -> ../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/python

So there are the goodies. Good to know.


Next download Python for Mac from here.
Unpack and run
python setup.py build:
Traceback (most recent call last)
File "setup.py", line 5, in
from setuptools import setup, Extension
ImportError: No module named setuptools

BUMMER !

Go to http://pypi.python.org/packages/2.6/s/setuptools/ and download the current .egg version. I got a setuptools-0.6c9-py2.6.egg.sh.

Ok, now I do:
chmod u+x ./setuptools-0.6c9-py2.6.egg.sh
./setuptools-0.6c9-py2.6.egg.sh
to get greeted by
./setuptools-0.6c9-py2.6.egg.sh is not the correct name for this egg file.
Please rename it back to setuptools-0.6c9-py2.6.egg and try again.

Fine,
mv ./setuptools-0.6c9-py2.6.egg.sh ./setuptools-0.6c9-py2.6.egg
./setuptools-0.6c9-py2.6.egg

Processing setuptools-0.6c9-py2.6.egg
Copying setuptools-0.6c9-py2.6.egg to /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages
Adding setuptools 0.6c9 to easy-install.pth file
Installing easy_install script to /Library/Frameworks/Python.framework/Versions/2.6/bin
Installing easy_install-2.6 script to /Library/Frameworks/Python.framework/Versions/2.6/bin

Installed /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg
Processing dependencies for setuptools==0.6c9
Finished processing dependencies for setuptools==0.6c9

And we give it another try:

python setup.py build

yields

running build
running build_py
creating build
creating build/lib.macosx-10.3-fat-2.6
copying _mysql_exceptions.py -> build/lib.macosx-10.3-fat-2.6
creating build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/__init__.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/converters.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/connections.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/cursors.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/release.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/times.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
creating build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/__init__.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/CR.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/FIELD_TYPE.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/ER.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/FLAG.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/REFRESH.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/CLIENT.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
running build_ext
building '_mysql' extension
creating build/temp.macosx-10.3-fat-2.6
gcc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -Dversion_info=(1,2,3,'gamma',1) -D__version__=1.2.3c1 -I/usr/local/mysql/include -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c _mysql.c -o build/temp.macosx-10.3-fat-2.6/_mysql.o -Os -arch i386 -fno-common
gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -bundle -undefined dynamic_lookup build/temp.macosx-10.3-fat-2.6/_mysql.o -L/usr/local/mysql/lib -lmysqlclient_r -lz -lm -o build/lib.macosx-10.3-fat-2.6/_mysql.so
ld warning: in build/temp.macosx-10.3-fat-2.6/_mysql.o, file is not of required architecture
ld warning: in /usr/local/mysql/lib/libmysqlclient_r.dylib, file is not of required architecture

I think the 2 warnings have to do with the "-arch ppc" in the 2nd invocation of gcc, but given that we also have the "-arch i386" there we're probably OK.

Next

sudo python setup.py install

Password:
running install
running bdist_egg
running egg_info
writing MySQL_python.egg-info/PKG-INFO
writing top-level names to MySQL_python.egg-info/top_level.txt
writing dependency_links to MySQL_python.egg-info/dependency_links.txt
reading manifest file 'MySQL_python.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'MySQL_python.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.3-fat/egg
running install_lib
running build_py
copying MySQLdb/release.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
running build_ext
creating build/bdist.macosx-10.3-fat
creating build/bdist.macosx-10.3-fat/egg
copying build/lib.macosx-10.3-fat-2.6/_mysql.so -> build/bdist.macosx-10.3-fat/egg
copying build/lib.macosx-10.3-fat-2.6/_mysql_exceptions.py -> build/bdist.macosx-10.3-fat/egg
creating build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/__init__.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/connections.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
creating build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/__init__.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/CLIENT.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/CR.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/ER.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/FIELD_TYPE.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/FLAG.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/REFRESH.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/converters.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/cursors.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/release.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/times.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
byte-compiling build/bdist.macosx-10.3-fat/egg/_mysql_exceptions.py to _mysql_exceptions.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/connections.py to connections.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/CLIENT.py to CLIENT.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/CR.py to CR.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/ER.py to ER.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/FIELD_TYPE.py to FIELD_TYPE.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/FLAG.py to FLAG.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/REFRESH.py to REFRESH.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/converters.py to converters.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/cursors.py to cursors.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/release.py to release.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/times.py to times.pyc
creating stub loader for _mysql.so
byte-compiling build/bdist.macosx-10.3-fat/egg/_mysql.py to _mysql.pyc
creating build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/PKG-INFO -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/SOURCES.txt -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/dependency_links.txt -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/top_level.txt -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
writing build/bdist.macosx-10.3-fat/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg' and adding 'build/bdist.macosx-10.3-fat/egg' to it
removing 'build/bdist.macosx-10.3-fat/egg' (and everything under it)
Processing MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg
Copying MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg to /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages
Adding MySQL-python 1.2.3c1 to easy-install.pth file

Installed /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg
Processing dependencies for MySQL-python==1.2.3c1
Finished processing dependencies for MySQL-python==1.2.3c1

Looks good. Let's try it:

python
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg/_mysql.py:3: UserWarning: Module _mysql was already imported from /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg/_mysql.pyc, but /Users/bhapca/Downloads/MySQL-python-1.2.3c1 is being added to sys.path

huh ?

>>> dir(MySQLdb)
['BINARY', 'Binary', 'Connect', 'Connection', 'DATE', 'DATETIME', 'DBAPISet', 'DataError', 'DatabaseError', 'Date', 'DateFromTicks', 'Error', 'FIELD_TYPE', 'IntegrityError', 'InterfaceError', 'InternalError', 'MySQLError', 'NULL', 'NUMBER', 'NotSupportedError', 'OperationalError', 'ProgrammingError', 'ROWID', 'STRING', 'TIME', 'TIMESTAMP', 'Time', 'TimeFromTicks', 'Timestamp', 'TimestampFromTicks', 'Warning', '__all__', '__author__', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__revision__', '__version__', '_mysql', 'apilevel', 'connect', 'connection', 'constants', 'debug', 'escape', 'escape_dict', 'escape_sequence', 'escape_string', 'get_client_info', 'paramstyle', 'release', 'result', 'server_end', 'server_init', 'string_literal', 'test_DBAPISet_set_equality', 'test_DBAPISet_set_equality_membership', 'test_DBAPISet_set_inequality', 'test_DBAPISet_set_inequality_membership', 'thread_safe', 'threadsafety', 'times', 'version_info']
>>>

Seems OK I guess.

Bottom line. The steps above seem to result in 2 different versions of Python coexisting side by side, one that came from Apple in

/System/Library/Frameworks/Python.framework/Versions/2.5/bin/python

and the one I installed in

/Library/Frameworks/Python.framework/Versions/2.6/bin/python

cron jobs terminate prematurely on Godaddy accounts

I've been using a large tarball file to upload my online store updates to godaddy and a cron job to unpack the tarball.

As the size of that tarball increased, due to adding more products, I noticed that the tar process would terminate prematurely, causing big issues, with the store left half-updated, in an inconsistent state.

I've been suspecting the folks at godaddy running some sort of watchdog timer and killing off processes that take longer to run.

Well, according to some reports, that indeed seems to be the case.

One option to go around this would be to break the large tarball in several smaller ones, that won't keep tar humming for too long. Also I should perhaps consider sending only actual delta changes instead overwriting the whole store in one go.

connection pools time-outs in GlassFish

I've been using JDBC connection pools and JMS connection factories for a recent project and upon deployment on GlassFish I did a sanity test to confirm that everything is OK. It passed the test.

So off I go, leaving it running on an Amazon EC2 instance. When I check on it about 3 days later I notice that it doesn't work anymore ... "server error", the UI informs me (which is a bad error message for the end users, whom might not know what a server (those working in restaurants) has to do with it). Anyway, by looking at the logs I see a few of these:

Exception: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

and

Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after statement closed.

Now the way I implemented this was in the "init" method of the servlet I would create my instances of javax.jms.Connection and java.sql.Connection as member fields of the servlet, hoping to reuse them for the lifetime of the servlet.

The errors in the logs seem to indicate that those connections get yanked out while my servlet is running, albeit idle. Not cool.

Looking at the admin console I see an "Idle Timeout:" configuration setting which defaults to 300 sec for both my JDBC connection pool and JMS connection factory. I changed that to 300000000. I have the feeling that this is going to fix the issue, but I need to run it for a while to confirm.

I'm not sure if this is the right thing of approaching this, so if you know any better please add a comment.

UPDATE:
It seems that changing the configuration settings for GlassFish is not enough to fix the problem.

Likely the wait_timeout configuration setting of MySQL has something to do with this too. By default mysqld will close idle connections after 8 hours, so we'd need to increase that to up to the max value of 31536000. One way to do that is through a configuration file (e.g. /etc/my.cnf, where we'd add a line with 'wait_timeout= 31536000' under the [mysqld] section.

Yet a better approach would be to catch the exceptions mentioned above in the servlet code and recreate the connection to the DB.

ANOTHER UPDATE:
It seems that catching the exception and re-opening the connection in the servlet code doesn't work very well. I get the following exception when trying to re-establish the connection:

Exception:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

Last packet sent to the server was 0 ms ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
...

Other people have experienced this too. One workaround indicated there, like I mentioned above, is to tinker with wait_timeout configuration of MySQL.

Another avenue to explore would be the "Connection Validation" capabilities of the JDBC connection pool. That's easily available through GlassFish admin console. According to the docs "Optionally, the application server can validate connections before they are passed to applications. This validation allows the application server to automatically reestablish database connections if the database becomes unavailable due to network failure or database server crash. Validation of connections incurs additional overhead and slightly reduces performance."

It'd be interested to see if the "table" validation method fixes this issue.