be careful when deleting web content

Until recently, when I was updating the inventory for www.negustore.com I used to delete old product pages, that I wasn't carrying any longer. To be honest, I didn't realize I was doing that, instead I discovered it through analytics.

So here I was, yanking out pages that were already indexed by Google, and sending incoming traffic to my custom 404 handler. After the required head-smack I changed my code to keep the product pages and change the availability to "not available" and remove the add to cart button. That's the sensible thing to do. Lots of people may still be interested in the product specs, even if they are not on sales anymore. Better yet I could add some adsense pointing to places that sells them.

Interesting thought, it'd be nice if adsense would give us more control to what's included in each ad block, e.g. product for sale, instead of redundant product info, in my case.

I'm still wondering, how does removing of pages that were previously indexed affect the overall page rank of the site. Common sense it tells it can't be good. The last thing a search engine wants is to send people over to broken links.

Same thing goes for moving things around, if possible don't do it, if there's no choice, use 302 permanent redirects handlers that still listen to the old URLs until they are removed from the indexes.

is it worth pursuing search engine friendly URLs ?

As someone who's running a few e-commerce sites, I'm always on the look for ways to attract traffic to my sites. One of the best sources of traffic are search engines, and the key to attract traffic from them is to observe the best SEO, or search engine optimization, practices.

One of those practices widely touted was to use SEFU, or search engine friendly URLs. That means avoiding dynamic URLs, that contain a query segment with parameters (anything following the '?' sign, in the form 'name=value', maybe separated by '&') and instead use static URLs that contain targeted keywords, separated by '-'. There are many ways to translate a dynamic URL into a static one. I used url rewriting capabilities provides by the mod_rewrite module of Apache web server. Yet I'd been thinking, if it's so easy to make this translation, why would the search engines make such a big fuss about. After all, the content they get is the same, so it shouldn't make any difference.

Yet it makes all the difference in the world. You see, when you use a dynamic URL, usually with cryptic values as parameters, like product numbers, or session id, or whatever, you make no clear assertion about the content the URL is pointing too. That is, a subsequent fetching of the same URL may produce an entirely different content, since after all it's the result of a dynamic script, right ? The content may change based on internal state like the date or time, or based on the parameters passed. So understandably the search engines are reluctant to place too much weight on the content. Yes, you could use the title tag, or metadata to describe the content, but still, nothing guarantees that the mapping between the URL and the content will stay the same.

On the other hand, when you use a static URL, with embedded keywords, you add semantics to the URL, making its destination clear, giving it a sense of permanence. Now that's something that the search engines can bank on.

So I changed the product URLs on my www.greencirclestore.com site to be SEFU and the results were excellent. The amount of traffic from search engines doubled and the crawling activity kicked into high gear. All in all I'm pretty pleased with the results.

"Unknown Error. Please reload the page." when using the File Manager part of Hosting Control Center from godaddy

I've been using the File Manager from godaddy to unzip some tarballs with updates to my web site. I had to rely on this somehow manual approach as any attempt to do the same programatically, through scripts launched by cron jobs would be cut short. That's because there seems to be a limit on how long a user-initiated process could run on godaddy.

In some cases I'd get an "Unknown Error message" after some unzipping finishes and I try to start a new one. That's presumably because the file manager tries to figure out all new files created as a result of unzipping. However beside showing that useless error message, for just a few seconds before hiding it (awful usability), the error condition seems to recover only after logging out and logging back in after at least 5 minutes.

I discovered one way to avoid the error in the first place. Don't start a new unzipping until the first one stops for good. When does that happen ? Not when the system tells you, but when you see that the checkbox for the archived being unzipped becomes unselected. Depending on the complexity of the archive, that may happen a few minutes after the unzipping finishes.

The "iron triangle" concept in Project Management

The PM literature refers to the triple constraint of scope, time and cost as the "iron triangle" to help visualize the interlocked dependencies between these constraints. Imagine a triangle made of iron, with each corner of the triangle corresponding to the scope, time and cost, respectively, while the iron core represents the quality of the result produced by a given project. If you try to stretch any corner, you're bound to break the quality at the core.

Im not sure I buy this metaphor. In my view both time and cost are resources, together with other, like people, skill-set, etc. Each of these have a bearing to the outcome of the project. You may be tackling an impossible problem, so whatever resources you throw at it you're still far from meeting the goals of the project, as defined by its scope. That is, until you come across the mix of resources that makes the impossible possible.

So sow about a silk string instead of the iron triangle ? At one end is the scope, at the other are the resources. The string itself is the quality. If you pull too hard at one end it's bound to break.

Principles of defensive programming

I came about this topic while watching a course presentation. They mentioned that we should always validate the input provided by the user and the results returned by the code we didn't write (library calls).

Based on my experience I'd add using assertions to document and check for invariants within our code. Another use of assertions is when exposing an API to other parties. If there are preconditions that the calls to the API must satisfy, it's pretty effective to enforce them through assertions.

Another area to watch for is to always check the results (success status) of the function calls we make and act accordingly. Sometimes we tend to be too optimistic and assume that everything will work smoothly. This is somehow alleviated by the use of exceptions, which prevents continuing the processing under false assumptions, which usually yield unexpected results, hard to trace back to their root cause. Unless the exception is muted, that is, caught and ignored, (really bad practice) any problem surfaces quickly.

302 redirects and iframe

I was quite surprised to discover that the behavior of posting a form while targeting an iframe depends on whether the form handler performs an HTTP 302 redirect.

I have a web page, with an iframe embedded and a form that targets that iframe. If I post the form and the handler returns a normal 200 result, the iframe will end up with the content sent by the handler, while keeping the surrounding content intact. That's the behavior that I was intuitively expecting.

However if the form handler returns an 302 code, instructing the browser to do a redirect, the whole page gets redirected, so the iframe and surrounding content vanish, being replaced by whatever content is served by the redirection.

This is quite surprising to me, as I would have expected only the target iframe to be affected by the posting of the form. After all that's the semantic of the target, isn't it ? This odd behavior happens in both FireFox and Safari on Mac, so it may not be a bug, but by design. I'd be curious about the rationale behind this.

How to debug PHP scripts on your Mac

I use a MacBook as one of my development machines. For LAMP development I've been using the excellent setup provided by MAMP. 


Coming from a background in other languages that have readily available debuggers I've been longing for a debugger to help stepping through PHP code and inspecting the content of variables. I found out that some commercial IDEs come with such capabilities out-of the box, but I've been looking for a stand-alone solution, similar to GDB. I can highly recommend xdebug for that. 


Installing it on MAMP was a breeze, by following the excellent instructions from technosophos. The only changes from those instructions were that the latest versions of MAMP (I use version 1.8.4) come with an xdebug.so already present (i.e. /Applications/MAMP/bin/php5/lib/php/extensions/no-debug-non-zts-20060613/xdebug.so), so all you need is uncommenting the zend_extension entry in the php.ini. 


Coupled with the MacGDBp client, the xdebug provides exectly what I've been looking for. Just pay attention to the need to press the reset button in order to initiate a new debugging session when your web page loads.

shoptoit sucks !

I've been using shoptoit.com for some PPC (pay-per-click) advertising campaigns for my online store, on and off, for some time. Although the ROI is pretty low, I've been thinking to start some new campaigns. 


It's quite amazing how poorly implemented that site is. For merchants they don't provide even such simple tools for updating the list of products they wish to be advertised, all at once, in batch mode. So if you have some products that are out-of date, you're out-of-luck. There's no apparent way to replace them, through a new feed. There's not even a way to delete a product listing, not even manually. Mind boggling. Hello, CRUD, anyone ?


Adding to this deplorable state, any requests to their customer service, via email, on 3 separate addresses, seem to fall on deaf ears. They're probably too busy spewing nonsense on social networks, as they appear to be overly active on twitter and facebook, instead of caring for paying customers. 


Given this, I can highly recommend avoiding them. I'll be trying to get my remaining balance that's in my account and move to better providers.

Terminating app due to uncaught exception 'NSInternalInconsistencyException'

One reason for getting this exception may be due to a typo in the name of the NIB file used to init an object.

In my case I was loading a view controller like so:


YellowViewController *yellowController = [[YellowViewController alloc] initWithNibName:@"YellowViev" bundle:nil];



whilst my bundle contained an YellowView.xib. I was using an "v" instead "w" in my NIB name.

So, yeah, check the names. XCode is pretty forgiving (and misleading) here. It would instantiate an object (from an non-existent NIB), in my case a view controller with a view member set to nil. Perhaps a better approach would be to return nil instead a half-baked, incorrect object. Instead it set me to a wild goose chase, to check the view outlet in the NIB and the ensuing scratching of the head.

The error message before the crash wasn't too helpful either:

ViewSwitcher[30112:20b] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: '-[UIViewController loadView] loaded the "YellowViev" nib but no view was set.'

Solving the (HY000/2002): Can't connect to local MySQL server through socket '/var/mysql/mysql.sock' on Mac OS 10.5

I've been using the Apache version that comes pre-installed with the Leopard, and its companion installation of PHP, that can be enabled as described here. I figured that if Apple took the trouble to pre-install these, I'd better use them instead of re-installing them from scratch.

I also installed MySQL using the dmg package from mysql.com.

At this point PHP and MySQL seemed to be working fine on my system, each tried separately.

Things went ugly when I tried to call MySQL from within a PHP script. Calling $dbc = mysqli_connect(MSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB) yielded the HY000/2002 error, complaining about the inability to connect through /var/mysql/mysql.sock.

The docs for mysqli_connect indicate that if the host to connect to is the local host, the system will attempt to use a pipe instead of TCP/IP, perhaps for improved efficiency. So it looks like the /var/mysql/mysql.sock pipe is not present. A netstat -a | grep mysql indicates that that's indeed the case, instead there seems to be a /tmp/mysql.sock pipe on my system.

Nice, all I have to do is make use of that. A quick change to $dbc = mysqli_connect(MSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB, ini_get("mysqli.default_port"), '/tmp/mysql.sock') makes it work. Good times.

Not this is clearly a thing peculiar to my setup, I'd rather fix my setup instead of changing my code to work around this, especially since I need the code to be generic enough to run on a production server, with a different setup. So I take a look at the output provided by phpinfo. Here I see that MYSQL_SOCKET is /var/mysql/mysql.sock and mysqli.default_socket is not set. Then all I need to fix my system is to set mysqli.default_socket to '/tmp/mysql.sock'. To do that, I changed my /private/etc/php.ini to have the 'mysqli.default_socket =' line changed to 'mysqli.default_socket = /tmp/mysql.sock' and restart Apache (System Preferences -> Sharing -> Web Sharing disable and enable again).

Now my system is good to go, I can connect to MySQL from within PHP simply with $dbc = mysqli_connect(MSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB).

Note: The same issue may affect connecting through the regular mysql_connect, in which case you'll have to change your mysql.default_socket setting in a similar way.

Push To My Phone wins Judge's Choice Award in the 2009 Calling All Innovators Contest, presented by Forum Nokia

My latest project, Push To My Phone, has been awarded a prize in a competition I entered.

It's been an interesting experience, now having participated in 2 editions of the contest. I don't think I will enter in future editions, mainly because the way people at Nokia treat the participants. Basically if you're not one of the top price winners, who get monetary awards, you won't get much. As a Judge's Choice Award winner you're supposed to get a membership in their Launchpad program, though I'm still waiting on that. If you're one of the thousands of participants that don't get an award you don't get anything. No recognition at all, not even a thank you.

After the first edition I posted some feedback that was acknowledged, but promptly ignored in the second edition. Actually things went downhill, at least in my perception. I have more suggestions for improvement, but why bother.

I'm not sure what the goal of this contest is for Nokia, but I can tell you that they managed to alienate a few developers. If you look at the number of views on their discussion boards, in the threads related to the contest, you can see that the interest is waning from one edition to another.

There are way better opportunities out there.

dynamically added web content in IE fails to display properly

If you're doing content modifications through DOM operations driven by JavaScript you must be aware that IE doesn't pick the CSS style dictated by the class name of the newly added elements.

So if you add an element like this:

<div id="test_id" class="test_class">example<div>

and your style comes from a CSS include like this:

<link rel="Stylesheet" href="css/default.css" type="text/css" media="screen,print" />

and you have styles corresponding to test_class (e.g. .test_class selector) in the externally included default.css file, they won't be applied.

This is a rather egregious issue, affecting only IE, that bit me several times.

One workaround is to use inline CSS, through the style attribute of the element. That seems to be working fine.

It'd be interesting to see if inline CSS in the <style> element gets picked up too. Also I wonder if the same issue is present for id selectors (e.g. #test_id selector).

Error when running Xcode sample projects

I've been through some sample code downloaded from the net and after opening a project and attempting to build it I was greeted with the following error:

"There is no SDK with specified name or path 'Unknown Path'". This is likely because those samples were built with a different version of the SDK than those I had installed.

A simple fix is to go to Project->Edit Project Settings, then go to "Base SDK for All Configurations" drop-down list and pick a version that you have installed on your system instead of the grayed (missing) one that's displayed.

installing/upgrading Python + MySQLdb on Mac OS X

I've been working on a project that required the use of the urllib2.urlopen that takes a 3rd argument to indicate the guard timeout to break the blocking on an unresponsive socket. Alas this handy feature is only available in the versions of Python starting with 2.6. As Leopard comes with 2.5, I was faced with the prospect of upgrading to 2.6, hopefully without breaking existent functionality.

First step is to see where Python is installed by default:
which python
yields
/usr/bin/python
and
ls -l /usr/bin/python
yields
lrwxr-xr-x 1 root wheel 72 21 Feb 2008 /usr/bin/python -> ../../System/Library/Frameworks/Python.framework/Versions/2.5/bin/python

So there are the goodies. Good to know.


Next download Python for Mac from here.
Unpack and run
python setup.py build:
Traceback (most recent call last)
File "setup.py", line 5, in
from setuptools import setup, Extension
ImportError: No module named setuptools

BUMMER !

Go to http://pypi.python.org/packages/2.6/s/setuptools/ and download the current .egg version. I got a setuptools-0.6c9-py2.6.egg.sh.

Ok, now I do:
chmod u+x ./setuptools-0.6c9-py2.6.egg.sh
./setuptools-0.6c9-py2.6.egg.sh
to get greeted by
./setuptools-0.6c9-py2.6.egg.sh is not the correct name for this egg file.
Please rename it back to setuptools-0.6c9-py2.6.egg and try again.

Fine,
mv ./setuptools-0.6c9-py2.6.egg.sh ./setuptools-0.6c9-py2.6.egg
./setuptools-0.6c9-py2.6.egg

Processing setuptools-0.6c9-py2.6.egg
Copying setuptools-0.6c9-py2.6.egg to /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages
Adding setuptools 0.6c9 to easy-install.pth file
Installing easy_install script to /Library/Frameworks/Python.framework/Versions/2.6/bin
Installing easy_install-2.6 script to /Library/Frameworks/Python.framework/Versions/2.6/bin

Installed /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/setuptools-0.6c9-py2.6.egg
Processing dependencies for setuptools==0.6c9
Finished processing dependencies for setuptools==0.6c9

And we give it another try:

python setup.py build

yields

running build
running build_py
creating build
creating build/lib.macosx-10.3-fat-2.6
copying _mysql_exceptions.py -> build/lib.macosx-10.3-fat-2.6
creating build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/__init__.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/converters.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/connections.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/cursors.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/release.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
copying MySQLdb/times.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
creating build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/__init__.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/CR.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/FIELD_TYPE.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/ER.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/FLAG.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/REFRESH.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
copying MySQLdb/constants/CLIENT.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb/constants
running build_ext
building '_mysql' extension
creating build/temp.macosx-10.3-fat-2.6
gcc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -Dversion_info=(1,2,3,'gamma',1) -D__version__=1.2.3c1 -I/usr/local/mysql/include -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c _mysql.c -o build/temp.macosx-10.3-fat-2.6/_mysql.o -Os -arch i386 -fno-common
gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -g -bundle -undefined dynamic_lookup build/temp.macosx-10.3-fat-2.6/_mysql.o -L/usr/local/mysql/lib -lmysqlclient_r -lz -lm -o build/lib.macosx-10.3-fat-2.6/_mysql.so
ld warning: in build/temp.macosx-10.3-fat-2.6/_mysql.o, file is not of required architecture
ld warning: in /usr/local/mysql/lib/libmysqlclient_r.dylib, file is not of required architecture

I think the 2 warnings have to do with the "-arch ppc" in the 2nd invocation of gcc, but given that we also have the "-arch i386" there we're probably OK.

Next

sudo python setup.py install

Password:
running install
running bdist_egg
running egg_info
writing MySQL_python.egg-info/PKG-INFO
writing top-level names to MySQL_python.egg-info/top_level.txt
writing dependency_links to MySQL_python.egg-info/dependency_links.txt
reading manifest file 'MySQL_python.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'MySQL_python.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.3-fat/egg
running install_lib
running build_py
copying MySQLdb/release.py -> build/lib.macosx-10.3-fat-2.6/MySQLdb
running build_ext
creating build/bdist.macosx-10.3-fat
creating build/bdist.macosx-10.3-fat/egg
copying build/lib.macosx-10.3-fat-2.6/_mysql.so -> build/bdist.macosx-10.3-fat/egg
copying build/lib.macosx-10.3-fat-2.6/_mysql_exceptions.py -> build/bdist.macosx-10.3-fat/egg
creating build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/__init__.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/connections.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
creating build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/__init__.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/CLIENT.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/CR.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/ER.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/FIELD_TYPE.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/FLAG.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/constants/REFRESH.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb/constants
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/converters.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/cursors.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/release.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
copying build/lib.macosx-10.3-fat-2.6/MySQLdb/times.py -> build/bdist.macosx-10.3-fat/egg/MySQLdb
byte-compiling build/bdist.macosx-10.3-fat/egg/_mysql_exceptions.py to _mysql_exceptions.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/connections.py to connections.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/CLIENT.py to CLIENT.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/CR.py to CR.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/ER.py to ER.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/FIELD_TYPE.py to FIELD_TYPE.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/FLAG.py to FLAG.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/constants/REFRESH.py to REFRESH.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/converters.py to converters.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/cursors.py to cursors.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/release.py to release.pyc
byte-compiling build/bdist.macosx-10.3-fat/egg/MySQLdb/times.py to times.pyc
creating stub loader for _mysql.so
byte-compiling build/bdist.macosx-10.3-fat/egg/_mysql.py to _mysql.pyc
creating build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/PKG-INFO -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/SOURCES.txt -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/dependency_links.txt -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
copying MySQL_python.egg-info/top_level.txt -> build/bdist.macosx-10.3-fat/egg/EGG-INFO
writing build/bdist.macosx-10.3-fat/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg' and adding 'build/bdist.macosx-10.3-fat/egg' to it
removing 'build/bdist.macosx-10.3-fat/egg' (and everything under it)
Processing MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg
Copying MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg to /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages
Adding MySQL-python 1.2.3c1 to easy-install.pth file

Installed /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg
Processing dependencies for MySQL-python==1.2.3c1
Finished processing dependencies for MySQL-python==1.2.3c1

Looks good. Let's try it:

python
Python 2.6.2 (r262:71600, Apr 16 2009, 09:17:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg/_mysql.py:3: UserWarning: Module _mysql was already imported from /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.3-fat.egg/_mysql.pyc, but /Users/bhapca/Downloads/MySQL-python-1.2.3c1 is being added to sys.path

huh ?

>>> dir(MySQLdb)
['BINARY', 'Binary', 'Connect', 'Connection', 'DATE', 'DATETIME', 'DBAPISet', 'DataError', 'DatabaseError', 'Date', 'DateFromTicks', 'Error', 'FIELD_TYPE', 'IntegrityError', 'InterfaceError', 'InternalError', 'MySQLError', 'NULL', 'NUMBER', 'NotSupportedError', 'OperationalError', 'ProgrammingError', 'ROWID', 'STRING', 'TIME', 'TIMESTAMP', 'Time', 'TimeFromTicks', 'Timestamp', 'TimestampFromTicks', 'Warning', '__all__', '__author__', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__revision__', '__version__', '_mysql', 'apilevel', 'connect', 'connection', 'constants', 'debug', 'escape', 'escape_dict', 'escape_sequence', 'escape_string', 'get_client_info', 'paramstyle', 'release', 'result', 'server_end', 'server_init', 'string_literal', 'test_DBAPISet_set_equality', 'test_DBAPISet_set_equality_membership', 'test_DBAPISet_set_inequality', 'test_DBAPISet_set_inequality_membership', 'thread_safe', 'threadsafety', 'times', 'version_info']
>>>

Seems OK I guess.

Bottom line. The steps above seem to result in 2 different versions of Python coexisting side by side, one that came from Apple in

/System/Library/Frameworks/Python.framework/Versions/2.5/bin/python

and the one I installed in

/Library/Frameworks/Python.framework/Versions/2.6/bin/python

cron jobs terminate prematurely on Godaddy accounts

I've been using a large tarball file to upload my online store updates to godaddy and a cron job to unpack the tarball.

As the size of that tarball increased, due to adding more products, I noticed that the tar process would terminate prematurely, causing big issues, with the store left half-updated, in an inconsistent state.

I've been suspecting the folks at godaddy running some sort of watchdog timer and killing off processes that take longer to run.

Well, according to some reports, that indeed seems to be the case.

One option to go around this would be to break the large tarball in several smaller ones, that won't keep tar humming for too long. Also I should perhaps consider sending only actual delta changes instead overwriting the whole store in one go.

connection pools time-outs in GlassFish

I've been using JDBC connection pools and JMS connection factories for a recent project and upon deployment on GlassFish I did a sanity test to confirm that everything is OK. It passed the test.

So off I go, leaving it running on an Amazon EC2 instance. When I check on it about 3 days later I notice that it doesn't work anymore ... "server error", the UI informs me (which is a bad error message for the end users, whom might not know what a server (those working in restaurants) has to do with it). Anyway, by looking at the logs I see a few of these:

Exception: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

and

Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after statement closed.

Now the way I implemented this was in the "init" method of the servlet I would create my instances of javax.jms.Connection and java.sql.Connection as member fields of the servlet, hoping to reuse them for the lifetime of the servlet.

The errors in the logs seem to indicate that those connections get yanked out while my servlet is running, albeit idle. Not cool.

Looking at the admin console I see an "Idle Timeout:" configuration setting which defaults to 300 sec for both my JDBC connection pool and JMS connection factory. I changed that to 300000000. I have the feeling that this is going to fix the issue, but I need to run it for a while to confirm.

I'm not sure if this is the right thing of approaching this, so if you know any better please add a comment.

UPDATE:
It seems that changing the configuration settings for GlassFish is not enough to fix the problem.

Likely the wait_timeout configuration setting of MySQL has something to do with this too. By default mysqld will close idle connections after 8 hours, so we'd need to increase that to up to the max value of 31536000. One way to do that is through a configuration file (e.g. /etc/my.cnf, where we'd add a line with 'wait_timeout= 31536000' under the [mysqld] section.

Yet a better approach would be to catch the exceptions mentioned above in the servlet code and recreate the connection to the DB.

ANOTHER UPDATE:
It seems that catching the exception and re-opening the connection in the servlet code doesn't work very well. I get the following exception when trying to re-establish the connection:

Exception:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

Last packet sent to the server was 0 ms ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
...

Other people have experienced this too. One workaround indicated there, like I mentioned above, is to tinker with wait_timeout configuration of MySQL.

Another avenue to explore would be the "Connection Validation" capabilities of the JDBC connection pool. That's easily available through GlassFish admin console. According to the docs "Optionally, the application server can validate connections before they are passed to applications. This validation allows the application server to automatically reestablish database connections if the database becomes unavailable due to network failure or database server crash. Validation of connections incurs additional overhead and slightly reduces performance."

It'd be interested to see if the "table" validation method fixes this issue.

java hangs after installing "Java for Mac OS X 10.5 Update 4"

I had been experiencing some issues with Java on my MacBook after an update was pushed by Apple.

Had to jump through some hoops, but I came up with a workaround. For more details see this post.

How to download and install Nokia .wgz widgets OTA

According to Nokia we should be using an "AddType x-nokia-widget .wgz" directive to change the settings of web server to add a new mapping from the filename to a new MIME type.

Tough some people don't have access to server settings, let alone that Nokia's instructions don't include the complete MIME type (application/x-nokia-widget) so I'm not sure that what they suggested would work well.

Another way is to include the appropriate HTTP response headers through a script running on the server side.

This one, writteh in PHP, seems to be doing the job just fine. Pointing the S60 browser to its URL downloads the widget and starts its installation.

$filename = 'example.wgz';
 
header('Content-type: application/x-nokia-widget');
header('Content-Disposition: attachment; filename="'.$filename.'"');
readfile($filename);

java.rmi.server.UID is not random

Well, here's an unfortunate result of having different ways of doing the same thing in Java.

I needed a way to generate a random unique ID to be used as an authentication token. So I thought hmmm a sort of UID ought to do it. So the first candidate that Google suggested was

java.rmi.server.UID

Running a quick test gave me the following results in 2 subsequent runs: '-31a4aa0:121f7402438:-7f1f' and '-31a4aa0:121f7402438:-7f1e'. Epic fail. So much for randomness.

Looking for alternatives,

java.util.UUID

seems to be a better choice. In particular its

randomUUID()

factory method seems to claim cryptographic strength.

How to run GlassFish on the default HTTP port 80 on Solaris

If GlassFish is run as non-root process, with reduced privileges, as recommended, it won't be able to bind to ports < 1024. You'll get an exception:

java.net.BindException: Permission denied: 80 at com.sun.enterprise.web.connector.grizzly.SelectorThread.initEndpoint

To get around this you could set up a domain that has the ability to bind to standard Unix ports, including the port 80. For this you use the --serviceproperties net_privaddr flag of asadmin create-service command.

You can use the following steps. Replace the italic portion with your specific settings.

/opt/gf21/glassfish/bin# ./asadmin create-domain --adminport 4848 --instanceport 80 --profile cluster --user admin --domaindir /opt/gf21/glassfish/domains ptmp

Please enter the admin password>

Please enter the admin password again>

Please enter the master password [Enter to accept the default]:>

Please enter the master password again [Enter to accept the default]:>

Using port 4848 for Admin.

Using port 80 for HTTP Instance.

Using default port 7676 for JMS.

Using default port 3700 for IIOP.

Using default port 8181 for HTTP_SSL.

Using default port 3820 for IIOP_SSL.

Using default port 3920 for IIOP_MUTUALAUTH.

Using default port 8686 for JMX_ADMIN.

On Unix platform, port numbers below 1024 may require special privileges.

Domain being created with profile:cluster, as specified on command line or environment.

Security Store uses: JKS

Domain ptmp created.

You need to create a file with the admin credentials:

cat > /opt/gf21/glassfish/domains/ptmp/.aspass

AS_ADMIN_USER=admin

AS_ADMIN_PASSWORD=something

AS_ADMIN_MASTERPASSWORD=something

^D

/opt/gf21/glassfish/bin# ./asadmin create-service --passwordfile /opt/gf21/glassfish/domains/ptmp/.aspass --type das --serviceproperties net_privaddr /opt/gf21/glassfish/domains/ptmp

The Service was created successfully. Here are the details:

Name of the service:application/SUNWappserver/ptmp

Type of the service:Domain

Configuration location of the service:/opt/gf21/glassfish/domains

Manifest file location on the system:/var/svc/manifest/application/SUNWappserver/ptmp_opt_gf21_glassfish_domains/Domain-service-smf.xml.

The service could be enabled using svcadm command.

Command create-service executed successfully.

Ckeck that the newly created service is online. 

# svcs -a | grep ptmp

online         14:26:14 svc:/application/SUNWappserver/ptmp:default

If not make it so.

# svcadm enable svc:/application/SUNWappserver/ptmp:default

An extra sanity check:

# netstat -a | grep 80