How to read multiple lines from the standard input stream in Java

I was facing this problem, of reading lines of text from the console input, as I was under some intense time-pressure, while I was taking an online test. I was convinced that it must be something simple like:

String line = System.in.readln();

Not such luck. This seemingly simple task in Java ain't that simple. Googling for this revealed (in multiple places) something along these lines:

import java.io;

try {

BufferedReader in = new BufferedReader(new InputStreamReader(System.in));

return in.readLine();

} catch (IOException e) {

System.err.println(e);

}

which seems straightforward, until you try running this sequence repeatedly, perhaps in a loop.

If you feed this a multi-line input, pasted in the console, like so:

one
two
three

then you'd notice that the first gets read fine, then the subsequent reads block, without catching the lines:

two

three

Bad times. There's a fatal flaw in the code above, cited on so many sites, from one of which I copied it too. The readers hooked up to System.in don't get closed, and if that sequence of code is run in a loop we'd end up with multiple readers for the same single standard input stream. Now one could argue that an exception should be thrown in such case, where multiple readers are trying to slurp data from the same source. Alas it doesn't, and instead the behaviour is truly puzzling.

My faulty code had this function called repeatedly:

public static String readline() {

try {

BufferedReader in = new BufferedReader(new InputStreamReader(System.in));

return in.readLine();

} catch (IOException e) {

return null;

}

}

The fix consisted in doing this instead:

static BufferedReader in = new BufferedReader(new InputStreamReader(System.in));

public static String readline() {

try {

return in.readLine();

} catch (IOException e) {

return null;

}

}

The static variable gets initialized only once, and now calling readLine() repeatedly works fine.

Alternatively in Java 1.6 one could also use:

public static String readline() {

return System.console().readLine();

}

with the caveat that this approach using console() doesn't work by default when running in Eclipse.

How to deal with inconsistent display of characters with diacritics on the web

A friend asked me to build a web site, targeted to Romanian visitors. Romanian language has some characters with diacritics, as documented in Wikipedia:

Romanian uses a breve on the letter a (ă) to indicate the sound schwa /ə/, as well as a circumflex over the letters a (â) and i (î) for the sound /ɨ/. Romanian also writes a comma below the letters s (ș) and t (ț) to represent the sounds /ʃ/ and /t͡s/, respectively.

 So I thought I'd use a UTF-8 encoding for the HTML, together with some fonts that support those characters and this should be easy enough. To my surprise the results were pretty bad:

Here I have 5 paragraphs, showing the same text, each with a different font. First 2 are fonts downloaded on demand from Google ('Arvo' and 'Noticia Text'). The following 3 are stock fonts ('serif', 'sans-serif' and 'Verdana'). You can see that the stock fonts are failing badly, as marked with red. Those characters are either over or under-sized, relative to the others (they should have the same x-hight). The first 2 fonts aren't much better, since they fail in other contexts, depending on the size of the font chosen. So this yields the first insight - changing the font size may smooth out such issues. To check this I focus on the paragraph in the blue box and I tinker with the 'font-size' CSS selector. Seems to work:

The 4th paragraph looks better now. Applying the same trick in another context fails miserably:

That's using the 'serif' font, and on Mozilla it would fail regardless of the chosen size (at least on Windows XP). Incidentally, the same combination works perfectly on Safari on OS X, at any size:

Going back to Mozilla, if I change the 'font-family' to 'Arvo' it works fine (while that same combination would break in Safari):

To make matter worse, the initial set-up that would fail on Mozilla on Windows XP works fine on Mozilla on OS X:

Yet the same change in font-family that made it work on Mozilla on Windows XP brakes it on Mozilla on OS X (you can't make this up):

To get this right you need to test different combinations of 'font-family', 'font-size', browser and OS. I guess the biggest surprise for me was that for the 2nd paragraph, even if I used 'Noticia Text', included though @font-face from Google web fonts, which contains all glyphs with diacritics that I was looking for, it still failed to display properly on Windows. Doesn't that defeat the whole purpose of @font-face ?

So there you have it, another type of cross-browser inconsistencies that we must be aware of, together with the usual workarounds involving serving browser-specific CSS, consolidated in included files that can be applied to the whole site.

Spare your users the high cost of data roaming

You may have heard reports from the media, regarding the high cost of roaming charges that some people incurred. It's really no surprise, as some network carriers are practicing predatory pricing every time you leave the borders of their home country.

Although the end users are ultimately responsible for those charges, we, as developers, should strive to warn them when they're about to do some data transfers while using the apps we developed, if they're likely to be roaming at the time.  That would help people that haven't turned off data roaming, through the settings provided by iOS.

The problem is that AFAIK there's no API exposed by iOS to determine if the user is roaming. So we'll need to get creative. First off, we have the SystemConfiguration.framework, which allows us to determine whether a Wi-Fi or cellular connection is in use. If it's the former, we have a non-issue. There won't be any roaming charges for Wi-Fi connections. If it's the latter we need to dig deeper. We could use the CoreLocation.framework to determine the current location of the device.

Then we could use NSLocale and its ability to determine the county code of the current locale (likely established when the device is first set up).

If the country of the locale differs from the country of the current location, there's a high probability that the user is roaming, so we could warn them about potential charges before any data transfer is initiated by the app.

This heuristic is not fool-proof, but it's better than nothing, and it can go a long way towards preventing some unwanted charges. An alternative is to track changes in the country reported by the core location framework and use that as a hint of possible roaming. A possible complication may be caused by the user not authorizing the app for location lookups. There's a way around that, which I'll explore in a future blog post.

The best approach would be to have an API to tell precisely when a user is roaming, and better yet, the data rates for that particular destination, but that will be the topic for another blog post.

Notify your users about updates to your iOS apps

Most apps undergo updates, to fix problems, or to add new features. It's a fact of life. As is the fact that most updates go unnoticed by the end user. That's because by default Apple simply shows a badge on the icon of the "App Store" app, like so:

If you have a lot of apps installed, which are likely to produce lots of updates, I think most people would develop selective blindness towards those badge notifications, and would postpone the updates.

Another way to notify the users about an update available for a given app is to send a more vocal push notification (text alert, maybe with sound), assuming that the user agreed to receive such notifications. That may however be perceived as too pushy (or desperate), and that may be why people developed a more elegant way of notifying about updates, in a more appropriate context, which increases the chance of people acting on them. That's by simply showing the notifications when people are launching the old app, like so:

That makes sense, because now the end users are in a context where they demonstrate engagement with the app (they just started it) and the notifications have less chance to be perceived as off-putting. So this is a design idiom that makes sense to be followed.

In some instances the notifications must go further in providing more details about the upcoming update, and possibly assist the user with the upgrade. One such scenario is, for example, if the user moves an app that was initially a stand-alone app, with its own icon on the home screen, to the Newsstand. In that case the icon on the home screen would disappear and a cover corresponding to that app would appear in the Newsstand, like so:

That move has the potential to confuse people, and a notice shown by the old app before the upgrade could clarify the new location.

Another move scenario is moving an app to another developers' account. This tends to happen when a bespoke app is first developed as an experiment, with few expectations, and it's published under the developer's account. If the app is a success, the contracting clients often want to have the app moved to their own account. This is challenging because from Apple's perspective the app in the new account becomes a brand new app, completely independent from the old one. The upgrade prompt should set clear expectations about the old app being discontinued, with all future development and support slated to go in the new one.

If the app has in-app purchasing content, care must be taken to migrate purchases made in the old app to the new one. The usual purchase restoration mechanism provided by Apple is of no use here, because Apple has no notion about the 2 apps being related. So you'll need to roll out your own purchase migration mechanism. For example, when the old app is launched, you could make a snapshot of the past purchases, upload them to a server, and produce a redeemable code that could be used in the new app to restore the purchases. If you already have another way of identifying the users, perhaps through some user account/login scheme, you can use that instead to assist the migration. In iOS 5 and later this is facilitated by the newly introduced Account framework, that allows tying separate apps to the same users through their credentials on social networks like Twitter.

Another consideration would be the transfer of the name of the app. Since there can't be 2 apps with the same name, you'll need to allocate a temporary new name, then delete or rename the old app, to free up the old name, and later apply it to the new app.

How to install MySQLdb on Mac OS X Lion

After Apple rendered all my development tools (MacBook, iPad and iPod touch) obsolete and incompatible with iOS 6, I purchased a new MacBook Pro, running Lion. So I tried installing the software I've been using on the old MacBook to the new one.

One of that software is MySQLdb. Yet I didn't remember how I installed it in the first place. So I had to start from scratch. I'm a proponent of the principle "if it ain't broken don't fix it". Since Lion comes with Python 2.7 pre-installed, I figured I'd use that.

Next missing piece was MySQL. I used a .dmg from  http://dev.mysql.com/downloads/mysql/, specifically mysql-5.1.63-osx10.6-x86_64.dmg. As the name indicates that's for a 64 bits architecture.

One important insight gathered from this site was that the architecture of Python, MySQL and the MySQLdb I've been trying to build should match. To check where I was standing I started the Python interpreter, made sure that MySQL server was running and used Activity Monitor to look at their processes like so:

So I clearly needed to build the 64 bits variant of MySQLdb. After downloading the sources from http://sourceforge.net/projects/mysql-python/ I followed the instructions from the README file with the following amendments:

1. Did a which mysql_config to see its location and updated the site.cfg to point there like so: mysql_config = /usr/local/mysql/bin/mysql_config

2. Instead python setup.py build did a ARCHFLAGS="-arch x86_64" python setup.py build.

3. Instead sudo python setup.py install I did sudo ARCHFLAGS="-arch x86_64" python setup.py install

To check the result I did import MySQLdb at the Python prompt and checked that no errors occur.

Note that following the step #2 you'll see a bunch of 'implicit conversion shortens 64-bit value into a 32-bit value' warnings. Those could probably be ignored, though there's a chance that some data loss may occur during those implicit conversions if the values on the right hand side of those assignments are large enough (probably unlikely). I tried a fix, to eliminate them, by using platform independent types like size_t instead of int and by matching the types of the variables in LHS return types of the functions in RHS of the assignments on the problematic lines. I placed a patch that attempts to fix those warnings at http://sourceforge.net/tracker/?func=detail&aid=3541063&group_id=22307&atid=374934.

Connecting external devices to iPhone

Recently I heard about some opportunities involving creating apps running on iPhone, meant to talk to specialized hardware devices, so that got me thinking about ways in which the iPhone can connect and talk to external devices.

First category is wired connections. Here the choices are simple, since there's only a single connection port, the docking port (also used to charge the device). In order to be able to communicate through that port to the external device you'd need to be enrolled in Apple's MFi program. However there's another option, that doesn't require participation in this program, and uses the audio jack available on iPhone. That jack outlet provides connectivity to both headsets and microphone, so that gives you access to an input/output channel. Your external device should be capable to generate the required voltage that applied on the microphone pin would be detected in the iOS app as a sound, and should be able to understand the audio output received on the headset pins, as generated by the app. This is nothing new, as some people have already implemented it as either dedicated solutions like Square, or more general-purpose solutions like HiJack. For low data rates you may be able to get by with a simple detection technique, whereas for higher data rates you may need to implement a full-blown modem.

The other category is wireless connections. Here your best bet may be using the network connectivity (either WiFi or 3G) to connect to a server over TCP/IP or higher protocols. That's probably the approach that involves the least effort. Another appealing avenue is to use Bluetooth, however the public SDK only allows you to talk to another iOS on Mac device, via Game Center, or Bonjour. If you wanted to talk to your specialized external device over bluetooth, you would also need to be enrolled in the MFi program, and use the External Accessory Framework to talk to the device.

Another wireless channel, albeit one-way, is to use the camera to interpret either a pattern like QR code or barcode, or a pulsating light. Another one-way channel in the opposite direction is the screen of the iPhone. You could have your app display a sequence of patterns that could be interpreted by an external devices. Some of you greybeards may remember a version of Windows transferring data to a Timex watch in that manner.

The accelerometer falls in a category of its own, since it's not quite wired, neither wireless. It's obviously an input channel into the app, whereby the app can detect movement, if the iPhone is physically attached to an external device. That could be used for a monitoring/tracking the movement of physical goods, either intentional or due to theft or mishandling.

That pretty much sums it up. If you can imagine other ways of communication between the iPhone and external devices, please comment.

Using gdb to debug Objective-C code

I've been using gdb for many years, initially for debugging code for embedded software, written in C/C++ and running on specialized hardware, and more recently for software meant to run on iOS devices, written in Objective-C. Knowing the way to use it for C/C++ served me well, as most of that stuff is applicable to Objective-C too, including the most commonly used commands. I was aware of, and had been using some commands specific to Objective-C, like print-object, but not much beyond that.

At some point I came across a presentation that informed me about the ability to evaluate and print the value of complex expressions, including method calls. Of course you have to be mindful of possible side-effects, but that's a pretty powerful capability to have in the middle of a debug session.

So recently I had to inspect the number of entries in a dictionary. Even though I could have used the Xcode's ability to hover over variable's name and "inspect" some of its content, which for NSDictionary works really well, in that it shows how many key/value pairs it contains (in my case 10), I thought I should use the command-line gdb to find that out. So I did:

(gdb) po [existentMetadataForPrefix count]

0xa does not appear to point to a valid object.

Hmm, not what I expected. The 0xa mentioned there in hex is 10 in decimal, so the value appears OK. It looks like po (shorthand for print-object) tries to send a message to the object pointed by 0x10. In fact that message is "description", so what I get is fair enough. Obviously I shouldn't be using print-object on an integer value (NSUInteger) returned by count.

To print primitive values in C/C++ one would use straight print. Doing so yields:

(gdb) p [existentMetadataForPrefix count]

Unable to call function "objc_msgSend" at 0x1b1e08c: no return type information available.

To call this function anyway, you can cast the return type explicitly (e.g. 'print (float) fabs (3.0)')

What the ... ? This doesn't make any sense. That 0x1b1e08c address is not the address of existentMetadataForPrefix, as shown by:

(gdb) p existentMetadataForPrefix

$2 = (NSMutableDictionary *) 0xf520c10

Then, what's at that address ? Can find out, like so:

(gdb) x 0x1b1e08c

0x1b1e08c : 0x08244c8b

Huh, looks like that might be the actual address of the function objc_msgSend. Can that be ? Let's double-check:

(gdb) x objc_msgSend

0x1b1e08c : 0x08244c8b

It is indeed, so what's going on here ? AFAIK that objc_msgSend function is called when a message like count is sent to an object like existentMetadataForPrefix.  Let's set a breakpoint:

(gdb) b objc_msgSend

Breakpoint 3 at 0x1b1e08c

And try again:

(gdb) p [existentMetadataForPrefix count]

Unable to call function "objc_msgSend" at 0x1b1e08c: no return type information available.

To call this function anyway, you can cast the return type explicitly (e.g. 'print (float) fabs (3.0)')

Nope, the breakpoint doesn't get hit. Ok, at this point it's pretty clear that I need to learn more about how to use gdb with Objective-C. In doing so I discovered this excellent blog post that explains that:

We can also print the result of messages which return primitive values. However, gdb is not smart enough to figure out the return type in this case, so we have to tell it by adding a cast to the expression:

So all I had to do was this:

(gdb) p (int)[existentMetadataForPrefix count]

$1 = 10

Which, in all fairness was in fact suggested by gdb, but was shrouded in confusion, at least for me, by the mention of objc_msgSend.

I hope this would help you in case you run into a similar problem, and I can definitely recommend checking out that blog post. It's full of very interesting gotchas on this topic.

Following through the examples in the "Agile Web Application Development with Yii 1.1 and PHP5" book

I just started reading the Agile Web Application Development with Yii 1.1 and PHP5 book. Chapter 3 explains the virtues of TDD (test-driven development) and tries to walk us through setting up a development environment that supports automated testing through PHPUnit and Selenium.

At the same time, right at the beginning of the chapter 2 there's this note:

"There are several versions of Yii from which to choose when downloading the framework. We will be using version 1.1.2 for the purposes of this book, which is the latest stable version as of the time of writing. Though most of the sample code should work with any 1.1.x version of Yii, there may be some subtle differences if you are using a different version. Please use 1.1.2 if you are following along with the examples."

This clearly makes sense - it's better to use the same version of the tools, as the author, if we're expecting to see the same results as shown in the book. The realty is that by the time some readers, like myself, get to read a book some of the software tools, and their dependencies, have evolved to the point where what they produce does not even remotely match what the version that the author used produced.

So I followed the advice, got Yii version 1.1.2, and I tried to follow along. It wasn't a smooth ride. I'll try to walk you through some of the pitfalls that I experienced, and the ways I overcame them, hopefully helping you if you're experiencing similar problems.

First problem appeared when I tried:

sudo pear install phpunit/PHPUnit 

shown on page 45

I got an error telling me that the version of pear I've been using was too old (had been using a MAMP setup common on Macs, which may have grown old, at version 1.*). Even though I don't have the exact message, based on my googling history it said something along these lines: "requires PEAR Installer (version >= 1.9.4)".

The solution for that was to upgrade pear, like so:

sudo pear upgrade pear

and do a
pear --version 
to check that the version went up.

However, if we follow what the book says, literally, when it tells us to install  phpunit/PHPUnit, like so:

sudo pear install phpunit/PHPUnit

we'll get the latest version of PHPUnit, which, of course, doesn't match the older version of the Yii framework, which we were advised to install. This opens a whole can of worms.

First off, when you try to run the functional test cases, by doing:
phpunit functional/SiteTest.php 
you'll be informed that:

PHP Warning:  require_once(PHPUnit/Extensions/SeleniumTestCase.php): failed to open stream: No such file or directory

If you check the location of PHPUnit install, which on my system is at /Applications/MAMP/bin/php5/lib/php/PHPUnit, you'll see that indeed that file is missing. How so ? Turns out that most recent versions of PHPUnit and Selenium now get that file installed by doing:

sudo pear install phpunit/phpunit_selenium

but don't do that, because you'd just be wasting your time ! The thing is, you don't want to use the most recent PHPUnit, because if you do, as you would be if you followed the instructions in the book, you'll next hit this error:

Warning: require_once(PHPUnit/Framework.php): failed to open stream: No such file or directory in .../framework/test/CTestCase.php on line 11

Yeah, Yii version 1.1.2 requires PHPUnit/Framework.php, which is not present in the most recent versions of the PHPUnit.

So, "yes it is" - a complete mess, that is. To save your sanity, uninstall the newer versions of the tools:

sudo pear uninstall phpunit/PHPUnit_selenium (if you installed it already)
sudo pear uninstall phpunit/PHPUnit

Then install the version that matches the one used in the book:

sudo pear install --alldeps phpunit/PHPUnit-3.3.17

This should bring you to clear sailing through chapter 3.

how to find the app version from the iOS crash reports

If you have an app that has a longer history, with several versions of the app having been released along its lifetime, you may find yourself getting a crash report from an end user and not knowing for sure to which potential version of the app it corresponds.

This is a pretty glaring oversight from Apple, because they don't include any information about the bundle version of the app in the crash reports they produce. You'll often see a "Version:         ??? (???)" string that's pretty useless in determining the actual version of the app that produced the crash.

Luckily other people did come up with a solution for this. I'm writing this mostly as a note for myself, after spending more than an hour trying to dig up this info. Hopefully this blog post will help funnel other searches towards the solution.

The key to solving the problem is to use the dwarfdump command line tool to determine the UDID of the app, for different versions of the app that you published, and look up those UDID in the crash report, until you find a match.

So for each version of the app you have in your "Archives" view of the Xcode organizer,

you'd try to find the actual binary of the app. You may need to use the "Show Package Contents" option of the Finder several times to dig down through archives, towards the binary file.

Then you'd drag that binary over a Terminal window, to complete the call to dwarfdump, and you'd make a note of the UDIDs that are output by the tool:

Then you'd simply look up those UDIDs in the crash report, while ignoring the dashes in the format of UDIDs that are produced by dwarfdump but are not present in the crash report:

 If you get a match, like I do here for the armv7 architecture, it means that the version of the app you have selected in the Organizer corresponds to the crash report being analyzed.

User-friendly push notifications for iOS

iOS apps allow notifying the end users about things of interest. Notifications may include alerts, sounds and icon badges. They are "pushed" to users' devices automatically, without users' intervention. This ability has been present in the platform for a while now, and many apps are using it. However, in my opinion, most of them are doing it wrong.

The way they're doing it is that as soon as you launch an app that you just installed, you'll be greeted with something like this:

There are 2 issues with this prompt. First is the timing. I just installed this app and I'm now trying it for the first time. Unless I'm familiar with the brand, at this point I don't really know enough about this app, to trust it to allow it to bug me with future notifications. Is this really the best time to ask me for this ? Probably not. I always reject it. And there goes your chance. Once the end user rejected the prompt for allowing push notifications you don't get a second chance. That is, the app can't programatically make this prompt appear again. The only way to revert this initial rejection is if the end user goes to Settings and manually overrides it for this app, under the "Notifications" section. Or if the user uninstalls the app and keeps it uninstalled for at least one day.

Perhaps a better way would be to ask the end user to allow notifications only after she used the app for a while, to the point where she would have built up the trust for the app. Asking for such notifications is done programatically, via calls to the

- (void)registerForRemoteNotificationTypes:(UIRemoteNotificationType)types

method of UIApplication, so it's entirely within the control of the app. We could track users' engagement with the app, perhaps by keeping a record of how many times the app was launched or how much time has been spent in the app. Then we can choose to show the prompt only after the user has shown a sufficient level of interest in the app. That would increase the chance of the prompt being accepted.

The other issue with the prompt could be asking for permission for some potentially intrusive notifications, as it's the case in this example. Remember, there are 3 possible types: badges, alerts and sounds. While for most apps, a badge update would make sense, would alerts and sounds be appropriate ? For a magazine app, like in this example, that's hardly the case. When you do the interaction design for the app you'll need to ponder what types of notifications it makes sense to send, and request permission only for those. That's controlled through the parameter passed to the registerForRemoteNotificationTypes method.

Apple could also improve the visual representation of these prompts, perhaps by including some icons or color coding that better depicts the types of notifications the end user is asked to accept.

Choosing a name for your iOS app

As you can imagine, you can't have 2 apps with exactly the same name in the App Store. So grabbing an app name is like grabbing a good domain name. First come first serve. For domain names you probably know that some people registered desirable domains, with no intention to actually use them, only to resell them to the highest bidder. Apple learned something from that, and they try to prevent namejacking. So once you create an app in iTunes Connect, occupying a name, you have a 120 days grace period to submit your app for review. If you fail to do so, the app will be automatically removed from the system and you'll lose the right to use that name ever again under your account.

When that happens you'll get an email from Apple saying that:

You did not upload a binary for your app, ***, during the 120-day grace period. As a result, the app has been deleted from iTunes Connect.
This app cannot be restored, nor can you use the App Name or SKU for any other app under your account in the future.

The same happens when you explicitly delete an app from your account, either before your grace period expires, or if the app is already live.

Deleting it will permanently remove it from iTunes Connect, along with any associated In-App Purchases. The App Name and SKU will not be reusable, even once the app is deleted. 

There are a a few ways to mitigate the risk of losing your app name:

1. Once you grabbed a name, submit the app, any app, perhaps with limited functionality, or even unrelated functionality. Something that Apple would accept in their store. Example apps would be a native wrapper for a web site, PDF publication or a simple app perhaps built off some sample code available from either Apple or 3rd parties. For a competent developer 120 days should be enough to build a version 1.0 of most apps.

2. If you're approaching the end of the grace period, you could try to open a new account, maybe under a different name, of a relative or associate, and move the name to the new account to get a new extension. You'd have to relinquish it from your current account, before it can be registered to the new one.

3. Register the domain matching the app name. That may deter people form trying to use it.

4. Register the name as a trademark. That may give you a legal recourse in claiming ownership with Apple or infringers.

If it comes to worse, and you lose your name, you can still publish your app using variations of that name.

How to deal with "Cannot connect to iTunes Store" errors

Even after I got plenty of experience building apps with in-app purchase capabilities, having gone through the lessons learned at my first attempt, today I was still hit by this error. My purchase attempt on the sandbox would still fail with this laconic message. This is ridiculous. Apple still doesn't provide enough information, or even a small hint of what could be wrong.

Like I mentioned in my post referred to above, you should first check your provisioning profile, to make sure it's bound to a fully qualified app id, not one with a wildcard. In my case it wasn't, even if I should have known better. D'uh ! That's easy to fix, right ? In Xcode I just go in the build settings for the project and set the code signing identity for Debug builds. Any sane person would expect that to be enough. Not so, the build settings for the actual target is still stuck at the old, incorrect, value. That's arguably a bug in Xcode. Why would one have the same setting available in 2 different places is beyond me. Anyway, I had to fix the setting for the target to match that for the project. To my frustration the same "Cannot connect to iTunes Store" failure kept popping.

At that point the only thing I could think of is that Apple's sandbox lost its marbles. So I created a new test account. Tada ! It worked. This is insane. Having to deal with such issues, with deadlines looming, it's the last thing one could wish for. Unfortunately it's not the first time it happens. I had other issues in the past, so ridiculous as having tests accounts used on the sandbox locked, after repeatedly being denied login. After filing a bug report for that lock-out Apple came back with some sort of explanation citing recent changes/"fixes" to the sandbox and prompting me to try again.

I hope my experience could be useful to you if you run into some similar issues. Do you know any other relevant tricks ? If so please comment.

Adaptive volume level for audio on mobile communication devices

Mobile devices are meant to be used in various environments, as such I find it pretty surprising that the designers of user experience for these devices fail to address one obvious fact: the ambient noise can vary dramatically from one environment to the other. To be fair, each device has a volume control, meant to adjust the level of the audio volume above that of the background noise, however in some cases unexpected bursts of noise may occur in an otherwise quieter environment. It'd be great if the volume would adjust automatically to make up for the increased noise, and later subside when the noise is gone.

The alternative to automatic adjustment is to do it manually, which is the current state of the art. While that's merely a nuisance for regular audio output, like when listening to a podcast, or while in a phone call, it could have more severe effects when applied to the volume of the ring tone. I missed many calls on my mobile phone while I was out on the street, with heavy traffic around me. Even though I have my phone set to both ring and vibrate at the same time, I would still miss calls.

In thinking about solutions to this problem, the obvious approach is to use the microphone, if available on the device, to listen to the environment and react appropriately on increased noise. The drawback of doing such a continuous listening is that it may consume CPU and battery power, and that's probably why it hasn't been considered. Perhaps a better approach would be to let the user decide when that's worth doing. After all, this listening should only happen when audio is playing, so it may not be such a drag on battery.

One could apply further optimizations, by employing location information correlated with past data collected on the same location as a hint for quieter places, or data from the motion sensors as a hint that the user is on the move, hence more prone to the vagaries of the environment.

I remember how my first mobile device, a NEC pager, solved this beautifully. Upon receiving a message, it would first start blinking an LED. If the message was ignored it would start beeping ever so slightly, than it would gradually increase the volume to the point where it was so loud that it was almost painful to hear.

In any case, this should be handled at the level of the platform, since it's a concern that cuts across several apps that produce audio output. I'm looking forward for the first platform that will handle this effectively. My previous experience on Symbian/S60 and my current experience on iOS uncovered this as still lacking. What do you think, have you missed this capability too ?

How to deal with "A network timeout error occurred. Please try again later." in iTunes Connect

I first encountered this error about a week ago, while trying to set up a new purchase item for an existent app. Given the recommendation to try again later, I assumed it's a transient error and I ignored it.

About a week later I tried again, with the same result:

Hmm, now this doesn't look any more like a transient error, unless a really unlikely coincidence it at play here. In any case I alerted Apple. Then I figured I should also try with a different browser, and sure enough it works fine with Safari.

So if you're getting this too, try switching to Safari and you may be able to work around it.

Hidden, recurrent costs of developing iOS apps

So you're thinking of building an iOS app for that great idea you have. Awesome ! But if you're not an expert in iOS development you'll need to hire one to build it for you, or maybe you're thinking of using one of those cookie-cutting app makers, that promise "no coding required".

If you want something to stand out from the competition, or if you have some special requirements, your only choice is custom development, with "plenty of coding required". Assuming you're doing it the right way, which will be the subject of a future post, you'll soon figure out that this is going to set you back a certain amount of coin. Just be aware that the sum you're contemplating at that point, based on the quote you got from the developer, is most likely only going to cover the cost of development. Is that all there is ? Well, not really, you'll also need to consider some recurrent, or revolving, costs.

Why recurrent costs ? Well, for one, if you have the app published under your own account, which is probably going to be the case if you're building a custom app, there's the $99 annual membership fee for Apple's Developer Program. Beyond that you'll need to think about the maintenance of the app. What ? Maintenance, like car or building maintenance ? Exactly.

You see, what happens is, these apps are not built in a vacuum, from scratch. Instead they're built on top of an existent platform, the iOS system, which keeps evolving. No doubt you're familiar with the fact that Apple periodically issues updates to the iOS system, indicated by different versions, like iOS 2.1, 3.1, 3.2, 4.2, 4.3, and so on. All these updates add new functionality, and, here's the rub, sometimes change existent functionality. That's right, stuff that used to work in older and current versions of iOS will stop working in future versions.

Why's that ? Two reasons. The first one is controlled. It's called deprecation. As they go along, folks at Apple figure out better ways to accomplish the same thing, or in some cases, the old ways of doing things stands in the way of progress, or conflicts with new planned functionality. In those cases they flag some functionality as deprecated, and provide some alternatives. You're given a period of time, usually unspecified, between several releases, to migrate to the new functionality. That means that for the time being the old functionality still coexists with the new one, but at some point in the future, solely at Apple's discretion, it will go away, and if you didn't migrate, you'll be left hanging.

The other reason is uncontrolled. It's because of bugs and defects that are the usual unintended consequences of changing software code. Every time Apple releases a new version of iOS, that new version has new functionality, fixes for known problems, and new, not yet known, problems. That's the nature of the business.

The biggest changes to the platform, and the most potential for bugs, are when the major version changes. The major version is indicated by the first number in the versioning scheme. For example in version 4.3.5, 4 denotes the major version. When iOS 5.*.* comes about, you'd better watch out.

I've been building iOS apps for a living for more than a year now. Based on that experience I can tell you that if your app has a fair amount of functionality, touching different areas of the platform, chances are high that it will break on the next major release. I had problems both when iOS 4 was released and when iOS 5 got released.

You may ask, how come, since iOS 5 hasn't been released at the time of this post. That leads to the solution. You see, Apple issues beta releases available for developers prior to the official releases. So all you need to do is have your developer proactively test the app on those betas, as they come along, and look for issues. This gives it the "hidden" nature of the cost. You can't know in advance how many problems you'll encounter. So you'll need to budget for that. On my latest project, which took about a year to develop, I had to spend a week to fix problems in iOS 5 beta.

How can one fix those problems ? In some instances they are problems in Apple's code. In those instances the developer must file a bug report with Apple, to let them know about the problem. The developer should next look for possible workarounds, because even if a bug report has been filed, there's no guarantee that Apple will fix it in time for the next release. You can gauge the likelihood of that to happen, by monitoring the incoming betas. The higher the number of the current beta, the closer it's to the GM (gold master) release, after which all bets are off.

In other instances, changes in the platform may rattle some dormant bugs already present in the app. Those bugs are the developer's responsibility. The good thing is that with that responsibility comes power and control too. That is, a competent developer should be able to fix them. That's opposed to the previous scenario, where you're entirely at Apple's mercy, to fix their mess, unless you have a developer that can come up with a workaround. If you work with a less gifted developer, your option is to find a second opinion, or, if the problem hasn't been fixed in time before the release that introduced the breakage, enlist the help from Apple DTS (Developer Technical Support). You get 2 free support incidents per year with your membership in their developer program, and you can purchase more at a low cost. I never had to use that myself, so I couldn't comment more on that.

So there you have it, a rundown of the issues that may crop up post-release. On all my projects I offer my clients a maintenance plan that charges a monthly fee to test the app on the latest beta and to handle any issues encountered. So they can have the piece of mind knowing that their initial investment is protected.

How to publish your iOS app in the app store if you outsourced its development

Let's say you're a business person with a great idea for an app, yet you have no interest or inclination in doing software development, hence you hand it over to an outsourcing partner. Assuming it all goes well, in due time you'll have a working app that's ready to be published to the world. The development team produces the source code for the app, and possibly some installation package (IPA) that allows you to test it on your device. All looks good and you decide to have the app published under your own account, assuming you enrolled in Apple's iOS Developer Program.

First off, why would you want to use your own account ? It allows you to promote your brand, by showing you as the "Seller" of the app in the app store. It also allows you to directly collect any proceeds from selling the app, or any in-app purchased content, if the app or its content it's not free. In most cases these are strong enough reasons to shell out the required annual membership fee (which at the moment stands at the equivalent of $99 USD). If these don't matter to you, you're better off using the account of your outsourcing partner for publishing.

Assuming you applied and got your membership in Apple's iOS Developer Program you'll have to assist your developer in publishing the app under your own account. Here you can work under 2 possible scenarios:

1. New account

If this is a brand new account, that is, you don't have any existent apps live in the App Store, the best course of action is to hand over to your (trusted) developer the credentials for logging into your account and let them do their thing. This may be stepping beyond the terms of service imposed by Apple, but it's the hassle-free option. You'd cede the access only temporarily, until your app is set up, then you can change the password of the account to take back control.

The alternative workflow, endorsed by Apple, is to add the developer to your account, in a "Developer" role, assuming you have a company account, that is, not an individual account. Company accounts allow having different members under the same account, with different roles. The workflow envisioned by Apple may work well for companies with in-house development expertise, but I found it quite lacking in cases where companies outsource the development as a whole, and have no such expertise (like most of my clients). The reason is that only the person in the "Team Agent" role can submit apps and app updates to Apple for review and inclusion in the App Store. The other members of the account can contribute towards the development of the app, by writing source code, testing, etc, but there's only one person that acts as a gatekeeper for the submission process. By default the Team Agent is the person who applied for the account, who, as a representative of the contracting party, in most cases has no interest or knowledge of the technicalities of building an app. That's why I'd recommend to give your outsourcing partner access to that role.

2. Pre-existent account

This is the case where you have already an account, that was used to publish some other apps, possibly developed by other developers. In that case, besides giving them access to the Team Agent role, you should do one more thing. You see, all apps get digitally signed before they are submitted to Apple for inclusion in the App Store. This signing takes place automatically when the app is being built by the Team Agent and relies on a unique development certificate that must be set up by the Team Agent. If you have ceded that role to a developer, who had that set up for your account, and now you're switching to a new developer, the new developer must have access to that certificate too. When that certificate is set up it produces 2 parts. One is a public part, which is the certificate itself, which encloses a public key. This public part gets stored in Apple's development portal and can be accessed by anyone who has access to the account. The private part gets stored locally on the computer of the person who set up the certificate. If a new person is to use the same certificate for signing purposes she must also get access to that private key. For that you'd need to ask your previous developer to export the private key, as shown in the figure below, and pass it to the new developer.

It's best to ask your developer to give you a copy of the private key, as soon as the certificate gets created, even before you embark on new development projects.

If for some reason you can't receive the key from the previous developer, not all is lost, the new developer in Team Agent role has the option to revoke the old, existent certificate and create a brand new one, with a private key that would be in her control. Although it sounds like a pretty drastic measure, it's really quite benign, as described in Apple's official docs. Your new developer may need to do this anyway if the key passed from the previous developer fails to import properly. I can attest for the effectiveness of the method, as I performed it several times for different clients.

Reverse-engineering iOS apps through console output and logging

One useful practice in software development is the use of logging, to record details about the status of the app, or other events of interest at runtime. This can be a life saver when problems occur and they need debugging, either live or post-mortem. The logs are also useful during the development of the app, as a way to quickly asses its sanity, at a glance, as opposed to placing breakpoints and using the debugger to inspect the value of variables of interest. In some instances, where timing issues could affect the behaviour of the app, stopping the execution with breakpoints is infeasible, possibly resulting in heisenbugs. In such instances logging can be a better alternative. So you'll find logging used heavily on all types of apps.

In iOS we have this function, part of Foundation framework:


NSLog

Logs an error message to the Apple System Log facility.

void NSLog (
   NSString *format,
   ...
);

Its use is encouraged in many introductory books and tutorials for iOS programming. By default this function outputs content to the device console available through the Organizer view of Xcode. Now, while this is all good for debugging apps that we're developing, I was very surprised to discover output produced by 3rd parties apps, that were installed straight from App Store, in this case "Globe News". I obtained this output by simply running the app on my iPad set up for development, connected to my laptop running Xcode:

In many cases such output may contain sensitive information, like details of various web services endpoints used by the app, or other information that could be used for reverse engineering. This may well be a bug in this particular combination of Xcode/iOS versions, but as developers we need a way to guard against this.

Another concern is the overhead incurred by having our code peppered with such NSLog statements. Each time such a statement is executed, depending on how efficient its internal implementation is, its input parameters may be processed and a string may get formatted, even if there isn't an output sink available. And on mobile devices, like those powered by iOS, where every wasted CPU cycle translates in battery drain, this is of particular concern.

A popular way to control logging is to enable it only in debug builds, by wrapping the calls to logging functions in some macros. For iOS we could use this setup:

#ifdef DEBUG

#    define DLog(...) NSLog(__VA_ARGS__)

#else

#    define DLog(...) /* */

#endif

#define ALog(...) NSLog(__VA_ARGS__)

This snippet can be placed in the _Prefix.pch file, that makes the macro available in all source files. Using this macro enables logging only on debug builds, hence getting rid of all problems mentioned above. Your released app, available in the App Store wouldn't even have any logging compiled in, hence it would be better protected against reverse engineering.

Obvious shortcomings of Apple's App Store

Having published more than 40 apps in the App Store, on behalf of my clients, I'm in a good position to comment on possible ways to improve it.

Perhaps the most glaring omission is the inability to quickly revert to a previous version of an app, once a newer update has been approved and went live. Sometimes mistakes occur, and those mistakes could easily go live.  Based on my experience, once an account has a fair number of apps published (like over 30), and an unblemished history, with few app rejections, it seems that Apple's review process gets noticeable lax (as a better word for sloppy). In one instance an app update was glaringly broken on retina display devices (i.e. iPhone 4), yet it was approved and went live.

In such instances your only recourse is to either pull the app from the store, and/or to prepare and issue a new update, but that has to go back to the waiting queue. This could have terrible business consequences, because there wouldn't be a working app available for about a week. You could try to speed it up, by requesting an expedite review. Although the onus is on the developer to test each update before publishing it, it'd be very helpful to be able to revert to a previous, functioning, version.

A related idea would be to have some sort of a staging area, whereby a new app, or an update of an existent app, could be made available through an obscure URL to a set of testers, after it has been reviewed, but before being made available to the public at large. This may be already supported through the option of manually releasing the app after the review completes, though the fact that there's a single URL that points to the app in the store, irrespective of its version, may suggest that the URL points to the latest live version. This staging area mechanism would be similar to the Ad-Hoc distribution, only more scalable.

Another thing missing is an API to access the App Store functionality programatically. Some apps lend themselves to a cookie cutting model, whereby the same template is used to produce multiple apps, perhaps being differentiated by the content they package and some branding. In such instances the ability to programatically interact with the store would be very useful.

Crash reports fail to symbolicate using xcode 4.0.2

I got some crash reports recently, for some work in progress iOS app that I distributed to my clients via an IPA. To my surprise the reports fail to symbolicate, hence are pretty useless in hunting down the root problem.

I never had this problem before, and its occurrence seemed to coincide with my recent move to using the Xcode 4.0.*. Doing some Googling uncovered a post with a bit of background regarding this issue. It turns out that the newer versions of Xcode changed the way IPAs are built, and in the process broke the symbolication. D'oh !

Now, as a rule, whenever I upgrade stuff I try keeping a copy of the old version, as a precaution. Luckily I have an Xcode 3.2.5 still installed, as a fallback mechanism. All I need is to rebuild my IPAs using the older version of Xcode, until Apple fixes their tools. The post quoted above suggests fixing the Perl script that causes the problem. I'm not sure about that. It may work for a particular version of Xcode, as a last resort, if you really need Xcode 4.0.*. Falling back to Xcode 3.2.* seems a simpler, safer alternative, at least for my needs.

Google effect in action

I've been running a web site that acts as a wrapper for Amazon's aStore. This was started as an experiment a few years ago, yet it didn't accomplish much, beyond a learning experience in how to use PHP to perform some on-the-fly screen scrapping and content embellishing. It was also something I could use when experimenting with PPC advertising. Having been pulled in some lucrative projects I didn't pay any more attention to this. In fact functionality that used to work, like search engine friendly URLs (SEFU), is now broken...

Anyway, glancing over my analytics account today I see a smashing 338% traffic increase during the last month:

Wow ! All this traffic coming from Google. To what do I owe the honour ? No inbound links, none of the usual favourable auspices. The only logical explanation is Amazon dropping their affiliates form California, about at the same time as my jump in traffic, which likely caused the poor folks to pull the plug on their sites. Or perhaps a glitch in Google's secret sauce for bestowing wealth in cyberspace ? Anyway, the people are coming in, goals as measured by analytics are fulfilled, meaning people are adding stuff to their carts, good times rolling.

Of course the earnings exploded :)

One thing that strikes me is the conversion rate, which at 7.69% stands much higher than industry average (which I believe stands at less than 2%). So yeah, this passive income thingy is not entirely hype. If you can bring people over, the money will follow.