California Dreams

My most recent Caltroid update added map view with two location listeners: one coarse (=network) and one fine (=GPS) location listener. Or so I thought. Even though I was calling LocationManager.removeUpdates() for both location listeners, I could still see the GPS icon active at the top, and logcat showed GPS activity still going on. Users were reporting drained batteries. Not good.

It took me a few false starts until it occurred to me that I was also using MyLocationOverlay to automatically put the user’s location on the map. Well, this obviously requires location updates. I had somehow falsely assumed that as soon as the map view went away, MyLocationOverlay would stop listening to location updates. That is not the case. You must explicitly call MyLocationOverlay.enableMyLocation() to start updates, and MyLocationOverlay.disableMyLocation() to stop updates.

After figuring this out the fix was easy. In my map view’s onResume() I request location updates for my two location listeners and enable MyLocationOverlay, and in onPause() I do the opposite. No more drained batteries!

My latest update to Caltroid added two notable new features: the next available train is highlighted and scrolled into view automatically, and the simple locate function was replaced with a custom Google Map view. The map has Caltrain stations as an overlay, and the map starts out centered and zoomed so that all stations should be visible. While the map is active, it will update the users current position (using GPS and/or network location), and showing the nearest Caltrain station (as the crow flies). It is possible to select this station as the starting station from the map. Additionally, it is possible to tap the directions button on the map to get driving directions to the nearest station.

Working on the map feature was a pleasure now that I actually have an Android device.

If you look at the map view carefully, you will notice the text is written in transparent grey boxes with rounded corners. A stackoverflow question showed me how to do this with custom shape resource. I also based the map overlay largely on this blog post.

I had agonized for the longest time about how to get highlighting for the the list view. Then I stumbled into a couple of stackoverflow questions that showed me the way. In the end the best example I found was the List14 sample in Android API demos. I just need to call convertView.setBackgroundDrawable() with the right drawable, and make sure to add the following attribute for the list view: android:cacheColorHint="#00000000"

My first Android phone is the Nexus One. I tried to wait for an AT&T one, but the battery in my 8525 wasn’t so good anymore so I decided I could wait no longer. So far I am pretty happy with the phone.

I ordered the unsubsidized phone, which leaves me an option to go with T-Mobile or staying with AT&T. Time will tell which one I choose. T-Mobile coverage is my main concern. If I knew roaming would work well in the US without obnoxious costs, I might switch. On the other hand, I have been reasonably happy even with AT&T’s EDGE speeds. It is fast enough for checking email, do light browsing, and navigation works as well. Most of the time when I need the higher speeds I am in some location where I can use wifi.

Setting up the phone turned out to be a breeze. I just went through all the settings on first boot and everything worked. I was initially somewhat concerned about being able to put in the settings for AT&T network, but I didn’t really need to do anything fancy, just looking in the wireless options and selecting the only carrier it found (AT&T).

Since this is my first actual Android device, I put my Android apps through some tests on actual hardware. Everything worked! I’ve also been able to work on some new features (namely location related) that I didn’t much attempt with just the emulator. Some updates will roll out shortly.

Getting adp working with Nexus One turned out to be a little more work. First I updated the SDKs and the Eclipse ADT plugin, but even after this the phone was just showing with question marks in the launch dialog and would not let me copy the apk and run it. I found a thread on xda-developers which gave me the answer. I am on Ubuntu 8.04 and this is what I did:

1. Create file /etc/udev/51-android.rules with contents SUBSYSTEM=="usb", SYSFS{idVendor}=="0bb4", MODE="0666"
2. Restart adp: adb kill-server && adb start-server
3. Turn on USB Debugging on the phone in Application > Development settings
4. Connect the phone with USB cable to the computer

I get great battery life on the Nexus One. And what I mean by that is that I can go for two full days before I need to recharge with my normal usage. I keep wifi, bluetooth and GPS off unless I am actually using them. I talk less than five minutes on the phone per day, browse a handful of websites for a few minutes, and read email several times a day. Surprisingly I also find myself playing some games, probably half and hour to an hour a day. You could say on most days I am not using the phone for anything that’d require a smartphone.

During my first week the phone had some stability issues, crashing several times when I had left it in the charger overnight. But once I uninstalled some applications that kept services running (SIPDroid, Fring and some other IM clients), I haven’t had a single crash. Best uptime when I remembered to check was over 300 hours.

I’ve encountered some annoying bugs as well. During my first week the phone got into a state where tapping the messaging icon always launched the browser. The only way I could recover was to reboot. This hasn’t happened since. Far more common problem is the soft keyboard not registering taps, or thinking I pressed one of the four buttons at the bottom (back, menu, home or search, with accompanied vibration feedback). Switching back and forth between apps fixes this eventually. I also seem to have situations when I don’t seem to get any GPS information. Some people have reported that this might require a reboot, but I haven’t been trying persistently enough to confirm if this is the only cure.

Voice recognition generally works great. There have been a few bizarre mistakes, like once I did a voice search saying “swype for android” and voice recognition thought I said “life on crack”. I find I miss a physical key I could press and do arbitrary voice commands, though.

One of my major issues with the phone, besides it not working on AT&T’s 3G network, is the inability to call out with Bluetooth without handling the phone. With my 8525 running Windows Mobile 5 and 6 I was able to tap my Bluetooth headset and say a name in my addressbook, and it would make the call. With Android nothing happens when I tap the headset, and I need to find my contact by handling the phone. This seems bizarre to me, since as far as I know California law requires hands free calling, and Google is based in California. Answering calls does work as I expect.

I haven’t been thrilled with voice quality on calls, which has been a surprise. I seem to get a fair bit of static. Also the earplugs and microphone that shipped with the phone have been practically useless; the other party can hardly hear anything when I try to use them.

I haven’t bought any apps yet, although I was tempted to get Locale. However, Locale author(s?) seem to have pissed off their potential customers by silently removing the free beta when they introduced their relatively expensive paid version while at the same time apparently making the paid version worse than the free version was. I’d also be interested in being able to buy Swype, since I don’t seem to be a very good typist with the builtin soft keyboard. Unfortunately Swype is not (yet?) available on the Android Market.

I don’t want to sync my data with Google, so I am looking for some way to sync with my PC. So far I haven’t had much luck. There are apparently paid solutions for Windows, and maybe Mac, but haven’t found anything for Linux. The most promising effort to me seems to be the Funambol Android Sync Client. AFAIK you could run your own Funambol server, which would solve my sync needs nicely. Except that at this point only contact sync is supported. If my phone loses data now, I am going to be sad…

I installed the Google update which added multitouch to Google apps, and I really like it in the browser.

I haven’t found the need to root my phone yet, but I will probably do that down the line to get some of the features that are not available otherwise.

Android Market handles sales through Google Checkout. I haven’t tried selling anything else online before, but what this setup provides for me as the seller leaves a lot to be desired. One issue you will have trouble with is getting the data needed to file taxes.

Google provides a Google Checkout Notification History API that lets you programmatically query sales data. For my purposes the API requests are really simple: just post a small XML document with the date range I am interested in, get back XML documents that contain my data. If there is more data that fits in a single response, look for an element that specifies the token for the next page and keep pulling until you get all data.

Below is a really simple Python script that uses M2Crypto to handle the SSL parts for the connection (needed since Python doesn’t do secure SSL out of the box). You will also need to grab certificates. You should save the script as gnotif.py, save the certificates as cacert.pem and create gnotif.ini as described in the script below all in the same directory. When you execute it, it will ask for start and end date (in YYYY-MM-DD format) and then fetch all the data, saving them in response-N.xml files, where N is a number.

#!/usr/bin/env python
# Script to query Google Checkout Notification History
# http://code.google.com/apis/checkout/developer/Google_Checkout_XML_API_Notification_History_API.html
 
# Supporting file gnotif.ini:
#[gnotif]
# merchant_id = YOUR_MERCHANT_ID_HERE
# merchant_key = YOUR_MERCHANT_KEY_HERE
 
import base64
import re
from ConfigParser import ConfigParser
 
from M2Crypto import SSL, httpslib
 
ENVIRONMENT = "https://checkout.google.com/api/checkout/v2/reports/Merchant/"
XML = """\
<notification-history-request xmlns="http://checkout.google.com/schema/2">
%(query)s
</notification-history-request>
"""
 
config = ConfigParser()
config.read('gnotif.ini')
MERCHANT_ID = config.get('gnotif', 'merchant_id')
MERCHANT_KEY = config.get('gnotif', 'merchant_key')
 
rawstr = r"""<next-page-token>(.*)</next-page-token>"""
compile_obj = re.compile(rawstr, re.MULTILINE)
 
auth = base64.encodestring('%s:%s' % (MERCHANT_ID, MERCHANT_KEY))[:-1]
 
ctx = SSL.Context('sslv3')
# If you comment out the next 2 lines, the connection won't be secure
ctx.set_verify(SSL.verify_peer | SSL.verify_fail_if_no_peer_cert, depth=9)
if ctx.load_verify_locations('cacert.pem') != 1: raise Exception('No CA certs')
 
start = raw_input('Start date: ')
end = raw_input('End date: ')
 
data = XML % {'query': """<start-time>%(start)s</start-time>
<end-time>%(end)s</end-time>""" % {'start': start, 'end': end}}
 
i = 0
 
while True:
    c = httpslib.HTTPSConnection(host='checkout.google.com', port=443, ssl_context=ctx)
    c.request('POST', ENVIRONMENT + MERCHANT_ID, data,
             {'content-type': 'application/xml; charset=UTF-8',
              'accept': 'application/xml; charset=UTF-8',
              'authorization': 'Basic ' + auth})
 
    r = c.getresponse()
 
    f=open('response-%d.xml' % i, 'w')
    result = r.read()
    f.write(result)
    f.close()
 
    print i, r.status
 
    c.close()
 
    match_obj = compile_obj.search(result)
    if match_obj:
        i += 1
        data = XML % {'query': """<next-page-token>%s</next-page-token>""" % match_obj.group(1)}
    else:
        break

As you take a look at the data you will probably notice that you are only getting the sale price information, but no information about the fees that Google is deducting. Officially it is a flat 30%, but I have found out a number of my sales have the fee as 5%. So we need to get this information somehow. Luckily you can toggle a checkbox in your Google Checkout Merchant Settings. Unfortunately there is a bug, and the transaction fee shows as $0 for Android Market sales. I have reported this to Google, and they acknowledged it, but there is no ETA on when this will be fixed.

I also haven’t found any way to programmatically query when and how much did Google Checkout actually pay me. (I can get this info from my bank, but it would be nice to query for that with the Checkout API as well.)

Last but certainly not least, working with the monster XML files returned from Google Checkout API is a real pain. If someone has a script to turn those into a format that could be imported into a spreadsheet or database that would be nice…

Not a big update to Caltroid at this time, but I wanted to get out the release because Caltroid now works in 320×240 resolution (QVGA), and there are even devices out with that resolution, like the HTC Tattoo. Of course bigger screens will work as well (I’m looking at you, Verizon Droid). The other minor user visible change is that when you locate the nearest station with GPS, the destination station will stay the same whenever possible.

Under the hood I fixed the spaghetti layout into pretty clean RelativeLayout affair, except for the red strip container which is LinearLayout so I could use layout weight to size the departure and destination stations the same. Apparently this can be a performance hit, but it is probably overall faster than it used to be. (Yeah, bad me, I didn’t measure.)

Figuring out how to make an apk that would work in all SDK versions took some time as well. I ended up creating layout-320x240 and layout-land-320x240 directories to put in the QVGA specific changes needed. I should change to layout-small instead, but for some reason I could not get this to work. I also added android:targetSdkVersion="4" into AndroidManifest.xml and built with 1.6 SDK.

Final change in place is that I exported using the wizard in the Eclipse plugin, which should zipalign the apk and make the app use less memory.

Oh, I should note that Caltroid passed 100 users last month. Thanks to everyone who purchased it, and made it the best rated Caltrain schedule application for Android!

I read about a survey of iPhone developers which found that a third had made under $250 and 52% under $15,000. Not exactly reassuring for anyone wanting to earn a living with iPhone apps.

As far as I know, there hasn’t been any similar studies done of Android developers, so I decided to set up a simple poll to see how we compare. I couldn’t find out what the iPhone survey counted as earnings, but I am limiting my first poll to just application sales and in-application advertisements. I have read Mark Murphy’s 40 Android Business Models posts, so there are definitely more ways to make money, but let’s start with the survey I set up.

With that set up, head over to the poll page.

Caltrain updated schedules today, August 31, 2009. I released updated versions of my Caltrain schedule applications last night, so this is a little delayed notice.

My first application is CaltrainPy, which started out as a Python app for my phone, but it did not work so great on the phone after all, at least on Windows Mobile. It is fine for desktop use, though. I added schedule scraping functionality later to support the next two applications. Fully open source and all that.

CaltrainJS is a web version, optimized for Windows Mobile browser. It of course works with other browsers as well, although it hasn’t been optimized for them. CaltrainJS is free to use, but not really open source.

The last and most fully featured one is the Android port, Caltroid, which is a commercial version. I’ve added a little Google Comments gadget on the Caltroid homepage in the hopes that people would find this an easier way to communicate with me on what new features they’d want, what is broken and so on. The Android Market comment system doesn’t let me respond to comments in any sane manner, but the Comments gadget does. The gadget isn’t perfect, but hopefully better than nothing. This is also first time I am testing out Google Gadgets, so time will tell if I’ll keep using them.

The only application to gain small new features is the Python version, which basically just got some properties to identify which schedule it embeds and what schedule date it was confirmed to be able to process. I have some additional ideas for improvements, but due to currently being somewhat sleep deprived those will have to wait a bit. But if you do have ideas, please let me know.

The Chandler ecosystem gains a new member: ChandlerQE for Android. Your phone is always with you, so you can quickly send notes to the Chandler Hub, and then use Chandler Desktop or your desktop browser to manage your items.

To get started, you must first create an account on Chandler Hub with your desktop browser. It is easy, and free. Of course, if you are already using the Hub you can just use your existing account. This step cannot be done with the browser in Android, because Chandler Hub does not work in a mobile browser (which is the main point why I made ChandlerQE for Android). If you are new to Chandler, be sure to check out the quick product tour.

When you first launch ChandlerQE for Android, it will ask for the Chandler Hub username and password. It will also try to automatically connect to the Hub to fetch your collections every time you change your username or password. The main UI consists of the title field, the content field, the collections list, and the send button. Enter the title of your note in the title box, then content in the content box, and select the collection you want to send the note into. If your collection is not listed but you know it is on the server, select the “Update from server…” item from the list, and after the list is updated, select the collection you want. Finally hit the Send button. After a little while you should get message in the UI informing you that the send was successful.

ChandlerQE for Android 1.0 Screenshot

Background

Unless you lived under a rock the past few years, you have probably heard of the Chandler project. Mitch Kapor of Lotus fame founded the Open Source Applications Foundation (OSAF) in 2002 to create great Open Source software. The flagship project is and was Chandler, the note-to-self organizer, which aimed to be the killer personal information manager. While Chandler didn’t gain mass adoption similar to Firefox, it is still pretty innovative and used by thousands of people. There is even a book written about Chandler development: Dreaming in Code.

OSAF is running a free instance of the Chandler Server, aka Cosmo, called the Chandler Hub. The server is written in Java. Its original purpose was to enable sharing between Chandler Desktop clients, but the server quickly grew a UI as well, so it can be used on its own. Unfortunately the web UI is not usable in a mobile browser such as the one that ships with Android, nor will the desktop client – which is written in Python – run in a mobile device. So we needed a native client, hence ChandlerQE for Android.

Even while I was employed by OSAF between 2003 and 2008 we were thinking of ways to get revenue to make the non-profit foundation self sustaining. Many different models were considered and dropped. Currently OSAF relies purely on volunteers and donations. I have been tinkering with Android since early 2008 and when the Android Market went live with paid applications a light bulb went on. People have been willing to pay small amounts of money for mobile applications, so why not try that with Chandler? So I proposed to OSAF that I will create a Chandler-on-Android application, sell it on the mobile marketplaces since I am already doing that with some of my other projects, and donate a percentage to OSAF. OSAF said yes, so here we are! Depending on how well this takes off, we might consider something else in the future.

And if you need one more reason for ChandlerQE for Android, there already was ChandlerQE for iPhone. That just won’t do for us Android fans. So what are you waiting for, go grab ChandlerQE for Android!