Sunday, December 29, 2013

K6ps's Ultimate Smartphone Test

I'm writing this in English, in case it interests any smartphone developers worldwide.

The Test

This is the test i intend to put all my current and future handsets through:

  1. I enter my car, switch on the car.
    1. The handset should switch to "car mode". Only unlocking the handset (insert security code, mazelock, fingerprint scan, face recognition, etc.) plus at most one additional simple movement (tap a button, touch an NFC tag, etc.) should be required to switch it to car mode.
    2. When going to car mode, the handset should:
      1. Switch to ringing mode, if it was silent before
      2. Connect to external audio system, if/when it is available (such as Bluetooth headset or car built-in audio)
      3. Launch my favorite music player, if/when external audio system is available and it is stereo (i.e. not if only mono Bluetooth headset is available)
      4. Launch my favorite navigation system and bring it to foreground. The navigation system should show traffic information, e.g. traffic jams, problems on road, police, etc. (such as Waze)
      5. Not go to sleep mode and darken the screen (unless battery is getting emptier than a particular treshold),
  2. I navigate to an address in my calendar entry, address book entry, or a web site.
    1. It should start navigation in my favorite navigation app
    2. Copy and paste of address is the most complicated task that should be needed.
    3. If both a headset and car audio system are active, the navigation voice messages should only come to the headset.
  3. I switch on the car audio player
    1. The handset should resume playing music where it left off last time.
    2. If navigation app is in foreground, the music player should stay in background.
    3. Controlling the music should not require me to take my eyes off the traffic/road (if the car has external music controls (e.g. play, stop, previous, next, volume up, volume down buttons on wheel), the handset should be controllable by these).
    4. If both a headset and car audio system are active, the music should play only in the car audio system.
  4. Navigation (voice) message comes while music is playing.
    1. The handset should smoothly pause or mute the music before the voice message.
    2. It should smoothly resume the music after the voice message.
  5. Incoming call comes while music is playing and navigation is active
    1. The handset should display caller name (if it is in my phonebook) or number on screen with minimal disturbance of the navigation screen
    2. It should smoothly pause or mute the music and start ringing.
  6. I answer incoming call
    1. Answering incoming call should not require me to take my eyes off the road (E.g. it should be automatic, push of a button on headset, push of a button on steering wheel, etc.)
    2. If music was playing, it should be smoothly paused or muted.
    3. If navigation app is in foreground, it should stay in foreground. Only a minimal visual indicator of an active call is allowed.
    4. If both headset and car audio system are active, the call should only be in headset.
    5. If i turn off the headset while call is active, the call should switch from headset to car audio system.
  7. I initiate outgoing call
    1. Initiating outgoing call should not require me to take my eyes off the road (E.g. it should be a push of a button on headset or on steering wheel and voice dialing that resolves contact names also in Estonian)
    2. If music was playing, it should be smoothly paused or muted.
    3. If navigation app is in foreground, it should stay in foreground. Only a minimal visual indicator of an active call is allowed.
    4. If both headset and car audio system are active, the call should only be in headset.
    5. If i turn off the headset while call is active, the call should switch from headset to car audio system.

Some Results

Nokia N9/SailfishOS

The SailfishOS was very recently ported to N9 and it has many issues so far. So i feel it is not time yet to start the test. However, i've installed SailfishOS to my N9 and have played with it a bit.

Nokia N9/MeeGo + ProfileMatic + QuazarMX + Waze unofficial Qt port

Works pretty well, except:

  • Fails completely on the "outgoing calls" section, because it has no built-in voice recognition. There are a few third-party voice command apps, but all of them rely on the Google Voice service, which has no Estonian language support. There is zero chances than any of such apps correctly resolve and Estonian name to a phonebok entry.
  • The Waze port is very raw and has several issues. Also, it is very unlikely that it will be ever updated for N9.
  • Not possible to launch navigation with Waze from address book or calendar entry. Can't copy/paste addresses from some places either, e.g. from an e-mail message body (with the built-in MeeGo e-mail app).

Nokia 5230/Symbian S60

Works surprisingly well. It (the S60) is the only platform so far that i have seen to pass the "outgoing call" test. Some issues:

  • I did not find any app for S60 that switches profiles automatically on certain conditions. However, switching them manually was very straightforward.
  • I did not have any interactive navigation app, but the built-in Nokia Maps was fairly good too.

Others

I have played a little with iPhone 4s and a few Android devices, but i have not systematically performed this test with these. However, i'd expect Android and iOS devices to succeed on everything else (because of multitude of apps available) but fail on the "outgoing calls" section (because of no Estonian language voice support). I have never played with any handheld device of Windows Phone, Blackberry, the newer versions of Symbian, and others.

No comments: