How to test connected car applications

The automobile as an application platform

The days of the automobile solely as a means of transportation are long gone. Today, new vehicles act as an extension of the user's home. And as part of said home extension, that includes the user's cell phone -- and everything that comes on the phone too.

Let's explore some common application integrations between cell phones and cars, how they can interact with one another in day-to-day examples, and what engineers should be on the lookout for when they design these tests.

Audio -- car on and off. When the car turns on, it will likely connect to the phone and start whatever application(s) the car thinks were running in the background. For example, if the app has sound, customers could suddenly be faced with a barrage of sound when they turn on the car. Think of the car's startup as a new form of interruption -- like a phone call -- and test for it.

Send to feature. Many applications have an OS-level "send to" feature to send data to a friend over text or Facebook messenger. Test what happens when the app and console are connected over Bluetooth and the user "sends to" the vehicle itself. It might be a nonsense command, but perform the test anyway. Nonsense commands can often lead to performance problems.

Long car drives. If an application uses GPS or cellular data, it could run in the background for a long time, regardless of whether it's plugged into a USB port or not. Track how fast the app drains the data and battery over this time, and if the user will even know what's going on. For mapping software, don't just use a simulator; take the car on a three-day road trip instead.

Usability and safety. Consider if the application's user will ever be the driver -- and if the app is safe to operate under driving conditions. One workaround is to program in an "operate at your own risk" popup message. Another option is to consider something modeled after the Apple API, which can tell if someone is in a moving vehicle and restrict what the user can and can't do. It's possible to restrict screens to a "read only" dashboard when the API determines that the vehicle is in motion. However, it can't account for whether the user is the driver or a passenger. Some applications ask the user to perform a series of time-sensitive keyboard tasks that are essentially impossible to perform while driving but are simple for a passenger.

Ideally, drivers would turn on their phone's Do Not Disturb mode or enable a lock screen while driving, but developers should consider these safety restrictions when they test their applications. These discussions might just change one feature in a subtle way that saves lives and makes the application simpler for everyone.

Connected applications and APIs

If a driver can perform an action on a set of car keys, they can probably do it through a phone's connected API. Things like locking and unlocking doors, honking the horn, opening the trunk, using remote start or starting a car alarm are all accessible from car keys and a phone-connected car API. However, this setup creates a security problem. You don't want anybody to walk over to a car, connect via Bluetooth and unlock it.

Manufacturers generally take two steps to resolve this. First, they tie the owner's phone number to the vehicle so only that phone can unlock the device. Second, the manufacturer may have the API calls report back to servers that can track when the functions are called. The servers track adoption, but also allow a manufacturer to turn off a stolen automobile.

But all that security comes at a price. If a function call goes from a phone, to an auto console, then to a server, only to then go back to a console, to the phone and the car horn, there's a lot of room for error.

For example, I served as a tester for a car manufacturer on one of these applications. It was common to wait 60 seconds to 100 seconds for a result. Sometimes the result would come back, "Error, unable to sound car horn," and then the car would honk. Sometimes the car would honk, and then I would get an "Error, unable to sound car horn" message. Because the architecture contained several layers that control where the API call is during the process, it was extremely difficult to determine what was happening where. Essentially, the architecture couldn't be easily observed to detect potential problems.

Testing these sorts of APIs involves more than setting up the data, making the function call and seeing the right result. Testers need to observe how the API handles the requests and determine how long is an acceptable amount of time to see a result. Also, see how long it took at every layer of the architecture. If a mobile app interacts with the console, check the performance when switching between applications on the phone.