A testing strategy can be defined at two different levels:
- An Enterprise-wide test strategy that is applicable to the QA group in the entire organization.
- An application, product or project level strategy that defines the testing requirements and a test plan applicable for the entire life of that application, product or project.
The following is not to be considered a strategy document or even a template but may be used as a starting guide towards developing a more comprehensive mobile application test strategy specifically for a mobile application, product or a mobile application development project. It merely highlights a few important points to keep in mind while developing a testing methodology for a mobile application. With that in mind, here are some key steps to employ:
- Unit testing to verify that a particular module of source code is working properly. Unit tests are designed to test a single class or component or module in isolation. Developers run unit tests, and only for the components that they are working on. Developers must ensure that their code is written to support integrating unit tests, and their data access is designed to support dependency injections if necessary.
- Installation testing to check how the application behaves during installation and uninstallation, for differences in behavior between installing from App store versus installing via USB, for installing fresh versus upgrading from store, and for the installation experience OTA (Over the Air – through URL for download)
- UI/UX verifications
- Input modes – Touch screen, Long touch, short touch, Track ball/ Track wheel, Keypad, button size and position, Font size and color, Sound profile (Test cases around multiple inputs at the same time)
- Screen regulation/orientation –If your application is supported on various devices that have different screen resolutions, make sure you test early with the device that has the smallest screen
- Test where you rotate the device from portrait to landscape display, and vice versa, on all of the pages within your application
- Test input reactions when the screen orientation is changed. Try using the soft keyboard while changing the orientation repeatedly
- Ease of use. Make sure the most used features are easy to find instead of being buried in layers of menus
- Soft keyboards –
- Does the soft keyboard automatically appear if the users main action is to enter some text?
- Does the first layer of the soft keyboard include shortcut “@” and “.com” keys if the highlighted field is for entering email addresses?
- Can the soft keyboard be dismissed and re-displayed easily?
- Hard keys – Be sure to include a lot of testing around the use of the devices with available hard keys such as Start, Home, Menu, and Back. In order for the user to have a great experience, these should all interact with your application similar to how they interact with the device’s native applications.
- Functional flow testing– Verify the functionality from a user’s perspective, based on functional flow or business flow requirements.
- Testing on various network strengths
- No network, Low, Medium, High and testing during changes of network strengths Low/High and High/Low
- Test intermittent network scenarios that a user might encounter in the real world:
- Walk out of Wi-Fi range so the connection automatically switches to 3G/2G (for example, in a large building like a hospital or airport, or outdoors)
- Ride in an elevator or on a train where the network connection may go up and down
- No network connection available at all
- Testing on various network types
- 2G (GPRS, CDMA, EDGE)
- 3G, Wi-Fi, Different types of plans based on service providers.
- Only Wi-Fi connection
- Testing in various battery strengths – High, Low strengths. Observe the battery consumption rate as the application is run in background or foreground.
- Testing against different mobile devices. Login on mobiles of different models and made by different manufacturers and verify especially data manipulation scenarios (ex: delete, add, edit)
- Interoperability testing – If you are developing an app that is intended to run on different Operating Systems (Windows, Android, iOS, etc), make sure you test your app on each operating system and how it interfaces with any common backend services you have may have to support the native apps.
- Monitoring memory usage patterns
- Check on application launch, when app runs in the background and foreground, on app exit and when app runs for a long time.
- When no other 3rd party applications are installed, some third party applications are installed, lots of third party applications that may use some of the same hardware resources as your app (such as the mic) are installed to test interactions
- SD Card interactions – If your application can potentially store or retrieve items on the device’s SD card, then it is important to test the application behavior when there is an SD card present and when it isn’t.
- Interruptions – examples of interruptions – incoming call, receiving message, receiving alerts, camera activated, device shutdown, remove battery, lose and recover network connection
- Stress & Performance testing – Some examples of what to test:
- Verify system behavior on large data transfers.
- Perform the same operations over and over again, particularly those that load large amounts of data repeatedly.
- Leave your application running for a long period of time, both interacting with the device and just letting it sit idle and verify your app behavior.
- Other testing considerations
- Security testing – especially around login, signup and when there is confidential data to be protected.
- API (Ex: Twitter, Facebook API integration) and web service testing (such as with your own back-end services)
Finally, here are some recommendations for testing with tools such as emulator and automation:
|Type of testing|||Manual testing – (Device)|||Manual testing – (Emulator)|||Automated testing||