Sonoma Partners Microsoft CRM and Salesforce Blog

How to Plan Test Data for Testing Mobile Applications

Today's post is written by Jen Ford, Principal QA at Sonoma Partners.

When brainstorming test data for testing out a mobile app, I go with what I learned in the first grade: The 5 W's - Who, What, Where, When, and Why.  In this blog post, I will address the “Who” and the “What".

The “Who” - Security

Security is extremely important to consider in a mobile app.  If you are connecting to a back-end system, you want to make sure that the security that exists on the back-end is mirrored in the mobile application.  This means that users shouldn’t see anything in the app that they can’t see in their own environment.  This is a great place to start looking for defects.  Not properly handling security within the app can lead to crashes and data loss when syncing to the back-end system.  For example, let’s consider a requirement for expense submission and approval.

Requirement: Users will create expenses and submit them for approval.  The manager will receive an e-mail summary of the expenses, and will approve the expenses.

  • All users should be able to create, submit, and view their own expenses.
  • A sales person cannot view anyone else’s expenses.
  • A manager can view their own expenses, and all the expenses of their direct reports.
  • No users should be able to approve their own expenses.
  • A manager can view, edit, and approve the expenses of all their direct reports.

Using these requirements, we can determine what permissions each role should have.  This is as simple as jotting this down in an excel spreadsheet or table, just like the below:

Expense Security Scenarios

ID #

Overview

Salesperson

Sales Manager

ES-01

Create new expenses

Yes

Yes

ES-02

Create new expenses on behalf of others

No

No

ES-03

View their own submitted expenses

Yes

Yes

ES-04

View others’ submitted expenses

No

Yes

ES-05

Update own expenses before submitting

Yes

Yes

ES-06

Update others’ expenses before submitting

No

Yes

ES-07

Update own expenses after submitting

No

No

ES-08

Update others’ expenses after submitting

No

Yes

ES-09

Approve their own expenses

No

No

ES-10

Approve others’ expenses

No

Yes

Now we know that there are 10 scenarios to test for each User Role. The next step for each Expense Security Scenario is to identify what test data needs to be set up to accomplish each test above.

The “What” - Test Data

Careful thought processing before starting testing is the key to smooth test execution. 

By putting forth the effort to plan out your test data, you will consider all scenarios up front.  This will lead to ease in execution, or the ability to bring in other people to test if the timelines are tight.  Planning your test data and setting it up, if it needs to reside in a system that is integrating with the application, can be done while development is going on.  The important thing here is to figure out what is happening on each page of the app. 

Consider our expense submission and approval example above.  First, let’s think about what expenses we will want to set up in our environment to successfully test the submission page (see the sample below).  In the analysis, you will see that there are scenarios that will validate the “happy path”, and scenarios that are expected to fail.  I recommend jotting down every scenario you can think of, even if it ends up being 100 different scenarios.  Then, when you are done, take a look back at it and determine what you do not need to do.

Maybe you have redundant scenarios or maybe you have scenarios that can be combined. In the chart below, take a look at scenario #2.  I could have positioned this as two different scenarios: one to make sure that an expense with an earlier date should be able to be entered, and one where an expense that is not a whole dollar amount should show the cents when submitted.  I combined them because I expect both to succeed.  If you identify your test data and flow for each page of your custom app up front, you will be able to quickly hit the ground running when it is time to test.

Expense Creation Scenarios for ES-01

ID #

Scenario

Date

Amount

Expected Result

ES-01-01

Add an expense with today’s date, and any $ amount.

[today]

$100.50

Expense added successfully.

ES-01-02

Add an expense with an earlier date, and any $ amount.

[yesterday]

$50.55

Expense added successfully.

ES-01-03

Add an expense with a negative amount.

[today]

-$75.00

Expense not added.  Error message displays.

ES-01-04

Add an expense >= $1,000.00

[today]

$1,234.56

Expense added successfully.

ES-01-05

Add an expense with a future date.

[future date]

$100.00

Expense not added.  Error message displays.

ES-01-06

Add a $0 expense.

[today]

$0.00

Expense not added.  Error message displays.

ES-01-07

Add an expense with a blank amount.

[today]

 

Expense not added.  Error message displays.

When you combine the two charts above, we can see the test cases flesh out.  We have 7 test cases for Expense Security Scenario ES-01.  Considering we need to test this across 2 different user types, we now have 14.  And there are nine Expense Security Scenarios to go!  Not every one of these will need 7 scenarios each.  If we were to flesh out ES-02, that one is quick for both the sales person and the sales manager: one test case each to make sure that they cannot create new expenses on behalf of others.

For each of the Security Scenarios, we’d repeat the exercise by going through and identifying the necessary tests. Excel is great for keeping track of these.

Record Counts: Online & Offline

Now we need to consider how much data we need to sync down to the device when it is online and when it is offline.  To do this, let’s make another chart and figure out what to expect.  Interviews with key stakeholders will help you answer these questions.  Test with too little data, and you can run into unanticipated performance issues.  Test with too much data, and you could be spending extra time performing unnecessary tests.

For our Expense Scenario above, I would ask the following:

  • How many expenses would a user submit per day?
  • How many users report to a sales manager?
  • How long should past data be available on the device?
  • How much of this data needs to be available offline?
  • Is there a scenario for a system administrator who will have access to all records in the system?
  • If so, how many users are there?

And we may get these answers:

Scenario

Response

# Expenses per User per day?

20 expenses / day

How many users report to a Sales Manager?

10 users

How long should data entered in the past be available on the device?

1 month

How much of this data needs to be available offline?

All data

Will a System Administrator use this app?

Yes

# of Total Users

100 users

Looking at these amounts, we can see that a sales manager will have significantly more data than a sales person, and that data needs to be available on the device for 1 month.  The requirements are the same, whether the device is online or offline, so:

  • Each sales person will then have a max of 20 expenses per day for 1 month (31 days) = 620 expenses
  • Each sales manager will then have this amount of data for 10 users = 6,200 expenses PLUS their own expenses: 620. Total expenses for a sales manager = 6,820.
  • A System Administrator will then have this amount of data for 100 users = 62,000 expenses

Now we know that when we're creating our test data for the mobile app, we will not only need to consider the various scenarios to test for data entry, but also how much data we need to pre-populate at the start to ensure that the devices can handle the load. We will be modeling real-life scenarios from the start of testing, which will uncover performance and usability issues early on.

Happy Testing!

Topics: Enterprise Mobility