10 Points App Testing Formula For A Successful Mobility Programme

Your 10 Points
App Testing Formula

For A Successful
Mobility Programme

1

Executive summary 3
4
A driving force ridden with challenges 6

• Speed of app release 11
• App complexity due to back-end integration 12
• App user experience
• App performance
• Security and compliance

10 App testing best practices

• Start the test process early on
• Invest in test automation
• Test your app on adequate number of real devices
• MDC is a must!
• Cover the test-spectrum end-to-end
• Use the right tools
• Do not ignore localisation testing
• Unleash the power of Artificial Intelligence
• Explore crowd testing
• Treat testing as an ongoing function

Top tools and framework

Hand over the job to experts

2

Executive summary

From being perceived as a security challenge by the CIO’s office, enterprise mobility has evolved
into a must-have technology with strategic business importance today. Enterprises across sectors
are adopting mobility to improve productivity and transform business operations. According to
data published by Strategy Analytics, the mobile enterprise business applications market will
touch $63 billion by 2020. Another report, by Research and Markets, predicts that the enterprise
mobile apps market will reach $98.03 billion by 2021, posting 15.24% growth annually. In 2016,
it stood at $48.24 billion.

Size of mobile Size of mobile
enterprise apps enterprise apps

market by market by

Source: Strategy Analytics Source: Markets and Markets

As enterprise mobile apps adoption sky-rockets, so do the associated challenges. With mobile
apps turning into essential user-connect channels for enterprises, their performance and user-
experience have become crucial to enterprises’ success in today’s highly digitalised world.
The apps that enterprises build must score well on all parameters ranging from functionality,
performance, user experience and adoption. Given this scenario, mobile app testing has emerged
as a business-critical function demanding special attention. it stood at $48.24 billion.

According to Gartner, progressive web apps (PWAs) will replace 50% of general-purpose,
consumer-facing mobile apps by 2020. PWAs aim to disrupt the mobile app paradigm by bridging
the web experience with native app functionality. Application leaders responsible for mobile app
strategies must determine when — not if — they need to factor in PWAs as part of their overall
mobile development strategy, further said the study. Keeping abreast of trends, pCloudy has
added innovations in its ecosystem. Its advanced offerings have an increased focus on Internet
of Things (IoT), PWAs and Artificial Intelligence (AI)-powered app testing bots.

This whitepaper examines the business challenges associated with mobile apps and illustrates
how a few app testing good practices can help tackle these with ease.

Enterprise mobile apps are reshaping the way companies conduct their business. Whether it’s
about speeding up business processes, creating employee-friendly work-spaces, customer
engagement, brand building, or changing their business models, mobile apps provide immense
opportunities to achieve these goals and more.

3

A driving force ridden with challenges

Even as mobile apps have become critical tools for businesses, enterprise mobility is not free
of challenges. The DevOps era has brought with it its own set of benefits and issues.

New ways of Continuous Integration, Continuous Delivery and Continuous Deployment means
daily updates or multiple times a day has become the norm. This gives rise to challenges such
as resistance from users, pushing for cultural changes in companies, building an automated
code repository and maintaining the same, and more.

Let’s explore these factors in depth.

Speed of app release

This factor needs to be examined from two angles with respect to mobility.
From the development perspective, the enterprises must be geared
to introduce new mobile apps, version upgrades, and updates very
fast. From a major version upgrade every year and quarterly
releases traditionally, organizations have to attune to weekly or
even daily updates. Leading organizations like Facebook, Google,
and Amazon release several updates on a daily basis, and the ones
that cannot meet such frequent updates expectation may be left
behind. Organizations have to consider various aspects including agile

development methodology, continuous testing to stay in the race.

App complexity due to back-end integration

Enterprise mobility evolved into a major trend after the
introduction of the first Android phone (by HTC) in September
2008. As companies look to embrace mobility, have to deal with
issues about legacy modernization.

Their mobile apps must seamlessly integrate with backend
systems such as core banking, payment platforms, ERP, SCM,
CRM, insurance policy administration systems, securities
trading platforms, sales force automation, and distribution
management suites.

Also, companies also look at integrating their mobile apps with other advanced technology systems
cloud, IoT, analytics and machine learning. Any legacy modernization or application integration
issues including latencies may instantly result in a dip in user experience, and correspondingly,
customer disgruntlement. Following a few good practices in app testing may help nip these
issues in the bud.

4

App user experience

It’s an established fact that a great user experience contributes
substantially to the adoption of a mobile app. The converse
is equally true. A dip in user experience may mar an app’s
ability to retain users. Several aspects such as visual
sequencing, finger-touch sizes, navigation frameworks,
and response times go into creating an intuitive user
experience. A simple, clutter-free, and user-friendly interface

helps create a seamless user experience for your apps.

While nearly all industries feel the need for enterprise mobile
apps, the mobility challenges they face create roadblocks in their
efforts toward achieving their digital transformation goals. However,
adherence to a few app testing good practices can help organizations turn
their enterprise mobility programmes successful bypassing the hindrances. Let’s explore these
in more detail.

App performance

Is your application fast enough to prompt the user to use your app?
Higher the latency, higher are the chances that the customer will
close or worse, uninstall the app. By focusing on mobile application
access speed, organizations can ensure that they do not lose their
customers to competitors.

Security and compliance

Widespread adoption of mobile apps has led to rising concerns about security
and privacy. Enterprises need to ensure that their apps follow strict information
protection guidelines. The issue has become highly sensitive due to a
proliferation of mobile wallets, insurance apps, trading apps, and
m-commerce. Same is true with mobile apps meant for healthcare,
communication, and entertainment that collect users’ personally
identifiable information (PII).

At another level, organizations also have to ensure that enterprise
resources are not misused by users—internal or external. Companies
have to employ newer ways to authenticate and authorize users (two-factor authentication,
OAuth 2.0, etc.) to accomplish their compliance, security and data privacy objectives. The mobile
apps they build must be rugged enough to adequately protect enterprise data as well as users’
privacy concerns, and correspondingly, their brand reputations.

5

10 App testing best practices

Components of mobile Most organisations acknowledge the
app testing business-critical nature of mobile
application testing. However, many are
Usability testing wary of the challenges that the process
presents. It’s a common experience of
Interoperability testing CIOs familiar with app testing that new
bugs continue to surface even as the Go-
User experience testing Live date nears. Adherence to a few best
practices may help IT leaders ensure that
Low-level resource their mobility programmes deliver the
testing desired outcomes without compromising
on predefined timelines. Let’s explore
Performance testing these practices a little more in detail.

Business continuity 1. Start the test process
testing early on

Performance The CIOs and team leaders who have
testing worked on application testing projects may
be aware that some of the issues that tests
Security testing throw up pertain to the software design
phase itself. This happens when, due to
deadline-pressures, the project team
is rushed into mobile app development
without adequate planning. However, it
proves expensive and time consuming
if planning and requirement-gathering
related issues are spotted only after the
development is complete. It adversely
impacts the quality and performance of
the app since the planning-level issues
are fundamental to an app’s purpose or
feasibility.

CIOs need to adopt shift left testing wherein
testing starts right at the beginning of
software development. This approach
stresses on culling out probable issues
that could develop and averting them then
and there. It puts quality assurance into
place at the roots, ensuring time, efforts
and energies are not wasted in making
mistakes and then doing damage control.

6

SHIF T =
LEFT

EARLY COST TIME RELEASE
Smooth
Detection Effectiveness Saving

In case of a collaboration app, for example, if developers fail to incorporate the critical require-
ments of a particular team due to imperfect requirement gathering, developing the same later
may affect the app’s quality with new bugs being introduced during the rush hour development
work. Starting the test process along with development helps in identifying bugs early on. Tak-
ing corrective steps proves less expensive and faster in this scenario.

2. Invest in test automation

Much like the traditional software testing, even mobile app testing has two components: tasks
that are repetitive and ones that require custom-coding. It makes sense to automate the
mundane, repetitive tasks while running manual tests for the rest. Among several other benefits,
test automation allows you to run multiple tests in parallel, reuse the same code multiple times,
and achieve high efficiency by running tests 24x7.

7

Automation may prove to be a boon in the context of m-banking or m-commerce where a minor
bug can have a disastrous effect on a company’s revenues and reputation. Automated tests can
help detect issues with greater efficiency and speed so that fixes can be issued without delay.

Calculating the ROI

Initial Investmen t Manual Testing
Cumulative Cost Saving
Cumulative Costs ($ )
Automated T esting

Time

ROI = Net Value/Investment Cost
* The maintenance cosy of TA is usually not tri val but if you keep enhancing
your TA framework, improve application testability and over all testing processes
high ROI is very much achiev able

3. Test your app on adequate number of real devices

In May 2017, Google announced that it had more than 2 billion active devices on Android on
a monthly basis. Nearly a year-and-a-half before this, Apple had announced that the total
number of active Android devices stood at 1 billion. These announcements provide insights into
the potential size of the installed base of smartphones.

Given the vastness of the smartphone user-universe, it’s a no-brainer that 7 8 simulators and
emulators may not be adequate to test mobile apps thoroughly. Arguably, it’s impossible to
invest in every unique device available. However, a detailed analysis of the total number of
unique devices used by your target audience should help determine the adequate number of
real devices to run the tests on.

4. MDC is a must!

Modern enterprises, have a business footprint at multiple locations (nation-wide or global). As
a result, they face a few unique challenges when it comes to mobile app testing. Not only are
their development and testing teams based in distant regions, but so do their target users. As
device procurement is done at different office locations, it typically results in redundant device
purchase and cost overruns.

8

Since team members have to share the Flawless application
devices for testing, it affects efficiency performance
and speed of test operations. The multi-
location nature may also open up DevOps depends on how
and collaboration challenges. Mobile complete your
device cloud (MDC) can be a highly useful
approach in such a scenario. Besides testing coverage
trimming spends due toon duplicate has been
device purchases, MDC eliminates wait
times by automatically provisioning
devices to specific team members for
specific time duration.

MDC may be the answer to cost, security and compliance issues companies face in their mobile
app testing environment. It can help perform manual or automated testing on real devices,
facilitate scheduling and DevOps capabilities, among other things.

Organisations can choose from public, private or an on-premise model to see what best suits
their app testing needs. pCloudy provides a Public Device Cloud service wherein the devices are
shared with everyone and its online capabilities can help test apps across a range of Android
and iOS mobile app hosted on the cloud. Its Private Device Clouds are privately hosted with
dedicated mobile devices. The On-premise Device Cloud service is generally a mobile digital
lab solution deployed at the client’s premises. The table below explains it further

5. Cover the test-spectrum end-to-end

Flawless application performance depends on how complete your testing coverage has been,
besides its effectiveness. To ensure your mobile application succeeds on every key parameter—
functionality, performance, user experience, and adoption—your testing scope needs to be
comprehensive. Your app testing project should cover all the testing areas including performance,
backend integration, interoperability, and user acceptance. The team you appoint— internal or
external—should be experienced in all these areas.

6. Use the right tools

While the importance of using automation tools cannot be ruled out, it is necessary to identify
the right tools and frameworks. Tool selection should ideally factor in aspects such as support for
multiple platform simulators as well as real devices and OS versions. Choose a set of tools that
cover the complete app lifecycle management to deliver desired results. (See box: Top tools and
frameworks)

9

7. Do not ignore localisation testing

Localisation is an important discipline that global organisations have to be cognizant about.
Organisations rolling out enterprise-wide mobility programmes need to ensure their apps comply
with country-specific regulations and the region-specific features work with finesse. For instance,
insurance carriers in the United States have to follow the guidelines stipulated by individual
state governments besides the federal regulations. The insurance apps, therefore, must be
tested for a smooth functioning of state-specific laws and features. Similarly, in many countries
devices come with region-specific versions, such as devices running FlymeOS. In case your app
is targeted at such country or geography, your enterprise mobile app must be tested thoroughly
for such customised OS versions built for a particular geography.

8. Unleash the power of Artificial Intelligence

The app testing journey does not get over with the submission of the app with an app store.
Post submission, the app is subjected to a series of checks including security, technical-,
and content-compliance. At this stage, it is recommended to opt for an Artificial Intelligence-
powered (AI) tried and tested app certification tool to provide insights into app behaviour and
performance. Such tools, powered by deep machine learning algorithms, test and monitor
your app for a dozen issues like crashes, memory spikes, high CPU usage, slow response,
and battery drain. Using AI, these tools help you quickly change the status of your app from
‘Publishing’ mode to ‘In the store’.

9. Explore crowd testing

Forward-looking organisations like Amazon, Microsoft and Google use crowdsourcing for

development and testing. Using a crowd-testing platform to invite real users to test your app can

be of immense advantage to e-commerce

ventures given the millions of customers

they target. A mobile app’s

As a huge pool of qualified testing performance gets
professionals from across geographies truly tested only
tests your product, they bring with them after it goes live
a rich local market understanding, besides
the benefits of speed and efficiency
obtained through parallel, round-the-
clock testing.

10

10. Treat testing as an ongoing function

It’s a common mistake committed by organisations to cut down their testing budgets once

an app is launched. In reality, a mobile app’s performance gets truly tested only after it goes
live. The task of the testing team is to continuously monitor the app’s performance in the
real world and find issues. An enterprise mobile app needs to succeed on parameters such as
functional excellence, intuitive user interface, and a compelling business need even after going
live. Typically, when an app fails on any one of these parameters post-launch, the user interest
in the app wanes resulting in uninstall. Testing, therefore, must be considered as an inseparable
part of app maintenance to ensure the app continues to deliver business benefits throughout the
lifecycle.

Top tools and frameworks

pCloudy recommends the following tools and frameworks integrations that promise to turn mobile
app testing a cakewalk.

OpKey: OpKey is an Automation framework which allows users to create Automation scripts in a
script-less way. Seamless integration with OpKey can help create and run automation scripts for
Mobile Apps.
Appium: Appium is the preferred choice for Mobile App Automation. When using Appium, select a
platform that allows for all kinds of Appium scripts to be run effortlessly. The platform should also
facilitate concurrent unlimited parallel runs of Appium scripts on multiple devices.
Selenium: Run your Selenium scripts for cross-browser testing on Mobile Web.
Espresso: Run Espresso scripts on multiple Android devices in parallel. For Espresso to deliver
amazing results, get a platform you can either upload the scripts to or look for solutions that offer
a built-in feature to run scripts.
Calabash: Run Calabash scripts on both IOS and Android Devices.
Robotium: Run Robotium scripts on Android Devices.
Jenkins: pCloudy has released a plugin for Jenkins which provides you access to the platform for
performing automation testing of mobile apps from within Jenkins on real devices.
Android Studio: pCloudy has a plugin for Android Studio which provides you access to the devices
for performing manual and automation testing of mobile apps from within Android Studio IDE.
Eclipse: pCloudy has released a plugin for Android Studio which provides you access to the
platform for performing manual and automation testing of mobile apps from within Android Studio
IDE.
JIRA: pCloudy has introduced a new feature in its latest update 4.6 where testers can directly log
bugs on JIRA while testing the app on pCloudy. You can attach relevant screenshots, video links
or bugs by a single click.
JMeter: pCloudy has recently launched a plugin for JMeter to bring you an insight into the
Performance Testing of your app.
Rest API: pCloudy has an exhaustive set of Rest APIs to control devices, access pCloudy features.
You can use the APIs from your choice of tool/language or framework.

11

Hand over the job to experts

As seen in this whitepaper, mobile app testing is an involved exercise ridden with several
complexities. A plethora of smartphone and tablet brands, device models, a wide range of OS
platforms, custom ROMs, regional-flavours and versions, besides internet data options and
browsers make mobility a vast and complex ecosystem.
In addition, project teams have to deal with issues related to scope creep, DevOps and tight
schedules. Project continuity in the event of the exit of key resources is another challenge that
CIOs face. Given these complexities, the best option is to ensure your project is in the right hands.
Hire mobile app testing experts—external or in-house—who have experience in your industry,
preferably your own line of business. Getting experts to do the job can help you navigate through
the complexities and spell long-term success for your enterprise mobility programme.

About pCloudy

pCloudy, the key product of Smart Software Testing Solutions is revolutionizing the testing
environment in mobile space. pCloudy offers multiple cloud based mobile app testing solutions
from small and medium-sized businesses (SMB) to large enterprises. It offers on-demand public
cloud based mobile testing over more than 500 Android and iOS devices and private cloud and
on-premise setup offerings for enterprises.
pCloudy has also been recognized by top industry analysts including Gartner that included the
products in latest report on mobile testing tool landscape.

12