Test and Quality Assurance – The challenges and pitfalls

Testing and quality assurance (QA) methodologies have undergone important developments over the last few years and as wealth managers and private banks digitise, organisations must deliver faster and more reliable IT solutions. Enabling this shift has been the use of Agile and DevOps methodologies as well as smart automation technologies and software-as-a-Service solutions. Orbium’s Fabrice Devos looks at how you could create a faster, smarter, more robust testing and QA programme.

As the pace of technology increases in-line with clients’ expectations, prioritising digital projects can be a challenge. From the integration of front-office and consumer-facing apps with back-office core systems, to prioritising cloud and microservices, it’s easy to become #digilost.

Compounding this problem – from an implementation perspective, technology is often delivered by different teams, many of which may not even be part of the same company. Therefore, ensuring that technology is free from bugs and usable needs to be the culture across the project, as opposed to just the domain of the test teams.

How to achieve quality faster through “shift left”

Business teams, developers and testers should unite by working closely together. The waterfall approach of “big-bang” testing (integrating everything at once and then test) has its drawbacks. For instance, the methodology is ill-prepared to handle the rapid iterations that functional and performance testing demands. Testing should happen early in the software development life-cycle as part of the “shift left” approach.

“Shift left” is an approach that strives to build quality into the software development process by incorporating testing early and often. The benefits include:

• The ability to integrate teams, encouraging them to work closely and share valuable insights. For example, testers can be moved directly into scrum teams to work on stories together with developers.
• Design tests in parallel with developers allowing greater, more in-depth testing scripts.
• Gives developers and testers the flexibility to help each other and reduce bottlenecks.
• Early detection of bugs, shortening the critical metric of time between creating a bug and resolving it.
• Increased transparency of code dependency issues. In this instance, issues are raised during the nightly (or more often) regression testing.
• Virtual elimination of the test bottleneck that occurs after development is completed.

Testers should focus on reducing testing into small components and functions, testing only the changes made in an application. This helps reduce the feedback time from days to minutes.

Project teams are advised to run a larger number of small tests in parallel. This will enable them to ensure that applications perform as expected before promoting code to the next stage in the software delivery pipeline. This can be supported by new technology and tools.

How do you make testing smarter?

Continuous testing enables continuous delivery. The utopian endpoint being the automation of the entire pipeline, with zero manual interventions during any part of the process. Such an approach sees an evaluation of quality at every stage of the continuous delivery process, starting right from the requirements-gathering phase (“shift left”) and continuing after an upgrade or piece of code has been released. It also leverages artificial intelligence (AI) and machine learning elements to support automation – enabling the software to work smarter and free up the development team’s workstream.

With significant portions of the pipeline automated (for example regression testing or test execution), there remain areas where manual intervention is still required. These gaps in automation are mostly seen in the following:

• Moving from development to test environments
• Provisioning/configuration of required elements (test data, test environments, etc.)
• Integrating a feedback loop into the testing cycle after the application has been released

What should be addressed when implementing a test culture?

Some of the biggest testing challenges relate to poor test requirements, an inadequate understanding of the testable criteria and/or a lack of appropriate test coverage. Often, approaches adopted for requirements gathering, analysis and engineering are not automated. In some circumstances, they still use analogue elements such as paper, whiteboards or notes. For most firms, neither requirement coverage plans, nor the automatic generation of test cases are the norm today.

Recommendations and trends:

• Model-based testing (MBT) and AI solutions for better requirement management. Designing test cases with appropriate requirement coverage.
• Leveraging smart automation technologies for test design.
  • Models can be used to represent the desired behaviour of a system under test or to represent testing strategies and test environments. It also provides for automatic generation of test artefacts (including test cases and test automation scripts).
  • Greater use of AI to get the appropriate coverage, a reduction in testing time, as well as a greater ability to automatically generate tests in response to changing requirements.
  • Project teams are specifying requirements and/or stories through dynamic visual models instead of text-based ones

To really achieve continuous high performing tests, certain prerequisites are required. Enterprises need to have an infrastructure that supports their test data, the ability to “stand up” environments quickly with the right data and the ability to auto-generate and maintain test cases and integration of the automation test suites so that the test can be run. In addition, automation needs to happen at the application programme interface (API) level and Agile teams need to be involved with testing.

Some of the challenges – culture, mindset, skill sets – still need to be tackled by organisations before continuous testing can be fully achieved. At the same time, several changes could easily be achieved and addressed immediately, including model-based testing, test-data automation and test environment virtualisation.

Agile, DevOps and automation are key elements of IT nowadays. Continuous testing is needed and helpful, speeding up implementation and making the software more robust. Orbium’s propriety software – TestMaster is proving to be an incredibly flexible suite that significantly increases productivity by proving faster execution times, wider test coverage and an integrated feedback mechanism. The result is more robust software implemented faster and at a lower cost.