KPMG’s Software Testing Market and Insights Report

This week I spotted that KPMG had published their insights into the Software Testing Market so I decided to look it over.

You can read the report for yourself by registering on KPMG’s website, but I’ve shared a few of my observations below.

Growing Demand for Software Testing in the UK

The report mentions that the UK software testing market has shown rapid growth, outpacing the overall technology sector. The market for software testing grew to £1.1 billion in 2023, with job listings for software testers almost doubling since 2022. This increase is primarily driven by the expansion of digital transformation efforts, especially in areas like cloud migration, artificial intelligence (AI), and machine learning (ML).

Sectors like financial services, media, and telecommunications are the main contributors to this demand. Financial institutions, in particular, rely heavily on robust testing services due to the complexity and security concerns in handling sensitive data.

Key Trends in Software Testing

The report identifies several major trends that are shaping the future of software testing, both globally and in the UK. Understanding and staying on top of these trends is essential for testers who wish to stay ahead in their careers:

  1. AI and ML-Driven Testing: AI and ML are revolutionising software testing by automating test case generation, reducing human errors, and speeding up the testing process. The report claims that these technologies are critical for large-scale operations, allowing companies to detect and resolve issues faster. I agree that there is a place for this but we need to balance the productivity gains with our desire to be more sustainable. I’d argue that we always need to question our use of AI and balance the pros in efficiency against the potential downsides.
  2. Codeless Testing Tools: The report mentions tools like Tricentis TOSCA and Test.ai that are making it easier to perform testing without writing extensive code, saving time and effort. Codeless testing is clearly an area of potential growth. Since the report was written I’ve seen impressive demos from blinq.io and Applitools (Autonomous) demonstrating that these capabilities are almost ready for mainstream adoption. This has the potential to reduce the reliance of software engineering skills in testers, placing test tools in the hands of domain (business) experts rather than pure quality engineers.
  3. DevOps and Agile Testing: The integration of quality assurance within Agile and DevOps teams has become a staple. The report mentions that Tools and frameworks like QAOps, which blend quality assurance with DevOps practices, are critical for automating and streamlining software testing processes.
  4. Security and Cybersecurity Testing: With the rising frequency of cyber threats, the need for robust security testing has never been more critical. As UK companies continue to face cybersecurity breaches, demand for testers with expertise in security testing will increase.
  5. Generative AI: One of the most transformative trends, Generative AI, is expected to automate many repetitive testing tasks, further improving software quality and reducing time to market. I can see huge potential in this, after my recent experience working to produce a documentation “helper” for quality engineers using LLMs to advise on document quality.

Career Takeaways for UK Testers

Given these trends, there are several key areas that UK testers should focus on to advance their careers:

  • Upskill in AI and ML Technologies: With AI and ML becoming central to automated testing, testers should seek out certifications or training in these areas to remain relevant in the job market.
  • Master Codeless Testing Tools: Becoming proficient with tools like Tricentis TOSCA Copilot, Test.ai, Applitools Autonomous, Blinq.io or similar tools has the potential to help testers save time and improve efficiency in their workflows.
  • Security Testing Expertise: Cybersecurity testing is a growing field. Gaining certifications in security testing or familiarising yourself with industry standards will help to position you as an asset to companies facing new and evolving cyber threats.
  • Get Comfortable with DevOps and Agile: Understanding how to integrate testing into DevOps and Agile processes is essential for modern software development environments. Investing time in learning QAOps techniques will help you to work within integrated engineering teams.

Conclusion

Software testing is evolving rapidly, with AI, increased automation and cybersecurity at the forefront. Testers who focus on these emerging areas and stay informed about industry developments will find themselves in a stronger position as the demand for skilled professionals continues to rise.

Figure One – Open AI robot demo

AS I mentioned in my last blog post, I’ve started to do some work using Generative AI. One of the great things about this is that it’s one of those rare areas of IT where people in the “outside world” are actually aware of AI and what it can potentially do.

People are now sending me examples of clever uses of AI and I have to say that this is the most impressive so far….

In the demo a human has a conversation with a humanoid robot called “Figure One” developed by OpenAI.

The human asked Figure One what it could see in the environment, and Figure One described seeing a red apple on a plate, a drying rack with cups and plates, and the human standing nearby. When the human asked for something to eat, Figure One gave them the red apple, explaining it was the only edible item available on the table.

Figure One then asked the human where the dirty dishes on the table should go next, and the human correctly deduced they should go in the drying rack. When instructed, Figure One picked up the dishes and placed them in the drying rack.

This demonstration is impressive for a few key reasons:

  1. Environmental awareness: The robot showed it could visually perceive and describe the objects and layout of its environment, including identifying edible items versus dishes etc. This spatial awareness and object recognition is a fundamental requirement for any AI system operating in the real world.
  2. Task reasoning beyond simple perception: The robot demonstrated basic reasoning about the appropriate actions to take given the context. It inferred that the human wanted food, so it provided the apple. It also correctly deduced that used dishes belong in the drying rack after the human prompted it.
  3. Instruction following: The robot could understand and follow the human’s spoken instructions, like “put them in the drying rack.” This ability to comprehend natural language commands and translate them into physical actions can be difficult for AI systems.
  4. Dexterous manipulation: To successfully pick up the dishes and place them in the rack required visual/mechanical coordination and dexterity.

While this is a simple scenario, Figure One exhibited several core competencies that AI and robotics researchers have been working towards for decades – perception, reasoning, language understanding, and dexterity. Integrating all these capabilities into one system that can interact in an environment is an impressive feat of artificial intelligence. It suggests progress towards more capable household/service robots that can understand their surroundings and assist with tasks based on dialogue.

My first “Hackathon” and AI

It’s some time since I added a blog post to this site. Since my last article (3 years ago), my role has changed and I’ve been less of a hands-on tester than before. I’m still involved in helping Software, DevOps and Quality Engineers, evaluating new products and trying to find ways to make their lives easier investigating tool integrations and more recently AI.

AI seems to be on everybody’s lips so I was keen to separate the myth from reality when I took part in a hackathon hosted by Google with a team from Lloyds Banking Group’s Quality Engineering Centre of Excellence (QECoE). The hackathon focused on exploring ways that generative AI could be applied to help in the workplace.

Our team at the Hackathon

Our team’s challenge was to develop an application that could enhance quality assurance processes by assessing and providing feedback on testing documents compared to “golden standards”. We created separate golden standards for different types of test plans like accessibility, functional, and performance testing. The application we built used Python to make API calls to Google’s Gemini Pro large language model, with a Streamlit web frontend. When a user uploads a document, it is compared with the relevant golden standard checklist and feedback is provided.

Since many of us weren’t experienced coders, we used AI assistants like ChatGPT and Claude for help with coding the application. Their code completion capabilities were amazing! After the hackathon, we added chat functionality so the app could interactively guide users on improving their documents based on feedback.

Our innovative solution allowing live assessment of document quality against configurable golden standards won in the “documentation” use case category. Presenting it live to the judges was a great experience.

You can read more detail here:
LinkedIn: Producing “Gold Standard” Documentation using AI