We keep manual QA for non-deterministic, exploratory stuff, things that do not have an expected input and output and where spotting secondary information and insights (e.g. "why is the chrome network console logging so much when I click on this button?"). things that have a deterministic input and expected output get automated. This whole area is a terminology minefield, but one term I like from the context-driven testing community for category 2 (deterministic) is "checking". The term I like for category 1 is "exploring". If a computer can "check" a post condition given some preconditions then a human does not need to waste time on it. If you need up to go after sidequests and look for unknown factors, it requires a human to explore. Having said that, there is a role for automation to speed up exploration (e.g. record a flow through the app for some key scenario, so a human can play it at 10x speed, prefill data to bring up an app into some well known state for exploration etc), but the judgement stays human. If the QA is flooded with tickets for exploration, giving them tooling to speed up data setups, cleanups, reviews etc might help.
I've picked up a framework for deciding what/how much to test manually from James Whittaker's ACC matrix a while ago, and it's been incredibly helpful to create a policy around when something requires manual testing, how much and what the manual testing should actually cover. It's covered reasonably well in the book "How google tests software". Having a policy around that means that there's no misunderstanding what gets automated and by whom, and what's left for exploring and by whom.
I think the work of QAs is going to shift from executing the test cases themselves to instructing agents to do the work and checking if the execution was done properly.
We keep manual QA for non-deterministic, exploratory stuff, things that do not have an expected input and output and where spotting secondary information and insights (e.g. "why is the chrome network console logging so much when I click on this button?"). things that have a deterministic input and expected output get automated. This whole area is a terminology minefield, but one term I like from the context-driven testing community for category 2 (deterministic) is "checking". The term I like for category 1 is "exploring". If a computer can "check" a post condition given some preconditions then a human does not need to waste time on it. If you need up to go after sidequests and look for unknown factors, it requires a human to explore. Having said that, there is a role for automation to speed up exploration (e.g. record a flow through the app for some key scenario, so a human can play it at 10x speed, prefill data to bring up an app into some well known state for exploration etc), but the judgement stays human. If the QA is flooded with tickets for exploration, giving them tooling to speed up data setups, cleanups, reviews etc might help.
I've picked up a framework for deciding what/how much to test manually from James Whittaker's ACC matrix a while ago, and it's been incredibly helpful to create a policy around when something requires manual testing, how much and what the manual testing should actually cover. It's covered reasonably well in the book "How google tests software". Having a policy around that means that there's no misunderstanding what gets automated and by whom, and what's left for exploring and by whom.
I think the work of QAs is going to shift from executing the test cases themselves to instructing agents to do the work and checking if the execution was done properly.
Of course that's already happening. But I think this is going to happen more and more