At some point, most companies face the question of how to assess testers' performance and determine measurable criteria. Should it be based on the quantity of defects discovered, the rejection rate, or the number of tests written and executed? The variety of metrics devised for this purpose depends on the creativity of the test manager, often resulting in a multitude of options. Why is there a universal need for measurements? What are the outcomes of numerically evaluating testers? This article covers everything you need to know.
Why the Issue of Metrics Comes to the Fore#
One fateful day, the test manager or their superior awakens with a realization: "It is imperative to evaluate and measure the performance of testers." What prompts this sudden realization? Several common triggers drive this kind of thinking.
The Feeling of Lacking Control Over the Situation#
Effective management involves addressing crucial questions regarding task allocation, recognition, feedback, and improvement within the realm of testers' work evaluation. Managers seek reliable metrics to assess the quality of testers' performance and gain valuable insights into their contributions. However, implementing metrics can pose challenges. Despite these hurdles, constructive feedback during testing serves as a vital tool for growth and improvement. By measuring testers' performance, managers can gain a comprehensive understanding of the current dynamics within their domain, enabling them to make informed decisions and foster a culture of continuous improvement.
Employees Are Deprived of Adequate Attention#
The evaluation of tester performance plays a pivotal role in providing employees with clarity and guidance concerning their work. Oftentimes, testers may feel uncertain about their performance, lacking a clear sense of direction and understanding of their developmental journey. In response, upper-level managers urge their lower-level counterparts to conduct regular evaluations, utilizing metrics tailored for testers, to gauge their work outcomes accurately. Through this feedback, testers gain valuable insights into their performance, enabling them to identify areas of improvement and fostering their professional growth and development.
Employees Need Measurable Goals to Increase Motivation#
Providing feedback to an employee, especially when suggesting areas for improvement such as "bug better" or "bug more," can be challenging, particularly if the employee believes they are already performing well. However, by measuring the outcomes of their work, any sense of subjective bias can be eliminated, enabling employees to gain precise insights into the specific performance indicators they need to focus on and enhance.
It Is Impossible to Demonstrate the Absence of Issues#
Developers express grievances about defects, project managers voice concerns about meeting deadlines, and customers raise complaints about bugs. In the midst of these challenges, the test manager endeavors to substantiate the efficacy of their department by showcasing the responsibilities shouldered by their team members and the notable accomplishments they have achieved. To accomplish this, it becomes imperative to first measure and quantify the outcomes to provide concrete evidence of the department's contributions and success.
"The true value of a tester's work lies in their comprehensive testing approach, uncovering vulnerabilities, improving user experience, and ensuring utmost quality, fostering trust in the end product."
The Implementation of Metrics Leads to Unintended Outcomes#
Now that we have metrics in place - whether a few key values or comprehensive Key Performance Indicators (KPIs) - what can we typically expect as the standard outcome of implementing such innovations?
Time wastage: Collecting metrics necessitates expanding tool usage, introducing new fields, collecting and processing new data, which inevitably leads to a time-consuming process for both the test manager and employees. Although this time waste is usually insignificant, such as 1-2 hours per week, allocating a portion of working time to tasks that do not directly enhance product quality and test results can evoke a mild sense of demotivation.
Employee demotivation: Motivation theory distinguishes between intrinsic motivators and extrinsic incentives. Initially, employees may be motivated to excel solely for the sake of achieving excellent results, driven by intrinsic motivation. However, the moment external rewards are tied to the outcomes, the intrinsic value of the results diminishes. The focus shifts from delivering high-quality products and benefiting the team to attaining a specific quantity of defects per month in order to secure quarterly bonuses.
The disparity is significant. Complex bugs that require effort to reproduce are avoided due to time constraints, and instead, the emphasis is placed on creating multiple minor defects for higher productivity. Concerns about the percentage of rejected defects may deter employees from tackling controversial bugs. Consequently, duplicates and lower quality defects proliferate, leading to disputes over severity and accusations of biased evaluation. This lack of objectivity gives rise to demotivation and conflicts.
Diminished work quality: Employee demotivation and an excessive focus on numbers rather than genuine results create a scenario where all selected indicators exhibit a 100% increase, yet the overall work quality of the team significantly declines. While feelings may not be as objective as numbers, relying solely on numerical measurements disregards the true essence of work quality.
Advocates of numerical performance measurement often attribute the issue to poorly chosen metrics that fail to accurately reflect the actual outcome. Indeed, if the metrics could genuinely represent the quality of work 100%, this problem would not arise. Unfortunately, in the context of testing, determining such metrics is impractical. Unlike in a factory setting where daily output and defect percentages can be easily quantified, testing does not lend itself to such definitive measurements.
What to Do Instead#
While metrics have no universal place in testing (except, of course, in a factory setting), the challenges highlighted above still require solutions. If not through metrics, then how can we address them?
The answer lies in three core practices:
- Consistent and meaningful feedback - regular, constructive assessments that give employees clear direction
- Timely assessment of tasks - prompt evaluation so testers understand their performance in context
- Genuine communication with employees and colleagues - open dialogue that builds trust and shared purpose
Fostering regular feedback, both constructive and timely, plays a vital role in providing employees with a clear understanding of their prospects, areas for improvement, and expectations. Meaningful face-to-face communication on a monthly basis can contribute significantly to employee satisfaction and development. Moreover, cultivating sincerity in communication with employees and colleagues helps build stronger relationships, while emphasizing shared interests in project outcomes rather than relying solely on numerical data.
Final Thoughts#
While metrics can be a useful tool in certain contexts, they may not always be the most effective or appropriate approach when it comes to evaluating the work of testers. The challenges and potential drawbacks associated with implementing metrics - such as time wastage, employee demotivation, and a diminished focus on genuine results - highlight the need for alternative strategies.
By prioritizing regular feedback, timely assessments, and sincere communication, organizations can create an environment that encourages continuous improvement, motivates employees, and ultimately enhances the quality of work produced.
It is important for managers and organizations to recognize that evaluating tester performance goes beyond numbers and metrics. By embracing a holistic approach that values individual growth, open communication, and a focus on genuine results, organizations can create a more supportive and productive testing environment that benefits both the testers and the overall quality of the software being developed.