Unlike a continuous measurement value, which cannot be accurate (on average), any lack of precision in an attribute measurement system inevitably leads to accuracy problems. If the error coder is not clear or undecided on how to encode a defect, different codes are assigned to several defects of the same type, making the database imprecise. In fact, the vagueness of an attribute measurement system is an important factor in inaccuracies. However, a bug tracking system is not an ongoing payment. The assigned values are correct or not; There is no (or should not) grey area. If codes, locations and degrees of gravity are defined effectively, there is only one attribute for each of these categories for a particular error. Analytically, this technique is a wonderful idea. But in practice, the technique can be difficult to execute judiciously. First, there is always the question of sample size. For attribute data, relatively large samples are required to be able to calculate percentages with relatively low confidence intervals. If an expert looks at 50 different error scenarios – twice – and the match rate is 96 percent (48 votes vs.
50), the 95 percent confidence interval ranges from 86.29% to 99.51 percent. It is a fairly large margin of error, especially in terms of the challenge of choosing the scenarios, checking them in depth, making sure the value of the master is assigned, and then convincing the examiner to do the job – twice. If the number of scenarios is increased to 100, the 95 per cent confidence interval for a 96 per cent match rate will be reduced to a range of 90.1 to 98.9 per cent (Figure 2). As performing an attribute analysis can be tedious, costly and generally uncomfortable for all stakeholders (the analysis is simple versus execution), it is best to take a moment to really understand what should be done and why. First, the analyst should determine that there is indeed attribute data. One can assume that the assignment of a code – that is, the division of a code into a category – is a decision that characterizes the error with an attribute. Either a category is correctly assigned to an error, or it is not. Similarly, the appropriate source location is either attributed to the defect or not. These are « yes » or « no » and « correct allocation » or « wrong allocation » answers.
This part is pretty simple. In addition to the sample size problem, logistics can ensure that listeners do not remember the original attribute they attributed to a scenario when they see it for the second time, also a challenge.