Seattle Market Administrator
October 2, 2000

Proprietary handler verification reports are issued monthly. The Seattle MA laboratory strives to complete all reports by the 8^th of each month, provided handlers have relayed their corresponding test data as early as possible. Immediate copies of reports are available via e-mail or fax. Color copies are automatically sent out via US mail.

Test Summary table (p. 1)

This table lists producer means for fat, protein, and other solids based on identical samples run by MA and Daisy’s Dairy throughout July, 2000. Proprietary handler reports are intended to include 100% of producers. Producer means can be based on anywhere from 4-32 samples, depending on the producer’s bulk tank and pickup situation. Producer means do not incorporate tank weighting – they are simple averages of pure infrared results. Therefore, the entries on p. 1 are not equivalent to true pay test values.

One producer (Nancy, #33) is excluded from the July comparison due to insufficient sampling. One fat comparison (John, #14) and one protein comparison (Jerry, #26) are automatically flagged due to their unusual discord; these measurements are excluded from the final averages for the plant.

Fat and other solids are within tolerance, and producers can be paid according to the handler’s pay tests computed for the entire month (note: pay test values differ from the handler values listed on p.1 due to inclusion of many more measurements as well as weight averaging of all producers). We’ll consider the producers flagged with H and L later (see Appendix).

Graphical Comparisons (p. 2)

Upon entry of IR data into p.1, a visual presentation is generated instantly via Excel. The fat and protein "flyers" described earlier are now readily seen in relation to the core data. MA and handler IR results agree well for fat and imply similar calibration slopes and biases. Agreement in other solids is also good, despite an apparent mismatch in slope and bias (expect to consistently see the weakest agreement for this component because it spans a very limited range and incorporates uncertainties via difference calculations).

The handler’s protein mean falls out-of-tolerance-low due to disagreement in both slope and bias. With the exception of Jerry (producer #26), however, the relationship between handler and MA results is very consistent.

For a handler with only a few producers, clean regressions may not be possible if producer means are employed (too few points). For this reason, for handlers with fewer than seven producers, all sample-to-sample comparisons are shown on p. 1 (table) and regressed on p. 2 (plots).

Historical Comparison (p.3)

From the table above and plots below on p.3, it can be deduced that Daisy’s laboratory performance for butterfat over the past 12 months has been exemplary (92% success rate). MA and handler overall fat means track very closely with no evidence of consistent bias. The average difference between labs over 12 months is very slightly positive (handler high).

Protein agreement has not been as tight (67% success rate) and there is some evidence of a repetitive bias (handler low) over the latest three months.

Agreement in other solids is fair (75% success rate) and should be better given the implementation in 1999 of a uniform, region-wide, raw milk calibration system.

Final Pay Test Values (p. 4) –

 Supplement to be provided in out-of-tolerance situations only

From p. 2, we saw that the handler and MA protein data correlate very well despite the fact that the means differ by more than the tolerance. In principle, this means the original handler data can be adjusted to a "best fit" about the 1:1 diagonal, thus bringing MA and handler data into excellent agreement. This is done by rotating the handler data by a slope factor and shifting it by a bias – both adjustment factors can be backed out of the protein regression on p.2 (minus flyers). This same technique can be applied to the handler’s weighted average protein pay test values for the entire month.

In an out-of-tolerance situation, these weighted average pay test values are listed on the left side of p. 4 under Original Pay Test. The transformed values appear on the right side under Adjusted Pay Test. Note that the difference between the adjusted mean and original protein pay test mean is 0.046 – nearly the exact difference found on p.1 (0.047) based on head-to-head comparison of IR results for random subsamples.

A key underlying premise in this whole approach is that handler data is basically trustworthy. Testing by handlers generally captures a level of detail that random MA sampling cannot. So long as the graphs on p.2 imply a consistent relationship between handler and MA on a reliable subset of samples, handler data (or more important, handler pay tests) can be "corrected" whenever necessary.

 

If a handler determination for a producer differs from the MA determination by more than four times the respective tolerance, that producer is flagged with an H or L. This "outlier" designation does not imply fault – it merely alerts the MA staff as to a possible problem. This report mechanism is designed to safeguard individual producers – report conclusions based on group means do not always accomplish this.

Outliers can arise from many origins. One of the commonest causes is simple number crunching or transcription error on the part of the handler or the MA. Another possibility is sampling inconsistency. In either case, an H or L could trigger a deeper look at the producer in question to resolve these types of problems.

An outlier can also be caused analytically. For example, take the case where a handler’s fat slope is way off, but nearly all producers lie near the part of the plot where the data regression intersects the 1:1 diagonal (see p. 2). The handler’s overall mean will be indistinguishable from the MA mean. But suppose there is one producer with very low fat and another with very high-fat milk. Because the handler’s IR was incorrectly calibrated, the low-fat producer will be severely overpaid and the high-fat producer underpaid (or vice-versa, depending on the fat slope). The Seattle verification system will flag these two producers and if necessary, report their individual adjusted pay tests directly on the cover letter accompanying the month-end report. Thus, handlers may need to adjust payment to one or two individual farmers, even when their overall test program was within tolerance for the month. Generally, however, action on individual outliers is rare.