Accred Qual Assur (2006) 11: 388–390 DOI 10.1007/s00769-006-0105-0

Anthony J. Russell ¨ Ulf Ornemark Lena H˚ard Gunnar Nordin

Received: 5 October 2005 Accepted: 14 February 2006 Published online: 25 March 2006 C Springer-Verlag 2006
Presented at 5th Eurachem Workshop on Proficiency testing in analytical chemistry, microbiology and laboratory medicine, Portoroˇz (Slovenia) 25–27 Sept 2005 A. J. Russell (
) NATA, 7 Leeds Street, Rhodes, NSW 2138, Australia, e-mail: [email protected] Tel.: +61-2-9736-8222 Fax: +61-2-9743-5311 ¨ U. Ornemark · L. H˚ard · G. Nordin EQUALIS, Box 977, 751 09 Uppsala, Sweden

GENERAL PAPER

Accreditation of PT providers – is it worth the money?

Abstract This topic is being presented from two perspectives, namely the views of an accreditation body and those of an accredited provider of external quality assessment schemes. The first perspective, from an accreditation body, is based on the experience of Australia’s national laboratory accreditation body (NATA), which has accredited fourteen proficiency testing (PT) providers and is currently processing an application from another. The second perspective is based on the experience of EQUALIS, the national quality assurance organisation for laboratory

An accreditation body’s view Background The need for accredited proficiency testing (PT) providers in Australia arose from two main sources. Firstly, there was a growing demand from PT providers during the 1990s for access to a credible, independent process to assess and recognise their technical competence. Secondly, in its laboratory accreditation activities, the national accreditation body (and Australian laboratories) needed additional access to competent proficiency testing to support the laboratory accreditation system. The scopes of 12 of the 15 NATA accredited PT providers cover various clinical/medical PT programs. The accredited scopes of the other three providers include PT programs for residues in waters, residues in soils, residues in foods and agricultural materials, forensic science tests (controlled substances), tests on electrical appliances and accessories, and biological tests on veterinary products, pharmaceuticals, cosmetics, foods and waters. Fourteen of the current accreditations are based in Australia and one in Thailand.

medicine in Sweden, which was granted accreditation in 2002. Keywords Proficiency testing . External quality assessment . Accreditation . Value for money

All of the programs are open to laboratory participants on an international basis. It is natural that an accreditation body would answer “Yes” to the question of whether accreditation of PT providers is worth the money; otherwise, they would not offer the service. But the question needs further analysis. Is it worth the money to different groups of stakeholders? Stakeholders of PT From NATA’s perspective, there are at least three interested parties in this question, namely the accredited PT providers, the laboratories using the PT providers and the laboratory accreditation bodies that may use the PT programs to assist their laboratory accreditation decisions and ongoing surveillance of accredited facilities. However, Fig. 1 illustrates the full range of stakeholders for competent PT services. For accreditation bodies (and laboratories) the benefits of using accredited PT providers are obvious and are of additional value to the use of other PT providers where

389

Accreditation Bodies Laboratories Laboratories

PT Providers Reference Reference ference Material Material Producers Producers

Professional Bodies

of PT in an international marketplace would mean that, in the long term, it will be a necessity for the competence of providers to be demonstrated through processes such as accreditation.

Standards Standards Bodies Bodies

A provider’s view Direct Direct Customers Customers (Test (Test Users) Users)

Indirect Indirect Customers Customers (e.g. (e.g. Regulators) Regulators)

Fig. 1 Stakeholders of proficiency testing

there is no independent assessment of their competence to operate specific PT programs. This reduces the risk of poor decisions based on PT data. Providers’ feedback It would not be appropriate for an accreditation body to answer the question on value for money on behalf of accredited PT providers, unless their views were sought. Accordingly, in preparing for this topic, representatives of all of the currently NATA accredited PT providers were interviewed by telephone or email. The results of this informal survey were, on balance, very positive, and the respondents identified a number of specific indicators of value for money. Amongst others, they included the following benefits (values) of the accreditation process: – Increased confidence of their own clients (participating laboratories); – Improvements in the technical aspects of their PT operation (such as in homogeneity and stability testing); – Opportunities to benefit from interaction with technical experts (their professional peers); – Opportunities (in some cases) to have their own staff participate in assessments of other PT providers; – Opportunities to establish networking with other PT providers through the awareness created by a growing number of accredited PT providers; – Opportunities to market their programs domestically and internationally; – Peace of mind of senior management that independent experts had judged their PT scheme’s competent and compliant with international codes of practice, e.g. ILAC-G13:2000 [1] and ISO/IEC Guide 43-1 [1]; – Provision of additional credibility for certified values for reference materials based on use of accredited PT providers (and use of competent laboratories identified through such PT). In particular, the observation of one of the accredited PT providers is of particular significance, stating that it was their belief that the growing awareness of the importance

Background EQUALIS (External Quality Assurance in Laboratory Medicine in Sweden) has provided regular EQA services since 1992. Initially this was done on project basis with support from the National Board of Health and Welfare, but in 1996 a non-profit limited company was set up. During 2005, EQUALIS manages 83 schemes and collaborates on some sixty other schemes with providers in Nordic and European countries. The number of EQA schemes grew rapidly the first years, so the need for a quality management (QM) system became more obvious. Since EQUALIS was operating from its own premises, outside the medical laboratory atmosphere, there was no existing system to rely on. In February 1998, the work to implement a QM system based on requirements in ILAC-G13 [1] and ISO Guide 43-1 [2] started. The Swedish accreditation body (SWEDAC) assessed the progress of the work in 2000 and 2001 and granted accreditation in May 2002. At present, 53 EQA schemes in clinical chemistry, microbiology, immunology and transfusion medicine are accredited. Part of the accreditation has a flexible scope, thereby simplifying changes to EQA schemes, e.g. replacement of test materials and changes of measurands, in between assessments. A newly developed information system for evaluation of participants’ results is also included in the scope of the accreditation. EQUALIS’ objective is to accredit all its schemes. In reality, however, this is neither relevant nor possible. Some schemes may not meet all metrological requirements, e.g. concerning test material properties, reference measurement procedures, and/or measurand specifications [1]. Other schemes have a small and/or insecure basis for participation or are offered as one-off exercises. Experience from assessments and audits After that the accreditation was granted, SWEDAC has performed annual assessments with the help of technical assessors from the laboratory medicine profession and/or other providers of PT/EQA schemes. The assessments have been a positive experience and have led to further developments of the QM system. The documented procedures in the QM system are based on a handful of templates. There are also written agreements with all sub-contracted suppliers of test materials, and EQUALIS performs occasional audits at selected suppliers. EQUALIS’ requirements for documented information about the test samples’ homo-

390

geneity, stability and composition, have increased the quality of the EQA services. Regular staff meetings constitute opportunities to discuss problems and to report on client feedback and non-conforming activities. Based on ILACG13 and the C-AQ IFCC guidelines [3] a Swedish document was produced as support to assessors and PT/EQA providers [4]. Meeting new requirements The transfer of administrative procedures and results reporting, from fax and normal mail to Internet, means that more emphasis is put on improving the practice for information security management. At the beginning of 2003, SWEDAC announced increased requirements in this field. To comply with the accreditation standard (SS-EN ISO/IEC 17025 and/or SS-EN ISO 15189), medical laboratories (and EQUALIS) must perform a documented review of all information management activities. Guidelines from SWEDAC [5] and from ISO [6] are used to implement the new requirements. EQUALIS also provides a few calibration services, e.g. for prothrombin time in the coagulation area [7]. The calibrators and control materials are by definition in vitro diagnostic (IVD) medical devices and must comply with EU’s directive 98/79/EC [8]. In meeting the requirements for information security management and for IVD products, the basic document control structure of the QM system has proven suitable. Separate chapters in the quality manual regulate the new areas, but the structure from ILAC-G13 is maintained.

Conclusions Although more than 60 European PT/EQA providers are already accredited by a dozen or more accreditation bodies, the topic is much debated. Two reports, by Eurolab [9] and by Eurachem/EQALM [10], contain feedback from surveys among some 160 providers. Many of them acknowledge the benefits of accreditation for reasons similar as EQUALIS (below). The implementation of a QM system was necessary for EQUALIS to structure the work. The resources currently allocated to maintain the QM system correspond to one full time employee, which is much more than the costs associated with accreditation fees and the assessment team. The total costs correspond to approximately 3% of the turnover. For the following main reasons, EQUALIS can conclude that going through the accreditation process was “worth the money spent”: – Improved general quality of the PT/EQA services, – Improved insight into the properties of the test materials and sub-contractors’ services, – Improved QM system through regular auditing, – Ability to meet new requirements related to information security management and IVD. The implementation and maintenance of a QM system requires resources, which for small, specialised PT/EQA providers can constitute a large burden. The surveys [9, 10], however, show that many PT providers, or their parent organisation, have other accreditations or certifications. Much of the basic work is thus already done.

References 1. ILAC (2000) ILAC-G13, Guidelines for the requirements for the competence of providers of proficiency testing schemes, International Laboratory Accreditation Cooperation, http://www.ilac.org 2. ISO/IEC Guide 43-1 (1997) Proficiency testing by interlaboratory comparisons: Part 1: Development and operation of proficiency testing schemes, International Organization for Standardization, Geneva 3. Guidelines for the requirements for the competence of EQAP organizers in medical laboratories, IFCC, Education and management division, Committee of analytical quality, version 3/2002

4. Riktlinjer f¨or kompetenskrav f¨or arrang¨orer av kompetenspr¨ovningsprogram och program f¨or extern kvalitetsbed¨omning, SWEDAC DOC 04:10 (2nd edn), draft June 2005, www.swedac.se. 5. SWEDAC DOC 04:6, Krav p˚a laboratoriets dataverksamhet och informationsspridning, SWEDAC, Bor˚as, 2005-01-31 6. ISO/IEC 17799 (2000) Information technology—Code of practice for information security management, International Organization for Standardization, Geneva

7. Hillarp A, Egberg N, Nordin G, Stigendal L, Fagerberg I, Lindahl TL (2004) Thromb Haemost 91:300 8. Directive 98/79/EC of the European parliament and of the Council of 27 October 1998 on in vitro diagnostic medical devices 9. Eurolab technical secretariat (2005) Report No. 1 Feb. 2005, http://www.eurolab.org ¨ 10. Ornemark U, Henriksen GM (2005) Eurachem/EQALM Report, October, http://www.eqalm.org