Accred Qual Assur (2012) 17:371–374 DOI 10.1007/s00769-012-0892-4
PRACTITIONER’S REPORT
Accrediting PT/EQA providers to ISO/IEC 17043 Christian Lehmann
Received: 24 January 2012 / Accepted: 20 March 2012 / Published online: 8 April 2012 Springer-Verlag 2012
Abstract After many years and having several different attempts for the accreditation of proficiency testing provider (PT provider), there is finally one stand-alone standard defining the requirements for the competence of PT providers and therefore an internationally harmonised basis for the accreditation of proficiency test providers. Since February 2010, the ISO/IEC 17043:2010 has replaced ISO/IEC Guide 43:1997 and ILAC G 13:2007. The philosophy of the standard about subcontracting work is different to this of the standards mostly used for accreditation like ISO/IEC 17025:2005 or ISO/IEC 17020:2004, etc. Besides the planning of the proficiency tests (PT), the performance evaluation and the authorisation of the PT reports the ISO/IEC 17043:2010 allows subcontracting for the rest of the work when providing PTs. This is a challenge for the assessors to judge about the competence of a PT provider. In numerous paragraphs, the standard sets very detailed requirements. Nevertheless, there is room for interpretation. For these cases, for example, contracts for subcontractors, procedure for the advisory board, minimum requirements for PT certificates, etc., some proposals are given to enable harmonised approach for the assessment of PT providers. Keywords Accreditation Proficiency testing provider ISO/IEC 17043 Requirements
Presented at the Eurachem PT Workshop, October 2011, Istanbul, Turkey. C. Lehmann (&) Deutsche Akkreditierungsstelle GmbH, Spittelmarkt 10, 10117 Berlin, Germany e-mail:
[email protected]
Introduction Quality assurance tools as the use of reference materials and the participation in proficiency testing (PT) play an important role for the quality management in laboratories. Therefore, it is essential for the laboratories to use highquality products. This suggests itself to guarantee the quality of these tools by accreditation of proficiency testing provider (PT provider) and reference material producers, respectively. In 1997, the Australian accreditation body NATA (National Association of Testing Authorities, Australia) granted the first recognitions of PT providers using ISO/ IEC Guide 43-1:1997 [1] as a basis. Since then, more and more accreditation bodies worldwide started to offer the accreditation of PT providers. The first accreditation in this field in Germany was granted in 2003. The accreditation bodies not only used ISO/IEC Guide 43-1:1997 [1] but also ILAC G 13:2000 [2], which is a combination of ISO/IEC Guide 43-1:1997 [1] with parts of ISO/IEC 17025:2000 [3]. These guidance documents were used stand-alone or in combination with ISO/IEC 17025 [3, 7] or ISO/IEC 17020:2004 [8], respectively. In 2007, PT providers were accredited according to more than five different approaches worldwide. This was not a useful basis for an international recognition of the accreditation of PT provider. In 2005, ILAC (International Laboratory Accreditation Cooperation) initiated the revision of the ISO/IEC Guide 43:1997 [4] and its conversion into a 17000 series standard, which could be used for the accreditation of PT providers in all fields and serve as a basis for international harmonisation and recognition in this field. ILAC G 13:2007 [5] was used as the basis document for this revision. In addition, the general assembly of ILAC resolves (resolution GA 9.13) that in the meantime the ISO/IEC Guide 43:1997 [4] and
123
372
Accred Qual Assur (2012) 17:371–374
ILAC G 13 [2, 5] are the common basis for the accreditation in this field. Since February 2010, the ISO/IEC 17043:2010 [6], which was accepted as an EN standard also, has replaced ISO/IEC Guide 43:1997 [4] and ILAC G 13:2007 [5]. A 2-year transition period has been determined by ILAC, that is, by February 2012, all accreditations of PT providers shall be based on ISO/IEC 17043:2010 [6] to be accepted. Today in Germany, ten PT providers are accredited. They provide PTs in the fields of mechanical testing, food analysis, olfactometry, medical testing and environmental testing. All of them are assessed according to ISO/IEC 17043:2010 [6]. The standard is mostly very detailed but in some fields there is room for interpretation. To ensure harmonisation of the assessments and provide help for assessors as well as PT providers, some additional guidance is necessary. For this purpose, the German accreditation body Deutsche Akkreditierungsstelle GmbH (DAkkS) has produced a guidance document [9].
Scope of the standard
• • • • •
Statistical evaluation Interpretation of the results Performance evaluation Reporting (authorisation of the report) Communication.
In reality, there are various kinds of PT providers differing in the work which they are performing themselves and which they are subcontracting. The standard takes this into account and allows subcontracting to a certain degree. Only for the planning, the performance evaluation and the authorisation of the report subcontracting is not permitted. This is to avoid that retailers of proficiency tests can be accredited. In some cases, this means for the assessment team that it has to evaluate the competence of the PT provider on these three tasks. In addition, the assessment team faces the challenge to differentiate between the competence of the subcontractors and the PT provider. The situation gets more difficult if an advisory board is installed for additional technical competence. Finally, the assessors have to decide who has the competence in the fields the PT provider wants to be accredited for.
The standard sets out general requirements for the competence of PT providers who are providing PTs to determine the competence of laboratories performing testing and calibration as well as inspection bodies. Interlaboratory comparison provided for method validation or certification of reference materials are not covered by the standard. The standard is bound to cover all fields of testing and calibration as well as inspection and should serve as a general basis. For specific fields, it acknowledges the use of specific standards, which should be used in addition. This should avoid the application of sector-specific standards for the accreditation of PT providers.
Planning
Competence of a PT provider
Performance evaluation
Compared with laboratories, which are quite similar in their organisation, PT providers can be very different in their organisation, for example, subcontracting of parts of the work is done on a regular basis. The procedure providing a PT includes the following tasks:
For performance evaluation of laboratories in the field of testing, the z-score or various derivates are commonly used in PTs. Aside from the assigned value, the denominator or so called standard deviation of proficiency assessment (SDPA) has a crucial impact on the assessment of the performance of the participants. The bigger this figure, the easier it is for the individual participant to have a satisfactory performance. Hence, it makes sense to discuss this in detail to find out something about the competence of the PT provider. The adequacy of the SDPA, the reasons for the choice and the relation to homogeneity and uncertainty are indicators to estimate the competence.
• • • • • •
Planning Selection of the material including sample preparation, homogeneity and stability testing Invitation of the participants Statistical layout Dispatch of the sample Collection of the results
123
In clause 4.4.1, the standard requires 21 items to be recognised and documented when planning a PT. The practice shows that the plan is sometimes seen as a formality. But it is not. Discussing the PT plan of a newly developed PT is a good opportunity to assess the background and the experience of the personnel responsible for a specific PT. Hence, the DAkkS requires for initial assessments for each field that should be accredited at least one PT plan. In surveillance audits, PT plans of newly developed PTs or changed PTs are always assessed.
Accred Qual Assur (2012) 17:371–374
Authorisation of the report The authorisation of the report is more a formality. It is important to state who takes the responsibility for the PT. It does not give opportunities as good as the other subjects mentioned to find out whether a PT provider is competent or not.
373
have been validated and are maintained. If the test or calibration methods are not accredited, the assessors have to audit these methods during the onsite assessment. For inhouse methods, a full validation study is required. For standard test methods, the PT provider has to prove that the performance criteria of the standard are obtained. Advisory group
Subcontracting tasks If PT providers subcontract tasks, they must ensure that the subcontractor is competent. For testing and calibration, this is easy if the subcontractor itself is accredited for the tasks in question. If it is not, the PT provider has to conduct an audit to prove that the subcontractor fulfils the relevant requirements. The records have to be detailed to give the assessor an impression of the quality of the audit and the work of the subcontractor. In addition, the PT provider has to prove that the relevant clauses of the ISO/IEC 17043:2010 [6] are met. The handling of the subcontracting gives the assessors information about the competence of the PT provider.
Harmonisation of the assessment Each accreditation body faces the situation that it has to harmonise the assessment practice of its assessors. The ISO/IEC 17043:2010 [6] is compared with other standards very precisely in a lot of clauses and sometimes the audit seems to be a tick off task. But, for other clauses, some additional information should be given to avoid a different treatment of the PT providers. Personnel For the competence of the technical management, a university study and 3-year work experience in the field is required by DAkkS. This includes experience with quality management systems. In certain cases, the competence must be proven as equivalent. For external personnel, the minimum content of the contracts was defined. It should include a job description, the time frame, regulation for confidentiality, impartiality and conflicts of interest, regulations for training and embedding in the management system, treatment of personal data and information about the termination of the contract. Equipment The standard requires in clause 4.3.6 that the test methods and equipment used for homogeneity and stability testing
The standard allows the PT provider to have an expert group or technical experts to gain additional technical expertise. In addition to the requirements in clause 4.4.1.4 of the ISO/IEC 17043:2010 [6], a documented procedure is required, which describes the work of group. This should include a description of the tasks, selection of members and the frequency of the meetings. The meeting minutes shall contain the decision making process. In addition, there is a list of members and other experts required with the technical expertise of the persons and their field of activity for the PT provider. It must be traceable who is deciding. Data analysis and records The standard requires in clause 4.7.1.1 that the software must be validated before it is used in routine work. For commonly used commercially available software, it is precluded that the software is validated. Hence, it is only necessary to perform an entrance check with a data set. For self-made software like Excel sheets, it has to be proven that all functions of the software are working properly, for example, with various kinds of data sets. These checks must be documented. Certificates of successful participation In the clause communication with the participant (clause 4.9.5), it is described that the statements of participation and performance of the participants given by the PT provider must not be misleading. In Germany, it is common practice to provide in addition to the PT report certificates of participation in the PT with a statement regarding the performance of the participant. Sometimes these certificates are even required in tenders. Unfortunately, there is no further information in this standard about the minimum content of these certificates like ISO/IEC 17025:2005 [7] has for test reports. To ensure a harmonised assessment, it was necessary to set the following requirements for the minimum content of certificates. • • • •
Name and address of the PT provider Subject and parameters to be analysed Identification of the participant and code number Number or other unique identification of the PT
123
374
• • • • • •
Accred Qual Assur (2012) 17:371–374
Reference to the final report, if relevant Number of pages Name function and signature of the responsible person Date of the PT Results of the participant Performance criteria
If there is a statement on the certificate like ‘‘successful participation’’ information must be given on the criteria this statement is based on. For example, ‘‘for successful participation the laboratory must have more than 90 % of the analysed parameters with a z-score smaller than 3’’.
These additional requirements are based on the experiences derived from various assessments in Germany, exchange of experience with PT providers as well as assessors and discussions in national and international expert groups. It is bound to harmonise the assessment practice in Germany. These additional requirements may be different for each accreditation body based on its experience. It should be seen as a help for accreditation bodies and PT providers preparing for accreditation. The requirements will change with gaining more experience with the application of the new standard ISO/IEC 17043:2010 [6].
Review of requests, tenders and contracts This clause of the ISO/IEC 17043:2010 [6] is not often relevant for most of the routinely organised PTs. Two different scenarios have to be differentiated. Usually, the participants register in an established scheme. The conditions are well known and cannot be changed or selected by the participant. The contract comes into force by signing the registration form. In these cases, no contract review, etc. is required. In these cases, the participants must be given the possibility to clarify everything what is important for their participation before registering. For this, the documentation of these contracts are sufficient. If a client orders a special PT that is not organised routinely, for example, for a group of laboratories applying for a tender, a full contract review according to clause 5.4 of the standard is required. Subcontracting services In addition to the remarks given above a contract with the subcontractor is required. In this contract, the following shall be regulated at least: • • • •
• • •
Names and addresses of the contracting parties Precise description of the work and the specifications of the required products and services Confidentiality statement Acceptance of onsite audits by the PT provider, if necessary in presence of representatives of the accreditation body Exclusion of further subcontracting Commitment to the fulfilment of the relevant clauses of the ISO/IEC 17043:2010 [6] Archiving and availability of all relevant raw data and records.
123
Conclusion The application of the ISO/IEC 17043:2010 [6] worldwide will lead to a harmonisation of accreditation of PT providers. The standard is mostly very detailed, but in some fields, there is room for interpretation. In these fields, some additional advice should be provided to the assessors by the accreditation body. The possibility for the PT provider to subcontract nearly everything on regular bases is a challenge for the assessment of the competence. If a PT provider subcontracts a lot of its tasks, the planning of the PTs, the performance evaluation as well as the evaluation of the subcontractors by the PT provider are good indicators for the competence of the PT provider.
References 1. ISO/IEC Guide 43-1 (1997) Proficiency testing by interlaboratory comparisons—part 1: development and operation of proficiency testing schemes 2. ILAC G 13 (2000) ILAC Guidelines for the requirements for the competence of providers of proficiency testing schemes 3. ISO/IEC 17025 (2000) General requirements for the competence of testing and calibration laboratories 4. ISO/IEC Guide 43 (1997) Proficiency testing by interlaboratory comparisons Part 1 and 2 5. ILAC G 13 (2007) ILAC Guidelines for the requirements for the competence of providers of proficiency testing schemes 6. ISO/IEC 17043 (2010) Conformity assessment—general requirements for proficiency testing 7. ISO/IEC 17025 (2005) General requirements for the competence of testing and calibration laboratories 8. ISO/IEC 17020 (2004) General criteria for the operation of various types of bodies performing inspection 9. 71 SD 0 007 (2011) DAkkS Regeln zur Akkreditierung von Anbietern von Eignungspru¨fungen/Ringversuchen Rev. 1.0
