검색
검색 팝업 닫기

Ex) Article Title, Author, Keywords

Article

Split Viewer

Letter to the Editor

Progress in Medical Physics 2024; 35(1): 16-19

Published online March 31, 2024 https://doi.org/10.14316/pmp.2024.35.1.16

Copyright © Korean Society of Medical Physics.

Formulation of a New En Score in the Proficiency Test

Chul-Young Yi , In Jung Kim , Jong In Park , Yun Ho Kim , Young Min Seong

Ionizing Radiation Metrology Group, Korea Research Institute of Standards and Science (KRISS), Daejeon, Korea

Correspondence to:Chul-Young Yi
(cyyi@kriss.re.kr)
Tel: 82-42-868-5370
Fax: 82-42-868-5671

Received: December 11, 2023; Revised: February 21, 2024; Accepted: February 21, 2024

This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

A new En score of the proficiency test (PT) is formulated; it is applicable when a correlation exists between the reference and participant’s values. Based on the uncertainty propagation rule given in ISO/IEC Guide 98-3 (GUM:1995), the En score covering the correlation case is newly developed for the PT. The new En score will be applied in a future PT organized by the Korea Research Institute of Standards and Science (KRISS) dosimetry team. The new En score will enhance measurement traceability and contribute to improving the quality management system of participants in the KRISS PT by avoiding performance underestimation.

KeywordsEn score, Proficiency test, Dosimetry audit, Calibration, Traceability

In accordance with KS Q ISO/IEC 17025 [1], testing and calibration laboratories shall verify that the entire measurement process, including facilities, equipment, personnel, and methods, meets the requirements. They shall ensure result validity by establishing a monitoring procedure, which shall in turn be planned and reviewed. Whenever appropriate, laboratories shall monitor their performance by comparing their results with those of other laboratories. This monitoring should be planned and reviewed and should include one or two of the following but not limited to: (a) participation in proficiency test (PT) and (b) participation in interlaboratory comparisons other than PT.

The laboratories accredited by the Korea Laboratory Accreditation Scheme (KOLAS) shall regularly participate in PT during the period set by the Korean Agency for Technology and Standards Notice No. 2022-0047 (KOLAS-R-007) entitled “the PT operation guidelines.” To implement KOLAS-R-007 in the national measurement system, the Korea Research Institute of Standards and Science (KRISS) directly provides PT or sometimes gives the reference value to PT in compliance with the requirements of KS Q ISO/IEC 17043 [2] and organized by the accredited PT provider of KOLAS. The PT results are used to assess the performance of the quality management system (QMS) of the accredited institute by KOLAS, and all corrective actions shall be undertaken when the PT result is assessed as “not acceptable.”

In 2021, the KRISS performed PT over the dosimetry audit service provider for the administered dose in cancer therapy and reported the results in Progress in Medical Physics [3]. In doing so, the KRISS supports the implementation of the Nuclear Safety and Security Commission Notice No. 2019-6 entitled “Technical standards for radiation safety management in the medical field” because the dosimetry audit shall be run on the basis of QMS. Furthermore, the KRISS PT is used to ensure the conformity of QMS. The PT provided by the KRISS contributes to confirming the competency of the dosimetry audit service provider and building the public confidence of the dosimetry audit itself.

PT results must be analyzed and transformed into a performance statistic for interpretation and assessment according to the defined objectives. The purpose of this step is to find the deviation of the result from the reference value to allow a comparison with performance standards. In KS Q ISO/IEC 17043, the En score is defined as one of the following statistical approaches:

En=x  yUx2 + Uy2

where x is the result of the participating laboratory, y is the reference value, Ux is the expanded uncertainty of x, and Uy is the expanded uncertainty of y. The numerator of the above equation is the difference between the participant’s result and the reference value, and the denominator is the expanded uncertainty of the difference. Eq. (1) is valid only when the participant’s x is independent of the reference y and the coverage factors and effective degrees of freedom coincide. The En score for the case of correlation is not explicitly given in KS Q ISO/IEC 17043; however, the statistical approach follows the method described in KS Q ISO 13528 [4] and KS Q ISO/IEC Guide 98-3(GUM) [5].

According to KS Q ISO/IEC Guide 98-3, when a correlation exists between x and y, the new En score given in Eq. (1) becomes:

En=x  ykicxi2uxi2 + jcyj2uyj2 + 2ijcxicyj uxi,yj 

where x is the result or output estimate of a participant obtained from an appropriate function or a measurement model of the input estimates x1, x2, …, xM, i.e., x=f(x1, x2, ..., xM); y is the reference value or output estimate determined from a function of the input estimates y1, y2, ..., yN, i.e., y=g(y1, y2, ..., yN); k is the coverage factor to obtain the expanded uncertainty of (xy); cxi is the sensitivity coefficient of the i-th input estimate xi given by the partial derivative (∂f/∂xi); uxi is the standard uncertainty of xi; cyj is the sensitivity coefficient of jth input estimate yj given by (∂g/∂yj); uyj is the standard uncertainty of yj; and uxi,yj is the estimate of the covariance between the input estimates xi and yj. The coverage factor was determined to correspond to confidence level. Covariances appearing between input estimates xi and xj or between yi and yj shall also be included in Eq. (2) for completeness.

Considering the relationship:

fncxn2uxn2+fncyn2uyn2 2fncxncynuxnuyn

where fn is the correlation coefficient given by uxn,yn/(uxnuyn), we obtain the following conservative estimate of Eq. (2):

En*=x  ykicxi2uxi2 + jcyj2uyj2  nfncxn2uxn2 + nfncyn2uyn2

where the correlation is assumed to be over the input estimates with the subscript n.

The denominator of Eq. (4) is equal to or less than that of Eq. (2). This approach has been accepted for two decades in the uncertainty evaluations of the international dosimetry key comparison results carried out among national metrology institutes (NMIs) in the consultative committee of ionizing radiation, section I (Comité consultatif des rayonnements ionisants section I, CCRI[I]) of the General Conference on Weights and Measures (Conférence générale des poids et mesures, CGPM) [6,7]. In this study, the parameters fk2 and fj2 in references [6,7] correspond to fn. When the condition that cxn=cyn and uxn=uyn is met, Eq. (2) is equal to Eq. (4), i.e., En=En*. When the measurement procedures used in PT are verified and validated according to KS Q ISO/IEC 17025, the related parameters are likely to satisfy the condition above.

If the input estimates are in perfect correlation, then fn equals to 1. Perfect correlation is frequently encountered in many cases of calibration and testing. Eq. (1) underestimates the En score when a correlation exists between x and y. The independent terms are mostly those for the statistical uncertainties involved in the measurements in both laboratories.

When the laboratory value is traceable to other NMIs, it exhibits some systematic difference from the KRISS reference value y by an amount of

d=yDNMIDKRISS

where d is the systematic difference in the laboratory value traceable to another NMI, DNMI is the degree of equivalence (DOE) of another NMI, and DKRISS is the DOE of the KRISS. The DOE given in mGy/Gy is the relative difference between the established standard of NMIs and the key comparison reference value. The DOE is presented with its uncertainty on the KCDB website (https://www.bipm.org/kcdb/). The relative standard uncertainty of d is then given by:

u2dd2=jcyj2uyj2y2+u2DNMI + u2DKRISSDNMIDKRISS 2

where u(d) is the standard uncertainty of d, u(DNMI) is the standard uncertainty of the DOE of another NMI, and u(DKRISS) is the standard uncertainty of the DOE of the KRISS. In Eq. (6), the correlation between the DOEs of another NMI and KRISS is assumed to be minor and thus ignored. When the results of a direct comparison between KRISS and another NMI are available, we need to follow the procedure in references [6,7] to obtain d instead of Eq. (5).

Eqs. (2) and (4) can be rewritten for the case of the laboratory value traceable to other NMIs as follows:

En=x  d  ykicxi2uxi2 + jcyj2uyj2 + u2d + 2ijcxicyj uxi,yj 

and

En*=x  d  ykicxi2uxi2 + jcyj2uyj2 + u2d  nfncxn2uxn2 + nfncyn2uyn2

where we assume (x−d) is correlated with y. Eq. (2) or (4) shall be used in the case where the value of the other NMI is traceable to the KRISS.

The correlation coefficient ranges from 1 to 1 and is zero when no correlation is present. If a correlation is confirmed but the correlation coefficient is not explicitly known, then a correlation coefficient of 0.5 would be a conservative estimate. In technical report series No. 398 [8], kQ,Q0 is given as a chamber-specific factor, and a correlation among values of kQ,Q0 is reasonably accepted. When different values of kQ,Q0 are involved in the KRISS PT, i.e., in the case of the absorbed dose to water from proton and electron beams, we can then use 0.5 for the correlation coefficient in the values of kQ,Q0 as a conservative estimate during the evaluation of the new En score.

In this study, we describe the new En score in PT, which is useful particularly when a correlation exists between the participant’s value and the reference value. The new En score will reduce underestimation. Therefore, it will help enhance the discerning level of the participant’s performance in PT and be applicable in the PT program organized by the KRISS dosimetry team.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

All relevant data are within the paper and its Supporting Information files.

  1. International Organization for Standardization (ISO). Reference number KS Q ISO/IEC 17025. General requirements for the competence of testing and calibration laboratories. ISO. 2017.
  2. International Organization for Standardization (ISO). Reference number KS Q ISO/IEC 17043. Conformity assessment - general requirements for proficiency testing. ISO. 2010.
  3. Yi CY, Kim IJ, Park JI, Kim YH, Seong YM. Proficiency test for the dosimetry audit service provider. Prog Med Phys. 2022;33:72-79.
    CrossRef
  4. International Organization for Standardization (ISO). Reference number KS Q ISO 13528. Statistical methods for use in proficiency testing by interlaboratory comparison. ISO. 2015.
  5. International Organization for Standardization (ISO). ISO/IEC Guide 98-3. Uncertainty of measurement - part 3: guide to the expression of uncertainty in measurement (GUM:1995). ISO. 2008.
  6. Allisy PJ, Burns DT. Consultative Committee for Ionizing Radiation (CCRI) uncertainty workshop document. Workshop(I)/05-07. Bureau International des Poids et Mesures. 2005.
  7. Burns DT, Butler DJ. Updated report on the evaluation of degrees of equivalence in regional dosimetry comparisons. Consultative Committee for Ionizing Radiation (CCRI) meeting document. CCRI(I)/17-09. Bureau International des Poids et Mesures. 2017.
  8. International Atomic Energy Agency (IAEA). IAEA TRS-398 (V.12). Absorbed dose determination in external beam radiotherapy. An international code of practice for dosimetry based on standards of absorbed dose to water. IAEA. 2000.

Article

Letter to the Editor

Progress in Medical Physics 2024; 35(1): 16-19

Published online March 31, 2024 https://doi.org/10.14316/pmp.2024.35.1.16

Copyright © Korean Society of Medical Physics.

Formulation of a New En Score in the Proficiency Test

Chul-Young Yi , In Jung Kim , Jong In Park , Yun Ho Kim , Young Min Seong

Ionizing Radiation Metrology Group, Korea Research Institute of Standards and Science (KRISS), Daejeon, Korea

Correspondence to:Chul-Young Yi
(cyyi@kriss.re.kr)
Tel: 82-42-868-5370
Fax: 82-42-868-5671

Received: December 11, 2023; Revised: February 21, 2024; Accepted: February 21, 2024

This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

A new En score of the proficiency test (PT) is formulated; it is applicable when a correlation exists between the reference and participant’s values. Based on the uncertainty propagation rule given in ISO/IEC Guide 98-3 (GUM:1995), the En score covering the correlation case is newly developed for the PT. The new En score will be applied in a future PT organized by the Korea Research Institute of Standards and Science (KRISS) dosimetry team. The new En score will enhance measurement traceability and contribute to improving the quality management system of participants in the KRISS PT by avoiding performance underestimation.

Keywords: En score, Proficiency test, Dosimetry audit, Calibration, Traceability

Introduction

In accordance with KS Q ISO/IEC 17025 [1], testing and calibration laboratories shall verify that the entire measurement process, including facilities, equipment, personnel, and methods, meets the requirements. They shall ensure result validity by establishing a monitoring procedure, which shall in turn be planned and reviewed. Whenever appropriate, laboratories shall monitor their performance by comparing their results with those of other laboratories. This monitoring should be planned and reviewed and should include one or two of the following but not limited to: (a) participation in proficiency test (PT) and (b) participation in interlaboratory comparisons other than PT.

The laboratories accredited by the Korea Laboratory Accreditation Scheme (KOLAS) shall regularly participate in PT during the period set by the Korean Agency for Technology and Standards Notice No. 2022-0047 (KOLAS-R-007) entitled “the PT operation guidelines.” To implement KOLAS-R-007 in the national measurement system, the Korea Research Institute of Standards and Science (KRISS) directly provides PT or sometimes gives the reference value to PT in compliance with the requirements of KS Q ISO/IEC 17043 [2] and organized by the accredited PT provider of KOLAS. The PT results are used to assess the performance of the quality management system (QMS) of the accredited institute by KOLAS, and all corrective actions shall be undertaken when the PT result is assessed as “not acceptable.”

In 2021, the KRISS performed PT over the dosimetry audit service provider for the administered dose in cancer therapy and reported the results in Progress in Medical Physics [3]. In doing so, the KRISS supports the implementation of the Nuclear Safety and Security Commission Notice No. 2019-6 entitled “Technical standards for radiation safety management in the medical field” because the dosimetry audit shall be run on the basis of QMS. Furthermore, the KRISS PT is used to ensure the conformity of QMS. The PT provided by the KRISS contributes to confirming the competency of the dosimetry audit service provider and building the public confidence of the dosimetry audit itself.

PT results must be analyzed and transformed into a performance statistic for interpretation and assessment according to the defined objectives. The purpose of this step is to find the deviation of the result from the reference value to allow a comparison with performance standards. In KS Q ISO/IEC 17043, the En score is defined as one of the following statistical approaches:

En=x  yUx2 + Uy2

where x is the result of the participating laboratory, y is the reference value, Ux is the expanded uncertainty of x, and Uy is the expanded uncertainty of y. The numerator of the above equation is the difference between the participant’s result and the reference value, and the denominator is the expanded uncertainty of the difference. Eq. (1) is valid only when the participant’s x is independent of the reference y and the coverage factors and effective degrees of freedom coincide. The En score for the case of correlation is not explicitly given in KS Q ISO/IEC 17043; however, the statistical approach follows the method described in KS Q ISO 13528 [4] and KS Q ISO/IEC Guide 98-3(GUM) [5].

According to KS Q ISO/IEC Guide 98-3, when a correlation exists between x and y, the new En score given in Eq. (1) becomes:

En=x  ykicxi2uxi2 + jcyj2uyj2 + 2ijcxicyj uxi,yj 

where x is the result or output estimate of a participant obtained from an appropriate function or a measurement model of the input estimates x1, x2, …, xM, i.e., x=f(x1, x2, ..., xM); y is the reference value or output estimate determined from a function of the input estimates y1, y2, ..., yN, i.e., y=g(y1, y2, ..., yN); k is the coverage factor to obtain the expanded uncertainty of (xy); cxi is the sensitivity coefficient of the i-th input estimate xi given by the partial derivative (∂f/∂xi); uxi is the standard uncertainty of xi; cyj is the sensitivity coefficient of jth input estimate yj given by (∂g/∂yj); uyj is the standard uncertainty of yj; and uxi,yj is the estimate of the covariance between the input estimates xi and yj. The coverage factor was determined to correspond to confidence level. Covariances appearing between input estimates xi and xj or between yi and yj shall also be included in Eq. (2) for completeness.

Considering the relationship:

fncxn2uxn2+fncyn2uyn2 2fncxncynuxnuyn

where fn is the correlation coefficient given by uxn,yn/(uxnuyn), we obtain the following conservative estimate of Eq. (2):

En*=x  ykicxi2uxi2 + jcyj2uyj2  nfncxn2uxn2 + nfncyn2uyn2

where the correlation is assumed to be over the input estimates with the subscript n.

The denominator of Eq. (4) is equal to or less than that of Eq. (2). This approach has been accepted for two decades in the uncertainty evaluations of the international dosimetry key comparison results carried out among national metrology institutes (NMIs) in the consultative committee of ionizing radiation, section I (Comité consultatif des rayonnements ionisants section I, CCRI[I]) of the General Conference on Weights and Measures (Conférence générale des poids et mesures, CGPM) [6,7]. In this study, the parameters fk2 and fj2 in references [6,7] correspond to fn. When the condition that cxn=cyn and uxn=uyn is met, Eq. (2) is equal to Eq. (4), i.e., En=En*. When the measurement procedures used in PT are verified and validated according to KS Q ISO/IEC 17025, the related parameters are likely to satisfy the condition above.

If the input estimates are in perfect correlation, then fn equals to 1. Perfect correlation is frequently encountered in many cases of calibration and testing. Eq. (1) underestimates the En score when a correlation exists between x and y. The independent terms are mostly those for the statistical uncertainties involved in the measurements in both laboratories.

When the laboratory value is traceable to other NMIs, it exhibits some systematic difference from the KRISS reference value y by an amount of

d=yDNMIDKRISS

where d is the systematic difference in the laboratory value traceable to another NMI, DNMI is the degree of equivalence (DOE) of another NMI, and DKRISS is the DOE of the KRISS. The DOE given in mGy/Gy is the relative difference between the established standard of NMIs and the key comparison reference value. The DOE is presented with its uncertainty on the KCDB website (https://www.bipm.org/kcdb/). The relative standard uncertainty of d is then given by:

u2dd2=jcyj2uyj2y2+u2DNMI + u2DKRISSDNMIDKRISS 2

where u(d) is the standard uncertainty of d, u(DNMI) is the standard uncertainty of the DOE of another NMI, and u(DKRISS) is the standard uncertainty of the DOE of the KRISS. In Eq. (6), the correlation between the DOEs of another NMI and KRISS is assumed to be minor and thus ignored. When the results of a direct comparison between KRISS and another NMI are available, we need to follow the procedure in references [6,7] to obtain d instead of Eq. (5).

Eqs. (2) and (4) can be rewritten for the case of the laboratory value traceable to other NMIs as follows:

En=x  d  ykicxi2uxi2 + jcyj2uyj2 + u2d + 2ijcxicyj uxi,yj 

and

En*=x  d  ykicxi2uxi2 + jcyj2uyj2 + u2d  nfncxn2uxn2 + nfncyn2uyn2

where we assume (x−d) is correlated with y. Eq. (2) or (4) shall be used in the case where the value of the other NMI is traceable to the KRISS.

The correlation coefficient ranges from 1 to 1 and is zero when no correlation is present. If a correlation is confirmed but the correlation coefficient is not explicitly known, then a correlation coefficient of 0.5 would be a conservative estimate. In technical report series No. 398 [8], kQ,Q0 is given as a chamber-specific factor, and a correlation among values of kQ,Q0 is reasonably accepted. When different values of kQ,Q0 are involved in the KRISS PT, i.e., in the case of the absorbed dose to water from proton and electron beams, we can then use 0.5 for the correlation coefficient in the values of kQ,Q0 as a conservative estimate during the evaluation of the new En score.

In this study, we describe the new En score in PT, which is useful particularly when a correlation exists between the participant’s value and the reference value. The new En score will reduce underestimation. Therefore, it will help enhance the discerning level of the participant’s performance in PT and be applicable in the PT program organized by the KRISS dosimetry team.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflicts of Interest

The authors have nothing to disclose.

Availability of Data and Materials

All relevant data are within the paper and its Supporting Information files.

References

  1. International Organization for Standardization (ISO). Reference number KS Q ISO/IEC 17025. General requirements for the competence of testing and calibration laboratories. ISO. 2017.
  2. International Organization for Standardization (ISO). Reference number KS Q ISO/IEC 17043. Conformity assessment - general requirements for proficiency testing. ISO. 2010.
  3. Yi CY, Kim IJ, Park JI, Kim YH, Seong YM. Proficiency test for the dosimetry audit service provider. Prog Med Phys. 2022;33:72-79.
    CrossRef
  4. International Organization for Standardization (ISO). Reference number KS Q ISO 13528. Statistical methods for use in proficiency testing by interlaboratory comparison. ISO. 2015.
  5. International Organization for Standardization (ISO). ISO/IEC Guide 98-3. Uncertainty of measurement - part 3: guide to the expression of uncertainty in measurement (GUM:1995). ISO. 2008.
  6. Allisy PJ, Burns DT. Consultative Committee for Ionizing Radiation (CCRI) uncertainty workshop document. Workshop(I)/05-07. Bureau International des Poids et Mesures. 2005.
  7. Burns DT, Butler DJ. Updated report on the evaluation of degrees of equivalence in regional dosimetry comparisons. Consultative Committee for Ionizing Radiation (CCRI) meeting document. CCRI(I)/17-09. Bureau International des Poids et Mesures. 2017.
  8. International Atomic Energy Agency (IAEA). IAEA TRS-398 (V.12). Absorbed dose determination in external beam radiotherapy. An international code of practice for dosimetry based on standards of absorbed dose to water. IAEA. 2000.
Korean Society of Medical Physics

Vol.35 No.1
March 2024

pISSN 2508-4445
eISSN 2508-4453
Formerly ISSN 1226-5829

Frequency: Quarterly

Current Issue   |   Archives

Stats or Metrics

Share this article on :

  • line