Inviting comments on standard IVD evaluation protocol drafted by ICMR and CDSCO 2

ICMR-CDSCO/IVD/GD/PROTOCOLS/04/2024 Indian Council of Medical Research and Central Drugs Standard Control Organization Department of Health Research and Drugs Controller General of India Ministry of Health and Family Welfare Government of India Document No.: ICMR-CDSCO/IVD/GD/PROTOCOLS/04/2024 Subject: Inviting comments on standard IVD evaluation protocol drafted by ICMR and CDSCO Licensure of In-Vitro Diagnostics (IVDs) under Medical Devices Rules 2017 requires a detailed evaluation protocol for the performance evaluation of IVDs to evaluate their quality and performance. To facilitate this process, the Indian Council of Medical Research (ICMR) and CDSCO have come together to draft standard evaluation protocols for use by IVD manufacturers testing labs in India. Currently, the HMPV real time PCR IVD evaluation protocol has been developed by ICMR and CDSCO. The protocol is now being placed in the public domain for comments from relevant stakeholders. This window of opportunity will close on 15th March 2025, and, once finalized, there will be minimal scope for change in these documents. Therefore, all interested stakeholders are requested to provide their comments before 15th March 2025, at ivdevaluation@gmail.com as per the enclosed format. Once the public consultation period concludes, all comments will be reviewed and considered in finalizing the draft protocols before final clearance by ICMR and CDSCO. Dated: 28th January 2025 Place: New Delhi STANDARD IVD PERFORMANCE EVALUATION PROTOCOL STAKEHOLDER FEEDBACK FORM S.N. Name of the Document Page Line Current Text Proposed Explanation/Reference Protocol No. No. No. Text Name: Designation and Affiliation: Page 1 of 11 STANDARD PERFORMANCE EVALUATION PROTOCOL DRAFT FOR STAKEHOLDER COMMENTS HUMAN METAPNEUMOVIRUS REAL-TIME PCR 2 3 ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 4 5 JANUARY, 2025 New Delhi, IndiaHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 6 Human Metapneumovirus Real Time PCR Performance Evaluation Protocol 7 Table of Contents S.No. Content Page Number 1. Performance evaluation protocol for Human Metapneumovirus real- 2 time PCR kit 2. Information on Operational and Test Performance Characteristics 12 Required from Manufacturers 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 31 Performance evaluation protocol for Human Metapneumovirus real-time PCR kit 32 I. Background: 33 CDSCO and ICMR, New Delhi, have aimed at facilitating the availability of Quality-Assured 34 Diagnostics kits appropriate for use in India. Hence the following guidelines shall establish the 35 uniformity in performance evaluation of in-vitro diagnostic kits (IVD). The performance 36 evaluation is to independently verify the manufacturer’s claim regarding in-vitro diagnostic kit 37 (IVD) performance. 38 This recommendation focuses on the laboratory performance evaluation of Human 39 Metapneumovirus (hMPV) virus real time PCR kit. All clinical samples tested in the study should 40 be evaluated in accordance with the candidate test’s instructions for use. 41 42 II. Purpose: 43 To evaluate the performance characteristics of hMPV real-time PCR kits in the diagnosis of hMPV 44 infection/ disease. 45 III. Requirements: 46 1. Supply of kits under evaluation (Along with batch/lot No. Expiry & required details). If the 47 kit to be evaluated works in a closed system format, the manufacturer needs to supply 48 the required equipment. 49 2. Evaluation sites/laboratories (With required equipment) 50 3. Reference test kits 51 4. Characterised Evaluation panel 52 5. Laboratory supplies 53 IV. Ethical approvals: 54 Exempted from Ethics approval as per ICMR’s Guidance on Ethical Requirements for Laboratory 55 Validation Testing, 2024. A self-declaration form as provided in ICMR guidelines to be submitted 56 by the investigators to the institutional authorities and ethics committee for information. 57 V. Procedure: 58 1. Study design/type: Diagnostic accuracy study using clinical/spiked samples 59 2. Preparation of Evaluation sites/laboratories: 60 Identified IVD kit evaluation laboratories should be well-equipped and establish their 61 proficiency through ALL of the following: 2 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 62 A. Accreditation from NABL for at least one of the Quality management systems for at least one 63 respiratory viral pathogen molecular testing (NABL accreditation for testing Lab / calibration 64 lab as per ISO/IES 17025, Medical Lab as per ISO 15189, PT provider as per ISO/IEC 17043), or 65 CDSCO approved Reference laboratory. 66 B. Staff training: All the staff involved in hMPV virus IVD evaluation should undergo hands-on 67 training and competency testing on following 68 ⮚ Preparation & characterization of reference sample panel (at least 2 staff) 69 ⮚ Handling of hMPV RT-PCR kits received for performance evaluation 70 (Verification/Storage/Unpacking etc). 71 ⮚ Testing, interpreting, recording of results & reporting 72 ⮚ Data handling, data safety & confidentiality 73 3. Preparation of hMPV RNA evaluation panel 74 A well characterised panel of hMPV positive human samples is a critical requirement for 75 evaluation of these RT-PCR IVD kits. A statistically significant number of clinical samples should 76 be used for the evaluation. 77 The sample type for hMPV detection is nasopharyngeal/oropharyngeal swab. If a kit claims to 78 detect hMPV across several sample types, attempt should be made to evaluate the assay across 79 all the sample types. In case all the sample types mentioned in the IFU are not available with the 80 lab, the performance evaluation report should clearly mention the sample type against which the 81 kit is evaluated. There should be no ambiguity about the type of sample used for evaluation. 82 4. RNA extraction 83 RNA extraction should be performed using standard techniques. If the manufacturer of the index 84 test recommends a specific RNA extraction kit, the same needs to be provided by the 85 manufacturer if the evaluation lab is unable to procure the same. 86 5. Real-Time PCR System 87 PCR should be performed using IVD-approved machines. If any equipment(s) is specified in the 88 IFU of the index test, it should be used for the evaluation, and it should be provided by the 89 manufacturer if not available within the lab’s IVD evaluation scope. 90 Real-time closed systems/devices awaiting evaluation should be provided by the manufacturer 91 along with all necessary components, supplies and reagents. 92 6. Internal control/Extraction control 93 The index test must have an internal control (housekeeping gene), with or without an extraction 94 control (RNA added before extraction to a sample). 3 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 95 7. Reference assay: 96 FDA approved real-time PCR assay/ ICMR-NIV Pune in-house Real Time PCR Assay should be used 97 as the Reference Assay. 98 All positive samples should be confirmed positive by the reference assay. 99 All negative samples should be confirmed negative by the reference assay. 100 101 8. Sample size for performance evaluation: Sample size is calculated assuming 95% 102 sensitivity and specificity of the index test, 95% confidence level, absolute precision of 5% and 103 ≤5% invalid test rate. A minimum of 77 (rounded to 80) positive clinical samples and a minimum 104 of 77 (rounded to 80) negative clinical samples are required for performance evaluation. 105 However, for negative samples, a minimum of 115 specimens are suggested to account for a 106 rigorous cross reactivity panel. 107 108 109 9. Sample panel composition: 110 A. Human samples 111 A.1 Positive samples (n=80): Clinical samples positive by the reference real-time PCR 112 assay 113 A.1.1 Strong positive (Ct value <25) = 20 samples 114 A.1.2. Moderate positive (Ct value between 25-30) = 40 samples 115 A.1.3 Weak positive (Ct value >30-35) = 20 samples 116 Note: 117 If possible, attempt should be made to include all lineages of hMPV in the positive sample panel. 118 119 A.2 Negative samples (n=115): All negative samples should be negative by reference real- 120 time PCR assay. Distribution of the negative samples should be as follows: 121 A.2.1 NP/OP swab from individuals with respiratory infection that are negative for hMPV 122 RNA = 30 samples 123 A.2.2 NP/OP swab from apparently healthy individuals with no respiratory symptoms = 124 20 samples 125 A.2.3 Cross reactivity panel (Table 1): Samples negative for hMPV RNA but positive for 126 other common respiratory viruses = 65 samples 4 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 127 Table 1: Cross reactivity panel for performance evaluation of HMPV real time PCR kit S.N. Pathogen Minimum no. of Additional positive samples comments needed (n=65) i. RSV A 5 In case adequate ii. RSV B 5 number of one RSV type is unavailable, supplement with the available RSV type iii. Measles 5 - iv. Mumps 5 Buccal swab is the preferred sample type for Mumps, and the same (or throat swab) should be used for evaluation v. Seasonal Influenza A 10 (5 of each) - (H1N1pdm09 and H3N2) vi. Seasonal Influenza B 5 - (Victoria, with/without Yamagata) vii. SARS-CoV-2 5 - viii. Respiratory 5 Representation Adenovirus from all respiratory types is desirable ix. Human 5 Representation Respiroviruses 1 and from all types is 3, Human desirable Rubulaviruses 2 and 4 x. Rhinovirus 5 In case samples xi. Enterovirus 5 available with the lab are not typed into Rhinovirus and non-Rhinovirus Enteroviruses, please use 10 such samples to represent these 2 pathogens 5 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 xii. Seasonal 3 OC43 AND 229E coronaviruses xiii. Cytomegalovirus 2 Lower respiratory specimen positive for CMV is acceptable 128 129 If available, samples positive for relevant bacterial pathogens and other relevant viruses 130 (with which majority of the population is likely to be infected), should also be included in 131 the cross-reactivity panel. 132 133 10. Evaluation method: 134 The index test and the reference assay should be run simultaneously on the sample panel, 135 and results should be recorded. 136 11. Test reproducibility 137 A. Sample size for lot-to-lot reproducibility 138 Three lots of an assay should be evaluated. Sample size for lot-to-lot reproducibility should 139 be as follows: 140  First lot of the assay: should be tested on statistically significant number of positive 141 and negative samples as calculated in the protocol. 142  Second lot of the assay: should be tested on 25 samples (15 positive samples 143 comprising 10 low positive AND 5 moderate/high positive samples, and 10 negative 144 samples). 145  Third lot of the assay: should be tested on 25 samples (15 positive samples comprising 146 10 low positive AND 5 moderate/high positive samples, and 10 negative samples). 147  There should be no lot-to-lot variation. 148 149 Refer the flowchart below (Fig. 1): 6 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 150 151 152 153 12. Blinding of laboratory staff 154 To ensure rigor of the evaluation process, laboratory staff performing the evaluation 155 should be blinded to the status of the clinical samples. The PI of the evaluation exercise 156 should remain unblinded, i.e., privy to the status of the samples. Another senior 157 laboratory staff selected by the PI may remain unblinded and carry out coding of samples 158 and dispensing them into similar-looking vials to be used for testing, and maintaining the 159 database of results. Staff performing the reference test and the test under evaluation, 160 interpretation of the test result, and entering the results against the coded samples in the 161 database, should remain blinded to the status of samples till the completion of evaluation. 162 The data should be analyzed only by the PI of the evaluating lab. Refer to Fig. 2. 163 164 Fig.2: Blinding in evaluation exercise 7 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 165 166 167 168 13. Acceptance Criteria 169 Expected sensitivity: ≥95% 170 Expected specificity: ≥98% 171 Cross reactivity with other viruses as outlined in the negative sample panel: Nil 172 Invalid test rate: ≤5% 173 174 13. Publication Rights: 175 The PI(s) of the evaluating labs shall retain publication rights of the evaluation as lead author(s). 176 177 After following due procedure as defined in this document, once any kit is found to be Not of 178 Standard Quality, thereafter, no request for repeat testing of the same kit will be acceptable. 179 Any request of re-validation from the same manufacturer for the same test type will only be 180 entertained if valid proof of change in the kit composition is submitted. 181 8 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 182 VI. References: 183 1. U.S. Food and Drug Administration: Testing for Human Metapneumovirus (hMPV) Using Nucleic Acid 184 Assays - Class II Special Controls Guidance for Industry and FDA Staff. 2009. Available at: 185 https://www.fda.gov/medical-devices/guidance-documents-medical-devices-and-radiation-emitting- 186 products/testing-human-metapneumovirus-hmpv-using-nucleic-acid-assays-class-ii-special-controls- 187 guidance#3 [Accessed on January 11, 2025] 188 2. Amarasinghe, G.K., Ayllón, M.A., Bào, Y. et al. Taxonomy of the order Mononegavirales: update 189 2019. Arch Virol 164, 1967–1980 (2019). https://doi.org/10.1007/s00705-019-04247-4 190 191 VII. Performance evaluation report format 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 9 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 211 PERFORMANCE EVALUATION REPORT FOR HUMAN METAPNEUMOVIRUS (HMPV) REAL-TIME 212 PCR KITS Name of the product (Brand /generic) Name and address of the legal manufacturer Name and address of the actual manufacturing site Name and address of the Importer Name of supplier: Manufacturer/Importer/Port office of CDSCO/State licensing Authority Lot No / Batch No.: Product Reference No/ Catalogue No Type of Assay Kit components Manufacturing Date Expiry Date Pack size (Number of tests per kit) Intended Use Number of Tests Received Regulatory Approval: Import license / Manufacturing license/ Test license License Number: Issue date: Valid Up to: Application No. Sample Positive samples (provide details: clinical/spiked, strong, moderate, Panel weak) Negative samples (provide details (clinical/spiked), including cross reactivity panel) 213 214 Results Reference assay ……….……………… (name) Positive Negative Total Name of Positive HMPV virus real-time PCR Negative Total 215 Estimate (%) 95% CI Sensitivity Specificity 216 217 ● Details of cross reactivity with other viruses: 218 ● Conclusions: 10 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 219 o Sensitivity, specificity 220 o Performance: Satisfactory / Not satisfactory 221 (Sensitivity and specificity have been assessed in controlled lab setting using kits provided by the manufacturer from 222 the batch mentioned above using ….. sample. Results should not be extrapolated to other sample types.) 223 Disclaimers 224 1. This validation process does not approve / disapprove the kit design 225 2. This validation process does not certify user friendliness of the kit / assay 226 Note: 227 This report is exclusively for Human Metapneumovirus………….. Kit (Lot No……) manufactured by …………… 228 (supplied by ……….) 229 The kit has been validated against the pathogen (as a whole) with statistically significant sample size, and 230 NOT against different lineages of the pathogen. 231 Evaluation Done on …………………… 232 Evaluation Done by …………………………. 233 Signature of Director/ Director-In-charge …………………… Seal …………………………. 234 ********************************End of the Report**************************** 235 236 237 238 239 240 241 242 243 244 245 246 247 248 11 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 249 Annexure-1: Information on Operational and Test Performance Characteristics Required from 250 Manufacturers 251 The manufacturer should provide the following details about the IVD: 252 1. Instructions for Use 253 2. Scope of the IVD: to diagnose hMPV. 254 3. Intended Use Statement 255 4. Principle of the assay 256 5. Intended testing population (cases of ARI/ILI/SARI) 257 6. Intended user (laboratory professional and/or health care worker at point-of-care) 258 7. Lot/batch No. 259 8. Date of manufacture 260 9. Date of Expiry 261 10. Information on operational Characteristics 262 i. Configuration of the kit/device 263 ii. Requirement of any additional equipment, device 264 iii. Requirement of any additional reagents 265 iv. Operation conditions 266 v. Storage and stability before and after opening 267 vi. Internal control provided or not 268 vii. Quality control and batch testing data 269 viii. Biosafety aspects- waste disposal requirements 270 11. Information on Test Performance Characteristics 271 i. Type of sample-NP/OP swab, other respiratory specimen 272 ii. Volume of sample 273 iii. Any specific sample NOT to be tested 274 iv. Any additional sample processing required 12 | PageHMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 275 v. Any additional device/consumable like sample transfer device, pipette, tube, etc required 276 vi. Name of analyte to be detected 277 vii. Pathogens targeted by the kit 278 viii. Time taken for testing 279 ix. Time for result reading and interpretation 280 x. Manual or automated(equipment)reading 281 xi. Limit of detection 282 xii. Diagnostic sensitivity 283 xiii. Diagnostic specificity 284 xiv. Stability and reproducibility 285 xv. Training required for testing 286 xvi. If yes, duration 287 xvii. Details of Cut-off and /or Equivocal Zone for interpretation of test 288 xviii. Interpretation of invalid and indeterminate results to be provided 289 xix. It is recommended to provide data demonstrating the precision 290 291 *Please mention “Not applicable” against sections not pertaining to the kit. 292 293 294 295 296 297 298 299 13 | Page