Technical Assessment FAQ (Compiled from Provider Inquiries)
1. Question: What guidance can you provide in answering question 1 on form M00151? Is form M00116 required if we are performing an NGS cancer test that outputs genomic alterations but includes proprietary components in the process of determining those alterations?
Answer: No. Form M00116 is required when the test result is novel or a proprietary result that requires independent clinical validity and utility assessments. If your test measures genomic alterations (such as mutations), wherein the utility and validity of such measurements are already established (like predictive biomarkers in cancer), the validity of performing such tests is established. Utility will be determined by ensuring the required test conditions for biomarker coverage are met. M00116 is reserved for new technologies or wherein validity of utility of a test is not established on the literature or cannot be established because there cannot be comparators to determine validity.
2. Question: If I am performing a hotspot (targeted) test defined by CPT codes 81445 or 81450, do I need to submit all the additional material on page 2 of the Checklist (form M00151)?
Answer: No. These additional documents are reserved for Comprehensive Genomic Profiles (CGP) or novel tests requiring form M00116.
3. Question: What is the appropriate number of samples in each variant type (SNV, indels, CNVs, fusions, TMB, etc.) to be included in this assessment to determine whether or not we need to run more samples? Clarify difference between analytical and clinical validation on form M00119 V5. Are both necessary for all components, or is clinical sample testing sufficient?
Answer: It is important to do an analytical validation to demonstrate the boundary conditions for the operations of the test. The different variant classes should be handled as distinct, and the contrived samples (like cell lines) should have enough of each variant class to accurately determine the capabilities of the test and set appropriate thresholds for the limit of detection (LOD).
Regarding the number of samples for the clinical validation, per CAP/AMP guidelines you should use at least 59 samples. Ensure these are representative of the kinds of tumors being tested and that there is enough of each variant class tested to show with some confidence that the test operates as expected in a clinical setting.
Analytical validation sets the detection parameters of the test across different variant types, typically in contrived samples like cell lines. For example, SNV detection in this setting can leverage SNPs and non-clinical variants that prove the test’s ability to accurately detect such variants with a large number of variants. Clinical validation tests these parameters in clinical samples, wherein the test will be typically limited to clinically relevant variants. Because these are relatively few in number, using only the latter will not result in a clear understanding of the working parameters of the test. For a CGP, both data sets are required and should be present in any well-conducted validation.
4. Question: For form M00153 for solid tumors, under section 6, the variant table, what is the intent of column 3 versus column 5? Is column 3 supposed to represent the number of samples with the variant by the comparative or orthogonal method and column 5 the number of samples with the variant by the assay we are assessing? Is MolDX asking for the number of samples? What about variants determined to be artifacts?
Answer: That is correct. Column 3 would be the samples that have the specified variant via the comparative or orthogonal method. Column 5 would be the samples in the validation where the variant was identified by your method. If it is determined via testing method that a finding is an artifact during the routine application of your test, then it would not be reported as a detected variant. If you ex post facto determine that a variant is detected that would have normally been reported as an artifact, then it is a false positive and should be discordant to your expected result.
5. Question: What about genes or variants that our assay covers but are not listed on the form (M00153 or M00154)?
Answer: It is not required that you enter this information here. You should provide aggregated information in section 3 of this form for the performance of the test as a whole, and if you are performing CGP test, more information is required on form M00119. For the latter, provide a list of genes in your validation dossier.
6. Question: Which NM number was used for the positions that are listed?
Answer: The amino acid positions are from the most clinically relevant transcript. This is usually the longest known transcript. Those listed are the most well-known. A quick review of the literature for these can give you the genomic positions.
7. Question: Do we need to submit a TA for each single gene assay or sub-panel that is done as part of a larger hotspot panel, but with masking of the other genes? For example, JAK2 V617F that is reported as a single gene assay yet is part of a larger myeloid mutation panel or a colorectal cancer panel that includes BRAF, HRAS, KRAS, NRAS and PIK3CA that is part of a larger 50 gene cancer panel? If we submit a TA for the larger panel, is that enough or do we also need to submit for every sub-panel and single gene reported?
Answer: The TA must show that the test for which a claim is being submitted was validated. If the test was validated as part of the validation of a larger panel or platform validation, that is fine. If you wish to offer tests to patients consisting of single genes or small groups of genes from a large panel and submit claims for these tests, we ask that you register each gene or group of genes (i.e., each test) with a unique Z-identifier and provide us with an executive summary in your TA request noting what tests (identified by Z-code) corresponds to which validation documents submitted.
8. Question: For the Analytical and Clinical Validation documents to be submitted for the TA of NGS tests, does Palmetto GBA want the raw data generated for validation of these tests?
Answer: No, the raw data are not required. For CGP tests, please provide a copy of the approved validation summary for the assay. For “targeted panels” (non CGP), only the forms are required.
9. Question: What will be the consequence for labs that do not submit a TA by the deadline set forth in LCDs where TAs are required?
Answer: If labs do not submit the required TAs for tests that are currently covered, coverage will be rescinded until a TA is completed satisfactorily.
As an aside: Labs should keep in mind that a majority of coverage for NGS tests in cancer (within the scope of the draft LCD) are defined and billed as “targeted panels” or “hotspot tests.” According to the TA process, most labs using these tests will only need to complete forms M00151 (the checklist) and either M00153 or M00154 for the relevant test to complete the TA process.
10. Question: Is it necessary to have confirmation of every specific variant in the list on form M00153 if it is covered on our panel?
Answer: No. It would not be reasonable to know a priori that all variants are found in your validation set, furthermore it would not be feasible to ensure every variant, particularly rare ones, are tested. However, there are some common variants we would ensure are covered in your test, such as those commonly seen in EGFR and BRAF. Test coverage for the intended use of the test is required, however.
11. Question: Could immunohistochemistry serve as an orthogonal test for MSH2/MSH6/MLH1 mutations on form M00153?
Answer: It could, however this is not a great validation method (A: IHC can show loss due to factors other than mutations, resulting in false negatives; B: you cannot confirm the actual mutation identified). It may be better to confirm with Sanger sequencing if you have discrepant findings — which would be included in your validation — to reduce possible errors.
12. Question: For the specific variants listed, what is the “expected VAF?” These are clinical samples and the orthogonal test does not report allele frequency. Should we list the allele frequency at which our assay detected them?
Answer: If the orthogonal test is another NGS test utilizing the same specimen, then you would have an expected VAF to compare. If you do not have such a comparator, leave this blank.
13. Question: How should we account for a single unique specimen that has multiple mutations in the same gene (e.g., APC or TP52)? Is that considered two unique specimens?
Answer: For form M00153 it is one specimen. We want to ensure you are validating the platform on an acceptable number of clinical samples. For form M00119, we want to know the variants expected, so there is the opportunity to further define such cases.