Can I submit software programs other than SAS? What software programs should I submit? Are sponsors required to submit executable programs?
Do I need to rename my software programs so that they all have the same extension e.g. “.txt”?
Can I make use of macros in my software programs and if so, should macros be part of the submission package?
What kind of documentations for software programs should I include in the submission package?
Do I need to follow any particular style and conventions when writing software programs that will be part of a submission package?
A single topic generates so many questions! Get the answers in this blog.
Both FDA and PMDA, the US and Japanese Regulatory Agencies respectively, and more recently the NMPA (the Chinese agency[1]), have some “statement” in their guidance related to the submission of Software Programs. Despite that, in some cases sponsors still have different understanding or opinions on what software programs we need to include in our data submission package.
Let us try to answer these questions and in doing that, in addition to the available regulatory agency guidance[2] [3], I will also make use of the two recent presentations made at PHUSE US Connect 2019 [4] and PHUSE CSS 2018 [5].
Can I submit software programs other than SAS?
The use of SAS is not mandatory in the analysis of clinical study data. Instead, other software such as “r” could be used; for example, “.r” together with “.sas”, is also one of the “acceptable file formats for use in eCTD (as per the FDA guidance Specifications for File Format Types Using eCTD Specifications) . Of course the same requirements such as 21CRFPart11, apply to “r” or any other software programs used to collect, transform and to analyze clinical trials data.
Read an interesting blog clarifying the topic and how FDA reviewers make use of “r” and also the statement FDA made in 2015 about Statistical Software.
What software programs should I submit?
All three agencies require the submission of software programs used to create the complete analysis datasets and analysis results. FDA and PMDA specifically require the submission of software programs used to generate tables and figures for the primary and secondary efficacy results (PMDA requires analysis programs used to generate any efficacy results), while the NMPA does not make any distinction on the type of analysis results for which software programs should be submitted.
It is also common for phase I Pk studies to submit software programs used to generate key Pk analysis results.
Moreover, the PMDA has specific requirements with regards to software programs used for Pk analysis (see M. Dalton presentation for more details), while the FDA, in an update made in 2018 to its Study Data Technical Conformance Guide, added an additional requirement for “Section 14 CLINICAL STUDIES of the Prescribing Information” (the type of analysis results software programs to be submitted here might vary depending on therapeutic area, indication, etc. as such adverse events summary results software programs might be submitted, pk analysis results, etc.).
What about programs used to migrate legacy datasets to SDTM standards? There is no need to submit those software programs, unless specifically requested by the reviewer. In the data submission, SDTM is considered as source data and as such there is no need to provide the software programs used to convert legacy datasets to SDTM. For this reason in the define-xml provided along with the SDTM data package, there is no need to specify how you derived SDTM datasets from original legacy datasets (see “wrong” example in figure 1) [6].
Figure 1: SDTM define-xml containing reference to raw datasets
Where should you save submitted software programs in the eCTD? Module 5 of eCTD has a dedicated folder “programs” under analysis folder for both “adam” and “legacy”.
Are sponsors required to submit executable programs?
The PMDA states very clearly in its guidance that software programs do not need to be executable: “The main purpose of requesting the submission of these programs are to understand the process by which the variable for the predictive analyses were created and to confirm the analysis algorithm. Therefore, it is not necessary to submit the programs in a format or content that allows the PMDA to directly run the programs under its given environment”.
Although the FDA guidance is less explicit than PMDA in this regards, it states that the main purpose of requesting the submission of these programs is to understand the process by which the variables for the respective analyses were created and to confirm the analysis algorithms and results. Sponsors should submit software programs in ASCII text format. Executable file extensions should not be used; we can also affirm that the FDA has no plans to re-use the sponsor original code under its given environment.
On the other hand, it seems that the NMPA “desires” to re-use sponsor submitted software programs “Programming codes submitted in submission package should be readable (with comments), understandable, executable”.
Do I need to rename my software programs so that they all have the same extension e.g. “.txt”?
There has been some debate on the origin of the requirement of all submitted software programs having the same extension, “.txt”, regardless of the software used. This seems more a requirement for the FDA and confirmed through some informal communication with some sponsors as confirmed by M. Dalton in her presentation previously mentioned; so the recommendation here is to rename all software programs so that they all have the .txt extension and the name of the software program used as part of the name of the file e.g. “adae.sas” will be renamed to “adae-sas.txt”.
The PMDA instead clearly mentions to keep the original extension of the software programs used.
The NMPA says “Programming codes in submission package are generally in TXT format”, but here it is not clear whether or not the file extension should be changed as we could argue, for example, a SAS software program is a text file.
Can I make use of macros in my software programs and if so should macros be part of the submission package?
Both FDA and PMDA require sponsors to make their own judgement on whether or not a macro should be part of the submission package. Moreover, the PMDA also states that if submitting macro programs is difficult, sponsors can omit macros in the submission package. However, they require that any analysis algorithms covered by non-submitted macros, are described in the submitted documentation e.g. in the analysis data reviewer guide.
The NMPA instead requires software programs to be stand-alone “do not include external program calls, which in particular avoid using external macro programs”.
What kind of documentations for software programs should I include in the submission package?
If Analysis Results Metadata (ARM) [7] are included in your define-xml, then this could contain some details about your submitted programs e.g. source datasets, any selection criteria applied, statistical procedure applied and code used, etc. However, the recommendation is to also provide some details in the Analysis Data Reviewer Guide. See examples in template package made available by PHUSE [8].
Do I need to follow any particular style and conventions when writing software programs that will be part of a submission package?
Clarity of software programs should be a “must” regardless if those programs are part of a submission package or not. PHUSE provides some “Good Programming Practice Guidance” [9] and these have been referenced in 2018 poster presented by FDA at PHUSE-CSS and previously referenced. In that poster the FDA presenters also recommend the “tidyverse” style guide.
In conclusions, I hope I was able to answer the questions that I mentioned in the beginning of this post. Of course, as usual, my final recommendation is to always check with the assigned reviewer whenever there is a doubt.
References:
- “Guideline on the Submission of Clinical Trial, NMPA Center for Drug Evaluation Data”
- “FDA Study Data Technical Conformance Guidance”, November 2020
- “Japan Pharmaceuticals and Medical Devices Agency - Revision of Technical Conformance Guide on Electronic Study Data Submissions”, Jan. 2019
- “Submission of Software Programs to Regulatory Agency”, M. Dalton, PHUSE US CONNECT 2019, Paper SA04
- “FDA Study Data Technical Conformance Guide Updates for Software Program Submissions” P. Schuette and W. Zhang. PHUSE-CSS 2018
- “How to ensure quality in data submission”, A. Tinazzi, PharmaSUG-China 2019
- “CDISC define-xml”
- “ADRG Packages V1.2”
- “PHUSE Good Programming Practice”
About Angelo Tinazzi
Angelo Tinazzi is Senior Director, Statistical Programming, Clinical Data Standards and Clinical Data Submission at Cytel. He is a well- published and recognized expert in statistical programming with over 20 years' experience in clinical research. The application of CDISC standards in different therapeutic areas is part of his core expertise since 2003 in particular in the context of data submission to health authorities such as the FDA and PMDA.
Angelo is an authorized CDISC instructor and member of the CDISC ADaM Team as well as the CDISC European Committee where he also manages the Italian-speaking CDISC User Network.
