AUDIENCE FOR THIS CHAPTER

This chapter is directed at librarians and faculty who work with health sciences students who need to understand complex systems to improve patient and population health. As well it is directed at librarians and faculty who work with engineering students who need to understand the complexity of interacting standards and clinical contexts in order to create products for the health care environment. Both are part of interdisciplinary teams that require a greater understanding of the medical product life cycle and reporting systems [3–5]. This chapter may also benefit librarians and faculty who work with programs in regulatory science or regulatory affairs that develop new methods and computational tools for assessing safety and risk [6–9]. These regulatory science degree programs are often interdisciplinary and housed within a variety of academic departments including health sciences, biopharmaceutics, engineering, business, law, and environmental science. Also, medical schools and large medical centers are increasingly focusing on innovation and rapid design thinking [10]. As a result, librarians are called to work with entrepreneurial teams developing new drugs and biologicals, health information systems, diagnostic equipment, medical instruments and devices, or health care quality improvement processes [6, 11–15].

While we often assume that students today rapidly learn how to use new products and technologies, this rapid assimilation may not always come with an understanding of the purpose of these tools, or the systems underlying their creation and regulation [16]. As a result of this lack of understanding, medical products are used inconsistently by health care providers, and post-market adverse events are underreported due to time or culture constraints [17]. Students and professionals also may not fully understand the role that standards play in transitions of care and post-market analysis. By graduating health care professionals who lack understanding of these systems, we may inadvertently impact the ability to extract meaningful data about existing products and potentially impair the future creation of safe and effective products. As well, we may not be properly preparing graduates who can assure that engineering standards remain up to date with current understanding of biomechanics, physiology, technology, security, and standards of care [18]. As we look to a future of increased precision medicine and artificial intelligence, adherence to standards will facilitate increased clinical decision support enabled by learning health care systems.

THEORETICAL FRAMEWORK

Health science librarians frequently focus their teaching on the Association of American Medical Colleges (AAMC) Entrustable Professional Activity (EPA) 7 for entering residents, that of evidence-based practice [19]. They assist students and providers in finding, evaluating, and synthesizing the best available evidence (e.g., journal articles, clinical practice guidelines, etc.) for clinical decision making. They may be less comfortable with their role in AAMC EPA 5 documentation, EPA 9 interprofessional teams, and EPA 13 system failures.

Even further, they may fail to look more closely at Accreditation Council for Graduate Medical Education (ACGME) Core Competencies for graduating residents, specifically Core Competency 5, practice-based learning and improvement (PBLI), and 6, systems-based practice, which were added in 1999 and 2002 [20–24]. These competencies were created to acknowledge that health care providers no longer work as solo practitioners, and increasingly need to understand complex systems and work with interdisciplinary team members—nurses, systems administrators, insurance companies, dieticians, social workers, pharmacists, biomedical engineers, and others—to provide care for patients. Previous authors have written about the difficulty in teaching and assessing systems-based practice in health care, even though it has been standard in engineering for decades [25–28]. In a mixed-methods study by Ackerman, et al. of a cardiology outpatient clerkship, students preferred gaining clinical skills through direct client-patient interactions, over systems-based practice objectives focused on workflow, patient user experience, and follow-up communication [25]. Systems-based practice is most frequently addressed through quality improvement exercises such as clinical audits or morbidity and mortality rounds [21, 29–30]. Librarians have mapped these competencies to the ACRL Framework, though high-level documents may fail to translate to logistical examples of teaching for these competencies [31].

Teaching standards to health care professionals presents one opportunity for greater understanding of systems-based practice and practice-based learning and improvement. Systems-based practice requires the learner to adapt to changes in health care and reporting systems [20]. While much is focused on human-centered design and creating and using systems that understand people/user needs, less in health care is focused on the opposite, creating individuals and populations who understand standards underlying these systems and the need for interoperability with other systems [32–34].

HISTORY OF STANDARDS IN HEALTH CARE

In the United States, the Food, Drug, and Cosmetic Act (FDCA) passed in 1938 first gave authority for food and drug safety to the Food and Drug Administration (FDA). The FDA amended the Food, Drug, and Cosmetic Act in 1976 to include medical devices 201(h) and to define three classes of devices:

Class I—do not require premarket approval

Class II—require premarket notification (FDA 510(k)) and post-market surveillance

Class III—approved by the premarket approval (PMA) process including clinical trials for quality, safety, and effectiveness that are similar to drugs

For drugs and devices marketed prior to the amendment, it required the device manufacturer to undergo the premarket authorization process and prove the safety and efficacy of the device to continue marketing it.

The addition of FDA Medical Device Reporting (MDR) in 1984 required manufacturers to report complaints and incidents to the FDA. In 1990, the Safe Medical Device Act amended the FDCA to require device traceability and added requirements by distributors and health care facilities to report post-market incidents to the FDA. In 1995, reporting forms were standardized and foreign device manufacturers were required to comply with the same regulations, and a 1998 FDCA amendment adjusted the Safe Medical Device Act to require distributors to report complaints only, not incidents. The 2002 Medical Device User Fee and Modernization Act focused on premarket and reprocessed devices. The 2016 21st Century Cures Act improved the regulation of combination products, created procedures for new indications for approved drugs, expedited processes for biologics and medical devices in response to health needs, and set parameters for collecting sustainable real-time post-market safety and adverse reporting data from networked devices. The FDCA was extended in 2017 with the Food and Drug Administration Reauthorization Act (FDARA).

In the United States, the FDA adopts technical, engineering, or information exchange specifications or terminologies developed by national or international standards developing organizations, or other government agencies. These are incorporated into Current Good Manufacturing Practice regulations for the manufacturing of products under the Center for Devices and Radiological Health (CDRH), the Center for Biologics Evaluation and Research (CBER), or the Center for Drug Evaluation and Research (CDER). While veterinary devices and drugs are regulated by the FDA, veterinary biologics fall under the jurisdiction of the USDA Animal Plant Health Inspection Service (APHIS) under 9 CFR E: 101-118.

Oversight standards for interoperability, privacy, and security of networked devices fall under the Department of Health and Human Services Office of the National Coordinator for Health Information Technology (ONC), which maintains www.healthIT.gov.

In Europe, the Global Harmonization Task Force on medical devices formed in 1992 just prior to the creation of the European Union. The International Medical Device Regulators Forum replaced this in 2011. Formal medical device regulation (EU 2017/745) requiring greater post-market follow-up was created in 2017, for application in 2021 [35–36]. Implementation was delayed until spring 2024 as the European Database on Medical Devices (EUDAMED) prepared to register devices and assure unique identifiers.

Several medical device classification/nomenclature systems exist around the world. Some of the more common are the United Nations Standard Products and Services Code (UNSPSC), the Global Medical Device Nomenclature (GMDN), the Universal Medical Device Nomenclature System (UMDNS), the Generic Implant Classification (GIC), and the European Medical Devices Nomenclature (EMDN). Use of a particular system often is decided based on a nomenclatures structure (hierarchy or polyhierarchy), licensing (free or copyright), granularity of description, and use by specific disciplines or partnering organizations. EUDAMED requires the use of EMDN because, unlike the proprietary polyhierarchy UMDNS system, it is a freely available hierarchy [37–38].

CLASSIFICATION OF HEALTH CARE PRODUCTS COVERED BY STANDARDS

Health-related engineering standards cover medical devices, information technology, drugs, biologicals, and facilities. This section will address each of these segments in turn.

The definition of “medical devices” is poorly understood. This term incorporates an array of equipment encountered in diverse settings and disciplines. While the phrase is used frequently in engineering settings, it is rarely encountered in health science curriculums. Medical devices may include laboratory and imaging diagnostic equipment, remote and bedside patient monitors, drug delivery systems, drug manufacturing materials and equipment, medical implants, personal protective equipment, and surgical instruments and robots. Laboratory equipment combines reference (tests and analysis) and metrology (measurement) standards with materials and network capabilities. Previous authors have classified medical devices by their function (therapeutic, diagnostic, and analytical), data type (standard DICOM, HL7, XML, or nonstandard image data), connections to networks, and data flow.

Pharmaceutical drugs, chemical substances that affect physiology or psychology, are regulated separately from biologicals or biologics, originating from living cultures or blood, a category that includes vaccinations, blood products, and a growing array of immunotherapies.

Health information technology (HIT) incorporates electronic health records, information and communications technology (ICT), telehealth, standard file formats unique to health care (such as DICOM radiology images), algorithms, security/privacy, health information exchange (HIE), and a growing number of networked medical devices in what is sometimes referred to as the Internet of Medical Things (IoMT) [39].

Health information technology systems frequently use permanent identifier standards and terminology standards designed to represent the context of the health care setting or injury. Identifier and terminology uptake may vary across countries based on mandates, incentives, and the degree to which health care is publicly administered. Controlled vocabularies that seek to define context can be a good entry point for librarians, and some systems may map clinical terminology to educational objectives, or to controlled vocabularies used to index literature, such as the National Center for Biotechnology Information (NCBI) Medical Subject Headings (MeSH).

While the focus of this book is on technical/engineering standards, the FDA recognizes a growing number of combination products, and these products often require consulting multiple categories of standards. For example, human drugs and biologics are regulated by the FDA, and their manufacturing, packaging, and delivery systems are subject to materials and manufacturing standards. Tissue-engineered medical products (TEMPs) used as implants in regenerative medicine are composed of both biological and synthetic materials. Health care facilities that are subject to standards for air quality, water, waste, materials, energy, design, and networks that impact patient safety, may also choose to pursue the Leadership in Energy and Environmental Design (LEED) standard for health care, or install SMART operating rooms. Environmental standards for water, air, and waste may force health care providers to find new methods for necessary tasks such as cleaning and sterilization. Standards for data applicable to HIT, ICT, and HIE can assure that data generated by microelectromechanical systems (MEMS), which incorporate mechanical and networked electronic elements, are compatible with other systems. At the same time, data generated by these products need to meet privacy and security standards. Standards must be compatible with multiple organizational standards and medical practice guidelines, which directly impact diagnostics or patient treatment. For instance, ISO medical laboratory quality standards are compatible with the Laboratory Medicine Practice Guidelines (LMPGs) from the American Association of Clinical Chemistry, which conforms to the National Academy of Medicine Committee on Standards.

HEALTH CARE ENVIRONMENT

The health care environment is unique in many ways. Health care systems are complex. Providing care involves a variety of interdisciplinary stakeholders, multiple systems, and frequent transitions of care [32]. Patients themselves are members of their own health care team. Highly trained professionals use health care devices and systems. In many cases, the outcome of their use of medical devices may be dependent on skill and technique, while other times their use of devices (such as health information systems) is considered secondary to their main job [40].

Health care accounts for the highest number of professional malpractice claims. Patient safety itself has standards, and it is safety that becomes the number one goal driving the use of standards [41]. While studies show that implementing HIT does reduce medication errors and improves compliance, the overall impact on patient safety requires further study [42]. Furthermore, in addition to inherent medical and surgical risks associated with health care, the expectation for security and privacy standards is high due to the Health Insurance Portability and Accountability Act (HIPAA). Health care providers are at risk of security breaches and ransomware attacks.

Health care increasingly is moving away from clinic-centered care to a continuum of care emphasizing prevention—and intervention when risk is determined [43]. While regulatory agencies and manufacturers have traditionally used a system of premarket clinical trials and postmarket reaction to problems after reported incidents, the potential exists through machine learning for more dynamic risk assessment relying on Markov modeling. This can facilitate real-time decision support and earlier intervention [44].

As health care becomes more automated, medical education runs the risk of focusing too much on preparing physicians to work within a system, and not enough on preparing students to change the system when it fails to advance health equity [24]. As a result, health education is now focusing more on personalized medicine, and this starts with recruiting diverse participants for medical device and drug clinical trials [45].

Products often interact with other products and with the human body, through either physical contact, chemical interaction, or technical connection. Physical contact requires strict standards for withstanding and assuring sterility. Chemical contact between products requires preventing incompatibility. Technical connections require interoperability with a complex health system that includes monitors, medical records, financial systems, insurance claims, patient portals, and quality improvement systems. As well, alarms or alerts designed to increase patient safety may inadvertently cause harm if they contribute to provider fatigue in the health care environment.

EDUCATION OF HEALTH CARE PROVIDERS

Previous authors have emphasized the importance of case-based experiential learning for regulatory science [6]. While case-based learning is common in medicine, it may be less commonly seen in relation to medical product development and product use in health care settings. For health science professionals, understanding regulatory science begins with an understanding of evidence-based practice, critical appraisal, and experience applying real-time data to patient care. A modern example of the important role of standards exists within the current world problem of antibiotic resistance. Health care providers should consult Clinical and Laboratory Standards Institute (CLSI) standards for species to accurately correlate in vitro culture and sensitivity results with patient clinical parameters in order to select antibiotic protocols that prevent antibiotic resistance [46]. Once students have an advanced understanding of evidence levels, and the application of population-level evidence to clinical practice, they are prepared to understand how their own documentation, using standard calculations, file formats and terminologies in health information systems, impacts the creation of future practice-based evidence or real-world evidence (RWE) [47–49].

A step beyond evidence-based medicine involves teaching students quality improvement methods that dive deep to the level of consulting and assessing current standards. These are best taught in interdisciplinary team settings, reflective of the work environment. Students should identify the presence of post-market reporting systems and understand that health care providers are not powerless to change unsafe and ineffective products; however, they need outcomes data, and they need to collect it in a standard way. Because many government and manufacturer reporting systems have not been transparent or consistent across countries or states in the past, and because health care providers do not have access to device and operator-specific information from scientific studies, many health care providers have become proponents of international medical device registries, which provide data independent of industry [18, 50–51]. Only through regular use of reporting systems and registries can health care providers accurately identify where adverse events are associated with standards or variability in standards, and not medical error or patient factors [18]. As Rome states, “Clinician and patient engagement in post-market surveillance and comparative effectiveness research remains imperative” [52].

Adverse reporting databases in the United States include the FDA Adverse Event Reporting System (FAERS) Public Dashboard, which displays human drug and biologic adverse reports since 1968 by region, report type, seriousness of report, type of reporter (health care provider vs. consumer), age, and sex [53]. The database is updated weekly and searchable by generic and proprietary product names. The FDA also maintains the Medical Device Recall Database and the Manufacturer and User Facility Device Experience (MAUDE) Database of Medical Device Reports (MDRs) submitted to the FDA by mandatory reporters (manufacturers, importers, and device user facilities) and voluntary reporters such as health care professionals, patients, and consumers [54]. The FDA Sentinel Initiative provides training and data from partner institutions’ electronic health and billing systems to evaluate post-market drug and biologic safety. Working with data created from electronic health records and reporting systems informs how students document in health information systems in the future [55].

Health science students should also practice with medical terminologies/ontologies used in HIT systems so that they understand their power to collocate like cases and enable collective data. The use of virtual patients in electronic health record simulation systems helps to make these ontologies transparent through structured data input and drop-down menus, in place of free text fields. Teaching with the use of actual or simulated hospital electronic health record systems can make these standards even more transparent to users and encourage health professionals to be involved in the ongoing development of these terminologies. Beyond learning medical terminology, the use of ontologies helps to define complex relationships and contexts. For example, the recent update to ICD-10 diagnostic codes have received mixed reviews due to their insistence on exact descriptions. While this can be frustrating for practitioners, it can provide an interesting exercise for students. Organizations regularly release humorous lists of ICD-10 codes, such as W56.01 “bitten by dolphin” and Y93.D “arts and crafts injury.” While engaging, these codes also serve to make students consider who might want to collocate this data. Students can use online terminology browsers or metathesauri to identify codes.

Exposure to health care environments, or simulated health care environments, helps product developers understand time constraints, cognitive load, tissue and chemical exposure, anatomic barriers, and other limitations that arise in certain settings where a product may be used. Existing health science simulation laboratories are underutilized as learning environments for broader entrepreneurial groups. When products are complex combination products, such as drug delivery systems or smart wearables, relevant standards bridge multiple standards developing organizations, and they may be best identified by interdisciplinary teams working in simulated settings.

As well, incorporating case scenarios with remote networked devices and transitions of care to other health care facilities will help teams understand the need for interoperable standards like Fast Health Care Interoperability Resources (FHIR) in increasingly complex health information networks. At the same time, teams may recognize the limitations of proprietary systems in health information exchange, and the potential barrier that proprietary data can be to patient safety.

As devices become more networked, and the potential for computational modeling, real-time data dashboards, and RWE increases, we are moving from a reliance on post-market reporting systems to point-of-care risk mitigation. We can prepare students for this by providing opportunities to interpret adverse event databases, crowd-sourced datasets, and data generated by personal devices. Future jobs will require professionals to use these data for risk management at the population, patient, and individual device level. Health care professionals aided by quantitative decision support tools that weigh multiple factors need skills in applying data to patient care, assessing risk management tools for accuracy and bias, and communicating risk effectively to the public.

REFERENCES

1. 1 E. Hu, “All things considered. Number of the year: How an obscure government code—834—became big in 2013,” National Public Radio. https://www.npr.org/sections/alltechconsidered/2013/12/20/255834823/how-an-obscure-government-code-834-became-big-in-2013
2. 2 T. D. Aungst and R. Patel, “Integrating digital health into the curriculum—Considerations on the current landscape and future developments,” Journal of Medical Education and Curricular Development, 7, Jan/Dec 2020, p. 2382120519901275. https://doi.org/10.1177/2382120519901275.
3. 3 Y. Yazdi, “Developing innovative clinicians and biomedical engineers: A case study,” American Journal of Preventive Medicine, 44(1), Jan. 2013, pp. S48–50. https://doi.org/10.1016/j.amepre.2012.09.013.
4. 4 A. Thakur, S. Soklaridis, A. Crawford, B. Mulsant, and S. Sockalingam, “Using rapid design thinking to overcome COVID-19 challenges in medical education,” Academic Medicine, 96(1), Jan. 2021, pp. 56–61. acm.0000000000003718.
5. 5 P. E. Patterson, “A systematic approach for introducing innovative product design in courses with engineering and nonengineering students,” Biomedical Sciences Instrumentation, 43, 2007, pp. 91–94.
6. 6 J. E. Adamo, E. E. Wilhelm, and S. J. Steele, “Advancing a vision for regulatory science training,” Clinical and Translational Science, 8(5), Oct. 2015, pp. 615–618. https://doi.org/10.1111/cts.12298.
7. 7 T. M. Morrison, P. Pathmanathan, M. Adwan, and E. Margerrison, “Advancing regulatory science with computational modeling for medical devices at the FDA’s Office of Science and Engineering Laboratories,” Frontiers in Medicine, 5, Sep. 2018. fmed.2018.00241.
8. 8 P. Spindler, K. F. Bach, M. Schmiegelow, N. Bedlington, and H.-G. Eichler, “Innovation of medical products: The evolution of regulatory science, research, and education,” Therapeutic Innovation & Regulatory Science, 50(1), 2016, pp. 44–48.
9. 9 D. Drago, S. Shire, and O. Ekmekci, “Improving regulatory education: Can we reconcile employers’ expectations with academic offerings?,” Therapeutic Innovation & Regulatory Science, 50(3), 2016, pp. 330–336.
10. 10 W. Liu, J. Yang, and K. Bi, “Factors influencing private hospitals’ participation in the innovation of biomedical engineering industry: A perspective of evolutionary game theory,” International Journal of Environmental Research and Public Health, 17(20), Oct. 13, 2020. https://doi.org/10.3390/ijerph17207442.
11. 11 N. Shin, K. Vela, and K. Evans, “The research role of the librarian at a community health hackathon—a technical report,” Journal of Medical Systems, 44(2), Dec. 18, 2019, p. 36. https://doi.org/10.1007/s10916-019-1516-x.
12. 12 A. Singh, D. Ferry, and S. Mills, “Improving biomedical engineering education through continuity in adaptive, experiential, and interdisciplinary learning environments,” Journal of Biomechanical Engineering, 140(8), Aug. 1, 2018, pp. 0810091-8. 1.4040359.
13. 13 A. W. Eberhardt, O. L. Johnson, W. B. Kirkland, J. H. Dobbs, and L. G. Moradi, “Team-based development of medical devices: An engineering-business collaborative,” Journal of Biomechanical Engineering, 138(7), Jul. 1, 2016, pp. 0708031-5. 1.4032805.
14. 14 A. J. Carroll, S. J. Hallman, K. A. Umstead, J. McCall, and A. J. DiMeo, “Using information literacy to teach medical entrepreneurship and health care economics,” Journal of the Medical Library Association, 107(2), Apr. 2019, pp. 163–171. jmla.2019.577.
15. 15 C. Rika Wright, Z. Shamika, and S. Arif, “Project-based learning: Engaging biomedical engineering sophomores through a collaborative vein-finder device project with nursing,” Salt Lake City, Utah, June 23, 2018. https://peer.asee.org/30903.
16. 16 N. Selwyn, “The digital native–myth and reality,” in Aslib Proceeding. Bradford: Emerald Group Publishing Limited, 2009.
17. 17 M. L. L. Leape, Making healthcare safe: The story of the patient safety movement. Cham, Switzerland: Springer, 2021. https://link.springer.com/book/10.1007%2F978-3-030-71123-8
18. 18 F. Henshaw et al., “Engineering standards for trauma and orthopaedic implants worldwide: A systematic review protocol,” BMJ Open, 8(10), Oct. 18, 2018, p. e021650. https://doi.org/10.1136/bmjopen-2018-021650.
19. 19 P. Cocks, W. Cutrer, K. Esposito, and C. Lupi, “EPA 7 schematic: Form clinical questions and retrieve evidence to advance patient care,” in Core Entrustable Professional Activities for Entering Residency, V. Obeso, D. Brown, and C. Phillipi, eds. Washington DC: Association of American Medical Colleges, 2017.
20. 20 A. G. Lee et al., “Teaching and assessing systems-based competency in ophthalmology residency training programs,” Survey of Ophthalmology, 52(6), Nov/Dec 2007, pp. 680–689. https://doi.org/10.1016/j.survophthal.2007.08.021.
21. 21 A. M. Tomolo, R. H. Lawrence, and D. C. Aron, “A case study of translating ACGME practice-based learning and improvement requirements into reality: Systems quality improvement projects as the key component to a comprehensive curriculum,” Postgraduate Medical Journal, 85(1008), Oct. 2009, pp. 530–537. https://doi.org/10.1136/qshc.2007.024729.
22. 22 S. Guralnick, E. Fondahn, A. Amin, and E. A. Bittner, “Systems-based practice: Time to finally adopt the orphan competency,” Journal of Graduate Medical Education, 13(2), Apr. 2021, pp. 96–101. jgme-d-20-00839.1.
23. 23 J. D. Gonzalo, C. H. Chuang, S. A. Glod, B. McGillen, R. Munyon, and D. R. Wolpaw, “General internists as change agents: Opportunities and barriers to leadership in health systems and medical education transformation,” Journal of General Internal Medicine, 35(6), Jun. 2020, pp. 1865–1869. s11606-019-05611-5.
24. 24 E. G. Castillo, J. Isom, K. L. DeBonis, A. Jordan, J. T. Braslow, and R. Rohrbaugh, “Reconsidering systems-based practice: Advancing structural competency, health equity, and social responsibility in graduate medical education,” Academic Medicine, 95(12), Dec. 2020, pp. 1817–1822. https://doi.org/10.1097/acm.0000000000003559.
25. 25 S. L. Ackerman et al., “The action research program: Experiential learning in systems-based practice for first-year medical students,” Teaching and Learning in Medicine, 28(2), 2016, pp. 183–191. https://doi.org/10.1080/10401334.2016.1146606.
26. 26 M. A. Dolansky, S. M. Moore, P. A. Palmieri, and M. K. Singh, “Development and validation of the systems thinking scale,” Journal of General Internal Medicine, 35(8), Aug. 2020, pp. 2314–2320. s11606-020-05830-1.
27. 27 M. M. Plack et al., “Systems thinking and systems-based practice across the health professions: An inquiry into definitions, teaching practices, and assessment,” Teaching and Learning in Medicine, 30(3), Jul/Sep 2018, pp. 242–254. 10401334.2017.1398654.
28. 28 P. B. Batalden and D. C. Leach, “Sharpening the focus on systems-based practice,” Journal of Graduate Medical Education, 1(1), Sep. 2009, pp. 1–3. https://doi.org/10.4300/01.01.0001.
29. 29 P. Varkey, S. Karlapudi, S. Rose, R. Nelson, and M. Warner, “A systems approach for implementing practice-based learning and improvement and systems-based practice in graduate medical education,” Academic Medicine, 84(3), Mar. 2009, pp. 335–339. https://doi.org/10.1097/ACM.0b013e31819731fb.
30. 30 J. K. Johnson, S. H. Miller, and S. D. Horowitz, “Advances in patient safety. Systems-based practice: Improving the safety and quality of patient care by recognizing and improving the systems in which we work,” in Advances in Patient Safety: New Directions and Alternative Approaches, vol. 2, K. Henriksen, J. B. Battles, M. A. Keyes, and M. L. Grady, eds. Rockville, MD: Agency for Healthcare Research and Quality (U.S.), 2008.
31. 31 E. A. Brennan, R. S. Ogawa, K. Thormodson, and M. von Isenburg, “Introducing a health information literacy competencies map: connecting the Association of American Medical Colleges Core Entrustable Professional Activities and Accreditation Council for Graduate Medical Education Common Program Requirements to the Association of College & Research Libraries Framework,” Journal of the Medical Library Association, 108(3), Jul. 1, 2020, pp. 420–427. https://doi.org/10.5195/jmla.2020.645.
32. 32 G. M. Samaras and R. L. Horst, “A systems engineering perspective on the human-centered design of health information systems,” Journal of Biomedical Informatics, 38(1), Feb. 2005, pp. 61–74. https://doi.org/10.1016/j.jbi.2004.11.013.
33. 33 J. E. McLaughlin, M. D. Wolcott, D. Hubbard, K. Umstead, and T. R. Rider, “A qualitative review of the design thinking framework in health professions education,” BMC Medical Education, 19(1), Apr. 4, 2019, p. 98. s12909-019-1528-8.
34. 34 B. Holtz, K. Vasold, S. Cotten, M. Mackert, and M. Zhang, “Health care provider perceptions of consumer-grade devices and apps for tracking health: A pilot study,” JMIR mHealth and uHealth, 7(1), Jan. 22, 2019, p. e9929. https://doi.org/10.2196/mhealth.9929.
35. 35 P. Marešová, B. Klímová, J. Honegr, K. Kuča, W. N. H. Ibrahim, and A. Selamat, “Medical device development process, and associated risks and legislative aspects-systematic review,” Frontiers in Public Health, 8(308), July30, 2020. https://doi.org/10.3389/fpubh.2020.00308.
36. 36 International Medical Device Regulators Forum, “International Medical Device Regulators Forum Strategic Plan 2021–2025,” 2020. https://www.imdrf.org
37. 37 European Commission, “EUDAMED Database.” https://ec.europa.eu/tools/eudamed
38. 38 World Health Organization, “Standardization of medical devices nomenclature: International classification, coding and nomenclature of medical devices EB150/14 Add. 1.” https://apps.who.int/gb/ebwha/pdf_files/EB150/B150_14Add1-en.pdf
39. 39 T. Yaqoob, H. Abbas, and N. Shafqat, “Integrated Safety, Security, and Privacy (ISSP) Risk Assessment Framework for medical devices,” IEEE Journal of Biomedical and Health Informatics, 24(6), Jun. 2020, pp. 1752–1761. https://doi.org/10.1109/JBHI.2019.2952906.
40. 40 R. E. Hoyt, W. R. Hersh, and E. V. Bernstam, Health informatics: Practical guide. Author: Lulu.com, 2018.
41. 41 T. Miclăuş et al., “Impact of design on medical device safety,” Therapeutic Innovation and Regulatory Science, 54(4), Jul. 2020, pp. 839–849. https://doi.org/10.1007/s43441-019-00022-4.
42. 42 Y. K. Alotaibi and F. Federico, “The impact of health information technology on patient safety,” Saudi Medical Journal, 38(12), 2017, p. 1173.
43. 43 D.-W. Kim, J.-Y. Choi, and K.-H. Han, “Medical device safety management using cybersecurity risk analysis,” IEEE Access, 8, 2020, pp. 115370–115382. https://doi.org/10.1109/ACCESS.2020.3003032.
44. 44 S. R. Fox-Rawlings, L. B. Gottschalk, L. A. Doamekpor, and D. M. Zuckerman, “Diversity in medical device clinical trials: Do we know what works for which patients?: Diversity in medical device clinical trials,” The Milbank Quarterly, 96(3), 2018, pp. 499–529. https://doi.org/10.1111/1468-0009.12344.
45. 45 Clinical and Laboratory Standards Institute (CLSI), Understanding susceptibility test data as a component of antimicrobial stewardship in veterinary settings, 2019. https://clsi.org/standards/products/veterinary-medicine/documents/vet09
46. 46 L. W. Green, “Public health asks of systems science: to advance our evidence-based practice, can you help us get more practice-based evidence?,” The American Journal of Public Health, 96(3), Mar. 2006, pp. 406–409. https://doi.org/10.2105/AJPH.2005.066035.
47. 47 United States Food and Drug Administration, “Real-world evidence.” https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence
48. 48 European Medicines Agency, “Data Analysis and Real World Interrogation Network (DARWIN EU).” data-analysis-real-world-interrogation-network-darwin-eu
49. 49 G. Raman et al., “AHRQ methods for effective health care,” in Quality of Reporting in Systematic Reviews of Implantable Medical Devices. Rockville, MD: Agency for Healthcare Research and Quality, 2012.
50. 50 R. E. Gliklich, N. A. Dreyer, M. B. Leavy, “Registries for evaluating patient outcomes: A user’s guide.” Washington, D.C.: Agency for Healthcare Research and Quality, 2014.
51. 51 B. N. Rome, D. B. Kramer, and A. S. Kesselheim, “Approval of high-risk medical devices in the US: implications for clinical cardiology,” Current Cardiology Reports, 16(6), 2014, p. 489. https://doi.org/10.1007/s11886-014-0489-0.
52. 52 United States Food and Drug Administration, “FDA Adverse Event Reporting System (FAERS) Public Dashboard.” https://www.fda.gov/drugs/questions-and-answers-fdas-adverse-event-reporting-system-faers/fda-adverse-event-reporting-system-faers-public-dashboard
53. 53 United States Food and Drug Administration, “MAUDE—Manufacturer and User Facility Device Experience.” https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfmaude/search.cfm
54. 54 D. Church and S. Budsberg, “Plenary topic: Can practice-based evidence complement and promote EBVM?,” presented at the Veterinary Evidence Today, Edinburgh, Scotland, 2016. https://doi.org/10.18849/ve.v1i4.80.
55. 55 A. Coronato, “Engineering high quality medical software: Regulations, standards, methodologies and tools for certification,” in The Institution of Engineering and Technology, London, United Kingdom, 2018.
56. 56 A. Coronato and A. Cuzzocrea, “An innovative risk assessment methodology for medical information systems,” IEEE Transactions on Knowledge and Data Engineering, 2020, pp. 1–1. https://doi.org/10.1109/TKDE.2020.3023553.
57. 57 World Small Animal Veterinary Association (WSAVA), Microchipping—The importance of ISO, 2020. https://wsava.org/wp-content/uploads/2020/01/Microchipping-The-Importance-of-ISO.pdf
58. 58 E. K. Burrows, H. Razzaghi, L. Utidjian, and L. C. Bailey, “Standardizing clinical diagnoses: Evaluating alternate terminology selection,” AMIA Joint Summits on Translational Science Proceedings, 2020, pp. 71–79.
59. 59 Observational Health Data Sciences and Informatics (OHDSI), The Book of OHDSI. San Bernardino, CA: OHDSI, 2019. https://ohdsi.github.io/TheBookOfOhdsi
60. 60 D. Raths, “HL7, OHDSI to collaborate on single common data model: Data standards organizations will integrate FHIR, OMOP models to create a single source for the sharing and tracking of data,” Mar. 1, 2021. https://www.hcinnovationgroup.com/interoperability-hie/standards/news/21212141/hl7-ohdsi-to-collaborate-on-single-common-data-model