We reviewed the source article and the most recent publications that we could locate for each assessment instrument included. If the instrument you are reviewing does not have a value on a given characteristic, this means that either 1) this characteristic is not relevant for that type of assessment (e.g., norms for qualitative assessment guides) or 2) we were unable to find this information in the published source article or most recent publication we found.

It is important to note that this does not mean that data on a specific characteristic do not exist as we did not conduct a comprehensive literature search on each instrument or contact the developer of each instrument to locate all metadata fields. If this information is important to you, we recommend that you conduct an updated more comprehensive search or contact the developer of the instrument.

In addition to this repository, there are other web-based compilation of assessment instruments for D&I research. A few of these include:

• The Society for Implementation Research Collaboration Instrument Repository

https://societyforimplementationresearchcollaboration.org/measures-collection/

• The Implementation Outcomes Repository

https://implementationoutcomerepository.org/

• Washington University in St. Louis Measuring Health Policy Implementation Database

https://www.health-policy-measures.org/

Several characteristics are usually relevant for deciding upon which assessment instrument to use. First, the team needs to ensure that the project has a clear research question and stated outcomes, and a TMF (or combination thereof) that supports the question and outcomes. In selecting an appropriate assessment instrument, it is important to find one that fits closely with the key constructs in your selectedTMF. You should also consider the assessment instrument’s appropriateness for a specific context or with the population of interest. What instruments are ‘best’ depends upon the question you are trying to answer and often pragmatic factors such as time to administer, costs, burden, and reliability are important. We recommend that 1) in collaboration with your community or clinical partners you identify which characteristics should be prioritized in making your decision and 2) whenever possible that you include more than one instrument, and potentially more than one type of instrument (e.g., quantitative and qualitative) if appropriate. Ultimately, this selection process includes balancing feasibility and  scientific rigor in order to select the best instruments for your project.

There are a variety of places to get information on these tools. Often, the tool has a paper or reference that provides information and the information for the tool development team. Additionally, some of these tools have websites or are located within other toolkits that can be found online.

For each assessment instrument, we provide information on key characteristics (i.e., metadata) that can help you decide which assessment instruments will work for your project. Characteristics are organized by 6 domains:

1. General Overview: Brief Description, Citing literature (original/application), Website, Version, Attachments, & Related instruments
2. Summary Statistics: Qualitative/quantitative, Type of instrument, Number of items, & Sub-scale language
3. Implementation Science Considerations: Constructs, TMFs, Strategies, & Implementation Science outcomes
4. Intended Focus: Levels of data collection, Intended priority population, Intended setting, Policy, & Equity
5. Psychometric Properties: Scoring, Norms, Responsiveness, Reliability, Validity, & Factor analysis
6. Pragmatic Characteristics: Time to administer, Secondary data, Cost, Literacy, Interpretation, Training, Resources, User guidance, Obtrusiveness, & Interactivity

To learn more, go to the View Characteristics for D&I Assessment Instruments tab.

1)  The type of data collection event to be performed (e.g., individual interview, focus group interview, observation). 

2)  For interviews/focus groups, emphasis should be placed on using easily understood open-ended questions with attention given to their sequencing and timeframe (e.g., present, past, future). Implementation studies typically use semi-structured guides that reflect the topics/subject areas of particular interest and TMF constructs. (Patton, 2014 provides a nice overview of the specific types of questions that one might ask; questions should also reflect the type of evaluation being performed, i.e., formative vs. summative.)

3) For observations, consider protocols/guidelines/templates that specify particular foci and approaches for documenting observations.

4) The extent to which the instrument is tailored to the context in which the work is being carried out (e.g., clinic, school, other community setting) and participant type (e.g., system leader, organizational administrator, direct service provider, staff member, client/patient/service recipient). This is especially important when exploring implementation phenomena at multiple levels.

For guidance on this (including information in the appendices) please see Lengnick-Hall et al, 2023  https://link.springer.com/article/10.1186/s13012-023-01302-2. 

An overarching key concept is that regardless of which instrument is selected, the resulting data is only as good as how the instrument was ultimately applied. This is where the skills of the interviewer, observer, or other type of fieldworker becomes paramount.

A general overview of qualitative methods in implementation science is available from the white paper entitled National Cancer Institute: Qualitative Methods in Implementation Science White paper commissioned by the Implementation Science Division of Population Health at the National Cancer Institute https://cancercontrol.cancer.gov/IS/docs/NCI-DCCPS-ImplementationScience-WhitePaper.pdf

A follow-up webinar discussed advancements in the field since the publication of the white paper:

https://www.youtube.com/watch?v=Mxj7s2Fe89w

Commonly used methods and related publications and resources for qualitative methods in implementation science include:

Rapid Qualitative Approaches:

• Hamilton AB: Rapid Qualitative Analysis: Updates/Developments https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=3846 https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/3846-notes.pdf
• Hamilton, A. B., & Finley, E. P. (2019). Qualitative methods in implementation research: An introduction. Psychiatry research, 280, 112516. https://doi.org/10.1016/j.psychres.2019.112516
• Hamilton, A., Fix, G., Finley, E. (forthcoming). Pragmatic Healthcare Ethnography. Routledge. https://www.routledge.com/Pragmatic-Healthcare-Ethnography-Methods-to-Study-and-Improve-Healthcare/Hamilton-Fix-Finley/p/book/9781032487601?srsltid=AfmBOoo9oHcyiqxkFE3gYrrXQr73-dJorZ16xP6_p36j77TOiiYpV9T4
• St George, S. M., Harkness, A. R., Rodriguez-Diaz, C. E., Weinstein, E. R., Pavia, V., & Hamilton, A. B. (2023). Applying Rapid Qualitative Analysis for Health Equity: Lessons Learned Using “EARS” With Latino Communities. International journal of qualitative methods, 22, https://doi.org/10.1177/16094069231164938. PMID: 38463016 PMCID: PMC10923582. 

Ethnographic approaches:

• Gertner, A. K., Franklin, J., Roth, I., Cruden, G. H., Haley, A. D., Finley, E. P., Hamilton, A. B., Palinkas, L. A., & Powell, B. J. (2021). A scoping review of the use of ethnographic approaches in implementation research and recommendations for reporting. Implementation research and practice, 2, 2633489521992743. https://doi.org/10.1177/2633489521992743. PMID: 34056611 PMCID: PMC8153409
• Palinkas, L. A., & Zatzick, D. (2019). Rapid Assessment Procedure Informed Clinical Ethnography (RAPICE) in Pragmatic Clinical Trials of Mental Health Services Implementation: Methods and Applied Case Study. Administration and policy in mental health, 46(2), 255–270. https://doi.org/10.1007/s10488-018-0909-3. PMID: 30488143 PMCID: PMC7427407.
• Rabin, B. A., Cain, K. L., Salgin, L., Watson, P. L., Jr, Oswald, W., Kaiser, B. N., Ayers, L., Yi, C., Alegre, A., Ni, J., Reyes, A., Yu, K. E., Broyles, S. L., Tukey, R., Laurent, L. C., & Stadnick, N. A. (2023). Using ethnographic approaches to document, evaluate, and facilitate virtual community-engaged implementation research. BMC public health, 23(1), 409. https://doi.org/10.1186/s12889-023-15299-2. PMID: 36855118 PMCID: PMC9974043.