Outcomes/Output Monitoring and Tracking, Impact Reporting
Post-application and The Award Process

Tracking Researchers

Public Access Post-Award: Funded researchers are required to submit, or have submitted for them, to the National Institutes of Health’s Pub Med Central database an electronic version of the author’s final manuscript including all modifications resulting from the publishing and peer review process (the “postprint”) upon acceptance for publication, to be made publicly available no later than 12 months after the official date of publication.  This requirement applies to all publications related to funded research grants whether the research was funded in whole or in part. All scientific progress reports must include the PMC ID number (PMCnnnnn) for publications, if known at the time of the report due date.

Tools and Resources

ORCID

ORCID stands for Open Researcher and Contributer Identifier and is an alphanumeric code assigned to contributing members of the research community. This tool acts as a unique identifier for authors and contributors to be tracked over time. The ORCID is useful for consolidating and streamlining all of one’s scholarly activities which include their publications, patents, and affiliations.

Funders can utilize the ORCID to track the output and as well as the impact of their grantees to evaluate grant programs.

The ORCID also plays a role in strengthening connections and in maintaining interpersonal relationships between researchers and funders.

Lightoller is a O RCID-based tool developed by Gary McDowell, Ph.D. to track an awardee’s progress. This is built on long-term reliance of ORCID

Funding systems can sign into platforms such as ProposalCentral with ORCID login credentials or there is an option where an ORCID can be accessed and inputted (i.e SciENcv).

A benefit of the ORCID is that funding sources can be manually updated to an ORCID from both the grantee and the funder.

CrossRef

CrossRef is a registration agency operated by a not-for-profit member organization that registers DOI identifiers which are used to log and update the citations for a published body of work.

Funder ID (Fundref) is an identifier to disambiguate the funding organization.

Grant IDs is a unique identifier to disambiguate specific grants and awards.

Directly Following Up

Some organizations will continue tracking past grantees as well as the progress of fellows for a number of years (i.e. 5 years) after the award window ends by assessing grant deliverables.

Tracking publications and other dissemination

Tools and Resources

Dimensions is a database that aims to link datasets which include publications, clinical trials, and grants by utilizing an evaluation metrics matrix. By linking these datasets, it provides a connected, uniform platform for users to view various types of data-related information. Dimensions can be integrated with ProposalCentral.

OpenAlex is a new tool that is free and open-source with a focus on open source data.

Web of Science is a publication-focused search tool that can search based on funding agency, and it can be used to find or follow others working in a specific field. This product is charged based on the period of usage and per user(s). Members have reported approximately $10,000 per year for access.

Researchfish is a tool that focuses on data gathering, research tracking, and measuring impact. This aids in supporting impact evaluations for organizations.

 

Drug tracking

If drugs have been approved from funded projects, tracking can be done from clinical trials to market, although the timeline for this can be quite long, unpredictable, and difficult to follow due to the extensive periods of time involved.

Communicating Impact

Tools and Resources

HRA has held webinars on the importance of impact for funders:

  • Understanding Visual Intelligence & Data Visualization Webinar
  • Finding, Evaluating, and Promoting the Impact of your Research for Nonprofit Funders – how to use the new Dimensions functionality Webinar

Program Evaluation

Tools and Resources

[Cross-ref with resources under Program Evaluation Working Group]

Listserv Responses: Evaluation Platforms [May 2018]

  • At the 2022 Fall Member’s Meeting, George Santangelo, Ph.D. of the NIH presented tools developed and used by the NIH to measure impact and productivity:
    • IQRST (Influence, Quality, Reproducibility, Sharing of other data and resources, and Translation) metric is used to evaluate productivity. It can be used as an alternative to measure one’s progression in place of impact factor.
    • iCite is a tool for bibliometrics that is built off of PubMed IDs and machine learning to measure Relative Citation Ratio (RCR) values to infer scientific influence and Approximate Potential to Translate (APT) as a means to measure how an article will translate to clinical research.
    • iSearch is an AI tool that will be launched in the future as a method to connect individuals to certain publications and organizations.

Post-award surveys

[Member templates]

HRA Resources

History

Version
Description
Date
Version
1.0
Description
Initial Population
Date