Developers - Generating evidence for NICE health technology assessment

If you want NHS England to adopt your digital technology, undergoing a NICE health technology assessment can help. Here is what you need to consider.

NICE health technology assessment

The National Institute for Health and Care Excellence (NICE) provides guidance to NHS England on the clinical and cost effectiveness of selected new and established digital technologies. To do so, NICE carries out appraisals of digital technologies at the request of the Department of Health and Social Care.

If your digital technology is eligible for a NICE evaluation, you’ll need to generate evidence that supports your value proposition. Doing so will increase the likelihood of a positive recommendation by NICE. This will increase the likelihood that the NHS will adopt your technology.

Evidence for NICE health technology assessment

You’ll need to show NICE that your digital technology is:

• clinically effective
• safe, and
• cost effective

Some of this evidence will be identical to the evidence required to get a UKCA mark.

However, as a general rule, evidence to get a UKCA mark is more focused on the safety and efficacy of your digital technology. Efficacy evidence determines whether your technology produces the expected result under ideal circumstances.

On the other hand, evidence for NICE or adopters is more focused on relative effectiveness. Relative effectiveness evidence measures the degree of beneficial effect compared with the current standard of care. So you will need to prove that your digital technology is effective, in comparison with current routine practice. The key is to make sure your study reflects what happens in current clinical practice as closely as possible. For example, you should aim to recruit a population that closely matches the population who would use your technology in routine practice. And when selecting sites in which to test your technology, make sure you select ones that reflect established practice in the wider health and care system.

Proving your digital technology is clinically effective

If your technology is aimed at improving patient outcomes, it is essential to collect data on the most important and relevant outcomes for patients. Relevant outcomes are those that measure how a patient feels, functions or survives. The choice of outcomes to measure is also specific to the purpose of a technology. For example, a technology aimed at diagnosing cancer may need to collect longer-term patient outcomes such as cure rate or progression-free survival. You’d also need to gather diagnostic accuracy data for the technology.

A key element of clinical-effectiveness and safety data is having a long enough follow-up time to capture all relevant outcomes. For example, if your technology is aimed at improving survival in cancer patients, your study may need longer-term follow up. Or it may need to capture appropriate intermediary outcomes that can predict longer-term survival.

The quality of a study’s overall design, how it is done and its validity will also be reviewed by NICE and adopters. Critical appraisal is usually done to assess the quality of your evidence.

Typically, evidence will be critically appraised in line with the relevant risk-of-bias tool for that study design. For example, the Cochrane risk-of-bias tool for randomised trials.

See using PICO to generate evidence for AI development for further guidance on generating appropriate evidence.

Proving your digital technology is cost effective

You will also need evidence of different levels of cost effectiveness. This will depend on:

• the type of digital technology
• how a digital technology compares to current practice

Digital technology: same or better performance at similar or lower cost

You will need to do a cost-consequence analysis if your technology offers:

• the same clinical effectiveness as current practice
• better performance than current practice but at a similar or lower cost.

This type of analysis will set out all the relevant costs and consequences of using your technology. It also estimates the resource use and clinical benefits of your technology compared with current practice.

Digital technology: better performance at higher cost

If your technology offers better performance at a higher cost, a cost-utility analysis (CUA) is needed.

Processing data on costs and resourcing of your technology is essential regardless of the type of economic evaluation that will be needed.

How to generate evidence for a NICE health technology assessment

Generate your evidence in line with NICE’s evidence standards and methods guides, such as:

• NICE's health technology evaluations manual
• NICE's evidence standards framework for digital health technologies
• NICE's real-world evidence framework

Consider using NICE Advice. NICE Advice can help you demonstrate the value of your product and make valuable NHS connections, speeding up patient access to the best treatments and care.

NICE will also consider the quality of the data you have used when training, testing and validating your algorithms. See data quality considerations when training, testing and validating your algorithms for more information.