A maturity rating appears on a steering slide without a mention of what was tested, in what conditions, or by whom, and the group is requested to release the next tranche of spending on the strength of it. That single figure is being asked to carry an enormous amount of weight, and this page is about the knowledge it stands for: how that knowledge is built, how it is tested, and how it is kept true as the system it explains changes.
A rating is a claim about knowledge
A maturity level does not describe a device, a material or a piece of software directly. This shows how much has been proven about that subject, under conditions that resemble the conditions of use, and how much remains unknown. MITRE's AiDA reference on technology readiness levels puts this well: The levels serve as a helpful knowledge-based standard and shorthand for evaluating technology maturity, though they must be supplemented with expert professional opinion. The same reference sets out the supporting information each level ought to carry, and that list is not about the technology at all. The test log states who performed the tests, where and when, how the test environment differed from the operational environment, what problems were encountered, and what is planned to resolve them.
The GAO's Technology Readiness Assessment Guide frames the same idea from the buyer's side. It identifies four characteristics of a high-quality assessment, namely credibility, objectivity, reliability and usefulness, and it observes that relatively few federal agencies have any guide at all for judging whether a technology is mature enough to be integrated. It also records the cost of making an error: technologies that were not as mature as recommended have been the source of programme delays and cost increases. A rating that is unable to answer the questions on the supporting information list is not an assessment of maturity. It is an assessment of the presentation.
The environment is part of the digit
A level has no meaning without the environment attached to it, and this is the point where most second-hand ratings start to mislead. NASA's explanation of technology readiness levels shows the progression in concrete terms: a working model or prototype demonstrated in a space environment occupies a level well short of the top, and only a system proven during a successful mission reaches the highest one. Two closely related versions of the same hardware can sit at different points on the scale purely because of where they were exercised.
Sector funders make the same point explicitly. The Aerospace Technology Institute, which manages the ATI Programme, states that TRL definitions can be subjective and the scale imprecise, and that it therefore uses them as a guideline. The guidance says that a full-scale flight demonstration is typically closer to TRL 7 than TRL 6, but that this depends on the technology being tested, the system it forms part of, and the objectives of the demonstration. If a funding body says that the limits between adjacent levels are subjective, you should describe the operating environment in your own words before anyone argues about the score.
Why the least mature element sets the system figure
A single level awarded to a whole system conceals the structure underneath it. A system performs only as well as its least mature critical element, because that element is the part that will fail to be demonstrated when it matters. This is the mechanism behind the GAO finding about delays and cost increases: the programme proceeds on the strength of a composite figure while one critical element is still short of the evidence its position demands. The practical test is therefore not the system level itself but the identity of the elements in scope, and which of them is furthest from being demonstrated in the environment of use. The composite figure shows what was available when the scope followed the evidence gathering.
Configuration and time are part of the fact.
A maturity level is a particular build, in a particular setting, at a particular time. Changing the supplier, the version, the operating profile, the data feeding it or the team managing it means the identifier now refers to an entity that no longer exists. Two facts belong in the same phrase as the level: The identity of the configuration that was assessed, and the time of the most recent demonstration behind it.
A worked example of that movement appears in IDC's account of its product development work, where a hands-free breast pump programme began at a point at which the general technology was proven but not in the configuration needed for comfort and convenience, and ended with the product approved and in production. The technology did not change character; the configuration did, and the level moved with it. Anyone quoting the final figure for the earlier configuration would be describing a device that was never built.
Maintain a maturity log that survives examination
Maintaining maturity is mostly clerical work, and most of it happens before the demonstration. Describe the operating environment in plain terms and agree in advance what would be an acceptable demonstration within it. Log who ran the demonstration, when, what differed from expectation, and which questions remain open. The log should highlight the distinct questions answered by a development team demonstration in its own facility versus a trial by site operators at the operational location.
Then set a trigger list for re-opening the case. A change of configuration, a change of environment, a change of the accountable team, or a scheduled review should each send the log back to the evidence. Keep the earlier records when you update them. The history shows which elements have been retested since the last decision, and a rating without that history is a snapshot that lacks a time stamp.
Interrogating a rating you did not produce
When a supplier, a partner or a funding body hands you a level, a short set of questions applies, and the answers should be findable in the document. Which environment did the demonstration use, and how does it differ from the environment of use? Which configuration was assessed? Who ran the test? What has changed since? What does the level not cover?
The product development literature is honest about how much weight a single prototype has. EnCata, writing about technology readiness levels in new product development, recommends against closing the concept stage with mockups, looks-like prototypes or animations, and argues instead for a proof-of-concept prototype that demonstrates the key function and allows the main technical risks to be tested. That is the correct instinct applied to a pitch: A demonstration that exercises the risky part is worth more than a polished object that exercises the safe part.
Where organisational maturity models belong
Organisational maturity models answer a neighbouring question, and it is a mistake to let the two answer each other. MoldStud's guide to digital workplace maturity models uses assessments built from performance indicators, employee surveys, analysis of how digital tools are actually used, and reviews of existing processes. Those tools assess whether an organisation can absorb, manage and sustain a change. A technology maturity level evaluates whether the target state has actually been demonstrated. Both belong in the file, and they belong at different moments: the organisational model is properly used before an adoption plan is committed, and the technology rating before the authorisation is signed. Substituting one for the other produces a figure that settles no dispute, because each faces a query its method fails to answer.
The conditions a usable maturity level must meet
Collect the thread and it flows in one direction. A usable premise identifies the decision it will inform and the individual who will make it. The scope defines the area being assessed and the elements within it. The operational environment is defined before any demonstration is planned, along with the criteria for evidence. It gathers that evidence from live work where the operating team is involved. It scores only what was demonstrated and places the residual unknowns next to the score. The configuration identity and a review time are included, and it lists what would invalidate it.
A level produced this way is not a better value than the one on the slide. It is a different kind of object: A timed, limited, verifiable log of known facts allows a reader who was not present to trace the information back to the evidence. That is the whole of the practical value. Each action a maturity figure performs in a steering meeting, from releasing funds to deferring a launch, depends on the ability to trace it back to what was actually shown.
Sources
- Technology Readiness Assessment Guide: Best Practices for Evaluating the Readiness of Technology for Use in Acquisition Programs and Projects
- Technology Readiness Levels
- Technology Readiness Levels (TRL)
- Technology Readiness Levels (TRLs)
- Technology Readiness Levels (TRLs) and How They Fit with IDC's Process
- Technology Readiness Levels: Assessing Technological Maturity
- Assessing Your Organization's Readiness: Digital Workplace Maturity Models Explained
they must be supplemented with expert professional judgment
