# Overview We put forward a framework to articulate what a digital twin is, what it can do, and what it can become. Our framework intends to facilitate wider collaboration and discussion of digital twins across the industry. The objective of the framework is to evaluate the current state of digital twins across five key levels. The levels help the industry to use common language when describing a digital twin and its capabilities. The aim of the framework is to enable participation from our clients and collaborators at all stages of the development. In Figure 4 we present the evaluation framework. The metrics of the framework can be found in Table 2. The four metrics are: autonomy, intelligence, learning, and fidelity. While these four metrics are conceptually correlated, they should be treated independently; as the framework evolves and our understanding of digital twins grows, we can revise the framework and metrics accordingly. In Table 3, the framework moves through five levels, beginning with a simple digital model. As the model evolves, feedback and prediction increase in importance. At higher levels, machine learning capacity, domain-generality and scaling potential all come into play. By the highest levels, the twin is able to reason and act autonomously, and to operate at a network scale (incorporating lower-level twins, for example). Our vision for the built environment is to work collaboratively towards the development and adoption of ‘level 5’ digital twins. Our research has revealed that the progression and development of digital twins is far from reaching level 4 or 5. We are still a long way from an industry landscape populated by reasoning models, machine consciousness, and full autonomy. As digital twins evolve, however, they will control more and more operations, increasing autonomy, intelligence, learning, and fidelity providing value against a backdrop of minimal human intervention. While we considered other metrics that may be applicable such as maturity, we chose not to include it as a key metric. Maturity refers to the developmental stage of the digital twin, rather than its level of complexity. The stages of maturity run from the initial concept, through demonstration and development, and finally to commercialisation. Digital twins can be highly intelligent and highly autonomous, but nevertheless yet to attain maturity. Therefore, maturity was excluded as a key metric. In the following chapter, we apply the evaluation framework using the key metrics through a series of Arup case studies to demonstrate how it is applicable across digital twin projects. # Metrics ![[Arup_Digital_Twin_Figure_4.png]] ## **Autonomy** The ability of a system to act without human input. There are five levels of autonomy. At level 1, there is complete absence of autonomy, with the user controlling all aspects of the digital twin. A level 2 can be understood as user-assisted. At this level, prompts and notifications of system activity are expected, but autonomy is limited. A level 3 has partial autonomy, the twin has the ability to alert and to control the system in certain ways. A level 4 has high autonomy, the digital twin is able to perform critical tasks and to monitor conditions with little to no human intervention. Finally, a level 5 can operate safely in the total absence of human intervention. ## Intelligence The ability of digital twins to replicate human cognitive processes and to perform tasks. There are five levels of intelligence. At level 1, the twin has no intelligence. At level 2, the twin has reactive intelligence (the twin only responds to stimuli, cannot use previously gained experiences to inform their present actions). At level 3, the twin uses learning to improve its response and are also capable of learning from historical data to make decisions. At level 4, the twin understands the needs of other intelligent systems. Finally, at level 5, the twin is self-aware with human-like intelligence and self-awareness. ## Learning The ability of a twin to automatically learn from data in order to improve performance without being explicitly programmed to do so. Through machine learning, a twin classifies aspects of the systems (objects, behaviours) using reinforced learning. There are five levels of learning. At level 1, the twin has no learning component. At level 2, the twin is programmed using a long list of commands. At level 3, the twin is trained using a supervised learning approach (using labelled data able to provide feedback and prediction performance). At level 4, the twin is trained using an unsupervised learning (the twin uses no labels and tries to make sense of the environment on its own). At level 5, the twin uses reinforcement learning by interacting with its environment. With reinforcement learning, the twin learns from past feedback and experiences to find the optimal way to improve performance; the twin uses a reward system for good performance. ## Fidelity The level of detail of a system, the degree to which measurements, calculations, or specifications approach the true value or desired standard. There are five levels of fidelity. At level 1, the twin has low accuracy and can be considered as a conceptual model. At level 2, the twin has a low to medium range of accuracy and can be used to extract measurements. At level 3, the twin has a medium range of accuracy and can be used as a reliable representation of the physical world. At level 4, the twin can provide precise measurements and at level 5, the twin has a high degree of accuracy and can be used in the case of life safety and critical operational decisions. Fidelity, therefore, depends crucially on the requirements of a given asset operator, rather than constituting an absolute property of a digital twin. # Levels ## Level 1 A digital model linked to the real-world system but lacking intelligence, learning or autonomy; limited functionality e.g. a basic model of a map. ## Level 2 A digital model with some capacity for feedback and control, often limited to the modelling of small-scale systems e.g. building temperature sensors which feed information back to a human operator. ## Level 3 A digital model able to provide predictive maintenance, analytics and insights e.g. predicting the life expectancy of rail infrastructure, enabling repairs or replacements before asset failure. ## Level 4 A digital model with the capacity to learn efficiently from various sources of data, including the surrounding environment. The model will have the ability to use that learning for autonomous decision making within a given domain e.g. the model can automatically communicate real-time route recommendations through various modalities (app, signage, radio), allowing drivers to better plan their journey. ## Level 5 A digital model with a wider range of capacities and responsibilities, ultimately approaching the ability to autonomously reason and to act on behalf of users (artificial general intelligence). Intuitively, a level 5 model, such as a model of a neighbourhood in a smart city, would take responsibility for the tasks one would presently expect a human operator to manage, as well as to react to previously unseen scenarios. Another hallmark of this level would be the interconnected incorporation of lower-level twins e.g. take the level 4 example of traffic updates across a network. In a smart city scenario, numerous independent systems work in parallel to provide feedback to a central decision making network to deliver value to city-level leaders.