Automatic Checking of the Correctness of Clinical Guidelines in glare paolo Terenzianib, Luca Anselmaa, Alessio Bottrighib, Laura Giordanob, Stefania Montanib

Introduction

Background

In order to guarantee usability of the program to physicians not expert in Computer Science, in GLARE we aimed at defining a limited set of clear representation primitives, covering most of the relevant aspects of a guideline [15]. We distinguish between atomic and composite actions (plans), where atomic actions represent simple steps in a guideline, and plans represent actions which can be defined in terms of their components via the has-part relation. The has-part relation supports top-down refinement: a guideline itself can be seen as a composite action. Control relations establish which actions can be executed next, and in what order. We distinguish between four different control relations: sequence, controlled, alternative and repetition.

Four different types of atomic actions have been defined as well: work actions (actions to be performed at a certain step of the guideline), query actions (requests for information), decisions (selections among alternatives) and conclusions (explicit output of a decision process). Actions are described in terms of their attributes.
Acquisition and Execution tools.

As in most approaches in the literature, GLARE distinguishes between the acquisition phase (when a guideline is introduced into the system –e.g., by a committee of expert physicians) and the execution phase (when a guideline is applied to a specific patient). Therefore, the system is composed by two main modules, the acquisition tool and the execution tool. The tools strictly interact with a set of databases, including the terminological database (during acquisition) and the patient database (during execution).

The acquisition tool provides a graphical interface to acquire atomic actions, has-part relations and control relations between the components of plans. The guideline is depicted as a graph, where each action is represented by a node (different forms and colours are used to distinguish among different types of actions), while control relations are represented by arcs. By clicking on the nodes in the graph, the user can trigger other windows to acquire the internal descriptions (attributes) of the nodes. The interface also shows the hierarchical structure of the guideline in the form of a tree, where plans can be seen as parents of their components (see figure 1).


Figure 1: A window of GLARE’s acquisition tool graphical interface (concerning part of the gallbladder stones treatment guideline): on the left, the hierarchical structure of the guideline is displayed; on the right, the representation of control relations is shown in form of a graph

The execution tool is typically used “on-line”: a user physician applies a guideline with reference to a specific patient. This method is used for integrating guidelines into clinical practice. Moreover, GLARE is available for “off-line” execution, i.e. for education, critical review and evaluation purposes. The execution tool also provides a decision support facility, which allows physicians navigate through the guideline to see and compare alternative paths (stemming from decision actions).

We have already tested our representation formalism and acquisition tool prototype. Several groups of expert physicians, following a few-hour training session, used GLARE to acquire algorithms concerning different clinical domains (e.g., bladder cancer, reflux esophagitis, and heart failure), with the help of a knowledge engineer. In all the tests, our representation formalism and acquisition tool proved expressive enough to cover the clinical algorithms, and the acquisition of a clinical guideline was reasonably fast (e.g., the acquisition of the guideline on heart failure, starting from a non-structured textual representation, required only 3 days).

Methods

This first type of consistency checking is automatically triggered whenever the expert physician introduces a new term or value within the description of an action in a guideline. The acquisition module strictly interacts with the clinical vocabulary Database in order to provide expert-physicians with a standard terminology, and with a standard range of values for clinical findings. We currently support two modalities for the execution of the acquisition tool: the “safe” and the “advanced” modalities. In the “safe” mode, the expert physician is only allowed to introduce in the guideline terms/values that have already been defined within the clinical vocabulary Database. In order to make this task easier, the acquisition tool provides a means of browsing the clinical vocabulary Database, on the basis of the hierarchical organization of the data it contains. For instance, in the decision “GERD differential diagnosis” one criterion regards duration of heartburn. This datum can be found by browsing the database as follows: patient history (section)  subjective symptoms (class)  specific complaints (category)  heartburn (datum)  duration (attribute). The possible values of this attribute, as stored in the Clinical Database, are “null”, “less than 3 months”, “more than 3 months”.

Thus, in the “safe” mode, the acquisition tool enforces all data and values to be consistent with the dictionary provided by the Clinical Database.

In the “advanced” mode is we also allow expert physicians to introduce new terms/values, that are not already contained within the Clinical Database. In such a case, a warning is shown to the expert physician. If s/he decides to go on, s/he is directly responsible for the correctness of the new term/value.

1. 
   checks that each alternative is preceded by a decision; whenever alternative arcs exit a node, the acquisition module checks that such a node is a decisional one. This check allows for the fact that whenever alternative ways of achieving goals are considered in the guideline, the guideline also contains an explicit way of discriminating between them. This property is important especially at execution time, since, in such a way, the execution tool can provide specific support to the user physicians whenever they have to choose from alternatives;
   
2. 
   checks that decision actions are preceded by query actions specifying all the data involved in the decision criteria. Thus, at execution time, a decision action is executable only after all necessary data for that decision are available.
   

Checking semantic consistency: temporal constraints

In most therapies, actions have to be performed according to a set of temporal constraints concerning their relative order, their duration, and the delays between them. Additionally, in many cases, actions must be repeated at regular (i.e., periodic) times. Furthermore, it is also necessary to carefully take into account the (implicit) temporal constraints derived from the hierarchical decomposition of actions into their components and from the control-flow of actions in the guideline. Checking the consistency of such a set of implicit and explicit constraints is a very hard task, which cannot be performed manually by any expert. On the other hand, within the Artificial Intelligence community, several approaches to perform automatically the propagation of temporal constraints and to check their consistency have been developed [17].

Despite the large amount of valuable works, there still seems to be a gap between the range of phenomena covered by current AI temporal reasoning approaches and the needs arising from clinical guidelines management. In particular, in clinical guidelines,

1. 
   qualitative and quantitative constraints, as well as repeated/periodic events need to be considered at the same time; all types of constraints may be imprecise and/or partially defined;
   
2. 
   a structured representation of complex events (in terms of part-of relations) must be supported, to deal with structured descriptions of the domain knowledge;
   
3. 
   the distinction between classes of actions (e.g. an action in a general guideline) and instances of such actions (e.g., the specific execution of an action in a guideline) has to be supported;
   

Obviously, the interplay between issues (1)-(3) needs to be dealt with, too. For example, the interaction between composite and periodic events might be complex to represent and manage. In fact, in the case of a composite periodic event, the temporal pattern regards the components, which may, recursively, be composite and/or periodic events. For instance, consider Ex.1. In Ex. 1, the instances of the melphalan treatment must respect the temporal pattern “twice a day, for 5 days”, but such a pattern must be repeated for six cycles, each one followed by a delay of 23 days, since the melphalan treatment is part of the general therapy for multiple mieloma.

(Ex. 1) The therapy for multiple mieloma is made by six cycles of 5-day treatment, each one followed by a delay of 23 days (for a total time of 24 weeks). Within each cycle of 5 days, 2 inner cycles can be distinguished: the melphalan treatment, to be provided twice a day, for each of the 5 days, and the prednisone treatment, to be provided once a day, for each of the 5 days. These two treatments must be performed in parallel.

Unfortunately, no current approach in the AI and in the guideline literature proposes a comprehensive approach in which all the above phenomena can be represented, and correct, complete and tractable temporal reasoning can be performed. In GLARE, we define an approach addressing all the above-mentioned issues [1].

A complete automatic treatment of temporal constraints involves, besides the design of an expressive representation formalism, also the development of suitable temporal reasoning algorithms operating on them, to be applied both at acquisition and at execution time. However, subtle issues such as the trade-off between the expressiveness of the representation formalism and the tractability of correct and complete temporal reasoning algorithms have to be faced in order to deal with temporal constraints in a principled and well-founded way; few works in the area of computerized guidelines have deeply analyzed this topic so far.

As a starting point, we have chosen to rely as much as possible on STP (Simple Temporal Problem) [2], a well known and consolidated Artificial Intelligence approach coping with different types of temporal constraints. However, we had to extend it, in order to cope with the before-mentioned additional temporal issues. Specifically, We have chosen to model the constraints regarding repeated actions into separate STPs, one for each repeated action. Thus, in our approach, the overall set of constraints between actions in the guideline is represented by a tree of STPs (STP-tree henceforth). The root of the tree (node N1 in the example in Fig. 2) is the STP which homogeneously represents the constraints (including the ones derived from the control-flow of actions in the guideline) between all the actions in the guideline (e.g., in N1, the fact that the duration of the chemotherapy is 168 days), except repeated actions. Each node in the tree is an STP, and has as many children as the number of repeated actions it contains. Each edge in the tree connects a pair of endpoints in an STP (the starting and ending point of a repeated action) to the STP containing the constraints between its subactions, and is labeled with the list of properties describing the temporal constraints on the repetitions. For example, in Fig. 2, we show the STP-tree representing the temporal constraints involved by the example Ex. 1.

Figure 2: STP-tree for the multiple mieloma chemotherapy guideline. Thiny lines and arcs between nodes in a STP represent bound on differences constraints. Arcs from a pair of nodes to a child STP represent repetitions. Arcs between any two nodes X and Y in a STP of the STP-tree are labeled by a pair [n,m] representing the minimum and maximum distance between X and Y.
In order to check the consistency of the STP-tree, it is not sufficient to check the consistency of each node separately. In such a case, in fact, we would neglect the repetition/periodicity information. Temporal consistency checking, thus, proceeds in a top-down fashion, starting from the root of the STP-tree. Basically, the root contains a “standard” STP, so that the Floyd-Warshall’s algorithm can be applied to check its consistency (as shown in [2]). Thereafter, we proceed top down towards the leaves of the tree. For each node in the tree, we first check that the constraints on the arcs, considereds alone, are consistent. If so, we then merge such constraints with the constraints in the son node, and propagate the resulting constraints to verify the joint consistence. We also formally proved the following.

Property. Our algorithm to check the consistency of STP-trees is correct, complete, and operate in polynomial time.

Checking semantic “logical” consistency

Besides the property of being temporally consistent, several other semantic properties (e.g., logical consistency, safeness) should be checked on clinical guidelines. Specifically, we have identified four different classes of properties relevant in the clinical guideline context, and we have proposed a general and task-independent approach to check all of them.

1. 
   Properties concerning a guideline “per se”. One can check if the guideline contains a path of actions satisfying a given set of conditions (e.g., a path including actions X, Y and Z, or a path in which no action of type X is executed, or a path nor requiring a given laboratory test, or a path requiring only a given set of resources, and so on);
   
2. 
   Properties of a guideline in a given context. Specific contexts of execution may impose several limitations on the executable actions of guidelines, related, e.g., to the lack of certain resources (e.g., laboratory instruments). The consequences of such limitations may be automatically investigated. For instance, one can check whether there is or not a therapy for a patient affected by a given disease, in the case a specific set of resources is available (not available).
   
3. 
   Properties of a guideline when applied to a specific patient. For instance, the feasibility of a given action or path of actions on the specific patient can be checked.
   
4. 
   Integrated proofs. Any combination of the above types of checks can be performed. For instance, one may ask whether, given a patient with a specific disease and set of symptoms, and given an hospital with a specific set of resources, there is a path in the guideline which applies to the patient and satisfies a given set of properties.
   

In our approach, we provide a general way of proving all the above properties by loosely coupling GLARE with the model checker SPIN [6]. Roughly speaking, we have devised a tool to map clinical guidelines acquired by the GLARE system into the Promela language, which is the language used by the SPIN model-checker. Promela allows a high level model of a distributed system to be defined by modelling each agent in an extended pseudo-C code, including synchronization primitives and message exchange primitives. Specifically, GLARE’s guidelines are translated into a set of agents (e.g., the agents representing the guideline actions, the agent representing the physician executing the guideline, and so on). Once we have the translation of GLARE’s guidelines in Promela code, we can use SPIN as a general-purpose engine to prove any property that can be expressed in the temporal logic LTL. In fact, SPIN translates each process (each agent) into a finite automaton, and the global behaviour of the system is obtained by computing an asynchronous interleaving product of automata. The resulting automaton represents the global state space of the system and can built on-the-fly during the verification process. The property which has to be verified on the system is passed to the verifier through an interface, which maps it into a temporal formula, as required by SPIN. SPIN converts the negation of the temporal formula into a Büchi automaton and computes its synchronous product with the system global state space. If the language of the resulting Büchi automaton is empty then the property is true on all the possible execution of the system, otherwise the verifier provides a counterexample for the property (an execution path on which it is false).

In such a way, we provide a general-purpose approach to automatically check all the types of properties discussed above.

Results

GLARE provides a set of facilities to help physician during the acquisition of clinical guidelines, and to verify a-posteriori the correctness of guidelines being represented. To the best of our knowledge, no other guideline system in the literature provides such a large set of facilities. GLARE’s facilities have proven to be quite effective in our testing activities. In particular, in the cases in which a textually written guideline had to be entered into the GLARE systems, the adoption of GLARE facilities allowed us to detect different kinds of errors in the original guidelines. In certain cases, these errors were simply due to omissions (e.g., lack of a decision action to discriminate between alternative paths of actions). In other cases, they were due to the human impossibility to propagate constraints (and, in particular, temporal constraints) along long paths of actions. In all cases the physician experts agreed that the errors detected by the GLARE systems were “genuine” errors in the original guidelines, and agreed to correct them.

Acknowledgments

We acknowledge Prof Gianpaolo Molino and Dr. Mauro Torchio of Azienda Ospedaliera S. Giovanni Battista, Turin, Italy for their cooperation in defining and testing the GLARE system. The research reported in this paper has been partially supported by a grant from Koine Sistemi, Torino, and by PRIN’06.

References

1. L. Anselma, P. Terenziani, S. Montani, A. Bottrighi. Towards a Comprehensive Treatment of Repetitions, Periodicity and Temporal Constraints in Clinical Guidelines. Artificial Intelligence in Medicine, 2006, 38(2): 171-195.

2. R. Dechter, I. Meiri, J. Pearl, Temporal Constraint Networks, Artificial Intelligence , 1991; 49: 61-95.

3 J. Fox, N. Johns, A. Rahmanzadeh, R. Thomson, Disseminating medical knowledge: the PROforma approach, AI in Medicine, 1998; 14: 157-181.

4 L. Giordano, P.Terenziani, A. Bottrighi, S. Montani, L. Donzella, Model Checking for Clinical Guidelinesç an Agent-Based Approach, Proc. AMIA 2006, Washington D.C.; November 2006. p.289-293.

5. C. Gordon and J.P. Christensen, eds., Health Telematics for Clinical Guidelines and Protocols (IOS Press, Amsterdam); 1995.

6. G.J.Holzmann, The SPIN Model Checker. Primer and Reference Manual. Addison-Wesley; 2003

7. JAMIA, Focus on Clinical Guidelines and Patient Preferences, JAMIA, 1998; 15(3).

8. Special Issue on Workflow Management and Clinical Guidelines, D.B. Fridsma (Guest ed.), JAMIA, 2001; 22(1):1-80.

9. M.A. Musen, S.W. Tu, A.K. Das, and Y. Shahar, EON: A component-based approach to automation of protocol-directed therapy. Journal of the American Medical Information Association 1996; 3(6): 367-388.

10. L. Ohno-Machado, J.H. Gennari, S. Murphy, N.L. Jain, S.W. Tu, D.E. Oliver, et al., The GuideLine Interchange Format: A Model for Representing Guidelines, JAMIA , 1998; 5(4):357-372.

11. M. Peleg, A.A. Boxawala, et al., GLIF3: The evolution of a Guideline Representation Format, in: Proc. AMIA Annual Symposium; 2000.

12. S. Quaglini, M. Stefanelli, A. Cavallini, G. Miceli, C. Fassino, and C. Mossa, Guideline-based careflow systems, Artificial Intelligence in Medicine, 2000; 20(1):5-22.

13. Y. Shahar, S. Mirksch, P. Johnson, The Asgaard Project: a Task-Specific Framework for the Application and Critiquing of Time-Oriented Clinical Guidelines, Artificial Intelligence in Medicine, 1998; 14:29-51.

14. R.N. Shiffman, B.T. Karras, A. Agrawal, R. Chen, L. Menco, and S. Nath, GEM: a proposal for a more comprehensive guideline document model using XML, JAMIA, 2000; 7(5):488-498.

15. P. Terenziani, G. Molino, M. Torchio. A Modular Approach for Representing and Executing Clinical Guidelines. Artificial Intelligence in Medicine 23 (2001) 249-276.

16. S.W. Tu, M.S. Mark, A. Musen, A Flexible Approach to Guideline Modeling, in: Proc. AMIA’99; 1999. p. 420-4.

17. L. Vila, A Survey on Temporal Reasoning in Artificial Intelligence, AI Communications, 1994; 7(1):4-28.

Please address all correspondence to Paolo Terenziani

Prof. Paolo Terenziani,

Dipartimento di Informatica, Universita’ del Piemonte Orientale “Amedeo Avogadro”

Via Bellini 25\g, 15100 Alessandria, Italy

Email: terenz@mfn.unipmn.it -- Phone: +39 0131 360174