MD Marie Barais
email: marie.barais@gmail.com

Ercr Spurbo

Johannes Hauswaldt
email: johannes.hauswaldt@med.uni-goettingen.de

Daniel Hausmann
email: d.hausmann@psychologie.uzh.ch

Slawomir Czachowski
email: s.czachowski@torun.home.pl

Agnieszka Sowinska

PhD Paul Van Royen
email: paul.vanroyen@uantwerpen.be

Barais Marie

CS Camille Desmoulins

Van De Wiel

PhD Groell Nicolas

PhD Dany Antoine

PhD Montier Tristan

PhD Van Royen Paul

Jan Geert

Dinant

Margje Van De Wiel
email: m.vandewiel@maastrichtuniversity.nl

PhD Erik Stolper
email: cf.stolper@maastrichtuniversity.nl

Nathalie Morio

Amélie Calvez

MD Pierre Barraine
email: pierre.barraine@univ-brest.fr

MD Florie Scouarnec
email: florie.scouarnec@live.fr

MD Anne Sophie Mauduit

MD Bernard Le Floc'h

PhD Claire Liétard
email: claire.lietard@univ-brest.fr

Emilie Fossard

The LINGUISTIC VALIDATION of THE GUT FEELINGS QUESTIONNAIRE IN THREE EUROPEAN LANGUAGES LINGUISTIC VALIDATION OF THE GUT FEELINGS QUESTIONNAIRE

Keywords: feasibility study, questionnaire, gut feelings, family medicine, general practitioners. Gut Feelings Questionnaire Completely gut feelings, family medicine, general practitioners, diagnostic reasoning, decision making, problem solving, chest pain, dyspnoea, intuition

Background: Before using any prediction rule oriented towards pulmonary embolism (PE), family physicians (FPs) should have some suspicion of this diagnosis. The diagnostic reasoning process leading to the suspicion of PE is not well described in primary care.

Objective: to explore the diagnostic reasoning of FPs when pulmonary embolism is suspected.

The accuracy of the general practitioner's sense of alarm when confronted with dyspnoea and/or thoracic pain: protocol for a prospective observational study

Introduction Part 1 Chapter 2

The linguistic validation of the gut feelings questionnaire in three European languages.

Chapter 1 I was working in the emergency unit as a trainee when my tutor told me this in front of a patient about whom I had just been saying that I felt something was wrong, without being able to explain it using proper arguments, only that I could not help thinking about a pulmonary embolism. I was confused by my inability to explain what gave rise to such feelings. The only sign I had was the way she put her hand on her chest as she breathed loudly when I requested auscultation. My tutor understood my frustration and explained to me that there is indeed a place for intuition in medical decision making. It was the first time somebody had admitted it so frankly. Nobody had told us as students to use our intuition before, and it was puzzling to begin making decisions in the emergency unit with this new "partner" in the diagnostic process.

This was the first patient I had diagnosed as suffering from a pulmonary embolism. When I thought about this case, I was not satisfied with the word "intuition". It seemed vague and more likely to have been taken from a women's magazine than from a scientific article! My knowledge in this area was very limited, and closer to French common sense, epitomised by the Descartes quotation "I think therefore I am". After reading Damasio, I understood Descartes should have written "I feel therefore I am" and, that being the case, I would have abandoned my preconceptions about decision making earlier 1 ! So I began to read about intuition and the kind of place it has in medical decision making.

How to define intuition?

The common definition of intuition is "knowing or understanding without conscious use of reasoning" 2,3 . In cognitive science, there are many definitions of the word "intuition" and each definition is connected to a different concept. For Kahneman, a famous psychologist who received the Nobel prize for his work on the estimation of probabilities in decision making, intuition is an "informal and unstructured mode of reasoning" 4 . We have chosen the categorisation of Glöckner and Witteman who described "associative, matching, accumulative and constructive" forms of intuition, based on the way information is thought to be stored in memory and the type of integration process and retrieval from the simple to the more complex 5

. These four types are complementary and sometimes overlap because of different levels of abstraction.

Associative intuition results from the impact of an event on a person's feelings. By being exposed to a phenomenon, a person records elements, consciously and unconsciously, which become associated with certain somatic markers 1 . The positive or negative value attached to this marker will be activated if the person has to make a decision. Classical conditioning, evaluative conditioning, social learning and implicit learning are based on this associative and affective intuition. This type of intuition is used in advertising: if you recognized something in a supermarket from a TV advertisement which held a positive affective association for you, there is a strong likelihood you would choose this item in preference to a similar item you did not recognise.

The matching form of intuition is very close to the associative and adds a level of complexity in the description of the patterns stored in memory. Cues are generated by a situation and they are compared to the patterns experienced by the individual. This pattern matching will activate the script of an action. Klein studied the decisions made in the real-life context of fire commanders 6 . The analysis of rescue narratives revealed that the commanders were able to make a decision in less than one minute. They recognized in the case some familiarities with previous situations. Klein told the story of a firefighter captain who led his men into a burning house. He ordered the crew to evacuate suddenly, one minute before the floor collapsed. He understood the danger was very close when he noticed that the water applied to the apparent seat of the fire did not have the expected effect and that it was getting intensely hot and very quiet. The real fire had been in the basement 7 . The commander mentally simulated the strategy applied previously and chose the one which fitted the actual case. Klein described the model related to this theory as the Recognition Primed Decision (RPD) 6 .

The accumulative form of intuition is mathematically specific: memory traces or perceived experiences are accumulated following quick automatic processes. When the evidence favouring one option reaches a threshold, that option is selected. It is a mathematical model and precise predictions can be made. Busemeyer et al. took the example of jury members who have to decide a penalty program for a young offender convicted of a serious crime 8 . They have to make a decision from three options: the length of imprisonment, the other people imprisoned with him and the possibility of parole. Depending on the scenario (5 years with inmates with minor convictions and the possibility of parole in 2 years, or 30 years with hardcore criminals and no possibility of parole) jury members have to deliberate by thinking about the possible consequences of each possibility.

The constructive form of intuition is linked to the construction of mental representation formalised as networks of information, both furnished by the situation and activated within memory. During this process, one alternative will be represented with one advantage compared to other alternatives, and the disadvantages of the selected alternative will be neutralized: this is the dominance structure described by Montgomery 9 . When a mechanic listens to the sound of a faulty engine, he has to visualise which part of it could be broken: the visualisation, constructed according to his comprehension of the problem, will be compared to the previous experiences he has had and the most effective alternative will be automatically selected.

These four variations of the same word "intuition" summarize the main concepts linked in cognitive theories. Hogarth chose to present these concepts in connection with the timeline of the inferences involved: when the inferences look forward in time, they are concerned with discovery and prediction, whereas when the inferences look backward in time, they are concerned with justification 10 . Whatever the type of presentation, "intuition" is a generic term which is difficult to use in relation to diagnostic reasoning. It was difficult to connect one of these definitions to what I felt in front of the patient who was suffering from a pulmonary embolism. Could other terms have helped me to find out how this idea became self-evident?

In the field of research into medical decision making, several terms for "intuition" co-exist in parallel to define this non-analytical part of the process. The most common are the "first impression", the "heuristics", the "rules of thumb", the "clinical mind lines", then the "gestalt" and the "system 1" 11 . What do they have in common and how are they different from the concept of intuition?

"First impressions" are unavoidable when meeting a patient: as soon as the GP sees the patient in the waiting room, data concerning his/her health status are recollected. The way he/she stands up, the colour of his/her complexion are automatically noted by experienced GPs. From these first impressions, one diagnosis may emerge based on case recognition 12 .

Relying on first impressions is merely described as a source of errors in literature [13][14][15] . For instance, sticking to initial impressions is called the anchoring effect. Stopping reasoning too early and neglecting other alternatives is a premature closure error.

Linked to first impressions is the term "heuristics", which is defined as "simple rules in the mind's adaptive toolbox for making decisions with realistic mental resources" 16 . Several concepts are highlighted under this term, depending on different authors 5 . For Tversky and Kahneman, heuristics are useful in reducing uncertain and complex tasks, but also responsible for three severe and systematic errors: the bias of representativeness, availability and anchoring 17 . For instance, during an influenza epidemic, if a patient presents fever and aches, I will not think of paludism but of influenza because I am very familiar with this diagnosis at that time. The use of heuristics is unconscious and automatic according to Tversky and Kahneman 17,18 . For Gigerenzer, the use of heuristics is, on the other hand, efficient in the control of information searching, stopping the search, and making decisions 16 . Slovic and Finucane focused on the affect heuristic where the feeling of "badness" or "goodness" is associated, consciously or unconsciously, to a stimulus 19 . They stress the two sides of this kind of feeling: a very positive one because of its speed and sophistication and a negative one dependent on experience and possibly manipulated. This affect heuristic corresponds to the associative form of intuition described by Glöckner 5 .

The "rules of thumb" are synonymous with heuristics for some authors 16,20 . Nevertheless, the rules of thumb are associated with general rules, built from the experience of the physician or by word of mouth from a colleague. It forms tacit knowledge as it is immediate knowledge which is not really consciously acquired 20 . The rules of thumb seem to be easier to formalise and therefore different from heuristics 21 .

The "clinical mind lines" are close to the "rules of thumb" as defined by Gabbay 22 . Through an ethnographic study, he describes how GPs never use guidelines during their practice but mind lines which incorporate both explicit and tacit knowledge, based on the personal experience of the physician and shared reflection with colleagues, opinion leaders, and readings 23 .

"Gestalt" is also linked to first impressions and the way clinical data are brought together. The holistic approach is emphasized instead of the atomistic approach: according to gestalt theory, the physician perceives the situation globally with visual, auditory, tactile, or olfactory components in a top down fashion. The process is considered as "automatic-intuitive" and might lead to the same systematic errors as heuristics: representativeness, availability and anchoring 5 . The concept of gestalt has been studied, in particular, in the emergency context with acute cases (Pulmonary embolism 24,25 , coronary artery disease 26 , shock 27 ). The physician's use of gestalt, that is to say his/her personal estimation of the probability of the disease in the situation, was compared to the probability of the disease after evaluation with a numeric score.

"System 1" which exists alongside System 2, refers to the dual process theory which separates the way to make decision into two types. System 1, also called the non-analytical system, operates unconsciously 28 . It is unlimited, works fast and automatically. It treats several actions in parallel. It connects similar elements with previous situations and activates stored rules 28 .

System 2, or the analytical system, has the opposite characteristics: it operates consciously, is selective and limited in resources, slow, laborious and sequential 28 . It is a very powerful system because of its important computational capacity. The two systems interact throughout the decision-making process. This dual process theory is now integrated into clinical reasoning and into the medical educational process [28][29][30] .

As we can see, the concepts connected to the notion of intuition are multiple, various, and related to conscious and unconscious components. But these definitions remain theoretical and are not directly derived from work in the field.

And if what I felt was a sense of alarm?

The Dutch language has the idiomatic expression "niet pluis" linked to the uncomfortable feeling of physicians that something does not fit in a patient's clinical presentation. It means "a physician's gut feeling about a complaint or a disease, when no diagnosis is available; intuition, experience and knowledge frequently allow a doctor to distinguish between innocuous, nonalarming complaints and serious disorders that necessitate further treatment" 31 . The word "pluis" indicates the opposite situation.

The concept of "pluis/niet pluis" was the starting point of Erik Stolper for his work on gut feelings. He and his team formalised and described what was known by physicians, but has never been properly defined 32 . Although an idiomatic expression does not exist in each language, GPs in every country studied were very familiar with the concept of gut feelings 33 .

The determinants of the concept were then explored through a qualitative study by focus group with GPs. They revealed how gut feelings were considered "a diagnostic instrument which plays a substantial role in general practice". They distinguished two types of gut feeling, a sense of reassurance and a sense of alarm. Both these feelings concern the prognosis of the patient's situation even if a clear diagnosis is not readily available 34 . The next phase was a consensus on the criteria defining the sense of alarm and the sense of reassurance. A Delphi round process was undertaken with 27 Dutch and Belgian GPs involved in research in primary care and education 35 . They agreed on the following statements defining gut feelings 35 :

A 'sense of alarm' means that a GP perceives an uneasy feeling as he/she is concerned about a possible adverse outcome.

A 'sense of alarm' implies that a GP worries about a patient's health status, even though he/she has found no specific indications yet; it is a sense of 'there's something wrong here'.

A 'sense of alarm' activates the diagnostic process by stimulating a GP to formulate and weigh up working hypotheses that might involve a serious outcome.

A 'sense of alarm' means that, if possible, the GP needs to initiate specific management to prevent serious health problems A 'sense of alarm' will decrease as the diagnosis and the right management become clearer.

A 'sense of reassurance' means that a GP feels secure about the further management and course of a patient's problem, even though he/she may not be certain about the diagnosis: everything fits in.

The 'sense of reassurance' and the 'sense of alarm' constitute a dynamic element in a GP's diagnostic process.

From a theoretical point of view, gut feelings emerged from the non-analytical processing, to the consciousness of the GPs, reassuring them or alerting them that urgency was needed. It has been described as the third track, alongside medical decision-making and medical problem-solving, enabling the physician to commute between non-analytical and analytical diagnostic reasoning 36 . A Dutch Gut Feelings Questionnaire (GFQ) was created from the definition criteria and validated by a construct validation procedure using case vignettes 37 . The sense of alarm and the sense of reassurance were found as two opposite components, after a principal component analysis. The internal consistency of the GFQ proved to be high (Cronbach's alpha = 0.91). The Kappa with quadratic weighting was substantial (0.62, 95% CI: 0.55-0.69) 37 . A linguistic validation procedure was performed to obtain an English version of the questionnaire 37 .

Although such expressions as "gut feelings" or "pluis/niet pluis" do not exist in the French language, this description of the sense of alarm resonates immediately with what I felt during that consultation in the emergency unit, ahead of notions such as "intuition", "affect heuristic"

or "gestalt". I experienced this sense of alarm, worrying about the health status of the patient and weighing up the working hypothesis of pulmonary embolism that might involve serious consequences.

Why are gut feelings so decisive in general practice?

In general practice, headache, fatigue, and shortness of breath can be symptoms of several diseases e.g. from anxiety to cerebral metastasis of a pulmonary cancer. GPs may face all these diagnoses with these same sorts of nonspecific symptoms. The consultation sequence in general practice is complex: patients tell GPs their complaints, sometimes clearly, sometimes very vaguely. GPs have to translate patients' accounts into symptoms, looking for a significant coherence, a provisional diagnosis for a prognosis 38 . The clinical signs are incomplete compared to descriptions in the medical literature and often not discriminative 39 .

That is why consultations in primary care are described as complex and dealing with uncertainty and unpredictability, living on "the edge of chaos" 40 . Only some point-of-care tests are available to support the hypotheses. The stress of overlooking a potentially serious disease combined with time management aspects complicates the task of the practitioner. They have to select the cases which require urgent treatment and/or exclude serious conditions. Taking the wrong decision can lead to severe consequences, both for the medical status of the patient and at the judicial level for the physician. Judges in charge of medical cases have already recognized the usefulness of the sense of alarm as a tool for the prevention of error: in some countries GPs have regularly been blamed during trials in medical disciplinary tribunals for failing to have or to follow this sense of alarm 41 . This alarm bell should have pointed out the danger of the clinical case: the GP should have set in motion his course of action to avoid serious outcome from the patient's situation.

Judges already recognize the determinant role of the sense of alarm, but what is the evidence for the predictive value of gut feelings? In 2007, Van den Bruel found the physician's statement 'something is wrong' was the most discriminative sign for diagnosing serious infection in children 42 . In another prospective study on abdominal cancer in general practice, the GP's intuitive cancer suspicion was independently and closely associated with the diagnosis of a new cancer 43 . GPs defined gut feelings as a compass, steering them through busy office hours and enabling them to handle complex problems 34 .

Research questions of the thesis

Gut feelings and the questionnaire were defined by Dutch GPs after qualitative procedures.

One research question of this thesis was how to translate the concept and the questionnaire into other languages and contexts. The second research question focused on the symptoms of dyspnoea and thoracic pain: what is the diagnostic role of gut feelings in pulmonary embolism and can we rely on the sense of alarm when faced with dyspnoea and thoracic pain in general practice?

Objectives of the thesis

This thesis is divided into two parts.

The first part aims at translating the gut feelings concept into European languages and contexts:

• What was the process for translating gut feelings into French? I participated in the drafting of the article describing the Delphi procedure used to translate "gut feelings"

into the French language working with French expert GPs.

• How was the GFQ translated from English into French, German and Polish? Linguistic validation processes, following a procedural scheme, were undertaken in France, Germany and Poland.

• How was the feasibility of the GFQ tested in real practice? A mixed method study was followed in Belgium, The Netherlands, and France to obtain the final version of the GFQ.

• How did you define the salient terms on diagnostic reasoning used in the publications of the COGITA group? A glossary of terms was constructed following a literature review.

• The differences which occurred in the definitions and translations of gut feelings and the connected concepts led us to comment on the article "Recognition of sepsis in primary care: a survey among GPs," written by Loots et al, and their use of the wording "gut feelings" without describing to which concept they were referring.

The second part focuses on an acute disease in general practice with the symptoms of dyspnoea and thoracic pain.

• How did GPs come to suspect pulmonary embolism in real-life settings? A qualitative study with semi structured interviews explored this point. Next, we described more specifically the role of gut feelings in this diagnostic process.

• The results of this qualitative study led us to comment on the Hendriksen et al. article on the comparison of the diagnostic performance of "gestalt" and the Wells rule, for ruling out pulmonary embolism in primary care.

• What is the diagnostic test accuracy of the sense of alarm when applied to dyspnoea and chest pain? A prospective study using the gut feelings questionnaire was undertaken with French GPs. We will present the published protocol and report the first results of the prospective study.

ABSTRACT

Background

Physicians' clinical decision-making may be influenced by non-analytical thinking, especially when perceiving uncertainty. Incidental gut feelings in general practice have been described, namely, as "a sense of alarm" and "a sense of reassurance".

A Dutch Gut Feelings Questionnaire (GFQ) was developed, validated and afterwards translated into English following a linguistic validation procedure.

The aims were to translate the GFQ from English into French, German and Polish; to describe uniform elements as well as differences and difficulties in the linguistic validation processes;

to propose a procedural scheme for future GFQ translations into other languages.

Methods

We followed a structured, similar and equivalent procedure. Forward and backwardtranslations, repeated consensus procedures and cultural validations performed in six steps.

Exchanges between the several research teams, the authors of the Dutch GFQ, and the translators involved continued throughout the procedure.

Results

12 translators, 52 GPs and 8 researchers in the field participated to the study in France, Germany, Switzerland and Poland. The collaborating research teams created three versions of the 10-item GFQ. Each research team found and agreed on compromises between comparability and similarity on one hand, and linguistic and cultural specificities on the other.

Conclusions

The gut feeling questionnaire is now available in five European languages: Dutch, English, French, German and Polish. The uniform procedural validation scheme presented, and agreed upon by the teams, can be used for the translation of the GFQ into other languages. Comparing results of research into the predictive value of gut feelings and into the significance of the main determinants in five European countries is now possible.

BACKGROUND

Physicians' clinical decision-making is based on the interaction of analytical and non-analytical reasoning and gut feelings can be considered a part of the non-analytical reasoning process [1]. In 2009, the concept of gut feelings in general practice was described, by means of a qualitative study, as a sense of alarm and a sense of reassurance [2]. The sense of alarm is "an uncomfortable feeling experienced by the physician, that something does not fit in a patient's clinical presentation although he/she has found no specific indications". The sense of alarm "activates the diagnostic process and induces the doctor to initiate specific management to prevent serious health problems" [3]. The sense of reassurance means that a GP "feels secure about the further management and course of a patient's problem, even though he/she may not be certain about the diagnosis: "everything fits in"" [3]. Gut feelings are considered to play a substantial role in the diagnostic reasoning of GPs [1]. Two prospective studies proved how this sense of alarm could be efficient. When dealing with children with serious infections, GPs' gut feeling about parental concerns and the children's appearance had a high specificity and a high positive likelihood ratio [4]. Gut feelings that something was wrong were also a common reason for referral which proved to be a strong predictor of cancer in a Danish cancer pathway [5].

A Dutch Gut Feelings Questionnaire (GFQ) was created from the consensus criteria for gut feelings and validated by a construct validation procedure using case vignettes [6]. The validity of the GFQ was consistent: the internal consistency of the GFQ proved to be high (Cronbach's alpha = 0.91), the Kappa with quadratic weighting was moderate to good (0.62, 95% CI: 0.55-0.69) and factor analysis showed one factor with opposites for sense of reassurance and sense of alarm items. Two versions of the questionnaire were created: a vignette version and a real case version. A linguistic validation procedure was performed to obtain an English version of the questionnaire in general practice [6].

The aim of this article is to report on the translation procedure of the GFQ from English into French, German and Polish; to describe uniform elements as well as differences and difficulties in the linguistic validation processes; to propose a procedural scheme for future GFQ translations into other languages.

METHODS

Research teams are composed of French, German, Swiss, and Polish speaking researchers in different countries in primary care.

The linguistic validation procedure which the teams followed met the standardisation criteria found in the international literature [7-11]. It was in line with the way researchers had translated the Dutch questionnaire into English [6].

The linguistic validation process consisted of six steps: Forward-translation (step 1), backwardtranslation (step 2), first consensus (step 3), cultural validation (step 4), second consensus (step 5), and final version (step 6). Table 1 provides a summary of the different steps in all three versions and Figure 1 provides the procedural scheme followed.

We have obtained the approval of the ethics committee of the University de Bretagne

Occidentale for the study (N°05092012). Informed consent was obtained from all participants even thoughit was a non-interventional study.

Table 1 about here

Place Figure 1 about here

Forward-backward translations (step 1 and 2)

Two native-speaking translators for each language (French, German and Polish) who were familiar with medical terms, translated the questionnaire into their own language. They performed this translation separately and independently after receiving information about the goal of the questionnaire and the way it would be used in research. They were invited to add comments if needed (step 1).

Next, two native-speaking English language translators, familiar with medical terms, provided independently and separately two backward-translations, each using a different forwardtranslated version. They were also invited to add comments if needed (step 2).

Reaching a first consensus (step 3)

Each research team prepared a first draft for a consensus translation in their own language, putting all the differences and questions in an extended table. The four translators, each belonging to the French, German or Polish groups, were separately asked to read this first consensus carefully, including all the comments in the table, and to add their opinions to this Extensive communication between the translators, the coordinating scientific team, and the authors of the original Dutch version yielded a consensual GFQ version, in each language:

French, German and Polish.

Cultural validation (step 4)

These consolidated GFQ versions were sent to at least ten GPs (native speakers of French, German or Polish) based in France, Germany, Switzerland or Poland, asking them to check for grammatical errors and cultural misunderstandings. An accompanying letter explained the background of the GFQ and the purpose of their involvement.

Reaching a second consensus (step 5)

The results of the GPs' feedback were incorporated into an advanced version of GFQ by the research team. All previous stakeholders in this process and interested parties added some comments which were integrated. The four translators studied the comments and gave their final judgment.

Resulting in a final version (step 6)

After considering the translators' recommendations, each research group finally determined the definitive text of the questionnaire.

RESULTS

A French, German and Polish version of the English GFQ version is now available. Table 2 provides the GFQ in the four languages: the original in English, along with the French, German and Polish versions.

Table 2 about here

French procedure: adaptations and problems

These six steps were completed between October 2012 and May 2013.

Step 1 to 3

We only translated the real case questionnaire in the French procedure because we intended to use it for a study in real settings and had no research proposal related to the vignettes questionnaire.

Three translators were from the linguistic department of the University of Brest: two French native speakers and a British English native speaker. The fourth was a French GP whose native language is British English. The scientific team was composed of one GP trainee, working on a gut feelings master's thesis, and two members of the department of General Practice working on the same topic.

Several points needed to be discussed for the French translation:

For the fourth item: "I have an uneasy feeling because I am worried about potentially unfavourable outcomes," the proposition "I have an uneasy feeling" was translated as "Je suis gêné" "I am bothered". The phrasing "uneasy feeling" was not compatible in the French version For the sixth item: "What course of action have you chosen? (Please tick one answer.) I will wait and see", the concept of "wait and see" does not exist in the French language, and this expression is also used verbatim in English. We chose to translate it as "attendre, temporiser": "to wait, to temporise," meaning staying open to new things which could happen.

Step 4 to 6

We submitted the translated questionnaire to 12 GPs who were experienced in research in primary care. We analysed the 12 answers we received.

For the first item: "I feel confident about my management plan and/or about the outcome: it all adds up", 7 participants did not understand the proposition "about the outcome": they found it difficult to make such a judgement at this early stage of the diagnostic reasoning process. They asked about the kind of outcome: the expected outcome or the actual outcome, and the outcome of the management plan and the tests requested or the outcome of the treatment plan. The participants' lack of understanding was related to discomfort with the clinical reasoning process at an early stage in the case and not with the terminology. We chose to add "expected outcome" to the first proposition.

Seven participants wanted to add the referral to the emergency unit to item n° 6: "What course of action have you chosen?" "Refer the patient". For French GPs, referring to the emergency unit or to the specialist are two different situations. To the authors of the original Dutch version, the idea was to include the referral, not distinguishing between urgent and nonurgent. We maintained the original meaning of item 6 and added the following details: "refer the patient to a specialist, either within the emergency unit or elsewhere."

For the seventh item: "This patient's situation gives me reason to arrange a follow-up visit sooner than usual or to refer him or her more quickly than usual to a specialist", 5 participants asked that the wording "sooner than usual" be defined more precisely. They found the "usual" situation difficult to define. For the authors of the original Dutch version, "sooner than under usual care" means "sooner than he/she does in common daily situations, without hurrying". "To refer him or her more quickly than usual to a specialist" was also confusing for these 5 participants. They asked that "or to the emergency unit" be added. As for the sixth item, in accordance with the authors of the original Dutch version, we chose to maintain the generic term "to the specialist" without mentioning the emergency unit.

The French version of the English GFQ version is available (See additional file n°1).

German procedure: adaptations and problems

The six steps were completed between April 2014 and June 2015.

Steps 1 and 2

We translated both the real practice and the case vignette design from the BE version.

As the German language varies somewhat between regions and countries, we intended to find a supranational linguistic German version. Therefore translators, and members of the scientific team involved, were drawn from different countries and regions, e.g. Germany (D) (northern and southern regions) and Switzerland (CH).

All translators were from different institutions and lived and worked in Germany, Switzerland, the United Kingdom, or the United States of America. The research team was composed of one general practitioner with academic background from Germany (JH) and one University psychologist from Switzerland (DH), both doing academic research in the field of intuition and medical decision-making.

Step 3

Our actual execution of step 3 differed slightly from the adopted procedural scheme in three As an extra intermediate step, DH and JH independently proposed two leading versions as summaries of the four heterogeneous versions and comments of all the translators. Then a first consensus was reached between DH and JH, based on all the existing versions and comments, which tended to favour one of the proposed leading versions chosen by the preparation team. The consolidated table, including all versions and comments, was then sent to all the translators and the whole research team for further revision or comments. Another advantage of proposing two additional leading versions has been that a telephone conference involving everyone was not necessary. DH and JH had a meeting at the end of step three with the aim of checking, discussing and integrating the final comments, and planning further steps (e.g. cultural validation).

Step 4 to 6

Twenty physicians (mainly GPs) had been asked in February 2015 to do a cultural check of the penultimate version. Subsequently, 12 questionnaires from responders were systematically analysed, comment by comment, by the preparation team. Items 1, 3, and 8b were discussed by the research team in detail at a second meeting (in March 2015). The team voted to maintain the status quo, whereas items 4: "I have an uneasy feeling because I am worried about potentially unfavourable outcomes" (reformulated as: "weil ich... besorgt bin") and 6: "What course of action have you chosen? I will Wait and see"(first option added with: "die Situation abwartend offenhalten"which means stay open to what could happen) have been slightly adjusted. "Abwartendes Offenhalten" in GP semantics is the German equivalent to the English "watchful waiting" ("wait and see"), and has always to be weighed against "abwendbar gefährlicher Verlauf" (preventable dangerous outcome).

In an additional step, we asked for final comments from all significantly involved participants, including the whole research team and all the translators. Item 8a: "What do you consider to be the most likely diagnosis?" caused doubt until the very end of the German linguistic validation process. For this item, the following suggestions had been under consideration, with subtly different meanings: "die wahrscheinlichste"(the most likely) (also used by medics in the UK), "die bevorzugte"(the preferred), or "die zutreffendste"(the most appropriate). Finally, the following wording was chosen: "Was ist Ihrer Ansicht nach die zutreffendste Diagnose? Meine zutreffendste Diagnose ist..."in the sense of the most appropriate diagnosis.

The final German versions were called "Fragebogen zum Bauchgefühl bei ärztlichen Entscheidungen" (FBAE). Generally, the English case vignette design and the real practice version differed very little. The subtle differences in the German version can be found in items 6, 7, and 8b in the word "würde"(instead of "werde"), and in item 8a in the words "wäre"(instead of "ist") and "könnte"(instead of "kann").

The German version of the English GFQ version is available (See additional file n°2).

Polish procedure: adaptations and problems

Step 1 to 3

We translated both the real practice and case vignettes from the BE version into Polish.

All the translators were affiliated to different academic institutions and all had a linguistic background. There were two Polish certified translators with expertise in medical translation, one translator from the English Department of Nicolaus Copernicus University, and one American native speaker. The research team was composed of one general practitioner and a linguist, both from Nicolaus Copernicus University.

The problem that occurred at this stage involved the translation of Items 6 and 7 and was due to cultural differences: "This patient's situation gives me reason to arrange a follow-up visit sooner than usual or to refer him or her more quickly than usual to a specialist." First of all, phone consultations are not commonplace in Poland. GPs have no obligation to call their patients to arrange visits. Secondly, referring the patient more quickly than usual to a specialist is not possible at all in Polish primary care due to one national medical service provider which controls and manages the whole referral system. Yet, after discussion, we decided to include these items as they are present in the English version and proceeded to the next step. Finally, the translation of "would refer the patient" as "odesłałbym pacjenta" has negative connotations in Polish and implies ignoring and sending away the patient. For that reason, after consultation with the translators, we came up with a neutral expression "skierowałbym pacjenta gdzieś indziej." (I would refer the patient somewhere else), which communicates the meaning of sending a patient to someone else, rather than getting rid of the patient as it is the case with "odsyłać" in Polish.

Steps 4 to 6

We sent the translated questionnaire to 25 GPs via email and asked for a cultural check and evaluation of the equivalence between the translations and the BE versions. Two e-mail addresses turned out to be incorrect and, out of 23 GPs, only eight GPs with an academic background and experience in research in primary care responded. All of them evaluated the translations positively (real practice and case vignette). Four of the GPs provided constructive comments and feedback. The proposed linguistic corrections concerned Items 1, 3, 7 and 10.

These items were thoroughly discussed by the scientific team and consensus was reached.

In Item 1: "I feel confident about my management plan and/or about the outcome: it all adds up," there is no Polish adequate expression for "it all adds up." The closest expression: "wszystko składa się w jedną całość" was rejected and replaced with "wszystko zgadza się," (everything is fine) which is more comprehensible and more common in professional language among GPs.

In Item 3: "In this particular case, I will formulate provisional hypotheses with potentially serious outcomes and weigh them against each other," the phrase: "rozważę ich wzajemne związki" for "weigh them against each other" was replaced with "porównam je," (compare them) which more adequately renders the original concept and simplifies the translation. At the same time, the respondents found it more comprehensible than the previous choice.

In Item 7:" This patient's situation gives me reason to arrange a follow-up visit sooner than usual or to refer him or her more quickly than usual to a specialist," the phrase: "Obecny stan zdrowia pacjenta," which means "the patient's health condition" was replaced with "sytuacja pacjenta," which sits better within the holistic model adopted in general practice. It is the medical term used by GPs as it expresses not only a patient's somatic condition, but also his or her psycho-social condition.

In Item 10: "Please indicate what kind of gut feeling you have at the end of the consultation," the word "Intuicja" ("intuition") has been replaced with "przeczucie" ("gut feeling"), which is more appropriate in the everyday language of general practice.

The Polish version of the English GFQ version is available (See additional file n°3).

DISCUSSION

Main findings

The GFQ has been translated into three more European languages using a standardised procedure of linguistic validation. The collaborating research teams from France, Germany/Switzerland and Poland found and agreed on compromises between comparability and similarity on one hand, and linguistic and cultural specificities on the other. All the GFQ versions are available on the website http://www.gutfeelings.eu

Strengths and limitations of the study

Translators with a medical background worked on the questionnaire following the standardised procedure. This feature was important here to avoid misunderstandings in the specific area of medical decision-making. The cultural check stage was undertaken with GPs who were the principal recipients of the questionnaire. They gave a pragmatic point of view as they are active in the field of daily clinical practice.

The French, German and Polish teams were working in the same research network on clinical decision-making. The creators of the questionnaire were involved from the beginning of the process and acted as the vital link between the researchers. These two characteristics facilitated exchanges and probably prevented the translation from deviating from the original Dutch version of the questionnaire.

Similar items generated discussions in the three different research teams. Expressions such as "uneasy feeling" and "wait and see" do not correspond to existing linguistic concepts in French, German or Polish but may be reflected in analogy, at least in German, by "Alarmgefühl" and "abwartendes Offenhalten".

Comparing with existing literature

As far as we know, the GFQ is the first tool developed which measures GPs' gut feelings.

There is no alternative tool available at present. The sense of alarm was recognised by European GPs in their daily practice [12]. The transculturality of the gut feelings concept between Proto-Germanic and Romance languages was revealed after a Delphi procedure compared the Dutch and the French statements of the gut feelings criteria [13]. German research into this field had been sparked in 2004 by the Dutch expression "niet pluis" literally "there is danger here, something is amiss" which is commonplace for Dutch GPs but has no equivalent in German, although German GPs also expressed their incidental uneasiness ("Hier stimmt 'was nicht!") which was later coined as "Alarmgefühl". The French and German versions of the questionnaire logically followed this finding. The linguistic validation procedures followed here, in Polish, allowed us to expand the concept to include Slavic languages. We assume that the utility of the GFQ would also be transferable, working within this transcultural context and applying standardised linguistic procedures. The forward-backward translation, with cultural check, was preferred here to the Delphi procedure [14]. Exchanges between several translators with a medical background, GPs and a linguist allowed us to analyse in depth differences in wording.

The Dutch first authors on the gut feelings concept had an idiomatic expression in their language to express the sense of reassurance and the sense of alarm "pluis/niet pluis". A survey in 2005 identified idiomatic expressions in European languages about this specific term "gut feeling" [12]. Even if no specific expression existed to describe this feeling, European GPs recognised the description of the sense of alarm. Behind the linguistic aspects, GPs do share the same medical decision-making model. A consultation in general practice is complex: the patient may suffer from non-specific symptoms; he will use his own words and the GP has to translate into semiological language. The clinical signs are partial and rarely discriminative.

Few tests are available at the surgery to support his hypotheses. The stress of dealing with a potentially severe disease, as well as time management, complicate the task of the practitioner. The GP has to make a decision in this uncertain and incomplete area [15, 16].

Two different interacting modes which control the activity of reasoning were described: the intuitive mode or system 1 and the analytical mode or system 2 [17]. The analytical mode operates consciously; it is selective and limited in resources, slow, laborious and sequential. It is a very powerful system because of its important computing capacity but it is difficult to sustain over a long period. The intuitive mode has opposite characteristics: it operates unconsciously, it is unlimited, works fast and automatically. It considers several actions concurrently. It connects similar elements with previous situations and activates stored rules. This dual process theory is now integrated into clinical reasoning and the medical educational process [17-19]. The sense of alarm is recognised here as a feedback mechanism, compelling the physician to abandon his routine-based/schematic mode of reasoning in favour of an analytical and attentional one [20, 21].

Whilst the organisation of health care systems in The Netherlands and Belgium, where the original version was validated, are similar, the health care systems in France, Germany, Switzerland and Poland are organised differently in terms of structure, process and outcome [22-24]. The application of medical decisions has to integrate into each different type of organisation. The GFQ was modified to correspond to French, Polish and German systems.

French GPs distinguished between referral to a specialist and referral to the emergency unit.

In the first case, they sought the opinion of a specialist within their own network to obtain a second point of view of the patient, with non-formal emergency criteria. When they referred to the emergency unit, they needed to seek a second opinion with urgent and appropriate care.

We kept the original version of the questionnaire, with additions, on this specific point in the French questionnaire. In Poland the same item was problematic because of the national medical service provider which controls the referral system. Polish authors found a neutral formulation to express the sense of the proposition without insisting on the organisational aspect. No adaptations were needed in the German version: German and Swiss GPs did understand each proposition in the original formulation during the cultural validation. Their health care system is closer to the Dutch one on this particular point.

Implications for practice and future research

Translating the GFQ into different languages using a standardised procedure is of great value for further quantitative research. A study protocol has been designed to evaluate the feasibility of the questionnaire in daily practice in primary care. A quantitative phase will explore the average time taken to fill in the questionnaire, estimated by the GP, the disruption of daily routine caused by the gut feelings questionnaire with a four-point scale, and additional workload created by completing the questionnaire with a four-point scale. A qualitative phase, using semi-structured interviews with the GPs involved, will explore the integration of the questionnaire into daily practice.

The accuracy of gut feelings is another point to be studied. A prospective observational study, using the GFQ to measure the accuracy of the general practitioner's sense of alarm when confronted with dyspnoea and/or thoracic pain, is actually planned [25].

The GFQ may also be useful in the field of education. Gut feelings appeared in tutorial dialogue between Dutch trainees and their supervisors [26]. When they faced uncertainty during consultation, trainees had to take their gut feelings into account during the reasoning process [26]. We visualise the GFQ as a tool which will facilitate the explanation of how non-analytical reasoning forms part of the teaching of clinical decision-making [27,28]. A think aloud study is also planned, to check the way GPs understand each item when dealing with case vignettes.

Manipulating cues in case vignettes and measuring their influence on the results of the GFQ may be an interesting possibility. Modifications to the GFQ may occur in the future due to the integration of the results of new studies.

CONCLUSIONS

The gut feeling questionnaire is now available in five European languages: Dutch, English, French, German and Polish. The uniform procedural scheme presented, which the teams agreed on, can be used for the translation of the GFQ into other European languages.

Comparing results of research into the predictive value of gut feelings in several European countries, where the native language is one of these five, is now possible.

ETHICS APPROVAL AND CONSENT TO PARTICIPATE

We have obtained the approval of the ethics committee of the University de Bretagne Occidentale for the study (N°05092012). Informed consent was obtained from all participants even though it was a non-interventional study.

CONSENT TO PUBLISH Not applicable.

AVAILABILITY OF DATA AND MATERIALS

The dataset supporting the conclusions of this article is included within the article and its additional files.

COMPETING INTERESTS

None of the authors have any competing interests.

FUNDING

No current funding sources for this study. ❑ Not yet take action, but will invite the patient for a follow-up appointment either face-to-face or by phone.

❑ Arrange further testing (laboratory tests, X-rays, etc.). ❑ Arrange further testing, and in the meantime, I will start treatment (medicinal or other). ❑ Start treatment, but will not arrange a follow-up. ❑ Start treatment and will invite the patient for a follow-up appointment either face-to-face or by phone.

❑ Refer the patient. • Step n°1 = separate and independent forward translation by two native speakers into the intended language.

• Step n°2 = separate and independent backward translation of the two results as obtained from step n°1 by two English native speakers.

•

Step n°3 = first consensus version of the questionnaire obtained after comparison of the versions resulting from step N°1 and 2 by the research team.

• Step n°4 = cultural validation of the questionnaire by GPs asked to check grammatical errors and cultural misunderstandings.

• Step n°5 = second consensus with the summary of the GPs' comments and suggestions for modifications submitted to the four translators and the research team.

• Step n°6 = last consensus and definitive version of the questionnaire in the intended language.

Abstract

Objectives

The validated Gut Feelings Questionnaire (GFQ) is a 10-item questionnaire based on the definitions of the sense of alarm and the sense of reassurance. The purpose of the GFQ is to determine the presence or absence of gut feelings in the diagnostic reasoning of general practitioners (GPs).

The aim was to test the GFQ on GPs, in real practice settings, to check whether any changes were needed to improve feasibility, and to calculate the prevalence of the GPs' sense of alarm and sense of reassurance in three different countries.

Setting

Primary care, 6 participating centres in Belgium, France and the Netherlands.

Participants

We performed a think-aloud study with 24 experienced Dutch GPs, GP-trainees and medical clerks who filled in the GFQ after diagnosing each of 6 case vignettes. We then performed a feasibility study in two phases, using a mixed-method approach, with 42 French and Dutch GPs in the first phase and then 10 Belgian, 10 Dutch, and 10

French GPs in the second phase. All GPs filled in the GFQ after each of eight consultations with patients presenting new complaints and were subsequently interviewed about the use of the GFQ.

Outcome measures

GPs' experiences on using the GFQ in real practice, more specifically the average time needed for filling in the questionnaire.

The prevalence of GPs' sense of alarm and sense of reassurance

Results

The modified version of the GFQ, created without altering the sense of the validated items, was easy to use in daily practice. The prevalence of the GPs' sense of alarm occurred during 23% to 31 % of the included consultations.

Conclusions

After a two-step study and several minor adaptations, the final version of the GFQ proved to be a feasible and practical tool to be used for prospective observational studies in daily practice.

Strengths and limitations of this study

• Testing the use of a questionnaire such as the GFQ in two different settings (think-aloud in an experimental environment first, and then during office hours, in three different healthcare systems) was quite unique.

• The GFQ is directly derived from the consensual definition of gut feelings: its added value is the detailed and precise way it measures GPs' gut feelings.

• Quite a number of the GPs did not fill in the questionnaire right after the consultation but completed it later that day.

Introduction

The gut feelings which may arise during the process of diagnostic reasoning by general practitioners have been defined as a sense of alarm and a sense of reassurance [1].

The sense of alarm is an uncomfortable feeling, experienced by the general practitioner (GP), that something does not fit in a patient's clinical presentation although he/she has not (yet) found specific indications. The sense of reassurance means that a GP feels secure about the health status of the patient, even if he/she is not certain about the diagnosis. The sense of alarm activates the diagnostic process and initiates specific management to prevent serious health problems [1]. Gut feelings play an important role in the diagnostic reasoning process of GPs helping them to navigate in the complex and uncertain diagnostic situations encountered in practice [2]. It has been described as a third track, alongside medical decision-making and medical problem solving, enabling the physician to commute between non-analytical and analytical reasoning processes [2].

The sense of alarm and the sense of reassurance were defined following a qualitative analysis of the text of several focus groups on the topic and a subsequent Delphi consensus procedure [1,3]. A Dutch Gut Feelings Questionnaire (GFQ) was created from the definition criteria and validated by a construct validation procedure using case vignettes [4]. The objective of the questionnaire was to determine the presence, or absence, of gut feelings in GPs' diagnostic reasoning at the end of a consultation based on a clear consistent definition of the concept in order to avoid response bias. This questionnaire measures not only whether a gut feeling is present, (i.e. not just by a yes or no response, as is mostly done in clinical studies about gut feelings [5-7]), but also differentiates between the sense of reassurance and the sense of alarm by more precise statements reflecting the outcomes of the diagnostic reasoning process. In validating the questionnaire, a Principal Component Analysis (PCA) showed one component explaining 70.2% of the total variance with the sense of alarm and the sense of reassurance as opposites. The internal consistency of the GFQ proved to be high (Cronbach's alpha = 0.91). The kappa with quadratic weighting was substantial (0.62, 95% CI: 0.55-0.69) [4]. A linguistic validation procedure was performed to obtain an English version of the questionnaire [4] (Figure 1).

An international network group called COGITA, was established with the aim of coordinating and stimulating research into the role of gut feelings in general practice (See www.gutfeelingsingeneralpractice.eu). Linguistic validation procedures produced a French, Polish and German version of the GFQ [8], and a Spanish and a Catalan version [9] (publication in process).

The GFQ can be used in studies measuring the prevalence of gut feelings and their predictive value for a serious disease [10]. The questionnaire was, however, never evaluated by GPs in real settings during office hours. The aim of this study was to explore the practicability of the GFQ, i.e. feasibility and acceptability as experienced by GPs when using the instrument in daily practice, and to calculate the prevalence of the GPs' sense of alarm and sense of reassurance in three different countries.

We conducted a think-aloud study to explore whether the way experienced GPs, GPtrainees and medical students understood the GFQ items was in line with what we aimed for when composing the questionnaire. The next step was a feasibility study in daily practice with the original GFQ. By collecting quantitative data, we measured and compared the prevalence of GPs' gut feelings in different countries. As these two phases led to some adaptations, the modified questionnaire was retested during a second feasibility study. Figure 2 shows the steps taken to test the original version of the GFQ and to arrive at the final version as the result of both studies.

Procedure

Participants were asked to diagnose each case while thinking-aloud, and to fill in the GFQ afterwards, still thinking aloud. They were reminded to think aloud if they were silent for more than 5 seconds. The session took place in the GPs' offices or in a room at the university and lasted 30-80 minutes. All participants received a small gift at the end of the session.

Data analysis

All think-aloud protocols collected while participants filled in the GFQ were transcribed verbatim and analysed. We performed a thematic content analysis to summarize, per item, the problems participants encountered in interpreting and responding to the items in the GFQ [11].

Patient and Public Involvement

Patients were not involved.

Feasibility study 1

Participants

The participating GPs were recruited using a purposive sampling strategy according to criteria which could influence decision making: age, gender, location of practice (rural area means under 5000 inhabitants). Twenty French GPs from Brittany and 22 Dutch GPs, including 23 males and 19 females, aged from 28 to 64 years old, from different areas (33 urban and 9 rural areas), participated in this study. They were not given any financial incentive to take part.

Materials and Procedures

A mixed-methods approach was chosen. The GPs were instructed to fill in the GFQ, during their office hours, for 8 days in a two-week period. They were asked to include only the first consultation of the day with an adult patient, aged over 18 years, with a new reason for a consultation. After completing the 8 questionnaires, the participating GPs were asked to estimate the time they needed to fill in the GFQ in minutes and were interviewed at their office or by phone. The interview guide was composed of two open-ended questions, which aimed to explore the experience with the questionnaire in more depth: "What do you think about the questionnaire's integration into your daily practice?" and "Which elements should be improved following your experience of filling in 8 questionnaires?". Most interviews were held within two weeks of the two-week period and were audio-recorded and transcribed verbatim. The interviews lasted between 3 and 18 minutes.

Data analysis

The analysis of the qualitative data was descriptive, using a thematic content analysis.

The French and Dutch researchers coded the transcripts in an independent and open way, categorized their codes and established codebooks. After having reached consensus, they merged their codebooks, adapted the codes in the different texts and reanalysed the texts. Finally, they selected the most appropriate quotes to illustrate each code in each language [11]. QSR NVivo 11.0 Software was used to perform the analysis.

The quantitative data, i.e. the answers to items 1-5 and 10 (Figure 1), were analysed with a Chi² test using specific criteria. A sense of alarm was considered as present when the answer to item 10 indicated a sense of alarm or when the answer to item 10 indicated that it was not applicable and at least one of the scores for items 2 to 5 was higher than 3/5. A sense of reassurance was considered as present when the answer to item 10 indicated a sense of reassurance or when the answer to item 10 indicated that it was not applicable and the score for item 1 was higher than 3/5. Gut feelings were considered absent when the answer to item 10 indicated that it was not applicable and none of the scores for items 2-5 was higher than 3/5 and the score for item 1 was lower than 4/5. These cut-off criteria were chosen in line with the study protocol of the study on the accuracy of the sense of alarm when faced with chest pain and dyspnoea [10].

Patient and Public Involvement

Patients were not involved.

Feasibility study 2

Participants

The modified GFQ was tested in real practice in Belgium, the Netherlands and France, with 10 GPs from each country. The participating GPs were recruited using the same purposeful sampling strategy as in feasibility study 1. Ten Belgian GPs, 10 Dutch GPs, and 10 French GPs from Brittany, 15 males and 15 females, aged from 27 to 65 years, from different areas (26 urban and 4 rural areas), participated in the study. The participants were not incentivised to take part.

Materials and procedure

We used the same procedures as in the first feasibility study but presented the participants with the modified GFQ (Figure 3). Most interviews were held within two weeks of the two-week period and lasted 5-30 minutes.

Patient and Public Involvement

Patients were not involved.

Results

Think aloud study

The analysis of the think-aloud protocols revealed that some participants interpreted four of the GFQ items in a slightly different way than we intended. There were no systematic differences between the three groups. Based on these findings, we suggested small adaptations to the phrasing of two items and to the order of items to avoid misunderstanding.

Regarding item 1: "I feel confident about my management plan and/or about the outcome: it all adds up", Many participants were confused by the two elements of the question, i.e. management plan and/or the outcome. An experienced GP, for example, said: "I feel confident about the management I have in mind, but there's something wrong... It's a strange case" (GP n°24). The focus for this item is 'adding up', so we suggested a reversal of the wording of this first item: 'It all adds up. I feel confident about my management plan and/or about the outcome'.

Regarding item 3: "In this particular case, I will formulate provisional hypotheses with potentially serious outcomes and weigh them against each other". Several participants found this criterion stated the obvious. An experienced GP said: "Yeah, of course, you always do that in clinical reasoning" (GP n°23). Although this remark may show that the item does not actually discriminate, we decided to leave it in the questionnaire as it was one of the statements agreed upon in the consensus procedure [1] and it also fitted in with the other items in the construct and consistency validation procedures [12].

Regarding item 5 "This case requires specific management to prevent any further serious health problems", Many participants answered "yes", even if they had a sense of reassurance, due to safety netting or watchful waiting. However, this item defines the sense of alarm in the consensus definitions [1]. To emphasize the prevention of serious health problems, we suggested a reversal of the wording of this item, modifying to read: "To prevent any further serious health problems this case requires specific management".

Regarding item 10 "Please indicate what kind of gut feeling you had at the end of the consultation", several participants gave the impression that their answers to this last item were the conclusion of a rational reasoning process, based on a series of logical arguments built up from the previous items, to find an answer to item 10, rather than indicating their experience of a gut feeling. The ranking of the first nine items could induce a bias in the answer to the last one. To avoid this, we proposed starting with the item about gut feelings, thereby moving item 10 to the top of the list. We also proposed repeating this item at the end of the list for those participants who were not able to answer this question at the beginning. It was only for those participants who did not answer item 1 that we used item 11 as the indication of the presence or absence of a GF. We suggested changing the order of the items in line with the usual steps of the diagnostic process.

Feasibility study 1

Qualitative results

The interviews with GPs showed some important issues regarding the feasibility of the GFQ. The GPs mainly commented on deciding how to complete the questionnaire and also commented on some of the items. There were no differences between the comments of the French and Dutch participants. They encountered the same difficulties and misunderstandings. We summarize the main findings below and illustrate these with quotes.

The GPs were asked to fill in the GFQ after each consultation in daily practice. They needed to take some extra time to do this and most succeeded, but several GPs could not deal with it immediately after the consultation and postponed it until a more suitable moment.

"Of course, we're used to just following the routine, and it did very much interrupt the routine. But it took very little time. I was filling in how much time it took, and well, 2 minutes". (Dutch GP n°4) Some GPs responded about the timing needed to fill in the GFQ "At the end of a block of consultations, mostly. I occasionally did one or two immediately after the consultation if I had a gap in the flow of patients, as I had some time available and it was a new complaint, but I often did it at the end of the morning or the day, thinking Oh I must have seen some new people and I need to fill in the questionnaires". (Dutch GP n°2) "I did not fill in the questionnaire right after the consultation, I preferred to do it at the end of the day, because in fact, technically, during my consultation, I don't have the time to do it, well I am being honest, aren't I?". (French GP n°2) Some participants experienced problems in answering items 6, 8 and 9. In item 6, participants were asked about their management. Some participants found the list with possible courses of action incomplete and gave suggestions for improvement. "I tended to think "In some cases I can examine the patient and start therapy and make a follow-up appointment at the same time." So, I mean, I couldn't fit it all in one line".

(Dutch GP n°10)
In item 8, they were asked to provide the most likely diagnosis and the diagnosis that determined their management. Several participants said that they were confused about the difference between these two types of diagnoses and suggested how to improve the clarity of the question. In item 9, they were asked to indicate how confident they were (as a percentage) about their management determining diagnosis. This question also raised confusion.

"The only downside for me, was the worry of differentiating between the questions, between question number 8A and 8B, between the diagnosis and the hypothesis, I

often put the same answer in both boxes". (French GP n°5) Based on these comments, we proposed modifying the two diagnostic work-up items (items 6, and 9): we added more options for the course of action in item 6 and we decided to remove item 9 where participants had to assess their confidence in their policy determining diagnosis in terms of a percentage. The seven validated items concerned with gut feelings were retained.

"
The changes in the GFQ we proposed, based on the think-aloud study and the first feasibility study, were discussed during two consensus meetings of COGITA researchers (Marburg 2015, Tel Aviv 2016) (http://www.gutfeelings.eu/list/cogita-expert/). Afterwards, based on our concerted efforts, we formulated a modified version of the GFQ, did a linguistic validation of the new elements and changed the presentation of the questionnaire into a more visual and ergonomic format (Figure 3).

Quantitative results

Out of the 348 questionnaires collected during this second phase (8 to 10 per GP), 336 were analysable, 12 were non-analysable because of missing data. In total, 77 (23%) were concerned with a sense of alarm,242 (72%) with a sense of reassurance and there were 17 (5%) where no gut feeling was applicable. The internal consistency was high (Cronbach's alpha = 0.88). A PCA showed one component explaining 68.6% of the total variance, with the sense of alarm and the sense of reassurance as opposites.

There were no significant differences between the Dutch and French GPs. They expressed the same prevalence of a sense alarm and the same prevalence of a sense of reassurance. The median average time estimated by GPs for filling in the GFQ was 1 minute for the Dutch GPs and 2 minutes for the French GPs without significant difference between the Dutch and the French GPs.

Feasibility study 2

Qualitative results

The results of the analysis made clear that GPs had no major problems filling in the modified GFQ. The comments of the Belgian, Dutch and French participants did not differ. The practicability was good and using the GFQ took only a small amount of time.

"I'd fill it in, and then it was really just a quick question-and-answer process, so it was easy going". (French GP n°22)

"I don't remember having problems or saying to myself « it doesn't work" […] but

honestly, I actually had a feeling of fluidity". (French GP n°24)

The GPs did not consider it a burden.

"First consultation of each day, yeah, it was easy, the questions were precise enough so that it did not take three hours from the middle or at the beginning of a general practice consultation". (French GP n°21)

As in the first feasibility study, several GPs did not fill in the GFQ right after the consultation but at the end of the office hours or of the day. They did not want to interrupt the sequence of consultations with the questionnaire. They did not report recall difficulties when answering the questionnaire.

"And there are some, I guess about half of them, I filled in immediately after the consultation; the rest were done in the evening, when I get to the end of the list of patients and thus fill in the register, so I filled in some of them, but the majority, more than half of them, were completed just after the consultation". (Belgian GP n°8) Two Dutch GPs stopped filling in the GFQ after the first question and misunderstood the formulation of this first item: "Please indicate what kind of gut feeling you have at the end of the consultation. If you cannot answer this question now, please answer the following nine questions, then give your answer to question 1, which is repeated at the end of the questionnaire". They did not reply to the next nine items (n°2 to n°10). Some participants stressed the role of the instructions before filling in the questionnaire for the first time. They highlighted the distinction between gut feelings in their own decision-making process and feelings of empathy towards a patient regarding a bad prognosis.

"I can have an uneasy feeling -when it all fits. For example, I had a man with haematuria and no dysuria, no pollakiuria, a smoker. I thought, 'this is wrong, it's all about bladder or kidney cancer'. At the same time I thought, 'It's all right, I feel comfortable with the further approach I have in mind... I feel comfortable that the story is clear, namely, that it's very straightforward; I also know what to do now, but I know too that the outcome will not be very good". (Dutch GP n°30) Item 8 was a bit confusing for some participants. They did not understand that this item asked for the first three diagnoses that came into their minds and mentioned the same diagnosis in both items 8 and item 10.

"It was the question "Which diagnosis determined your course of action?" Yeah well,

I found that… it may be just me who did not understand the fact that it was potentially in the plural for question 8, but I had the impression of an overlap of questions 8 and 10 because, "Which diagnosis/diagnoses are you thinking about?" obviously includes the diagnosis which determines my course of action". (French GP n°22) Some participants also described how their gut feelings arose or disappeared during a consultation. They would have had some space in the questionnaire to describe their diagnostic reasoning process.

"So we just have some cognitive dissonance at the time, which is to say that, in the evening, we start thinking "perhaps I should have done something different" or "you feel at ease, you close that file and move on to something else"". (French GP n°26)

Based on this finding we added a sentence at the end of the questionnaire, allowing the participants to share some thoughts about their diagnostic reasoning.

Quantitative results

Out of the 263 questionnaires collected during this second phase (8 to 11 GFQs per GP), 259 were analysable, 7 were non-analysable because of missing data. Eighty two (31 %) were concerned with a sense of alarm and 177 (69%) with a sense of reassurance felt by the GPs. There was no significant difference between the Belgian, Dutch and French GPs' answers. They expressed the same prevalence of the sense of alarm and the same prevalence of the sense of reassurance. The median average time estimated by GPs for filling in the GFQ was 2 minutes for the Belgian, Dutch and French GPs. A PCA confirmed unidimensionality with one component explaining 72.3% of the total variance. The internal consistency was high (Cronbach's alpha = 0.90).

We compared the prevalence of the sense of alarm and the sense of reassurance in both feasibility studies with a Chi 2 test. There was no significant difference between the two samples.

After the second feasibility study, we added some minor changes to the items 1, 8, and 11. We rephrased item 1, adding "If you cannot answer this question now, please answer the following nine questions, then give your answer to question 1, which is repeated at the end of the questionnaire". We wrote both singular and plural forms of "diagnosis" in item 8, preferring the formulation "have in mind" instead of "thinking about" and suggested "max. 3": "What diagnoses (or diagnosis) do you have in mind? (max. 3)". We added the following sentence after item 11: "If you want to share some thoughts about your diagnostic reasoning, please use the back of this questionnaire".

(Figure 4) We also agreed on the instructions prior to filling in the questionnaire. In these instructions we explain how items 2-7 are derived from the definitions of the sense of alarm and the sense of reassurance and how to fill in item 1, item 8 and item 10. In order to minimize rationalizations afterwards we also emphasized to immediately fill in the questionnaire to grasp GPs' experience during the diagnostic process (preventing recall bias) for each patient that needs to be included in the study (preventing selection bias). The instructions should be embedded within the context and aim of any study [11]. (Text box 1). In this particular study we specified to fill in the questionnaire for the first consultation of the day with an adult patient, aged over 18 years, with a new reason for a consultation.

Discussion

Through a two-step study, we evaluated the feasibility and practicability of the GFQ in real practice. The main objective of this questionnaire was to determine the presence or absence of gut feelings in GPs' diagnostic reasoning and to differentiate between the sense of reassurance and the sense of alarm by precise statements which reflect the outcomes of the diagnostic reasoning process. The first step, a think-aloud study and a feasibility study, led to small modifications concerning the order of items, and to some small adaptations of the wording of two items. The modified version of the GFQ was created without altering the sense of the seven validated items. The second step, a repetition of the feasibility study but with the modified questionnaire, led to minor changes. The prevalence of gut feelings in the two phases of the feasibility study were similar in Belgium, France and the Netherlands showing that GPs experienced a sense of alarm in 23-31% of the reported cases.

Strengths and weaknesses of the study

Seventy GPs from Belgium, France, and the Netherlands were involved in the evaluation of the questionnaire in real settings. The same misunderstandings and difficulties in filling in the questionnaire occurred in all three countries. In addition, a similar prevalence of both the sense of alarm and the sense of reassurance was found in all three countries. Even though French GPs do not have an idiomatic expression for gut feelings, unlike Dutch and Belgium GPs ("pluis/niet-pluis"), the GFQ measures their sense of alarm and sense of reassurance in the same way [13]. The linguistic validation procedures used to translate the GFQ from Dutch to English and then from English to French, has been found to guarantee the cultural transposition from Dutch to French [8]. In spite of the differences between health care systems, the French and the Dutch versions of the GFQ do examine the same phenomenon. The GFQ is also feasible across practice settings in different countries. The internal consistency of the original Dutch language GFQ was high (Cronbach's alpha 0.90) as shown in the validation study [4] and continued to be high in the two cross-border feasibility studies (respectively 0.88 and 0.90). The outcomes of the factor analysis in both feasibility studies were similar to the original validation study. Our studies reaffirmed the transculturality of the gut feelings concept [13,14].

Testing the use of a questionnaire such as the GFQ in two different settings (thinkaloud in an experimental environment first, and then during office hours, in three different healthcare systems) was, as far as we know, quite unique. However, it enabled us to adapt the questionnaire in response to the participating Belgian, Dutch and French GPs' opinions and pragmatic concerns.

We started with the item asking for the presence of a gut feeling in diagnostic reasoning to capture their experience immediately after the consultation: the first item is now "Please indicate what kind of gut feeling you have at the end of the consultation". We repeated this item at the end of the questionnaire for those participants who were not able to answer this question at the beginning. It was only for participants who did not answer item 1 that we used item 11 as the indication of the presence or absence of a GF. There might be a risk that the last group will also use their analytical reasoning in finding an answer to item 11 but, in any case, we reduced that risk by also putting the question at the top of the questionnaire. To minimize rationalizations afterwards we emphasized in the instructions to immediately fill in the questionnaire to better grasp GPs experience during the diagnostic process.

The prevalence of the sense of alarm seemed to be higher in the second feasibility study than in the first one (23% vs 31%) but statistically there is no difference. Both studies took place in winter, with the same incidence of diseases. It can be an accidental finding which is confirmed by the fact that there is no statistical difference.

Further studies with the GFQ in clinical practice are needed to examine the prevalence of gut feelings in general practice and its predictive validity in different contexts.

The questionnaire was modified after the two phases of the study. Now we have a questionnaire formatted by GPs, for GPs, working in three European countries. A few weeks after the start of the studies, 600 questionnaires had already been included which is remarkable and might indicate how practical the questionnaire is in daily practice. Including research while practising is quite unusual for GPs [15]. Lack of time is usually given as the major cause of limited GP availability other than for patient care [16]. Quite a number of the GPs failed to follow the full instructions given prior to the feasibility studies. It did not always appear to be feasible to fill in a questionnaire right after a consultation. These GPs mentioned, however, that when responding to all the items, they were able to recapitulate the information regarding the patients involved without any problems. None of them mentioned that it could have induced a recall effect. We have highlighted this point for attention in the instructions (See Text box 1).

Several studies measured gut feelings with other definitions than the one we used here. For instance, Turnbull et al used in their questionnaire "my gut feeling is "something is wrong"": yes or no, whereas gut feelings were explained in the instruction booklet as "gut feeling that the child's illness may be more serious than is superficially apparent" [7]. Several other studies measuring gut feelings do miss a detailed and accurate definition of the sense of alarm [5,6]. In the study measuring the predictive value of gut feelings for serious infections in children [5], a precise definition of what was considered as "gut feelings" or 'instinct' is not available. In another study regarding the recognition of sepsis in primary care [6], the authors did not give details of the concept or definition to which they were referring when using the expression "gut feeling". In the questionnaire they used, one item was "How important were the following patient assessment aspects in the decision to refer?" "A gut feeling" was one of the possible choices. In our study we measured gut feelings more accurately. The concept of gut feelings in a closed question is not clear enough, and allows for differences in interpretation of different participants, especially within different languages and cultures. The sense of alarm and the sense of reassurance, as they were defined by Stolper et al., were considered, after linguistic validation procedures, as a transcultural concept validated in four languages [1,8].

Using such different definitions and measures of gut feelings in different contexts, it is not possible to compare the prevalence of the sense of alarm in different studies. The use of the GFQ is a uniform way of measuring the sense of alarm when diagnosing patients in primary care and to determine its prevalence.

Implications for practice and future research

To fill in the questionnaire right after a consultation gives GPs, or GP-trainees, the opportunity to reflect on their decision-making process. They may thus become aware of gut feelings and how they play a role in their diagnostic reasoning. The GFQ is a useful tool for eliciting reflection on diagnostic processes.

GPs' experience is probably positively related to the prognosis of serious diseases: the greater the experience, the more likely it is that GPs' predictions will be correct [7]. The GFQ can be used to study this relationship further. In the area of education, how both the sense of alarm and the sense of reassurance play a role in decision making should be addressed as an important non-analytical track of diagnostic reasoning, especially in general practice [2]. However, insight into the way gut feelings are used, and the role of experience should be refined through further studies.

With the final version of the GFQ, prospective observational studies in daily practice can be conducted. A study concerning the accuracy of GPs' sense of alarm when confronted with dyspnoea and/or thoracic pain has already been performed [10]. The results of this study will show the diagnostic test properties, such as the sensitivity and specificity, and the positive and negative likelihood ratios of GPs' sense of alarm, when applied to dyspnoea and chest pain. The relationship between gut feelings and the diagnosis of cancer can be calculated in this way, just as the relationship between gut feelings and the outcome of referrals, or non-referrals, to hospital specialists can be gauged. Knowing to what extent the sense of alarm acts on the decision of a GP in the real context of consultations for non-specific symptoms in primary care is the determining factor. The purpose of the questionnaire is to determine the presence or absence of gut feelings in diagnostic reasoning. These gut feelings are defined as a sense of alarm and a sense of reassurance. A 'sense of alarm' implies that a GP worries about a patient's health status, even though he/she has found no specific indications yet; it is a sense that 'there's something wrong here'. A 'sense of reassurance' means that a GP feels secure about the further management and course of a patient's problem, even though he/she may not be certain about the diagnosis: everything fits in. The items 2-7 of the questionnaire are derived from these definitions. In item 8 you will be asked to suggest a maximum of 3 diagnoses you have in mind concerning the patient. In item 10 you will have to write which diagnosis you used to determine your course of action. In order to avoid selection bias and to reflect your experience during the diagnostic process, we urgently ask you to fill in the questionnaire for each patient that needs to be included in the study directly after the consultation. Please, read the questionnaire, so we can discuss any questions you might have. Ο not yet take action, wait and see.

Ο not yet take action, but advise the patient to come back if the problem persists.

Ο not yet take action, but invite the patient for a follow-up appointment either face-to-face or by phone. Ο order further testing (laboratory tests, X-rays, etc.). Ο order further testing, and in the meantime, I will start treatment (medicinal or other). Ο start treatment, but will not arrange a follow-up. Ο start treatment and give the advice to the patient to come back if the problem persists. Ο start treatment and invite the patient for a follow-up appointment either face-to-face or by phone. Ο refer the patient.

Which diagnosis has determined your management?

……………………………………………………………….

11. This question is the same as question Ο not yet take action, wait and see.

Ο not yet take action, but advise the patient to come back if the problem persists.

Ο not yet take action, but invite the patient for a follow-up appointment either face-to-face or by phone. Ο order further testing (laboratory tests, X-rays, etc.). Ο order further testing, and in the meantime, I will start treatment (medicinal or other). Ο start treatment, but will not arrange a follow-up. Ο start treatment and give the advice to the patient to come back if the problem persists. Ο start treatment and invite the patient for a follow-up appointment either face-to-face or by phone. Ο refer the patient.

Research data policy

Materials described in the manuscript, including all relevant raw data, will be freely available to any researcher wishing to use them for non-commercial purposes, without breaching participant confidentiality.

BACKGROUND AND RATIONALE

General Practitioners (GPs) face both benign and serious diagnoses sometimes presented with the same vague and nonspecific symptoms. [1] That is why consultations in primary care are described as complex, dealing with uncertainty and unpredictability "on the edge of chaos".

[2] In these situations, occasionally GPs experience an uncomfortable feeling that something does not fit in a patient's clinical presentation. This feeling alerts the doctor. It activates the diagnostic process and induces him or her to initiate specific management to prevent serious health outcomes. The phenomenon is recognized by most GPs within Europe. [3] The concept of "gut feelings" was further investigated in the Netherlands in 2009 and is considered to be a specific kind of non-analytical reasoning. [4] Two types of gut feelings have been discerned: a sense of alarm (SA) and a sense of reassurance (SR). A sense of alarm implies that a family physician worries about a patient's health status, even though he/she has found no specific indications yet; it is a sense of "there's something wrong here" [4]. A sense of reassurance means that a family physician feels secure about the further management and course of a patient's problem, even though he/she may not be certain about the diagnosis: "everything fits in". [5] In situations of uncertainty, gut feelings may play a substantial role in the diagnostic process of GPs. [6] Until recently, no studies on this topic were available. An international network group called the European Expert Group on Cognitive and Interactive Processes in Diagnosis and Management in General Practice, or COGITA for short, was established, aiming to coordinate and stimulate research into the significance of non-analytical diagnostic reasoning such as gut feelings. GPs, psychologists, PhD students, linguists and medical students from eight European countries are active in this network group. The Dutch, Belgian, German, French, Polish, English, Swiss, and Swedish COGITA group members usually meet during an annual one-day conference. COGITA is a special interest group linked to the European General Practice Research Network (EGPRN). [7] One of the main objectives of the COGITA expert group was to define specific terms used in their publications. Therefore, a short glossary of terms related to the gut feelings topic was constructed. The objective of this paper is to present this Glossary of Diagnostic Reasoning Terms.

How we reached consensus

During a COGITA meeting (in Krakow, 2011) we first decided on a list of appropriate terms to include in the glossary. Several criteria were used for the selection of a list of terms. The frequency of use of terms in this area was the first one. For instance, "analytical and nonanalytical reasoning" were major concepts to define because of their occurrence in literature dealing with decision making. Theoretical consideration was another criterion for the selection of terms. The concept of "Pattern recognition" was seen to be important to define in the domain of non-analytical reasoning. Whether the target group of GPs has experience with or knowledge of certain research methods important for the research domain of diagnostic reasoning, was the last criterion: research method terms as "Delphi Round procedure" or "nominal group technique" seemed important to describe for further research.

Then, certain COGITA members put together a definition of each term, based on literature on the topic. The search terms used in the literature research were Mesh terms when they existed (diagnosis, heuristics, intuition, medical decision making, problem solving, pattern recognition, uncertainty, focus group, grounded theory, nominal group technique) or the words in free text if it did not. Our next step was a consensus procedure inviting comments and adjustments by other members of the COGITA group. Finally, we gathered all comments, adjustments and additions, discussed them in detail in a meeting and formulated the final text.

THE GLOSSARY AT A GLANCE

Each term starts with a short description after which the reader can follow a link to more information including relevant references. The glossary is available on the website http://www.gutfeelings.eu/glossary-introduction . Sixteen terms relate to the diagnostic reasoning process and eight terms relate to research methods.

Diagnostic reasoning part

In the diagnostic reasoning part, the following 16 terms or concepts were defined: analytical and non-analytical reasoning, Bayes and likelihood ratio, clinical mindlines, cognitive continuum, consistency, contextual knowledge, diagnosis, gut feelings, sense of alarm and sense of reassurance, heuristics, intuition, medical decision making and medical problem solving, pattern recognition, rules of thumb, system 1 and system 2 (dual process theories), tacit knowledge and uncertainty.

For instance, what are the differences between rules of thumb, clinical mindlines, heuristics and tacit knowledge? These four expressions refer to closely related concepts but the glossary describes their different features using relevant literature. How to deal with the umbrella term "intuition"? The glossary provides a definition based on references from Damasio AR [8],

Epstein S [9], Finucane M [10], Glockner A [11], Hogarth RM [12], Kahneman D [13], Klein G [14], and Slovic P [15]. We describe different kinds of intuition. Non-analytical reasoning and analytical reasoning, in system 1 and system 2 respectively, are defined and models of medical decision-making, medical problem-solving and gut feelings are described. As an example, the definition of "tacit knowledge" is provided in Box 1.

Research method part

In the research method part, the eight following terms were defined: construct validity, Delphi consensus procedure, focus group, Grounded Theory, linguistic validity, nominal group technique, saturation and triangulation. As an example, the definition of "triangulation" is provided in box 2.

STRENGTHS AND WEAKNESSES OF THE GLOSSARY

As far as we know, defining terms in the area of decision making by a European expert consensus based on a literature review and gathering them in an open access glossary is a unique initiative. This glossary was a prerequisite to conduct further research with the intention to create teachable knowledge as well as a basis for cross-border research in general and family medicine. We made the glossary freely available in order to share our results with other researchers, and to extend our scientific network. The glossary has been constructed based on the prevalent literature at this point in time and needs continuous effort of the expert group to be updated in line with new research findings and theoretical insights and elaborated with new relevant concepts.

THE FUTURE

Constructing this glossary was an original proposition by European researchers involved in the diagnostic reasoning domain. It provides pragmatic, consensual and referenced definitions useful for researchers working in this field. Members of the COGITA group invite interested researchers to propose additional terms and definitions to complete the glossary. New terms should be defined following the same procedure, according to the researchers' criteria. This approach is a helpful base for further research in this field.

POTENTIAL CONFLICT OF INTEREST

The authors declare that they have no competing interests

FINANCIAL RESOURCES

The COGITA group received some financial support from the European General Practice Research Network (EGPRN).

Box 1. The definition of tacit knowledge Box 2. The definition of triangulation TACIT KNOWLEDGE

Tacit knowledge (as opposed to formal or explicit knowledge) is the implicit (personal) knowledge that is not directly accessible and difficult to transfer to another person by means of writing it down or verbalizing ("that which we know but cannot tell"). The term and its definition were first developed by Michael Polanyi (1891-1976). Tacit knowledge has been described as "know-how" as opposed to "know-what" (facts), "know-why" (science) or "know who" (networking). [16-21] Tacit knowledge can be acquired via informal and implicit learning or gained through personal experience. Tacit knowledge may lead to routine action (or habits) and culture that we do not recognize in ourselves. By paying attention to the concept of tacit knowledge, we may have a starting point to make sense of the place of intuition in informal educational practice and in medical professional practice. In medicine, both patients and medical professionals are equipped with a wealth of tacit knowledge about health and health needs. It is a challenge to use this knowledge in decision-making processes, in parallel and complementary to explicit 'evidence-based' knowledge.

TRIANGULATION

Triangulation is a term in qualitative research methods derived from navigation, in which sailors try to discover their exact position on a map by taking bearings on two landmarks. The methodological triangulation is the most frequently applied approach, using different methods when studying a subject [22] such as the combination of ethnographic observations with interviews or the mixed methods approach which is mixing qualitative and quantitative methods in one study. [23] Denzin discerned three other types of triangulation next to the methodological triangulation: data, investigator and theory triangulation. [24] Data triangulation means that diverse sources of data are used studying a phenomenon in different settings; it results in a richer description of the phenomenon. [22] In investigator triangulation multiple observers in the same research field continually discuss their observations and interpretations through this de-biasing their personal preferences. Theory triangulation means that researchers approach their data with several hypotheses exploring how the data fit in each hypothesis.

93

Chapter 5 "I can't find anything wrong: it must be a pulmonary embolism": diagnosing suspected pulmonary embolism in primary care, a qualitative study PLos One 2014;9:e98112.

Introduction

Pulmonary Embolism (PE) is a serious pathology which has to be identified quickly: the mortality rate is high, with 18% of patients dying within 3 months [1].The incidence of PE ranged from 23 to 60 per 100 000 [2,3]. PE was clinically suspected in fewer than half of all fatal cases [4]. Because of the low rate of autopsy, the actual incidence is likely to be higher [2].

Uncertainty is an inherent part of primary care [5,6]. Signs and symptoms are often vague.

Dyspnea and thoracic pain are signs indicating multiple pathologies from the benign to a lifethreatening PE [7]. FPs have to select patients with a serious pathology in order to refer them to secondary care or an emergency unit. They are torn between missing a patient with a hypothetical PE and referring too many patients for harmful and costly investigations.

Pulmonary embolism was one of the most frequently reported missed diagnoses in primary care [8]. However, only 10% of cases of suspected PE turned out to be actual pulmonary embolisms [9].

How to safely exclude a diagnosis of PE is now well described. A literature review asserted the efficiency of the Wells rule and a D-dimer test in excluding the diagnosis [10]. This diagnostic approach was also validated in primary care. A Wells score < 4 combined with a negative point of care D-dimer test were proven safe and efficient for the exclusion of a PE diagnosis in primary care [11].

In fact before using any prediction rule oriented towards this particular diagnosis, the GP should have some suspicion of PE and it is precisely this initial stage which is unclear. The diagnostic process leading to the suspicion of PE is not well described in primary care. The objective of this study was therefore to explore how FPs came to suspect pulmonary embolism in real settings: how the suspicion of PE developed in the diagnostic reasoning process of French FPs. A second objective was to describe more specifically the role of gut feelings in this diagnostic process.

Materials and Methods

A qualitative approach was chosen because this type of research would enable us to explore the meanings of diagnostic signs and symptoms used in the diagnostic reasoning process of FPs [12]. Individual face-to-face semi-structured interviews were carried out with FPs.

Interviews were chosen because we wanted to access to the personal experience of each FP, and not that of the group of FPs as a whole. We aimed to reveal issues which had not been documented previously [13]. It was conducted within a grounded theory perspective in order to describe from the data the way FPs perform the diagnostic process [14].

Ethics statement

This study was approved by the ethic committee of the University de Bretagne Occidentale.

The participants provided their written informed consent to participate in this study. The ethic committee approved this consent procedure.

Research team

The research team included two FPs (MB and PB) with academic backgrounds and two female trainees (NM and AC) in family practice doing their Master's degree. After being coached in interview techniques, the two trainees conducted all the interviews. The theme of pulmonary embolism was citesd during the call to request an appointment.

Participant selection

Two groups of participants were selected. The first group consisted of 14 FPs who had referred a patient to the emergency unit of the local hospital in the area of Brest, Brittany and where a pulmonary embolism was ultimately diagnosed. Data of all the cases of pulmonary embolism from June to November 2011 were collected. The patient's FP was identified where he/she had been the physician who had sent the patient to the emergency unit. We had no information on the reason why the physician referred the patient to the emergency unit when we interviewed him/her. We undertook the FPs' interviews a few days after the positive diagnosis of PE. The first consecutive 14 FPs who agreed to participate made up the first group. A second group of FPs was chosen using a purposeful sampling method. The aim of selecting a second group was to include the widest possible range of perspectives, experiences, points of view, and, in particular, to enrich the sample with FPs from rural areas who had not referred their patients to the regional hospital. Sixty FPs were approached through a phone call during which the theme of the interview was presented. Fourteen agreed to participate and they made up the second group. Reasons to decline included prior engagements and lack of time.

Information on participant profiles is detailed in the "results" section.

Data Collection

The research team developed the interview form with a topic guide, drawn up to answer the research question (see table1). It was composed of open-ended questions for exploration and closed questions to refine the participants' answers. The first question focused on the case report of a hospitalized patient for the first group and a recollected consultation about a positive diagnosis of PE for the second group. In order to match what emerged from the interviews, we added new questions to the interview guide when participants raised aspects which had not previously been mentioned [15]. All the interviews were audio-recorded. The recordings were transcribed and checked. All the interviews took place at the FP's office. In order to improve validity and credibility, all transcripts were returned to participants for member checking. The duration of the interview was between 5 and 40 minutes. We considered data saturation achieved when no new code emerged from the analysis of the verbatim accounts. It occurred after the twelfth interview in the first group and after the ninth interview in the second group.

Data analysis

A thematic analysis was conducted using the technique of constant comparison, originating from grounded theory [16]. The first stage: open coding was done by two researchers (AC and NM) working independently without any framework for the written data. After the first stage, they shared their results. Any discrepancies between the two researchers were discussed with a third member of the research team (MB) until a consensus was reached. Through an iterative process of constant comparison, an axial coding framework was developed at the second stage. The axial coding involved linking categories found within the open coding. The same procedure of working independently before sharing the results was applied. The codebook was revised, with the 3 researchers going back to the data until mutual consent was reached. QSR N vivo 10.0 Software was used to perform the analysis.

Results

Sample

Participants

Characteristics of the participants and practices are summarized in table 2. One of the FPs interviewed had been a mentor to one of the trainees during the training period.

Case characteristics

10 FPs in the first group were correct in their suspicion and referred their patient to the emergency unit for PE. The 4 other FPs referred their patient for other reasons without having any suspicion of PE but thought about pericarditis, infection in a COPD context, pneumonia and coronary heart disease. In the second group, one FP had not even suspected a PE.

Analysis

Analysis of the text fragments resulted in 65 open codes, which after an inductive interpretation and categorization process could be structured in 16 axial codes and 3 main categories (table 3).

Key points

A polymorphic clinical picture Clinical signs, which allowed FPs to form their suspicions, were so varied as to be hardly recognizable, according to the FPs interviewed. Chest pain in a PE context could occur both during effort, as well as during deep inspiration and could even be reproduced by physical palpation. The location, intensity and duration also varied without any specificity. The dyspnea was described with variable intensity from a one-off incident to almost intolerable tachypnea.

Tachycardia was considered to be a helpful sign. Combined dyspnea and thoracic pain with tachycardia was directly associated with a PE diagnosis.

"As for the symptomatology, it is very diverse. Personally, I saw many… some punctiform pains, some which appeared to be muscular, others which increased on palpation… It looked like nothing!" (P17, female FP, rural practice, 31, group 2) "He was gasping like a stranded fish! " (P10, male FP, rural practice, 57, group 1) "She was a bit breathless" (P18, female FP, urban practice, 51, group 2) "I was very surprised because the heartbeat was very fast" (P27, male FP, urban practice, 57, group 2)

The association between dyspnea, thoracic pain and symptoms of thrombosis facilitated the identification of PE.

"If the thrombosis had not been there, I think it might have passed unnoticed because it was completely atypical." (P23, female FP, urban practice, 31, group 2)

"She told me she had been breathless for 10 to 15 days, so I noted 10 to 15 days and she had been suffering with pain in her right calf since the previous weekend." (P14, male FP, urban practice, 50, group 1)

The patient's anxiety was viewed differently by different FPs. The anxiety was integral to the global picture for the FPs who had recently established a positive diagnosis. They considered it a strong clue guiding them to suspect PE. For other FPs, the anxiety was related to the dyspnea and had no specific connection with the diagnosis of PE.

"Well, yes, I did find something else… and that's anxiety. I mean, I noticed, the dyspnea really frightened the patient, compared with other dyspnea. What alarmed me was a dyspnea that was frightening for the patient, not a usual "lambda" common dyspnea. There was anxiety and stress." (P25, male FP, rural practice, 62, group 2) "When something is wrong I think they are all anxious." (P16, female FP, urban practice, 34, group 2)

The absence of indicative clinical signs for other diagnoses in patients with specific complaints was in itself a strong clue, likely to evoke the diagnosis of PE. Compared to coronary heart disease and pneumonia, PE was described as a pathology with very few symptoms. Four FPs did not suspect PE before referring their patient to the emergency unit. The signs and symptoms presented were interpreted in different ways. The pattern of other pathologies, such as pericarditis or coronary heart disease, came to mind. The diagnostic process, leading to suspected PE, was dependent on the possibility of eliminating other potential pathologies. For these reasons the diagnosis of PE was considered a complex issue in primary care.

"But it's true that if we have nothing to go on, no sign of cardiac insufficiency, and we see a

person who is breathless or is in pain… When there is not much to go on, I will consider it…" (P9, female FP, rural practice, 30, group 1) "She described effort-related chest pain, which lasted for 2 days. No dyspnea, no tachycardia.

I did an ECG which was normal, no signs of acute coronary insufficiency, no ECG deviation from normal. I had a case of clinical angina in a patient with a long history of coronary heart disease, even though the ECG showed no change. I sent her to the emergency unit for additional exploration into chest pain which was suggestive of acute coronary syndrome."

(P13, male FP, urban practice, group 1) "It's complicated. The clinical picture is different from one case to another sometimes very poor or virtually non-existent. No chest pain, no shortness of breath ... Sometimes just tachycardia.

And then a pulmonary embolism can cause death ... It is a complicated diagnosis, very complicated." (P13, male FP, urban practice, 59, group 1)

Tools used to help decision-making: ECG and the D-dimer test

The FPs were asked about the tools they used to help them in their decision making process.

The electrocardiogram had an ambivalent place: for some it was important to eliminate other diagnoses such as coronary heart disease. For FPs who had already considered PE, the ECG was seen as a waste of time. Its result would not change their management plan.

"In my case, I did an ECG which was normal. When there is shortness of breath I systematically do an ECG. In general there is no sign." (P25, male FP, rural practice, 62, group 2) "Yes yes, we have an ECG. We do the examination but it's true that I don't check for PE. If there is any doubt and if the patient is not well … I consider it a waste of time. It will be done at the hospital. The ECG is useful for other diagnoses. When you know the patient will be referred I don't waste my time" (P16, female FP, urban practice, 34, group 2) "If I consider there is typical thoracic pain or a pulmonary embolism and the ECG is normal, I will override the ECG result and refer the patient." (P13, male FP, urban practice, 59, group 1)

The D-dimer test was considered only a minor aid in the positive diagnosis of PE for two reasons. The first is a practical consideration: in France the test cannot be done at the FP's office but only in a laboratory. The FPs found this process too time consuming. The second reason was their need for a clear answer. The result of the test was not seen as sufficiently discriminating by some physicians. The high negative predictive probability was seen as a hindrance rather than a help in the diagnostic process.

"Yes, but in that situation, at that moment… Anyway I thought she should be hospitalized. I could have requested the D-dimer test at the laboratory, yes, I had considered that but, in view of the history of the patient, I wanted to be certain." (P24, female FP, urban practice, 60, group 2) "It is not very reliable. It has a negative predictive value, you know. This is often more annoying than anything else… » (P13, male FP, urban practice, 59, group) All the FPs interviewed stated they did not use the PE prediction rules. Most of the FPs did not know of any rules for this pathology. Younger FPs were familiar with it but did not use it in daily practice. Older FPs overreacted when the scores were mentioned because many were not familiar with them. They reacted as though they were being judged just because the interviewer had posed the question. All the FPs considered it an unsuitable tool in primary care. A scoring system was treated as a way of quantifying the severity of the condition in an emergency unit.

The FPs who did consider the scoring system only did so because they already believed the patient needed to be referred to the hospital. The result of the score would have had no influence on their decision-making process in the office. The FPs felt the scoring system was impersonal and at odds with a patient-centered approach and a good relationship with their patient. They insisted on the global view they had of the situation based on their examination and knowledge of the patient. The figure given by the score was seen as disconnected from the actual patient.

"Interviewer: and what about you? Do you use other things to diagnose PE, rules for instance?

FP: no, no. The rule… you mean the scale… I don't bother with that. I: Do you think they are useful in family practice? FP: no, that's for hospital physicians, they have time to classify patients." (P7, male FP, rural practice, 57, group 1) "We are working alone; we finished our studies a long time ago. Well, of course, we don't say so but we do our examination, we look for a thrombosis, look at their previous history. We don't call it "Wells or Geneva" but we do have our clinical examination procedure." (P8, female FP, urban practice, 48, group 1) "I don't normally use them, but my examination and my knowledge of the patient ticks a lot of the boxes. I can build a global picture." (P22, female FP, urban practice, 37, group 2) Some factors were highlighted by FPs working in rural areas. While FPs in urban areas could easily send the patient to the lab for a D-dimer test, where there was any doubt, rural FPs had to weigh the pros and cons carefully before referring to a specialist. They considered context to be particularly significant. They said that the distance their patient would have to travel to the hospital influenced their decision.

"We have nothing […] And there are only two doctors in the area who can do Doppler tests. It

is not that simple. You have to refer to the hospital. Here there is nothing, no lab. They have to go to the lab or the nurse has to come to their home. If we ask for the D-dimer in the morning, we will not have the answer until 3 pm. If we are in any doubt, we have to refer to the hospital." (P26, male FP, rural practice, 65, group 2).

"and, in an emergency context, you just have to call and you will have a scintigraphy or a CT angiogram, you have it very quickly. The lab test, it is done in one hour in the lab next door"

(P3, male FP, urban practice, 61, group 1)

The seeds of suspicion FPs told us of traumatic cases they had experienced, which paralleled the case they were discussing in the interview. A misdiagnosis with fatal consequences and a delay in diagnosis, which proved damaging for the patient, were described. These traumatic experiences reinforced the anxiety surrounding the pathology. They built a frightening picture of PE in the FP's mind. Some FPs even said that their approach to patient care was more conditioned by this traumatic experience than by objective elements.

"Well, she had a vicious bout of influenza, she was lying in bed and I didn't prescribe any anticoagulant. It was many years ago, at that time it was not codified as it is now. I feel guilty enough admitting it, but at least, I will step up to the plate. That's the story and ever since this case I think I have been traumatized by pulmonary embolism! It always crosses my mind."(P8, female FP, urban practice, 48, group 1) "Yes… In my case, I had a patient who died at the beginning when I was covering for an FP… Now as soon as I think there might be venous disease, I follow it up." (P18, female FP, urban practice, 51, group 2) "I saw a case of thrombosis during pregnancy, actually, it was not my patient but my brother's niece and she died at the end of her pregnancy of a pulmonary embolism. It was a dramatic situation." (P26, male FP, rural practice, 65, group 2)

A change in a patient's attitude or behavior pattern led one FP to the diagnosis. A patient usually seen at the office asked for a home visit in an emergency and that alerted the FP. In general, FPs know their patients very well, which allows them to detect a change in the patient's condition. These changes noted in the context of a consultation, along with a sudden change in the patient's condition, were sometimes the only symptoms driving the FP towards further investigations.

"Yes during home visits, each time, with people who usually came to the office" (P18, female FP, urban practice, 51, group 2) "With a man I know well, who never complains. So I know that when he calls, it's something important." (P24, female FP, urban practice, 60, group 1)

"And I know my patients very well, I have been here for 40 years and I can tell when they look

ill! Yes I saw his face, and I knew that he was in trouble" (P26, male FP, rural practice, 65, group 2)

The FPs talked about the use of their perception in diagnosis: they sensed when something was wrong, although they were unable to underpin this feeling with objective arguments. The perception of a serious prognosis decided where the patient would be sent for treatment. The FPs needed further investigation because of the sense of alarm they experienced. This feeling was described in different ways: having a "nose", "a sense", "an intuition". Eighteen FPs from both sampling groups told us about this feeling. It was unrelated to the FPs' gender, age, experience or the location of their practices.

"We sense the situation... Even if not all the signs are present… We feel things, but I can't explain them." (P5, female FP, 34, locum, group 1) « There is a notion of intensity, of underlying seriousness but you don't really know what's going on. It's true that we do think about some things in that way […] I don't know how to explain that. In some cases we said it was hard… and fortunately, we did further investigation.

Yeah we did the right thing then ... we started something a little faster than usual and it was beneficial." (P25, male FP, rural practice, 62, group 2) « When it comes into your head, when you're convinced, I sometimes call it "having a nose" it is true that sometimes you don't know why and you say to yourself "that's it" and you go with it. Then you have to follow it up." (P28, male FP, urban practice, 39, group 2) « Yes clinical signs and sometimes we have the… we have the "feeling", we feel… we feel the… with converging lines of evidence, we feel that something does not fit". (P19, female FP, urban practice, 37, group 2) « Well, sometimes we found pains which looked rather dubious… We used our noses and when you know someone well, you can tell when he is not his usual self." (P26, male FP, rural practice, 65, group 2)

Discussion

Main results

The suspicion of pulmonary embolism arose out of four considerations: the absence of indicative clinical signs for diagnoses other than PE, a sudden change in the condition of the patient, a gut feeling that something was wrong and an FP's experience of previously failing to diagnose PE. The FPs interviewed did not use rules in their diagnostic process.

Strengths and limitations of the study

As far as we know, this is the first study describing FPs' diagnostic reasoning processes in cases of pulmonary embolism. The FPs interviewed revealed their diagnostic errors, and sometimes recounted dramatic stories of failure to diagnose PE. This openness during the interview confirmed that the ambiance created by the interviewers was appropriate. The data were based on real life experiences and not on general opinions.

The first group of FPs interviewed was composed of FPs who had had a positive diagnosis of PE. We did not interview FPs who had missed a PE. In accordance with the objective of the study, we focused on the diagnostic reasoning process in cases where PE was suspected. Some FPs did not think about a PE, but about another serious disease which needed emergency care. In the second group, one FP had never even suspected a PE. We decided to include these interviews to broaden the analysis.

Another limitation of this study is the recall bias related to the FPs recruited in the second stage. We did not present a specific case to start with and sometimes their stories had occurred months or years before. In the first group the FPs did remember the cases very well which made it much easier for them to describe their diagnostic reasoning.

Two young family practice trainees carried out the interviews, which could have influenced the content of what FPs talked about. The FPs interviewed knew the study was conducted by young FPs wanting to know what occurred in primary care, rather an intern or pulmonologist who might be judgmental. However, a more hierarchical, rather than collaborative, relationship might have developed between the young interviewers and the experienced physicians. This situation may have influenced the answers of one FP who had previously been the tutor of one of the interviewers. This factor might also explain some reactions to the question about rules.

The sample was selected using two recruitment procedures. The purposive sample allowed us to interview rural FPs, female FPs, and younger FPs (see table 2). It reinforced the maximal variation of the sample.

None of the FPs interviewed used either prediction rules or the point of care D-dimer test although it is the recommended strategy according to international guidelines. Our aim was to describe what was done in real French practices, and not in an ideal situation. Relying on gut feelings in the first stage of the diagnostic process does not exclude the usefulness of clinical decision rules such as the Well's rule. On the contrary, gut feelings should trigger the next clinical process and especially the use of rules such as the Well's rule. The fact that the FPs interviewed revealed their real practice which, although far from what is recommended, is nevertheless another illustration of the openness achieved during the interview.

Key points

The diversity of the clinical pictures of PE did not provide a foundation on which FPs could base their decisions. FPs interviewed in this study, attached importance to the presence of symptoms of thrombosis and patient anxiety. They mainly emphasized the absence of objective and indicative signs in parallel with patients' complaints. A sudden change in the condition of the patient was considered as the most important indication. This point had already been raised, in cases where coronary heart disease was suspected [17], as a reason for referral of patients with chest pain [18], and in cases of meningococcal infection in children [19]. Background knowledge about the patient and person-specific discrepancy were tools used where a serious condition was suspected. The results of our study contribute to the improvement of a specific strategy in primary care: knowing the patient, his risk factors and being sensitive to a discrepancy in the patient's behavior seemed to be decisive for FPs when clinical signs were vague but serious conditions were suspected.

The FPs also based their decision on what they felt during the consultation. That uneasy feeling experienced by the FPs interviewed matched the sense of alarm described in the "gut feeling" concept [20]. This sense of alarm implies that an FP worries "about a patient's health status, even though he/she has found no specific indications yet"; it is a sense that 'there's something wrong here'" [20]. Their sense of alarm involves specific management whereby the patient has to be referred to an emergency unit, or to a specialist to prevent serious health problems. The gut feeling concept was originally formalized from statements raised in family medicine in the Netherlands and in Belgium [21]. Its transculturality was proven in a Romance language [22].

The FPs interviewed in our study affirmed the existence of the sense of alarm in the specific situation of a suspected PE. Van Den Bruel et al. found that a gut feeling had a higher specificity than a clinical impression in the context of serious infection in children in primary care. The authors recommended that this gut feeling should not be ignored in diagnostic reasoning but has to be used as a red warning flag [23]. One factor which contributed to gut feeling was the perception of parental concern. In our study FPs who had recently diagnosed PE, attached importance to the anxiety expressed by the patient. They felt that this anxiety was indicative of PE whereas other FPs considered the anxiety to be the result of the dyspnea. In our study the patient's anxiety and the sense of alarm perceived by the FPs steered the diagnostic reasoning process.

The FPs interviewed did not use the Wells rule during their diagnostic process because this prediction rule was not considered a helpful tool for detecting PE in primary care. The Wells score, combined with a qualitative D-dimer test, safely and efficiently excludes PE in primary care [11]. This procedure provides a concrete way to deal with the suspicion of PE depending on whether the probability is high or low: if the score is ≤ 4, the probability of PE is low and a D-dimer test is required and if the score is > 4, the probability of PE is moderate to high and further investigations in hospital (compression ultrasonography, pulmonary vascular imaging) are required [24]. The Wells rule brings a synthesis between clinical symptoms, clinical signs and the physician's assessment with the allocation of three points to the physician's clinical judgement of whether PE is more likely than an alternative diagnosis. The physician may express his feeling of alarm with these three points. This first stage in the suspicion of PE, including the sense of alarm, should drive the FPs forward to the second stage of the diagnostic process using the Well's prediction rule. This strategy has a clinical impact in the FP's decision and fits the criteria of a relevant rule for decision-making in primary care [25]. How can we explain this non-use of a safe decision strategy by the FPs interviewed in this study? The use of the point of care D-dimer test is not widespread in France: all the FPs interviewed had to refer their patients to an independent laboratory to have the blood sample analysed. Most of the older FPs had never heard of the Wells rule; indeed, even if the younger ones had learnt how to use it during initial training at the university, they had not used the scores in the cases they related. All the FPs interviewed agreed about their willingness to use their global evaluation of the case, guided by their knowledge of the patient. For them, using a score did not fit the patient-centered approach [26]. This is consistent with one of the main aspects of the definition of family practice where the patient-centered approach belongs to the core competencies of the discipline [5]. Clinical impression, global empirical clinical assessment (also called "gestalt") are the tools used to qualify a holistic rather than an atomistic approach based on clinical context [27-29]. Lucassen et al. compared the predictive value of the gestalt and clinical decision rule when used in combination with D-dimer testing for excluding pulmonary embolism [10]. The sensitivity of gestalt was similar to clinical rules (Wells, Geneva, Revised Geneva) but its specificity was lower [10]. Using a point of care D-dimer test combined with the Wells rule was useful in reducing false positives [10]. The FPs interviewed apparently gave priority to sensitivity rather than specificity. This may be explained by the fact that, in the context of low incidence serious diseases [30], the FPs appear to attach greater value to truepositive decisions (correct decisions to provide care to patients who need it) than to truenegative decisions (correct decisions to withhold care from patients who do not need it).

Providing care for patients whom they suspect may be suffering from a life threatening disease, is possibly considered much more important than withholding care from a patient in good health [25].

Moreover the FPs interviewed told us about diagnostic errors they or their colleagues had made when diagnosing PE and how the sense of alarm popped up from their experience and knowledge. This description of the sense of alarm fits into the Reason's model of error prevention in decision making [31]. The experienced FPs function as expert decision makers: as a result of their experience, they have established a set of spontaneous reactions to patterns of diseases they identify at a glance [32]. These rapid, effortless and unconscious ways of thinking are named heuristics: they are considered as powerful tools [33] but with specific errors [34]. When the FPs are confronted by a new situation they have to call upon slow and demanding analytical reasoning. The "uneasy feeling" perceived by the FP corresponded to a perception of cognitive dissonance: the mismatch situation with pre-established patterns triggered the sense of alarm in the FP [32]. The perception of alarm compelled the physician to quit his routine-based reasoning to an analytical reasoning by generating and considering the PE hypothesis [35,36]. The sense of alarm acted as a feedback mechanism, allowing the questioning of a possibly wrong decision at a very early stage of the diagnostic process. This feeling is based on medical, experiential and contextual knowledge. The traumatic experiences FPs narrated during the interviews are images which are tagged with negative affect in their memories [37]. This shortcut was named the affect heuristic [38]. In the situation where PE was suspected, we may hypothesize that this affect heuristic emphasized the FP's sense of alarm, forcing him to switch to analytical reasoning. The sense of alarm was used here as a tool to prevent the diagnostic error of missing a PE.

Implications for practice and future research

The accuracy of the sense of alarm needs to be evaluated in the context of PE. The data were collected in a qualitative retrospective manner. We cannot generalize from our results without testing them in a prospective way on a large sample of situations in primary care. A short questionnaire has been developed to determine gut feelings in primary care settings [39]. We are aiming to use this questionnaire to study the predictive value of the sense of alarm when confronted with dyspnea and/or thoracic pain at the office. The objective is to compare the results to the questionnaire, filled in at the end of the consultation, with the diagnostic outcome four weeks later. Before implementing the gut feelings concept in educational programs, we need to study its accuracy in real settings.

Conclusion

This study aimed to describe an early stage in the diagnostic process of FPs who suspected pulmonary embolism at their office. The absence of indicative clinical signs for diagnoses other than PE, a sudden change in the condition of the patient, and the FP's experience of previously failing to diagnose PE, as well as a sense of alarm were the main determinants of the decision to refer. A decision rule was not used at all. The sense of alarm was used as a tool to prevent the diagnostic error of missing a PE. The diagnostic accuracy of this aspect of gut feelings has to be evaluated before being recommended or taught.

Introduction

Dyspnoea and thoracic pain are signs indicating multiple pathologies from the benign to lifethreatening pulmonary embolism or cardiac diseases. General practitioners (GPs) are sometimes torn between missing a patient with a hypothetical life threatening disease and referring too many patients for harmful and costly investigations [1].

In 2009, the concept of gut feelings in general practice was described, by means of a qualitative study, as a sense of alarm and a sense of reassurance [2]. The sense of alarm is an uncomfortable feeling, experienced by the physician, that something does not fit in a patient's clinical presentation although he/she has found no specific indications. The sense of alarm activates the diagnostic process and induces the doctor to initiate specific management to prevent serious health problems [2]. It was considered to play a substantial role in the diagnostic reasoning of GPs [3]. A Dutch Gut Feelings Questionnaire (GFQ) was created from the consensus criteria for gut feelings and validated by a construct validation procedure using case vignettes. The internal consistency of the gut feelings questionnaire proved to be high (Cronbach's alpha = 0.91), the Kappa with quadratic weighting was moderate to good (0.62, 95% CI: 0.55-0.69). [4]. A linguistic validation procedure was performed to obtain an English version of the questionnaire [4].

A linguistic validation procedure produced a French version of the GFQ using the same procedure as the Dutch team [5,6] as described in the article presenting the questionnaire [4].

123

Limitations of the existing literature In the Netherlands, GPs have regularly been blamed, in medical disciplinary tribunals, for failing to respond to this sense of alarm or even because of the lack of it [7]. This alarm bell should have signalled a dangerous situation in the clinical cases in question: the GP should have reacted in order to prevent an error. The sense of alarm was seen as a means of guaranteeing optimal care. It would have acted as an error-prevention tool [8]. Taking the wrong decision can lead to serious consequences, both for the patient's health and, at a judicial level, for the doctor concerned.

The accuracy of gut feelings as a diagnostic test was studied in the field of paediatrics [9]. Gut feelings had a higher specificity than clinical impression in the context of serious infection in children in primary care. The authors recommended that gut feelings should not be ignored but used as a red warning flag [9]. Gut feeling was considered a fairly accurate tool in the case of chest pain [10]. GPs interviewed in a qualitative study explained how they had followed their sense of alarm when they correctly diagnosed pulmonary embolism [11]. They used the sense of alarm as a tool for preventing the diagnostic error of missing a pulmonary embolism. To our knowledge, data are scarce on the diagnostic accuracy of the sense of alarm in primary care, especially for patients with complaints of dyspnoea and chest pain.

Objective

The aim of the study is to calculate the diagnostic test accuracy of the sense of alarm when applied to dyspnoea and chest pain, using the gut feelings questionnaire [4]. The sensitivity and specificity will be indicated by means of a contingency table, using horizontal lines to indicate life threatening and non-life-threatening pathology (in accordance with dyspnoea and chest pain) and vertical lines to indicate the presence or absence of the sense of alarm. This research design will enable us to assess the extent to which GPs can trust their sense of alarm when confronted with dyspnoea and/or thoracic pain.

Methods and analysis

We will be implementing a prospective observational study using the French version of the GFQ.

Participants

Patients aged between18 and 80 years, consulting their GPs for dyspnoea and/or thoracic pain will be considered for enrolment in the study. GPs involved in the General Practice Faculty of Brest University will be selected for the study. The participants will not be incentivised to take part. In order to increase their involvement, the trainee assisting them at the surgery will be accuracy of the test used to return this diagnosis. Life threatening and non-life threatening diseases have previously been defined according to the pathologies or symptoms in the ICPC2 classification. This document was formulated, following a consensus procedure, by a group of experts on the topic. Members of the current research team, blinded to the actual outcomes shown on the index questionnaire, will judge each case in turn and will, by consensus, classify the expected outcomes as either life threatening or non-life threatening diseases. The study will take place between the 30 th January and the 31 th July 2015.

Data analysis

The GFQ consists of 10 items. The first 5 items in the questionnaire are derived from the consensus statements from the gut feelings concept which describes the sense of reassurance (item 1) and the sense of alarm (items 2-5) [2]. The items are rated using a 5-point Likert scale ranging from completely disagree to completely agree. Items numbered 6, 7, 8 and 9 relate to the patient's care. The tenth item assesses whether the patient's case elicited a gut feeling (a sense of reassurance or a sense of alarm) or whether it is impossible for the respondent to say or even whether a gut feeling is not applicable. A sense of alarm will be considered as present when the answer to item 10 indicates a sense of alarm or when the answer to item 10 indicates that it is not applicable and at least one of the scores of items 2 to 5 is higher than 3/5. A sense of alarm is considered as not present when the answer to item 10 indicates either a sense of reassurance or when the answer to item 10 indicates that it is not applicable and the score for item 1 is higher than 3/5. The contingency table will be composed of horizontal lines to indicate a life threatening or a non-life-threatening pathology (in accordance with dyspnoea and chest pain) and vertical lines to indicate the presence or absence of the sense of alarm. The sensitivity, the specificity, the positive and negative likelihood ratio (LR+, LR-) will be calculated from the constructed contingency table.

Discussion

As far as we know, this is the first study aiming to define the diagnostic accuracy of the sense of alarm when applied to dyspnoea and thoracic pain, using the validated gut feeling questionnaire. The setting of this study is of major importance: GPs will fill in the GFQ during their decision-making process. We are providing quantitative data from daily practice situations.

One limit of the study may be the selection of cases. The definitions of thoracic pain and dyspnoea from Mesh terms are broad and GPs may focus only on what they think to be serious cases. We will, of necessity, take inclusion procedure into account during the presentation of the study, and emphasise the definitions of the inclusion criteria, thoracic pain and dyspnoea.

It is more likely that patients will be referred in cases where GPs experience a sense of alarm than in cases where they do not experience any gut feeling. The likelihood of being admitted for a life-threatening pathology may be influenced by additional testing undertaken during secondary care. The analysis of the answers to the questionnaire will be undertaken as soon as the completed questionnaires have been received in order to minimize interpretation bias.

Another limit is related to the transferability of the results to other European countries. French GPs do not use blood point of care testing to help their decision making. D-dimer test and Troponin test cannot be done at the GP's office but only in a laboratory.

Introduction

The General Practitioner's (GP's) challenge is to make the right decision when faced with dyspnoea and chest pain, which are two common presenting and unspecific symptoms in primary care. GPs have to reassure the patient when there is a musculoskeletal origin or a non-severe disease and refer the patient to the emergency department when it is a cardiac or pulmonary serious disease. The GPs are likely to face all these diagnoses with the same sorts of nonspecific symptoms 1 . Dyspnoea represents from 0.9% to 2.6 % of cases as a reason for consultation [2][3][4][5][6] in primary care, and it is the fourth reason for visits by elderly patients 7 . Chest pain represents 0.7% to 2.7 % of patient consultations in general practice 3,6,[8][9][10] . Non-life threatening diseases, such as chest wall syndrome, were diagnosed in 24,5% to 53 % cases of chest pain 9,11,12 , whereas life-threatening conditions, including cardiac or respiratory diseases and cancer, accounted for 20% of cases 12 .

Many diagnostic tools have been designed to guide GPs through the diagnostic reasoning process of chest pain and dypsnoea 13,14 . In fact, before using any prediction rule for a specific diagnosis, the GP should have some suspicion of one specific diagnosis. Decision making when faced with chest pain or dyspnoea involves medical knowledge, knowledge of the patient and his/her background, GPs' experience and gut feelings alongside the classic signs 15,16 . GPs who diagnosed pulmonary embolism (PE) explained how their suspicions were raised: the absence of indicative clinical signs for diagnoses other than PE, a sudden change in the condition of the patient, an earlier failure to diagnose PE and a gut feeling that something was seriously wrong 17 . This feeling that "a GP worries about a patient's health status, even though he/she has found no specific indications yet" has been named the sense of alarm 18 . The gut feeling acts as a compass when faced with uncertainty, and feature alongside the medical decision making and the problem solving as a third track in diagnostic reasoning 19 .

A Dutch Gut Feelings Questionnaire (GFQ) was created from definitions of gut feelings and validated by a construct validation procedure using case vignettes 20 . The internal consistency of the gut feelings questionnaire proved to be high (Cronbach's alpha = 0.91); the Kappa with quadratic weighting was moderate to good (0.62, 95% CI: 0.55-0.69). The Principal Component Analysis confirmed one factor, with the sense of reassurance and the sense of alarm items as two opposites, explaining 70.2% of total variance. A linguistic validation procedure was performed to obtain an English version of the questionnaire. The GFQ was then translated into French after a linguistic validation procedure 21 . Through a three-step study, the feasibility and practicability of the GFQ were evaluated in real practice.

7 cases of chest pain per GP and 4 cases of dyspnoea per GP. The participants received 220€ as an incentive after completing the 11 questionnaires in order to motivate the GPs to complete the estimated number of questionnaires. The trainees assisting the GPs were also informed of the study and encouraged to help the GPs in including cases and answering the patients' questions related to the study.

Personal emails were sent to the participating GPs to inform them of how many cases of dyspnoea and/or chest pain were already included and how many still remained to be included. GPs who did not include any patients got a personal phone call. One email every two months was sent to the trainees to inform them about the study, their role and our availability for any questions.

Test methods

The index test was the sense of alarm felt by the GPs and determined by the GFQ, consisting of 11 items. Items 2 to 7 in the questionnaire are derived from the consensus statements from the gut feelings concept which describes the sense of reassurance (item 2) and the sense of alarm (items 3-7). The items are rated using a 5-point Likert scale, ranging from completely disagree to completely agree. Items numbered, 8, 9 and 10 relate to the patient's care. The first and eleventh items assess whether the patient's case elicited a gut feeling (a sense of reassurance or a sense of alarm) or whether it is impossible for the respondent to say or whether a gut feeling is not applicable. A sense of alarm is considered as present when the answer to item 1 or 11 indicate a sense of alarm or that it is not applicable and at least one of the scores of items 3-6 is higher than 3/5. A sense of alarm is considered as not present when the answer to items 1 or 11 indicates a sense of reassurance, or when it indicates that it is not applicable and none of the scores of items 3-6 is higher than 3/5. The final diagnoses were not available to the readers of the GFQ at this step.

The reference standard was the final diagnosis collected four weeks later: by phone contact with the GP or by asking the GP to phone the patient if the GP had no information. Life threatening (18 conditions), non-life-threatening diseases (11 conditions) and pathologies where the severity depends on clinical features and context (22 conditions), were previously defined according to the pathologies or symptoms in the ICPC2 classification. This document was formulated, following a nominal group procedure, by a group of 7 GPs. Members of the current research team, blinded to the actual outcomes shown on the index questionnaire, judged each case in turn and did, by consensus, classify the expected outcomes in the third category (pathologies where the severity depends on clinical features and context) as either life-threatening or non-life-threatening diseases.

[IC95 = 0.37; 0.77]). The sensitivity of the sense of alarm was 0.61 [IC95 = 0.48; 0.73] and the specificity was 0.713 [IC95 = 0.679; 0.75].

Table 3 provides the focus on the sense of alarm when faced with dyspnoea and the sense of alarm when faced with chest pain. The two positive likelihood ratios were slightly different (nonsignificant p = 0.34): higher concerning the dyspnoea LR+ = 2.862 [IC95 = 1.56; 4.791] than for chest pain LR+ = 1.820 [IC95 = 1.163; 2.625]. Responding to the GFQ caused no adverse effects for either the participating GPs or the patients.

Discussion

Summary of findings

During this prospective study with French GPs filling in the Gut Feelings Questionnaire where a patient consulted for dyspnoea and/or chest pain and the GP experienced a sense of alarm, it was found that the post-test odds that this patient had, in fact, a life-threatening disease were about twice as high as the pre-test odds. This means that, according to the Bayes theorem, the patient had a 35 % probability of having a life-threatening disease when the initial estimated probability was 20%; not experiencing a sense of alarm decreased the probability of having a life-threatening pathology from 20 % to 12%.

Strengths and limitations of the study

This is the first study which estimates the accuracy of the GP's sense of alarm when faced with dyspnoea and thoracic pain. The prospective design and the use of a validated questionnaire to measure gut feelings in real primary care settings are two major assets of this study.

One of the main limitations was the low number of GPs participating to the study. Despite several personalized emails with positive motivational invitations, less than half of the 62 GPs who agreed to participate actually included patients. We concentrated on this crucial step of GP recruitment using personal contacts with physicians, targeting the friendship networks 24 .

We multiplied the ways of informing and presenting the study using different media. We sent personal emails to inform GPs how many questionnaires the participants had already sent, and several personal emails with positive and encouraging messages to stimulate those who had not included any patients. A financial incentive was presented as a compensation for the time spent on the questionnaires. Despite all our efforts, 25 GPs participated in the study, eventually filling in 235 analysable questionnaires. We estimated the results were powerful enough to share as a first study on this topic. The most frequent comments we heard to justify this non-inclusion was lack of time, and the fact that chest pain and dyspnoea were considered to be infrequent reasons for a consultation. GPs between 30 and 40 years old were overrepresented in the sample of working GPs: as younger GPs see more younger patients in consultations, the prevalence of both chest pain and dyspnoea may feature among the lowest rates of reasons for consultation. Research is not a common activity for French GPs. Even if they were involved in the department as trainees, including patients and filling in the GFQ represented additional work they were not used to. Also being observed in the complex management of situations represented a major barrier and could explain the lack of GPs' participation in research in general 25 . In our study, the difficulty in the management of chest pain and dyspnoea might have been this barrier. The younger generation of French junior lecturers have been developing research skills: since their role was created, publications in peer review journals have increased noticeably among French general practice teams 26 . It is, nevertheless, possible to generalise from these findings and apply the results to older, more experienced GPs as one fifth of the sample of working GPs were over 60 years old. Further research with larger sample of GPs should be conducted to enhance the power of our results.

A selection bias might have occurred in the recruitment of cases by the GPs. The technical impossibility of checking the consecutive cases was the first problem. We did however stress the importance of including consecutive cases during the initial presentation and in every personal contact during the study. The second problem was the possible selection bias during the inclusion step. During the presentation of the study, many GPs spontaneously focused on chest pain as the typical myocardial infarction symptom. Among the different consultations for these two reasons, the one particularly salient consultation may have been included instead of consultations seen as more common. We assess that this bias is not very large because we tended to minimize this bias by stressing the definition of dyspnoea and chest pain in each email or phone call we addressed to the GPs. We also trained the GPs to detect all cases and not to focus solely on the serious ones presented at the surgery. The GPs and trainees were shown a slide presentation describing the MeSH definition of chest pain and dyspnoea, including case vignettes. Inclusion and non-inclusion criteria figured on the back of each questionnaire. We did not collect any elements on the patients' clinical presentations apart from the GFQ, because we wanted to reduce the burden of the study as it is still considered time consuming by most French GPs.

Comparison with existing literature

Several studies were about the predictive value of gut feelings in the area of serious infection in children 27 ; sepsis in primary care 28 ; children with respiratory tract infection in general practice 29 ; use of gestalt with regard to pulmonary embolism 30 and the role of intuition in the suspicion of cancer 31 . It is worth noting that all these studies used a binary question "do you have gut feelings?" [27][28][29] without using a proper definition of the concept. How did the GP respondents interpret the sense of "gut feelings" or "intuition" in these situations? Several meanings may overlap between these close but different concepts. The GFQ was created following the criteria of the sense of alarm and the sense of reassurance as they were defined by the GPs themselves.

The topics of chest pain and dyspnoea in primary care were selected because of their frequency and because of the challenge of the diagnosis behind these two symptoms. The clinical presentation of chest pain is not very helpful 33 and an evaluation based on symptoms and signs alone may not be accurate in diagnosing or excluding coronary heart disease 34 . Our results were similar to a Dutch study where 18 % of patients consulting for chest pain were suspected of having a life-threatening pathology 6 . The most requested point of care tests were the d dimer test and troponin because of the difficulty in making the correct diagnosis when faced with dyspnoea and chest pain 35 . The management of chest pain is specific to the general practice surgery: GPs use medical decision making strategies which are different from those in textbooks or in emergency departments 15,16,36 . For instance, whether the patient asked for a home visit 4 , whether he/she assumed the pain came from the heart, or whether his/her condition changed, when compared with his/her previous condition, have been proven to be effective tests in general practice 34 . As the pre-test probability of the diseases is different in the context of general practice, GPs have to follow clinical signs that may not figure in the historical description of diseases. Van den Bruel et al described how the three classic symptoms of meningococcal disease in children and adolescents: rash, meningism, and impaired consciousness occur very late in the disease history. Leg pain, cold hands and feet, and abnormal skin colour appear earlier and can be present at a first consultation with a GP 37 .

Many situations encountered by the GPs are not described in books or studies in the way in which they are observed at the time of the consultation. The GPs have to adapt to the person rather than to the diseases 38 .The positive likelihood ratio of more than 2 which we found here in this study should not be seen merely as a figure. Its meaning is broader in diagnostic decision making in general practice: faced with early symptoms and signs, the sense of alarm may be a reaction adapted to the low-level signs of disease seen at a very early stage. One way to avoid errors and bias is for the GP to follow his/her sense of alarm. The sense of alarm is a conscious feeling corresponding to the perception of cognitive dissonance: the mismatch between situation and pre-established patterns trigger the sense of alarm in the GP 39 . The perception of alarm compels the physician to quit his routine-based reasoning and switch to analytical reasoning by generating and considering different hypotheses 40,41 . The sense of alarm acts as a feedback mechanism, allowing the questioning of a possibly wrong direction of reasoning at a very early stage of the diagnostic process.

In a context where there is a low incidence of serious diseases 42 , the false positives noted after following the sense of alarm were seen as quite inevitable 17 . Providing care for patients suspected of suffering from a life-threatening disease, is possibly considered much more important than withholding care from a patient in good health.

Conclusion

Where a physician experiences a sense of alarm when a patient consults for thoracic pain and/or dyspnoea, the test-odds for diagnosing a life-threatening pathology [IC95 = 1.54-2.94] are doubled. The sense of alarm may be a reaction adapted to low-level signs of disease seen at a very early stage and can be seen as an efficient element for overcoming the difficulty inherent in the uncertainty of general practice.

Health threatening pathology

General discussion

What was already known

The two kinds of gut feelings experienced by general practitioners (GPs): the sense of alarm and the sense of reassurance, were initially described following qualitative studies in the Netherlands and in Flanders. They concern the prognosis of the patient's state of health during the consultation 1 .

The sense of alarm means that a GP perceives an uneasy feeling as he/she is concerned about a possible adverse outcome. It implies that a GP worries about a patient's health status, even though he/she has found no specific indications yet; it is a sense of 'there's something wrong here'. A 'sense of alarm' means that, if possible, the GP needs to initiate specific management to prevent serious health problems 2 .

The sense of reassurance means that a GP feels secure about the further management and course of a patient's problem, even though he/she may not be certain about the diagnosis:

everything fits in 2 .

The sense of reassurance and the sense of alarm constitute a dynamic element in a GP's diagnostic process.

Gut feelings have been described as a third track, alongside medical decision-making and medical problem-solving, which play their part in the continual interaction of non-analytical and analytical diagnostic reasoning processes 3 Even where an idiomatic expression did not exist in a language, these gut feelings were experienced by GPs in every country in the study 4 .

A Dutch language Gut Feelings Questionnaire (GFQ) was created from the definition criteria and validated by a construct validation procedure using case vignettes 5 . The sense of alarm and the sense of reassurance were found as two opposites of one component, explaining 70.2% of the variance after a principal component analysis. The internal consistency of the GFQ proved to be high (Cronbach's alpha = 0.91). The kappa with quadratic weighting was substantial (0.62, 95% CI: 0.55-0.69) 5 . A linguistic validation procedure was performed to obtain an English version of the questionnaire 5 .

In 2008, in Budapest, an international network of GPs, interested in the gut feelings topic, was established and called the European Expert Group on Cognitive and Interactive Processes in Diagnosis and Management in General Practice, or COGITA for short. The aim of the group is followed allowed us to expand the concept to include Slavic languages such as Polish. We assumed that the utility of the GFQ would also be transferable, working within this transcultural context and applying standardised linguistic procedures.

Then we aimed at testing the GFQ in real GP practice settings during office hours to check whether any changes were needed to improve feasibility, and to calculate the prevalence of the sense of alarm and of the sense of reassurance of GPs in three different countries. The

Second part

We began this second part by exploring how GPs came to suspect pulmonary embolism in real practice settings. Another objective was to describe more specifically the role of gut feelings in this diagnostic process. Before using any prediction rule oriented towards this particular diagnosis, a GP should have some suspicion of PE and it is precisely this initial stage which is unclear. The diagnostic process leading to the suspicion of PE is not well described in primary care. Therefore, a thematic analysis of 28 semi-structured qualitative interviews was performed by three researchers using a Grounded Theory coding paradigm. The suspicion of pulmonary embolism arose out of four considerations: the absence of indicative clinical signs for diagnoses other than PE, a sudden change in the condition of the patient, a gut feeling that something was wrong and a failure to diagnose PE in the past. The GPs interviewed did not use any prediction rule or score in their diagnostic process. The perception of alarm compelled the physician to quit his routine-based reasoning and switch to analytical reasoning by generating and considering the PE hypothesis 8,9 . When they had a gut feeling that there was something wrong, that it could be a PE, they then decided to refer the patient to the hospital specialist. The sense of alarm acted as a feedback mechanism at a very early stage in the diagnostic process, allowing the questioning of a possibly wrong diagnostic reasoning direction.

The objective of the next study was to calculate the diagnostic test accuracy of a GP's sense of alarm when confronted with dyspnoea and chest pain. A prospective study with 25 French GPs who filled in the GFQ after each consecutive consultation for chest pain and dyspnoea, collected 235 questionnaires with a sense of alarm or a sense of reassurance. Where the physician experiences a sense of alarm when a patient consults a GP for chest pain and/or dyspnoea, the implication is that the patient has a 35% probability of having a life-threatening disease whereas the initial estimated probability was 20% (LR+ = 2.12 [IC95 = 1.49; 2.82]).

The negative Likelihood ratio was (LR-= 0.55 [IC95 = 0.37; 0.77]). Where the physician does not experience a sense of alarm, the probability of having a life-threatening pathology decreases from 20 % to 12%. The sensitivity of the sense of alarm was 0.608 [IC95 = 0.475; 0.725] and the specificity was 0.713 [IC95 = 0.679; 0.75]. In vague and uncertain situations of chest pain, and/or dyspnoea, where it is difficult for a GP to make the right management decision, a gut feeling proved to be a helpful tool in their reasoning process.

Strengths and weaknesses

This thesis focuses on gut feelings in general practice: it is an original topic which has not been studied in France before. The international collaboration was a real strength of this work: three studies were carried out by Belgian, Dutch, French, German, Polish and Swiss researchers.

The fact that these studies were performed in real settings, mostly in daily general practices, was a strong feature. The cultural checks of the French, German and Polish versions of the GFQ were also done by GPs. Seventy-two GPs from France, the Netherlands and Belgium were involved in the evaluation of the questionnaire in real practice settings during the feasibility study. The participation of the COGITA group members was vital for the conception of the glossary and for the validation of the modified version of the GFQ during the feasibility studies. Twenty-five French GPs agreed prospectively to fill in 11 GFQs when faced with patients consulting for chest pain and /or dyspnoea. Our aim was to go beyond theoretical considerations by involving physicians working in daily general practice and considering their pragmatic points of view. We used several methods: a linguistic validation procedure, a mixed methods approach, a qualitative study with semi-structured interviews, and a quantitative prospective study.

The gut feelings concept proved to be transcultural according to several arguments. During the feasibility studies in Belgium, France and the Netherlands, the same misunderstandings and difficulties in filling in the questionnaire occurred. In spite of the differences between health care systems, the Dutch and French versions of the GFQ do examine the same phenomena.

Testing the use of a questionnaire such as the GFQ in two different settings (think-aloud in an experimental environment first; and then during office hours, in three different health care systems) enabled us to adapt the questionnaire in response to the participating Belgian, Dutch and French GPs' opinions and pragmatic concerns. The questionnaire was modified after the two-step study. Now we have a questionnaire formatted by GPs, for the use of GPs, working in three different European countries. These cross-border studies reaffirmed the transculturality of gut feelings 6,10 . In addition, a similar prevalence of both the sense of alarm and the sense of reassurance was found in all three countries. In addition, a Spanish qualitative study, using focus groups composed of GPs, described the existence of gut feelings in their own diagnostic processes 11 . They also considered the gut feelings concept an important tool in primary care. The sense of reassurance was treated cautiously by French and Spanish GPs:

they stressed the importance of remaining alert to the outcome of the case 6,11 . In the same way, the sense of alarm was considered more as "a trigger for the diagnostic process than as a need for management" 6,11 . Although the participating GPs live in different cultural European areas, work in different health care systems and use different languages, they all proved to be familiar with the gut feelings concept and to use it in a comparable manner. The transculturality of the gut feelings concept between proto-Germanic, Romance and Slavic languages has been revealed in several studies 6,11 . Gut feelings do exist in Dutch culture, as an idiom "pluis/nietpluis", and were first mentioned as a diagnostic reasoning tool more than a hundred years ago.

It was not present in the literature in France or in Germany. The gut feelings concept can be considered as an indicator of the transculturality of medical decision making as it "passes through classical cultural boundaries" 12 . For Welsch, the old cultures once considered different are now gathered in one interconnected culture. He took the example of the French and German nations, which now share European culture and are no longer purely French or German, homogenized and separatist cultures.

The accuracy of the sense of alarm, when GPs are faced with dyspnoea or chest pain, has never been studied using a validated questionnaire to measure gut feelings in real primary care settings. Knowing the test properties of the sense of alarm, such as the sensitivity and specificity and the positive and negative likelihood ratios concerning the possibility of a serious disease, contributes to the validity of this diagnostic phenomenon. When a GP is experiencing a sense of alarm in this situation, the post-test odds that this patient does, in fact, have a lifethreatening disease is about twice as high as the pre-test odds. Over-referral of patients was also important but this fact was considered quite normal by GPs. 13,14 . In the case of dyspnoea and thoracic pain, missing a life-threatening pathology (e.g. a coronary event) has much more important consequences than in cases where the prognosis is benign 15 . In vague and unclear situations of chest pain and dyspnoea, a GP weighs up several arguments in his/her decisionmaking process. Should he/she refer or not? According to the threshold concept 15,16 , a GP can intuitively define an action threshold which corresponds to a defined probability of disease, where both choices, to act or not to act, are equally justified. According to the findings of our research, if we were to put the action threshold to refer at a 30 % probability for a lifethreatening disease, the presence or absence of a sense of alarm could be one of the arguments that brings us above the threshold to refer, or leaves us below the threshold, and so allows the GP to choose a strategy of watchful waiting.

The major weakness of the thesis was the lack of participation of the GPs in the last prospective study. Despite several personalized emails with positive motivational invitations, fewer than a half of the 62 GPs who agreed to participate actually included patients and contributed to this study. We worked hard to improve this crucial step of GP recruitment by making several personal contacts targeting the friendship networks 17 . We multiplied the ways of informing and presenting the study using different media. We sent personal emails to inform GPs how many questionnaires the GP participants had sent and several personal emails with positive and encouraging messages to stimulate those who had not included any patients. The incentive part was presented as a compensation for the time spent on the questionnaires. Despite all our efforts, only 25 GPs participated in the study. The most frequent comments we heard to justify this non-inclusion was lack of time and lack of chest pain and dyspnoea as reasons for consultation. However, research is not a common activity for French GPs. Including patients and filling in questionnaires represented additional work they were not used to. For Hummers-Pradier et al. another element could explain the lack of GP participation in research in general: being observed in the often complex management of health care situations is a major barrier 18 .

In our study, the uncertainty of how to manage chest pain and dyspnoea might have been this barrier. A selection bias might have occurred in the recruitment of the cases by the GPs. The technical impossibility of checking whether the cases were indeed consecutively included was the first problem. We did stress the importance of including consecutive cases during initial presentation and in every personal contact during the study. The second problem was the possible selection bias during the inclusion step. During the presentation of the study design, many GPs spontaneously focused on myocardial infarction as a common explanation for chest pain. The GPs might then have selected the most salient consultation for dyspnoea/ chest pain. We tended to minimize this bias by stressing the definition of the dyspnoea and the chest pain in each email or phone call addressed to the GPs. It was the first diagnostic study with the GFQ in real practice situations at the GPs' surgery after the feasibility of the GFQ had been tested and confirmed. Training the French GPs to participate in research and to include patients during consultations are two of the several challenges for the forthcoming years in French academic departments of primary care.

Relevance for the clinical perspective

As we learned in the interview study, the sense of alarm experienced by the GPs interviewed, led them to diagnose pulmonary embolism 19 and to decide that urgent care was warranted in for these cases of dyspnoea and thoracic pain 20 . The sense of alarm can be seen as a complementary tool for learning how to prevent error in primary care. Interviewed about their diagnostic errors or near misses, GPs shared the gut feelings they followed which had prevented error or that, with hindsight, they should have listened to in order to avoid error 21 .

These feelings of alarm or unease were analyzed by Goyder retrospectively and considered to be due to salient features in the physician's understanding of the situation 21 .

The first step of gathering information during the patient-physician consultation is crucial for the next phases of correct clinical decision making or possibly erring. All the hypotheses and care propositions are based on the information gathered in the first minutes of the consultation.

This crucial step belongs to System 1 which acts permanently but unconsciously 22 . System 1, also called the non-analytical reasoning system, works fast and automatically with several actions occurring at the same time. System 2, the analytical reasoning system, operates consciously, slowly, laboriously and sequentially 22 . A schema of a pathology or a situation will be generated by System 1 after the first minutes of the consultation. A schema is "a high-level conceptual structure or framework that organizes prior experience and helps us to interpret new situations" [23][24][25] . In medical decision making, the recognition of a schema or a pattern match is often presented as the initiation phase of the diagnosis 26 . If the physician relies on his/her first impression leading to a schema of a well-known disease, he/she might make a fast and efficient decision. However, he/she might as easily be wrong and fall victim to several errors and the biases described below 27 . A premature closure occurs when the decision maker stops his/her reasoning process too early and neglects other alternative working hypotheses.

The availability heuristic operates to judge the likelihood of a phenomenon by how easily examples come to mind. The framing effect appears when decisions are made differently depending on how information is presented. The anchoring effect means that the decision maker sticks with initial impressions. These errors are described as psychological biases for every kind of decisions, not only medical decisions 28 . Instead of being purely rational, decision makers are looking for the easy option, rather than looking for the best solution from a theoretical point of view: they refer to their personal preferences, available alternatives and opportunities 29 .

One way for a GP to avoid these errors and biases in his diagnostic reasoning is to act upon the sense of alarm, if present. The sense of alarm is a conscious feeling corresponding to the perception of cognitive dissonance: the mismatch situation with pre-established patterns triggered the sense of alarm in the GP 30 . The perception of alarm compelled the physician to quit his routine-based reasoning and switch to analytical reasoning by generating and considering different working hypotheses 8,9 . The sense of alarm acted as a feedback mechanism, allowing the questioning of a possibly wrong decision at a very early stage of the diagnostic process.

Although a sense of alarm can also appear during the whole consultation and even at the end of it, this early stage is a strong feature of the sense of alarm: it might intervene even before using any prediction rule oriented towards a particular diagnosis, e.g. the suspicion of pulmonary embolism 19 . Donner Banzhoff et al. wrote about the inductive way of gathering information used by GPs 31 . While the patient is describing his/her situation, the GP evokes some hypotheses automatically. The use of open questions helps the inductive strategy to be developed during the first part of the medical interview. The second part consists of asking questions to test the diagnostic hypothesis with a deductive strategy, the hypotheticodeductive strategy. The context of a general practice is different from a hospital environment.

The prevalence of serious disease in general practice is low [32][33][34] and the GPs have a gatekeeper role. The probability of having a patient in good health in general practice is high.

Therefore, the presence of abnormal elements in a patient's presentation is more likely to catch the GP's attention than the confirmation of a normal sign. During the inductive phase, a different, unusual and dissonant element in a patient's presentation will alert the GP more than one where everything fits in. "The presence of an abnormality is more informative than confirmation of its absence" 31 . That is why a pattern failure approach seems to be more effective in primary care when using an inductive method rather than a deductive method and pattern recognition.

The gut feelings may also appear later in time, e.g., suddenly when the patient has left or after the consultation when thinking back to the surgery hours. Several GPs experienced this feeling of having a sense of alarm in a later phase of the diagnostic reasoning process 1 .

Alongside this type of reasoning, the relationship between the GP and the patient may also be taken into consideration. In most cases, GPs know their patients from previous meetings and consultations. A sudden or small change in the patient's presentation may trigger a GP's sense of alarm 19 . If this sense of alarm puzzles the GP, he/she will pursue his/her examining process by asking more questions and looking more closely, thereby avoiding premature closure. This "inductive foraging" way of listening to the patient, rather than the deductive way, allows the physician to become aware of what is disturbing, or unusual, in the presentation 31,35 and can be present at a first consultation with a GP 37 . One prospective study highlighted the high specificity and high positive likelihood ratio of gut feelings in terms of a sense of alarm in the presence of serious infectious illness, such as recognition of meningococcal disease in children and adolescents in primary care 38 . Convulsions and parents' concern that the illness was different from usual, were the strongest contextual factors associated with the GPs' gut feelings 38 (odds ratio 80.5, 95% confidence interval 6.2 to 1051 and 26.93 (9.02 to 80.41) respectively). So many medical problem situations encountered by the GPs are not fully described in books or studies. Schön wrote "in real-world practice, problems do not present themselves to the practitioner as givens. They must be constructed from the materials of problematic situations which are puzzling, troubling, and uncertain" 39 . He called "reflection-inaction" the fact that "practitioners themselves often reveal a capacity for reflection on their intuitive awareness in the midst of action and sometimes use this capacity to cope with the unique, uncertain, and conflicted situations of practice" 39 . The practitioner's diagnostic skill cannot be reduced to technical rationality: by integrating the messy and indeterminate component in each situation, the reflective practitioner is more efficient "in the art of solving problems" 40 . He uses his "tacit knowledge", a concept to name his personal knowledge built on experiences which cannot immediately be verbalized 41,42 . This capacity of GPs to adapt their reasoning process to the one particular patient is also mentioned by Montgomery 43 . GPs have to adapt more to the person in front of them than to the diseases they recognize: the strategy from the particular to the general rather than from the general to the particular was recorded when analysing GPs' daily decision-making [START_REF] André | GPs' decision-making--perceiving the patient as a person or a disease[END_REF] . "What is it best to do, for this individual, at this time, given these particular circumstances?" is the question GPs formulate for each meeting with their patients [START_REF] Greenhalgh | Is it time to drop the 'knowledge translation' metaphor? A critical literature review[END_REF] . Greenhalgh called it the "knowledge translation", the ability to apply general rules to particular situations [START_REF] Greenhalgh | Is it time to drop the 'knowledge translation' metaphor? A critical literature review[END_REF] . The sense of alarm helps the GPs to stay focused on the patient's situation instead of looking for the pattern of a disease.

Practising mindfulness is another way of expressing the GP's ability to take into account his/her own feelings, and the situation of the patient, during the consultation. For Epstein, practising mindfulness obliges GPs to consider diagnostic reasoning as a whole; integrating emotions, thoughts, actions and interpretations felt by the practitioner during the consultation [START_REF] Epstein | Mindful Practice[END_REF] . For him "mindfulness is a logical extension of the concept of reflective practice" [START_REF] Epstein | Mindful Practice[END_REF] . The concept of mindfulness is also helpful in integrating uncertainty as a productive component of clinical reasoning and not considering it as a regrettable and unavoidable aspect [START_REF] Engebretsen | Uncertainty and objectivity in clinical decision making: a clinical case in emergency medicine[END_REF] . The mindfulness practice drives the GP to "understand the patient's world to a sufficient degree to experience and express empathy" [START_REF] Epstein | Could this be something serious? Reassurance, uncertainty, and empathy in response to patients' expressions of worry[END_REF] . The alarm he/she feels can be seen as an efficient tool for overcoming the difficulty inherent in the uncertainty of general practice.

The different steps in this thesis contributed to the acquisition of a better overview of the whole process of the sense of alarm in the decision-making process. Causing the GP to quit the routine and stay more open to what the patient says is one effect of the sense of alarm. It acts as a trigger to become more mindful in the diagnostic reasoning process, not as a final outcome of that process but an as an alert to become more attentive to the prevention of a possible serious illness outcome. Considering the sense of alarm as a complementary tool for learning how to prevent error in primary care is an approach where error prevention is associated with decision making at a very early stage.

Relevance for medical education

The sense of alarm is a diagnostic tool to enhance the mindfulness of the GPs by forcing them to quit their routine-based reasoning. During medical studies, novices learn through a hypothetico-deductive route how to associate the symptoms and the signs presented by a patient within a disease script and with the appropriate treatment [START_REF] Elstein | Medical Problem Solving an Analysis of Clinical Reasoning[END_REF] . When they are experts, GPs use this hypothetico-deductive model in only 39% of the consultations, alongside other strategies, as inductive foraging and triggered routines 26,35 . Thanks to experience, their knowledge network has become much richer by the acquisition of additional knowledge from practice and additional links between the knowledge network nodes, so that they quickly recognise common and uncommon patterns. In an experimental context, instructing medical students explicitly to use both their analytical and non-analytical reasoning competencies increases their diagnostic accuracy [START_REF] Ark | The benefits of flexibility: the pedagogical value of instructions to adopt multifaceted diagnostic reasoning strategies[END_REF] . In the pragmatic context of practising in GPs' surgeries, trainees and supervisors evoke gut feelings, and other non-analytical reasoning, during tutorial dialogues [START_REF] Stolper | How do gut feelings feature in tutorial dialogues on diagnostic reasoning in GP traineeship?[END_REF] . The trainees experience a sense of alarm and its consequences in terms of referring a patient urgently. Inexperienced GPs feel the sense of alarm in 11.4% of the consultations whereas the experienced GPs feel it in only 7.4% [START_REF] Stolper | Gut feelings in general practice[END_REF] . Through their lack of expertise, trainees might have GF more frequently because they cannot analyse and explain the medical problem so well as experienced GPs . Rather than eliminating uncertainty from the diagnosis, GP trainees have to quit the myth of certainty and accept the inherent uncertainty of the discipline [START_REF] Griffiths | The nature of medical evidence and its inherent uncertainty for the clinical consultation: qualitative study[END_REF] . Becoming aware of the uncertainty may lead to new questions, to more and better knowledge of the patient's situation, to other working hypotheses, to a better diagnostic reasoning process [START_REF] Engebretsen | Uncertainty and objectivity in clinical decision making: a clinical case in emergency medicine[END_REF] . The example of diagnosing a pulmonary embolism is illustrative. The pathology was clinically suspected in less than half of all fatal cases [START_REF] Pineda | Clinical suspicion of fatal pulmonary embolism[END_REF] . If a GP student is taught about the clinical signs and symptoms of pulmonary embolism, as described in the hospital, he might have difficulty in his daily practice because only a few patients will present with all these relevant signs. After our qualitative study with GPs, we learned that listening to the sense of alarm, a sudden change in the presentation of a patient and the absence of indicative clinical signs for diagnoses other than pulmonary embolism were relevant strategies for diagnosing a pulmonary embolism which can actually be taught to GP students.

Teaching novice GPs how to integrate their feelings during the consultation as a chance to stay open to the patient, in a reflexive and mindful way of thinking, is also a way to enhance the therapeutic alliance with the patient. Considering the patient as an expert of his own disease, and the GP as an expert of diseases in general, helps us to understand how doctor patient relationships should be based on mutual collaboration. The shared decision making model is based on this mutual collaboration [START_REF] Charles | Shared decision-making in the medical encounter: What does it mean?[END_REF] .

Relevance for health care organization and policy

In the Netherlands, GPs have sometimes been blamed during medical disciplinary tribunals for failing to have experienced or to have followed this sense of alarm [START_REF] Stolper | How do disciplinary tribunals evaluate the « gut feelings » of doctors? An analysis of Dutch tribunal decisions, 2000-2008[END_REF] . Because it should have reactivated the GPs' diagnostic process, the sense of alarm was considered a contributing factor in providing optimal care. The Dutch trials used the sense of alarm as a tool for preventing error. Dealing with error strategies focuses on individual factors and rarely takes into account the organization of the health care in which the error occurred [START_REF] Reason | Human error: models and management[END_REF] .

Consideration and prevention of risk and error are not yet part of the French culture of primary care 18 . Quality improvement programs alone cannot guarantee quality of health care. Also a strategy of defining medical error, error reduction and risk prevention should be included [START_REF] Amalberti | Five system barriers to achieving ultrasafe health care[END_REF] .

Promoting gut feelings as an error reduction tool in GP diagnostic reasoning can help the discipline to integrate risk prevention and improve quality of care [START_REF] Monteiro | The 3 faces of clinical reasoning: Epistemological explorations of disparate error reduction strategies[END_REF] .

Further research

Many other research projects could be further developed. Quantitative studies on the diagnostic accuracy of the sense of alarm, in areas other than chest pain and dyspnoea, are needed. Suspecting cancer in primary care is another domain where gut feelings play a substantial role [START_REF] Johansen | How does the thought of cancer arise in a general practice consultation? Interviews with GPs[END_REF][START_REF] Donker | Determinants of general practitioner's cancer-related gut feelings-a prospective cohort study[END_REF] . It is particularly relevant to compare the outcomes of the final diagnosis, after months of follow-up, in a prospective way in relation to the initial sense of alarm measured by the GFQ.

Exploring gut feelings as experienced by patients is an interesting topic for future studies [START_REF] Buetow | When should patient intuition be taken seriously?[END_REF] . Is there any difference between the kinds of gut feeling felt by patients and their relatives and those felt by their physicians?

Several aspects of the effect of integrating the teaching of the gut feelings concept during medical school undergraduate and graduate education could be evaluated. The Physician Reaction to Uncertainty (PRU) is a questionnaire which evaluates 4 dimensions: anxiety due to uncertainty, concern about bad outcomes, reluctance to disclose uncertainty to patients and reluctance to disclose mistakes to physicians [START_REF] Gerrity | Physicians' reactions to uncertainty in patient care. A new measure and new insights[END_REF][START_REF] Gerrity | Physicians' Reactions to Uncertainty: refining the constructs and scales[END_REF][START_REF] Schneider | Validation and cultural adaptation of a German version of the Physicians' Reactions to Uncertainty scales[END_REF] . Scores on the anxiety scale, due to the uncertainty of students after following this course, and practising in real contexts, could be compared to the scores of students who had not yet received that tuition.

Continuing medical education sessions which integrate the gut feeling concept into decision making and risk prevention could also be organized for experienced GPs. To reflect on thinking processes and to critically examine assumptions and beliefs could be beneficial in improving the quality of care [START_REF] Berner | Overconfidence as a cause of diagnostic error in medicine[END_REF][START_REF] Eva | The difficulty with experience: does practice increase susceptibility to premature closure?[END_REF] .

Conclusion

The gut feelings criteria have been formulated from the GPs' descriptions of their own practices. The 'sense of reassurance' and the 'sense of alarm' constitute a dynamic element in a GP's diagnostic process, commuting between non-analytical and analytical diagnostic reasoning processes. The Gut Feelings Questionnaire was translated into the French, German and Polish languages following a standardized procedure of linguistic validation. The GFQ was then tested in real practice settings during office hours, to check its feasibility, in Belgium,

France and The Netherlands. The sense of alarm was identified as one major factor leading to the positive diagnosis of pulmonary embolism, after analyzing qualitative interviews with GPs. The GFQ was used in a prospective study with GPs to calculate the accuracy of the sense of alarm when faced with chest pain or dyspnoea at the GP's surgery. Having a sense of alarm in this situation drove the GPs to the diagnosis of a serious disease twice as often as those who did not have this sense of alarm. The sense of alarm can be seen as a complementary tool for preventing error in primary care at a very early stage in the decisionmaking process in general practice. Further research concerning the suspicion of cancer and the teaching of gut feelings should be promoted.

of the GFQ would also be transferable, working within this transcultural context and applying standardised linguistic procedures. The translated and validated GFQ makes it possible to

Compare results of research into the predictive value of gut feelings and into the significance of the main determinants in five European countries.

Then we aimed at testing the GFQ in real GP practice settings during office hours to check

Second part

First it was explored how GPs came to suspect pulmonary embolism in real practice settings.

Another objective was to describe more specifically the role of gut feelings in this diagnostic process. Before using any prediction rule oriented towards this particular diagnosis, a GP should have some suspicion of PE and it is precisely this initial stage which is unclear. The diagnostic process leading to the suspicion of PE is not well described in primary care.

Therefore a thematic analysis of 28 semi-structured qualitative interviews was performed, by three researchers, using a Grounded Theory coding paradigm. The suspicion of pulmonary situations of chest pain and or dyspnoea where it is difficult for a GP to make the right management decision a gut feeling proved to be a helpful tool in their reasoning process.

The general discussion addresses the strength and weaknesses of our studies and offers some comments on error prevention in primary care. It then discusses some educational notions.

The gut feelings criteria have been formulated from the GPs' descriptions of their own practices. The 'sense of reassurance' and the 'sense of alarm' constitute a dynamic element in a GP's diagnostic process, helping to commute between non-analytical and analytical diagnostic reasoning. The Gut Feelings Questionnaire was translated into French, German and Polish languages following a standardized procedure of linguistic validation. The GFQ was then tested in real practice settings during office hours to check its feasibility in Belgium, France and The Netherlands. The sense of alarm was identified as one major factor conducting to the

Samenvatting

De klinische besluitvorming van artsen is gebaseerd op de interactie van analytisch en nietanalytisch redeneren. Het pluis/niet-pluis (PNP) gevoel kan als een deel van het niet-analytisch redeneerproces worden beschouwd en omvat twee componenten, nl. het pluis gevoel en het niet-pluis gevoel. Via kwalitatief onderzoek in Nederland en Vlaanderen werden deze begrippen duidelijk gedefiniëerd. Het PNP gevoel kan dienstig zijn als prognostisch instrument tijdens de raadpleging om de uitkomst voor de gezondheidstoestand van de patiënt te voorspellen.

Het niet-pluis gevoel betekent dat een huisarts een ongemakkelijk gevoel ervaart omdat hij/zij zich zorgen maakt over een mogelijke ongunstige uitkomst. Het impliceert dat een huisarts zich over de gezondheidstoestand van een patiënt zorgen maakt, alhoewel hij/zij hiervoor geen specifieke aanwijzingen heeft; het is een gevoel van 'er is iets mis hier'. Een 'niet-pluis' gevoel betekent dat de huisarts, indien mogelijk, een specifieke behandelingsplan moet starten om te voorkomen dat zich ernstige gezondheidsproblemen voordoen.

Het pluis gevoel betekent dat een huisarts zich veilig voelt over de verdere aanpak en het natuurlijk verloop van het probleem van de patiënt, ofschoon hij/zij nog niet zeker is over de Deze doctoraatsthesis omvat twee delen.

Het eerste deel richt zich op het vertalen van het concept van het PNP-gevoel naar verschillende Europese talen en contexten:

• Hoe verliep het proces voor het vertalen van PNP-gevoel in het Frans? Een Delphiprocedure, met deskundigen van de Franse huisartsen, werd gebruikt om het PNP gevoel te vertalen in het Frans.

• Hoe werd de PNP-vragenlijst vertaald uit het Engels naar het Frans, Duits en Pools?

Een linguïstische validatieproces, volgens een bepaalde procedure, werd uitgevoerd in Frankrijk, Duitsland en Polen.

• Hoe was de bruikbaarheid van de vragenlijst binnen de dagelijkse praktijk? Een mixedmethods studie werd uitgevoerd in België, Nederland en Frankrijk om aldus een definitieve versie van de vragenlijst te bekomen

• Hoe kunnen we de meest voorkomende termen qua diagnostisch redeneren zoals gebruikt in de publicaties van de COGITA-groep definiëren? Een woordenlijst van termen werd opgebouwd na een literatuuroverzicht.

• De verschillen die zich hebben voorgedaan in de definities en vertalingen van het PNPgevoel en de daaraan verbonden begrippen hebben geleid tot een reactie op het artikel "Erkenning van sepsis in eerstelijnszorg: een enquête onder huisartsen" geschreven door Loots et al., en hun gebruik van de formulering "gut feelings" zonder te beschrijven naar welk concept ze verwijzen.

Het tweede deel richt zich op de diagnose van een levensbedreigende ziekte en de diagnostische waarde van het NP-gevoel in de huisartsenpraktijk bij de symptomen van kortademigheid en thoracale pijn.

• Hoe komen huisartsen tot het vermoeden van een longembolie (LE) in de dagdagelijke praktijk? Een kwalitatieve studie met semi-gestructureerde interviews onderzocht dit punt. Vervolgens werd meer in het bijzonder de rol van PNP-gevoelens in dit diagnostisch proces beschreven.

• De resultaten van deze kwalitatieve studie leidden tot een reactie op het Lucassenartikel over de vergelijking van de diagnostische eigenschappen van "gestalt" en de Wells-regel, om een LE in eerstelijnszorg uit te sluiten.

• Wat is de diagnostische test-nauwkeurigheid van het NP-gevoel wanneer toegepast op kortademigheid en pijn op de borst? Een prospectieve studie met behulp van de PNP-vragenlijst werd uitgevoerd met Franse huisartsen. Het protocol voor deze studie werd gepubliceerd en de eerste resultaten van de prospectieve studie worden gerapporteerd.

In deze thesis werden verschillende onderzoeksmethodes en designs gebruikt: een linguistische validatieprocedure, een mixed methods approach met kwantitatieve en kwalitatieve gegevens, een kwalitatief exploratief onderzoek met semi-gestructureerde interviews, een prospectieve kwantitatieve studie met een gevalideerde vragenlijst.

Eerste deel

We begonnen met de vertaling van de PNP vragenlijsten in het Frans, Duits en Pools met

  ways: first, by performing an intermediate step with two additional leading versions; second, by subsequently communicating by multiple e-mail exchanges and/or short physical meetings (instead of holding a telephone meeting), and third, by continuously involving the original Dutch authors (in particular ES).

  , DH, SC, AS, PVR, ES conceived the study, and participated in its design and coordination and helped to draft the manuscript. MB conducted the French part of the study, drafted the article and submitted it for publication. JH and DH conducted the German part of the study and drafted the German part of the article. SC and AS conducted the Polish part of the study and drafted the Polish part of the article. VRP and ES designed the study and reviewed the article. All authors read and approved the final manuscript.

Figure 1 .

 1 Figure 1. The procedural scheme followed for the English-French translation of the gut feelings questionnaire

Figure 1 .Figure 2 .Figure 3 .Figure 4 .Box 1 .

 12341 Figure 1. Original version of the GFQ

Figures

  Figures

  first step, a think-aloud study and a feasibility study, led to small modifications concerning the order of items, and to some small adaptations of the wording of two items. The modified version of the GFQ was created without altering the sense of the seven validated items. The second step, a repetition of the feasibility study but with the modified questionnaire, led to minor changes. The prevalence of gut feelings in the two phases of the feasibility study were similar in Belgium, France and the Netherlands. The questionnaire was modified after the two phases of the study. Now we have a questionnaire formatted by GPs and for the use of GPs working in three European countries. The final version of the GFQ proved to be a feasible and practical tool to be used for prospective observational studies in daily practice.The next study concerned the construction of a glossary of diagnostic reasoning terms relating to the research into gut feelings, by the COGITA group. The researchers reviewed relevant literature, aiming to define salient terms used in their publications. They described the terms, cross-reviewed the wording and reached consensus within the group. Two sections were created: (1) a diagnostic reasoning section that describes concepts such as analytical and non-analytical reasoning processes, clinical mind-lines, and intuition, and (2) a research methods section describing concepts such as linguistic validity and saturation. Defining terms in the area of decision making developed by a European expert consensus group, based on a literature review and gathered in an open access glossary, is a unique initiative. This glossary is a prerequisite for conducting further research, with the intention of acquiring knowledge which, in turn, can be transmitted through teaching, as well as creating a basis for cross-border research into general and family medicine. We made the glossary freely available in order to share our results with other researchers and to extend our scientific network. The glossary is considered as a source of background filling the gap of describing the specificity of diagnostic reasoning in general practice7 .

  whether any changes were needed to improve feasibility, and to calculate the prevalence of the sense of alarm and of the sense of reassurance of GPs in three different countries. The first step, a think-aloud study and a feasibility study, led to small modifications concerning the order of items, and to some small adaptations of the wording of two items. The modified version of the GFQ was created without altering the sense of the seven validated items. The second step, a repetition of the feasibility study but with the modified questionnaire, led to minor changes. The prevalence of the gut feelings in the two phases of the feasibility study, 23 and 31 % respectively, were similar in Belgium, France and the Netherlands. The questionnaire was modified after the two phases of the study. Now we have a questionnaire formatted by GPs, for GPs, working in three European countries. The final version of the GFQ proved to be a feasible and practical tool to be used for prospective observational studies in daily practice.The next study related the construction of a glossary of diagnostic reasoning terms regarding to the research of gut feelings, by the COGITA group. The researchers reviewed relevant literature, aiming to define salient terms, used in their publications. They described the terms, cross-reviewed the wording and reached consensus within the group. Two sections were created: (1) a diagnostic reasoning section that describes concepts such as analytical and nonanalytical reasoning processes, clinical mind lines, and intuition, and (2) a research methods section describing concepts such as linguistic validity and saturation. Defining terms in the area of decision making developed by a European expert consensus, based on a literature review and gathering them in an open access glossary is an unique initiative. This glossary was a prerequisite to conduct further research with the intention to create teachable knowledge as well as a basis for cross-border research in general and family medicine. We made the glossary freely available (published on the website www.gutfeelingsingeneralpractice.eu) in order to share our results with other researchers, and to extend our scientific network. The glossary was considered as a source of background filling the gap of the specificity of diagnostic reasoning in general practice.

  embolism arose out of four considerations: the absence of indicative clinical signs for diagnoses other than PE, a sudden change in the condition of the patient, a gut feeling that something was wrong and a failure to diagnose PE in the past. The GPs interviewed did not use any prediction rule or score in their diagnostic process. The perception of alarm compelled the physician to quit his routine-based reasoning to an analytical reasoning by generating and considering the PE hypothesis. If they had a gut feeling that there was something wrong, that it could be a PE, then they decided to refer the patient to the hospital specialist. The sense of alarm acted as a feedback mechanism at a very early stage of the diagnostic process, allowing the questioning of a possibly wrong diagnostic reasoning direction.The objective of the next study was to calculate the diagnostic test accuracy of a GP's sense of alarm when confronted with dyspnoea and chest pain. A prospective study with25 French GPs who filled in the GFQ after each consecutive consultation for chest pain and dyspnoea collected 235 questionnaires with a sense of alarm or a sense of reassurance. Experiencing a sense of alarm when a patient consults a GP for chest pain and or dyspnoea implies that the patient has a 35% probability of life threatening disease when the initial estimated probability was 20% (LR+ = 2.12 [IC95 = 1.49; 2.82]). The negative Likelihood ratio was (LR-= 0.55 [IC95 = 0.37; 0.77]). Not experiencing a sense of alarm decreases the probability of having a lifethreatening pathology from 20 % to 12%. The sensitivity of the sense of alarm was 0.608 [IC95 = 0.475; 0.725] and the specificity was 0.713 [IC95 = 0.679; 0.75]. In vague and uncertain

  diagnose: 'alles klopt'. Het pluis gevoel en het niet-pluis gevoel vormen een dynamisch element in het diagnostische proces van de huisarts. Het PNP-gevoel is beschreven als een derde spoor naast de medisch-besliskundige benadering en de medisch-probleemoplossende benadering en zij spelen elk hun rol in het continue samenspel van niet-analytisch en analytisch diagnostisch denken. Zelfs al bestaat er geen typische uitdrukking in elke taal, toch werd het PNP-gevoel in alle Europese landen herkend door huisartsen. In 2013 werd er een PNP-vragenlijst (GFQ) gemaakt op basis van deze definities en vervolgens gevalideerd via een constructvalidatie met case vignettes. Het NP-gevoel en het P-gevoel bleken twee tegenpolen te zijn van één component dat 70,2% van de variantie kon verklaren na een hoofdcomponenten analyse. De interne consistentie van de vragenlijst is hoog (Cronbach's alpha = 0.91). De kappa met kwadratische weging was eveneens aanzienlijk (0.62, 95% CI: 0,55-0,69). Een linguïstische validatieprocedure werd uitgevoerd om een Engelse versie van de vragenlijst te bekomen.

  behulp van een gestandaardiseerde procedure van linguïstische validatie. De samenwerkende onderzoeksteams uit Frankrijk, Duitsland/Zwitserland en Polen bereikten overeenstemming over compromissen tussen vergelijkbaarheid en gelijkheid aan de ene kant, en taalkundige en culturele eigenheid anderzijds. De vragenlijst is het eerste instrument dat ontwikkeld is om PNP-gevoelens te meten. Er is geen ander instrument beschikbaar op dit moment. De transculturaliteit van het PNP-gevoelens concept tussen Proto-Germaanse en Romaanse talen werd bevestigd na een Delphi-procedure waarbij de Nederlandse en de Franse statements rond pluis/niet-pluis werden vergeleken. De gevolgde linguïstische validatieprocedure maakte het mogelijk om het concept ook uit te breiden tot de Slavische talen zoals het Pools. Wij gaan ervan uit dat het nut van de vragenlijst ook overdraagbaar is, als we werken binnen deze transculturele context en een gestandaardiseerde taalkundige procedure toepassen. De vertaalde en gevalideerde vragenlijst maakt het mogelijk om onderzoeksresultaten naar de voorspellende waarde van PNP-gevoelens en naar de betekenis van de belangrijkste determinanten ervan in vijf Europese landen te vergelijken.Vervolgens hebben we ons gericht op het testen van de vragenlijst in de dagelijkse huisartsenpraktijk tijdens de kantooruren om na te gaan of er wijzigingen nodig waren om de bruikbaarheid te verhogen, en voor het berekenen van de prevalentie van het NP-gevoel van en het P-gevoel bij huisartsen in drie verschillende landen. De eerste stap, een think-aloud studie en een haalbaarheidsstudie, heeft geleid tot kleine wijzigingen in de volgorde van items, en enkele kleine aanpassingen voor de formulering van twee items. De gewijzigde versie van de vragenlijst is gemaakt zonder de kern van de zeven gevalideerde items te wijzigen. De tweede stap, een herhaling van de haalbaarheidsstudie maar met de gewijzigde vragenlijst, leidde tot kleine wijzigingen. De prevalentie van de PNP gevoelens in de twee fasen van de haalbaarheidsstudie, respectievelijk 23 en 31%, bleek gelijkaardig in België, Frankrijk en Nederland. De vragenlijst werd aangepast na de twee fasen van deze studie. We beschikten nu over een vragenlijst opgemaakt door huisartsen, voor huisartsen, werkzaam in drie Europese landen. De definitieve versie van de PNP-vragenlijst bleek een haalbaar en praktisch hulpmiddel om te gebruiken voor prospectieve observationele studies in de dagelijkse praktijk.De volgende studie was de opbouw van een woordenlijst van specifiek klinisch besliskundige termen met betrekking tot het onderzoek van PNP-gevoelens, door de COGITA-groep. De onderzoekers deden een onderzoek naar relevante literatuur, gericht op het definiëren van de meest gebruikte termen in hun publicaties. Zij beschreven de termen, toetsten de formulering en bereikte consensus binnen de groep hierover. Twee secties zijn gemaakt: (1) een sectie met betrekking tot diagnostisch redeneren waarin concepten zoals analytische en niet-analytische redeneerprocessen, klinische mindlines en intuïtie, en (2) een gedeelte over specifieke onderzoeksmethoden met concepten zoals linguistische validiteit en saturatie. Het is een uniek initiatief om klinisch besliskundige termen te definiëren op basis van consensus van Europese deskundigen en een literatuurstudie, en samen te brengen in een open access woordenlijst. Deze woordenlijst was een voorwaarde voor verder onderzoek met de intentie om leerbare kennis te verzamelen alsook als een basis voor grensoverschrijdend onderzoek in de huisartsgeneeskunde. We maakten het glossarium vrij beschikbaar (gepubliceerd op de website www.gutfeelingsingeneralpractice.eu) om onze resultaten te delen met andere onderzoekers, en ons wetenschappelijk netwerk uit te breiden. De woordenlijst werd beschouwd als achtergrondinformatie om het hiaat van de specificiteit van het diagnostisch redeneren in de praktijk in te vullen.Tweede deelAllereerst werd onderzocht hoe huisartsen komen tot het vermoeden van een LE in de dagelijkse praktijksetting. Een ander doel was om meer in het bijzonder te beschrijven wat de rol van PNP-gevoelens is in dit diagnostisch proces. Vooraleer een bepaalde predictiescore of -regel toe te passen voor LE, moet een huisarts eerst een vermoeden hebben op LE en het is juist deze eerste etappe die onduidelijk is. Het diagnostische proces dat leidt tot het vermoeden van LE is niet goed beschreven in de eerstelijnszorg. Daarom werd een thematische analyse van 28 semi-gestructureerde kwalitatieve interviews uitgevoerd, door drie onderzoekers, vanuit een Grounded Theory coderingsparadigma. Het vermoeden op een LE ontstaat vanuit vier overwegingen: de afwezigheid van indicatieve klinische tekens voor andere diagnoses dan LE, een plotse verandering in de toestand van de patiënt, een buikgevoel dat er iets mis was en een vroeger falen in het diagnosticeren van een LE. De geïnterviewde huisartsen gebruikten geen predictieregel of -score in hun diagnostisch proces. De perceptie van een NPgevoel dwong de arts te stoppen met zijn op routine gebaseerde redenering en over te stappen naar analytisch redeneren door het genereren en rekening houden met de LE-hypothese. Als ze een NP-gevoel hadden dat er iets mis was, dat het een LE zou kunnen zijn, besloten ze vervolgens om de patiënt door te verwijzen naar de ziekenhuis-specialist. Het NP-gevoel trad op als een feedback-mechanisme in een zeer vroeg stadium van het diagnostisch proces, wat toeliet om een mogelijks verkeerde richting qua diagnostisch redeneren in vraag te stellen.Het doel van de volgende studie was het berekenen van de diagnostische test-nauwkeurigheid van het NP-gevoel wanneer geconfronteerd met kortademigheid en pijn op de borst. In een prospectieve studie met 25 Franse huisartsen, die PNP-vragenlijsten invulden na elke opeenvolgende raadpleging voor pijn op de borst en kortademigheid, werden 235 vragenlijsten verzameld met een PNP-gevoel. Ervaren van een NP-gevoel wanneer een patiënt een huisarts raadpleegt voor pijn op de borst en/of kortademigheid houdt in dat de patiënt een 35% kans heeft op een levensbedreigende ziekte als het basisrisico 20% was (LR + = 2.12 [BI95 = 1,49; 2.82]). Niet ervaren van een NP-gevoel vermindert de kans op een levensbedreigende ziekte van 20% tot 12%. De gevoeligheid van een NP-gevoel was 0.608 [BI95 = 0.475; 0.725] en de specificiteit was 0.713 [BI95 = 0.679; 0,75]. Bij vage en onduidelijke situaties van pijn op de borst en/of kortademigheid waar het moeilijk is voor een huisarts om een juiste aanpak te bapelen, bleek een PNP-gevoel een nuttig hulpmiddel in hun redeneringsproces.DiscussieDe algemene discussie richt zich op de sterkten en zwakten van onze studies en biedt enkele opmerkingen over de preventie van de fouten in de eerstelijnszorg. Het bespreekt ook enkele opmerkingen voor de opleiding.Criteria voor PNP-gevoelens werden geformuleerd op basis van wat huisartsen beschreven uit hun eigen praktijk. Het pluis en niet-pluis gevoel vormt een dynamisch element in het diagnostische proces van huisartsen en helpt te pendelen tussen niet-analytisch en analytisch diagnostisch redeneren. De PNP vragenlijst werd vertaald in het Frans, Duits en Pools na een gestandaardiseerde procedure van linguïstische validatie. De vragenlijst werd vervolgens in de echte praktijk setting getest tijdens kantooruren om de bruikbaarheid ervan te bepalen in België, Frankrijk en Nederland. Na analyse van de kwalitatieve interviews bij huisartsen, werd het NP-gevoel geïdentificeerd als een belangrijke factor om tot een positieve diagnose van LE te komen. De vragenlijst werd ook gebruikt in een prospectieve studie bedoeld om de diagnostische nauwkeurigheid van het NP-gevoel te berekenen wanneer men in de huisartspraktijk in aanraking komt met thoracale pijn en kortademigheid. Een NP-gevoel in deze situatie dreef de huisartsen twee keer vaker naar de diagnose van een ernstige ziekte dan zonder dit gevoel. Het NP-gevoel kan beschouwd worden als een aanvullend instrument om te leren hoe je een fout in de eerstelijnszorg kan voorkomen. Het is het eerste model waar de preventie van fouten wordt geassocieerd met de besluitvorming in een zeer vroeg stadium in de huisartspraktijk. Verder onderzoek betreffende het vermoeden van de diagnose kanker of het onderwijzen van de PNP-gevoelens moet worden bevorderd.

Table 1 .

 1 A summary of the different steps of the linguistic and cultural validation 6 steps

7. This patient's situation gives me reason to arrange a follow-up visit sooner than usual or to refer him or her more quickly than usual to a specialist. ❑ Yes ❑ No 8. A. What do you consider to be the most likely diagnosis? (Please tick one answer.) My most likely diagnosis is …………………………………………………. There are several possible diagnoses; I am unable to choose one at this moment. ………………………………………………………… B. And which diagnosis will determine your management? ……………………………… 9. How confident are you in the diagnosis that you indicated under 8b as determining your management? ____% 10. Please indicate what kind of gut feeling you have at the end of the consultation: ❑ Something is wrong with this picture. ❑ Everything fits. ❑ Impossible to say, or not applicable.…………………… Additional file n°4: GFQ English version. The English version of the Gut Feeling Questionnaire

Table 2 .

 2 Four versions of the gut feeling questionnaire

	English	French	German	Polish
	1. I feel	1. J'ai confiance	1. Ich fühle mich	1. Jestem pewny co
	confident about	dans la prise en	sicher in Bezug auf	do mojego planu
	my	charge que je	meinen	postępowania i/lub
	management	propose	Behandlungsplan	wyników: wszystko
	plan and/or	et / ou dans ses	und/oder das	zgadza się.
	about the	résultats	klinische Ergebnis:
	outcome: it all	attendus : tout	Es passt alles gut
	adds up.	est cohérent.	zusammen.
	2. I am	2. Je suis	2. Ich bin besorgt	2. Jestem
	concerned	préoccupé(e)	über den	zaniepokojony
	about this patient's state	par l'état de santé de ce	Gesundheitszustand dieses Patienten:	stanem zdrowia tego pacjenta: coś tu się
	of health:	patient :	hier stimmt etwas	nie zgadza.
	something does	quelque chose	nicht.
	not add up	ne va pas.
	here.
	3. In this	3. Pour ce cas	3. In diesem	3. W tym konkretnym
	particular case,	précis, je vais	speziellen Fall werde	przypadku sformułuję
	I will formulate	formuler des	ich vorläufige	tymczasowe hipotezy
	provisional	hypothèses de	Verdachtsdiagnosen	z potencjalnie
	hypotheses with	pathologies	formulieren, mit	istotnymi wynikami i
	potentially	potentiellement	möglicherweise	porównam je.
	serious	graves que je	schwerwiegenden
	outcomes and	confronterai	Folgen, die ich
	weigh them	les unes aux	gegeneinander
	against each	autres.	abwägen muss.
	other.
	4. I have an	4. Je suis gêné	4. Ich habe ein	4. Mam niejasne
	uneasy feeling because I am	(e) parce que je redoute de	ungutes Gefühl, weil ich über mögliche	przeczucie ponieważ martwią mnie
	worried about	possibles	ungünstige Folgen	potencjalnie
	potentially	conséquences	besorgt bin.	niekorzystne wyniki.
	unfavourable	graves pour ce
	outcomes.	patient.
	5. This case	5. Ce cas	5. Dieser Fall	5. Ten przypadek
	requires specific	nécessite une	erfordert eine	wymaga
	management to	prise en charge	besondere	szczególnego
	prevent any further serious health	spécifique afin d'éviter d'autres problèmes de	Herangehensweise, um mögliche ernste Komplikationen zu	postępowania aby zapobiec dalszym poważnym
	problems.	santé graves	vermeiden.	problemom
		pour le patient.		zdrowotnym.
	6. What course	6. Quel plan	6. Wie sieht Ihr	6. Jaki rodzaj
	of action have you chosen? (Please tick one answer)	d'action avez-vous choisi (une seule réponse	weiteres Vorgehen aus? (Bitte nur eine Antwort ankreuzen.) Ich werde…	postępowania wybrałeś? (zaznacz jedną odpowiedź)

  For my own line of reasoning I would have preferred an extra question inserted here, like: what options are you thinking of? Differential diagnoses 1, 2 and 3. And which one do you consider to be the most important one, the one you absolutely want to exclude?

	not really true, is it? Because your management is not aiming to exclude
	pneumonia…". (Dutch GP n°6)
	And which one would you perhaps want to address?". (Dutch GP n°4)
	"So I can argue about that, but I find it more difficult to express it in a number, medically
	speaking… […] you'd say 'which diagnosis would determine your management? and
	that would be pneumonia. But you think the likelihood is very small: 5%. […] so you

wouldn't have an X-ray done or do a CRP test… and then under 8b it says "'which diagnosis would determine your management?' You write down pneumonia, but that's

  1. Please indicate what kind of gut feeling you have at the end of the consultation. If you cannot answer this question now, go further to the next 9 items and give your answer afterwards.

	Ο something is wrong with this picture. Ο everything fits. Ο impossible to say, or not applicable.
	2. It all adds up. I feel confident about my management plan and/or about the outcome.	Ο	Ο	Ο	Ο	Ο
	3. Something does not add up here. I am concerned about this patient's state of health.	Ο	Ο	Ο	Ο	Ο
	4. In this particular case, I will formulate provisional hypotheses with potentially serious outcomes and	Ο	Ο	Ο	Ο	Ο
	weigh them against each other.
	5. I have an uneasy feeling because I am worried about potentially unfavourable outcomes.	Ο	Ο	Ο	Ο	Ο
	6. To prevent any further serious health problems this case requires specific management	Ο	Ο	Ο	Ο	Ο
	7. This patient's situation gives me reason to arrange a follow-up visit sooner than usual or to refer him	Ο	Ο	Ο	Ο	Ο
	or her more quickly than usual to a specialist.
	8. Which diagnosis/es are you thinking about? ……………………………………………………………….. ………………………………………………………………..

9. What management have you chosen? (Please tick one answer.) I will…

  Please indicate what kind of gut feeling you have at the end of the consultation. If you cannot answer this question now, please answer the following nine questions, then give your answer to question 1, which is repeated at the end of the questionnaire.

	Ο something is wrong with this picture. Ο everything fits. Ο impossible to say, or not applicable.
	2. It all adds up. I feel confident about my management plan and/or about the outcome.	Ο	Ο	Ο	Ο	Ο
	3. Something does not add up here. I am concerned about this patient's state of health.	Ο	Ο	Ο	Ο	Ο
	4. In this particular case, I will formulate provisional hypotheses with potentially serious outcomes and	Ο	Ο	Ο	Ο	Ο
	weigh them against each other.
	5. I have an uneasy feeling because I am worried about potentially unfavourable outcomes.	Ο	Ο	Ο	Ο	Ο
	6. To prevent any (further) serious health problems requires specific management of this case.	Ο	Ο	Ο	Ο	Ο
	7. This patient's situation gives me reason to arrange a follow-up visit sooner than usual or to refer him	Ο	Ο	Ο	Ο	Ο
	or her more quickly than usual to a specialist.
	8. What diagnoses (or diagnosis) do you have in mind? (max. 3) ……………………………………………………………….. ……………………………………………………………….. ………………………………………………………………..

1. If you have already given an answer, there is no need to answer this question again. Please indicate what kind of gut feeling you have at the end of the consultation: Ο something is wrong with this picture. Ο everything fits. Ο impossible to say, or not applicable. 1. 9. What management have you chosen? (Please tick one answer.) I will…

  10. Which diagnosis has determined your management? ………………………………………………………………. 11. This question is the same as question 1. If you have already given an answer, there is no need to answer this question again. Please indicate what kind of gut feeling you have at the end of the consultation: Ο something is wrong with this picture. Ο everything fits. Ο impossible to say, or not applicable. If you want to share some thoughts about your diagnostic reasoning, please use the back of this questionnaire.

Table 2

 2 Characteristics of the 28 FPs interviewed for data collection

	Range	Group 1 i

  . The sense of alarm was first described in general practice: GPs are familiar with their patients and used to observing, listening and being open for what is different and dissonant and what is as usual. Many situations are not described as they appear in general practice: they are "different from what the textbooks say" 36 . Also, in meningococcal disease, the three classic symptoms in children and adolescents, rash, meningism, and impaired consciousness, occur very late in the disease history. Leg pain, cold hands and feet, and abnormal skin colour appear earlier

  3 . Should we teach gut feelings and, if so, how? Relying on gut feelings can be a tool, identified by GPs, for actively dealing with uncertainty 34, . Integrating the gut feelings concept into the postgraduate education of young GPs may be a relevant way to help them cope with uncertainty. For Malterud et al., certainty is neither possible nor desirable in general practice54

ACKNOWLEDGMENTS

We are indebted to numerous general practitioners from France, from Northern and Southern Germany and Switzerland, from Poland as well to many others for their help. We are indebted to Prof. Antonius Schneider, Munich, for his support in the German cultural validation process.

We thank Sébastien de Villèle, Jean Yves Le Disez and Helen McCombie-Boudry from the Bureau de Traduction Universitaire Université de Bretagne Occidentale and Jean Sébastien Cadwallader who provided the French translations for the linguistic validation procedure; Alex Gillman who provided medical writing services on behalf of the Collège Brestois des Généralistes Enseignants.

Acknowledgments

We are grateful for the help of the 72 participating GPs, and of Alex Gillman who provided medical writing services on behalf of the Collège Brestois des Généralistes Enseignants.

ACKNOWLEDGEMENTS

The authors want to thank the members of the COGITA group who participated in building the glossary.

RUNNING TITLE

Glossary of diagnostic reasoning terms

Acknowledgments

We are grateful for the help of the FP participants who shared their time with us, Alex Gillman for the re-writing and Pr F Couturaud, Département de Médecine Interne et Pneumologie, Centre Hospitalier Universitaire de la Cavale Blanche, Brest.

Acknowledgments

We thank Alex Gillman who provided medical writing services on behalf of the Collège Brestois des Généralistes Enseignants; Sébastien de Villèle, Jean Yves Le Disez and Helen

McCombie-Boudry from the Bureau de Traduction Universitaire Université de Bretagne Occidentale and Jean Sébastien Cadwallader who provided the translations for the linguistic validation procedure.

This study was approved by the ethical committee of the University de Bretagne Occidentale.

Funding statement

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors

Funding statement

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Data analysis

We conducted the same thematic content analysis, using the codebook from the first feasibility study. The quantitative data, i.e. the answers to items 1-6 and 11, were analysed with a Chi² test.

Data sharing statement

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

The Gut Feelings Questionnaire in daily practice: a feasibility study in three European countries Accepted in the BMJ Open.

Competing interests

None of the authors have any competing interests.

Methods

Think-aloud study

Participants

Three groups, differing in their level of experience in Dutch general practice, participated in a think-aloud study on diagnostic reasoning. Participants were 8 experienced GPs (7 female; average experience in GP practice was 18,6 years, ranging from 6 to 29 years), 8 first-year GP-trainees (5 female; average clinical experience before their traineeship was 24,5 months, ranging from 9 to 53 months) and 8 advanced medical students (7 female) doing their internship in general practice at Maastricht University. The experienced GPs were recruited through a snowball strategy in the Netherlands, whereas the trainees and medical students were approached via the Department of Family Medicine at Maastricht University in the Netherlands.

Patients were not involved.

Materials

Six case vignettes were developed based on actual accounts from patients. Each case vignette briefly described the complaints, medical history, and results from history taking and physical examination. The cases described patients with myalgia, asthma, cardiomyopathy, pancreatic carcinoma, panic disorder and pulmonary embolism. In the real-life situation, three cases had produced a sense of alarm and three a sense of reassurance. Four cases had previously been used in the validation study [4]. The original Dutch GFQ was used (Figure 1). A glossary of diagnostic reasoning terms by the COGITA expert group Eur J Gen Pr 2017; 23 (1): 53-56.

Abbreviations

GFQ Gut

ABSTRACT

The role of gut feelings in diagnostic reasoning is recognized by most GPs throughout Europe and probably throughout the world. Studies on this topic have emerged from different countries but there is the risk that authors will use different terms for similar concepts. The "European Expert Group on Cognitive and Interactive Processes in Diagnosis and Management in General Practice", COGITA for short, was founded in 2008 to conduct cross-border research in the area of non-analytical diagnostic reasoning. Academic GPs, PhD students, psychologists, linguists and students meet once a year to share their experiences, exchange results and initiate new studies on the topic.

A milestone in their research is this publication of a short glossary of diagnostic reasoning terms relating to the gut feelings research topic. It was constructed by the COGITA group members following a literature review, which aimed to define salient terms, used in their publications. They described the terms, cross-reviewed the wording and reached consensus within the group. Two sections were created: (1) a diagnostic reasoning section that describes concepts such as analytical and non-analytical reasoning, clinical mindlines, and intuition, and

(2) a research methods section describing concepts such as linguistic validity and saturation.

The glossary, including relevant literature, has been published on the website www.gutfeelingsingeneralpractice.eu. In the future, the glossary will be modified if necessary and completed by members of the COGITA group. Method: Semi-structured qualitative interviews with 28 FPs. The regional hospital supplied data of all their cases of pulmonary embolism from June to November 2011. The patient's FP was identified where he/she had been the physician who had sent the patient to the emergency unit. The first consecutive 14 FPs who agreed to participate made up the first group. A second group was chosen using a purposeful sampling method. The topic guide focused on the circumstances leading to the suspicion of PE. A thematic analysis was performed, by three researchers, using a grounded theory coding paradigm.

Results: In the FPs' experience, the suspicion of pulmonary embolism arose out of four considerations: the absence of indicative clinical signs for diagnoses other than PE, a sudden change in the condition of the patient, a gut feeling that something was seriously wrong and an earlier failure to diagnose PE. The FPs interviewed did not use rules in their diagnostic process.

Conclusion:

This study illustrated the diagnostic role of gut feelings in the specific context of suspected pulmonary embolism in primary care. The FPs used the sense of alarm as a tool to prevent the diagnostic error of missing a PE. The diagnostic accuracy of gut feelings has yet to be evaluated. Table 1 Topic guide for the interviews in both sampled groups of family physicians (FPs)

Aim

To explore how FPs come to suspect pulmonary embolism using two groups: FPs who had recently diagnosed a case of PE; and FPs chosen using a purposeful sampling method For the first group

Methods and analysis

Prospective observational study. Patients aged between 18 and 80 years, consulting their GP for dyspnoea and/or thoracic pain will be considered for enrolment in the study. These GPs will have to complete the questionnaire immediately after the consultation for dyspnoea and/or thoracic pain. The follow-up and the final diagnosis will be collected 4 weeks later by phone contact with the GP or with the patient if their GP has no information. Life threatening and nonlife-threatening diseases have previously been defined according to the pathologies or symptoms in the ICPC2 classification. Members of the research team, blinded to the actual outcomes shown on the index questionnaire, will judge each case in turn and will, by consensus, classify the expected outcomes as either life threatening or non-life-threatening diseases. The sensitivity, the specificity, the positive and negative likelihood ratio of the sense of alarm will be calculated from the constructed contingency table.

Ethics and Dissemination

responsible for communicating the information to patients. In France, patients are not used to being involved in research programs at their GP's surgery. Informing them of the study design and answering their questions are too time consuming for French GPs. The trainee's role here will be to facilitate this information phase. Consecutive patients, for whom dyspnoea and/or thoracic pain are the reason for contact, will be enrolled over a period of 12 weeks. Dyspnoea is defined as difficult or laboured breathing (MeSH Medical Subject Headings definition, the National Library of Medicine controlled vocabulary thesaurus used for indexing PubMed citations.). Chest pain is defined as pressure, burning, or numbness in the chest (Mesh definition). The GPs will be informed directly of the results of the study.

Non-inclusion criteria are: patients in palliative care, and patients known to have coronary heart disease. Patients known to have pulmonary diseases are not excluded because of the possibility of the co-existence of pulmonary embolism or secondary infection and other pulmonary pathologies such as COPD in the same patient.

In order to limit selection bias in cases, the GPs and trainees will be trained to detect all cases and not to focus solely on the serious one presented at the surgery. The GPs and trainees will be shown a power point showing the Mesh definition of thoracic pain and dyspnoea, including case vignettes. Inclusion and non-inclusion criteria will figure on the back of each questionnaire.

Sample size

The size of the sample will be estimated according to the following data. The incidence of the sense of alarm was 7 % for the respiratory ICPC code chapter and 15 % for the circulatory ICPC code chapter in the first Dutch study [12]. The prevalence of consultations for dyspnoea in France in primary care is 1.77 % and 1.51 % for thoracic pain [13]. We defined our initial population as 40 volunteer GPs, each following up, on average, 800 patients in their practice.

We included a physician level and a patient level in our calculation. The number of cases required for a power of 80% and an error rate of 5% is 211 for thoracic pain with 34 GPs and 123 for dyspnoea with 31 GPs. Taking into account, the Lasagna effect , we estimated 7 cases of thoracic pain per GP and 4 cases of dyspnoea per GP. Epi Info™ 6.04 Software will be used to perform the analysis.

Data Collection

The final diagnosis will be collected four weeks later: by phone contact with the GP to find out how the patient's condition has progressed; or by phone contact with the patient if the GP has no information. One researcher from the team will collect follow-up data and gather information on the way used to achieve final diagnosis. We will not include specific analysis on the

Ethics and dissemination

This study was approved by the ethical committee of the University de Bretagne Occidentale and declared to the French data protection authority (CNIL) n°1637307 v 0 (12-12-2012). The GPs will sign a written consent after receiving information, both by e mail and by post. Patients will sign a written consent after information had been delivered orally by a trainee, as well as in writing.

The findings of this study complete the description of the sense of alarm by contributing an essential quantitative component. We will ensure that the results are widely disseminated

through publication in open access journals as well as conference presentations.

Authors' contributions

MB, PB, BLF, PVR, CL and ES conceived the study, and participated in its design and coordination and helped to draft the manuscript. CL performed the statistical analysis. FS and ASM carried out the linguistic validation procedure and supervised the nominal group. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests

Chapter 7

The accuracy of the general practitioner's sense of alarm when confronted with dyspnoea and/or thoracic pain: prospective observational study

Abstract Background and Objectives

Dyspnoea and chest pain are signs shared with multiple pathologies ranging from the benign to life-threatening diseases. Gut feelings such as the sense of alarm and the sense of reassurance are known to play a substantial role in the diagnostic reasoning of general practitioners (GPs). A Gut Feelings Questionnaire (GFQ) has been validated to measure the general practitioner's sense of alarm and/or sense of reassurance. The aim of the study was to estimate the diagnostic test accuracy of GPs' sense of alarm when confronted with dyspnoea and chest pain.

Methods

Prospective observational study in general practice. Patients aged between 18 and 80 years, consulting their GP for dyspnoea and/or chest pain, were considered for enrolment in the study.

These GPs had to complete the GFQ immediately after the consultation for dyspnoea and/or chest pain. The follow-up and the final diagnosis were collected 4 weeks later by phone contact with the GP. Life-threatening and non-life-threatening diseases have previously been defined according to the pathologies or symptoms in the ICPC2 classification. The index test was the sense of alarm and the reference standard was the final diagnosis at 4 weeks.

Results

From November 2016 to the end of December 2017, 25 GPs recruited 241 patients and filled in GFQ questionnaires. Of these, 235 detected a sense of alarm and/or sense of reassurance.

The positive likelihood ratio for the sense of alarm was 2.

Discussion

Where the physician experiences a sense of alarm when a patient consults for dyspnoea and/or chest pain, it has been found that the post-test odds that this patient has, in fact, a lifethreatening disease are about twice as high as the pre-test odds.

Registration

Clinical Trial: NCT02932982

The accuracy of the GP's sense of alarm when confronted with dyspnoea and/or chest pain was not known. The aim of this study was to calculate the diagnostic test accuracy of the sense of alarm when applied to dyspnoea and/or chest pain, using the GFQ.

Method

We conducted a prospective observational study using the French version of the GFQ. The protocol of the study has been published 22 .

Participants

Patients aged between 18 and 80 years, consulting their GPs for dyspnoea and/or chest pain were considered for enrolment in the study. Consecutive patients, for whom dyspnoea and/or chest pain were the reason for contact, were enrolled over a period of one year. Dyspnoea was defined as difficult or laboured breathing (MeSH = Medical Subject Headings definition, the National Library of Medicine controlled vocabulary thesaurus used for indexing PubMed citations). Chest pain is defined as pressure, burning, or numbness in the chest (Mesh definition).

Non-inclusion criteria were: patients in palliative care, and patients known to have coronary heart disease. Patients known to have pulmonary diseases were not excluded because of the possible co-existence of a life-threatening event (e.g. embolism, secondary infection) along with other pulmonary pathologies such as COPD.

GP Participants

GPs in the General Practice Department of Brest University were informed of the study through a personal email. Personal phone calls, and presentations during academic meetings promoted the study. The GPs taking part received a videotaped presentation with an explanation of the objectives and the study design. The GPs had to fill in the GFQ right after each consultation where dyspnoea and/or thoracic pain were the reason for the consultation.

The sample size was estimated before the start of the inclusion. The prevalence of consultations for dyspnoea in France in primary care is 1.77 % and 1.51 % for thoracic pain 3 .

In the first Dutch study, which aimed at estimating the incidence of gut feelings in general practice consultations, the incidence of the sense of alarm was 7 % for the respiratory ICPC code chapter and 15 % for the circulatory ICPC code chapter 23 . We defined our initial population as 40 volunteer GPs, each following up, on average, 800 patients in their practices.

We included a physician level and a patient level in our calculation. The number of cases required for a power of 80% and a type 1 error rate of 5% was 211 for thoracic pain with 34

GPs and 123 for dyspnoea with 31 GPs. Taking into account the Lasagna effect, we estimated

Data analysis

The scores to the GFQ were assessed by two independent researchers blind to the final

Research questions of this thesis

The first part aimed at translating the gut feelings concept into European languages and contexts:

• How to translate the GFQ from English into French, German and Polish?

• How feasible is the GFQ when tested in real practice?

• How to define salient terms on diagnostic reasoning research used in the publications of the COGITA group?

The second part focused on the diagnosis of a life-threatening disease and the diagnostic value of the sense of alarm in general practice applied to the symptoms dyspnoea and thoracic pain.

• How did GPs come to suspect pulmonary embolism (PE) in their daily practice?

• What is the diagnostic test accuracy of the sense of alarm when applied to dyspnea and chest pain?

What this thesis adds First part

We started with the translation of the GFQ into French, German and Polish using a standardised procedure of linguistic validation. The collaborating research teams from France, Germany/Switzerland and Poland found and agreed on compromises between comparability and similarity on one hand, and linguistic and cultural specificities on the other. The GFQ is the first tool developed that measures GPs' gut feelings. There is no alternative tool available at present. The transculturality of the gut feelings concept between Proto-Germanic and Romance languages was revealed after a Delphi procedure comparing the Dutch and the French statements of the gut feelings criteria 6 . The linguistic validation procedures which

Summary

Physicians' clinical decision-making is based on the interaction of analytical and non-analytical reasoning and gut feelings can be considered a part of the non-analytical reasoning process.

The two kinds of gut feelings of general practitioners (GPs), the sense of alarm and the sense of reassurance, were described after qualitative studies in the Netherlands and in Flanders.

They concern the prognosis of the patient's state of health during the consultation. The sense of alarm means that a GP perceives an uneasy feeling as he/she is concerned about a possible adverse outcome. It implies that a GP worries about a patient's health status, even though he/she has found no specific indications yet; it is a sense of 'there's something wrong here'. A 'sense of alarm' means that, if possible, the GP needs to initiate specific management to prevent serious health problems. The sense of reassurance means that a GP feels secure about the further management and course of a patient's problem, even though he/she may not be certain about the diagnosis: everything fits in. The sense of reassurance and the sense of alarm constitute a dynamic element in a GP's diagnostic process. Gut feelings have been described as a third track alongside medical decision-making and medical problem-solving and play their part in the continually interaction of non-analytical and analytical diagnostic reasoning processes. Even if an idiomatic expression does not exist in every language, the gut feelings were experienced by GPs in all European countries. In 2013 a Dutch language Gut Feelings Questionnaire (GFQ) was created from the definition criteria and validated by a construct validation procedure using case vignettes. The sense of alarm and the sense of reassurance were found as two opposites of one component explaining 70.2% of the variance after a principal component analysis. The internal consistency of the GFQ proved to be high (Cronbach's alpha = 0.91). The kappa with quadratic weighting was substantial (0.62, 95% CI: 0.55-0.69). A linguistic validation procedure was performed to obtain an English version of the questionnaire.

This thesis was divided into two parts.

The first part aims at translating the gut feelings concept into European languages and contexts:

• What was the process for translating gut feelings into French? A Delphi procedure, with

French GP experts, was used to translate "gut feelings" into the French language.

• How was the GFQ translated from English into French, German and Polish? Linguistic validation processes, following a procedural scheme, were undertaken in France, Germany and Poland.

• How was the feasibility of the GFQ tested in real practice? A mixed method study was followed in Belgium, The Netherlands, and France to obtain the final version of the GFQ.

• How did you define the salient terms on diagnostic reasoning used in the publications of the COGITA group? A glossary of terms was constructed following a literature review.

• The differences which occurred in the definitions and translations of gut feelings and the connected concepts led to a comment on the article "Recognition of sepsis in primary care: a survey among GPs" written by Loots et al, and their use of the wording "gut feelings" without describing to which concept they were referring.

The second part focuses on the diagnosis of a life-threatening disease and the diagnostic value of the sense of alarm in general practice with the symptoms of dyspnoea and thoracic pain.

• How did GPs come to suspect pulmonary embolism in real-life settings? A qualitative study with semi structured interviews explored this point. Next, more specifically the role of gut feelings in this diagnostic process was described.

• The results of this qualitative study led to a comment on the Lucassen article on the comparison of the diagnostic performance of "gestalt" and the Wells rule. for ruling out pulmonary embolism in primary care.

• What is the diagnostic test accuracy of the sense of alarm when applied to dyspnoea and chest pain? A prospective study using the gut feelings questionnaire was undertaken with French GPs. The protocol for this study was published and the first results of the prospective study are reported.

Linguistic validation process, mixed method with quantitative and qualitative data, qualitative exploring method with semi structured interviews, prospective quantitative study with a validated questionnaire were the different methods used in this thesis.

First part

We started with the translation of the GFQ into French, German and Polish using a standardised procedure of linguistic validation. The collaborating research teams from France, Germany/Switzerland and Poland found and agreed on compromises between comparability and similarity on one hand, and linguistic and cultural specificities on the other. The GFQ is the first tool developed that measures GPs' gut feelings. There is no alternative tool available at present. The transculturality of the gut feelings concept between Proto-Germanic and Roman languages was revealed after a Delphi procedure comparing the Dutch and the French statements of the gut feelings criteria. The linguistic validation procedures followed allowed us to expand the concept to include Slavic languages such as Polish. We assume that the utility positive diagnosis of pulmonary embolism after analyzing qualitative interviews of GPs. The GFQ was also used in a prospective study aiming at calculating the accuracy of the sense of alarm when facing a thoracic pain and a dyspnea at GP's office. Feeling a sense of alarm in this situation drove the GPs to the diagnosis of a severe disease 2 times more than without.

The sense of alarm can be seen as a complementary tool for learning how to prevent error in Comment "Ruling out pulmonary embolism in primary care: comparison of the diagnostic performance of "gestalt" and the wells rule"

Comment published online June 24, 2016, on Ann Fam Med. Link: http://www.annfammed.org/content/14/3/227.long/reply#annalsfm_el_29874

What about GPs' Gut Feelings when facing dyspnoea and thoracic pain?

In their article "Ruling Out Pulmonary Embolism in Primary Care: Comparison of the Diagnostic Performance of "Gestalt" and the Wells Rule", Janneke M.T. Hendriksen et al. compared the accuracy of "Gestalt" and the Wells rule for ruling out Pulmonary embolism (PE) in suspected cases in primary care [1]. Combined with d-dimer testing, both Gestalt, which uses a cut off of less than 20%, and the Wells rule, which uses a score of 4 or lower, are safe for ruling out PE in primary care. The number of patients who need to be referred for further testing was substantially lower when using the Wells rule instead of gestalt probability (efficiency = 45% vs 25%), as well as when using the stepped approach, that is combining gestalt, the Wells rule and D-dimer testing (efficiency = 47%). This conclusion is certainly helpful for daily practice in primary care.

However, what do the authors understand by gestalt, since a clear definition is missing in the article? They described gestalt as an "implicit physician's estimate" and asked the GP participants to rate an estimated probability of PE being present using a visual analogue scale with a range from 0% to 100% for consecutive adult patients seeking care with symptoms raising suspicion of PE. What they called "gestalt measurement" was in fact the assessment with a Visual Analogic Scale of the probability of one pre-defined pathology. To our view, the authors measured the same diagnostic process with gestalt scale and item 2 of the Wells score, i.e. that there is "no alternative diagnosis" than PE with a high impact on the final score.

Moreover, for the theorists who developed the concept, gestalt is a holistic top down approach with pattern recognition as opposed to the atomistic approach where each element is individualized [2,3].

In fact, before using any prediction rule oriented toward a particular diagnosis, the GP should have some suspicion of PE and it is precisely this initial stage which is unclear.

The major point is the perspective of the PE diagnosis directly put to the GPs. What is crucial in daily practice? Before using any prediction rule oriented towards this particular diagnosis, the GP should think about PE or have some suspicion of PE when faced with patient's symptoms and it is precisely this initial stage which is unclear. Which elements drive the practitioner to suspect PE in the first place? In our qualitative study, the main determinants of PE suspicion were the absence of indicative clinical signs for diagnoses other than PE, a sudden change in the condition of the patient, the GP's experience of previously failing to diagnose PE and a sense of alarm preventing diagnostic error in missing PE [4]. We think that gut feelings, and the sense of alarm in particular, do intervene at an early stage in diagnostic reasoning. This occurs earlier than Gestalt as it was described in this study. The sense of alarm is an uneasy feeling about a patient's health status, even though he/she has found no specific indications yet [5]. So, according to the definition of Gestalt theory, the sense of alarm is a holistic approach to the case, leading to a prime hypothesis. Gestalt as an estimation of probability of a PE diagnosis is used at the second stage: the GP is already considering PEhe did recognize the pattern of PE after asking questions to the patient-and focused on this one diagnosis, as a reflective process following a possible sense of alarm. The Wells rule intervenes as a third step in diagnostic reasoning: when considering this hypothesis, which individual elements consolidate or contradict the PE diagnosis?

We consider the study to be important in diagnostic reasoning within the process of PE diagnosis. But it did not address the crucial question of the suspicion of PE, at an early stage, in daily primary care practice. Gut feelings act as a compass steering the practitioner through the diagnostic process. It may prevent the GP from too early excluding an important working hypothesis. A holistic approach to diagnosis seems to be more sensitive than specific but, at this stage of the diagnostic reasoning process, it is important not to lose sight of hidden diagnoses too early. Gut feelings and then Gestalt could prevent diagnostic error at an early stage in the diagnostic process. We want to know the contribution of the gut feelings in that process. So we designed a study aiming to calculate the diagnostic test accuracy of the sense of alarm when applied to dyspnoea and chest pain, using the gut feelings questionnaire [4,6].

The findings of this study complete the description of the sense of alarm by contributing an essential quantitative component. We consider the study to be important in diagnostic reasoning within the process of serious infection diagnosis, but their article raises some points we want to discuss.

The authors did not detail to what concept or definition they were referring to, when using the expression "gut feeling". In the questionnaire the participants had to answer the question "How important were the following patient assessment aspects for the decision to refer?" and "a gut feeling" was one of the answer possibilities. We suppose the authors did use the Dutch wording "niet pluis" in their questionnaire as described by Stolper et al. as a sense of alarm? 1,2 . The sense of alarm is an uneasy feeling about a patient's health status, even though he/she has found no specific indications yet 2 . Nevertheless their study shows how non-analytical reasoning plays an important role in the diagnostic process of Dutch GPs facing a possible serious infection in adult patients. GPs all over Europe will recognise this gut feeling 4 . Besides this was already proven in at least one other prospective study with Flemish GPs facing a serious infection in children 5 . Van den Bruel et al. found that a GP's gut feeling "that something was wrong" with the patient substantially increased the risk of serious illness (likelihood ratio 25.5, 95% confidence interval 7.9 to 82.0)".

From their study, we do not know how the gut feelings steered the GPs' diagnostic reasoning. How was it linked to the "general appearance" which was another item of the questionnaire?

What were the cues that made the gut feeling emerging? These questions are essential to get better insight into how gut feelings arise and influence the diagnostic process. We agree with the necessity of studying the predictive value of the sense of alarm 6 but the authors made the whole diagnostic reasoning process appearing as if it was a sum of not-connected elements instead of an interacting mix of analytic and non-analytic tracks 7 Ressentir et tenir compte de ce sens de l'alarme a permis de suspecter une pathologie grave 2 fois plus fréquemment que s'il était absent. Le sens de l'alarme peut être considéré comme un outil de prévention de l'erreur en soins primaires. C'est le premier modèle de prévention de l'erreur associant la prise de décision à un stade aussi précoce du raisonnement.

Title :

The sense of alarm as a tool preventing error in primary care. Keywords : General Practice, Diagnosis, Clinical Decision Making, Intuition, Uncertainty, Gut Feelings.

Abstract :

The gut feelings criteria have been formulated from the GPs' descriptions of their own practices. The 'sense of reassurance' and the 'sense of alarm' constitute a dynamic element in a GP's diagnostic process, helping to commute between nonanalytical and analytical diagnostic reasoning. The Gut Feelings Questionnaire was translated into French, German and Polish languages following a standardized procedure of linguistic validation. The GFQ was then tested in real practice settings during office hours to check its feasibility in Belgium, France and The Netherlands. The internal consistency was high (Cronbach's alpha = 0.88). The sense of alarm was identified as one major factor conducting to the positive diagnosis of pulmonary embolism after analyzing qualitative interviews of GPs. The GFQ was also used in a prospective study aiming at calculating the accuracy of the sense of alarm when facing a thoracic pain and a dyspnea at GP's office. Feeling a sense of alarm in this situation drove the GPs to the diagnosis of a serious disease two times more than without. The sense of alarm can be seen as a complementary tool for learning how to prevent error in primary care. It is the first model where error prevention is associated with decision making at a very early stage in general practice. Further research concerning cancer suspicion and teaching the gut feelings should be promoted.