Abstract
Background Consultations in primary care for allergies are common. It can be difficult to differentiate between IgE-mediated (atopic) symptoms — which respond to allergen-specific interventions — and those that are non-atopic, without performing objective tests that are largely unavailable in UK general practice.
Aim To develop and test a screening tool that can accurately discriminate between atopic and non-atopic individuals.
Design and setting A validation study that took place in 2012 in adult volunteers aged >16 years in Scotland.
Method A questionnaire screening tool was developed using questions from a large cohort study and through consultation with experts. Participants answered the questions and had skin prick tests for four aeroallergens (house dust mite, cat, dog, and mixed grasses). Participants were classified as atopic if any average wheal diameter was ≥3 mm bigger than the negative control. Sensitivity, specificity, and positive and negative predictive values of individual and combinations of questions were calculated.
Results In all, 143 participants completed the questionnaire and underwent skin prick testing. Of these, 81 (56.6%) were atopic. Negative predictive values for the individual questions ranged from 48.2% (55 not atopic out of 114 negative answers) to 72.0% (18/25). An optimum combination of four questions was identified, in which a negative answer to all four questions was reported by 24 participants, 21 (87.5%) of whom were not atopic.
Conclusion The authors have identified a set of questions that correctly predict negative skin prick tests to common aeroallergens 88% of the time. These may be useful to exclude patients who do not warrant further investigation and who can reliably be advised that allergen avoidance is neither necessary nor helpful.
INTRODUCTION
Allergic (IgE-mediated) disorders as a whole are responsible for substantial morbidity, healthcare utilisation, and costs to the NHS.1,2 In addition, a significant number of people mistakenly believe they have allergies and use both NHS and their own time and resources pursuing unnecessary investigations (for example, conventional skin prick testing [SPT] and specific IgE blood testing, both of which are associated with a high level of false positives),3 and in pursuing alternative allergy testing (such as hair analysis or kinesiology, which have no scientific basis). As a result they may unnecessarily avoid exposure to presumed triggers.4
However, it can often be difficult to differentiate between allergic (that is, IgE-mediated) and non-allergic (non-IgE-mediated) symptoms. To date, the only way of identifying non-atopic status (defined as negative allergy tests to one or more airborne allergens and independent of clinical symptoms) is to carry out SPT or blood tests for the presence of specific IgE. However, these diagnostic tests are expensive (in the case of blood tests) and/or are often unavailable in the primary care setting (SPT).5 Furthermore, it is difficult for generalist healthcare professionals and people buying over-the-counter blood-testing kits to interpret the results because of the high rate of false positives.
Food allergy is a less common manifestation of IgE-mediated disease. It is commonly overestimated, particularly among parents, who often attribute childhood skin rashes (which may be virally induced) to food allergy. IgE-mediated food allergy is uncommon in those who are non-atopic (that is, not sensitised to common airborne allergens as shown by negative skin prick/specific IgE tests to those allergens). In a 2005 study, less than 0.2% of children who were non-atopic went on to test positive to one of 12 food allergens.6 In adults, 10–12% of the general adult population think they have some type of food allergy or intolerance,7,8 even though IgE-mediated food allergy can only be confirmed in 1–2%.6 This represents a fivefold overestimation of food allergy, which is likely to have significant cost and societal implications.1,9
A screening tool that excludes allergy and allows the general public, nurses, or doctors to identify people who are non-atopic could bring significant resource and health benefits. A simple, inexpensive, non-invasive method of screening has the potential to reduce non-essential testing and referral (and therefore healthcare use and costs to the NHS), and to reassure patients about the safety of continued exposure to allergens they wrongly suspect of causing their symptoms. Similar studies using a diagnostic tool to exclude disease in other medical fields include the use of a D-dimer test in patients with suspected venous thromboembolism10,11 and the measurement of serum levels of procalcitonin as a marker for bloodstream infections.12
How this fits in
The purpose of this study was to identify simple questions that can accurately predict the absence of atopy without the need for expensive and time-consuming objective tests, for example, skin prick tests or allergen-specific IgE blood tests. These questions can now be developed into a screening tool to be used by healthcare professionals and the general public to exclude allergy. This will prevent unnecessary avoidance of allergens, and could also be useful in excluding an allergic basis for adverse reactions to food, which is extremely uncommon in those who are non-atopic.
Data from a large paediatric cohort study13 suggest that approximately five to 10 questions from a patient’s clinical history may accurately predict non-atopic status, but this requires testing in clinical practice to confirm utility. The aim of this study was therefore to investigate whether key clinical questions could accurately identify patients who are non-atopic.
METHOD
Participants
Adults aged >16 years registered with general practices, or members of the general public, in Lothian, Scotland, were invited to participate in this study. It was performed in 2012. The exclusion criteria were:
<16 years old;
pregnancy;
uncontrolled asthma;
a previous history of anaphylaxis;
antihistamines taken in the previous 48 hours; and
unwilling or unable to give informed consent.
Recruitment
The study was advertised in general practices, in liaison with the Scottish Primary Care Research Network. Posters were displayed in the waiting areas of the practices, and flyers were handed out to those who expressed an interest. Flyers were also sent to patients attending designated asthma clinics. E-mail, posters, and websites within the University of Edinburgh were used to inform the general public. Potential participants were asked to contact the researcher by phone or e-mail and were sent the participant information sheet to read before their appointment. Participants were asked to book an appointment with the researcher for clinics at their GP surgery or in the clinical research facility at the Royal Infirmary of Edinburgh. Consent forms were completed and exclusion criteria checked at the beginning of the appointment. Permission was obtained for the results of the skin prick testing to be sent to the participant’s GP.
Questionnaire development (index test)
The authors identified five questions from the Ashford Birth Cohort Study13 that had the potential to accurately predict non-atopic status, but which had not been tested in clinical practice, and used them as the basis for the questionnaire (Box 1). Cullinan and colleagues (2004)13 defined atopy as a positive skin prick test (≥3 mm) to at least one of mixed grasses, cat fur, and house dust mite. Parents in the Ashford Cohort answered the five allergy questions, and parents and children were skin prick tested. The number of people who were non-atopic in relation to each negative response ranged from 63 to 77%; that is, if a parent reported that they had no history of hayfever then they had a negative allergy test to grass 77% of the time (high negative predictive value). The authors amended the wording of these questions and supplemented them with six additional questions, identified through consultation with experienced allergy clinicians, which also had the potential to discriminate between allergy and non-allergy. Five allergy clinicians were sent a short summary of the study prior to the consultation (Appendix 1) and asked to identify any additional questions that they felt were able to discriminate between atopy and non-atopy in a person presenting with suspected allergy. All suggestions were included, and the final 11 questions are shown in Box 2.
Ashford Birth Cohort Study screening questions
Do you have a personal history of hayfever?
Do you have a personal history of asthma?
Do you have a personal history of eczema or asthma as a baby (age <2 years)?
Do you have a personal history of other allergy?
Do any of your parents or siblings have a history of hayfever?
Questions used in screening tool
Are you aged <40 years?
Do you have, or have you ever had, hayfever?
Do any of your parents or siblings (brothers or sisters) have, or have they ever had, hayfever?
Do you have, or have you ever had, asthma?
Did you ever have eczema or asthma as a baby (aged <2 years)?
Do you have, or have you ever had, any other allergy?
Do you ever have any symptoms of itch or sneeze?
Do your allergy symptoms vary when you go from place to place (for example, on holiday)?
Is there a specific trigger that always sets off your allergy symptoms?
Do your allergy symptoms start within 30 minutes of being exposed to a specific trigger?
Do your allergy symptoms improve after treatment with antihistamines?
Questionnaire completion
Consenting participants answered the resulting 11-question screening tool relating to their atopic status. Data were recorded anonymously on data collection sheets. Clinical information and index test results were not available to the assessors of the reference standard.
Skin prick testing (reference standard)
Skin prick testing was performed using positive (histamine dihydrochloride) and negative (allergen diluents) controls, mixed grass, house dust mite (Dermatophagoides pteronyssinus), dog, and cat allergens on the volar aspect of the forearm according to a standardised technique using individual sterile lancets. Skin wheal diameter (mm) was measured after 15 minutes with a measuring grid. Positive responses to allergen were defined as mean wheal sizes ≥3 mm bigger than the negative control. Negative responses were mean wheal sizes that were the same size or smaller than the negative control. Non-atopic status was defined as negative responses to all four skin prick tests. Atopic status was defined as a positive response to one or more aeroallergen.
Data analysis
Sensitivity, specificity, and positive and negative predictive values of the individual questions and combinations of questions were calculated using positive skin prick test results as the gold standard for atopic status. Confidence intervals (CI = 95%) were calculated for the sensitivity and specificity using standard methods for proportions. Positive and negative predictive values were calculated as the probability that the questionnaire responses agreed with the skin prick test results, where the positive predictive value (PPV) was the proportion of patients with a positive history who were atopic, and the negative predictive value (NPV) was the proportion of patients with a negative history who were not atopic.
Logistic regression techniques were used to identify the combination of questions that demonstrated the strongest association with non-atopic status. Responses to all questions were considered individually, then a multivariate model was constructed by including the questions with the strongest associations one by one until no other questions significantly contributed to the fit of the final model. The contribution of each question was evaluated using likelihood ratio tests. Once these questions were identified it was possible to calculate the sensitivity, specificity, and positive and negative predictive values for this combination of questions. Participants were defined as questionnaire positive if at least one response to these questions was positive, and questionnaire negative if all responses were negative. All statistical analyses were undertaken using Stata (version 13).
Sample size
Based on a sample size of 150, with 95% CIs of ± 4.8%, specificity would be estimated at 90%. The sample size was calculated using an online statistical tool.14
RESULTS
In all, 143 participants completed the questionnaire and underwent skin prick tests. No adverse reactions occurred. There were no indeterminate index or reference standard results or missing data. The mean age of participants was 41.1 years (range 18.8–84.9) and 76.9% were female. A total of 81 participants (56.6%) were atopic. The rates for each allergen were 62/143 (43.4%) for grass, 67/143 (46.9%) for house dust mite, 49/143 (34.3%) for cat, and 33/143 (23.1%) for dog. Individual question analysis is shown in Table 1.
The authors explored any correlation between questions and, although many of the questions were related to each other, none was in perfect agreement. Logistic regression techniques identified Questions 2, 8, 3, and 9 as being independently associated with the risk factor of being non-atopic (Table 2).
A negative response to Question 8 was the most associated with being non-atopic (that is, negative skin tests). After adjusting for the other questions, patients were four times more likely to have negative test results if they answered ‘No’ to that question. Table 3 shows the relationship between the responses to these four questions and atopic status.
Comparison of the questionnaires and skin tests gave a sensitivity of 96.3% (78/81, 95% CI = 89.6 to 99.2), specificity of 33.9% (21/62, 95% CI = 22.3 to 47.0), PPV of 65.6% (78/119, 95% CI = 56.3 to 74.0), and NPV of 87.5% (21/24, 95% CI = 67.6 to 97.3).
This shows that the combination of Questions 2, 8, 3, and 9 had a reasonably high NPV. Most (87.5%) of those who gave negative responses to all the questions were non-atopic (had negative skin tests).
DISCUSSION
Summary
The authors were able to identify four questions that were reasonably predictive of non-atopic status in patients with suspected allergy. This has the potential to be useful in differentiating between IgE-mediated and non-IgE-mediated symptoms (and so to drive treatment choices and avoidance advice) in primary care, where diagnostic tests are not routinely available.
Strengths and limitations
The authors were able to recruit members of general practices and the general public for this study, and almost achieved the target sample size of 150. The screening tool is intended for use by healthcare professionals in primary care and the general population, which is where it was tested,15 and was compared with the best gold standard available. The authors’ definition of atopy, a positive skin prick test to common (country- and climate-specific) aeroallergens, has been used in other studies,13 and the choice of skin prick testing as a gold standard is safe and feasible for aeroallergens.16 However, this definition of atopy gave a prevalence of 57% in the sample population, which is high compared with other similar studies, presumably because there was a higher motivation to participate among people who are atopic.17 This may have resulted in loss of power to detect important questions that predict non-atopy.
A further limitation is the sample size. The numbers were small in the detailed analysis, where combinations of questions were examined, and increasing the sample population would have contributed more data to each combination of positive/negative results to the skin prick testing and the questions, potentially increasing the negative predictive values and improving precision.
Due to the nature of the recruitment — advertising with posters and leaflets — the authors were unable to record how many people saw the information but declined to take part, thus preventing any consideration of how representative the findings are.
Comparison with existing literature
Previous work in this field has mainly focused on developing questions that can accurately predict sensitisation to a suspected allergen (that is, whether positive answers to clinical questions predict positive skin prick or specific IgE test results).17–19 The focus of this current study, however, was different. The authors sought to identify simple questions for which negative answers could predict negative skin tests in a general population, with a view to being able to assess atopic status without the need for a formal diagnostic test.
This combination of questions compares well with other studies that have reported an NPV for atopy, including the ALATOP20 in vitro multispecific IgE test, which reported sensitivity of 89.57%, specificity of 98.06%, PPV of 98.65%, and NPV of 85.59%. Similarly, in a study looking at the accuracy of allergy skin prick tests,17 SPT was predicted to be positive in 42.6% of cases and was actually positive in 36.1%. In SPT results with a cutoff value of 3 mm, prediction sensitivity was 77%, specificity was 65.3%, PPV was 65%, and NPV was 86%. In studies of different diagnostic tests, Wang and colleagues11 reported a slightly lower sensitivity of 64% and a higher NPV of 94%, concluding that the D-dimer assay may have a role in tailoring treatment to optimise prevention of venous thromboembolism. Similar results were found for the predictive value of procalcitonin in excluding bloodstream infections and managing antibiotic usage (83% sensitivity and 94% NPV).12
When a test has a high sensitivity, a negative test rules out the diagnosis,21 and this study reports a high sensitivity (96%) and NPV (88%). The authors are therefore fairly confident that this result using the four questions has a high NPV (as it is almost 90%), although the numbers were relatively small.
Implications for research and practice
The authors were able to identify four questions that were predictive of non-atopic status. The results provide useful data for the development of a screening tool for non-atopic status in people with suspected allergy, although the questions need further validation in a larger, independent population of consecutively enrolled patients. This would increase the numbers in the combination analysis, and increase precision.
The screening tool could then be confidently used by healthcare professionals or patients to accurately predict non-atopic status.
Acknowledgments
The authors would like to thank the participants who generously contributed their time to undergo allergy testing. They would also like to thank the healthcare practitioners involved in conducting this study, particularly the clinical research facility nurses and general practice nurses. The authors are also grateful to the lay representatives for their support and advice.
Appendix 1. Study summary for allergy specialists
Developing and testing a screening tool to accurately predict non-atopic status in patients with suspected allergy
Background
Disorders such as asthma, rhinitis, and urticaria are extremely common in Scotland, potentially affecting up to one in three of the population.1 The commonest manifestations are respiratory and dermatological, and are often caused by exposure to allergens such as pollens and house dust mites, mediated through the production of allergen-specific IgE antibodies and subsequent histamine release. Allergic (IgE-mediated) disorders as a whole are responsible for substantial morbidity, healthcare utilisation (>4% of GP consultations and 1.5% of hospital admissions in Scotland are for allergic diseases) and costs to the NHS.1 In addition, there are significant numbers of people who mistakenly believe they are allergic and who utilise both NHS and their own time and resources pursuing unnecessary investigations (for example, conventional skin prick testing (SPT) and specific IgE blood testing, which are both associated with a high level of false positives) and pursuing alternative allergy testing (for example, hair analysis or kinesiology, which have no scientific basis), as a result of which they may unnecessarily avoid exposure to presumed triggers.4 For example, 10–12% of the general adult population think they have some type of food allergy or intolerance,7,8 even though IgE-mediated food allergy can only be identified in 1–2%.6 This represents a fivefold overestimation of food allergy, which is likely to have significant cost and societal implications.1,9
However, it can often be difficult to differentiate between allergic (that is, IgE-mediated) and non-allergic (that is, non IgE-mediated) symptoms. To date, the only way of identifying non-atopic status (defined as negative allergy tests to one or more airborne allergens and independent of clinical symptoms) is to carry out SPT or blood tests for the presence of specific IgE. These diagnostic tests are, however, expensive and often unavailable in the primary care setting, and are furthermore difficult to interpret by healthcare professionals and also people buying blood-testing kits over-the-counter. Negative allergy tests alone are a relatively accurate predictor of non-allergy,22 although it is not clear at present whether a negative allergy history accurately predicts negative allergy tests.
Outline of this project
This study will investigate whether key clinical questions can accurately identify patients who are non-atopic. We propose to develop an instrument that is comprised of validated questions, which can identify those patients for whom an allergy test is so likely to be negative that it is not worth doing (that is, a very high negative predictive value). This has the potential to be useful both for clinicians and patients in streamlining care in a cost-effective manner.
Collaboration with colleagues from Imperial College London has enabled us to analyse unpublished data from the Ashford Birth Cohort,13 providing useful information about parental and child allergy status. Parents answered the following five allergy questions and parents and children were skin prick tested:
Do you have a personal history of hayfever?
Do you have a personal history of asthma?
Do you have a personal history of eczema or asthma as a baby (age <2 years)?
Do you have a personal history of other allergy?
Do any of your parents or siblings have a history of hayfever?
Atopy was defined as a positive SPT (≥3 mm) to at least one of mixed grasses, cat fur, and house dust mite. The numbers of people being non-atopic in relation to each negative response ranged from 63 to 77%; that is, if a parent said that they had no history of hayfever then they had a negative allergy test to grass 77% of the time (high negative predictive value). A negative response to all the questions was associated with a high rate of negative allergy tests in adults (83%). Results from the same study in children were similar (89%), although numbers were smaller. High specificity (ideally 95%) is required for the tool to be useful in clinical practice.
These five questions will be used as the basis for the screening questionnaire. Additionally, we will identify any other questions that, according to experienced allergy clinicians, discriminate between allergy and non-allergy, and test them with the questions identified above to create the combination with the highest negative predictive value.
_________________________________________________________________________________________________________________________
We now wish to find out which questions, in addition to those listed above, discriminate between atopy and non-atopy in someone presenting with suspected allergy.
Name: __________________________________________________________
Position: __________________________________________________________
Questions:
1. __________________________________________________________
2. __________________________________________________________
3. __________________________________________________________
Notes
Funding
This study was funded by the Chief Scientist Office, grant reference number: CZG 2 576. Jessica Harris was supported by the National Institute for Health Research Bristol Cardiovascular Biomedical Research Unit.
Ethical approval
Ethical approval was obtained from the South East Scotland Research Ethics Committee 02 (SES REC 02), reference 12/SS/0094. Research and development management approval was obtained from NHS Lothian. Amendment approval to extend the recruitment to the general public was obtained from SES REC 02.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors have declared no competing interests.
Discuss this article
Contribute and read comments about this article: bjgp.org/letters
- Received May 23, 2016.
- Revision requested June 21, 2016.
- Accepted August 23, 2016.
- © British Journal of General Practice 2017