Evaluation of an interactive e-learning module “Toothache Clinic” for delivering information on dental pain: a prospective cohort study

This multivariate, prospective, monocentric, observational study evaluates the experimental application of an interactive e-learning module on the topic of dental pain, which deals with the content of interdisciplinary diagnostics and therapy of dental pain. The module (“Toothache Clinic”) was offered to dentistry students during the first, second and fourth clinical semesters. A total of n = 138 students took part.

The questionnaire used in the study comprised 43 items (assigned to three domains) as well as questions on academic progress, gender, information technology (IT) background and module evaluation. The explorative factor analysis resulted in a reliability value (Cronbach’s Alpha) of 0.94. The results show a good rating for the module of 2.09 ± 0.55 (1 = excellent to 6 = unsatisfactory) across all groups. 94.17% of the volunteers would recommend the module to their fellow students. With a rating of 1.96 ± 0.51 (1 = excellent to 6 = unsatisfactory), women rated the module significantly better than men who rated it 2.27± 0.67 (p = 0.016). There were no significant differences in the ratings of the module between the three semesters. Furthermore, the results show no significant correlation between module assessment and prior information concerning technology education. In summary, it can be said that the students rated the e-learning module (“Toothache Clinic”) positively, regardless of their IT (information technology) knowledge and academic progress. Keywords: pain medicine, dental, e-learning, gender, education.

Introduction

The diagnosis and treatment of dental pain is one of the everyday and nevertheless most important tasks of both dentists and dental students [1-5].

The suffering of a patient with dental pain is usually considerable, which is why the importance of a correct diagnosis together with appropriate therapy cannot be overestimated. The acquisition of competence in this field of dentistry is of great importance and in only very few cases can actually take place through direct patient contact during the course of studies. A solid education before the first patient contact is therefore important for all participants [6-9]. It is difficult to convey the necessary skills in the form of traditional lectures, because clinical skills, such as the individual preparation of a pain anamnesis, and even the implementation of a special therapy, depend on numerous factors. In this context, the worldwide experience of recent years with e-learning, especially in the field of medicine, has shown that interactive, case-based patient simulations are an excellent tool for training clinical skills such as the acquisition of diagnoses from findings [10-14]. In addition, there are further advantages of e-learning courses which, with the appropriate concepts and didactics, can impart a similar amount of knowledge as conventional classroom teaching [15]. This is both time and cost effective, as well as much more variable in the presentation of the teaching contents, so that different types of learners can be addressed in the best possible way [13,16,17]. Finally, the German Rectors’ Conference and the Council of Science and Humanities call for the sustainable establishment of e-learning in teaching [18,19]. In doing so, they are following the efforts of the European Union, which has been promoting the development of e-learning in Europe for several years with various programmes. After the conception of an electronic module on the issue of dental pain (“Toothache Clinic”), which is freely available on the Internet, we were interested in the following research questions:

1. How do the students rate the e-learning module (“Toothache Clinic”)?

2. Do students from different semesters evaluate the e-learning module (“Toothache Clinic”) differently?

3. Does students´ gender (male or female) have an influence on the assessment of the e-learning module (“Toothache Clinic”)?

4. Does the students’ previous IT (information technology) knowledge correlate with the evaluation of the e-learning module (“Toothache Clinic”)?

Materials and methods

The e-learning module

The e-learning module (“Toothache Clinic”) was created with an open source authoring tool called “Webkit Freiburg”, licensed under “Creative Commons”, which is based on the “Adobe Flash” technology [20]. It consists of a total of n = 83 pages, whose contents can be edited interactively using various functions (tips, feedback, dropdown, radio button, checkbox, free input, magnifying glass, image sequence, drag and drop). Multimedia elements such as anamnesis sheets, findings sheets, extra- and intraoral photographs, X-ray images, videos, animations, concept maps) optimize the presentation of the individual didactic contents. A total of five patient cases from different areas of dentistry (case 1: conservative dentistry, case 2: prosthodontics, case 3: periodontology, case 4: oral surgery and case 5: orthodontics) are integrated, all of which have a common characteristic, namely the presence of acute dental pain. The respective learning objectives (n = 30 in total) are based on the specifications of the “Swiss learning objectives catalogue for dentists” from 2008 [21] and the ADEE (Association of Dental Education in Europe) specifications [22, 23]. The patients were recruited from the patient courses of the University Dental Clinic. Their consent to the use of their findings and illustrations was obtained previously. The page order is fixed, but pages can be skipped via the menu, and the user can also jump back to previous pages. The given sequence begins after the initial examination, a case vignette and the display of the case-specific learning objectives with the general anamnesis (step 1), followed by the pain anamnesis (step 2) and the clinical findings (step 3). This is followed by diagnosis (step 4) and therapy (step 5). The case ends with the page “Behandlungsende” (end of treatment) (step 6) and the output test.

The questionnaire

The questionnaire designed according to the “Ulmer Qualitätskriterien für Lernprogramme in der Medizin” (Ulm quality criteria for learning programmes in medicine) was divided into several sections [5,9,24]. After the “Code zum Wiedererkennen” (code for recognition), “Allgemeine Angaben” (general information) was queried: age, gender, number of semesters, Abitur grade, previous technical knowledge, vocational training. In the section “Spezielle Fragen” (special questions) an assessment of the entire module and information on the processing time as well as the time of day were requested. The individual cases were assessed in sections four to eight. Further sections included “Bedienbarkeit und Technik” (handling and technical aspects) with 14 items, “Inhalt und Funktionsumfang” (content and functional range) with 8 items, as well as “Didaktik und Eignung für die Ausbildung” (didactics and educational value) with 21 items. The evaluation of the items was assigned by a Likert scale from 0 (“does not apply at all”) to 3 (“applies completely”). Free text comments were also possible.

Blended learning

The “Toothache Clinic” is part of the integrated Frankfurt dental medicine e-learning strategy “FranZI”. Integrated into the curriculum, the e-learning contents are provided in class. Embedded in a blended learning scenario, a mixture of e-learning and classroom teaching arises. For the application of e-learning, students have four computer workstations with Internet access in the library of the dental clinic as well as four further computer workstations with Internet access in the so-called “LernLab” at their disposal. The students of the first clinical semester also have the possibility to use the Internet at their treatment units in the course room (n = 24 computer workstations) of the phantom course for conservative dentistry. Further opportunities for online and offline computer work are provided by the main medical library on the campus of the University Hospital as well as the WLAN network available in the dental clinic, provided that a portable computer is available.

Study period and setting

For this study the data of three parallel semesters (first, second and fourth clinical semester) during the winter semester 2010/11 were collected (Table 1). The ethics application of the monocentric and self-initiated study was assigned the reference number 154/11 by the ethics commission of the Faculty of Medicine of the Goethe University Frankfurt am Main.

Preparatory measures by those learning

At the beginning of the 154-day evaluation phase, the students received a 15-minute introduction to the use of the module with the following contents: Announcement of the URL of the e-learning module, assurance of complete anonymity of the incoming and outgoing tests, reference to the concept maps, reference to any increased loading times, indication of an e-mail address for technical support, explanation and issue of the evaluation forms and user names, announcement of the deadline, enquiry of the e-mail addresses of the semester speakers, answers to any technical or procedural questions the students may have.

Preparatory measures by those teaching

The teaching staff of the department (conservative dentistry) was informed about the application of the elearning module in blended learning scenario. In addition, they were given access data to enable them to use the module.

Statistics

For the statistical analysis of the results, the Wilcoxon test with a significance level of p ≤ 0.05 was applied. The Wilcoxon test was performed using the statistical analysis package “WinStat for Excel”, version 2003.1.

Results

After returning the evaluation forms, a population of n = 104 students remained (Table 2).

Response rate

The response rate for all semesters was 74%. The first clinical semester had a response rate of 94%, the second clinical semester of 75% and the fourth of 58%. The average drop-out rate was 25%.

Exploratory factor analysis

The explorative factor analysis (main axis factor analysis) showed that there are three factors. This is in line with the theoretical construct according to which the evaluation sheet was drawn up. The reliability values of the three factors are shown in table 3. The entire questionnaire received a reliability value of 0.94.

Non-group-specific results

All test person groups gave the five cases an average school grade (1 = excellent to 6 = unsatisfactory) of 2.09 ± 0.55 (Table 4). On average, the first case (paediatric dentistry) was rated significantly better with 2.04 (± 0.57) than case two (prosthetics) with 2.18 (± 0.68) and case five (orthodontics) with 2.29 (± 0.80).

81% of the subjects found the e-learning module to be helpful in exam preparation. 93% rated the module as helpful in preparing for clinical courses. 84% found it helpful as emergency service preparation. 94% of those surveyed would recommend it to their fellow students.

Gender-specific results

Men rate their own previous IT knowledge significantly better (2.14 ± 0.87) than women (2.52 ± 0.76) at a pvalue of 0.038. The other specific questions and the three factors were not answered significantly differently. The five cases were rated significantly better overall by women (school grade 1.96) while all cases averaged 2.27 (p = 0.016) for men (Table 5). In the individual assessment, the first, third and fifth cases were rated significantly better by women than cases two and four.

Semester-specific results

There are no significant semester-specific differences in the overall and individual assessment of the five cases, nor in the previous IT knowledge, nor in the three factors and all other specific questions.

Correlation of previous IT knowledge

The previous IT knowledge does not correlate in any way with the evaluation of the module.

Discussion

The interdisciplinary concept and the division of the module into five cases are mostly rated positively in the free text comments. One working group recommends the use of e-learning modules as an optimal instrument for conveying interdisciplinary content [11]. This was confirmed by the good acceptance of the “Zahnschmerzambulanz: Advanced” (toothache clinic: advanced).

On average, the three factors were rated slightly better by the women, but not significantly so. One striking deviation concerns items 10.4 (“Die medizinischen Lernziele sind deutlich ausgewiesen” / “The medical learning objectives are clearly stated”) and 11.6 (“Der Einsatz multimedialer Elemente fördert das Verständnis vom angebotenen Lerninhalt” / “The use of multimedia elements promotes understanding of the learning content offered”), which were rated significantly better by women than by men. This could indicate that women have a greater “natural” motivation to e-learning in virtual patient cases, which could be explained by differences in learning behaviour between the genders [25]. Men usually have better spatial perception than women. The latter therefore find e-learning an appropriate learning medium, as it visualises content in different ways and thus supports the learning process [13, 16, 17]. The good female evaluation shows that this seems to have been successful in this module. However, it should also be questioned how e-learning modules can also be optimised to meet male needs. For example, this could be achieved by clearly demonstrating the practical benefits that elearning can have for them [11].

A questionnaire was used for the evaluation, which was examined with regard to the criteria objectivity, validity and reliability, standardised in structure and content, and is thus comparable between different universities. It is based on the “Ulmer Qualitätskriterienkatalog für medizinische Lernprogramme” (Ulm quality criteria for learning programmes in medicine) [24]. The reliability of such a questionnaire is measured by means of the Cronbach Alpha [26]. At over 0.9 for the entire questionnaire, the reliability of the used questionnaire calculated by Cronbach’s Alpha – as a prerequisite for its validity – is excellent. It can thus be regarded as very reliable. Other studies also evaluate their questionnaires using Cronbach’s Alpha, which makes the reliability directly comparable as required by a working group [1,24].

The results confirm that students feel more confident in their abilities through virtual patient simulations in the clinical part of the study and therefore support the use of such modules [12]. The factors “didactics and educational value” as well as “ handling and technical aspects” were assessed between 2.03 and 2.13 on average. The students rate the module as most suitable for self-study. As indications for the further use of this module group lessons or use in class are rated as strongly below average. This could be based on the fear of students that this or similar modules would be saddled on top of the previous curriculum, which already has an enormous workload. Students may also see no meaningful indication or method of how the module could be integrated into classroom teaching and therefore reject these items. This is where one working group calls for more detailed studies on how virtual patient cases could be better integrated into and adapted to the curriculum [10].

Apart from the different levels of previous computer knowledge, which have no influence on learning success in e-learning, men and women approached the module with the same prerequisites. The poorer IT skills of the women in the self-assessment correspond to the findings of other studies. However, they have no influence on the evaluation of e-learning content [27].

Women do not consider the module more recommendable than their fellow male students. However, they tend to recommend it more frequently for preparation for clinical courses and examinations. This non-significant positive evaluation is even more pronounced in the grading of the five cases, which on average is significantly better than the evaluation by men. The overall better rating of cases by women could be related to the fact that interactive patient simulations are closer to the needs of the explorative-developing learning method of women described by Gunn than to the practical-instrumentalizing learning behaviour of men [27]. This is also supported by the fact that women generally learn more successfully with e-learning courses than men. According to one study, women tend to be more inclined towards self-directed learning behaviour, which fits very well with the voluntary and interest-based conception of this module [17]. The generally positively evaluated usability of the module seems to benefit women in particular. Furthermore, women seem to prefer interactive content more than men [28].

The homogeneous evaluation of the module by all three semesters involved, as well as the lack of correlation with their scolar (Abitur) and university (Physikum) grades, shows that the prior knowledge of the study participants had no influence on the evaluation of the module, because achievements in medical studies normally correlate with these grades [29]. This might be connected to the fact that all information necessary for the solution of the cases was kept in the module itself. The prior level of knowledge that was absolutely necessary for processing was therefore – as intended – low. However, it also indicates that not only the contents offered were evaluated, which would probably have led to greater fluctuations between the semesters and examination marks if the conditions had varied, but also didactics, conception and implementation. The positive result of the evaluation, independent of previous knowledge, thus underlines the good validity of this study.

Different levels of experience in dealing with a computer had no significant effect on the assessment of the modules. In the future, this question is likely to become more and more of a secondary concern, since IT competence correlates significantly and positively with the age group of the students, i.e. the younger students bring with them probably greater computer knowledge and skills. The development team aimed for a processing time of 30 to 45 minutes per case, based on the time for a classic school lesson. The evaluation showed that this goal was achieved with a median of 30 minutes. The duration of use of e-learning modules is rarely stated in literature but is usually between 30 and 60 minutes [28,30,31]. Nevertheless, in the free text comments students requested that in future modules the contents per case should be reduced, possibly in favour of an increase in the number of individual cases.

Internal validity (Maturation and selection / recruitment of test persons

During the twelve-week evaluation phase, various exams in the subjects of orthodontics, oral surgery, periodontology and prosthodontics were included in the three participating semesters, which were very important for the course of studies. Depending on the time in which the module was worked on, these could have resulted in a different prior educational background.

As this study was conducted within the framework of the Department of conservative dentistry, only students of the three semesters who also participate in courses at this polyclinic were included. Functionality, accessibility and framework dates were explained and demonstrated to each of the three semesters at the beginning of the study and any questions were answered. Students who did not take part in this event or were inattentive would have had different prerequisites. In order to counteract this effect, all information provided during the event was distributed several times by email to all students of the three semesters. Study participants could not be obliged to submit the evaluation forms. Therefore, it cannot be ruled out that a higher proportion of those who are fundamentally more positive about e-learning will have submitted questionnaires, which would have lead to a falsification of the results in favour of a more positive outcome. This was to be counteracted by a preferably low drop-out rate made possible through direct personal contact with the students when collecting the evaluation forms.

The multicentered implementation and evaluation of the module not only through written questionnaires but also through the storage of all user activities using a database would also be very desirable for the future. One working group found strong discrepancies in 10-15% of the evaluation forms submitted for an e-learning module compared to the actual user behaviour recorded in databases [28].

Conclusion

In summary, it can be concluded that dental students rated the pain-e-learning module “Toothache Clinic” positively, regardless of their gender, previous IT knowledge and academic progress.

1. Gerhardt-Szép S (2009) Conception, implementation and evaluation of the interactive, interdisciplinary and case-oriented eLearning course "Zahnschmerzambulanz" at the Polyclinic for Dentistry at the University Hospital Frankfurt am Main. Frankfurt am Main.
2. Ziegeler C, Wasiljeff K, May A (2018) Nondental orofacial pain in dental practices - diagnosis, therapy and self-assessment of German dentists and dental students. Eur J Pain 23: 66-71. Link: https://goo.gl/gbbSXo
3. Alsafi Z, Michelotti A, Ohrbach R, Nilner M, List T (2015) Achieved competences in temporomandibular disorders/orofacial pain: a comparison between two dental schools in Europe. Eur J Dent Educ 19: 161-168. Link: https://tinyurl.com/y79b547m   
4. Ali R, O'Sullivan DJ, Gray GB, Vowles RW, Hooper SM (2009) Teaching dental pain with and without underlying oral physiology: learning implications. J Dent Educ 73: 1090-1094. Link: https://goo.gl/8NNQtt
5. Gerhardt-Szép S, Dreher S, Rüttermann S, Weberschock T (2017) Conception and implementation of a novel E-learning module with EbM learning contents in operative dentistry. Z Evid Fortbild Qual Gesundhwes 128: 72-78.             Link: https://goo.gl/DN62b7  
6. Watt-Watson J, Lax L, Davies R, Langlois S, Oskarsson J, Raman-Wilms L (2017) The Pain Interprofessional Curriculum Design Model. Pain Med 18: 1040-1048. Link: https://goo.gl/EY9XbM
7. Borromeo GL, Trinca J (2012) Understanding of basic concepts of orofacial pain among dental students and a cohort of general dentists. Pain Med 13: 631-9. Link: https://goo.gl/MeRq58
8. Teich ST, Alonso AA, Lang L, Heima M (2015) Dental Students' Learning Experiences and Preferences Regarding Orofacial Pain: A Cross-Sectional Study. J Dent Educ 79: 1208-1214. Link: https://goo.gl/pQnH7f
9. Dreher S, Weberschock T, Giraki M, Uhse A, Parvini P, Rüttermann S, Gerhardt-Szep S (2017) Dental training in evidence-based dentistry: A replication study. J Dent Probl Solut 4: 066-071. Link: https://goo.gl/2g37eb
10. Cook DA, Triola MM (2009) Virtual patients: a critical literature review and proposed next steps. Med Educ 43: 303-311. Link: https://goo.gl/hfFwkT
11. Grijpink-van den Biggelaar K, Drop SLS, Schuwirth L (2010) Development of an e-learning portal for pediatric endocrinology: educational considerations. Horm Res Paediatr 73: 223-230. Link: https://goo.gl/cgjVRF
12. Kopp V, Stark R, Fischer MRG (2007) Fostering diagnostic competencies in medical education by implementing the approach of case-based learning through worked-out examples. GMS Z Med Ausbild 24: Doc107. Link: https://goo.gl/5Dd6VF
13. Ochsendorf F, Beschmann H, Weberschock T (2008) TIP-TOP Dermatology practical course: successful combination of classroom-learning und E-Learning. GMS Z Med Ausbild 25: Doc23. Links: https://goo.gl/5e4r71
14. El Azem A, Benington PC, Khambay BS, Ayoub AF (2014) Evaluation of an interactive multi-media device for delivering information on Le Fort I osteotomy. J Craniomaxillofac Surg 42: 885-889. Link: https://goo.gl/Zz3zYn
15. Cook DA, Levinson AJ, Garside S (2010) Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis. Acad Med 85: 909-922. Link: https://goo.gl/da6FmE
16. Neuhaus KW, Schegg R, Krastl G (2008) Integrated learning in dentistry: baseline data and first evaluation at the Dental School of Basel. Eur J Dent Educ 12: 163-169. Links: https://goo.gl/eWusy5
17. Paechter M, Fritz B, Maier B, Manhal S (2007) eSTUDY - eLearning during your studies: How do students evaluate and use eLearning? Graz 2007. Link: https://goo.gl/1wcSBE
18. Schultz E (2016). Manual guideline for higher education institutions to establish information and communication strategies.
19. Council of science (2008) Recommendations for quality improvement of teaching and studying. Berlin 2008. Link: https://goo.gl/mJqaWj
20. Toothache Clinic (2018) E-Learning module Link: https://goo.gl/UCcLu9
21. H. Bürgi, B. Rindlisbacher, Ch. Bader, R. Bloch, F. Bosman, C. Gasser, W. Gerke, J. P. Humair (2008). Swiss Catalogue of Learning Objectives for Undergraduate Medical Training, Bern 2008. 2nd edition. Link: https://tinyurl.com/ycvbx82y    
22. Field JC, Kavadella A, Szep S, Davies JR, DeLap E, Manzanares Cespedes MC (2017) The Graduating European Dentist-Domain III: Patient-Centred Care. Eur J Dent Educ 1: 18-24. Link: https://goo.gl/BhyvCT
23. Field JC, Walmsley AD, Paganelli C, McLoughlin J, Szep S, Kavadella A, Manzanares Cespedes MC, Davies JR, DeLap E, Levy G, Gallagher J, Roger-Leroi V, Cowpe JG (2017) The Graduating European Dentist: Contemporaneous Methods of Teaching, Learning and Assessment in Dental Undergraduate Education. Eur J Dent Educ 1: 28-35. Link: https://goo.gl/3cCVQF
24. Scholz W, Fassnacht U, Öchsner W, Stracke S, Waldmann U, Friedl R, Liebhardt H (2006) Ulm quality criteria catalog for medical learning programs. GMS Z Med Ausbild 23: Doc17. Link: https://goo.gl/D4CVWU
25. Mitterauer L, Frischenschlager O, Haidinger G (2007) Sex differences in study progress at Medical University of Vienna. GMS Z Med Ausbild 24: Doc111. Links: https://goo.gl/Kx93nD
26. de Vet HCW, Mokkink LB, Mosmuller DG, Terwee CB (2017) Spearman-Brown prophecy formula and Cronbach's alpha: different faces of reliability and opportunities for new applications. J Clin Epidemiol 85: 45-49. Link: https://goo.gl/2fCvFf
27. Gunn C (2003) Dominant or different? Gender issues in computer supported learning. JALN 7. Link: https://goo.gl/pb1vMi
28. McNulty JA, Sonntag B, Sinacore JM (2009) Evaluation of computer-aided instruction in a gross anatomy course: a six-year study. Anat Sci Educ 2: 2-8. Link: https://goo.gl/n26Dka
29. Syed Ali A, Schulze J, Seibert-Alves F, Gentsch S, Nürnberger F (2008) Admissions criteria and success in medical fields of study. GMS Z Med Ausbild 25: Doc35. Link: https://goo.gl/PqeiFg
30. Perryer DG, Walmsley AD, Barclay CW, Shaw L, Smith AJ (2000) Development and evaluation of a stand-alone web-based CAL program. A case study. Eur J Dent Educ 4: 118-123. Link: https://goo.gl/zQ6iyb
31. Prensky M (2000) Digital game-based learning. McGraw-Hill, New York.  Link: https://goo.gl/1sqsBV