Abstract
Background. We changed the biostatistics curriculum for our medical students and have created a course entitled “Multivariate analysis of statistical data, using the SPSS package.” Purposes. The aim of this course was to develop students’ skills in computerized data analysis, as well as enhancing their ability to read and interpret statistical data analysis in the literature. Methods. In the current study we have shown that a computer-based course for biostatistics and advanced data analysis is feasible and efficient, using course specific evaluation questionnaires. Results. Its efficacy is both subjective (our subjects felt better prepared to do their theses, as well as to read articles with advanced statistical data analysis) and objective (their knowledge of how and when to apply statistical procedures seemed to improve). Conclusions. We showed that a formal evaluative process for such a course is possible and that it enhances the learning experience both for the students and their teachers. In the current study we have shown that a computer-based course for biostatistics and advanced data analysis is feasible and efficient.
1. Introduction
Physicians need to use and interpret statistical and epidemiological data on a daily basis. Colditz and Emerson [1] found that more than 40% of 700 medical articles they reviewed utilized advanced statistical procedures, like regression MANOVA and the like. Paltiel et al. [2] believe that today’s medical decision-making requires enhanced capabilities in communication as well as data analysis. To develop these skills and those necessary to conduct and interpret clinical research, medical students would benefit from training in evidence based medicine (EBM), epidemiology, and biostatistics.
That is why the field of biostatistics has become an integral part of the medical milieu [3]. Studies have shown that physicians have a difficult time grasping statistical logic, understanding its importance relative to the treatment of patients, and conducting their own research [3]. Sterne [4] believes that the problematic issue of statistical inference, as well as how to teach it, is as old as statistics itself. The tendency of medical literature today to ask for unneeded and problematic statistical procedures is an outcome of the variability of teaching methods of these subjects. Sterne calls for the implementation of guidelines to standardize what should be taught to medical students. Medical students themselves [5, 6] understand the need to learn statistics, especially as an essential language to understand the medical literature and as a tool that helps in clinical decision-making. Colditz and Emerson [1] identified an emerging need for the development of a biostatistical core curriculum for medical students. Alas, this need has yet to be fulfilled. Clayden [7] identified the following key curriculum-related questions: who should teach this content (statisticians or clinicians), how should it be taught, and when should it be taught.
In contrast to those who advocate for adding biostatistics to medical curricula, Simpson [8] believes it is anachronistic and unrelated to the real aim of clinical decision-making. This debate is ongoing, and no side has had the final word about what and how we should teach biostatistics [9–11].
Mintz and Ostbye [12] believe that even though the biostatistics field is considered important, it is not rewarding for those who teach it to medical students. The most prevalent method of teaching biostatistics to medical students is still the lecture system, in which the main emphasis is on mathematical formulas, rather than their proper usage [13].
Looney et al. [14] found that although most medical students reported that computerized statistics would be helpful, most courses in U.S. medical schools were in the form of lectures, with little or no use of computers.
Appleton [15] believes that the emphasis in medical statistics teaching is erroneous—the courses are too long and detailed and are of no real relevance to the students. Mintz and Ostbye [12] agree with that and identified key concepts that are essential for any statistics usage. Dixon [16] also believes a core curriculum in biostatistics is feasible after he evaluated 36 different courses across the UK. Clayden [7] described several essential components: coordination between biostatistics experts and medical educators, use of computer programs, and proper evaluation formats to evaluate their success. Dixon [16] also calls for the use and evaluation of computer-based biostatistics tutoring.
Ostbye [17] created such a course in the 1980s and found it to be efficient and cost-effective. Knapp and Miller III [18] thought that teaching biostatistics to medical students with computers was both feasible and effective because of the widespread use of computers in medical practice, as well as the availability of user-friendly statistical programs. Although they studied research clinicians rather than medical students, Hutton Jr. et al. [19] found that a computer-based biostatistics course was as good as an equivalent course given as a series of lectures.
Following these guidelines and in concordance with the BGU Faculty of Health Sciences initiative to integrate theory with practice, we created a new course: Multivariate analysis of statistical data, using the SPSS package. Henshaw [20] has mapped the main challenges regarding teaching biostatistics to nurses, and we followed his footsteps in mapping the main problems regarding teaching computerized biostatistics to medical students. We believe, just as Henshaw does, that the proper way to help students become familiarized and knowledgeable in the field of biostatistics is helping them build their certainty and ability to manipulate data analysis. As do Ambrosius and Manatunga [21], we do not want or need our medical students to be trained statisticians; however, we do believe that, in order to practice evidence-based medicine, one of the many essential skills they must learn is how to speak the language of statistics.
Newcombe [22] believes that there has been no real evaluation process of biostatistics in the UK, and that formal debriefing of students regarding content and feelings about a course is insufficient. Stone and Qualters [23] agree and call for a better evaluative process of medical education outcomes as part of the professionalism concept that is so popular today. Medical education research today [19, 24] emphasizes the need to evaluate not only the acquisition of knowledge but also the acquisition of clinical/academic skills and whether that acquisition of skills is long lasting. In the present study we compared the skills before and after an intervention project as others have done before us [20, 21]. Similar to Rosenbaum and Kreiter [25], we evaluated the effect of using computerized learning by using evaluative questionnaires before and after the workshop. Similar to Garg et al. [26], we evaluated whether there was a change in the feelings of readiness and competence of the students about understanding and using computerized data analysis.
We can summarize our study’s aim as developing the students’ skills in the field of computerized data analysis, as might be needed in their M.D. theses, as well as enhancing their ability to read and interpret statistical data analysis from the current professional literature. Our primary assumption is that the ability to understand and use computerized data analysis is feasible for medical students and that we can evaluate it.
2. Methods
2.1. The Course
The main themes of the course evolved upon questions such as what is data analysis? How can we use computers for that? How can we deal with descriptive and analytical statistics, as well as understand computerized statistical output?
The course was structured as a two-week, 30-hour workshop, which was designed for medical students in their 4th year of studies (in a 6 year program), who had an introductory course in biostatistics during their first year of medical school. The workshop included the following: classroom lectures; working with real data files (from lecturers prior studies)—coding and variable definitions; simulations; and use of computer data analysis. For students to be exposed to the benefits of multi-disciplinary statistical applications, faculty included the following: epidemiologists, biostatistics experts, and advanced years medical students. At the end of the course, students were evaluated through their exposure to an unknown data file, on which they were to do the data analysis. The course is given during the first semester. The students started with formal lectures in statistics, followed by “hands on” experience in data analysis using the SPSS statistical program (SPSS, Chicago, IL).
We also wanted to see if the course was successful in developing and enhancing a new skill, beyond the subjective level of the personal experiences of the students. We also evaluated whether the course was able to dispel the anxiety and feeling of incompetence so often associated with data analysis in general and with its computerized part.
2.2. Research Population
The course is mandatory, so the subjects of the study were all 4th year students during the academic year 2005-2006, at the Faculty of Health Sciences of Ben-Gurion University of The Negev, Israel ().
2.3. Materials
We used 6-point Likert scale questionnaires to evaluate our subjects. They were paper based and anonymous and were designed especially for our study.(1)A specific debriefing questionnaire regarding the course (6-point Likert scale, 38 questions), which was formulated in previous courses and showed a high reliability (α-Cronbach of 0.93). This is the standardized course evaluation questionnaire/form.(2)Questionnaires regarding feelings of readiness and competence, attitudes towards data analysis and how to plan their own data analysis (6-point Likert scale, 18 questions). These questionnaires were based on prior studies [26, 27] that deployed similar aspects of teaching and evaluation. We checked their content validity by using external evaluators and their reliability by using statistical measures (α-Cronbach was 0.77).(3)An objective assessment of knowledge through multiple answer questions on specific aspects of data analysis. Content validity was checked by three expert evaluators. There were 11 multiple choice objective questions. Each question had a set of answers, only one of which was true, and each answer was validated by three experts. We then counted the number of correct answers.The second part of the questionnaire was mainly subjective and referred to the feelings and perceived capabilities of the responders. We tried to phrase the questions in the before and after questionnaires as similar as possible; they were all answered using a 6-point Likert scale (1 “not at all,” 6 “to a great extent”). The second (subjective) part included 18 questions such as As a physician, to what extent do you think there is a need to learn statistics? Or, In general, how do you evaluate your ability to use statistics? The third part was mainly objective and evaluated content knowledge (such as: What is the proper way to evaluate the association between 2 nominal variables? With answers like chi square, Kappa, and Spearman Rho).
2.4. Procedure
On the first day of lectures, students completed the precourse questionnaires, which helped us gather key demographic data and information regarding their general opinions towards statistics, data analysis, and the course. The precourse questionnaire also included content specific questions in the application of biostatistics tests (see Table 3). At the end of the course, students completed postcourse questionnaires regarding the course (as was done in the past), as well as a questionnaire complementary to those they completed at the beginning of the course—there were questionnaires regarding feelings of readiness and competence in data analysis, how they intend to manage such an analysis in their own thesis and questions regarding how long they think these skills will last.
In the analytical part, we had to eliminate sources of error that might have biased our results and so we discarded the subjects who completed the study questionnaires after hearing an introductory lecture on statistics, making them familiar with the study questions and thus helping them find the right answers—8 subjects were discarded. We also had to discard those who did not answer both the before and the after questionnaires, so we could have a truly paired analysis of our results.
All students’ data were gathered by the research assistant. The questionnaires were coded and stored using the Epi-Data program and analyzed using the SPSS computer program (SPSS 13.0, Chicago, IL).
2.5. Statistical Analysis
We analyzed the data first by using descriptive statistics (mean and SD, graphs) and then moved to analytical statistics using paired parametric (e.g., paired -test) for quantitative variables and nonparametric tests (McNemar test) for categorical variables.
2.6. Sample Size Calculations
We used the WINPEPI sample size software, in the paired tests sample size calculator. Under the assumptions that , power is 80%, and the mean difference to be detected is 0.5, with a standard deviation of 1 the needed sample size for -test was at least 27 pairs.
3. Results
From the 61 students enrolled, 34 precourse and 56 postcourse questionnaires were collected. Of the 34 precourse questionnaires, eight were completed by students who had listened to an introductory lecture on biostatistics and thus could have been biased. Of the questionnaires answered, 62.5% were female. 85.4% of the responders were born in Israel; the average length of stay in Israel for students born out of Israel was 19.4 years, eliminating the problem of misunderstanding the study questionnaires due to language problems. Most (86.8%) of our students had no academic experience of statistics prior to medical school. The average age of responders was 25.87 (±2.15), and most of them (90.4%) had an advanced high school math course.
We tested the reliability of the second part and found it to be acceptable (α-Cronbach of 0.77).
Looking at our data, one gets the impression that our subjects believe that they are quite proficient with computer use and data analysis and lack any fear of computers or statistics (Table 1). But this is far from true, because this involves both before and after evaluations. When performing a subgroup analysis, one gets a different understanding (see Table 2 for the subgroup analysis). One can see from these results that some of the attitudes towards statistics and computer use have changed and one can assume that the course had an effect on the attitudes of our students, even though not in all fields examined. Our students became more aware of the need to learn and use statistics (a graphic visualization of this is depicted in Figure 1); they wanted to get more training in that field, the course enhanced their capabilities in that area, and they became more proficient in the use of computerized statistics as well as their ability to read articles using uni- and multivariate analysis. One surprise was in the fear of computers, which became a bit more prominent after the course, even though this was not statistically significant.
(a)
(b)
The third part of our questionnaire was an objective one—when we did a subgroup analysis we found that most of the correct answers were after the course (), compared with only 0.94 (±1.28) correct answers before the course. A visual presentation of the magnitude of this change is seen in Figure 2.
(a)
(b)
For comparing the pre- and posttest, we ended up with 31 subjects who have filled all questionnaires—this is due to the fact that not all students who completed the questionnaires both pre- and postcourse were able (for example, 2 subjects had given birth during the course, etc.) or willing to fill both questionnaires. Considering the fact that we used a 6 point Likert scale for the subjective part and had 31 subjects in the analysis, we felt it was feasible to use a student’s paired t-test to find out whether the attitudes of our subjects changed during the course (see also sample size calculations). The results are shown in Table 2.
We found the following comparisons to be significant: the extent to which they as physicians need to learn statistics, the need to get a thorough education in statistics as a physician, the ability to use computerized statistics, their ability to read articles using univariate analysis, and their ability to read articles using multivariate analysis. All these changed in the hypothesized direction. A similar trend was found in the objective part (see Table 3). For all the questions, we found a significant change; all were in the hypothesized direction—higher percentage of correct answers. We also compared the total scores on the pre- and postcourse quizzes and found a significant difference among our pairs (). It is worth mentioning that two questions showed little improvement at the end of the course—a question about survival and a question about multivariate analysis, both of which are more advanced and complicated statistical issues.
3.1. Qualitative Analysis
We believe that the qualitative aspect of our course is of importance, and thus we also surveyed the written remarks of our students. A theme that repeated was that even though the contents of the course are not easy and even scary, the need to do a realistic test is essential, because only then could they confront the learned material and realize it is not as frightening as first perceived. Another important theme was the importance and efficacy of integrating clinical contents and real life studies, as the students saw those as anchors to their lives and a way to clarify the relevance of the course to their professional career.
4. Discussion
As mentioned, we believe that the proper way to help students become familiarized and knowledgeable in the field of biostatistics is helping them build their confidence in their ability to manipulate data analysis.
Our results suggest that this has been accomplished and that our students feel more at ease conducting and interpreting statistical data and procedures. This is seen both in the subjective part of our study, as well as in the objective part in which the improvement was very impressive. As Ambrosius and Manatunga [21] did, we introduced our students to a new language—the statistical language, a language that hopefully will help them understand data analysis and common statistical procedures done by others or by themselves in the future.
Another aim of our course was to develop the students’ skills in the field of computerized data analysis, which might be needed in their M.D. theses. With it came an aim of the study itself—can we accomplish this, and is this change measurable? We found that after the course students felt more competent and more ready to tackle the research project that is part of the requirements for their M.D.
Another issue we pondered was whether students understand and use computerized data analysis, and if can we evaluate this process; the answer to these questions is a profound yes—our comparisons of pre- and postcourse questionnaires demonstrated that students felt that their ability to run data analyses on their own had improved. They had no problem answering the study questionnaires as they went along (this was our personal impression both as lecturers of the course and as the study coordinators), so we can conclude that an evaluation of the process is both feasible and efficient.
As the qualitative analysis has shown us, the need to do a realistic test is essential, because only then could they confront the learned material, and the importance and efficacy of integrating clinical contents and real life studies was essential for making the course a successful one. We can sum up by saying that the course was successful in developing and enhancing a new skill—computerized data analysis. This was achieved not only at the subjective level of feelings of readiness and competence. We believe that the course was able to dispel the anxiety and feeling of incompetence so often associated with data analysis in general, and especially with its deep involvement of computers. This was also achieved in the objective level of finding the right statistical procedure for a complex set of situations, which are practical, real life situations.
Our findings align with other studies. Since the 1980s, Knapp and Miller III [18] thought that teaching biostatistics to medical students with computers was both feasible and effective because of the widespread use of computers in medical practice, as well as the availability of user-friendly statistical programs. Ostbye [17] has shown, that such a course is effective in achieving the study aims (which were teaching students statistics without the awe that usually engulf it) just as we did. Hutton Jr. et al. [19] found that a computer-based biostatistics course was as good as an equivalent course given as a series of lectures, considering measures that are both objective (like % of correct answers in a written exam) and subjective. Hewett and Porpora [24] reported their experience in a course in which the students are not only taught with the aid of computerized lessons, but they have to do an entire project, including data analysis, similar to what has been found in our study. Looney et al. [14] found that most of the courses in medical schools in the US were lectures, with little or no use of computers, even though most of the medical students found that computerized statistics would be more useful to them. Medical students themselves [5, 6] do understand the need to learn biostatistics, a theme that was also shown in our study.
It seems that one of the keys to a successful computer-based course is its being intriguing and interesting enough for the students. As Melnyk and Fineout-Overholt [27] have shown, you have to ask the right question. A qualitative theme that arose from our debriefing was the importance and efficacy of integrating clinical contents and real life studies, as the students saw those as anchors to their lives and a way to clarify the relevance of the course to their professional career. As Knapp and Miller III [18] have said—relevance is of the essence. If the students find the course to be relevant to their lives it is more appealing and less intimidating, and so the real challenge is to find the relevant cases, studies, and articles so they can immerse themselves in them.
One issue that must be stated is the issue of feasibility and costs involved in such a course. One might ponder why not to combine our 1st and 4th year statistics courses into one and thus reduce costs and shorten time until we have new physicians to work. This question was asked a dozen times by our administrators who wanted to reduce costs. But, our motto in the Ben-Gurion Medical School talks about the spiral of learning. This means you get acquainted as an undergraduate with all kinds of medical terms which you will learn as time goes by to understand more thoroughly as your knowledge base broadens. This also means that with time you become acquainted with more advanced teaching methods, and thus you are able to expand your knowledge and know how to be a better clinician and researcher. This approach has been used in our institute for 3 decades and has become world renowned [28–30].
4.1. Study Limitations
No study is devoid of shortcomings, and ours has its own. First and foremost, we still lack the answer to the question of whether the newly acquired skill is long lasting. We intend to see if this skill was retained at the end of medical school and to what degree. We will do so by administering the questionnaires, as well as by calculating the percentage of on-time submission of thesis projects by our subjects in comparison to previous years. One must recognize the possibility of other effects upon this acquisition of the new skill, but in that time period they only learn this very intensive course and have no exposure to other computerized courses or statistical issues.
Another shortcoming is our emphasis on questionnaires, which means that we lack a truly objective way of assessing the newly acquired skill. Another issue is developing other evaluation forms for data analysis that do not rely so heavily on the subjective part.
To avoid biased results, we had to discard subjects who had not completed both pre- and postcourse questionnaires (as mentioned in the methods). We ended up with only 31 subjects, which is half of our subjects and thus in the future we shall look for more thorough ways of data collection.
Last but not least is the lack of an external control to our study, which was mainly due to time and budget limitations.
4.2. Study Implications
In the current study we have shown that a computer-based course for biostatistics and advanced data analysis is feasible and effective. Its effectiveness is both subjective (as our subjects felt better prepared to do their thesis, as well as to read articles that incorporate advanced statistical data analysis) and objective (their know-how and when to apply the proper statistical procedures have been enhanced immensely). We do believe that future courses should implement medical statistics using computers and involving both clinicians and biostatisticians in the teaching of the course. We do believe that involving real studies of the tutors and lecturers will have added value and make thing more real and relevant. We do believe there is a need to rehearse and broaden the course in more advanced years, so that the learning will last longer.
We also showed that a formal evaluative process for such a course is possible and that it enhances the learning experience for the students. We do believe that using questionnaires is a feasible way doing so, even though one can make them a bit more concise (a fact seen in our questionnaires which not always added information).
We call for other researchers to try a similar curriculum and evaluate it so we can learn if this innovative method is a better way to teach biostatistics.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.