The Development of Ship Simulation Assessment to Complement Written Maritime Examinations in Ireland
Document Type
Conference Object
Disciplines
Education | Educational Assessment, Evaluation, and Research
Abstract
This paper is a summary of research which examined the development of ship simulation assessment as a complementary form of assessment to written examinations for competency-based assessment in Coastal Navigation within the Seafarer’s Training, Certification and Watchkeeping Code 1995 (STCW 95 Code) for qualifying stage Officer of the Watch (Navigation) (OOW) students in Ireland in 2005. Current thinking on assessment principles and the use of simulation for assessment were researched by literature review. A semi-structured questionnaire determined views from Canada (Vancouver and Newfoundland), Sweden, the Netherlands, and Ireland. Simulation assessment is used in various forms outside of Ireland, but in a limited form in Ireland. The STCW Code and examination syllabus from the Maritime Safety Directorate (MSD) of the Department of Transport, Ireland, were used to determine relevant tasks for inclusion in a simulation assessment paper. A simulation task paper was written and a marking system was developed which reflected OOW required task skills, underlying knowledge and proficiency. A purposeful sample of nine qualifying stage OOW students which represented about 50% of the population for 2005, participated in quasi-experiments, using ship simulators, to test the assessment paper. Assessments were marked and compared to the actual results obtained in the students’ official written Coastal Navigation examinations. Marks awarded for the simulation assessment averaged 62%, in contrast to the students’ official written examination average mark of 72%. A comparison of both sets of results resulted in a Spearman Rank Difference correlation +0.73, indicating a strong relationship. Simulation assessment paper task analysis indicated easy and difficult tasks for review. Participants’ views, obtained using an etic reaction questionnaire, showed that the students enjoyed simulation and found it realistic; believed it related well to tasks during training; and found some task completion difficult. After the written examination and simulation assessment process, an emic focus group interview with the participants confirmed the etic view and resulted in valuable insights and suggestions for improvements. Students felt that simulation assessment was more difficult than the appropriate written examinations. The research concluded that simulation assessment is a suitable means for determining competency, as required by STCW 95, in contrast to the existing written examination. Simulation assessment which is relevant to International standards is valid, reliable, practical and authentic. It can be developed and used successfully to complement existing professional written examinations for OOW students in Coastal Navigation, using continuous assessment. This research is the first of its kind in Ireland, and lays the foundation for the future expansion and progression of assessment systems in Irish maritime education and training.
Recommended Citation
Kavanagh, Bill. The Development of Ship Simulation Assessment to Complement Written Maritime Examinations in Ireland. 2006 International Conference on Marine Simulation and Ship Manoeuvrability (MARSIM) in Marine Institute Willem Barentsz, Terschelling, Netherlands.
Publication Details
Presented in June 2006 at the Conference: International Conference on Marine Simulation and Ship Manoeuvrability (MARSIM) in Marine Institute Willem Barentsz, Terschelling, Netherlands. © 2012 IMSF