2024 MSSE Program Annual/Biennial Assessment Report

Academic Year Assessed: 2023-2024
College:  Letters and Science
Department: Master of Science in Science Education
Submitted by: John Graves

Program(s) Assessed              
List all majors (including each option), minors, and certificates that are included in this assessment:

Master of Science in Science Education Program

The Assessment Report should contain the following elements, which are outlined in this template and includes additional instructions and information. 

1. Past Assessment Summary.
2. Action Research Question.
3. Assessment Plan, Schedule, and Data Source(s).
4. What Was Done.
5. What Was Learned.
6. How We Responded.
7. Closing the Loop

1. Past Assessment Summary.

Fifteen student in-person Capstone presentation artifacts were assessed on oral communication and 94% scored at or above the sufficient level, meeting the threshold of 75%. Fifteen student writing artifacts from MSSE education and science courses and 22 Capstone papers were assessed on written communication. Seventy-seven percent of the artifacts and 77% of the Capstone papers scored at or above the sufficient level.

The oral presentation is an area of strength of the program. For written communication, we have focused on writing in the four core courses that lead up to the Professional Capstone paper and partnered with the MSU Writing Center so that students receive additional writing support.  

The results of the assessment were shared with all stakeholders: MSSE staff, the MSSE Steering Committee, and the MSSE Instructors Retreat.  As a result, MSSE has partnered with the Writing Center and encourages students to utilize those services.  Additional oral practice for the Capstone Presentation has also been encouraged.

2. Action Research Questions for 2023-2024 Assessment Cycle.

• Can students communicate in written form evidence of the art and discipline of science education and its role in contributing to our understanding of teaching science?
• Can students communicate in written and oral form comprehension of at least two science content disciplines through successful completion of those disciplines, meeting course learner outcomes?

3. Assessment Plan, Schedule, and Data Source(s).

 Assessment planning, scheduling and data source information was created for MSSE Program Learner Outcomes (Table 1).

 Table 1.  Assessment Planning Schedule MSSE 

The threshold value is for 80% of assessed students scoring at the Achieving level or higher (Table 2).

Table 2.  Threshold Values

4. What Was Done.

The completed assessment was consistent with MSSE’s Assessment Plan.

Art & Science PLO

For the PLO, addressing the articulation of an understanding of the art and discipline of science education was assessed using the Art & Science Rubric (Figure 1). Artifacts from MSSE 501 Inquiry in Science Education and EGEN 511 Engineering Methods were analyzed. The target assignment in each course asked students to provide a response to the prompt: How would you describe science teaching as an art?  How do you understand the discipline of science education and its role in contributing to our understanding of teaching science? Provide an example from your own teaching experience.

Prior to the independent assessment, the assessors completed a pilot assessment to determine the continuity of assessment scoring.  The pilot was found to have similar scores from the assessors.

Figure 1. Science as Art & Discipline Rubric

Science Content PLO

At the direction of the MSSE Director, quizzes, final exams and final project scores from seven science content courses were gathered. Average scores for each artifact were analyzed. In some cases, rubrics were used to score final projects and these are referenced along with the average project scores in the analysis. Representative examples of each quiz, test, and/or scoring rubric are referenced in the discussion. Since the MSSE Program offers approximately 75 science content courses, the PLOs are specific for each course.  Evaluating mid-term and final exam scores and final projects seemed the best approach to answering the question: is the Program Learner Outcome of students learning science content being met? (Figure 2).

Figure 2. Science Content Rubric

5. What Was Learned.

Teaching as Art & Science Results

The results of the science as art and disciple outcome indicated that 97.5% of the students demonstrated understanding at or above the Achieving level, (N=20). The threshold of 80% of the students meeting the disciplinary content understanding learner outcome was met and exceeded. The number of students scoring Excellent or Outstanding was 82.5%, with 15% scoring at the Achieving level.

One student who scored at the Excellent level wrote, “Our standards (Not NGSS in my state) and their focus on science facts as well as standardized tests linked to funding has reduced a teacher's ability to teach what is most important in a science classroom – the Nature of Science (NOS).  Science education should be a means to show students the skills “good science” uses to learn about and improve the world.  If students are taught these things, then they will be able to investigate their own world, analyze data, critique others, and think for themselves” (Appendix A).

A student who scored at the Achieving level, wrote in part, “Science teachers in high school and below have an awesome responsibility to educate and encourage students without raising unrealistic expectations. The discipline of science education can help with those goals, especially if they are viewed as bumpers on the bowling alley and not barbwire-encrusted fences. Today’s science standards might very well be viewed with disdain in some not-too-distant future, and it is better to bend like the willow rather than snap like the ancient oak” (Appendix B).

Science Content Knowledge Results

The results of the science content final exam for BIOE 526 Symbiosis for Teachers: Eat, Prey, and Love indicated that the average score was 100% (N=11). This was assigned as a group grade and scored using the BIOE 526 Action Plan Unit (Appendix C). The threshold of 80% of the students meeting the science content learner outcome was met and exceeded.

The results of the science content final exam for CHMY 593 Kinematics, Equilibrium & Thermodynamics indicated that the average score was 89.8% (N=9). The range of scores was 55.6%-100%. The threshold of 80% of the students meeting the science content learner outcome was met and exceeded (Appendix D).

The results of the science content midterm exam for ERTH 520 Fundamentals of Oceanography indicated that the average score was 74.86% (N=6). The range of scores was 48%-94%. The threshold of 80% of the students meeting the science content learner outcome was not met (Appendix E).

The results of the science content final exam for ERTH 520 Fundamentals of Oceanography indicated that the average score was 83% (N=7). The range of scores was 54%-98%. The threshold of 80% of the students meeting the science content learner outcome was met and exceeded (Appendix F).

The results of the science content final exam for ERTH 521 Geology of the Moon indicated that the average score was 100% (N=4). The range of scores was 100%-100%. The threshold of 80% of the students meeting the science content learner outcome was met and exceeded (Appendix G).

The results of the science content final exam for LRES 585 Water Quality in the Classroom for Teachers indicated that the average score was 95.3% (N=9). The range of scores was 88.13%-98.75%. The threshold of 80% of the students meeting the science content learner outcome was met and exceeded (Appendix H).

The results of the science content Week 10 Quiz for NUTR 526 Nutrition for Fitness and Performance indicated that the average score was 85.6% (N=6). The range of scores was 16.67%-100%. The threshold of 80% of the students meeting the science content learner outcome was met and exceeded (Appendix I). The results of the science content Week 12 Quiz for NUTR 526 Nutrition for Fitness and Performance indicated that the average score was 88.2% (N=6). The range of scores was 50%-100%. The threshold of 80% of the students meeting the science content learner outcome was met and exceeded (Appendix J).

The results of the science content final exam for PSPP 547 Biomimicry: The Technology of Biology indicated that the average score was 96% (N=7). This was assigned as a group grade based on the PSPP 547 Rubric (Appendix K).  The threshold of 80% of the students meeting the science content learner outcome was met and exceeded (Appendix L) (Table 3).

Table 3. Final Quiz and Exam Scores

6. How We Responded.

The Program Assessment Report was communicated to faculty and stakeholders as follows:

• The Certificate Program Academic Advisor and MSSE Staff reviewed the data soon after it was compiled
• The report was shared with the participating instructors soon after the data was compiled
• Core Instructors reviewed the data at the fall Core Instructors Retreat in September 2024
• The MSSE Steering Committee reviewed the data at their semi-annual meeting in November 2024     
• The MSSE faculty reviewed the data at the Spring 2025 MSSE Instructors Retreat

To address the results of science teaching as art and discipline, a presentation of the results will be shared at the Spring ‘25 MSSE Instructor Retreat.  This will be followed by a round-table discussion for instructors to discuss the learner outcomes and share examples of ways this outcome can be addressed in their courses.

From the assessment results and anticipated feedback from all stakeholders, instructors were directed to review the quiz and test questions students failed. We plan to address the creation and evaluation of quiz and test questions at the Spring ’25 MSSE Instructor Retreat by presenting strategies for writing and evaluating test questions, followed by a round table discussion so instructors can share strategies they find useful in writing and evaluating quiz and test questions that address course learner outcomes.

No information outside of this assessment is necessary for informing programmatic change.

No additional support or resources (e.g. workshops, training, etc.) are needed currently.

7. Closing the Loop(s).

The loop will be closed as addressed above.

 Appendices

Appendix A: Excellent Paper Example

Appendix B: Achieving Paper Example

 Appendix C: BIOE 526 Representative Examples

Appendix D: CHMY 593 Representative Example

 Appendix E: ERTH 520 Representative Midterm Example

Appendix F: ERTH 520 Representative Final Exam Example

Appendix G: ERTH 521 Representative Final Exam Example

Appendix H: LRES 585 Representative Final Exam Example

Appendix I: NUTR 526 Representative Quiz Example

Appendix J: NUTR 526 Representative Final Exam Example

Appendix K: PSPP 547 Final Project Rubric

Appendix L: PSPP 547 Final Project Representative Example