Academic Year Assessed: 2019-2020 College: CLS

Department: Liberal Studies Submitted by: Bridget Kevane, Director

 

Program(s) Assessed:

Indicate all majors, minors, certificates and/or options that are included in this assessment:

 

Majors/Minors/Certificate Options
Liberal Studies Quaternity, Environmental Studies, Global/Multicultural
  Minors: Global Studies, Global Health, Sustainability Studies
   

 

Annual Assessment Process (CHECK OFF LIST)

 

  1. Data are collected as defined by Assessment PlanYES x      NO
  2. Population or unbiased samples of collected assignments are scored by at least two faculty members using scoring rubrics to ensure inter-rater reliability.YES x    NO
  3. Areas where the acceptable performance threshold has not been met are highlighted.  YES x    NO     NA
  4. Assessment scores were presented at a program/unit faculty meeting.  YES x    NO
  5. The faculty reviewed the assessment results, and responded accordingly (Check all appropriate lines)Gather additional data to verify or refute the result. Identify potential curriculum changes to try to address the problemChange the acceptable performance threshold, reassess Choose a different assignment to assess the outcome Faculty may reconsider thresholdsEvaluate the rubric to assure outcomes meet student skill levelUse Bloom’s Taxonomy to consider stronger learning outcomesChoose a different assignment to assess the outcome
                OTHER:    We decided to continue emphasizing the importance of alternative/diversity                                of perspective with accompanying assignments in LS490R. If there is a drop                                in this element by next LS490R assessment, we will address again then.
  6. Does your report demonstrate changes made because of previous assessment results (closing the

           loop)?         YES x      NO

 

  1. Assessment Plan, Schedule and Data Source.a.  Please provide a multi-year assessment schedule that will show when all program learning outcomes will be assessed, and by what criteria (data). (You may use the table provided, or you may delete and use a different format).

    **Because of changes in LS, including the addition of two new minors and the loss of part of the LS Assessment team, we will be working on a new chart this academic year. Students in the minors are not required to take the LS490R so are not represented here.
    ASSESSMENT PLANNING CHART
    PROGRAM LEARNING OUTCOME 2017-2018 2018-2019 2019-2020 2020-2021 Data Source*
    Be able to apply critical & analytical thinking to discussions, readings and writings         LS 101USclass discussions and writings
    Be able to communicate effectively         LS 101US oral presentations
    Understand different disciplines (ways of knowing) from multiple perspectives     x   LS 101 and LS 490RSeminar essays and research papers
    Produce a well-defined research question/thesis and a balanced context     x   LS 490 RSeminar research papers
    Be able to use evidence, cite sources, and establish a balanced context about an issue     x   LS 490RSeminar research papers
               

    *Data sources can be items such as randomly selected student essays or projects, specifically designed exam questions, student presentations or performances, or a final paper. Do not use course evaluations or surveys as primary sources for data collection.b.  What are your threshold values for which you demonstrate student achievement? (Example provided in the table should be deleted before submission)


    Threshold Values
    PROGRAM LEARNING OUTCOME Threshold Value Data Source
    Understand different disciplines (ways of knowing) and engage with multiple perspectives The threshold value for this outcome is for 75% of assessed students to score above 2 on a 1-4 scoring rubric. Randomly selected student essays
    Produce an original, creative and persuasive analytical research paper The threshold value for this outcome is for 75% of assessed students to score above 2 on a 1-4 scoring rubric. Randomly selected student essays
    Be able to use evidence, cite sources, and establish a balanced context about an issue The threshold value for this outcome is for 75% of assessed students to score above 2 on a 1-4 scoring rubric. Randomly selected student essays
         
  2. What Was Done
    • Was the completed assessment consistent with the plan provided?   YES x_    NOIf no, please explain why the plan was altered.
    • Please provide a rubric that demonstrates how your data was evaluated.
    (Example provided below should be deleted before submission – your rubric may be very different, it just needs to explain the criteria used for evaluating student achievement).

    Example: Rubric for outcome #6



    Rubric for Undergraduate Research Undergraduate research is defined by the Council on Undergraduate Research (CUR) as an inquiry or investigation conducted by an undergraduate that makes an original intellectual or creative contribution to the discipline. Bosso and Mabrouk state that undergraduate research is a transformative collaboration between undergraduates and supervising faculty on research that extends knowledge or creative works that offer novel insights. These activities utilize the standard research or creative practices of the relevant discipline with the intent of contributing the resulting original work to the scrutiny of the greater university scholarly and artistic communities. Depth and breadth of research are assessed using the following criteria:
    •  A well-defined question.
    •  Description of the context and existing information about an issue.
    •  Adequate references, including original sources, alternative perspectives, and criticism.
    Criteria Exemplary (4) Good (3) page1image48091744Acceptable (2) Unacceptable (1)
    Well-Defined Research Question - Purpose The writer's central purpose or argument is readily apparent to the reader. The writing has a clear purpose or argument, but may sometimes digress from it. The central purpose or argument is not consistently clear throughout the paper. The purpose or argument is generally unclear.
    Context Balanced presentation of relevant and legitimate information that clearly describes the context and existing information about an issue. In-depth analysis of a significant topic. Reader gains important insights. Information provides reasonable background information and displays evidence of a basic analysis of a significant topic. Reader gains some insights. Information supports a central purpose or argument at times. Analysis is basic or general. Reader gains few insights. Context is not clearly identified. Analysis is vague or not evident. Reader is confused or may be misinformed.
    Use of References Compelling evidence from professionally legitimate sources is given to support claims. Attribution is clear and fairly represented. Professionally legitimate sources that support claims are generally present and attribution is, for the most part, clear and fairly represented. Although attributions are occasionally given, many statements seem unsubstantiated. The reader is confused about the source of information and ideas. References are seldom cited to support statements.
    Quality of References References are primarily peer reviewed professional journals or other approved sources (e.g., government documents, agency manuals, ...). The reader is confident that the information and ideas can be trusted. Although most of the references are professionally legitimate, a few are questionable (e.g., trade books, internet, sources, popular magazines,). The reader is uncertain of the reliability of some of the sources. Most of the references are from sources that are not peer reviewed and have uncertain reliability. The reader doubts the accuracy of much of the material presented. There are virtually no sources that are professionally reliable. The reader seriously doubts the value of the material and stops reading.
    Criteria Exemplary (4) Good (3) Acceptable (2) Unacceptable (1)
    Consideration of Multiple Perspectives &Positionality Identifies, articulates, and explains the reasoning behind differing points of view. Considers and discusses alternative views rationally and impartially. Identifies and articulates differing points of view as points of opposition. Demonstrates a somewhat inflexible position and an unexplained bias in discussing points of view. Refers to other readings or experiences, but references and explanations about how the information provides insights are underdeveloped or awkward. Ignores explicit and implicit points of disagreement.
  3. How Data Were Collected
    1. How were data collected? (Please include method of collection and sample size).
      The Liberal Studies director requested that each of the two instructors submit four randomly chosen papers, totaling eight in all. Randomization for these artifacts was done by looking at the order in which papers were submitted and assigning a number. For example, the first paper submitted was “1”, the eight paper was “8”, and so on and so forth. From there each instructor used a random number generator website to come up with 4 randomly chosen numbers for each section to determine which papers to submit, which were then renumbered as artifacts 1 – 8. The assignment assessed was their final capstone paper.

    2. Explain the assessment process, and who participated in the analysis of the data.

    Program Manager, Jennifer Storment, posted eight randomly selected student papers on the LS shared drive. Professor Kevane, along with the rest of the Liberal Studies team, assessed the students’ work individually. At least two assessors evaluated each assignment. Neither Instructor Teresa Greenwood nor Instructor John Townsend-Mehler reviewed his or her own students’ work.
    NOTE: Student names must not be included in data collection. Totals of successful completions, manner of assessment (publications, thesis/dissertation, or qualifying exam) may be presented in table format if they apply to learning outcomes.
  4. What Was LearnedBased on the analysis of the data, and compared to the threshold values provided, what was learned from the assessment?
    • Areas of strengthEfforts to emphasize the importance of alternative perspectives resulted in an increase of student research papers that presented “the other side” of contested topics.
    • Areas that need improvement
    Instructors for the LS490 capstone should continue to reinforce the multiple, alternative perspectives from credible and legitimate scholarship to inform students’s work. Research Core Scoring Results
    R-Core Assessment-LS 490R Capstone
    Exemplary = 4
    Good = 3
    Acceptable = 2
    Unacceptable = 1

    Paper#
    Assessor    		 Well- Defined Research Q/ Purpose
    Context
    Use of References
    Quality of Reference s
    Consideratio n of Alt.Perspectives
    Paper Average,4.0 scale
    1 Assessor 1 (BK) 4 4 4 4 2 3.33
      Assessor 3 (JTM) 2 2 3 3 2 2.67
      Avg:     3.5 3.5 2 3
    2 Assessor 1 (BK) 3 3 3 4 2 3.00
      Assessor 3 (JTM) 2 3 2 3 1 2.20
      Avg: 2.5 3 2.5 3.5 1.5 2.6
    3 Assessor 1 (BK) 4 4 4 4 3 3.75
      Assessor 3 (JTM) 2 3 4 3 3 3.25
      Avg:   3.5 4 3.5 3 3.5
    4 Assessor 1 (BK) 4 3 4 4 2 3.40
      Assessor 3 (JTM) 3 4 4 4 3 3.60
      Avg: 3.5 3.5 4 4 2.5 3.5
      1-4 Avg: 3 2.5 3.5 3.625 2.25 3.15
    5 Assessor 2 (JS) 3 3 3 3 2 2.80
      Assessor 4 (SCF) 3 2 3 3 2 2.60
      Avg: 3 2.5 3 3 2 2.7
    6 Assessor 2 (JS) 3 3 3 3 2 2.80
      Assessor 4 (SCF) 2 2 3 2 2 2.20
      Avg: 2.5 2.5 3 2.5 2 2.5
    7 Assessor 2 (JS) 3 4 4 3 2 3.20
      Assessor 4 (SCF) 3 3 3 3 2 2.80
      Avg: 3 3.5 3.5 3 2 3
    Assessor 2 (JS) 2 2 2 3 2 2.20
    Assessor 4 (SCF) 2 2 2 3 2 2.20
    Avg: 2 2 2 3 2 2.2
    5-8 Avg: 2.625 2.625 2.875 2.875 2 2.6
    Overall Average, Papers 1 - 8
    2.8125
    2.5625
    3.1875
    3.25
    2.125
    2.875
    Assessor 2 (JS) 2 2 2 3 2 2.20
    Assessor 4 (SCF) 2 2 2 3 2 2.20
    Avg: 2 2 2 3 2 2.2
    5-8 Avg: 2.625 2.625 2.875 2.875 2 2.6
    Overall Average, Papers 1 - 8
    2.8125
    2.5625
    3.1875
    3.25
    2.125
    2.875

    8
     
    Research Q/ Purpose
    %
    Context
    %
    Use of References
    %
    Quality of References
    %
    Consideration of Alt.Perspectives
    %
    Total Unacceptable (1)
    0
    0%
    0
    0%
    0
    0%
    0
    0%
    1
    6%
    Total Acceptable (2)
    4
    33%
    5
    31%
    3
    19%
    1
    6%
    12
    75%
    Total Good (3) 7 58% 7 44% 7 44% 10 63% 3 19%
    Total Exemplary (4's)
    1
    8%
    4
    25%
    6
    38%
    5
    31%
    0
    0%
      *12 100% 16 100% 16 100% 16 100% 16 100%
      *4 outlier scores thrown out)                  

    5.  How We Responded
    • Describe how “What Was Learned” was communicated to the department, or program faculty. Was there a forum for faculty to provide feedback and recommendations?We were pleased to see that the emphasis that LS instructors/staff agreed upon in 2019 improved the outcome regarding consideration of multiple perspectives. In 2019, there was zero consideration of a nuanced consideration of different positions with regard to topics such as green energy, affirmative action or voting rights. In this cycle that criterion improved. Also, because this is a semester long endeavor where students work closely with instructors on their research positions, we are pleased that seven of the papers had good research proposals/purposes that demonstrated a complex understanding of the topics at hand.
    • Based on the faculty responses, will there any curricular or assessment changes (such as plans for measurable improvements, or realignment of learning outcomes)?YES      NO x
      If yes, when will these changes be implemented?

  5. Closing the Loop

a) Based on assessment from previous years, can you demonstrate program level changes that have led to outcome improvements?

See above. We will continue to monitor multiple perspectives and explaining the importance of this criterion in a research paper.