Introduction

This handbook is intended to provide an overview of the Outcomes Assessment Committee structure, purpose, and to provide Cerro Coso faculty and staff guidance in the development and meaningful assessment of outcomes.

Outcomes Assessment Committee

The Outcomes Assessment Committee is an associated committee of College Council, under the Institutional Effectiveness Committee (IEC). The committee has a participatory governance structure with representation from all employee groups and is charged with providing oversight for the College’s outcome assessment processes and documents in order to improve student learning and achievement. The committee maintains its faculty emphasis, being the largest representative group, and a faculty member is the SLO Coordinator, and committee chair. The faculty represent the members of the standing Outcomes Assessment Committee of the Academic Senate. Student Learning Outcomes and Assessment related to academic courses and programs is a Title 5 “10 + 1” area.

The committee maintains a website for SLO resources and data are housed. Formal and informal resources are available for faculty, staff, students, and the public. These resources highlight best practice and effective strategies in learning outcome assessment and can provide guidance for faculty and staff and a context through which to interpret the information for students and the public.

Charge

To provide oversight for the College’s outcome assessment processes and documents in order to improve student learning and achievement.

Purpose

• Ensure that outcome assessment is ongoing, systematic, and used to assess and improve student learning and achievement.
• Promote ongoing, pervasive, and robust dialogue about student learning.
• Ensure the ongoing evaluation and fine-tuning of organizational structures to support student learning.
• Maintain student learning improvement as a visible priority in all practices and structures across the College.
• Ensure all practices, services, and structures provide student with an optimal learning environment.
• Ensure that student learning and administrative unit outcomes are specifically linked to program reviews.

Composition

• Student Learning Coordinator (faculty) – Chair
• Vice President, Academic Affairs
• 1 additional management representative
• 6 additional faculty representatives
• *Representatives from sites (may be management, classified, and/or faculty)
• 2 classified representatives
• Student representative

Evaluation and Assessment

The outcomes assessment process itself is evaluated both formally and informally. Formally, the committee is evaluated through an annual self-evaluation of the committee’s work and goals, scored with a rubric by the Institutional Effectiveness Committee. The following standards are analyzed:

• Student learning outcomes and assessment are ongoing, systematic, and used for continuous quality improvement
• Dialogue about student learning is ongoing, pervasive, and robust
• There is evaluation of student learning outcomes processes
• Evaluation and fine tuning of organizational structure to support student learning are ongoing
• Student learning improvement is a visible priority in all practices and structures across the college
• Learning outcomes are specifically linked to program reviews

And once every two years, the committee is additionally assessed by the strategic planning survey which asks employees about their awareness of and satisfaction with outcomes assessment. Informally, the SLO Coordinator, by virtue of frequently meeting with faculty and staff to discuss best practices and being a member of the curriculum committee and IEC, ensures continuous monitoring of outcome assessment practices and is often the first to hear of gaps and suggested revisions to the process for improvement.

Assessment at Cerro Coso

Cerro Coso Community College is committed to the ongoing assessment of student learning in academic programs and student services through a systematic, college-wide assessment plan. The results of assessment provide clear evidence of student learning, and student experiences, and are used to make further improvement to instruction and services.

The College embraces the idea that learning assessment is a natural extension of instruction and student services and that all departments and units have a responsibility to regularly evaluate the knowledge and skills that comprise student learning and student achievement and make adjustments in operations or teaching methodology when outcomes are not met.

Philosophy

Self-assessment is a natural extension of instruction and student services, and all members of the College share in this responsibility. Student populations are becoming more diverse and a rapidly changing employment economy creates challenges to meet all students’ needs effectively. Consequently, the teaching methods of today may not work as well for tomorrow’s learners. We need to continually assess what is working and what requires improvement. Another trend that makes self-assessment a natural academic activity is the culture of teaching and learning is shifting from independence and autonomy to interdependence and collaboration. Intra-departmental, collaborative assessment is a natural extension of this culture. We want to ensure that students are learning, so we should be interested in verifying this. Finally, we are accountable to external organizations and students, as consumers, for our learning effectiveness. Assessment certifies the quality of the education we offer.

• We value assessment as a process for continuous quality improvement and evolution of programs and services. We believe when done intentionally, and with meaningful analysis, the assessment processes lead to improvement in student achievement and services.
• We value a process that is simple, but not simplistic. Outcome assessment should be simple enough to be manageable and sustainable, but it should thorough enough to assess and improve instructional programs and services.
• We value quality over quantity. Learning Outcomes are intentional, measurable, and succinct. They represent the major skills, knowledge, abilities a student will acquire at the successful completion of a course, or program.
• We value assessment of support programs and services as an effective means to ensure that student learning occurs in an environment that values the student experience.
• We value assessment of courses and programs as a faculty-driven process to ensure that it is constructive and non-punitive. The process supports full faculty participation and the successful completion of an assessment cycle, including the definition of outcomes and assessments, assessment design and collection of data, analysis of the data, and implementation of improvements based on the data.

Definition of Student Learning Outcome Assessment

Student learning outcome assessment is an activity in which institutional and instructional effectiveness is certified by evidence of student learning, or experience with programs and services. Specific measurable learning behaviors are identified and assessed, and the results of the assessment are used to improve programs, courses, and services. Assessment, in this context, is not an evaluation of individual students or faculty.

There are several other concepts implicit to assessment:

• Its primary purpose is to improve student learning and services at Cerro Coso
• It is a process that is on-going and cyclical
• It does not encroach upon academic freedom
• The results are used constructively, not punitively
• Related to academic courses and programs, it is faculty-driven
• It is a collaborative process

It is a process by which individual learning outcomes are defined at the administrative service unit, institutional, program, and course level. For a particular outcome, expected student achievement /experience is compared with actual outcomes, using predetermined benchmarks. If the results are lower than what has been determined to be acceptable, a plan to improve student learning, or services is developed and implemented.

Assessment, in this context, is not related to grades or faculty evaluation. Although students provide evidence of learning, this is not an assessment of individuals, but an assessment of curriculum design and institutional best practices to the end that students are successfully learning.

Outcomes Defined

Outcomes are the end result – changes in the learners’ knowledge, skills, attitudes, and habits of mind that develop as a result from being involved in a course of study, program of study, activity, or service.

• An outcome must be measurable, and meaningful.

Cerro Coso has the following outcome categories:

• CSLO– Course Student Learning Outcome: Each CSLO identifies a specific learning goal a learner will achieve when successfully completing the course. CSLOs are defined at the individual course level and identify the knowledge, skills, and abilities a student will achieve upon successful completion of the course. Courses generally have 3-6 CSLOs.
  • Course Student Learning Outcomes are different from Course Objectives. Course objectives are incorporated throughout the COR in order to align with C-ID. Objectives nestle under CSLOs.
• PSLO– Program Learning Outcome: Describe what learners will know and be able to do when they complete a program of study. They are closely linked with the CSLOs in the courses that make up the program. Programs map CSLOs to PSLOs. *Programs = certificates and degrees, as well as sequences or groups of courses that allow students to achieve an academic objective, such as the general education pattern, basic skills improvement to the associate level work, and the honors program.
• ISLO– Institution Learning Outcome: ILO’s represent competencies learners will achieve while completing a program, and represent broad learning categories. Programs map CSLOs to ISLOs.
• GELO – General Education Learning Outcome: General Education courses are mapped to broad learning outcomes.
• AUO – Administrative Unit Outcome: Represent the key functions and services of student services, learning support, and administrative units.

Continuous Quality Improvement Practices

• All outcomes are assessed at least once every five years. Many are assessed more often, but the minimum requirement is at least once during each program review cycle. The program review template requires departments and units to complete a chart listing the due dates of each CSLO, PSLO, and/or AUO during the five year cycle.
• New courses are assessed the first semester taught, providing valuable reflection on the course structure, teaching strategies, and assessment methods. This assessment may be entered as a formative assessment, meaning it will not be reflected in the program’s data, or it may be entered as a summative assessment, and reflected in the program’s data.
• All courses in a program are mapped to program learning outcomes and also to applicable institutional outcomes (ISLOs) Appendix D
• Formative assessment is used to inform teaching/services during the course/event by checking understanding and learning at various points in instruction/service. These assessments may be captured in eLumen, if the faculty member, department, or unit wish.
• Summative assessment is used to evaluate learning at the end of instruction/service. These assessments are captured in eLumen and used for reporting.
• The development, assessment, and analysis of outcomes are the result of college-wide collaborations and dialogue.
  • CSLO’s and PSLO’s are defined in official curriculum documents and are the result of departmental dialogue guided by advice and feedback from the SLO coordinator and outcomes committee. The SLO Coordinator, who is always a faculty member, sits on the curriculum committee and serves in a technical review role to provide feedback on CSLO/PSLO design during the curriculum approval process.
  • AUO’s of an operational unit are defined by the supervising administrator based on core functions of the department. The SLO coordinator, supervisor, institutional researcher, department staff, and/or other mentoring staff member provides input and assistance to supervising administrators defining AUO’s and determining assessment instruments.

• For all CSLO’s, PSLO’s, and AUO’s, the specific assessment instrument is determined when the outcome is defined, and can be located in the course outlines of record and program review documents. For many instructional programs PSLO’s are mapped-over class assignments or entire CSLO’s from key or capstone classes. For example, the second PSLO of the Web Fundamentals certificate—demonstrate technical and creative mastery of the creation of web media, such as graphics, motion graphics, and interactive media—is assessed by a project scored with a rubric in Digital Media Arts C102.
• When outcomes are assessed, the expectation of all CSLO’s and PSLO’s for instructional programs is that collected data reflects all offerings at all campus locations, including online, and full time/part time faculty.
• AUO assessment will reflect all campus locations where the unit provides services.
• Once assessed, outcomes can be aggregated, disaggregated, and analyzed to drive improvements in instruction and services. For all CSLO’s, PSLO’s, and AUO’s, if gaps are detected appropriate remediation is expected to be designed and implemented and the outcomes reassessed at the next cycle or earliest available opportunity.
  • For instructional programs, faculty will be prompted to complete a reflection template after entering assessment results. This provides the faculty member an opportunity to reflect on teaching strategies, content, unique challenges, etc. and discuss improvements at the time the results are posted. In addition, outcome assessment gaps and improvements are discussed in both annual unit plans and the program review documents. Appendix C
  • For students services and administrative units, a reflection template will be provided after entering assessment results. (to be developed in fall 2018)
• When presenting course/program revision to CIC, faculty will discuss when a course /program was last assessed, and how the results influenced the proposal.
• Programs with less than 85% of their courses assessed are not eligible to complete Program Review. The target will move to in 90% in spring 2019.
• The AUP and Program Review templates require programs and units to identify all outcomes that were assessed, to identify gaps/themes, and link CSLO and PSLO data to budget requests. Outcome assessment information and results directly impact student behavior and achievement as faculty and staff identify best practices and collaboration opportunities both internally and externally with colleagues. Divisions, Units, Programs and Departments must directly correlate CSLO assessment and student success to requests for resources.
• The OAC reviews Annual Unit Plans for identified learning outcome gaps and themes, and uses this information to identify trends, training and professional development opportunities.
• The OAC recommends the following schedule of regular assessment within the 5 year Program Review Cycle:

• Years 1-3: All Course CSLOs assessed at least one time, re-assess any gaps identified
• Year 4: re-assess any gaps identified in year 3, assess PSLOs
• Year 5: write and submit Program Review

Roles and Responsibilities

Outcomes Assessment Faculty Coordinator

The Student Learning Outcome Coordinator provides college-wide leadership in the implementation of student learning outcome assessment. Under the supervision of the Vice President of Instruction, the faculty member receives reassigned time to provide leadership in maintaining and improving the outcomes assessment process.

Examples of Duties and Responsibilities:

• Serves as Chair of the Outcomes Assessment Committee with primary responsibility for the committee’s effectiveness in carrying out its charge act of promoting student learning by providing leadership in continuous and sustainable outcomes assessment and to foster a culture of inquiry
• Participates on the CIC committee as the technical reviewer of CSLO’s and PSLO’s in course and program proposals and as a resource to the committee on best practices in defining and assessing outcomes
• Participates on the Institutional Effectiveness Committee as the primary spokesperson for student learning and administrative unit outcomes, the outcomes assessment process, the integration of outcomes assessment results in the college’s integrated planning effort, and continuous review and improvement of the outcomes assessment process
• Participates in the annual evaluation of the outcomes assessment committee, identifies gaps in quality performance and collaborates with the committee to design improvements and implement changes
• Collaborates with the Vice President of Academic Affairs, the Program Review Coordinator, and the Accreditation Liaison Officer to maintain linkage among program review, outcomes assessment, and institutional planning, including resource allocation
• Attend Program Review meetings when needed to disseminate OAC recommendations related to Program Review documents
• Works with college department, unit, section, and division leadership to ensure college wide participation in the outcomes assessment process
• Acts as the primary point of contact for outcomes assessment related inquiries
• Assists in training college faculty, managers, and staff in outcomes assessment procedures and best practices
• Provides critical feedback to college constituents regarding outcomes assessment documents, plans, and integration of assessment results for continuous program improvement
• Monitors outcomes assessment timelines
• Maintains current knowledge of external developments in outcomes assessment, such as through literature review, participation on listservs, and attendance at conferences and workshops
• Supervises collection and archival of outcomes assessment data and processes for both internal and external use
• Provides administrative management of the eLumen Assessment Module for college wide use and oversees the module’s assessment process. Collaborates with Distance Education for eLumen-Canvas assessment integration.

The SLO Coordinator should have a strong understanding of curriculum, program review, and accreditation standards, and is a member of the Curriculum and Instruction Council. The following is a list of other skills identified as necessary for SLO Coordinators, based on input from SLO Coordiantors, curriculum chairs, and administrators throughout California:

An understanding of student learning outcomes and assessment

• Classroom teaching experience
• Educational research
• Sensitivity to diverse backgrounds
• Faculty leadership
• Strong interpersonal and motivational skills
• Organization and ability to keep current records
• Knowledge of institutional processes

The SLO Coordinator candidate is approved by both the IEC and the Academic Senate.

Outcome Assessment Committee Members

• Attend and actively participate in OAC meetings and related work
• Report on assessment issues and requests faculty input on assessment in their own area/unit.
• Assist colleagues with assessment guidance
• Assist committee in the review and evaluation of assessment reporting in Annual Unit Plans

Faculty/Department Chairs

Faculty or Department Chairs assume primary responsibility for all aspects of student learning outcome or administrative outcome assessment, although the process should be collaborative within departments and/or programs, and it may be necessary to rely more heavily on particular faculty members who have more expertise in a course's subject matter. Evidence of collaboration and dialog should be included in Department meeting minutes, Annual Unit Plans, and Program Review Documents. Individual faculty are responsible to gather CSLO Assessment information in their course, when an assessment is planned.

Institutional Researcher

Cerro Coso has access to the Institutional Researcher for support for unit plan, program review and student learning outcome assessment data. The IR office is available to provide guidance in the crafting of assessments that are valid and reliable and to assist in the collection of data that is not easily attainable through classroom-embedded assessments or through Oracle Discoverer. It is important that we have a researcher who is a member of our college culture and understands the complexities of serving students across multiple sites over a large geographic area.

Assessment of CSLOs in particular should be viewed as a reflective practice in intentional teaching. The primary focus should not be formal research, but instead focus is placed on reflection of course content in relation to CSLOs, and the learner’s demonstration of mastery at the end of the course.

Outcomes and Assessment Overview

What are Student Learning Outcomes

Student Learning Outcomes identify what students can DO to demonstrate that they are learning. There should be clear linkages between student behavior, the production of a learning artifact, and assessment of that artifact. Outcomes MUST be measurable.

Other characteristics of student learning outcomes include:

• They are NOT instructional objectives or goals
• They are an observable, behavioral outcome – what the learner will be able to do
• They focus on the end result, not on the learning process.
• They are learner-centered, rather than instructor-centered.
• They may or may not be content specific
• They should take a diverse student population into account
• They should be delivery platform independent (classroom, ITV, online)
• As much as possible and appropriate, they should require higher-level cognitive, affective, and/or psychomotor domains.
• Bloom's Taxonomy is recommended as a resource for the selection of outcome verbs.

Programs of study are created with what the student will know, and be able to do at the end of the program. With this in mind, program learning outcomes should be defined first, resulting from input from advisory committees or academic organizations for the discipline. Course student learning outcomes should emerge from program student learning outcomes. A curriculum map is useful in presenting how courses align or map to program learning outcomes.

Writing CSLO, PSLO, and GELO Outcomes

We refer to Bloom's Taxonomy of Educational Objectives (see Appendix A) for suggestions about appropriate observable outcomes (although Bloom's is not an exhaustive list). Bloom organized outcomes into three domains: cognitive, psychomotor, and affective. The cognitive domain relates to knowledge, the psychomotor domain relates to skills, and the affective domain relates to attitudes and values. If possible, we favor a set of outcomes that draw from each domain, although the psychomotor domain may not be appropriate for all programs or courses. Each of those domains has outcomes further organized according to depth of processing. We favor higher level outcomes that demonstrate critical thinking, a high degree of skill mastery, or personal integration of attitudes and values. Such higher level outcomes are listed in the right columns of the outcome tables. Refer to Appendix A for outcome examples, and Appendix B for examples of verbs that are difficult to measure.

Acceptable Results / Target

It is also useful to determine what the acceptable benchmark of student achievement will be. This has nothing to do with students passing courses or obtaining credit. Although we are measuring student learning in assessment, the objective is to determine how well we are doing with respect to instruction or student services. The question to be considered is: at what level would we determine that there is nothing more that we can do to improve student learning? There are student success factors that are outside of our control, so 100% student success is not realistic. However, something less than 100% will be appropriate, perhaps 90%, 85%, 80%, etc.

• The OAC has set 70% as the default – no assessment shall be set below this success rate

The determination of what will be acceptable is dependent upon many factors and, at first, may have to be a best guess among departments and program areas. That benchmark may differ from department to department, and it may differ between courses within a department. It may even differ between outcomes within a single course. An illustration of why this may differ is the following:

In some programs, entry level courses may have greater attrition than advanced courses because some students likely discover sooner rather than later that the program is not a good fit for their interests or aptitudes. This is a factor over which we have no control. Defining 75% as an acceptable result for assessment may be appropriate for an entry level course, during which many students are determining whether they are really interested in that program of study, whereas 95% might be appropriate for the capstone course of the same program because presumably by that time, students are confident about their academic goals. We would expect greater success, given the same quality of instruction.

Again, you are determining the point at which you believe institutional enhancements will no longer improve the results. This benchmark will inform you about what to do with the assessment data—make improvements or congratulate yourselves. There isn't a science to this. Determining appropriate levels is best achieved through continuous dialog within your department, as well as reassessment of the criteria after an assessment cycle.

The College’s Default Levels of Mastery:

• Meets expectations: The student demonstrated mastery. (Met)
• Almost meets / Barely meets expectations: The student is making progress toward but has not yet achieved mastery. (Optional use by each department). May be used to identify students who are within +-5% of the target.
• Does not meet expectations: The student did not demonstrate mastery. (Not Met)

Assessment Artifacts

Finally, the student learning outcome assessment definition needs to specify how the outcome will be measured. This includes an artifact and a method for scoring the quality of that artifact. Examples of common assessment artifacts include:

• Projects
• Portfolios
• Essays
• Speeches
• Performances
• Skill Demonstrations
• Athletic performances
• Exit Interviews
• Multiple Choice Exams
• Essay Exams
• Surveys
• Critiques

The artifact(s) chosen for the assessment should be appropriate for the outcome verb. For example, a learning outcome of describe is better measured by an essay than a multiple choice exam. Another consideration for the selection of an artifact is the relative ease or difficulty that the assessment can be conducted. Exams and surveys are easier to administer than portfolio assessments that are scored with a rubric. Departments should give careful thought about choosing an assessment that effectively measures the learning outcome, but is also reasonable to administer.

Principles of Assessment Tools

When considering which artifact/tool to use, consider the following:

• Isolate knowledge, and skills. Sometimes a CSLO has more than one variable to measure. Separate out the variables, so that each can be assessed. Consider if the student does not meet requirements in one variable, does it mean they have not mastered the entire CSLO. *This is especially true for math, or multi-step processes.
• Separate out levels of student mastery. Met / Not Met is acceptable, but would it be valuable to know how many of the students who met the outcome just barely met the target, vs. those who clearly met the target, or those how just barely missed the target vs. those who clearly missed the target?
• The tool should lend itself to repeated use, semester after semester. Consider how to guard against plagiarism/cheating. If using test bank generated questions, will these need to be revised if a new textbook is chosen?
• Develop a rubric that can guide in determining if the outcome was met.

Assessment Scoring

Some of the above artifacts can be simply scored for correctness, as is the case with multiple choice exams.

• Rubrics are appropriate for scoring projects, portfolios, essays, speeches, performances, skill demonstrations, critiques or essay exams.
• Response scales, such as Likert (respondents choose Strongly Agree, Somewhat Agree, Neutral, Somewhat Disagree, Strongly Disagree) may be useful in scoring surveys, interviews, or critiques. A scale might also be used to score a artifact holistically.

Assignment or course grades are not a valid means of assessing student learning outcomes for the following reasons.

• Course grades and many assignments reflect multiple skills and outcomes. We need to tease out a specific outcome for measurement.
• Course grades also may reflect criteria that have nothing to do with course learning outcomes, but are imposed within a course to motivate participation and the development of a learning community.
• Grades are an individual evaluation, whereas outcome assessment is collaborative and the results generalized.

However, certain types of course assignments can be leveraged for student assessment AND course assessment.

• Ensure that the same assignments and measuring tools are used in every single section of a course over multiple semesters and among all faculty. There must be a way to tease out a specific outcome and assess only that outcome.
• Collaborate with faculty to “norm” the assessment tool, rubric, and determine measurements for each level of mastery.

Administrative Outcomes

Administrative Unit Outcomes (AOU) identify what students (or clients) will experience or receive as a result of a given service. AOUs may also be business related, identifying particular goals related to efficiency or achievement.

AUO – Administrative Unit Outcome

• Represent the key functions and services of an administrative unit
• Enable administrative units to maintain focus on their role in student experience and success
• AUOS should be actionable by the unit

Consider / assess

• Service
• Efficiency
• Compliance
• Student /staff satisfaction - experience, or understanding as a result

Selecting Measures AUOs

• Data should not be difficult to collect or access
• Multiple methods of assessing each AUO
  • Direct and indirect measures
  • Qualitative and quantitative
• Data should represent all service points
• Connect to existing goals such as accreditation, strategic goals, etc.
• Use / connect to institutional data and resources when possible

Implementation, Planning, Budgeting

If a plan to improve student learning was developed, it should be implemented and reassessed in a new Assessment Study to verify that student learning has, indeed, improved. As has been previously mentioned, assessment is an on-going and cyclical activity. If the results of the previous study were acceptable, the next Assessment Study should focus on a different outcome.

Assessment results and plans for improvement must be integrated into our other institutional plans and processes. Because Cerro Coso Community College exists so that students may learn, there must be a link between the results of Assessment Studies and everything else that we do at Cerro Coso.

Outcome assessment results are thoroughly integrated into program reviews in the section titled “Achievement of Outcomes,” where the appropriate CSLO’s, PSLO’s, and AUO’s are reported on, analyzed, and serve as the basis of dialogue about change. They are also integrated into the annual planning template where departments and units report out on actions taken and on gaps identified in assessment activities undertaken in the immediately preceding academic year. The Unit Plan is included in the Educational Master Plan, which drives the College's Technology Plan, the Staffing Plan, the Facilities Master Plan, and the College's budget. Some improvements to student learning can be made with instructional practices, but sometimes institutional support is needed, and this process accomplishes that.

Publishing

Results of outcome assessments are published on the college website.

• PSLO assessments for every instructional program are posted both on the OAC page as well in the outcomes section of each program.
• PSLO results are posted as program reviews are completed; this process started in 2015, so a number of programs’ results are currently vacant but will be posted as the program reviews are completed.
• AUO assessments for student services, learning support services, and administrative services are published on the OAC page.
• Outcome assessment results are also reported out in annual unit plans and in program reviews.
• ISLOs are published on the college website and in the catalog

Specialized Assessment Practices

The Assessment Study

For programs wishing to delve deeper into assessment, they may plan an assessment study. The Assessment Study is the process by which a specific outcome is chosen (not necessarily CSLOs, PSLOs, or GELOs) and the results analyzed. It is important to understand that only 1 outcome is assessed in a particular study.

For detailed information on designing an assessment study, refer to Appendix E. It is recommended that the department/unit work with both the Outcomes Assessment Coordinator and the IR office if an assessment study is desired.

Appendix A: Instructional Student Learning Outcome Examples

The following are examples of several complete outcome statements:

CSLO Examples

Upon successful completion of the course, students will be able to:

Evaluate the merits of various learning theories and reflect on how they might be applied to their own practice. This will be measured with an essay, scored with a rubric.

Compose basic lyric and prose poems that integrate the key elements of poetry writing. This will be assessed through an appropriate sample of student poetry, scored by a rubric.

Apply the use of communication as a helping skill in human services. This is to be measured by an oral presentation, scored by a rubric.

Construct diagrams that accurately explain and demonstrate such earth science processes as the hydrologic cycle, the rock cycle, and the plate tectonic cycle. This will be evaluated through lab reports, scored with a rubric, based upon professional society guidelines.

Analyze case studies and identify the best possible solution to a problem. This will be measured by a multiple choice exam, with the criteria of success being that 80% of students answer correctly.

Demonstrate stage techniques in the performance of cold readings and monologues. This will be measured through peer critiques, scored through a faculty-developed rubric.

PSLO Examples

Upon successful completion of the program, students will be able to:

Develop and display a portfolio of visual art works from a variety of visual art disciplines that reflects a personal direction and individual creativity. The portfolio will be assessed by a rubric.

Demonstrate that they are prepared for one or more of the occupations specified in the program descriptions for digital animation. This will be measured by exit surveys, scored with a Likert scale.

Student Services SLO Examples:

Given the completion of a counseling session, 80% of students will be able to articulate, identify, develop, and clarify educational, career, vocational, degree, and transfer goals. This will be measured with a survey that uses a 5-point Likert scale to determine the extent to which students believe they are competent.

Given Web-based and printed instructions, 90% of students will be able to successfully apply, update, register, and drop classes. This will be measured using data collected from the computer in Admissions and Records and from BanWeb.

Given a detailed orientation, 90% of EOPS students will effectively use services and support to make satisfactory academic progress and achieve career goals. This will be assessed with a rubric that is completed by the counselor following meeting with a student.

Given print and electronic resources, 90% of veterans will be able to identify the necessary steps for obtaining educational services for veterans. This will be measured and scored by a survey in which steps are ranked in the order in which they should occur.

Administrative Unit Outcome Examples

• Provide a developmentally appropriate, play-based program for young children that is a model program for students and professionals related to the early education field.
  • Target: Average on all areas for each assessment
  • Assessment Method: ERS scores and CLASS evaluations in line with licensing and grant requirements.
• Provide materials and services that support the college’s programs and the interests of students, staff, and faculty.

• Target: At least 80% of responses for each AUO will be positive.
• Assessment method: The LAC will assess its AUOs using the student experience survey to address our students’ perception and will use a faculty survey every other Fall to learn of faculty perceptions.

• Optimize technology to enhance Information Competency instructional interactions across multiple modalities.

• Target: 80%
• Assessment Method: Satisfaction Surveys distributed to students (via the Student Experience Survey) and to faculty (via department survey)

Appendix B: Examples of Difficult to Measure Verbs

• “Demonstrate competency”, or “Demonstrate an understanding…”
  • These are both abstract and do not describe the measurable, action the student will be able to do at the end of the course/program. Using a measurable action verb provides the student, and potential employers with clear information on the skills and knowledge the student is prepared to use.
• Appreciate
• Access
• Develop
• Have more confidence
• Value
• Recognize
• Understand

Appendix C: Academic Reflection Template

Student Learning Outcome Reflection Template

Your responses in this reflection will go to the Department Chair, for discussion of the assessment tool, gaps, strategies, etc. within the department. Reflections can assist in creating a foundation to improve, and evolve the best practices of teaching and learning. The reflection template is for the entire assessment, so if multiple CSLOs were assessed, consider each CSLO when reflecting.

1. Describe any challenges that you had with the course or materials this term.
2. How did the results of student outcomes in this course section compare to previous sections of this course? Did you enact any changes or improvements to your instruction, or to the course material that could account for this change? Could any other changes explain a change in scores? If this is your first term teaching this course, put N/A.
3. Please discuss the strengths of the assessment (method (tool/assignment), artifact, rubric, timing of assessment in course, etc).
4. Please discuss the weaknesses of the assessment (method (tool/assignment), artifact, rubric, timing of assessment in course, etc).
5. Explain what you would do to change the course, if any, for future offerings.
   1. What actions will be taken to improve student performance? For example: particular skills or concepts that might need to be taught with additional attention, or using a different strategy; consider the curriculum- was enough time given to relevant learning goals?
   2. What training, professional development, or resources do you feel you need as a result of this reflection (i.e. teaching strategies, equity resources, etc)?
6. How many students did not submit the assessment assignment?
7. Please use this space to share any additional thoughts you feel relate to this particular course, and the assessment process or results.

Appendix D: Institutional Learning Outcomes

The ILOs are available at this link.

Appendix E: The Assessment Study

For programs wishing to delve deeper into assessment, they may plan an assessment study. The Assessment Study is NOT part of the regular cycle of assessment, rather is the process by which a specific outcome is chosen (not necessarily CSLOs, PSLOs, or GELOs) and the results analyzed. It is important to understand that only 1 outcome is assessed in a particular study.

This phase occurs over an appropriate period of time, to allow data to be collected from a sufficient sample. For the assessment of course student learning outcomes, this is usually 2-3 semesters. For program learning outcomes, it could be 2-3 years. There are 3 steps to the Assessment Study phase:

• Design the study
• Collect the data
• Analyze the results

Design the Study

In the previous phase, the assessment artifact, scoring method, and possibly the criteria for success will have already been defined. At this point, however, departments or program areas will need to work out the details of how the assessment will be conducted. The following issues/questions should be considered:

• What outcome will be assessed? Some suggestions...
  • A program's highest impact learning outcome
  • An outcome from a high impact course
  • An outcome that faculty are extremely passionate about
• Who is on the assessment team?
  • Subjective assessments must have 2 or more assessors to reduce bias
  • Objective assessments must have someone to tabulate the results
• What assessment artifacts or scoring devices need to be developed? For example:
  • Exam questions
  • Surveys
  • Interview questions
  • Likert scales
  • Rubrics
• What criteria will be used to determine whether the outcome of the study is successful or not (if not previously defined)?
• How will assessment artifacts be collected from students and archived until a review of those artifacts occurs? For example:
  • Video taping performances
  • Photocopying written works
  • Photographing visual works
  • Electronically storing digital works
  • Generating a mailing list for surveys, and budgeting for postage and self-addressed, stamped return envelopes
• What will constitute a sufficient sample?
  • Data should ideally be collected from multiple course sections across terms, sites, delivery modes, and instructors. Depending upon the number of sections offered each semester, a sufficient sample may require data collection over 2-3 semesters. A few courses, however, may provide a sufficient sample over a single semester, due to the number of sections offered and variation of delivery locations, modes, and instructors. ENGL C101 may be an example.
  • Similarly, program learning outcomes should be assessed with a sufficient sample, which may mean several groups of graduating classes over 2-3 years.
• How will the results be recorded?
  • Both objective and subjective data needs to be tabulated and compiled.
• Who will write the analysis of the findings?

Collect the Data

With thorough planning, the data collection process is fairly straightforward. There are a few points of note, however:

• The same artifact and scoring method must be used throughout the study. In other words, exams cannot be given to some students and not others. If the assessment artifact is an embedded class assignment, all course sections, regardless of instructor or delivery mode, must include the identical assignment. If multiple assessment artifacts are used, then all must similarly be used consistently.
• Given that only 1 outcome is assessed in a particular study, and course-embedded exams usually cover multiple course outcomes, the exam questions that pertain to the particular outcome need to be identified and the results somehow teased out and tabulated separately. Unless the entire exam only pertains to the single outcome, the general exam results cannot be used.
• Subjective evaluations must be conducted with a team of assessors to reduce bias. All assessors must evaluate all artifacts. In other words, the team cannot divide the work up to get through the process faster.
• It might be useful to document other data in association with learning outcome results, such as semester term, delivery mode, instructor, and/or campus site. This will be useful for analyzing the results and planning for improvement.
• Departments might consider incorporating retention/attrition data into the results. We should not merely be interested in the students that persisted to the end of the course, but also those who dropped out along the way.

Analyze the Results

• After tabulating the results and having already determined a benchmark of success, it will be clear whether students are achieving the outcome above, at, or below the expected level. If the result is at or above the expected level, congratulations are in order! This implies that there is nothing department faculty can do to improve the result. However, it may be worthwhile to discuss whether the criterion was set too low. This may be obvious if the department faculty can identify practices that could improve the result further.
• If the result is lower than expected, there should be discussion about why that is the case and what can be done to improve the result. This is where the identification of other data in association with the outcome data is useful. If on-site courses have a better result than online courses, what can be done to improve student learning in online sections? If results are better for 16-week semester courses than for 8-week summer courses, is there a way to improve the outcome for summer courses? Perhaps the solution is that particular course should not be offered during the summer because there is not enough time on task. If one instructor produced better results than others, what is that instructor doing that should be replicated throughout the department?

Please note that this data should not be used to penalize faculty or to point out failures. It should only be used to identify best practices and implement what works well more consistently. This is a constructive process and faculty should have that spirit about it. (This is also a good time to point out that while faculty are asked to discuss student learning outcome assessment as a part of the Faculty Evaluation process, this should simply be a discussion of the instructor's involvement in the process. The results of assessment are not included in faculty evaluation.)

Based on a collaborative departmental process, the results should be analyzed and a plan for improvement developed. Be sure to take detailed minutes of all meetings in order to provide evidence of collegial dialogue.