Revised: 07/31/2014

 

Outcomes Assessment for Continuous                           UPPS No. 01.03.04

Improvement                                                                         Issue No. 1

Effective Date: 04/09/2014

Review: June 1 E3Y

 

 

01.         POLICY STATEMENTS

 

01.01   As indicated in its mission statement and shared values, Texas State University values thoughtful reflection, collaboration, planning, and evaluation as essential for meeting the changing needs of those we serve. Outcomes assessment is an essential function for gathering critical information used to inform decision-making leading to continuous improvement.

 

01.02   In accordance with Comprehensive Standard 3.3.1 of The Principles of Accreditation: Foundations for Quality Enhancement of the Southern Association of Colleges and Schools Commission on Colleges (SACSCOC), Texas State’s accrediting body:

 

“The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional Effectiveness)

 

            3.3.1.1   educational programs, to include student learning outcomes

            3.3.1.2   administrative support services

            3.3.1.3   academic and student support services

            3.3.1.4   research within its mission, if appropriate

            3.3.1.5   community/public service within its mission, if appropriate”

           

01.03   Texas State has established an outcomes assessment program that effectively identifies and assesses outcomes for student learning in educational programs, administrative and academic/student support services, research, and community/public service initiatives.

 

01.04   This policy describes the process by which Texas State schedules, conducts, and approves assessment activities on campus to ensure useful data are collected.

 

01.05   For Texas State’s purposes, outcomes assessment activities are expected to be informative and not punitive, and are conducted predominantly for the purpose of continuous improvement of programs and services. The focus should be placed on the area, rather than singling out individuals.

 

01.06   Texas State encourages each area to employ methods that reveal weaknesses. Texas State acknowledges that this will involve submitting results with low success rates with respect to the target in question, and considers this a mark of effective assessment. The aim is not to show that the area is performing well. Rather, the aim is to determine how instruction/services might be improved.

 

01.07   The annual assessment cycle followed at Texas State includes two phases (Phase I – planning and Phase II – results reporting). In early fall of each year, assessment plans are developed/revised to include the mission statement, outcomes, methods and measures of assessment. In late spring for educational program outcomes and summer for all other outcomes, results, plans of action, and evidence of improvement are reported for each outcome.

 

01.08   All outcome assessment plans are maintained in the electronic repository, the Texas State Outcomes System. Reports are accessible to all Texas State faculty and staff with a user ID. Refer to the Outcomes Assessment website for specific cycle due dates, tools, and other pertinent information.

 

02.       DEFINITIONS

 

02.01   Area – Includes all units that provide assessments, i.e., office, department, unit, school, college, center, or institute, depending on the type of outcome.

 

02.02   Assessment – In higher education, includes gathering information in a prescribed manner and interpreting that information to make decisions about policy, curriculum, instruction, initiatives, or assessment practices.

 

02.03   Outcomes – The desired results that an area anticipates finding, expressed in general terms.

 

02.04   Methods – Strategies, along with tools or instruments, used to gauge progress toward attaining outcomes. Method statements should clearly indicate how an outcome will be assessed, who will be assessed, when and where it will be assessed, and what specifically will be assessed.

 

a.    Direct – Assessment tools that evaluate actual performance. Examples include exam scores, portfolios, observations, records of error, time, cost, efficiency, productivity, etc.

b.    Indirect – Assessment tools that evaluate perceived learning or performance based on opinions. Examples include student or employer perceptions, perceived satisfaction levels, perceived timeliness, perceived capability, etc.

 

02.05   Measures – Intended performance targets, expressed in specific terms.

 

02.06   Results – The findings amassed from the corresponding method.

 

02.07   Action Plans – Future steps to be taken toward attaining continuous improvement based on the reported results.

 

02.08   Evidence of Improvement – Current performance target level achieved as compared to previous findings, expressed in specific terms; intended to present result patterns from year to year, semester to semester, etc.

 

03.       EDUCATIONAL PROGRAMS OUTCOMES PROCEDURES

03.01  Educational program student learning outcomes identify the knowledge, skills, and attitudes that students are expected to demonstrate by the end of their course of study in a particular program. Chairs, directors, and program coordinators for each academic program will lead faculty discussions to gain collective input and consensus to create and assess appropriate, measurable student learning outcomes for each program.

 

03.02  All academic programs identified by a distinctive four-digit CIP code are required to identify and measure student learning outcomes.

 

03.03  Phase I in creating a system for assessment of student learning outcomes begins with a thorough review of the program mission. Mission statements should be:

 

a.    brief and clearly defined;

 

b.    distinctive to the program;

 

c.    reflective of the core purpose, primary functions and activities of the program; and

 

d.    consistent with the mission of the university, college, and department or school.

 

03.04  Once the mission is in place, at least five measurable student learning outcomes for undergraduate programs and three for graduate programs, as well as two support service outcomes (refer to Sections 04.05 through 04.09 below for support service assessment process) specific to the educational program are identified. These outcomes should align with the mission and overall goals of the program, as well as reflect the level of the program. Ideally, these outcome statements should demonstrate higher levels of cognitive, psychomotor, and affective learning domains. Outcomes:

 

a.    describe the knowledge, abilities, and attitudes of a successful graduate;

 

b.    focus on issues pertinent and important to the program;

 

c.    reflect learning throughout the program curriculum;

 

d.    focus on student learning rather than instructional or administrative processes; and

 

e.    focus on key areas where outcomes assessment will enable the program to improve.

 

03.05  Each identified outcome should have two methods of measurement that will be used to assess achievement of the corresponding outcome. Multiple methods of measurement are necessary to assure reliability and validity. Direct methods are preferred. Measures should:

 

a.    effectively and accurately reflect learning addressed in the outcome;

 

b.    apply to multiple sections of the same course, regardless of location or mode of delivery;

 

c.    allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;

 

d.    provide detail on who will be assessed, how the assessment will be conducted, when and where the assessment will take place, and what specific knowledge, skills, values, and attitudes will be assessed; and

 

e.    clearly indicate the desirable level or standard of performance (target).

 

03.06  Once the outcomes, methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:

 

a.    how and where the concepts, skills, and values are being taught;

 

b.    who will be involved in the implementation and measurement process;

 

c.    how individual student learning will be measured;

 

d.    what instruments (i.e., exams, assignments) and scoring rubrics will be needed to quantify student learning;

 

e.    when the learning will be measured; and

 

f.     how the data will be evaluated.

 

03.07  Phase II begins with collecting and analyzing the assessment data. Assessment results should be reported in a brief, but detailed summary for each method and measure. Discussions of findings should include:

 

a.    a reflection upon and discussion of the findings amassed from the corresponding method and measure;

 

b.    results based on reliable and valid data collection methods;

 

c.    success in achieving the desired performance target;

 

d.    indicators for improvement that can be gleaned from the results;

 

e.    evidence of learning related to the outcome aggregated for all sections of the course and disaggregated based on delivery, location, mode, and other pertinent factors;

 

f.     student accomplishments and success;

 

g.    improvement over previous assessment cycles; and

 

h.    effectiveness of previous action plans.

 

03.08  Based on the assessment results, a plan of action should be developed. Action plans should lead to continuous improvement in student learning. Action plans should also be feasible considering the availability of time and resources.

 

03.09  A synopsis of evidence of improvement gained from actions taken based on previous outcomes assessments should also be provided. This evidence of improvement should focus on continuous improvement in student learning and should be founded in the assessment results.

 

03.10  Once completed, outcomes reports are reviewed and approved by the appropriate program chair or director and college dean.

 

03.11  At the end of each phase of the assessment cycle, outcomes reports are audited by members of the Council of Associate Deans for Academic Assessment. Audit reports provide useful feedback for improving outcomes reports. These audit reports can be found on the Texas State Outcomes System.   

 

04.       SUPPORT SERVICE OUTCOME PROCEDURES

04.01 Support service outcomes identify core functions for providing administrative and academic/student support services. Department directors and administrators for each identified area across every division will lead staff discussions to gain collective input to create and assess appropriate, measurable outcomes for each support service area.

 

*04.02 All areas that directly report to a President’s Cabinet Member (units), as well as those that directly report to unit heads (departments), are required to identify and measure support service outcomes.

 

04.03  Phase I in creating a system for assessment of support service outcomes begins with a thorough review of the program mission. Mission statements should be:

 

a.    brief and clearly stated;

 

b.    distinctive to the area;

 

c.    reflective of the core purpose, primary functions and activities of the support service area; and

 

d.    consistent with the mission of the university, division, unit and department or office.

 

04.04  Once the mission is in place, three to five measurable outcomes specific to the support service area are identified. These outcomes should align with the mission and overall core functions of the support service area. Ideally, these outcome statements should reflect continuous improvement of services. Outcomes:

 

a.    focus on issues pertinent and important to the support service area as specifically defined in the mission statement;

 

b.    focus on key areas where outcomes assessment will enable the support service area to improve, with a focus on the end result and not the means to get there;

 

c.    clearly articulate the intended result or action desired; and

 

d.    align with the previous year’s identified action plans.

 

04.05  Each identified outcome should include two methods of measurement that will be used to assess achievement of the corresponding outcome. Multiple methods of measurement are necessary to assure reliability and validity. Direct methods are preferred. Measures should:

 

a.    align with the intent of the outcome;

 

b.    allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;

 

c.    provide detail on who will be assessed, how the assessment will be conducted, when and where the assessment will take place, and what will be assessed;

 

d.    clearly indicate the acceptable level or standard of performance (target); and

 

e.    align with the previous year’s action plans.

 

04.06  Once the outcomes, methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:

 

a.    who will be involved in the implementation and measurement process;

 

b.    what measures will be needed to quantify service levels;

 

c.    what instruments and scoring rubrics will be needed;

 

d.    when the services will be measured; and

 

e.    how the data will be evaluated.

 

04.07  Phase II begins with collecting and analyzing the assessment data. Assessment results should be reported in a brief, but detailed summary for each method and measure. Discussions of findings should include:

 

a.    a reflection upon and discussion of the findings amassed from the corresponding method and measure;

 

b.    results based on reliable and valid data collection methods;

 

c.    a level of achievement of performance targets;

 

d.    discussion regarding implementation of prior year’s action plans and how they impacted performance; and

 

e.    indicators for further improvement that can be gleaned from the results.

 

04.08  Based on the assessment results, a plan of action should be developed. Action plans should be feasible considering the availability of time and resources and should lead to continuous improvement of services.

 

04.09  A synopsis of year-to-year evidence of improvement gained from actions taken based on previous outcomes assessments should also be provided for each outcome. This evidence of improvement should focus on continuous improvement in services and should be founded in the assessment results.

 

04.10  Once completed, outcomes reports are reviewed and approved by the appropriate unit head and vice president.

 

04.11  At the end of each phase of the assessment cycle, outcomes reports are audited by the director of University Planning and Assessment. Audit reports provide useful feedback for improving outcomes reports. These audit reports can be found on the Texas State Outcomes System.  

 

05.       RESEARCH OUTCOME PROCEDURES

 

05.01 Research outcomes identify the scholarly and creative activities expected of faculty, staff and students in the research area. Academic deans and university research center/institute directors for each research area will lead discussions to gain collective input to create and assess appropriate, measurable outcomes for each research area.

 

05.02  All research areas including academic colleges with faculty and university research centers/institutes are required to identify and measure research outcomes.

 

05.03  Phase I in creating a system for assessment of research outcomes begins with a thorough review of the program mission. Mission statements should be:

 

a.    brief and clearly defined;

 

b.    distinctive to the research area;

 

c.    reflective of the core purpose, primary functions and activities of the area; and

 

d.    consistent with the mission of the university and college, as well as the university research strategic plan.

 

05.04  Once the mission is in place, at least three measurable research outcomes specific to the area are identified. At least one of the research outcomes must specifically address student involvement in research. These outcomes should align with the mission and overall goals of the research area. Ideally, these outcome statements should contribute to the achievement of the research strategic plan. Outcomes:

 

a.    focus on issues pertinent and important to the area, as specifically defined in the mission statement;

 

b.    focus on key areas where outcomes assessment will enable the area to improve; and

 

c.    clearly articulate the intended result or action desired.

 

05.05  Each identified outcome should have two methods of measurement that will be used to assess achievement of the corresponding outcome. Multiple methods of measurement are necessary to assure reliability and validity. Direct methods are preferred. Measures should:

 

a.    align with the intent of the outcome;

 

b.    allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;

 

c.    provide detail on who will be assessed, how the assessment will be conducted, when and where the assessment will take place, and what will be assessed;

 

d.    clearly indicate the acceptable level or standard of performance (target); and

 

e.    address areas where the research area can make a difference.

 

05.06  Once the outcomes, methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:

 

a.    who will be involved in the implementation and measurement process;

 

b.    what measures will be needed to quantify research initiatives;

 

c.    what instruments and scoring rubrics will be needed;

 

d.    when research initiatives will be measured; and

 

e.    how the data will be evaluated.

 

05.07  Phase II begins with collecting and analyzing the assessment data. Assessment results should be reported in a brief, but detailed summary for each method and measure. Discussions of findings should include:

 

a.    a reflection upon and discussion of the findings amassed from the corresponding method and measure;

 

b.    results based on reliable and valid data collection methods;

 

c.    a level of achievement of performance targets;

 

d.    discussion regarding implementation of prior year’s action plans and how they impacted performance; and

 

e.    indicators for further improvement that can be gleaned from the results.

 

05.08  Based on the assessment results, a plan of action should be developed. Action plans should be feasible considering the availability of time and resources and should lead to continuous improvement.

 

05.09  A synopsis of year-to-year evidence of improvement gained from actions taken based on previous outcomes assessments should also be provided for each outcome. This evidence of improvement should focus on continuous improvement and should be founded in the assessment results.

 

05.10  Once completed, outcomes reports are reviewed and approved by the appropriate academic dean or university research center/institute director and the associate vice president for Research and director of Federal Relations.

 

05.11  At the end of each phase of the assessment cycle, outcomes reports are audited by the assistant vice president for Research and Federal Relations. Audit reports provide useful feedback for improving outcomes reports. These audit reports can be found on the Texas State Outcomes System.   

 

06.       COMMUNITY/PUBLIC SERVICE OUTCOME PROCEDURES

06.01  Community and public service outcomes describe outreach activities across the university. Designated administrators within the Academic Affairs, Student Affairs, and University Advancement divisions will lead staff discussions to gain collective input to create and assess appropriate, measurable outcomes for their community/public service initiatives within their divisions.

 

06.02  Those areas specifically assigned by their division vice president are required to identify and report on community/public service outcomes. Although several areas within a division may be involved in measuring and assessing on their own specific community/public service initiatives, only one collective report will be prepared for each designated division.

 

06.03  Phase I in creating a system for assessment of community/public service outcomes begins with a thorough review of the program mission. Mission statements should be:

 

a.    brief and clearly stated;

 

b.    distinctive to the division;

 

c.    reflective of the core purpose, primary functions and activities of the division; and

 

d.    consistent with the mission of the university, division, unit and department or office.

 

06.04  Once the mission is in place, three to five measurable outcomes specific to the initiative are identified. These outcomes should align with the mission and overall core functions of the division. Ideally, these outcome statements should reflect continuous improvement of services. Outcomes:

 

a.    focus on issues pertinent and important to the initiative as specifically defined in the mission statement;

 

b.    focus on key areas where outcomes assessment will enable improvement of initiatives, with a focus on the end result and not the means to get there;

 

c.    clearly articulate the intended result or action desired; and

 

d.    align with the previous year’s identified action plans.

 

06.05  Each identified outcome should include two methods of measurement that will be used to assess achievement of the corresponding outcome. Multiple methods of measurement are necessary to assure reliability and validity. Direct methods are preferred. Measures should:

 

a.    align with the intent of the outcome;

 

b.    allow for the collection of accurate, valid, and reliable data for each method and measure in a reasonable and efficient manner within the academic year;

 

c.    provide detail on who will be assessed, how the assessment will be conducted, when and where the assessment will take place, and what will be assessed;

 

d.    clearly indicate the acceptable level or standard of performance (target); and

 

e.    align with the previous year’s action plans.

 

06.06  Once the outcomes, methods and measures are in place, a system should be developed for implementation and assessment. This system should identify:

 

a.    who will be involved in the implementation and measurement process;

     

b.    what measures will be needed to quantify service levels;

 

c.    what instruments and scoring rubrics will be needed;

 

d.    when the services will be measured; and

 

e.    how the data will be evaluated.

 

06.07  Phase II begins with collecting and analyzing the assessment data. Assessment results should be reported in a brief, but detailed summary for each method and measure. Discussions of findings should include:

 

a.    a reflection upon and discussion of the findings amassed from the corresponding method and measure;

 

b.    results based on reliable and valid data collection methods;

 

c.    a level of achievement of performance targets;

 

d.    discussion regarding implementation of prior year’s action plans and how they impacted performance; and

 

e.    indicators for further improvement that can be gleaned from the results.

 

06.08  Based on the assessment results, a plan of action should be developed. Action plans should be feasible considering the availability of time and resources and should lead to continuous improvement of services.

 

06.09  A synopsis of year-to-year evidence of improvement gained from actions taken based on previous outcomes assessments should also be provided for each outcome. This evidence of improvement should focus on continuous improvement in services and should be founded in the assessment results.

 

06.10  Once completed, outcomes reports are reviewed and approved by the appropriate designated administrator and vice president.

 

06.11  At the end of each phase of the assessment cycle, outcomes reports are audited by the director of University Planning and Assessment. Audit reports provide useful feedback for improving outcomes reports. These audit reports can be found on the Texas State Outcomes System.  

 

07.       REVIEWERS OF THIS UPPS

 

07.01  Reviewers of this UPPS include the following:

 

Position                                                         Date

 

Director, University Planning and            June 1 E3Y

Assessment

 

Associate Vice President for                      June 1 E3Y

Institutional Effectiveness

 

Assistant Vice President for Research    June 1 E3Y

and Federal Relations

 

08.       CERTIFICATION STATEMENT

 

This UPPS has been approved by the following individuals in their official capacities and represents Texas State policy and procedure from the date of this document until superseded.

 

Director of University Planning and Assessment; senior reviewer of this UPPS

 

Associate Vice President for Institutional Effectiveness

 

Provost and Vice President for Academic Affairs

 

President