University of West Florida Creates a Culture of Assessment
by Melanie DiLoreto, Administrative Assessment Fellow, University of West Florida
The University of West Florida was established in 1963 as an upper division institution. In 1983, UWF started accepting lower-division students. Since that time, the institution has grown steadily to its present population of over 12,000 students. Throughout this period of steady growth, new programs and degrees have been created and offered. As one of 11 state universities in Florida, UWF is uniquely positioned in the panhandle of Florida and is comprised of three campuses. UWF serves local students as well as many who come from other parts of Florida, other states, and countries. Approximately 15% of the student population lives on campus in University housing. Additionally, more than half of all local educators are prepared for their professional careers through programs offered in the Professional Education Unit at the University of West Florida.
The Current Assessment Push
Along with the steady growth of the university and the development of new programs and degrees comes an even greater need to make decisions based on accurate data. Because of this need, the upper administration created a university level assessment committee. This committee has developed a plan of action that has as its goal to make departments and their faculties more assessment savvy. To help accomplish this, a designated funding source was created to provide grants both for departmental level and single course, or faculty, level assessment projects. These projects will add to the development of a cadre of assessment activities that will in turn improve and enhance programs and degrees offered.
As the university has increased its interest in assessment activities and the use of data to help improve student learning and instruction, the School of Education has concurrently implemented several course and program level assessments using its data management tool, Tk20. During the past four semesters, the SoE has successfully used its assessment system to provide proof of student learning and attainment of competencies. These data were integral in the successful reaccreditation decisions rendered by both our state DOE and NCATE. Interestingly, though, the use of data to make decisions about programs, courses, and student learning has not always been viewed as necessary. In fact, if using data on a continuous basis was going to be required, then a change in the mindset of many faculty and staff would need to happen. Indeed, in order to make the use of assessment data a commonplace occurrence, something like a paradigm shift had to occur. Thus, the need for creating a culture of assessment became apparent.
Creating a Culture of Assessment
The development of a culture of assessment can be thought of as a dynamic endeavor that moves through phases. Being dynamic in nature, such a culture allows for the continuous improvement of program content, delivery, assessment, and student learning. As we worked to develop our assessment culture, we found we needed to move through four distinct phases. These phases allowed time for implementation as well as reflection by both the administration and faculty. The four phases we experienced as we strived to develop our culture of assessment are described in this article. Specifically, these four phases detail how faculty attitudes, use of data, and the implementation of Tk20 as a data management tool coalesced to create our assessment culture.
A Look Back in Time
After an intense review of the assessment activities that took place after the University of West Florida’s successful NCATE visit in 2003, the School of Education administration, faculty, and staff chose to adopt Tk20. Initially, many faculty members were unsure of the need for such a system. Several faculty members anecdotally commented, “But why do we need this? We’ve been just fine for years…” Additional comments included, “How much do students have to pay for our administrative requirements?” Furthermore, “You will never get all of the faculty members to use this system.” These comments came from the majority, not the minority, of faculty members within the Unit at UWF. Actually, the comment indicating that we would never get all of the faculty members to use the system was accurate since it was not the intent that every single faculty member use the system – although most faculty members did and still do use the system.
Once the decision was made to move forward with Tk20, its adoption was divided into four phases. These four phases helped faculty become engaged in data driven decision-making and helped foster a climate that allowed for a shift in attitudes as well.
Phases of Adoption:
- Phase 1: Focus on Successful Implementation
- Phase 2: Expand Functionality
- Phase 3: Perfect Uses
- Phase 4: Faculty Involvement
Phase 1: Focus on Successful Implementation of Tk20
The focus of Phase 1 was successful implementation. We knew that if we were to be successful in adopting Tk20, we must be inclusive of both internal and external stakeholders. Furthermore, we believed that we had to “keep it simple” and use a minimal number of functions of Tk20. To begin, we created a committee which developed the needs for data collection. This committee presented to all faculty in order to be a collective voice as opposed to one administrator dictating what would occur. The initial adoption included a small number of faculty members that were specifically selected to use their existing course-based assessments within Tk20 as opposed to the course management platform already in use at the university.
Awww, I see… At the completion of the first semester of adoption, many of those faculty members made comments such as, “I can now see the administrative needs for such a tool but what are the benefits to faculty for using it? When will more faculty members get to use this tool? Don’t you think we should standardize rubric formats so we can obtain consistent information?”
Phase 2: Expand Functionality
Will you show me… During Phase 2 of the adoption, the focus was to expand usage of functionality of the system. The adoption committee added course-based assessments to additional courses. In addition, a rubric template was created for all course-based assessments in which Tk20 would be the means for assessing student performance. Additionally, field experiences and student teaching and surveys were added functions in Tk20.
After completion of the second semester of using Tk20, faculty members often asked questions such as, “Will you show me how to collect data on my own students? Is there a way that Tk20 will ‘talk to’ other systems used at UWF for other reasons (distance learning platform, faculty records, etc.).” The transition in faculty comments indicated that the Tk20 adoption process was rolling-out smoothly.
Phase 3: Perfect Uses
How do I… Phase 3 of the adoption focused on perfecting uses. Instead of adding more functions to the system, the decision was made to improve those that we were already using. For example, we modified the use of course binders, rubrics, field experience evaluation forms, data collection reports, flowcharts, processes, procedures, etc. As a result, faculty members began commenting about their personal research. For example, “I’m working on a research project and I need certain information. How do I use Tk20 to obtain the information?” This was an indication that faculty members were taking ownership of the system and it was no longer ONLY an administrative requirement for accreditation.
Phase 4: Faculty Involvement
Can you imagine… Phase 4 of the adoption focused heavily on faculty involvement. Specifically, some faculty members were pulling their own reports for student monitoring; making suggestions for improvements; volunteering to help with data review and program improvements; and using the system for their own research. Comments from faculty exemplified this transition and included, “I know we used to gather student information from various sources in the past, but can you imagine if we had to do that again? Now that I’m used to Tk20, I can’t imagine using our old MS DOS-based student system. I used Tk20 to gather all of my information for this study.”
WHERE WE ARE TODAY
Most faculty members not only use Tk20 on a daily basis; but also, they can’t imagine doing their work without the system. They are using the data for their own course improvements and the data are compiled for use in both programmatic and unit changes. Furthermore, some faculty members compile student data, review their students’ academic history, monitor performance and responses to interventions, and use the data for their own research agendas.
Presently, the following functionality of Tk20 is used within many programs in the Unit at the University of West Florida:
- Course-based assessments
- Field experience applications, placements and evaluations
- Exam pass rates
- Application to the program
- Remediation monitoring
So, have we arrived? Do we have a firmly established culture of assessment? The obvious answer is no. However, we are gaining ground as each day passes. As you can see, the comments made by faculty before and after do show a shift. However, just as the development of a culture of assessment is a dynamic event, so is the maintaining and nurturing of that culture. As the adoption and use of Tk20 changed over time, so did the comments of the faculty members involved with the planning, adoption, and use of Tk20.