Last month, members of TAG met with representatives from IR’s Office of Planning & Institutional Effectiveness to discuss results from the TechQual survey that faculty were asked to fill out over the summer.
At this point, IR is working through the results and meeting with focus groups to get a deeper understanding of what the results mean for planning and operations. IR will not be sharing the full results directly with the University community, but the next IT Matters newsletter will include evaluation and reflection on the survey results as a whole.
During the TAG focus group, we looked specifically at faculty responses to the survey. Only full-time faculty were included in the survey, and there was a 21% response rate. This is the second time TechQual has been administered on campus – it was first administered in 2008. The survey respondents were split differently in 2008 than in 2011, so IR is not able to compare the answers directly, but there is some indication of changes over time.
The TechQual survey looks at three major areas: Connectivity & Access, Technology & Technology Services, and End User Experience. The survey instrument asks respondents to rate each item in three ways: in terms of 1) minimum service level, 2) desired service level, and 3) perceived service level.
Overall, the survey results indicate that for faculty, IR is not meeting mean desired expectations. In some areas, IR is not meeting mean minimum expectations. Adequacy gaps, or the difference between minimum and perceived performance, were larger for faculty respondents than for staff. There was evidence that faculty user experiences varied significantly, with some responses being overwhelmingly positive while others were negative.
Although direct comparisons could not be made to the 2008 survey data, there was a trend of rising user expectations for minimum service levels. At the same time, users’ perceived service levels were generally about equal or slightly higher, suggesting that IR is holding steady or improving in some areas.
The good news is that some of the key areas identified for improvement have already been addressed or will soon be addressed. For example, faculty respondents’ perceived level of service for wireless connectivity was below their reported minimum levels of service, but the campus wireless network is scheduled (pending the approval of funding) to undergo additional upgrades that will significantly improve connectivity in academic and administrative buildings.
Classroom technology had the largest adequacy gap for faculty respondents. We spent some time discussing what kind of classroom technology needs faculty experience. Most campus spaces are designed either for using technology (e.g., all seats face a display screen; projection screen requires dimming lights) or for not using technology (e.g., seminar-style seating). TAG members asked for classroom environments that allowed spontaneity and collaboration. We also discussed how faculty and IR can best communicate about technology problems in classrooms (e.g., if a broken projector is reported, can the instructors of the next scheduled classes be notified?). Jim is currently working with OIT and the Technology Support Center staff to figure out solutions to these kinds of questions.
On a more TAG-specific note, in the End User Experience portion of the survey, faculty respondents noted that for “Opportunities to Provide Feedback” their perceived service level was approximately equal to their minimum service level. This result suggests that TAG needs to continue to raise our profile as a communication channel between faculty and IR.
We ran out of time to fully discuss all of the faculty results, but other areas identified for improvement included mobile access and the website/my.scranton portal – aspects of which are currently being addressed by IR teams with faculty representation (the Mobile Apps Work Group and the Luminis Work Group).
[My notes on this meeting were a little rough, so if anyone notices errors or omissions, please let me know and I’ll correct it ASAP! – KY]
