Stage 1 Research Overview

1. Stage 1 Research Overview ITSP 2010

2. Stage 1 Research Goals • Quickly address specific questions generated within the planning committees; • Identify and begin to address common themes running through diverse committee discussions: • A better understanding of the specific communication/information needs of various constituents on campus; • More thorough assessment of how well current technology applications, services and support are meeting those needs; • A more up-to-date inventory of the alternative applications, services and support used in local departments/offices. • Explore and hone the Office’s research approach.

3. Mixed-Method Approach • Develop and pilot small-scale surveys aimed at variety of constituents, • Distributed pilot surveys to Tier 2 support staff and to faculty/graduate students in early April; • Revise and distribute large-scale surveys to select campus audiences, • Revised surveys in late April, need to be reviewed and distributed; • Conduct focus groups with a variety of constituents, • Facilitated discussions with three different groups of staff, a group of undergraduate students, and a group of ACTs in late April; • Use ethnographic research methods to study the day-to-day activities of various constituents, • Shadowed administrative and program assistants in late April; • Conduct individual interviews with faculty members • Because of time constraints, interview data from a related research project was used.

4. Development of Pilot Survey (Faculty/GS) Review existing surveys (national and local), • Educause examples • Instructional technology • Use/ level of interest • Level/type of assistance provided • School of Ed Survey • Instructional tech, work productivity tech, web-based tech • Skill level/level of interest • Investment priorities • ASSETT Survey • User profiles/comparisons with other schools, colleges, institutions • Investment preferences • ITS Faculty Survey (2005) • Instructional technologies, • 13/24 questions about course websites (WebCT and non-WebCT), • 11/24 questions about other “in-class” technologies (classroom projectors, clickers, etc.); • Frequency of use/assessment of importance, • Barriers to use (interest, reliability of tools, support, etc.)

5. Development of Pilot Surveys • Build upon/enrich existing surveys (national and local), • Inventory-type questions (what) + analytical ones (why, how, etc.) • Move beyond just instructional techs • Identify key data points for longitudinal research, • Frequency of use (including older technologies) • Barriers to use (including older technologies) • Begin to address common themes from committee discussions: • Better understand various communication/information needs; • Measure user satisfaction with current services; • Inventory techs used in local departments/offices. • Integrate specific questions generated by the committee’s themselves, • Integrate suggestions from diverse reviewers.

6. Lessons Learned from Pilot Survey NEED TO: • Improve access to and usability of existing data within ITS departments; • Develop better system for coordinating collaborative development process (unique challenges of disperse team); • Faculty members contributing specific questions, • Committee members reviewing survey instrument, • Developer building the survey instrument itself, • Strategic Communications Outreach and Documentation team • Invest in more efficient analytical tools; • Zoomerang survey report forms • Qualitative analysis tools (NVivo, Provalis) • Increase faculty response rate (on future surveys) • Distribution method (local vs. institutional contact) • Length of survey • Perceived relevance

7. Accomplished vs. Yet to Accomplish • Develop and pilot small-scale surveys aimed at variety of constituents, • Distributed pilot surveys to Tier 2 support staff and to faculty/graduate students in early April; • Revise and distribute large-scale surveys to select campus audiences, • Surveys revised; future reserachersneed to review and distribute; • Conduct focus groups with a variety of constituents, • Facilitated discussions with three different groups of staff, a group of undergraduate students, and a group of ACTs in late April; • Use ethnographic research methods to study the day-to-day activities of various constituents, • Shadowed administrative and program assistants in late April; • Future researchers need to conduct ethnographic observations of other constituents. • Conduct individual interviews with faculty members • Future reserachers need to conduct individual interviews.

8. Focus Groups • Context we provided • Part of IT Strategic Planning process • Identified broad themes (comm/info needs, current services) • Discussion format • Open-ended • Data collection methods • Observation and note-taking • Audio recordings • Analysis methods • Word searches for ITSP themes • We also used the results to: • identify cross-departmental concerns, shared perceptions, • raise/clarify job-specific needs and issues, • cross-check the survey results, • clarify/enrich survey results,

9. Shared Concerns/Perceptions • Repeated calls for: • effective room scheduling technology, • better support for widely-used apps (Word, Excel, Dreamweaver), • more useful financial data • Understand our particular needs • I’m not sure they’re aware of the people we support (the chancellor, vice-provost, etc.). The demands they have are significant, and sometimes the response that they need--it needs to be immediate. (admin assistant) • Its seems like there should be some outreaching from ITS into each department, almost having a sales representative or a service rep where they know what’s going on in all the different departments and understand their issues. (administrative assistant) • Listen better • When you go to a department and you say, “What do you need help with,” you need to actually listen to what they say rather than just respond with, “Oh, we do that.” You need to figure out where there’s a disconnect. (IT professional)

10. Shared Concerns/Perceptions • Provide guidance and standards • ITS just can’t be as responsive as we need them to be. There are just too many needs. They shouldn’t try to provide all of the services. But if they can’t provide them, they should come up with standards to help us. (IT professional) • Collaborate not dominate • The training that we need is not something that ITS necessarily has to do because, collectively, we all have that knowledge. So if there was some way that we could share that knowledge with one another. (IT professional) • That’s the sort of thing I want to see is that willingness. To have them say we’ll be on the team but we don’t always have to be quarterback, we don’t always need to carry the ball. We’ve got to share that role. (IT Prof) • CU’s IT is behind the times • …recommending things that are totally outdated. (Faculty) • …infrastructure is, to be quite blunt, archaic. (IT prof) • …consistently 5-10 years behind what’s current (IT prof) • …sometimes dealing with the donors, Boulder County, the City, and such, we’re a little behind the times in our technology and they expect us to meet them on that level…It makes us look backwards (administrative staff)

11. Shared Concerns/Perceptions • Provide guidance and standards • ITS just can’t be as responsive as we need them to be. There are just too many needs. They shouldn’t try to provide all of the services. But if they can’t provide them, they should come up with standards to help us. (IT professional) • Collaborate not dominate • The training that we need is not something that ITS necessarily has to do because, collectively, we all have that knowledge. So if there was some way that we could share that knowledge with one another. (IT professional) • That’s the sort of thing I want to see is that willingness. To have them say we’ll be on the team but we don’t always have to be quarterback, we don’t always need to carry the ball. We’ve got to share that role. (IT Prof) • CU’s IT is behind the times • …recommending things that are totally outdated. (Faculty) • …infrastructure is, to be quite blunt, archaic. (IT prof) • …consistently 5-10 years behind what’s current (IT prof) • …sometimes dealing with the donors, Boulder County, the City, and such, we’re a little behind the times in our technology and they expect us to meet them on that level…It makes us look backwards (administrative staff)

12. Understanding local needs/issues • We need an electronic document management system. Something where you can tag things, be able to search for things. I’ll give you an example: faculty appointments. Faculty affairs has an enormous file for every single faculty with all these paper documents, the Schools and Colleges also have a file,and the departments have their own file…The paperwork for dissertation committees is similar. This isn’t new tech, this is stuff that’s been around for a long time, its just not being used here. (administrative staff) • For research collaboration, we have faculty who work with people across the globe. They’re outside the campus firewall so if they want to share files they either have to host that on a cloud service outside of all of our services or they host it locally and we give them an identikey. This really doesn’t work because for 35 faculty there are probably 200 collaborators above them that they’re working with, and with grad students it multiplies up to about 2000 people. So how do we get those 2000 people to collaborate? We’ve got this “inside campus” (inside the firewall) and “outside”. But there’s no in-between where people can meet at and work. (IT professional) • I’m an inconsistent enduser—meaning I use certain technologies only at the beginning and at the end of the semester rather than on a daily basis. Training workshops don’t help, I just forget what I need to know. What I need is really good documentation. (administrative staff)

13. Clarifying/enriching survey results Reponses to question about centrally-provided file servers (Tier 2 survey) • 18%Helps 2%Hinders 7%No Effect 68%Don’t Use • You can pay for server support service from ITS but its really expensive…people aren’t afraid of moving outside. (faculty) • It costs far too much to use ITS services than to do it yourself or to use an outside service. So why would you? (IT professional)

14. Clarifying/enriching survey results 5-HELP 61%Helps 7%Hinders 30%No Effect 3%Don’t Use Dispatched Desktop Support 18%Helps 5%Hinders 10%No Effect 57%Don’t Use • They need to become more customer-service oriented. Many of them have decent computer knowledge but don’t seem to have the time or the ability to communicate with non-technical people. (administrative assistant) • Its impossible to find a contact for them, they’re kind of hidden behind a screen. It almost seems scary to contact them for any support, let alone a follow up to your original request. I know people in my department are afraid to contact them for a follow-up when something is still not working. (administrative staff) • The Tier model really puts us in a bad place. It dehumanizes the process, so you no longer have a relationship with ITS, they’re just, you know, 5-help or a black box. (IT professional)

15. Stage 1 Research Goals • Quickly address specific questions generated within the planning committees; • Identify and begin to address common themes running through diverse committee discussions: • A better understanding of the specific communication/information needs of various constituents on campus; • More thorough assessment of how well current technology applications, services and support are meeting those needs; • A more up-to-date inventory of the alternative applications, services and support used in local departments/offices. • Explore and hone the Office’s research approach.

16. Future research needs • Gather feedback from additional constituents (faculty, students, staff) on existing applications and services; • Consistent with recommendations in Chapters 1.1, 1.3, 2.1, 2.5, 2.7; • Using mixed-methods, conduct additional research targeting the everyday communication/information needs ofgraduate and undergraduate students; • Consistent with recommendations in Chapters 1.1, 1.3, 2.3, 3.1, 3.4; • Conduct ongoing interviews with a diverse set of faculty members to more thoroughly assess current technology use and future needs (ACT’s?); • Consistent with recommendations in Chapters 1.1, 1.3, 2.3, 2.7, 3.1, 3.2, 4.1, 4.2; • Gather and compile additional information for evolving Tier 2 Staff lists (e.g. activities supported, constituents served, etc.) • Consistent with recommendations in Chapters 1.1, 2.5, 2.7, 4.1, 4.2;

17. Future research needs • Research the reasons/motivations behind using alternative technologies, and the issues raised by wide-spread use; • Consistent with recommendations in Chapters 1.1, 1.3, 2.2, 2.7; • Stay abreast of research trends/innovations and feedback mechanisms in other research institutions, • Consistent with recommendations in Chapters 2.1, 2.2, 2.7, 4.1, 4.2; • Update the technologies that are used in the IT organization to collect, process, analyze, and present research data. • Take advantage of research opps that an internal cloud introduces • Qualitative analysis tools (Nvivo, Provalis) • Dynamic, password-protected website

18. Future research needs • Establish a mechanism for coordinating institutional and local technology, research/planning efforts, (Create a regular, web-based communication channel for these groups? Attend local, technology-oriented committee-meeting discussions?) • Consistent with recommendations in Chapters 1.1, 2.5, 3.1, 3.2, 4.1, 4.2; • Develop a virtual space where constituents can search, add to, evaluate and updateinventory of alternative applications, services, and support used by local departments/offices (see inventory list for details) • Consistent with recommendations in Chapters 1.2, 1.3, 2.2, 2.6, 2.7; • Provide an additional mechanism for diverse members of the campus community to voice individual technology issues and to understand how these might be linked to the ongoing planning efforts (see focus group summary for details) • Consistent with recommendations in Chapters 1.1, 2.7, 3.2, 4.1, 4.2;

19. Questions?