2016 Sessions


With over 300 sessions and 600+ presenters, the 2016 Forum has content for higher education professionals who use data for decisions. Browse the sessions below based on the topic areas and formats of interest to you. The final schedule for these sessions and the Web App will be available a few weeks prior to the conference.

Topic Areas







Formats








Assessment: Accountability, Institutional Effectiveness, and Accreditation

(CANCELLED) The Program Review Train by Jacob Williams, Ryan Johnson, Emilee Purcell
Program-level data is used for accreditation, academic planning, and resource allocation. With so much at stake, Ivy Tech has creatively reimagined the program review process to address the many data needs across its large, statewide, multi-campus system. After completing the second round of a completely overhauled program review process, we will share our successes, insights, and lessons learned from our transition from a paper-based afterthought to a dynamic and robust electronic data collection and reporting process.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
(Cancelled) Visualizing General Education Assessment Results for Impact and Utilization by Greg Michalski, Jametoria Burton
Assessing general education outcomes is a necessary reality for institutions of higher education. This session will outline an overall process, and provide some detail about the individual steps undertaken to successfully accomplish a structured general education outcomes assessment project involving multiple stakeholder groups. Key elements including the construction of targeted assessment rubrics, the scoring of student artifacts, the confirmation of results, and presentation and utilization of findings are included. As a part of the presentation and usability phase, several online interactive dashboard visualizations will be used as examples in the areas of English Composition, Oral Communications, and Humanities.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
“These are not the [data] you are looking for.” by Tim Stanley, Laura Jimenez-Snelson
In the original Star Wars (1977), Obi-Wan Kenobi changed the minds of the Storm Troopers with the wave of his hand. How often do we wish we could simply wave a hand and persuade decision-makers to focus their attention on the data that makes the most sense, fits their long term goals, and will support continuous improvement? Often IR officers and analysts are put in a position to help answer decision makers without fully understanding the question they hope to answer, or even less, the strategic direction the data are intended to inform. This purpose of this discussion group is to explore ways that IR analysts can tactfully help guide decision makers into thinking about their data requests in more strategic ways.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Discussion Group
A Model For Assessing Program Success Across Delivery Modalities by Bob Blankenberger, Doug Franklin
The presenters will offer a model for comparing student learning outcomes in degree programs across delivery modalities. The model uses a combination of assessment of student learning and program evaluation techniques. As higher education moves away from traditional delivery modalities to embrace new ways to reach students, state and federal oversight of higher education has increased expectations of accountability, and one of those expectations is that consistent student learning can be demonstrated across modalities. Participants in this session will be provided with a model for structuring a program assessment process across delivery modalities based on program evaluation techniques. They will also be provided examples of how to clarify program goals and how to translate these into program-level learning outcomes, sample assignments for gaining data on student learning, sample rubrics for measuring student learning, and methods for comparing the results across modalities.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
A Path Analysis of the Impact of College Environment on Learning Outcomes by Shuguang Wei, Min Chen
The college impact study is particular important to IR professions and campus decision-making in undergraduate education reform nowadays. The presenters built a conceptual framework based on Astin’s I-E-O (Input-Environment-Output) Model and Tinto’s four conditions (Expectation, Support, Assessment and Feedback, Involvement) for student success, and put Tinto’s four conditions for student success into the ‘Environment’ part of the I-E-O Model. Using the new framework, the presenters conducted a campus wide survey to all undergraduates in one research university in China. A SEM-oriented path analysis will be conducted to answer how Expectation, Support, Assessment and Feedback influence student learning outcomes with the mediator of student Involvement. This session plans to contribute some new ideas and practices to the field of college impact study.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Academic Advising: Using CAS & NACADA Concepts to Drive the Assessment CAR by Felisha Shepard-White, Karin Carlton
Assessment of advising and the impact the advising process has on retention and persistence continue to be major concerns within higher education. In addition, practitioners are embracing the notion of advisement as both a pedagogical process and a developmental learning experience for students. These paradigm shifts have several implications for the institutional researcher and assessment specialist. This presentation examines the efforts of one 4-year institution in developing an assessment plan for its centralized assessment unit that utilizes the Council for the Advancement of Standards in Higher Education (CAS) and the principles of academic advising set forth by the National Academic Advising Association (NACADA).
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Actual Campus Involvement, Self-reported Engagement, and First-year Outcome by Wen Qi
This study combines institutional data with self-reported survey data from Mapworks and NSSE to examine the relationship between actual participation in campus activities, student engagement, and academic performance and retention among first-year college students. Dividing the freshman cohort into three groups based on their actual participation in campus events and student organizations, this study examines group differences in the survey completion status, self-reported engagement behaviors, change in self-reported behaviors at the beginning and the end of the first year, and the relationship between self-reported engagement data and first-year cumulative GPA and retention. The presentation will illustrate an empirical example of how researchers can utilize institutional records and self-reported survey data to understand first-year college student engagement and its relationship to academic success and retention.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Alignment of Global Outcomes for Assessment and Accreditation by Stephen Benton, Ken Ryalls
Widespread agreement exists among influential higher education organizations in global student outcomes deemed critical to successful learning. Systems are needed for gathering student data that provide evidence of student achievement of those outcomes. IDEA Student ratings of instruction (SRI) offer a valid, reliable, and improved means for collecting indirect evidence of such achievement. Viewers of this poster presentation will learn (a) which global student learning outcomes are assessed in IDEA SRI learning objectives, and (b) which instructor behaviors are most highly correlated with student self-reported progress on those objectives.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
An Assessment Tool Grounded in Pedagogy to Aid Student Learning by Dana Dalton
Ensuring the quality of course content and that students are learning across all instructional modalities are the primary functions of assessment. The need for better student engagement, student and academic support, and appropriate assessment of learning and performance remains at the forefront as e-learning continues as the fastest growing enrollment sector in higher education. More emphasis should be placed on constructing measurable assessments to evaluate cognitive, affective, and psychomotor domains for students in online and blended environments. A rubric has been developed to evaluate the structural elements of course design across all modalities with emphasis on e-learning.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Assessing Co-Curricular Student Learning Outcomes by Matthew Pistilli, Brandi Gilbert
This discussion will address assessing student leaning outcomes within the context of co-curricular and experiential learning environments. This arena provides rich data with regard to how students apply the information they learn in the classroom. The discussion will focus on how institutional researchers, and student and academic affairs staff can collaborate to assess student learning outcomes in these settings. Those attending will have an opportunity to share what it is their institutions are doing in this area, as well as tactics for assessing more-elusive constructs. In addition, methods for collecting and analyzing data will be shared, as will the relative merits of qualitative and quantitative research as it pertains to this topic.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Discussion Group
Assessing Science Literacy at the Course, Program and Institutional Levels by Briana Keafer Morrison, Lindsey Brown
Science literacy is a common learning outcome for institutions and programs, but it can be challenging to assess. This poster describes a straight-forward approach for assessing science literacy and using results to inform meaningful change. To determine if graduates leave with adequate preparation in science for decision-making and civic affairs, our university uses Science Literacy Concept Inventory (SLCI) in targeted courses to assess basic science literacy. As part of an associated interest group, faculty members discuss SLCI results and devise ways to target specific scientific misconceptions in the classroom.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Assessing Student Co-curricular Involvement Using Rubrics and Focus Groups by Susan Thompson
As part of the most recent reaffirmation of its Southern Association of Colleges and Schools’ Commission on Colleges' Accreditation, Texas State University implemented a Quality Enhancement Plan (QEP) that involved reconceptualizing and reorganizing the entry of new freshmen into a unified orientation, advising, registration, mentoring, and learning experience to foster a close integration of major and career planning through a single center. This presentation will share the tools and processes developed to assess one outcome specified in the QEP: A rubric to evaluate student involvement as recorded on a co-curricular transcript, and focus groups conducted to determine student opinions on how their involvement in organizations and co-curricular activities enhanced their educational and career goals.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Assessing Written Communication: Is Motivation a Concern? by Joseph Rios
As written communication has been identified as a critical skill for success in higher education, institutions increasingly demand that students’ writing ability be assessed as a learning outcome. To address this concern, this presentation will describe a new written communication student learning outcome assessment that allows for authentic writing tasks, while balancing psychometric quality. Next, a discussion will be presented on the validity concerns (e.g., low examinee motivation) related to low-stakes writing assessments that employ constructed-response items. This will be followed by a review of how examinee motivation is typically identified, and the implications of motivation for performance on constructed-response items. A study evaluating the effects of motivation on essay scores from the new writing assessment will then be described. The presentation will conclude with a discussion on approaches to strengthen examinee motivation under low-stakes contexts.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Assessment: Get the Faculty to Do It – Hey, They Like It! by C. Ellen Peters
The presenters of this session worked with faculty to find a way for faculty to enjoy engaging in discussions about educational goals and the need to change them. Revision of educational goals is generally met with a sigh and an eye roll, but given the pressure for higher education to justify our very existence, strong, well-understood educational goals are essential. At many institutions, assessment is carried out either by assigning the task to a committee, or through a top-down approach, and is politically fraught. The presenters will share how they engaged a large group of faculty in activity-based discussion groups to develop a revised set of education goals, and had fun doing it.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Beyond Grading: Assessing Analytical and Reflective Skills of Students by Ebenezer Kolajo, Felix Amenkhienan
In today’s innovative economy, employers consider student competencies in some critical skills more important than their college majors. Embedding assessment of student learning outcomes in graded examinations enables faculty to reconcile the pitfall of separating grading from assessment. This presentation will discuss a study that measured students’ performances in an accounting course over two semesters. Questions pertaining to two learning outcomes, analytical and reflective thinking skills, were embedded in examinations given during each semester. The students’ mastery of analytical skill remarkably improved, but the rate was gradual in the reflective thinking outcome. The presenters will discuss how an action plan with more practice assignments might accentuate improvement in these outcomes.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Beyond IPEDS: National Data for Institutional Effectiveness (IE) Insights by Tafaya Ransom, Lan Ma
IR and assessment professionals are increasingly called upon to provide evidence of their institutions’ outcomes and achievements relative to their organizations’ missions. This presentation will provide an overview of two distinct national data sources, the National Science Foundation’s WebCASPAR database and the National Student Clearinghouse’s StudentTracker Service. Examples of how these data sources can be used to generate institutional outcomes insights will be discussed, especially where institutions are interested in understanding their role in broader contexts (i.e., regionally, nationally, across particular institution types, or within student subgroups). By providing an overview of these national data systems and sharing our experiences leveraging them, this session will offer participants practical knowledge and tools for enhancing their IE repertoire.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Collecting, Analyzing, and Reporting on Data from Small Populations by Allison BrckaLorenz, Sarah Hurtado, Thomas Nelson Laird
Quantitative and survey research depends heavily on large sample sizes, but there are a variety of reasons why larger sample sizes may not be possible. Participants in this session will discuss common issues and solutions associated with assessing small populations of college students and instructors, with considerations for special subpopulations (gender variant, multiracial, etc.) as well as considerations for small institutions. Participants will also learn about and discuss administration issues related to small populations, such as increasing response rates and identifying special subpopulations; strategies for analyzing and communicating about results from small populations; and approaches for communicating validity and data quality from small sample sizes.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Continuous Academic Improvement Cycle for Competency-Based Education by Kurt Gunnell, Kayla Maxwell
This discussion will address Competency-Based Education (CBE) within the context of academic program review and improvement. Examples will be provided from Western Governors University where internal program assessments are defined, compiled, and used for program assessment as delineated in the structures of competency-based assessments. These topics will be used as a primer for group discussion. A list will be compiled of other methods competency-education institutions are able to perform and manage program reviews.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Discussion Group
Coordinated Surveys for Assessment of Undergraduate Research Outcomes by Stephanie Foster
Undergraduate research has been identified as a “high-impact educational practice” to improve student success, learning, and retention. Assessment professionals are increasingly being asked to demonstrate the success of these programs. Participants will discover how institutional surveys can be coordinated and merged with institutional data to understand student success in undergraduate research programs, and explore how assessment data could be used for improvement from the perspectives of different campus partners. This session will focus on how assessment professionals can partner with other campus entities to support more effective use of assessment data.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Crowdsourcing: A Unique Approach In Creating Assessment Rubrics for GE by Susan Brooks, Mary Jo Geise
Trying to get buy-in to change? Trying to create something that reflects "best practices"? This poster presentation illustrates how crowdsourcing was successfully used to create an assessment plan for a newly re-envisioned general education (GE) program. Through the involvement of individual stakeholders, one GE committee successfully launched a new CORE+ Curriculum for General Education program. Draft assessment rubrics were developed utilizing stakeholder voices from beginning through implementation stages.
In this poster presentation, we will provide the background of our initial processes of developing a new general education program and defining our newly adopted CORE+ Curriculum learning outcomes. Presenters will share examples of draft assessment rubrics to gain further feedback on their effectiveness in assessing the CORE+ Curriculum learning outcomes as well as hearing from other institutions what they are doing in terms of assessing their general education programs.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Data AnalyticTool For Reliable &Timely Decisions, A College Case Study by Jennifer Owin, Ann Keith
Data analytics an important area where organizations purpose their raw data giving users the ability to make timely and reliable data decisions within their organization. The process of developing those analytical measures is an exacting process that requires solid planning, consistent team work, and continuous feedback. This poster will highlight the analytical reporting tool the Mary Lou Fulton Teachers College's Internal Research and Data team developed to give users the ability to identify, monitor, and report on students within their college. This poster will highlight what we have done to develop this system, discuss some of the road blocks that were successfully navigated, and examine about future next steps.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Data as Eye Candy: Improving Our Data Visualizations by Sherry Woosley
Because 65% of people are visual learners, improving our data visualizations could have a significant impact on our ability to affect decision making. In addition to traditional definitions and approaches, this presentation will take an unusual approach by drawing on art and design traditions. We will discuss specific art principles and design elements as well as their applicability to data visualizations. Using these tools, we will take a concrete data example through various iterations to improve its impact.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Degree Qualifications Profile and Tuning: Student Learning Impact and IR by Natasha Jankowski, Jillian Kinzie, David Marshall
Pulling from a New Directions for Institutional Research issue on Degree Qualifications Profile (DQP) and Tuning, along with findings from a national impact study on institutional involvement with DQP, this presentation will provide an overview of the work of IR and serve as a foundation for future discussions by outlining what the DQP and Tuning are, how IR has been involved, and what the future might hold for IR in these efforts. Examples from the field will be supported by evidence from an impact study on DQP efforts. Further, presenters will outline the different roles IR has assumed and might take on to participate in institutional initiatives around assessment of student learning.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Designing Assessment Data Collection Systems with the End in Mind by Mary Jo Geise, Susan Brooks, Helen Schneider
How do we create a data collection system that balances accreditation requirements with faculty needs for data to help with continuous improvement in their programs? This discussion will address using institutional objectives to design an assessment plan which will measure value-added to student learning in the context of an outcomes-based general education program. Implementation of such a plan will include reporting capabilities to illustrate student growth and provide for continuous improvement.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Discussion Group
Direct and Indirect Effects of Engagement on Grades by Robert Gonyea, James Cole, Louis Rocconi
Grades are perhaps the best predictor of a attaining a college diploma. Using NSSE data and matched year-end grades from nearly 20,000 first-year and senior students enrolled at 42 institutions, the authors tested path models to determine the interrelationships among student background, engagement, and campus environment. Total effects on GPA show that time spent studying, the use of learning strategies, and experiencing faculty using effecting teaching strategies had positive overall effects on grades, while coursework involving quantitative reasoning had a negative effect.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Do Common Performance Metrics Reflect Institutional Effectiveness? by Giljae Lee, Aaron Horn, Darwin Hendel
Several raw performance metrics have been proposed to evaluate institutional effectiveness in promoting degree completion, such as the six-year graduation rate and total degrees awarded. However, it remains unclear whether raw metrics can be used to accurately identify institutions that actually add value to the student experience. This study uses a value-added measure to examine the frequency of misclassifying institutions as effective or ineffective when using nine unadjusted performance metrics. Longitudinal data were obtained from IPEDS for public four-year institutions (n= 558) to create a trinomial classification of effectiveness for each performance metric based on the deviation from mean performance: below average, average, and above average. The results revealed a relatively high percentage of institutions with effectiveness classifications that differed between the value-added metric and an unadjusted metric (25% to 49% misclassified).
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Exploring Relationships Among SRI Survey Items to Overall Effectiveness by Joseph Ludlum, Tris Utschig, Karin DeAmicis
Student survey data about teaching effectiveness continues to play a major role in promotion and tenure decisions on many campuses despite skepticism over its relevance and appropriateness. We analyzed relationships among survey items to overall course and instructor effectiveness. Statistical regression showed items about instructor ability to communicate and engage as the best predictors of overall teaching effectiveness. Overall course effectiveness was best predicted by items such as how well activities / assignments facilitated learning and how well coursework evaluated learning. An exploratory factor analysis was also conducted. For all models, student effort items created separate factors from teacher and course related items. Student preparation was independent of these factors. Future exploration of course survey data, and implications for use in faculty development, and policy and decision-making are discussed.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
First-Destination and other Post-Graduation Outcomes by Jerold Laguilles, Mary Ann Coughlin, David Troutman, Jessica Shedd, Kristina Powers, Lou Guthrie, Cate Rowen
In light of the recent focus on the College Scorecard along with the national discussion on the cost of higher education and student loan debt, the post-graduation outcomes of college students are becoming even more important as indicators of institutional effectiveness. This panel will focus on the first-destination outcomes of college graduates while also recognizing that other outcomes are also relevant across institutional settings. Panelists will discuss not only the importance of these outcomes but also the nuances of collecting and reporting on these outcomes across different institutional settings. The goal of this panel is to create a session in which an IR practitioner could gain beneficial and practical information on this topic regardless of institutional type. In other words, this session is by IR professionals for IR professionals on a key and important topic for all institutions of higher education.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Panel Session
First-year Academic Challenge: A Qualitative Exploration of NSSE Results by Tracy Rokas
This session explores results of a qualitative study conducted by Belmont University’s Office of Assessment and Institutional Research, inspired by results from the 2014 National Survey of Student Engagement (NSSE), exploring second-semester freshmen perceptions of rigor, and the contexts in which students indicate learning the most. Findings from this research suggest the presence of important challenges faced by faculty, particularly those working with students earlier in their undergraduate career, to provide rigorous opportunities for students to expand their cognitive and critical thinking abilities, while also providing enough transparency to ensure students are able to access, experience, and even enjoy the journey. This research adds to current knowledge about effective practices for student engagement, and provides novel insights into the delicate balancing act often required to inspire early student success in the classroom.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Gender Identity & Sexual Orientation: Survey Challenges and Lessons Learned by Allison BrckaLorenz, Sarah Hurtado, Jana Clark
Participants in this discussion will learn about and discuss the assessment of and conversations about gender identity and sexual orientation on other campuses, and the challenges and potential solutions for writing more inclusive survey questions about complex identities. Challenges and potential strategies for surveying, disseminating results, and talking about difficult or sensitive topics on college campuses will also be discussed. Finally, participants will learn about the engagement, perceptions of campus support, and satisfaction, of students with varying gender identities and sexual orientations from a longitudinal, large-scale, multi-institution survey of students at four-year colleges and universities.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Discussion Group
Graduate student surveys: Assessment landscape, challenges, and solutions by Allison BrckaLorenz, Bridget Yuhas, Thomas Nelson Laird
Most assessments of the graduate student experience are institution-based and largely focused on satisfaction exit surveys. Very few of these surveys touch on the experiences of graduate students as instructors or teaching assistants. Further lacking are trends from large-scale surveys or national data on the experiences of graduate students, which is especially concerning given the increasing number of graduate students entering the classroom to teach undergraduate students. Participants in this session will discuss how the assessment of graduate student experiences varies on different campuses, as well as examine the challenges and potential solutions associated with assessing these experiences. Finally, participants will learn about one large-scale survey of graduate student experiences, see a selection of results gathered from two years of administration of this survey, and hear about how institutions have used these results for graduate student professional development purposes.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
High Impact Learning Experiences, Academic Engagement, and Success by William Armstrong
This session will focus on the effects of student participation in high impact learning and educational practices. Using course enrollment data from transcripts, high impact learning courses such as internships, undergraduate research, independent study, and education abroad will be analyzed for effects on academic engagement, success, and satisfaction with the undergraduate experience. Particular attention will be given to the net effects of such participation while controlling for entering student academic preparation variables. Findings are disaggregated by student demographic and socio-economic groupings.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Higher Ed Production Function: Performance, Accountability, & Efficiency by Justin Shepherd
The changes in higher education during the last forty years require that existing organizational frameworks be re-examined to address the modern challenges of performance budgeting and funding. This paper incorporates previous production function models with contemporary literature on higher education organizations and finances to develop a revised model that more appropriately fits the current situation of budget cuts, rising tuition, and accountability reforms. The proposed framework posits that institutional performance is a function of its inputs and funding reforms based on these performance outcomes may lead to changes in institutional behavior related to input acquisition rather than instructional processes.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
High-Impact Practices, Learning Outcomes, Retention, and Graduation by Yanli Ma, Juan Zhou
This session examines the relationship between student participation in high-impact practices (HIPs) and student learning outcomes, satisfaction, retention, and graduation. Data are derived from institutional student information system at a private liberal arts college and a public technology college based on NSSE results related to HIPs. Specifically, we explore whether participation in HIPs in the first year is related to satisfaction with overall educational experience, and to first- to second-year retention, and whether service learning is related to perceived personal and social gains. We also analyze whether participation in HIPs is correlated with satisfaction and graduation for seniors. This session will be of particular interest to AIR members from small to medium-sized or specialized 4-year institutions.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Incorporating economic data into the examination of alumni outcomes by Natalie Wright
Examining graduate outcomes is important both for institutional improvement and regulatory compliance. As the wider economic environment exerts a strong influence on graduates’ post-graduation outcomes, it is important to include macroeconomic information in any examination of post-degree outcomes. This presentation demonstrates how macroeconomic information can be incorporated into alumni survey data to provide a more accurate picture of how an institution’s educational programs affect graduates’ outcomes. Participants will learn how to obtain macroeconomic information and how to add this information to predictive models to examine graduates’ income and other important outcomes.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Involving Online Students in High-Impact Practices by Rong Wang, John Zilvinskis, Amy Ribera
Using a large-scale survey of student engagement, this study examined the extent to which taking all online courses affects senior students’ participation rates in high-impact practices (HIPs), such as internship and study abroad. Online students’ perceived gains in knowledge, skills, and personal development were also examined by whether or not they participated in an HIP. Overall, findings revealed online students’ participation rates in HIPs were relatively lower than students who did not take all of their courses online. Of the six HIPs, online students engaged most in a service-learning experience as part of a course requirement and least in study abroad. Online students who participated in a HIP reported greater gains in knowledge, skills, and personal development compared to online students who did not participated in a HIP. This study suggested institutions should pay special attention to the needs of online students and develop strategies for promoting their HIP participation.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Link Assessment & Planning to Ensure Effectiveness & Student Success by Shayla Moore Prince, Sesime Adanu
This presentation is aimed at addressing the link between assessment and strategic planning in measuring institutional efficiency and effectiveness. Further, demonstrate how these variables impact resource allocation and student success. These are very important to the field of Institutional Research when viewed from the perspective that knowledge and understanding of these issues are critical to practitioners in assisting their institutions attain not only continuous improvement, but also accreditation. This helps ensure public confidence in the quality educational programs delivered to students at the institution.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Discussion Group
Mapping & Assessing Institutional Outcomes for Graduate Health Professions by Melanie Davis, Jane Hawthorne
This poster presentation addresses the issue of assessing institutional outcomes in an effort to improve student learning outcomes and make data-driven decisions regarding program and curriculum review, which in turn fulfills accreditation assessment requirements. The objectives of the poster presentation are to 1) present a process for incorporation of Core Professional Attributes across the graduate curriculum, 2) describe mapping of the CPAs in the curriculum, and 3) describe the impact of assessing the CPAs on student learning outcomes.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Maximize Gains from NSSE: From Survey Administration to Results Sharing by Yang Zhang, Sonya Smyk, Kelly Jung-ts Lin
Two salient issues are associated with the NSSE survey: the national problem of declining survey response rates, and the challenge of turning NSSE survey results into actionable initiatives. Institutional researchers at a large flagship research university share their experience of how they achieved a significant increase in the NSSE response rate from 16% to 32% through executing a comprehensive survey marketing plan, and strategies they used to integrate NSSE results into the university’s strategic plan, and efficiently engage units at different levels during that process.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Measuring Programmatic Impact: Employment Commission & Licensure Databases by Jolene Hamm, Bernnadette Knight, Aris Bearse
The session will explain the use of a triangulated methodology whereby the data is captured from a series of databases, the graduate exit application, and the graduate employment follow-up survey. By using these three methods, the institution receives a clear picture of the employment of the recent graduating class. This methodology assists in answering the critical questions for an institution such as are students getting jobs? For those eligible for licensure, are they receiving their license? Are graduates prepared by the education the institution has provided them?
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Measuring the Success of Transfer Students: A New Metric Evaluated by Gerald McLaughlin, Richard Howard, Josetta McLaughlin, Jacqueline McLaughlin, Sandra Whalen
Institutions are assessed based on the success of their students. A growing number of concerns have been raised however about student success being typically measured as the graduation rate of a First-Time Full-Time Degree Seeking cohort. While the Student Achievement Measure has been proposed as an alternative, this research looks at another definition of transfers – those with one year credit from a community college. These data were reported to CSRDE by 117 senior institutions for cohorts detailed by gender and by ethnicity. These data are investigated for reliability and validity for use in describing institutional outcomes. Next steps are discussed.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Navigating the Path to Aligning Performance Metrics to Assessment by Sundra Kincey, Franz Reneau, Mark Howse
Institutions are challenged to develop creative ways to not only meet university-level goals related to graduation and retention, but also state mandates if they want to be able to compete in a global society. As such, institutions are now being mandated to achieve surmountable goals that are now being tied to performance metrics and possibly funding. This presentation will provide an overview of how one institution has revamped its assessment process to align programmatic assessment to state-mandated performance metrics to improve performance on key indicators by which institutions are evaluated.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
New Data on Academic Surveys from the National Science Foundation by Emilda Rivers, Lynn Milan, John Jankowski, Ronda Britt
The National Science Foundation has a legislative mandate to "provide a central clearinghouse for the collection, interpretation and analysis of data on scientific and engineering resources and to provide a source of information for policy formulation..." To carry out this mandate, NSF's National Center for Science and Engineering Statistics (NCSES) conducts a variety of surveys of the academic sector as well as workforce surveys of college graduates. During the past several years and continuing into the future, NSF has engaged in survey redesign and new survey design efforts. The speakers will report on the new recently fielded Early Career Doctorate Survey (ECDS) Pilot, redesign issues considered for the Survey of Graduate Students and Postdoctorates (GSS), and the latest data from the Survey of Earned Doctorates (SED), the Higher Education R&D (HERD) Survey and Research Facilities Survey, and the status of the Nonprofit Research Activities Survey that will be pilot-tested this summer.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Next-Level Learning Outcomes Assessment: Varying Perspectives, Common Goals by Ross Markle, Javarro Russell, Natasha Jankowski, Kathleen Wise, Charles Blaich
As student learning outcomes assessment has grown in both prevalence and importance, many institutions still struggle in several areas. First, building an effective assessment process requires a great deal of skill, particularly when institutions (and institutional research offices) have limited resources. Second - and perhaps more importantly - the field of assessment is shifting to a focus on improvement, rather than mere accountability. This requires a continuous process of assessment, rather than an episodic demonstration of institutional achievement. In this session, leading experts in the field will bring multiple perspectives to help attendees understand how institutional researchers can position themselves in this process. In addition, they will discuss best practices and cutting edge advances in higher education assessment that attendees can take back to their institutions.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Panel Session
On-site Visit 101: What They Didn't Teach You in Graduate School by Alexandra Anderson
This discussion will address preparing for a successful on-site accreditation visit. A successful accreditation on-site visit depends upon months of preparation and requires numerous details to consider from the welcome session to the exit interview. This topic is important because many administrators find themselves leading the on-site visit preparation process with little previous experience. The objective of this session is for participants to recognize the elements for planning an on-site visit. Participants will be members of a discussion regarding on-site visits and will be encouraged share their experiences.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Discussion Group
Performance Based Funding: Impacts on student success by Peter Trumpower
With the recent release of the College Scorecard, colleges are under increased scrutiny and being held accountable for student loan default rates, graduation rates, and other student success-related metrics. Various Performance Based Funding models have been implemented in several states, with more states to follow. This presentation details how Completion by Design grant-funded work was leveraged to address the student success factors used in the Ohio PBF model. Comprehensive changes to college practices and policies resulted in improved course persistence, redesign of placement practices and the developmental curriculum, and a record number of credentials awarded, despite declining degree-seeking student enrollment.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Promoting Faculty Engagement with Assessment at A Large Research University by Felix Wao, Chih Ming (Ryan) Chung
Most institutions have processes in place to gather program assessment reports. However, existence of a process may not equate to having a sincere culture of assessment. This discussion focuses on how a large research institution implemented initiatives to increase faculty ownership of and engagement with assessment by seeking information directly from faculty about their needs and interests, addressing those needs and interests and, most importantly, developing institution-wide activities aimed at cultivating a culture of assessment.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Discussion Group
Student Academic and Civic Engagement: Does It Matter to Career Success? by Tongshan Chang, Xiaohui Zheng, Susannah McCormick
This study uses the University of California undergraduate experience survey and alumni salary data to examine relationships between student academic and civic engagement and employment outcomes after graduation. The study is important because very limited research has been done to explore the effect of student engagement in various activities on career success. We hope that the session will provide information to help improve theories and enhance institutional practices in this field. We particularly hope the findings will help institutions to create rigorous programs and provide opportunities for students to improve learning mindsets and develop personal qualities required for career success.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Student Satisfaction Data: Its Value, and How To Use It. by Kim Oren
Hundreds of colleges currently collect student satisfaction data, and utilize those data in outcomes assessment programs, strategic planning initiatives, and other key institutional operations. Especially at institutions at which local recruiting is essential, it is important to facilitate positive word-of-mouth support. Satisfied students are the foundation of such support. This session will describe intensified activities and accomplishments toward increasing the quality and quantity of stakeholders’ actions in response to student satisfaction data, over an academic year at Mid Michigan Community College (MMCC). Such responses can include complex and resource-demanding improvement efforts, but also relatively simple and inexpensive communication campaigns addressing students’ misperceptions revealed by the data. The session will highlight discussions among MMCC faculty and staff on the importance, value, and limitations of satisfaction data in facilitating improvement efforts.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Student Satisfaction Data: Overcoming Barriers and Facilitating Use by Julie Bryant, Laura Fingerson, Katherine Coy, Stacy Vlahakis, Marcia Finch
Student satisfaction surveys are a key component in assessing the student experience. However, just collecting the satisfaction data is not enough. Representatives from four-year and two-year institutions, online and on-ground, will share how they have overcome barriers for administering satisfaction assessments and will provide examples of how student satisfaction data are being utilized for institutional change and dialogue. They will share how their IR offices are partnering with other college departments to actively use the data for planning and evaluation purposes. This session will combine the expert perspectives of the panelists with audience observations to identify how IR offices can effectively use student satisfaction survey results to improve their institutions.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Panel Session
Taking Your Campus Pulse: Effective Strategies for Assessing Campus Climate by Sherry Woosley
With the recent publicity and focus on campus climate assessments, many institutions are wondering if they should be conducting one. Campus climate studies are often a challenge, so understanding the issues prior to starting one is an important process. This session will discuss strategies to lay the ground work for a successful campus climate study, covering such topics as planning for and conducting the study, avoiding pitfalls, framing results, and other key considerations for assessing campus climate.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
The Institutional Effectiveness Paradigm: From Assessment to Improvement by Benji Djeukeng
Institutional effectiveness (IE) is important to AIR members because data-based decision making is at the core of it. This discussion will (a) apprise attendees of reasons higher education has been challenged by IE, (b) provide tips from IE implementation at a different institution, and (c) allow attendees to express their IE challenges and successes.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Discussion Group
Tips from the retrenchment road: IR’s role in institutional review by Terra Schehr
Like many campuses, Loyola has engaged in a large-scale institutional review to identify areas of competitive advantage, efficiency, and potential new revenue. This review and IR’s role in it will be described. Key decisions related to the review’s organization and process (e.g. use of consultants, setting financial targets, and use of governance) and the impact of those decisions on IR will be highlighted. Advantages and challenges for IR’s involvement in the process will be shared as will thoughts with the benefit of hind-sight. That is: if we could do it over again, would we have made the same decisions?
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Use Planning & Assessment to Measure Effectiveness & Learning by Shayla Moore Prince, Sesime Adanu
This group discussion is aimed at helping Institutional Effectiveness and Research professional identify strategies to keep their campus engaged in continuous improvement activities even after their accreditation visit. Specifically, this session will talk about how to keep faculty and administrators engaged in assessment after the accreditation visit. With this approach institutions are able to use their strategic plan as a roadmap to guide institutional decision making. Attendees will be able to: 1) Employ strategies to keep campus constituents engaged in assessment after the accreditation visit. 2) Implement an assessment process based on the strategic plan.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Discussion Group
Using Dual Enrollment Programs to Increase Accessibility by Misty Rodeheaver
Many four-year institutions explore ways to increase enrollment numbers while still attracting "college ready" students. As one of the many ways in which to do so while eliminating accessibility barriers, Buffalo State SUNY implemented a dual enrollment program, specifically targeting students that have been known to struggle on the collegiate level and to work with them for the entirety of their high school career. While only in its second year, the preliminary data not only suggest an increased level of access for an underserved population, but the data also indicate improved classroom performance as well as a majority deciding to attend Buffalo State SUNY upon completion of their high school program. This poster will present the viewers with three main talking points: 1. A brief overview of the program, 2. A description of the accessibility data collected, and 3. The plan for future program evaluation.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation
Using NSSE as a Catalyst for Improvement: Lessons from the Field by Jillian Kinzie, Fang Du, Sarah Owens
One of the more challenging phases of assessment is taking action on results. This session explores the latest field-tested lessons from nearly two dozen institutions that have successfully used the National Survey of Student Engagement (NSSE) to improve undergraduate education. Representatives from two institutions will discuss their use of data to improve the first year experience and to engage departments in enhancing student learning by creating dashboard displays, infographics, and customized reports. The session provides an opportunity to learn about approaches employed by institutions that have made effective use of results, and to discuss proven strategies for taking action.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Panel Session
Using Predictive Modeling to Assess Employability, Salary, and Satisfaction by Max Wartel
This presentation will discuss results of predictive modeling of variables contributing to various post-graduation occupation outcomes for undergraduate students. Outcomes discussed include employment, compensation, and satisfaction. In addition, results of more granular analyses conducted using groupings of majors constructed through cluster analysis will be discussed. Analysis of predictive variables was conducted using national data collected using The Outcomes Survey, a standardized survey instrument used at nearly 140 institutions across the country with over 44,000 undergraduates in the population, from students who graduated during the 2014-2015 IPEDS year as of 6-months post-graduation. The method discussed may be applicable on an institutional level and the presented findings have prescriptive potential. Participants will learn about the method of analysis, the findings, and prescriptive applications. Preliminary 2015-2016 IPEDS academic year data will also be discussed.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Speaker Session
Variations in the Instructional Behaviors of Graduate Student Instructors by Rong Wang, Allison BrckaLorenz, Thomas Nelson Laird
Graduate students who teach, or graduate student instructors (GSIs), play a significant role in influencing undergraduate students’ learning experiences and outcomes. Using multi-institution data from a large-scale survey of graduate student teaching practices, this project aims to explore the extent to which instructional behaviors vary across GSIs’ demographic background and the types of courses they teach. The extent to which GSIs employed effective teaching practices, and relationships between setting clear course goals and employing effective teaching practices among GSIs will also be examined. This poster presentation is important to AIR members because the findings of this study will not only help graduate schools to ensure the quality of the classes taught by GSIs, but also provide useful information to graduate faculty advisors and academic support staff for preparing future faculty members. Recommendations for ways GSIs may enhance their teaching are provided.
Topic Area: Assessment: Accountability, Institutional Effectiveness, and Accreditation   |   Format: Poster Presentation

Data Analysis and Research Methods for IR

(CANCELLED) Moving beyond averages: Quantile regressions of student performance by Fernando Furquim, Kristen Glasener
Research into the relationship between standardized admissions tests (such as the ACT and SAT) and college performance has yielded mixed findings. Researchers have focused too narrowly on these relationships on average, conditional on control variables. Examining how standardized tests affect the distribution of college performance may tell a different story. We address the limitations of previous research by employing quantile regression techniques, and using a large national dataset that includes detailed student-level information and important contextual factors. The goals of this session are to highlight limitations of previous modeling techniques and the additional insight provided by quantile regression.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
(CANCELLED) The Impact of ‘Late’ Registration on Student Outcomes – A 360⁰ Approach by Jacob Williams
As the college seeks to develop initiatives that improve student success and persistence, supporting decision making around registration policy and practices becomes an important IR function. Past research from Ivy Tech indicated that late registrants have lower retention rates and course success. To this end, Ivy Tech has a 360 degree research plan that examines the causes for ‘late’ registration and the impact it has on student’s academic outcomes. With students self-identifying their reasons for delay, it allows us to tie student outcomes to the circumstances and/or behaviors that lead to increased risk of failure. Session participants will learn about one institution’s process for developing a 360 approach on a student success topic, as well as discuss specific analytical methods and issues of policy and practice in applying research on registration dates.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
10 New IPEDS Insights as Revealed from a CAT-Scan Perspective by Jeffrey Cornett, Brittany Resmann
IPEDS has broadened its performance tracking to monitor four separate cohorts over an 8-year horizon. These improvements are important, but other insights can be gained by studying the student experience in greater depth. CAT-scan graphs display all student outcomes at all cohort points in time. Ivy Tech of Central Indiana (a 1+3 college) and Seminole State College of Florida (a 2+2 college) have developed CAT-scans and dashboard comparisons that track performance across all 4 new IPEDS cohorts. They will present their findings in relation to 10 proposed IPEDS insights – some of which will become apparent as a result of the new IPEDS approach, and others that require further research beyond IPEDS requirements.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
A Closer Look at the Trillion: Default Trends at Iowa's Community Colleges by Colleen Campbell, Nick Hillman
Outstanding student loan debt totals more than $1.18 trillion, but who defaults, and how can data be used to prevent this outcome? This session will provide attendees with an in-depth look at the findings and recommendations from a recent report published by the Association of Community College Trustees, which analyzed Department of Education and institutional data from all 16 community colleges in Iowa. The report’s authors will review the report’s findings, and discuss how IR professionals can analyze their institutions’ data, how to use their findings to drive policymaking, and how to integrate solutions across campus units.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
A Comprehensive Understanding of the Impact of a Change of Major on College Students by Hyejin Choi, Meihua Zhai, Manuel S Gonzalez Canche
A change of major is a complicated issue for institutional researchers and administrators to deal with. Despite its impact on students and institutions, its complexities have long prevented scholars from exploring the topic comprehensively. This study will reflect the dynamics and complexities of change of major and its impact on student retention, time to degree, and academic outcome. The study will use survival analysis and propensity score weighting so that the audience might gain a better knowledge of the issue, as well as possible statistical tools for analysis.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
A Difference of Two Years: College Pathways and Outcomes for Latino Students by Jing Feng, Ellen Sawtell, Sarah Leonard
National data have shown that Hispanic/Latino students are more likely to enroll in community colleges as their first postsecondary institution than any other racial/ethnic groups. The current study is conducted to examine patterns of transferring among Hispanic/Latino students who used community colleges as the entry point to their post-secondary education and how those patterns are related to degree attainment. The study looks at College Board data (student records on PSAT/NMSQT, AP, and SAT) and National Student Clearinghouse data. Participants will learn about the relationship between participation and performance on assessments and students’ post-secondary education experiences and outcomes.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
A Framework for Addressing a Data Request with IPEDS Benchmarking Data by Leslie Odom, Jessica Lillegaard
Institutional benchmarking is a common activity for many colleges and universities. These requests are typically received by institutional research (IR) offices to determine the most relevant and comprehensive data sources to use and potentially replicate the analysis year after year. For this session, the presenters will provide a benchmark framework based on one institution’s experience using the IPEDS Data Center. Topics will include explaining why IPEDS data are an appropriate benchmarking data source, summarizing the primary IPEDS Data Center features that were utilized, displaying the iterative analysis process to finalize the request, and illustrating process improvement enhancements.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Actionable Data For Free: Transforming Public Data into Useful Information by Bridgett Milner
Dashboards made with publicly available education data allow institutional researchers to assess the higher education landscape, and their own institution’s position within it. It takes just a little effort to create useful and informative displays. The presenters will share their dashboards, discuss their creation and use, and provide participants with useful takeaways. They will also discuss how they have linked their own institutional data to some of the publicly available data to produce even more informative displays.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Aligning Post-Graduation Outcomes: The Trifecta Approach by Heather Kelly, Matthew Brink
Demonstrating outcomes for college graduates is more important than ever. Where do your graduates get jobs? How much money do they make? Where do they go to graduate school? These are all questions asked by prospective students, parents, legislatures, and the federal government. This speaker session will highlight strategies being employed by a public research university to meet the National Association of Colleges and Employers (NACE) guidelines, including NACE’s recommended three pronged data collection approach to obtain increased knowledge rates leading to a career outcomes rate. NACE understands the importance of collaborating with IR professionals. Accordingly, the Assistant Executive Director of NACE will discuss their First-Destination Survey and NACE’s advocacy role in related key legislative issues.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
An Integrated, Program-Level Approach to Enrollment Planning by Ryan Johnson
This session will highlight UCR’s new comprehensive approach to long-term enrollment forecasting: a method that combines an integrated planning process with innovative statistical modeling. After program-level enrollment targets have been set through communication between central planners and academic departments, an interconnected network of statistical models evaluates how realistic those targets are, and exports the enrollment management strategies required for every academic program to hit its target over the period of time being forecasted. Attendees interested in developing a similar model will gain insight on where to begin that process.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Applying Item Response Theory to Examine Extreme Survey Response Style by Xiaolin Wang, Amy Ribera
Response style effect is a well-known limitation of surveys. By applying a two parameter logistic item response theory (IRT) model on extreme response style (ERS) to the National Survey of Student Engagement (NSSE), this study intends to reveal how demographic factors are related to the ERS tendency in the survey in order for wiser interpretations of the survey results.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
Assessing the Assessments – Using Analytics to Support an Early Alert Model by Mike Krywy
RRC’s Strategic Enrolment Management (SEM) plan includes an Early Alert initiative designed to identify and support at-risk students early in their program. Student triage is based on using grades digitally recorded in the Desire 2 LEARN Gradebook, along with observations by instructors to identify at-risk students and triage them to supports. This presentation will focus on the analytic techniques used to support and evaluate the initiative, including assessing correlations between early assignments/tests relative to performance on high stakes exams, identifying “killer courses” and their characteristics, and determining the extent to which “assess early, assess often” practices are being used.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Calculating Post-collegiate Earnings: FL, NY, & TX Explore Wage Record Data by Colin Chellman, Christy England Siegerdt, David Troutman
Over the past several years, institutional researchers have been struggling with identifying possible data sources and establishing methods to measure students’ annual earnings following their college experience. At the same time an increasing number of data sources have become available, ranging from self-reported salaries in Payscale to unemployment insurance wage records from state labor departments to income tax returns from the IRS via the College Scorecard. Researchers from public systems in Texas, New York, and Florida will discuss how student earnings vary by source, best practices for calculating earnings, and the benefits and limitations of each source.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Panel Session
CIRP Users’ Group, Survey Administration & Research, Longitudinal Data by Ellen Stolzenberg, Dominique Harrison
This discussion will address survey administration and informing institutional decision-making within the context of the CIRP suite of student and faculty surveys. Given the highly variable nature of institutional research and assessment, HERI offers customizable surveys so that institutions can make the most of their survey administration and subsequent use of data. The AIR Forum allows CIRP users (or those interested in using CIRP surveys in the future) to gather and discuss their successes and struggles in collecting and using survey data in their short- and long-term assessment plans. The central theme of the discussion is best practices in survey administration and informing institutional decision-making.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
Cost of Completing an MPH Degree by Christine Plepys
Data available on the cost of completing a master's degree in public health (MPH) is very minimal. The Association of Schools and Programs of Public Health devised a methodology to answer this question that compares apples-to-apples among ASPPH's membership to inform internal member benchmarking and decision-making efforts. This issue is important to AIR members as the federal government, internal administrators, parents, and society at large seek solutions to the rising costs of higher education. Viewers of this poster presentation will learn about a methodology used to provide comparison cost of degree data and learn the cost of the MPH degree and how it compares across different types of institutions.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
Cui Bono? How Employee Fringe Benefits Contribute to College Costs by Braden Hosch
This session illustrates how the costs of employee fringe benefits contribute to costs of higher education institutions and play a part in rising costs for students. Using data from the IPEDS Finance Survey and drawing upon methodology from the Delta Cost Project, this study investigates employee fringe benefits costs within the core educational mission (the education & related cost components) and shows how much these costs represent per student. Participants will see how these costs vary by state, sector, and over time and will understand how to examine these costs at their own institutions.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Data-driven approach to dropout prevention in Japan by Tomoya Hashimoto
In Japan, the rate of post-secondary dropouts has increased recently. Nearly 3 percent of all undergraduate students (about 79,000 students out of 3 million) dropped out in the 14/15 academic year. This number is much lower than that in the United States. However, since most Japanese people see dropouts as failures, the number is not low for Japan. Researchers in the United States have accumulated theory and practice using a date-driven approach to dropout prevention. Conversely, the accumulation of such theory and practice are insufficient in Japan. Moreover, it is difficult to apply knowledge of the topic by comparing the United States and Japan because the educational systems and environments in the two countries are considerably different. The objectives of this presentation are to discuss how we can apply a wealth of theory and practice of a date-driven approach to dropout prevention in the United States to a country like Japan, which seeks to accumulate knowledge on the topic.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
Decision-making with Analytics: Master the Complicated, Explore the Complex by Laura Blasi, Daryl Davis
Prepare to guide others through analysis using data visualization – in this case Tableau. Support for user decision-making is critical as more sophisticated questions can be explored using these tools. It is tempting to ignore all but the most basic tables, rejecting anything complicated, and user expectations often drive the data design. This sacrifices our collective ability to address complex issues. We will discuss key moments in the development of visualization techniques; analyze a case study of the integration and support for learners; outline related observations (from CCSSE N= 2,733); and adapt training materials that will be shared.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Defining Your Student Veterans and Assessing Their Outcomes by Rebecca Wood, Waddell Herron, Kristina Powers
Do you find the complex affiliations among students with military ties difficult to define and challenging when assessing this student group’s outcomes? As demands increase to meet new federal requests specific to military students, respond to military student ranking surveys, and to conduct analyses, IR professionals need to be able to disaggregate military student data to not only ensure consistency of the data but also accuracy in the tracking, reporting, and assessment of this student group. This session is designed to help IR professionals identify the varying affiliations within their military student population in order to better define and disaggregate within this student group for internal and external purposes. This session will also present the process and results of a research study involving two and three-year veteran student transfer cohorts and their retention, persistence, and graduation rates.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Designing, Validating and Successfully Implementing a Satisfaction Survey by Chih Ming (Ryan) Chung, Felix Wao, Lindsey Tate
The University of Oklahoma Office of Academic Assessment developed/administered a short student satisfaction survey (Spring 2015). Factor analyses (EFA, CFA, and SEM) validated results (N = 6,080), presented for use in institutional planning. This case study demonstrates the utility of in-house survey design to inform campus-wide decision-making. Upon completion, participants should be able to articulate benefits in designing/administering an in-house student satisfaction survey; identify methods for designing and validating home-grown survey; describe the use of survey results to inform institutional decision-making; discuss methods for adapting an existing survey for use in their institutions.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Determining the social value of occupations and related academic programs. by Brian Perry, Elizabeth Riesser
Understanding the economic value of a college credential is a question asked by many researchers and policymakers. Preparing for an occupation that’s growing and/or in demand can make students more employable and contribute to higher earnings. However, exclusive reliance on wage and demand data in evaluating occupational outcomes may miss the critical role some jobs play in their communities. This session will discuss the social value of occupations. The discussion will include defining social value, how we can measure it, and how it can aid our understanding of educational outcomes.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
Emotional Intelligence Assessment and Induction for First Year Teachers by Carrie Lloyd
In higher education and research on teacher education, specifically first year teaching, little is known about emotional stressors and how to significantly decrease teachers’ stress levels. As demands increase, teachers need specific skills to help them cope with those demands. A one-year emotional intelligence induction curriculum and support was offered to first year teachers to align with an emotional intelligence assessment called the Six Seconds Assessment (SEI). Quantitative and qualitative analyses measured the effect of this induction on emotional intelligence and stress.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
Examining Gaps in Aspiration, Access, and Completion by Parental Education by Mark Umbricht
Previous literature states that students whose parents have lower levels of education are less likely to aspire to, access, and complete a college credential. As these students, typically defined as first-generation students, are at less likely access and persist higher education, they are of particular interest to many institutions. However, most of the literature on this student group uses outdated data, such the National Education Longitudinal Study of 1988. This study will use two more recent nationally representative NCES datasets, the Educational Longitudinal Study of 2002 (ELS) and the High School Longitudinal Study of 2009 (HSLS), to examine changes in the first-generation population between 2002 and 2009, and identify gaps in aspirations, access, and completion by parental education.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Examining Individual and Campus-Level Engagement with Diversity Curriculum by Hannah Whang Sayson, Adriana Ruiz Alvarado
Considering today’s diverse student population, institutions must consider how to best foster students’ sense of belonging and a healthy campus climate—related outcomes linked to academic success. Diversity-related courses, now required by several schools, offer a potential means to those ends. While previous research has demonstrated the impact of curricular diversity experiences on individual students’ development, less is known about the impact of these experiences in the aggregate or about benefits that students perceive or receive themselves from their peers participating in diversity programming. This session presents a study that examines the impact of diversity related coursework on two related outcomes: students’ sense of belonging and experiences with discrimination and bias. In addition to main effects, the study highlights how program impact may differ based on individual student characteristics as well as aggregated student measures and campus composition.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Faster Research: Data Cleaning in R by Matthew Rysavy
Decision trees, K-means clustering, support vector machines? First things first, clean data. This presentation will show how R can be leveraged in order to automate and transform data so that researchers can get to modeling faster. Additionally, it will discuss how data cleaning fits in an overall data strategy alongside data governance. Several real world examples will be used.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Get them to Click it! Increasing Online Survey Response Rate by Chih Ming (Ryan) Chung, Felix Wao
In this interactive presentation, the presenters will be addressing challenges in administering online surveys and offering 20 strategies that may help you to improve online survey response rates. At the end of this session, you should be able to identify several strategies to utilize in your own online survey data collection.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
Going test-optional: The effect on diversity, admissions, and rankings by Kyle Sweitzer, Emiko Blalock
An increasing number of institutions have gone “test-optional” in admissions, not requiring applicants to submit standardized test scores (SAT/ACT). The trend has garnered much attention in the media, but surprisingly few analytical studies, and none directly assess the impact on an institution’s ranking. The objectives of this session are to explore how the impact of going test-optional varies across institutions in terms of admissions metrics and rankings, and how institutions do not necessarily experience the often-stated desired effect of increasing student diversity. We also discuss the history and rationale behind the test-optional movement, as well as the caveats and limitations to studying the impact of implementing a test-optional policy. This analysis should be important to AIR members due to the increasing number of institutions across all Carnegie classifications that have recently chosen to go, or are considering going, test-optional.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
How a psychological research approach can advance the effectiveness of IR by Sarah Hailey
Academic research psychologists who choose a career in IR enter the field with extensive training in research design, statistics, interpreting and reporting results, and knowledge of human decision making and behavior. This expertise can be leveraged to advance the capacity of IR offices to pursue more rigorous research endeavors and to shift the function of IR from reporting to providing essential decision support. The discussion will be led by IR professionals who represent diverse experience levels and institution types and invites IR professionals from all backgrounds to discuss the opportunities, challenges, and solutions for IR professionals trained in psychological research methods.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
How to Not Rely on Student Self-Reports: Estimating Classroom Effects by Serge Herzog
To estimate the effect of the classroom environment on academic outcomes, this study uses direct measures of peer attributes (i.e., academic preparation, gender composition, ethnic/racial diversity), class size, and instructor type based on institutional matriculation records at a public research university. Estimated effects on first-year student academic performance (grades, credits completed) and enrollment persistence control for students' socio-demographic profile, pre-college preparation, academic motivation, educational goal, and students' financial aid profile (type and amount of stipends received, amount of unmet need). Findings are based on weighted-sample regression models and compared to results from studies using student self-reported data.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Identifying Inclusive Cohorts in a Community College Setting by Ian Burke
This session investigates whether “full-time, first-time, fall” student cohorts are representative of broader student populations. To address this issue in context, the demographic composition of a community college student cohort including students entering in the Spring semester and at less than full-time status. The session also presents the results of a binary logistic regression model relating credential attainment to cohort inclusion criteria as well as transcription of prior learning credit (PLA) and registration for developmental education (DE), as the research was conducted in the context of evaluating TAACCCT grants effecting changes in PLA and DE policy at the presenter's home institution. Significant interactions between cohort inclusion criteria and student academic history are discussed. Attendees concerned about discrepancies between College Scorecard graduation rates and actual institutional performance may find this session of interest.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
If the major doesn’t fit, will they quit? Understanding major persistence. by Krista Mattern, Ty Cruce
Although exploration of different courses of study is an important part of students’ development during college, guiding entering students toward a good fitting major both in terms of their interests and academic preparation can shorten their time to degree while increasing their chances of having a successful, rewarding academic experience. In this presentation, we will provide an overview of a study that uses the theory of planned behavior and person-environment fit to examine the influence of factors such as major certainty, interest-major fit, and academic preparation on within-major persistence. Implications of the study findings for educational planning, taking the perspectives of both colleges and students will be discussed.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
Improving Data Collection to Better Assess Graduate Enrollment and Funding Patterns Using the GSS by Patricia Green, Kelly Kang, Timothy Lally, Monahan, Joseph Perez
The Survey of Graduate Students and Postdocs in Science and Engineering (GSS) is an annual census of all U.S. academic institutions granting research-based master’s degrees or doctorates in science, engineering, and selected health fields. The survey results are used to assess shifts in graduate enrollment as well as trends in financial support.
The panelists will discuss current data uses and needs, strategies for reducing reporting burden, and how proposed changes can improve the utility, quality, and timeliness of the GSS data. Attendees will have the opportunity to learn about the GSS data uses and comment on the feasibility of implementing potential changes to the survey.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Panel Session
Improving Web Survey Response Rates in Higher Education by Shannon Acton, Denise Nadasen
Online surveys are a common way for higher education institutions to evaluate student satisfaction and perceptions. However, these online surveys often have low response rates, reducing the value of the results. In a series of two experiments, we show that response rates of web based surveys can be significantly improved through the use of a few targeted treatments. Specifically, personalization of survey invitations, the schedule of survey reminders, and knowledge of the response deadline were manipulated in this controlled experiment, leading to significant improvement in response rates. Implications for best practice in institutional research survey design and administration are also presented and discussed.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Institutional Rankings in the Health Sciences: Proliferation and Standards by Jacqueline McLaughlin, Gerald McLaughlin, Josetta McLaughlin, Jingjing Zhang
A growing number of ranking platforms have been developed in an attempt to assess value, demonstrate success, and promote reputation in the health sciences. However, concerns about the quality and adequacy of these metrics inhibit many institutions from utilizing rankings to affect strategic change. IR professionals in health sciences must be prepared to engage in conversations about rankings (e.g. evaluate methodologies, interpret findings, discuss relevance) and help shape the audits that assess the quality of these metrics. This presentation examines current ranking systems in the health sciences and proposes standards for auditing and interpreting rankings.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Instructional Productivity Standards by Discipline and Level, Finally by Chatman
Producing student credit hours in a variety of disciplines and at different student class levels is THE core higher education function, but there are no national normative standards for the amount of instructional FTE required to produce an amount of SCHs. Standards, if any other than past local practice, tend to be from funding formulas that are in turn based on past practice. In an effort to establish standards, this session will share the best fit solutions of instructional FTE to student credit hours by discipline and level for public research universities using data from the National Study of Instructional Costs and Productivity (Delaware Cost Study) restricted access database. The relationship between instructional FTE needed based on the solutions and the actual instructional FTE used by institutions is an indication of instructional efficiency, funding, and faculty workload that can be used at the level of departments, colleges, and campuses.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Is Bigger Better? Effects of Text Box Size on Responses in Online Surveys by Jennifer May-Trifiletti, Lauren Conoscenti
It is well-established that a survey’s visual elements can cue respondents and thus influence responses, yet most of this research has focused on closed-ended items. What do respondents think when they encounter open-ended items such as, “What is your favorite part of Dining Services?” Do first-year students perceive a need to fill the space, as in their college application essays? Are older students disheartened when they see only apparently small spaces in which to share their unique and highly specific opinions? This project explored how and to what extent text box size influences various qualities of responses to open-ended items with an eye toward recommendations for designing open-ended items in IR surveys.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
It happens here too: The high school remediation effect on college outcomes by Yen To, Forrest Lane
There is evidence to suggest that developmental education (DE) in high school may improve college readiness (Tierney & Garcia, 2008) and avoid under-placement of students in college DE courses which can consequently raise overall student and institutional costs. The current study uses Propensity Score Matching on a nationally representative sample of 2,927,935 (weighted) students to identifying the impact that students’ participation in high school DE (i.e., remedial coursework) has on college outcomes. This study address three research questions: 1) Does high school DE enrollment predict students’ college DE enrollment? 2) Does high school DE enrollment predict students’ college persistence? 3) Does high school DE enrollment explain differences in college degree attainment? This study has implications for earlier assessment of student preparedness and intervention as an effective method in reducing college DE costs, institutional efficiency, and college student readiness and success.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
Making the most of the College Scorecard data by Wes Posson
The US Government recently released the College Scorecard data set, making huge waves in the world of Higher Education, and Institutional Research. This presentation will show how NYU created insightful reports and dashboards to utilize that vast amount of data. I will cover the structure of the data, available variables, data limitations, and my data cleaning process. The decisions that had to be made regarding data limitations will be addressed specifically. I will then present two separate analytic tools developed using Tableau: (1) a static report developed in the week after the release of the data and (2) an in depth, interactive dashboard. You will be able to see how NYU has utilized this data set to compare ourselves to our competitors in new ways, and how that can be easily adapted to other institutions. You will also see what stories NYU is able to tell about our school using this data.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Measuring and Researching Reverse Transfer to Inform Policy by Jason Taylor, Angela Bell, Matthew Giani, Jeremy Kintzel
This panel features researchers and policymakers from states participating in the national Credit When It’s Due (CWID) initiative. The CWID initiative supports states to develop and implement reverse transfer programs and policies to confer associate’s degrees to community college transfer students when students meet the requirements for the degree. Panelists will discuss reverse transfer metrics and how their states are collecting data on reverse transfer, as well as how data are being used to inform ongoing policy development and improvement.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Panel Session
Measuring Strategic Improvement: A Benchmarking Approach by Tynan Heller, Karen Battye
Most universities have strategic goals, but how can we measure progress towards these goals? Looking at internal historical trends is a traditional approach to measure scholarly work; however, this may not provide a full picture of progress. Likewise, looking only at program or institutional rankings can leave institutions without information they may need for improvement. With a unique approach of using external benchmarks to chart progress annually, this may help give a broader prospective. One method we will discuss compares the market share of scholarly work from year-to-year. Our discussion will focus on ways that institutions measure progress towards their strategic goals and the strengths and weaknesses of those approaches.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
Mediation Effect of Collaborative Learning for Student-Faculty Interaction by Lanlan Mu, Amy Ribera
Using NSSE data from senior students (n=95,491), this poster presentation explores the interplay between two effective educational practices--collaborative learning and student-faculty interaction. Controlling for SAT/ACT score, results from multi-group structural equation model shows collaborative learning has a positive mediation effect on SFI for self-reported gains in learning outcomes but a negative mediation effect on SFI for college grades. The poster will explore possible reasons for these results and identify strategies institutional researchers may use to share findings with units on campus to improve student learning and student-faculty interactions.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
Moving from STEM Readiness to STEM Success by Justine Radunzel, Krista Mattern
This discussion will focus on the role of student and institutional effects for helping students succeed in STEM (Science, Technology, Engineering, and Mathematics)-related fields. Illustrations from a recent multi-institutional study will be provided. Session participants will share their perspectives on the following questions: What information helps to identify students who are likely to need additional academic and social supports to succeed in STEM? How are student vocational interests measured and used for STEM majors? What institutional programs help move students from STEM readiness to STEM success? How might data collection strategies be expanded to include additional measures that help inform retention strategies for STEM majors?
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
Noncognitive Skills and Predictive Modeling: Integration and Implementation by Ross Markle
Both noncognitive assessment and predictive analytics are popular topics in modern higher education. Most institutions understand that these are worth pursuing, but how can they be effective implemented and used? In this session, we will provide structured guidance on the institutional implementation of both predictive modeling and noncognitive assessment, as well as how to integrate the practices. By showing data from a multi-institutional field trial, we will also show similarities and differences in the predictive efficacy of noncognitive factors. Participants can expect a research-based presentation with a focus on application and implementation, such that institutional researchers can use these tools to provide valuable data and inform institutional action.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Pell Grant Recipients: Financial Challenges and College Expectations by Ellen Stolzenberg
Using data from the 50th CIRP Freshman Survey, this poster highlights Pell Grant recipients’ challenges in meeting the full cost of their college education. With the maximum grant covering less than two-thirds of the average published tuition at public four-year colleges, Pell recipients need to draw upon other sources of aid, including loans, to attend college. This poster focuses on constructing a more comprehensive and detailed understanding of current Pell students’ financial pressures and challenges. In addition, the poster will address how Pell Grant status relates to students’ college choice process and college expectations, including initial intentions to stop-out and transfer. The visual display will encourage researchers to utilize similar analysis to understand Pell Grant students and to improve inclusion and support for this important student population.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
Practical Inter-Rater Agreement by David Eubanks
This session gives a practical demonstration of a way to calculate and visualize inter-rater agreement to assess reliability of ordinal or categorical data. These types of data are common from learning outcomes assessment, such as rubric ratings of student work, but also including admissions ratings of applicants, and even the course evaluations that students fill out. By the end of the session you will be able to read a graph of rater agreement and be able to judge performance against what we might get from rolling dice. Code in R will be made available so you can use these methods yourself.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Predicting Doctoral Student Success: Challenges and Future Directions by William Dabney
At The University of Texas at Austin, the Office of Institutional Reporting, Research, and Information Systems (IRRIS) is creating a statistical model that can help accurately predict doctoral students’ probability of success, defined as exiting with a PhD, based upon an assortment of variables. However, building this predictive model has not been without challenges due to the homogeneity of graduate students and other unique features in data collection, categorization, and sourcing. The poster presentation will cover the statistical models used, the independent and dependent variables examined, the data sources leveraged, the progression of sophistication within the models, and the challenges and considerations faced.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
Predicting persistence and degree completion from transfer student behavior by Edgar Sanchez
According to recent IPEDS data, about 40% of first-time degree-seeking students enroll at two-year colleges. Many of these students express plans to obtain at least a bachelor’s degree (e.g., 79% of ACT-tested high school graduates who initially enroll at a two-year college). Because it has become increasingly important to examine alternative pathways to baccalaureate degree completion this study examines the persistence and timely degree completion of a sample of ACT-tested two- to four-year transfer students tracked for 7 years using National Student Clearinghouse data. Weighted hierarchical logistic regression was used to explore the effects of stop-out, length of time taken to transfer, completion of an Associate’s degree before transferring, and precollege achievement on retention and timely degree completion for a sample of over 2,800 students. This presentation will discuss study findings, implications for practice, and benefits of locally developing and applying similar models.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Predictive Analytics on Graduation Rate: An Institutional Case Study by Eric Liu
This study demonstrates the power of predictive analytic methods on a high research activity university by Carnegie classification 2010. The highly accurate predicting result indicates the validity of the predicative model. Due to the unique characteristic of the student body, the follow-up research could lead to a qualitative path. The research findings aim to provide an academic oriented and replicable model. The findings will help the university administrative group locate the at-risk individuals and intervene at an early stage to help those students succeed in their college years.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
Predictive Analytics: Techniques and Applications by Bryce Pride
This discussion will explore the practical application of predictive analytics techniques to institutional data. This is an important area given the continual collection, yet underutilization of massive amounts of institutional data that could be used to support decision making processes and improve institutional effectiveness. The objectives of this session will be to briefly describe the predictive modeling techniques, identify practical applications of the techniques in higher education, and examine a practical example using institutional data.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
Ready or not? Generating data-driven insights about first-year readiness by Steve Wygant
Identifying characteristics that underlie student readiness for college is key for success in admissions, advising, placement and many other areas of higher education. Assessment and institutional research professionals have access to readiness data in the form of standardized tests and high school GPA that yield more useful information if used in combination. Data collected through surveys of students before they arrive on campus also yield data that can help us predict student success and target limited resources. Processes described in this session could be extended and/or generalized to facilitate comparative analyses of segments within any student population.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Redefining the National Data Framework: Discussing Best Practices in Data by Amanda Janice Roberson, Jennifer Engle
This session presents the findings of the Institute for Higher Education Policy’s (IHEP) and the Bill and Melinda Gates Foundation’s project to develop a postsecondary metrics framework based on best practice in the field. The presenters will discuss the research and vetting process to construct a framework, present the final metrics framework recommendations, and will provide insights into the best practices for measuring students’ access, progression, completion, cost, and outcomes of higher education.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Repayment Rates: What IR Professionals Need to Know and Why It Matters by Braden Hosch, Rachel Boon, Amanda Janice Roberson
This panel explores various approaches to calculate student loan repayment rates, including the repayment rate published in College Scorecard, and discusses their implications for consumer information, institutional improvement and accountability. Panelists will provide an overview of the policy landscape and the justification for examining student loan repayment rates after students exit higher education programs. Using institutional data from a statewide community college system and a public research university, the panel will show how repayment rates intersect credential completion, program type, and student and family characteristics. The panel offers advice for analytical approaches and collaboration on other campuses.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Panel Session
Retention Calculation Methods for Entities with Non-Traditional Enrollment by Stephen Nettles, Loraine Devos
This session offers a venue for Institutional Researchers, Institutional Effectiveness professionals, and higher-education administrators at non-traditional institutions and traditional institutions offering programs with multiple starts throughout the year to discuss metrics for assessing student retention in settings that do not conform to the traditional calendar (for example accelerated programs, certificates, online education organizations, or schools offering start dates throughout the year). The facilitators will lead a discussion on alternative methods of reporting student progression that provide a richer understanding of student performance.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
Sexy methods and thorny problems: Opportunities for improving assessment by John Hetts, Terrence Willett, Craig Hayward
The Multiple Measures Assessment Project (MMAP) research team used decision trees to predict the success of college students across 112 California community colleges. The decision trees incorporate high school performance data including cumulative high school GPA, completion of AP and college preparatory courses, highest math and English course completed, grade in most recent high school course in English or Math, and California Standards Test (CST) scores. Decision rules resulting from this research are being piloted by 28 colleges across California. Discussion will include comparison of predictive validity and content validity and approaches to maximizing accuracy of placement using available information.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Simplicity vs. Accuracy: A Comparison of Trees and Random Forest Methods by Mark Umbricht
Classification and regression trees can perform as well as regression models while providing several advantages, such as ease of interpretation and the ability to handle qualitative predictors without dummy variables (James et al., 2013). One disadvantage is that changes in the sample of data can result in different trees, potentially limiting their usefulness. An extension of tree-based methods called random forest fixes this problem by creating a large number of trees, then averaging them statistically. This method can vastly improve the error rates and reduce variability of tree based models at the expense of some interpretability. This presentation will provide an introduction to tree based models and random forest models. Then, using a student level, university dataset, the presentation will walk through examples of each model to predict the likelihood of graduation.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Strategic Enrollment Management and In-house Predictive Analytics by Yun Xiang, Vera Mauk
Data-driven decision making has emerged as a fundamental and integral piece of the culture of higher education. More recently, the enrollment prediction models has become an increasingly popular tool that are adopted by admissions offices to target specific individuals who have a high probability of enrolling. The enrollment prediction models provide admissions counselors with valuable information to use during the recruitment process. This session will illustrate the process by which a 4-year college, starting from scratch, formed an analytic skunk team to build predictive modeling work. The session will present this process step-by-step, from building the team and conceptualizing the work to evaluating and deploying the results.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Student Characteristics Affect Their Course Evaluation Completion? by Lynn Chen
Combining student socio demographic information and the status of their online course evaluation completion might sound like a big project. However, this can be done as the completion status of the student course evaluation is generally available through the online course evaluation platform or system. This study dovetails with Hatfield and Coyle’s study (2013) and extends to a broader timeframe and degree programs with a big dataset that combines student socio demographics and course evaluation completion status in a complete academic year. Results will show the factors that affect the relationship between student course evaluation completion and a student’s GPA, age, hours of credits taken, and degree program enrolled. Using a big dataset gives us more power to determine if course evaluation completion is affected by student social demographics, which may potentially help reveal factors that might improve course evaluation response rate.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
Student Lifecycle: Predicting Enrollment, Retention, and Graduation by David Troutman, Steven Wilkerson, Cathy Delgado
Institutional researchers are asked on a daily basis from executive officers the following questions: How many first time freshmen are we going to have this fall? Can you predict how many students are at risk to drop out? How many students are going to graduate in four and six years? This presentation will provide attendees with the statistical framework needed to answer these questions. Using logistic regression and multinomial logistic regression, attendees will understand how to develop a student success model, construct a longitudinal data set, and identify student characteristics (gender, Pell Status, and SAT) and time-varying factors (Financial Aid by year) associated with matriculation, retention, and graduation.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Students from 2nd Language Backgrounds: Engaging IR With Linguistics by Clifford Adelman
A new inquiry for IR: entering students from 2nd language backgrounds for whom the orthography of their native language is not a Roman alphabet. Which languages? which geographic concentrations? which specific linguistic challenges do they face? How can we improve understanding, instruction, guidance, and placement? These are increasingly powerful questions which IR can help answer.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Summer remediation and student outcome: a regression discontinuity analysis by Hui-Jeong Woo, Matthew Case, Erika Baldwin
This study will demonstrate how regression discontinuity design (RDD) can be utilized to examine whether participation in summer remediation program had a positive effect on students’ academic achievement in their first year. Regression discontinuity design infers causality in the absence of a randomized experiment. It is a pre-and post-test design, drawing causal effects of intervention by assigning a cut-off score. Students who scored below cut-off math and participated in summer remediation program are compared to those who scored above cut-off and were exempted from the program. Initial evidence suggests the summer remediation has a positive impact on student’s GPA. In this session, presenters will discuss how to adapt RDD to higher education setting when randomization is not plausible through a demonstration of using enrollment data with cut-off scores. Presenters will also discuss various research contexts in the field of IR where RDD can be utilized in decision making.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
The American Freshman 1966-2015: Highlighting 50 Years of Data Collection by Kevin Eagan
Technology, pedagogy, and state and federal policy have forced colleges and universities to evolve over the past five decades. Institutions have relied on technology to operate more efficiently, faculty have begun experimenting with new ways of teaching, and state and federal policies have forced more of the costs of attending college from society writ large to individual students and their families. Changes to the composition of students, however, have prompted some of the greatest responses from colleges and universities. This session unveils findings from 50 years of data collection on incoming college students conducted by the Cooperative Institutional Research Program’s Freshman Survey. CIRP will release its monograph describing five decades worth of change among incoming college students at AIR, and this session will highlight trends related to changes in student demographics, pre-college preparation and experiences, students’ college choice process, and expectations for college.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
The Disconnect in Career and Major for Underrepresented Racial Minorities by Patrick Bourke, Kevin Eagan
Research suggests underrepresented racial minority (URM) students are more decisive in their future careers than their majority peers. While URMs tend to be more decided in their future careers, research suggests that URMs tend to be undecided in their college major at higher rates, which presents an important opportunity for academic advisors on college campuses to help students find the best major for their intended career. This poster aims to display findings from national data about the factors that predict students’ decision to pursue particular careers and majors. The visual display will also highlight the stability of these choices during a student’s time in college. The presenters will engage the viewers in a discussion about the ability to assist URMs with career and major guidance and help institutions track students’ progression toward achieving their career goals.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Poster Presentation
The Future of Higher Education: A Shared Literature Review by Daniel Suvak
This session will address the future of higher education as presented in several recent publications.

Share your thoughts about your favorite reads and pick others' brains at this AIR book discussion on the Future of Higher Education. Discussion will focus on recent books on (1) College affordability (2) The promise and peril of technology (MOOCs, adaptive learning, etc.) and (3) Practices to improve access for motivated students.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
The Personal and Social Responsibility Inventory: A Measurement Model by Kevin Hemer, Robert Reason
The Personal and Social Responsibility Inventory (PSRI) is an Association of American Colleges and Universities (AAC&U) sponsored campus climate survey used in higher education assessment and research. The PSRI assesses campus climate on five dimensions: (a) striving for excellence, (b) cultivating academic integrity, (c) contributing to a larger community, (d) taking seriously the perspectives of others, and (e) developing competence in ethical and moral reasoning and action. This session examines the concept of campus climate and discusses the results of a confirmatory factor analysis to identify useful factors within each dimension. These factors may be used by IR professionals or other higher education professionals for institutional assessment and improvement.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Time-to-Degree: A Multilevel Model of Health Sciences Using National Data by Jingjing Zhang, Jacqueline McLaughlin, Audrey Jaeger
Graduate education has garnered significant attention in recent years amid conversations concerning access, affordability, and global competitiveness. Specifically, advocates have encouraged capacity building in the health sciences, as disciplines in this area are seen by many as critical to serving society’s needs and addressing many national and global challenges. In this study, data from the NSF’s Survey of the Earned Doctorate and from IPEDS are used to identify student and institutional characteristics associated with time-to-degree for graduate students in the health sciences. Multilevel models are used to examine trends from 2002-2012 for 20,003 respondents at 278 higher education institutions.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
To Triangulate Data Or Not To, This Is the IR Question! by Fang Du, Stephanie Stephens-Hargrave
This discussion will attempt to spur discussion and collect wisdom on the topic of when and how IR professionals shall triangulate the many data sets we possess in order to inform administrative actions. Two case studies from two IR offices in different universities will be shared as examples for triangulation. Discussion participants will have dialogues on key elements for effective triangulation, such as appropriate research questions, compatibility of data sets and so on. Participants will also explore together possibilities of data triangulation in future work.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
U.S. News Education Rankings: Review of Last Year and the Upcoming Rankings by Robert Morse
The session provides updates on all of the U.S. News education ranking projects, including Best Colleges, Best Graduate Schools and Best Online Programs. We will discuss how the recently updated Carnegie Classifications will impact the upcoming Best Colleges rankings to published in 2016. We will explain methodology changes made to these projects in the past year, discuss existing project expansions (e.g., rankings of graduate nursing programs), talk about new ideas being considered for the upcoming edition of Best Colleges, and review other relatively new ranking projects, including Best Global Universities and Best Arab Region Universities. We will discuss why we give data and unpublished rankings to the institutional research community, and how AIR members can obtain those data from U.S. News.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
U.S. News: Insights and analysis from the Best Global Universities rankings by Robert Morse
Global university rankings are becoming more important each year to both the U.S. and the international IR communities. Many IR offices now track and need to understand the various global rankings because of how much they impact various constituencies on their campus. This session on the U.S. News Best Global Universities rankings published in October 2015 provides updates and methodology details on both the top 750 overall rankings and the 22 important subject rankings. There will be the release of unpublished results and analysis from the latest global rankings. We will explain methodology changes made to the global rankings project in the past year, discuss existing project expansions and talk about new ideas being considered for the upcoming edition of Best Global rankings. We will also review another relatively new regional ranking project-the Best Arab Region Universities rankings and provide a brief overview of the latest results and methodology.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Understanding the Results of the NACUBO Tuition Discounting Study by Kenneth Redd, Lesley McBain
This session will help IR professionals answer several key questions about tuition discounting on their campuses. The session begins with a demonstration of the survey instrument and formulas NACUBO uses to calculate the institutional and the student tuition discount rates. Second, the session discusses the results of the most recent TDS study, with a focus on trends in institutional and student discount rates over the past ten years. The session concludes with new data that show the effects of discount rates on schools' net tuition revenue, trends in funded institutional grants, and need- versus non-need grants.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Uses of the Carnegie Classification of Institutions in Higher Education by Phoebe Wakhungu, Chris Stewart, Victor Borden
The session examines the use of the Carnegie Classification of Institutions of Higher Education among institutional researchers, other college and university administrators, faculty and staff, and interested media outlets. Data will be drawn from a survey of institutional research practitioners, internal tracking system used by the Classification staff, published articles, and the media. Presentation will be followed by a discussion as to how the session participants are using the classification. Participants will also be asked for input as to how the Classification staff can provide avenues for input into future revisions.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Using Data to Assess and Support Interdisciplinary Research Initiatives by Kyle Sweitzer, Bethan Cantwell, Patricia Goodall
In this session, IR professionals from two universities will discuss metrics used to support and benchmark interdisciplinary research. Indiana University is implementing an approach to help faculty more easily find interdisciplinary collaborators on campus. Michigan State University has found a way to identify peer institutions to benchmark interdisciplinary units. The session is organized around the following three questions: What data can be used to facilitate potential interdisciplinary research collaborations on campus? What data can be used to measure an institution’s interdisciplinary initiatives against similar initiatives at peer institutions? What role can IR professionals serve in measuring interdisciplinary research?
Topic Area: Data Analysis and Research Methods for IR   |   Format: Discussion Group
Using EDM in the evaluation of behavioral indicators on CC student outcomes by Erez Lenchner
In the study of factors that influence community college student outcomes, administrative records can play a vital role in uncovering inherent behavioral patterns. Precisely, the extent to which present day EDM allows investigators to explore the effect of student behaviors on academic outcomes in community colleges has yet to be documented thoroughly. While studies have explored the relationship of individual behaviors to student outcomes, investigators have not focused on the temporal tipping-points which distinguishes at-risk behavior from otherwise benign behavior. This study begins to address this gap by examining the timing and frequency of various student behaviors, specifically: late course registration; schedule changes; and actual development of peers network. It shows that indicators can be derived from CC data system, and that information can be updated regularly. It further evaluate both positive and negative behavior indicators impact on student success.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Using growth models to investigate student course performance by Natalie Wright
Examining student engagement and performance in online courses is important for retention and program improvement efforts. This presentation demonstrates how growth models can be applied to the student data available in online learning platforms to investigate student engagement and performance across courses. Participants will learn the basics of growth modeling, be provided with a step-by-step illustration of using growth models on learning platform data, and consider what kinds of institutional research questions can be addressed with growth modeling.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Using Latent Class Analysis in IR: Subgroups of Non-Proficient STEM Majors by Erika Baldwin, Hui-Jeong Woo, Matthew Case
Latent class analysis (LCA) is a quantitative method used to determine the existence of latent (unobserved) groups of individuals based on observable indicators. Although this method has been widely used in other fields, it has not been applied in institutional research (IR). This presentation will demonstrate how LCA results can guide data-driven IR decisions regarding campus programs, services, and policies. This presentation will walk through LCA by examining latent subgroups of first-time freshmen who enter with declared STEM majors yet have not exhibited proficiency in math at matriculation. In this study, non-proficient STEM majors are classified into subgroups based on pre-matriculation indicators (i.e., standardized and placement test scores, high school GPA and college preparatory course taking) with sex, race, first-generation college status and socio-economic status used as predictors of latent class. Implications of results for IR and practitioners will also be discussed.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Using Propensity Score Matching to Create Matched Groups for Analyses by Bobbie Frye
The propensity score matching (PSM) technique simulates an experimental design, controls for selection bias, and creates near equivalent experimental and comparison groups. It is critical that researchers apply methodologies like PSM to control for selection bias and leverage PSM to create matched groups for outcomes analyses in institutional research studies. The technique enables researchers to meet the increasing demands of external and internal stakeholders, demonstrating a wise appropriate use of resources and facilitating the practice of informed-decision making.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Using R to Create a Random Forest Model to Predict NCLEX Exam Performance by Sarah Berg
The national licensing exam for nursing (NCLEX) is a high-stakes exam that assesses if our students have received sufficient training and determines if they will be able to practice as a Practical or Registered Nurse. The outcome of this national exam has extensive ramifications, including maintenance and attainment of accreditation, so it is critical for us to be able to predict which students are likely to fail the NCLEX on the first attempt. Towards this end, a random forest model was created (using the R statistical programming language) to identify those individuals most likely to fail the NCLEX on the first attempt. This presentation will share the results of the modeling analysis, providing R code where applicable. The presentation will also discuss how the College intends to utilize this prediction data in order to improve outcomes for graduates. Model and parameter selection will also be discussed, including the benefits and limitations of those decisions.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Using Student Satisfaction Data to Improve Institutional Metrics by Julie Bryant
Institutions with higher student satisfaction also have higher graduation, lower loan default rates and higher alumni giving. What particular student experiences are correlated with these institutional metrics? Where should institutional researchers focus their analyses if improving student completion and increasing financial resources are priorities for the institution? Results from a multi-year study will be shared and discussed, allowing participants to identify priorities for their own campuses.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
What Translates to STEM Major Degree Completion for First-time Freshman? by Yi Cao
Recent research on college STEM majors showed that the percentage of first-year college students who declare a STEM major is substantially low. Furthermore, for students declaring a STEM major, they leave the STEM pipeline at various transition points along their education career, and only 37% of first-year STEM majors earned a STEM degree or certificate within six years. This study will address what academic and financial aid factors predict first-time freshmen declaring STEM majors who persist and ultimately complete STEM degrees. The findings will help university administrators, instructors and policy-makers to provide targeted services to meet the needs of STEM students as well as to construct early indicators for successful STEM degree completion.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Panel Session
Who will declare a STEM major?: The Role of Achievement and Interests by Krista Mattern, Justine Radunzel
In light of new initiatives and programs being implemented to promote STEM (Science, Technology, Engineering, Mathematics) interest and participation among students, this study examines a multidimensional model of STEM major choice based on students’ achievement levels, their high school coursework and grades, their expressed and measured interests in STEM, and student demographics. Study data consists of more than 90,000 students from more than 40 four-year institutions. Students’ interests in STEM are measured using the ACT Interest Inventory and their expressed major preference. Study findings suggest that students who are better prepared academically in mathematics and science and have STEM-related interests are more likely to initially declare a STEM major. Differences in STEM major choice rates exist among STEM major clusters and student demographic groups. The implications of the findings for promoting interest in STEM fields among different student groups are discussed.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session
Who’s going to leave? A multilevel approach to investigating attrition. by Doodnath Persad
Students leaving before completing their postsecondary course of study result in a lost investment of personal and institutional resources as well as public expenditure. Institutional research professionals are continuously challenged to assist in research endeavors to identify ‘at risk’ students and measure the impact of strategic initiatives designed to reduce student attrition. In this session, findings from a study investigating the relationship between selected student level factors (i.e. entry level qualifications, GPAs, age, student status and sex) and student attrition using hierarchical logistic regression models will be presented.
Topic Area: Data Analysis and Research Methods for IR   |   Format: Speaker Session

IR Operations

(CANCELLED) Creating Your Own Data Diva/Dude: Bringing IR Functions into a Unit by Marisa Yates, Scot Lingrell
This discussion will address the life cycle of incorporating a data and reporting role (an analyst) into a department or division, from conceptualization and initial position creation through navigating the actual implementation of the role. The challenges associated with both implementing as well as building the capacity for this type of position are numerous. While a position within a specific unit is likely to have a very limited role, that role also needs to possess multiple talents in order to provide effective coordination for that unit and to work with the central IR office to collaborate and assist the institution, and to more effectively coordinate tasks that need to be accomplished between offices.
Topic Area: IR Operations   |   Format: Discussion Group
(CANCELLED) Institutional Effectiveness Implementation Maturity Model by Lisa Ncube
The poster presentation will explore the benefits of using a Product Lifecycle Management principles to develop an Institutional Effectiveness Implementation Maturity Model that provides a system platform on which institutional effectiveness metrics can be managed and sustained.
As IE/IR have become increasingly important to institutions of higher education, it is equally important that institutions determine their level of IE implementation maturity. By determining their level of implementation maturity institutions can take appropriate measure to rise to the next level of maturity.

Objectives the session

1. Describe an innovative model for metrics that can be used to measure efficiency, effectiveness, and the level of maturity of IE.
2. Explore the benefits and utility of the Institutional Effectiveness Implementation Maturity model
3. Demonstrate application of the IE Implementation Maturity Model within institutional settings
Topic Area: IR Operations   |   Format: Poster Presentation
(Cancelled) Project Management Methodology Used in Responding to Publishers’ Surveys by Liana Crisan-Vandeborne, Jay Johnson
Whether leading a project for the office or participating as part of a project team, institutional researchers work on several projects each year. While formal project management methodologies exist that may be overkill for the type of work done in an office of institutional research, there are aspects from these methodologies that could help organize our work. In this session we will discuss how project management methodology from the Project Management Institute (PMI) relates to institutional research. We will demonstrate how our office has used this methodology to organize responses to publishers’ surveys.
Topic Area: IR Operations   |   Format: Speaker Session
“I’m here to chew bubblegum and [talk data], and I'm all out of bubblegum.” by Tim Stanley, Taylor Lovell
In the 1988 sci-fi classic “They Live” the protagonist uses a kick-in-the-door approach to get the attention of his audience. Often IR could benefit from a more assertive delivery of information to decision-makers. Analysts are frustrated by decision makers that don’t use (or even seem interested in) the data they produce. UVU’s IR office has used more aggressive tactics in an attempt to stimulate demand for data. Passively publishing good research to the web has been insufficient. We’ve invited ourselves to committees across campus, deployed information on digital signs, scheduled regular check-ins with campus leaders, etc. This discussion group will include sharing methods that have worked for UVU and gather best practices from others.
Topic Area: IR Operations   |   Format: Discussion Group
A framework to assess and promote institutional research capabilities. by Victoria Diaz, Pierre Mercier
Institutional research has evolved from its early role of producing descriptive data reports to becoming an active participant in organizational decision-making (Swing, 2009), driving research endeavours for the timely identification of challenges and opportunities (Delaney, 2009) and leading a culture of knowledge management (Huynh, Gibbons, & Vera, 2009). In this poster, we describe an alternative view to previously proposed models (Bichsel, 2012; Taylor, Hanlon, & Yorke, 2013) that distinguishes activities, from functions, areas of interest and resources, to facilitate the assessment of maturity levels and to identify opportunities for development. Practical implications of adopting this framework for IR Directors include: increased ability to strategically propose capability areas for further development; improved IR promotion via the potential managerial and reputational benefits for the organization; and, more accurate assessments of required resources.
Topic Area: IR Operations   |   Format: Poster Presentation
At the President's Table: Institutional Leadership for IR Professionals by Maya Evans
As college presidents increasingly expand the scope and membership of their senior leadership teams, there is a greater presence of institutional researchers at the president’s table. Yet the expectations for the contributions of IR professionals at the table are more undefined than traditional senior leadership roles. Additionally, IR professionals sometimes lack the training to expand their contributions from research and institutional intelligence to vision setting, strategy formulation, and execution. This session focuses on explaining the key skills and strategies that IR professionals can use to help set and drive the presidential agenda. Using concrete examples, participants will gain an understanding of the primary areas of focus for college presidents, and how they can use their IR skillset to meaningfully advise the president.
Topic Area: IR Operations   |   Format: Speaker Session
Beyond the Water Fountain - Dynamics of Internal Relationships in IR by Donald Femino, Peter Hart
The IR Office relies on collaboration to collect information from data experts across campus, work with various constituent groups and report back with meaningful information. Building and maintaining internal relationships is an essential part of getting the job done. This discussion group will focus on best practices in collaboration, how to remain visible and how to assess the efficacy of the IR Office at the institution.
Topic Area: IR Operations   |   Format: Discussion Group
BI & IR: Shotgun Wedding, Or Marriage Made In Heaven? by Henry Childers
In the face of increased demand for information, the University of Arizona recently decided to combine its Business Intelligence (BI) and IR teams. This seems both an attractive and improbable proposition. This presentation will discuss possible reasons why this move was made, how it is working out, and what other institutions are doing in terms of coordinating or keeping separate their BI and IR teams. Recommendations for other institutions based on this experience will be discussed.
Topic Area: IR Operations   |   Format: Speaker Session
Data verification by visualization - promoting staffs' data literacy by Yuen-Hsien Tseng
Data accuracy and consistency is the tacit requirement for institutional research (IR). However, during the data collecting process, it often occurs that the data collectors can hardly tell the accuracy and consistency of the submitted data, due to, partly, indifference to the data definition/description and, mostly, lack of awareness of data value and data literacy. This common phenomenon suggests the need to promote data literacy among the staff to improve data quality for IR.

In this poster, we first propose the use of data visualization to address the data verification issue during the data collection process. A set of real-world examples is illustrated to show the effectiveness of this approach. We then elaborate on the concept of data literacy (a set of knowledge, attitudes, and skills about data) and the promotion of the awareness of data literacy to potentially solve the data accuracy and consistency issue.
Topic Area: IR Operations   |   Format: Poster Presentation
Development & Implementation of Policy Statements for IR Offices by Rajiv Malhotra, Elizabeth Beaulieu
IR-specific policy statements can play an important role in managing workflow efficiency within offices of IR. Policy statements have potential to improve workflow efficiency and help IR researchers balance the trade-off between prompt response to incoming data requests from various parties within the university and conducting valuable internal projects that support data-informed decision-making. The presentation centers on the recent launch of official Office of Institutional Research and Assessment policy statements at MCPHS University. The session will offer information and tools that will aid AIR members in considering development of such policies in their own offices.
Topic Area: IR Operations   |   Format: Poster Presentation
Educating and Training Stakeholders Toward a “Self-Serve” Data Model by Dana Weigel, Lauren Conoscenti
Increasing work load can be a huge issue for Institutional Research offices regardless of staff size and institution type. This discussion group will consider ways of helping those on your campus become more self-reliant with resources already available to them. Whether it’s better utilizing a survey software tool or being better informed about what data is in the university Fact Book, educating the campus community can help reduce ad hoc requests and other projects to allow IR staff the time to dedicate to required reporting and other long-term projects.
Topic Area: IR Operations   |   Format: Discussion Group
Effective Data Communication Strategies for Institutional Researchers by Yang Zhang, Krystyna Aune, Ronald Cambra
Effective communication between institutions, researchers and data users is key to successful and efficient data-informed decision-making. In this session, data users, communication scholars, and institutional researchers will help institutional research professionals advance their knowledge and capacity in effective data communication strategies. Presenters will first introduce Weick’s Organizational Information Theory (OIT) (1995), which provides a theoretical framework to explain how organizations make sense of the vast amount of information that is available to decision-making. Drawing from communication theories and our university’s examples, presenters showcase how effective data communication strategies can be achieved in dynamic and information-rich university settings.
Topic Area: IR Operations   |   Format: Speaker Session
Engaging Faculty in the Scholarship of Student Success by Margaret Dalrymple, Linda Shepard
Traditionally, research on student success has been conducted by the institutional research office, which then informs administrators and faculty alike of their findings. The purpose of this presentation will be to present and discuss specific processes for involving faculty more directly in the scholarship of student success. The rich, naturally occurring inquiry expertise and varied interdisciplinary backgrounds of faculty makes them a universally ideal partner. The numerous outcomes have been positive, with all involved benefiting. Two institutions have begun investing in these partnerships and will share their techniques and experiences from the first year of implementation.
Topic Area: IR Operations   |   Format: Speaker Session
Exploring opportunities for collaboration between IR, Faculty, and Students by Tim Stanley, Ron Hammond, Traci Wennerholm
As the demand for institutional research and data to inform decision-makers continues to increase, offices of Institutional Research must rely on collaboration with people outside their departments to help meet the demand. Faculty and students provide rich resources for innovation, energy, and enthusiasm around topics of interest to institutional research. At UVU we have found many opportunities for these collaborations to advance the goals of the IR office, faculty, and students simultaneously. Discussion will revolve around examples of successful experiences, benefits, strategies, and other tips for leveraging collaboration between IR staff, faculty, and students.
Topic Area: IR Operations   |   Format: Discussion Group
FERPA and its role in Institutional Research – Best Practices by Geoff Matthews, Tim Stanley
Often offices of institutional research focus their energy on research innovations but neglect important issues of compliance. The Family Education Rights and Privacy Act governs access to data by a student, but also plays a role in how those data are to be protected. Institutional Research departments receive many requests for data, and when those requests call for records of identifiable students, care needs to be taken that departments operate within the confines of the law. This presentation will provide an explanation of these rules coupled with applicable examples, and one department’s strategies for avoiding problems.
Topic Area: IR Operations   |   Format: Speaker Session
Increasing the Overall Efficiency and Effectiveness of IR/IE Offices by Hirosuke Honda, William Knight, Heather Kelly, Yasushi Morimoto
How can we improve the overall efficiency and effectiveness of IR/IE offices? The four insider case studies of a public baccalaureate university, a community college, and two public research universities will analyze and illustrate the topic by applying a four-quadrant framework. Participants will learn how to apply the four-quadrant framework to analyze their office projects and use it as a communication tool with office staff and campus stakeholders to further improve IR operations.
Topic Area: IR Operations   |   Format: Speaker Session
INSTITUTIONAL DATA AND TRANSGENDER STUDENTS: ADAPTING EXISTING STRUCTURES by Shawn Potter, Red Tremmel
In an effort to create an environment that values diversity and promotes the dignity of all people, Tulane University convened a task force to recommend how the institution should adapt existing structures and processes related to campus records and materials to meet the needs of transgender students. The Office of Assessment and Institutional Research has played a key role in this process. This presentation addresses the challenges faced by the task force, as well as the obstacles encountered by transgender students as they strive to learn and live in an atmosphere that correspond with their gender identity. The session objectives are to share the challenges related to adapting the existing structures and processes connected to campus records and materials to meet the needs of transgender students as well as foster discussion about the issues affecting transgender students and how universities can incorporate changes to enhance their student experience.
Topic Area: IR Operations   |   Format: Speaker Session
Institutional Research Certificate Program at Penn State University by Mark Umbricht
With initial support from AIR, Penn State offers an on-line, Graduate Program for institutional researchers. This session will use Terenzini’s (2013) revised tiers of institutional research intelligence as a framework to show how IR professionals, from current institutional researchers to graduate students interested in the field, can gain skills and knowledge to support institutional planning, analysis, and policy formation. Examples from current IR courses will be used to show how students can gain all three tiers of IR intelligence from the certificate program.
Topic Area: IR Operations   |   Format: Poster Presentation
Interaction between IR and IT to enhance organizational effectiveness by Ching-Hui Lin, Wei-Cheng Chien, Cheng-Hsing Hsu, Victor Borden
Learning from the long US history of IR practices, this poster considers how a country developing IR at this point in history quickly assimilates those lessons to an international context? As the Taiwan Ministry of Education encourages the development of IR practices in the country, participating institutions learn quickly how to adopt IR models to their local circumstances (Webber & Calderon, 2015). Although every country has a unique culture that shapes IR operations, similar foundational principles and collaborations cross borders. Responses to unique cultural contexts enrich our understanding of tailoring IR to meet institutional needs, locally and globally.
Topic Area: IR Operations   |   Format: Poster Presentation
IR & BI Collaboration: Can we really make it happen? by Kristin Moser, Malissa Martin
The offices of IR and BI at the University of Northern Iowa have a unique relationship not often seen at other institutions. Where other institutions have typically found conflict, the teams at UNI have been able to work collaboratively to enhance the BI analytics capability on campus. The presenters will discuss the path taken to reach this arrangement and will discuss strategies for collaboration at other institutions.
Topic Area: IR Operations   |   Format: Speaker Session
IR and PR: An Unlikely Partnership by Linda LeFauve
Institutional research professionals must sometimes navigate the complicated relationship between their office and public relations. IR claims public relations is too focused on perception; PR claims IR is too focused on facts. When the facts don’t speak for themselves, however, what can we in IR learn from our PR colleagues? How can we use a public relations strategy to move analyses from background information to conversation starter? How can we get agenda-changing discoveries in front of the people best positioned to appreciate its implications? The session uses real examples from institutional research and provides practical ideas for applying these same techniques.
Topic Area: IR Operations   |   Format: Speaker Session
IR: Transition Pathway of Chinese Higher Education Research Institutions by Junchao Zhang
Following a different developmental path, institutional research in China emerged as a sub-field of higher education research in the form of higher education research institutes (HERI), rather than typical institutional research offices located in American universities. In the past 40 years, many HERIs were built in Chinese universities, and they generally follow two different directions: an academic research approach and an administrative affairs approach. As more and more Chinese university leaders start to change from experience-based management style to data-driven and evidence-based, HERIs are facing new challenges to accommodate those new needs. This study will display the development of HERIs in China and analysis their current situation and problems, and then further discuss HERIs’ new trend of organizational and role transformation as IR offices based on a national 4-year HEIs Survey.
Topic Area: IR Operations   |   Format: Poster Presentation
Making the Most of Your Data: The Institutional Effectiveness Newsletter by Molly Isaacson
Institutional Effectiveness offices house a wealth of data used for a variety of purposes. Often, researchers respond to specific data requests while other data goes unused. By creating a newsletter, data can be shared to highlight initiatives and surveys while also providing increased visibility for Institutional Effectiveness projects. There is no extra cost to this and it's a great way to show students and faculty that the data are used and may even increase response rates on student surveys. Audience members will understand how this can benefit their own campuses through customizing a newsletter to their needs.
Topic Area: IR Operations   |   Format: Poster Presentation
Maximizing the effectiveness of Institutional Effectiveness by Amy Garczynski, Steve Nettles
This discussion will address how small Institutional Research (IR) departments can maximize their impact within the context of developing team members without adversely affecting their workload. Specifically, we will share our experience developing analysts who are new to IR to expand beyond analysis to collaborate cross-departmentally and disseminate results. Participants will discuss best practices and pitfalls in mentoring team members, as well as resolve dilemmas in expanding teams.
Topic Area: IR Operations   |   Format: Discussion Group
Online Certificate in IR at Florida State University by Samantha Nix
The focus of this poster is an online certificate program designed to provide academic and professional development opportunities for institutional researchers, administrators, doctoral students, and faculty from all higher education institutions. The program is designed to accommodate the working professional's schedule. The program goals are (1) to enhance knowledge and understanding of the core principles of IR; (2) to facilitate use of national databases; and (3) to promote the use of IR to improve administrative and policy development processes. The 18-credit hour curriculum focuses on IR theory, institutional administration, quantitative research methods, utilization of national databases, and IR practice.
Topic Area: IR Operations   |   Format: Poster Presentation
Promoting Effective Documentation: Tools and Resources by Erika Hill
Documentation plays an important role in IR offices by helping to explain the history of the data and outlines the processes the office has used to complete tasks. It also helps offices provide consistent and accurate data and helps with on-boarding of new employees. Unfortunately, documentation may not comprehensive, may have multiple versions, may not be updated on a regular basis, and in some cases --documentation may not exist at all. Over the past five years, the mid-size IR office at the University of Utah has experienced challenges and successes with their documentation processes. This discussion session will cover lessons learned, tools and resources, and provide a chance for attendees to share their own experiences.
Topic Area: IR Operations   |   Format: Discussion Group
Strategic Agenda-Setting of Institutional Research in Taiwan by Shihuei Ho, Yao-Ping Peng, Tseng Wei-Li, Yao-Wen Chang
As higher education around the world faces increasingly challenging times in responding to increasing forms of accountability and competition, new stakeholder expectations, and rapidly changing technologies, institutional research will inevitably evolve to become a stronger force. This study is concerned with institutional research in Asia and seeks to understand the establishment and development of the Offices of Institutional Research in Taiwan, by comparing their function and practice in public and private HEIs. Based on detailed analysis with five well-known HEIs, we show a growing level of planning and establishing strategy of offices of institutional research. Implications for IR practitioners, contributions to institutional effectiveness, and institutional synergy are discussed.
Topic Area: IR Operations   |   Format: Speaker Session
Supporting Academic Areas: One size does NOT fit all by Linda Ferguson, Margaret Dalrymple
The fundamental element of Institutional Research is to provide service and data to the campus constituents. Our most overlooked and underutilized data consumers are in the academic areas. Upon recognizing the diversity of aptitude and data capability across campus, it is quickly realized that a one-size-fits-all approach will not do. This presentation will highlight reports, from static to dynamic, and include a demonstration of our new information gateway system. We will discuss the various ways of providing data, communicating, and accommodating needs and expectations so as to support the academic areas with the quality service of institutional research.
Topic Area: IR Operations   |   Format: Speaker Session
The Graduate Experience for IR Professionals by Mark Umbricht, Samantha Nix
As demands for data analysis rise in our increasingly data-driven world, IR offices will require individuals who are highly educated and skilled. Graduate education is an excellent way to master new technical and analytical skills, increase issues intelligence and contextual intelligence, and receive a credential which may be needed for advancement. This discussion will address the topic of graduate education for IR professionals including master's, doctoral, and IR certificate programs. Attendees will discuss the type of skills they want to learn, what type of education they may be interested in, and what type of graduate education can be beneficial for these goals.
Topic Area: IR Operations   |   Format: Discussion Group
The Role of Partnerships in Achieving the Vision by Valerie Holton, Kathleen Shaw
Strategic partnerships across the institutions of higher education are critical to achieve the vision outlined in the Statements of Aspirational Practice for Institutional Research (www.airweb.org). This presentation highlights the role of partnerships in recognizing an expanded definition of “decision makers”, grounding efforts in a student-focused paradigm, and enhancing coordination of institutional studies. Using Virginia Commonwealth University as a case study, this presentation will highlight the role of internal partnerships in the development of enterprise data systems to track and assess the university’s engagement with and impact on our communities. Participants will learn strategies to identify, develop and maintain strategic partnerships within their institutions.
Topic Area: IR Operations   |   Format: Speaker Session
Transforming IR Practice Guided by the Statements of Aspirational Practice by Michele Hansen, Robert Janik
Indiana University-Purdue University Indianapolis was selected as a one of ten Founding Institutions for the Statements of Aspirational Practice for Institutional Research. IR functions were reorganized around the three major themes: 1) An Expanded Definition of Decision Makers, 2) A Student-Focused Paradigm, and 3) Institutional Research Leadership 2.0. Presenters will describe how IR functions and practices were transformed to ensure that timely and actionable information about student learning, success, and institutional effectiveness was available to all key decision makers across campus. This presentation will also focus on how enabled business intelligence technology such as Tableau was optimized to put data in the hands of decision makers and to allow users to explore, interact with, and analyze information tailored to specific decisions. Presenters will also explain some IR leadership functions such as managing data and being effective data narrators.
Topic Area: IR Operations   |   Format: Speaker Session
Using a Technical Position in IR to Improve Institutional Effectiveness by Catherine Trouth
This presentation will address how a technical position normally found in IT can be brought into an IR office and used to create solutions that improve program effectiveness across the institution. In addition, the session will address creating a permanent position from a contract position by creatively using the technical ability of IR staff. Participants will learn how to think outside normal IR positions to use their expertise to address problems around campus while also building the goodwill and trust necessary to grow the IR department.
Topic Area: IR Operations   |   Format: Speaker Session
Washington Update: National Data Policy Conversations by Teri Hinds, Christine Keller, Jamey Rorison
This session will feature data policy experts from the Institute for Higher Education Policy, Association of Public and Land-grant Universities, and American Association of Community Colleges bringing participants an up-to-the-minute assessment of the climate in Congress and within the Department of Education with respect to higher education in general, proposed data privacy legislation, and the expected timetable and drivers for reauthorization of the Higher Education Act. The presenters are all seasoned data policy experts and will engage in a panel conversation of the issues of the day with a particular focus on national policy as it applies to institutional research, institutional effectiveness, assessment, and accreditation.
Topic Area: IR Operations   |   Format: Panel Session
What Every IR/IE Rookie Should Know: Class of 2016 by Angel Jowers, Eric Atchison, Gordon Mills
Three institutional researchers representing a public master’s university, a public university system office, and a public research university will share experiences from working in IR and IE. The target audience is newcomers to institutional research (IR) and/or responsible for coordinating, planning, and assessment and helping others to use assessment results for continuous improvement (IE). This presentation will also allow time for a question and answer session with the panelists as well as an opportunity for the audience to share lessons they learned during their initial experience of working in IR and IE.
Topic Area: IR Operations   |   Format: Panel Session

IR Studies for Campus Decision Support

(CANCLLED) A Student Exit Survey Employed In an Undergraduate Health Studies Program by Paula Griswold, Jessica Dolecheck
This work focuses on the results of a student exit survey used in assessment and planning of an undergraduate Health Studies program at the University of Louisiana at Monroe (ULM). Employing this survey provides ULM administrators, program directors, and faculty with vital information obtained from online and traditional Health Studies graduates on advising, benefits of community engagement, quality of a practicum experience, and successes of experiential learning. Participants will examine the benefits of an undergraduate student exit survey, explore how an exit survey can enhance an undergraduate program, and identify important student information to capture in an exit survey.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
A Critical Test for Critical Courses by Andrew Zehner
This session offers two useful outcomes. First, participants interested in research outcomes and results will learn how strong (or weak) the link is between particular undergraduate courses and on-time graduation as much as seven semesters later. These results shed a different light on academic progress than the more common assessment using DFW rates. Second, participants interested in research design and assessment follow-through will enjoy the narrative of the presenters' efforts to use assessment to modify and improve a recently adopted policy at their university.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
A Noncognitive Roadmap to College Success by Becky Bobek
A multidimensional set of achievement, psychosocial, and education and career planning factors predict college retention, satisfaction, and graduation, yet the noncognitive factors receive far less attention in many colleges than cognitive ones. It is important for colleges to understand which psychosocial and education and career planning factors have high potential for promoting specific outcomes. This understanding would allow colleges to focus more efficiently and cost-effectively on these noncognitive factors. In this presentation, the author identifies psychosocial and education and career planning constructs that are likely to contribute to positive outcomes for critical transitions during college. Implications of this research for institutional assessment, reporting, and intervention will also be discussed.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
A Regional University’s Processes in Dealing with a State Tuition Reduction by Daniel Matthews, Sarah Swager
Washington State passed legislation in July 2015 that reduced student tuition by 5% in the Fall of 2015, and an additional 15% at the six state-funded baccalaureate institutions in the coming year. This presentation will address how this decision has affected a regional institution in terms of budget projections in the finance department, enrollment models in institutional research, and new advising requirements in student success. This multiple-department presentation will cover details on how funding was determined, state funding models were implemented, and payback was made to achieve budget reconciliation, and address additional unintended outcomes and institutional decisions.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Academic Program Planning: Bringing Data to the Conversation by Shannen Robson, Tim Stanley
One of the most important tasks of a university is to prioritize prospective program offerings in a systematic way. This presentation will explain how IRI collaborated with Academic Affairs to develop a new program feasibility process at the presenters' university. The presenters will review how they assembled labor market, employer demand, student interest, and comparative peer institution datasets, and then used this information to assist Academic Affairs to identify and evaluate potential program offerings. There will be a discussion of the challenges and benefits of this collaborative process and the ways IR professionals can critically inform programmatic decision making on campus.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Academic Social Norming: Using IR to Promote Successful Academic Behaviors by Mark Hanson, Sidney Fisk
Student behaviors are an important piece of the college completion puzzle, and IR offices are steadily being called upon to play a more active role in data-driven campus collaborations to improve student outcomes. The social norms approach provides a way to have an active, direct, and data-driven impact on student academic behaviors and attitudes linked with academic success. Session objectives are to explain the social norms approach, highlight strategies for academic norming data collection and analysis, and explain how to translate results into an effective academic norms messaging campaign.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
An Analysis of Factors Related to First-Time, First-Year Persistence by Christopher Pena
Student persistence is one of the most common strategic metrics at post-secondary institutions, but administrators struggle to understand the factors that lead students to persist or depart. This session will describe how IR staff at the University of Denver responded to an institutional need to understand and promote student access and success by studying and analyzing factors related to student persistence. Speakers will summarize current persistence reporting practices at the University of Denver and present the results of a logistic regression analysis exploring potential associations between one-year persistence and student characteristics identified in prior research studies. Specifically, the presenters will outline salient variables identified in the literature, report the results of their analysis, and discuss how their findings are being implemented in advising, financial aid, and student support service offices.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
An IR Faculty Success Toolkit by William Knight
The employment of tenured faculty members is the largest long-term investment most institutions make. IR offices can provide decision support concerning faculty success in a variety of ways including faculty satisfaction, retention, promotion and tenure rates, workload, and salaries. The presentation will provide information in all of these areas, including copies of institutional surveys, interview questions, analyses, and the BSU Faculty Retention Task Force Report.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Analyzing Course Evaluation Data to Inform New Faculty Training by Ann Marie Russell, Thomas McGuinness
At Bates College, the analysis of more than 130,000 course evaluations was a key part of a data-driven approach to enhance the training of new faculty. Multivariate analyses identified the characteristics of faculty and courses at risk of receiving weaker evaluations. Presenters will discuss the results of their research, and how it had an impact on academic decision-making on campus. Attendees of this session will gain a greater understanding of the factors that contribute to variation in evaluations of junior faculty, and will explore possibilities for conducting analyses of course evaluation data on their own campuses.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Analyzing User Engagement in a Private Social Media Educational Group by Lisa Elliot
Educators often adopt social media platforms such as closed Facebook pages or Google+ private Communities, to facilitate course engagement. IR professionals may be asked to determine if this strategy is effectively engaging students. This poster session offers a case study of the Google+ Private Community established for the Deaf STEM Community Alliance, to illustrate this question. Specifically, the poster session will (a) describe the functions of the DHHVAC; (b) share methods by which DHHVAC engagement is assessed; and, (c) discuss strategies for generating engagement in an educational forum supported by a social media platform.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
Apples to Apples?: Comparing ECORE and Face-to-Face Course Performance by Susan Donoff, Tan Tran, Rachana Bhatt
While the explosion of online learning is indisputable, consensus on student performance is far from certain. According to Allen and Seaman, “a minority of academic leaders continue to believe the learning outcomes for online education are inferior to those of face-to-face instruction.”. The present study aims to address this concern by comparing the grades of online versus traditional students in subsequent face-to-face coursework. The authors utilize a regression model and data on approximately 8,000 students within seven University System of Georgia institutions. By including demographic variables in our analysis, the authors assess whether the difference in learning outcomes is due to the format of the course (online versus face-to-face) or if the differences are due to the characteristics of students who elect to take the online version of the course instead of the face-to-face one.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Assessing Student Success Initiatives: Challenges And Results by Julio E. Moreno, Paul Rusinko
Higher education institutions are always interested in understanding what type of impact new implementations and initiatives are making on students, and this is especially true when those initiatives relate to student success. The development of evaluation plans and tracking of key performance measures can be a big challenge when there are many initiatives but few human resources available for that assessment. This case study highlights the decision making and processes followed by one institution to create an effective assessment strategy for a large number of simultaneous student success initiatives.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Camouflage to the Classroom: Understanding the Student Veteran Experience by Travis Tilman, Kevin Eagan
Despite billions of dollars spent on hundreds of thousands of student veterans, there remains a lack of understanding of the student veteran experience and how it differs from that of traditional students. This session is targeted toward participants who intend to better understand how veterans are using academic services, their habits of mind, and how these behaviors are related to overall performance in terms of GPA. In this session, presenters will highlight findings from a multi-institutional study on student veterans’ use of campus services, development of habits of mind for lifelong learning, and predictors of academic performance. Recommendations for tailoring of transition programs to meet the specific needs of student veterans will be discussed.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Career Outcome Results for Recent Graduates: a Multi-Disciplinary Approach by Emily Shipley
The perceived return on investment of a college education is under increasing scrutiny as the cost of a college degree and student loan indebtedness rises. Prospective students and their families, alumni, accrediting bodies, and government agencies desire more data on post-graduation career outcomes from institutions of higher education in order to evaluate and compare institutions. Capturing reliable and accurate information on these outcomes is a difficult task, but one that Institutional Researchers are often called upon to complete. This presentation will explore one institution’s adoption of a multi-disciplinary approach to the collection of post-graduation career outcome data, including the benefits, disadvantages, and lessons learned from utilizing this method.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Challenges of International Students from China in American Institutions by Xiaobing Cao, Qingwen Dong, Tongshan Chang
The major issue to be discussed in this session is challenges facing international students in American institutions. This discuss group is critically important to institutional researchers and administrators to understand the fundamental causes of the challenges international students face. Specifically, the objectives of this session are twofold: introducing and analyzing major issues facing international students studying in American institutions, and developing recommendations for faculty, researchers, administrators as well as students to deal with their challenges.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Closing Higher Education's Equity Gaps by Kathleen Zaback, Andy Carlson, Sharmila Mann
In late 2015, the State Higher Education Executive Officers and Complete College America (CCA) released a brief utilizing CCA's multi-state data set to examine equity gaps across higher education. This session will present the results of a report that examines data researchers have never before had access to at this scale, and provides additional insight on some of the larger trends that face the higher education system as it seeks to improve outcomes. Large equity gaps exist in the current system, and those gaps look different across groups and states. Presenters will discuss existing literature that helps explain some of these trends, and new interventions that show promise in helping to address outcomes. This session will provide an overview of the results of this report and invite institutional researchers to consider how they might look at these data to help inform campus-level interventions.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Data Governance and Business Intelligence: A Tale of Two Schools by Dale Amburgey, Tarji Kinsey, Dominic Lombardi, Jessica Steinmann
The dramatic increase of data requirements in today’s higher education landscape has led to the topic of Business Intelligence (BI) reaching critical mass in the Institutional Research profession. As many institutions are finding out, even the best BI efforts can be thwarted by a weak or non-existent data governance infrastructure. Join us as two private institutions share the journey they are taking to enhance data-driven decision-making by introducing the tenets of data governance and business intelligence in their organizations. Both universities will explain what has worked and, more importantly, what has not.
Topic Area: IR Studies for Campus Decision Support   |   Format: Panel Session
Debunking the online quality myth: Using data to evaluate online education by Christopher Foley, Sharon Wavle
Online education is inferior to traditional face-to-face education, right? Not so! This session will present a variety of methods that Indiana University utilizes to assess the effectiveness of online education compared to traditional campuses classes. The session will provide an overview of standard success metrics (e.g., GPA, retention, graduation rates) for students involved at various levels of online education (e.g., taking a single course vs. a completely online program). We will present exploratory analyses of student outcomes (e.g., performance in sequential courses) as well as an analysis of the types of students most likely to take advantage of online education.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Diverse Approaches to Collecting and Responding to Sexual Assault Data by Marne Einarson, Kirsten Skillrud, Laura Palucki Blake, Erika Farfan, Michael Le
Particular attention is currently being paid to campus climate regarding sexual assault. The U.S. Department of Education and Congress, along with student activists, are pressuring colleges and universities to purposefully address the issues of sexual assault and student safety. One way institutional research professionals can assist in meeting demands and providing decision support on the topic is to collect data from students on their experiences with, and perceptions of, sexual assault on their campus. This panel presentation will provide multiple perspectives on the development, collection, and use of surveys related to sexual assault, including campus-specific and collaboratively developed survey models.
Topic Area: IR Studies for Campus Decision Support   |   Format: Panel Session
Do Macroeconomic and Financial Aid Indicators Impact Graduate Enrollments? by Diana Barbu, Adrian Barbu
Most studies examined the impact of macroeconomic indicators on individuals’ decisions in regards to graduate school enrollments. This study focuses on institutional level data and examines how institutions (public, private for- and not-for-profit) are impacted by the variations in macroeconomic and financial aid indicators in terms of graduate student enrollments. The macroeconomic indicators used in the study are unemployment rates, income, gross domestic product, population at the state and national level and the financial aid indicators are undergraduate and graduate loans and graduate grants. Preliminary results show that graduate enrollments in public and private institutions are impacted by financial aid and macroeconomic factors. These findings can assist administrators and IR professionals with revenue prediction, planning, and strategic decision making.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Engaging marketing techniques to improve data-informed decision-making by Holly Stovall, Colin Jenney
Increasing awareness and understanding of the role of Institutional Research is paramount. Although this task is challenging, there are methods that can be employed to better brand an IR office and engage IR constituents. Processes presented in the panel session include producing Data’s Anatomy (a TV-style data series), disseminating branded data facts emails, which educate constituents while showcasing IR, and creating a Presidents’ report, which were specifically structured around their self-stated data needs.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Evaluating AP Credit Policies with Placement Validity Research by Kelly Godfrey, Cathy Brigham
Oftentimes, institutional researchers may be asked for their support in determining the appropriateness of the AP placement and credit policies in place at an institution or system of institutions. This session, led by a College Board researcher, discusses requirements, benefits, and procedures for conducting placement validity studies. Detail on data considerations, challenges, limitations, and presenting to various stakeholders will be discussed, with opportunity for attendees to ask questions and discuss concerns specific to their institutions.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Examining Faculty Satisfaction and Students’ Academic Outcomes by Hannah Whang Sayson, Casey Shapiro, Marc Levis-Fitzgerald
Faculty are inarguably a key aspect of the college experience and contribute to student learning through their work both in and out of the classroom. As colleges and universities continue to “unbundle” faculty’s teaching, research, and service roles, rely on non-tenure-track faculty, or otherwise reduce their availability of faculty resources, it is incumbent upon institutions to ensure that these shifts do not adversely affect students. This presentation highlights one institution’s use of multiple data sources to explore the relationships between faculty attitudes and student outcomes. By examining faculty’s job satisfaction and beliefs about their value to their institution, the present study sheds light on how institutions might better support their students by better supporting their faculty.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Exploring admission strategies and resource allocations on graduation rates by Eric Liu, Eric Yang
Graduation rate is an important performance indicator for better accountability. By analyzing the high school GPA percentile and institutional expenditure from Carnegie doctoral institutions, this study discloses significant relationship that will assist decision makers to adjust admission strategies and college resource allocations.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
Gates Excellence Instrument by
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Getting a Handle on Employment Data: How do we use it to tell our story? by Brianna Moore-Trieu, Chris Olson
Two private non-profit colleges will share information on how they acquired and used non-self-reported alumni employment data from the Employment Development Department (EDD). Presenters will compare the benefits and limitations of using this wage data along with other sources of information about alumni outcomes; such as the College Scorecard, LinkedIn University pages, PayScale and alumni surveys. Presenters will show how each data source contributes a unique element to telling the story of a graduate. The session will also review the impact of benchmark choice when interpreting salary information and discuss suggested ways to use employment data within an institution.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Harnessing Data without Getting Trampled by It; A Branch Campus Case Study by Deborah Kepple-Mamros, Joseph Rives
Learn how one branch campus identified “at risk” students, and built their capacity to solve retention problems while working at both the institution-level and the local level. The result was 1) creation of a data warehouse at the institution-level to organize and make accessible more useful date, 2) the creation of sub-committees responsible for retention for each of the three campus locations, and 3) the use of tools like Weka to analyze and visualize the local data, and a website to communicate the information across the institution. This presentation will highlight how gathering and analyzing small data sets from the branch campus led to changes in academic services.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
How Good is Your Retention Rate? by Natalie Solverson
The institution analyzed for this presentation has a strong first-to-second year retention rate. Our goal was to determine if the observed rate matched the predicted rate of retention to facilitate conversations around institutional programming and resources. Participants will learn about the statistical method we applied, the challenges we faced, and considerations for undertaking a similar project at their own institutions.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
How to visualize flow data, simple, but catchy (Case: Change of major) by Hyejin Choi, Manuel S Gonzalez Canche, Julius Gantt, Meihua Zhai
Institutional researchers are requested to report higher education data to reflect diverse aspects of the issue in a concise, accurate and informative way. Aligned with this demand, visualization of college data becomes more rules than a choice with its strength in summarizing and analyzing dynamics of higher education reflected on data. However, with a wide range of selections, institutional researchers should acknowledge what tools are available and how they can be used differently and effectively depending on what information they are requested to provide. This session will briefly introduce three selected tools to visualize flow data.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
How We Increased Participation Rates in Course Evaluations and NSSE by Michael Duggan, Angela Carlson-Bancroft
Institutional Research worked with Marketing, Student Affairs, and Academic Affairs to develop and implement a plan to increase participation rates in NSSE. Our rates increased from 25% to more than 40%. Our poster provides information on our contact plan and incentives.
We implemented a new course evaluation system which had a participation rate of about 94%, compared with 78% for the prior system. We worked closely with IT and our Registrar to come up with a plan to increase participation. We discuss how we utilized a grade hold in the fall semester with a resulting boost in responses.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
Improving Student Outcomes with Early Alert Technology: A Comparison Study by Laura Damm, Lawrence Piegza
Spurred by recent national education initiatives and the growing adoption of performance funding models, colleges are under more pressure than ever to not only improve student outcomes, but to document their progress along the way. Offering comparative study of performance data from the Mississippi statewide community college system, this session will demonstrate the effectiveness of early alert technology in achieving better outcomes in terms of retention and graduation rates. In exploring the adoption of data-driven software technology at select schools within the system, participants will learn effective strategies for leveraging data-driven technology to inform and improve student success initiatives, as well as learning effective strategies for collecting and using data points to measure student success program efficacy.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
International Students: Safe Environments and Cross-Cultural Interactions by Rosemarie Lerma, Michele Hansen
Summer Bridge Programs are one of the first steps toward creating safe environments and ensuring positive cross-cultural interactions. The primary purpose of this study was to examine whether an integrated or community specific summer bridge program would best meet the academic and daily life adjustment needs of international students on a large urban campus. Presenters will describe the results of a multiple methods investigation, the implications of the results, and how IR offices can provide evidence based decision support for campus leaders.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
IR Reporting for College Leadership: Show and Tell. by Lou Guthrie
This show and tell discussion will address Institutional Research reporting tools used to help their college's Trustees and Executives make decisions. Come prepared to share examples of your latest unique, or innovative, reports, including the types, content, frequency and data visualization used. Successes, lessons learned and challenges will be discussed. We will conclude with the trends and best practices that we see based on the discussion.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Learning Communities’ Student Impacts: A Multi-Method Study by Stefano Fiorini, Tao Liu
This study provides an assessment of Living-Learning programs at a large Midwestern university and examines their impact on student retention, graduation and academic performance in the first and second years. These programs can enhance student involvement and progression towards degree by enhancing the academic and social interactions of students in higher education. However, assessment can be biased due to student self-selectivity. This study provides multiple techniques to address this issue and gives special attention to the most successful approaches.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Leveraging Early Alert Technology for Efficient Intrusive Advising by Damon Blythe, Lawrence Piegza
As enrollment at the nation’s two-year colleges continues to rise, schools must be on the lookout for ways to support student success while making the most of available resources. This discussion will explore efficient methods for routing academic support services to at-risk students within the particular context of community colleges. Centering on a model early alert strategy implemented at Trenholm State Community College, the discussion will address a variety of relevant issues, including: practical applications of intrusive advising principles, the use of risk modelling for early intervention, tactics for efficient deployment of support resources, and techniques for measuring the effectiveness of student support strategies.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Mining Big Data: Survey Factors as Arsenal to Solve the Retention Puzzle by Patrice Lancey, Uday Nair, Rachel Straney
Tinto’s seminal model spawned decades of empirical research and influenced higher education practitioners who embraced their role as intentional actors in the student departure process. This session focuses on how a large research institution used a comprehensive longitudinal data base to mine for actionable findings to proactively support strategies to increase retention rates. Results are shown for Entering Student Survey variables correlated with FTIC first-year retention rates and a specific GPA score range described as the “murky middle.” Participants will understand the power of data mining capacity to identify and impact students at risk for attrition at their institution.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
National Survey of Institutional Research Offices by Darlena Jones, Jason Lewis
This national survey establishes a baseline of IR office capacities. The inquiry seeks to document the characterizations of IR Offices as they exist in 2015, including scope of work, reporting lines, and staffing. Guiding questions include: How many people work in the IR Office? What tasks are assigned to the IR Office? What fiscal investments do institutions make in their IR Offices? Survey responses represent IR Offices at 1,575 unique institutions. This session provides an overview of the findings and opportunity for dialogue about the need for benchmarking in this arena.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
NEAIR Best Presentaiton: Understanding the Impact of Test Optional Policy by Yuko Mulugetta
Over 850 institutions have adopted a “test-optional admission policy” (TOP), hoping to promote campus diversity by removing the barriers against various minority groups often presented by standardized testing. The TOP and diversity causal relationship, however, is not well researched. Using the data of six cohorts from Ithaca College, this study employs a quasi-experimental design and reveals that the treatment group (non-test submitters) did indeed increase the probability of a student being a minority, controlling for non-TOP related changes observed in the two comparison groups (test submitters) before and after TOP implementation. The study concludes that TOP positively affected diversity at each stage of the enrollment funnel: application, admission, enrollment and retention.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
No more playing Tetris with class schedules by John Petoskey
Class scheduling is often treated like a game of Tetris. Schedules are created one by one after orientation sessions. Inevitably room runs out and the students/pieces wouldn't fit in the oddly shaped spaces that remained. Then the whole game breaks down. Wingate University decided to change the game by creating all the incoming student schedules at once. This process has been able to reduce empty seats, better distribute classes/times/professors, improve student outcomes, remove under enrolled courses, and create needed sections. This session will discuss the research, excel tool , and guiding principles used in this process.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Predicting Academic Achievement and Persistence from a New Student Survey by Cinnamon Danube
IR professionals must identify risk factors for poor academic achievement and attrition. In Fall 2014, campus stakeholders collaborated with IR to create a New Student Survey (NSS). The goal was to determine whether midterm grades and survey responses were related to first term academic achievement and persistence into the spring after accounting for background factors. Models accounted for 54% and 16% of the variance in academic achievement and persistence, respectively. Poor academic achievement predictors were receiving a poor midterm grade, having visited the campus Students First Center (indicative of registration/financial aid issues), not having prepared for and/or attended class, and not feeling adequately prepared for college. Attrition predictors were receiving a poor midterm grade, having low academic self-efficacy, having plans to leave, and not engaging in extracurricular activities. Strategies for developing and interpreting data from new student surveys are described.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
Predicting Admission Melt: A Way of Increasing Financial Aid Efficiency by Robert Maddox
At many institutions, a financial aid package is presented to each admitted applicant at the point of decision release. In order for each admitted applicant to receive a carefully reviewed and competitive financial aid package, financial aid offices must begin packaging admitted applicants well in advance of the decision release deadline. Utilizing key admission variables, this poster describes how Emory University predicted admission melt - the process where applicants whose decision status is moved from admitted to a non-admitted status before the decision release. Analyzing the applicants admit status on key admission variables reduced the number of applicants who were unnecessarily packaged by the financial aid office by approximately twenty percent. Variables and methodologies for carrying out this project are discussed.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
Predicting At-Risk Students in Teaching Preparation Programs by Linda Clark
The issue to be addressed:
Proactively identify teacher certification candidates who may be at risk to fail
Importance to the Field:
Many undergraduates enter higher education with the intent to become teachers, only to experience failure in the certification process. Developing a pro-active identification process, implement programming for these students can enhance the pool of certified teachers.
The objectives of the session:
Demonstrate how entering student characteristics can be used to predict students who may be at risk in courses that are prerequisites to teacher certification

Demonstrate a model that other institutions can utilize to develop their own predictive ability
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Predictions of Persistence: Building a Model of Student Success by Ann Boudinot-Amin, Cheryl Harris
Many academic institutions face problems with declining undergraduate enrollment and retention. This session will describe a project to improve understanding of persistence and retention characteristics by using data from several years of graduating student surveys and other variables such as test scores, GPA, average course load, and demographics to build a predictive model to help researchers understand how data points taken much earlier in a student’s career may predict outcomes, including degree attainment and student satisfaction. Recommendations for using these results in decision support and potential interventions will also be discussed.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Prerequisites alignment analysis by Maria Vasilieva
Adopting the student success in the next class as one of the main outcomes will allow the college to build a robust alignment of the prerequisites. The research method developed at Pima Community College helped to reveal the hot points in the prerequisites sequence where the courses were not perfectly aligned. This analysis is supported by the findings, that, in general, for both developmental and college-level course, the grade in the previous course is an important student success predictor factor.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Reality Check: Driving Academic Unit Planning through Performance Data by Bryan Beck, Bryan Harvey
Planners frequently struggle to match meaningful performance data with decisions of consequence. As the culmination of a strategic planning process, departments were asked to develop plans responding to specific campus-wide objectives, with performance data in six areas. Dissatisfied with the extent of alignment, the campus-wide planning committee developed a rubric, based on the performance data, for a third-party assessment of plans, and also compared plans with performance data. These analyses allowed assessment of departmental plans in terms of both what they said and what the available evidence suggested they ought to have said. This session will present the process, outcomes, and lessons learned.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Recent Patterns of Graduate Student Debt: 2008 to 2012 by Karen Webber, Rachel Burns
This presentation identifies factors that contribute to graduate student debt and examines changes in recent student borrowing. Analyses use data from the 2008 and 2012 National Postsecondary Student Aid Studies and data from the Delta Cost Study and find continuing trends in borrowing, particularly for students in certain degree programs. Results show that more graduate students borrowed, and average borrowing in 2012 was approximately 50% higher than similar averages in 2008. Presenters will discuss findings and implications for policy and practice.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Reporting on All Students: Examining the Progression of Transfer Students by Loraine Devos, Stephen Nettles
The number of transfer students is increasing, transforming the student population of many institutions. Yet, most Institutional Research Offices continue to collect retention and degree completion rates for freshmen only, ignoring how transfer students perform on these metrics. It is essential that Institutional Researchers develop a system for tracking the progression of transfer students and examine their specificities relative to first-time students. Indeed, their efforts will give impetus to support initiatives tailored to these particular students. The presenters will discuss a study conducted at a large university. The study compared the progression of transfer students to that of entering freshmen and first-time students with the same number of credits. Attendees will also learn about the role played by student characteristics, financial aid, and early performance in accounting for the differences in student progression.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Retention Strategies: How do we identify and retain at-risk students by Billy Evers, Jacqueline MacNeil
This is a discussion on how IR and IE offices can utilize their expertise in moving the retention conversation forward. Through the use of quantitative and qualitative data analysis multiple departments can come together to identify, qualify, and intervene on behalf of students that are at risk.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Scholarly Writing: Advice from Editors by Gary Pike, Ann Gansemer-Topf, Angela Henderson, Sharron Ronco, Kristina Powers
This session is for individuals interested in learning more about writing for scholarly publications. A panel of journal editors will share insight, advice, and suggestions about writing for higher education journals in general, and institutional research-related journals specifically. Information about a variety of journals, their requirements, and related review and selection processes will be shared, including AIR Professional File, Journal of the First Year Experience and Students in Transition, New Directions for Institutional Research, and Review of Higher Education.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Seniors’ Perspectives about Advising Inform Student Support Decision by Sandip Thanki, Janice Le-Nguyen, Mick Haney
While it is commonly-recognized the importance of students’ perspectives about academic advising for the campus community to make evidence-based decision about academic advising programs, senior students’ experiences have not been empirically studied. Drawing on data of 293 senior students participated in 2015 National Survey of Student Engagement, this proposal is intended to address three research questions: 1) what are the senior students’ experiences with academic advising? 2) How are senior students’ experiences with academic advising related to perceived gains? And 3) how are senior students’ experiences with academic advising related to satisfaction with the quality of campus relationships?
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
Statement of Aspirational Practice for Institutional Research by Leah Ross, Jason Lewis
This session is designed to encourage dialogue about the AIR Statement of Aspirational Practice for Institutional Research published in February 2016. The ideas presented in the aspirational statement stem from the experiences and contributions of more than 260 AIR members. How does this vision for data and decision support resonate with you? What opportunities and challenges do the ideas present for your institution or organization? Join us for a conversation about this dynamic piece of work that is poised to grow with the field of institutional research. Examples of institutional activities in response to the aspirational statement will be shared.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
STEM Exemplar Institutions: Talent Development & Faculty Teaching Practices by Tanya Figueroa, Ashlee Wilkins, Sylvia Hurtado
Targeting six diverse institutions identified as ‘exemplars’ in producing STEM bachelor degrees among underrepresented racial minority students, this poster presentation demonstrates how the teaching practices of STEM faculty are influenced by their perceptions of student talent and/or achievement. We further investigate whether institutional context influences the way faculty view talent and ability at their respective institutions. This poster contributes to an understanding of how the collection of qualitative data by institutional researchers can help institutions better leverage faculty and instruction to become more productive in conferring STEM bachelor degrees among URM students. Insightful ways to scale up or strengthen innovations in the classroom for the expansion of talent development are offered.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
Students’ Experiences with Advising Inform First-Year Program Decisions by Sandip Thanki, Mick Haney, Janice Le-Nguyen
Drawing on 177 first-year student data from NSSE 2015 and institutional records, this discussion examines first-year student’s experiences with academic advising, including how they perceive the interactions with academic advisors and how it affects their retention and overall first-year experiences. The results indicate first-year students have overall positive perceptions regarding their experiences within their first year and with their advisors. Students who have used academic advising services were retained at a significantly higher rate for one-year than their peer counterparts. Using the data and results, campus administrators and academic advisors have made evidence-based decisions to enrich students’ first-year experiences.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Ten-year follow up of postsecondary enrollment delayers and nondelayers by Suchitra Gururaj
Students who delay immediate postsecondary enrollment may forego individual benefits, or, by not persisting once enrolled, represent misplaced resources for higher education institutions. This study revisits extensive P-20 state student-level data collected of the 2004 Texas high school graduating cohort and analyzed when graduates were six years out to predict persistence patterns for delayers and nondelayers. This ten-year follow-up study sheds light on these students’ graduation outcomes, including their use of diverse institution types, to shed light on the long-term effects of postsecondary enrollment delay. The goal of this session is to identify moments of pause, redirection, as well as possible intervention, through the education pipeline.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
The College Experiences that Influence Post-Bacc STEM Pathways by Tanya Figueroa, Sylvia Hurtado, Kevin Eagan, Krystle Cobian, Damani White-Lewis
Using longitudinal data gathered by the Higher Education Research Institute on a cohort of diverse STEM student graduates, this paper uses multilevel modeling techniques to identify how undergraduate experiences, perceptions of undergraduate training experiences, and institutional contexts predict two dichotomous outcomes among STEM bachelor's degree recipients 11 years after college entry: 1) whether the student had ever matriculated into a STEM graduate/professional program versus not; and 2) whether the student had ever joined the STEM workforce versus not. Implications from the findings can inform institutional policies and initiatives seeking to encourage students to pursue STEM post-college pathways, increase minority participation in advanced STEM programs, and provide a model that can be replicated by IR departments using alumni studies to understand student career decisions.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
The Overwhelming Influence of Unmet Financial Need on Student Success by Craig Rudick
Using FAFSA data we explore the influence on student success of a number of financial variables. Our analyses indicate that of these, Unmet Need has by far the strongest association with student success. In fact, we find that Unmet Need is the single best pre-College variable in our data at predicting retention, explaining more variance than HS GPA and ACT combined. We investigate how Unmet Need affects both the rate and timing of attrition, finding that it is the dominant factor associated with 1st-Spring attrition, but plays less of an impact on the attrition of upper-division students. Over the past several years we have observed a large increase in the number of students attending UK with high Unmet Need. We use predictive models to estimate the impact on the retention rate of this increased Unmet Need burden, and create simulations which optimize the distribution of need-based financial aid to affect retention.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
The Relationship between General Education, Student Engagement, and Success by Resche Hines, Angela Henderson
This poster illustrates findings from one institution’s analysis of the impact of first year general education courses on student retention. The analysis examines the ways in which completion of general education courses within the first year affect the probability of student retention and provide a meaningful proxy for student engagement’s impact on student retention. Findings presented will increase participants’ understanding of how integrative learning and student engagement can be leveraged not only to enhance strategies to improve student retention, but also to synthesize information about which first year courses are most important for student success.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
The Role of IR in Developing Processes for Evidenced Based Decision-Making by Susan Ledlow, Daryl Davis
To foster deep reflection on data that results in meaningful change, institutional researchers must provide leadership in developing processes for reviewing and acting upon data. Collaboration between the Offices of Institutional Research and Career and Workforce Education revitalized Valencia College’s Academic Review process. The offices jointly determined program data needs and developed a tool to visualize program and student success data. Academic programs now receive their data in advance of individual program reviews facilitated by Career and Workforce staff. The new process has both made program data more accessible and generated College-wide interest in using data for improvement initiatives. This effort is presented as a case study demonstrating why and how IR Offices should co-create data analysis and decision-making processes with stakeholders. Participants will share their own experiences in co-creating evidence based decision-making processes.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
The role of Student Satisfaction in Academic Engagement by Osundwa Wanjera
This paper empirically demonstrates that student engagement is linked to satisfaction through various channels. Although research favors the benefits of engagement, little is known about the mechanisms through which satisfaction influences engagement given previously applied models are inadequate. Hence the role of satisfaction in engagement remains imperfectly understood. This paper fills this research void by testing whether these relationships are sensitive to choice of estimation method and particularly the use of panel data structural simultaneous equation models. The analysis finds that there is a positive and significant bidirectional relationship between satisfaction and engagement even when the model specifications are altered.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Title IX, survey design, campus sexual assault, reporting by Kathleen Chaballa
Anticipating that reporting on issues surrounding unwanted sexual experiences on campus will become mandatory in the future, this discussion group focuses on how to design an effective survey that will yield actionable data. The facilitator will provide examples of the University of Denver survey, pointing out both the successes and challenges of their first survey, conducted in 2015, and will briefly discuss changes made to the 2016 survey, before opening up the group to questions and discussion.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Transferring Up vs Down in Retention Models for Intervention by Benjamin Silva, Jeff Hoyt
The emergence of the National Student Clearinghouse allowed for the differentiation in discontinuing students to account for transfer-outs in retention modeling. This session presents a further expansion of the outcomes in utilizing the Carnegie classification system to discriminate between transfers to lower versus upper tier programs. Including variable transfer institute levels allows for additional freedom in the relationship with the predictors which in turn offers new interpretations to transfer-out motivations. Additionally we consider the inclusion of transcript requests as an indicator variable in order to facilitate the application of the model in directing campus interventions for improved institutional retention.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Transforming Institutional Research into Institutional Results by Purva DeVol, Lou Guthrie
With its 1st institutional effectiveness website, first-ever scorecards for the college/divisions, & corresponding dashboards that use external benchmarks, Kankakee Community College will share how they are using data to help make critical decisions. Moving from producing IR data to using the data to drive change & produce results was not an easy task. Specific examples of how KCC's IR presents data their administrators, faculty, & student services staff use to make decisions will be shared. KCC's leadership team has a scorecard for each area they lead with KPIs that align with KCC's scorecard w/benchmark data added. Impressive progress from 19.7% completion in 2014 to 26.9% completion in 2015 shows the value of the approach. This best practice is transferable to other CCs. Objective of session is to provide a template CCs can use to give them the ability to track internal & benchmark data to help their CC improve & to set reasonable goals to increase student outcomes.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Undeclared: Does 15 Always Finish? by Dana Weigel
Universities nationwide are constantly looking for ways to improve retention and graduation rates. At Western Washington University, administrators have shown interest in the “15 to Finish” initiative, but because students at WWU are not required to declare a major until their junior year, researchers wanted to know what impact this may have on both retention and graduation. To help University administration understand factors affecting student success, the Office of Institutional Research looked at several cohorts of incoming freshmen to see if increased credit load, time of major declaration, pre-college academic preparation levels, and demographic characteristics were correlated with retention and graduation. This poster outlines methodology, results, and how the office plans to present these findings to the university community.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
Understanding Postsecondary Success: A P-16 Metropolitan Partnership by Katherine Rote
An in-state, public university obtained the unit level student records from a high school district in the university’s metropolitan area. Greater levels of collaboration and information sharing between secondary and postsecondary institutions is crucial in order to make appropriate, data-driven changes so that students can achieve greater levels of educational attainment leading to meaningful credentials and careers. Many states are implementing state-wide longitudinal databases that make this type of data analysis feasible. Analyses of students’ records examine whether conduct, course performance, and certain types of coursework in high school influence a student’s likelihood of persisting or experiencing success in college. Findings are presented.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
Using a Multi-Factorial Dashboard Tool to Track Success in Writing classes by Maria Vasilieva
To track the changes in student success after reducing class-size a multi-factorial dashboard (MFD) was developed using Microsoft Excel (a low cost, in-house development project) to ensure that the improved success was the result of reduced cap-size and not ancillary factors. Many variables were included to ensure that the reduced cap-size did not have any unforeseen negatively consequences. During the evolution of the MFD several changes were made to improve data presentation and understanding. Modification to the excel document as we developed this MFD allows for a single-stop student success tracking tool.
Topic Area: IR Studies for Campus Decision Support   |   Format: Discussion Group
Using IR Data to Identify Students At Risk of Leaving Before Year Two by Ty Cruce, Edgar Sanchez
With the nation currently not on track to meet its goal of greater college degree attainment by 2025, Lumina Foundation (2015) has stressed that more attention be given to student pathways through postsecondary education. At the institution level, knowing which students are less likely to persist to the second year of college can help faculty and administrators to target more resources toward improving these students’ chances of success. Early identification of students who are at risk of leaving the institution is an important task often conducted by IR offices, and this session will demonstrate how institutional researchers can analyze data available at the time that students are applying to college to predict students’ chances of departure.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Using Predictive Analytics to Guide Retention Interventions by Dev Nambi, Max Savishinsky
In this session, University of Washington staff share how historical student data is being used to identify the likelihood of current students leaving the university within a specific period of time. Using a wide range of risk factors that correlate with students stopping out, the university can pinpoint students likely to leave and design, time, and deploy proactive interventions that are responsive to, and customized for, the most influential factors that put individual students at risk – not simply low scholarship. By combining predictive data with customized advising interventions, the university has improved its understanding of persistence, and anticipates notable improvements in retention rates. Presenters will review the logic and data underlying the predictive model, and attendees will discuss the ethical and practical implications of using predictive data, as well as the opportunities for outreach, student interventions, and cross-campus IR collaborations.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Using Statistics and Machine Learning to Predict Student Success by Brent Drake, Monal Patel, Ian Pytlarz, Xi Zhang
Most higher education institutions have implemented student success initiatives towards helping students succeed through their college career. Higher education institutions also are faced with limited resources for implementing student success initiatives. At Purdue, we are using statistical modelling and machine learning techniques to predict student outcomes before they happen, allowing Purdue to target student success resources at students that are most likely to perform poorly. Our presentation will focus on the methodology we used for building our models, the features of the data which ended up being most important to student success, how we plan to continue improving our models, and finally how Purdue is operationalizing these learnings.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Using Text Analytics to Understand Student Course Scheduling Preferences by Greg Michalski
The problem of course scheduling, particularly at large, complex, institutions has been examined (though not entirely solved) from differing perspectives ranging from the qualitatively administrative to the quantitatively mathematical/computational. This session presents new findings and actionable results obtained from a large case study at a prominent four-year, multi-campus state college involving (n = 5,024) responses to comprehensive online Class Schedule Preference (CSP) designed to capture both quantitative and qualitative (i.e., open text response) data directly from actively enrolled students. Open text data was extracted and mined using a widely recognized, commercially available, text mining application. The results of several models are explored, compared, and applied to improve understanding of the course scheduling problem with specific reference to existing and emerging empirical work in the areas of text mining and course scheduling research.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Utilizing Course Evaluation Results to Assess First Year Seminar Outcomes by Ada Kwanbunbumpen, Katie Busby
Course evaluations are administered to solicit feedback on course and teaching effectiveness. Some institutions also gather indirect evidence of student learning though their course evaluations. This poster session will recount the methods used to extract, analyze, and present longitudinal course evaluation data and to determine the extent to which first year seminar learning outcomes were achieved using indirect assessment methods. This session will address the challenges faced during the analyses (e.g., low response rates, maintaining confidentiality, and changes in items across time). Participants in this session will be able to use course evaluation results effectively to inform assessment and evaluation efforts.
Topic Area: IR Studies for Campus Decision Support   |   Format: Poster Presentation
What Impact Do Student Services Have on Student Outcomes? by Lou Guthrie
How do student services costs impact student outcomes? For example, does spending more on co-curricular activities increase your persistence levels? Would your executives like the answers to these types of questions? We can help by sharing the results the Maximizing Resources study. It was complete in Sept. 2015 and covered the following SS: admissions, recruitment, advising, counseling, career services, financial aid, registrar/student records, tutoring, testing services, co-curricular activities, & others. Costs are examined internally as % of the SS budget devoted to each function. An excellent way to see if your college's costs align with priorities. Costs are compared to other CCs in the study as $ per FTE student and $ per student users of the services. Finally costs were examined in relationship to student outcomes including graduation/completion rates, transfer rates, credit college-level retention & success, & persistence rates.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Who lands a job first? Exploring time to employment post-graduation by Rachana Bhatt, Angela Bell, Lori Hagood, Tan Tran
The labor market outcomes of college graduates is of great interest to higher education administrators and researchers. In particular, how long it takes for students to find employment is becoming important indicator of institutional performance. This research examines how employment outcomes vary across students with respect to their postsecondary choices (type of institution attended, degree, major) and student characteristics (race, gender). We conduct this research using a new state longitudinal data system (SLDS) from Georgia. This presentation will offer insight on how to work with SLDS as well as highlight trends in labor market outcomes for college graduates in Georgia.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session
Working for Pay and High-Impact Practice Participation by John Zilvinskis, Alexander McCormick
High-Impact Practices (HIPs), such as study abroad, research with faculty, and service-learning, have garnered the attention of educators and researcher alike; however, since the participation in these educational opportunities may be cost prohibitive, concerns regarding access exist. One behavioral factor of interest related to these concerns is student employment and how it is related to HIP participation. Using data from the 2015 administration of the National Survey of Student Engagement, the authors explore diverse patterns of engagement when considering the differing circumstances of part-time and full-time, first-year and senior students with regard to paid employment on campus, off campus, or both. The findings can help guide institutional practice with regard to promoting HIP participation for all students.
Topic Area: IR Studies for Campus Decision Support   |   Format: Speaker Session

IR Technologies

(Replaced with Best Presentation) Course Enrollments - Our Interactive Tableau Dashboard, from the Why to How by Mark Leany
How full are course sections? How many waitlist students need the course versus just want a better time/teacher? How do enrollments compare to last year? These are some questions that decision-makers at our university need answered. And they need results weeks (or months) before every term. Over several years, we have worked on this process. This year, we finally created an interactive Tableau dashboard that delights the main stakeholders. In this session, we will show you how we did it. We will explain the history of the prior variations. We will discuss some of the issues that we addressed along the way. And we will show you the final result and some of its features. We do not expect you to duplicate our dashboard. However, the information you will learn from this session may help create your own course enrollments and management tool.
Topic Area: IR Technologies   |   Format: Speaker Session
Actionable Intelligence: Big Data For Student Success by Barrie Fitzgerald
Even before students enter the classroom, they provide institutions with data on a daily basis. Typically, institutions collect the data and use small portions of data to assist with academic success. By changing the use daily data, one campus provided faculty with early indicators on student success and a communication system to assist with students’ success. Utilizing these methods to communicate and facilitate strategies can not only increase retention and graduation, but also offer students an opportunity at a better life. The presenters will review techniques and technologies to give students a higher chance of success.
Topic Area: IR Technologies   |   Format: Speaker Session
Automating the Collection of Survey Data: Welcome to the 21st Century by Michael Bolen, Jennifer McCarthy
This presentation will discuss how and why an IR office at a large university successfully implemented an automated mandatory graduation survey within each student’s application to graduate, as well as an automated reporting tool to quickly put actionable data into the hands of senior leaders. Significant consideration will be given to navigating the institutional politics surrounding this proactive initiative, as well as the technological requirements necessary for implementation. Attendees will be encouraged to apply the lessons learned and techniques employed at their institutions.
Topic Area: IR Technologies   |   Format: Speaker Session
Challenges of Assessing Impacts of Community Engagement by Suchitra Gururaj
This discussion will address the practice of collecting meaningful data about community engagement within the context of myriad factors, including the development of competency-based learning; students’ motivations to strengthen their desirability on the job market; institutional assessments of their public service missions; as well as student, faculty, and institutional goals to measure truly reciprocal learning experiences.
Topic Area: IR Technologies   |   Format: Discussion Group
Choosing and Implementing an Epic Accreditation Management System by Lenore Benefield, Paul Snyder
A feasible, practical, affordable planning, assessment, and accreditation management system can be developed using Microsoft SharePoint. This presentation will describe how Florida Gulf Coast University developed and implemented its own system using SharePoint to accommodate IE needs to satisfy accreditation and state reporting requirements. Presenters will discuss the criteria they used to choose between a customized in-house system and a plethora of commercial choices. SharePoint is a web application platform that provides lists, libraries, search tools, calendars, customizable pages, and integrates with Microsoft Office applications.
Topic Area: IR Technologies   |   Format: Speaker Session
Creating a Dashboard with MS Access by Yenny Anderson
If you have a static dashboard that you wish to "spruce up" and you are comfortable using MS Access, then you must attend this session. Using MS Access as the foundation of a static dashboard is very economical primarily for institutions that have not been able to produce an interactive and dynamic dashboard. This presentation will demonstrate how simple it is to use MS Access to develop a static dashboard.
Topic Area: IR Technologies   |   Format: Poster Presentation
Creating Data Visualizations Using R: An Introduction for Non-Programmers by Arie Spirgel
Creating graphs is a central part of the IR workload, but it can be frustrating. If you make graphs using Excel, you might spend hours mindlessly pointing-and-clicking, which then has to be repeated if updates to the graphs are required. The presenter will begin by arguing in favor of two points: First, IR professionals would be better off using R (a free and open-source programming language), and second, even those without any programming experience can learn R well enough to reap its majesty. A demonstration will be given on how to create and modify a scatter plot using ggplot2, an R package for building highly customizable visualizations. The primary goal of the presentation is for audience members to leave feeling empowered to learn to use R to improve the quantity, quality, and reproducibility of their work.
Topic Area: IR Technologies   |   Format: Speaker Session
Creating Tableau Dashboards with IPEDS Data: How To, Tips, and Tricks by Sean Hoffman
The IPEDS Data Center offers a wealth of institutional metrics for benchmarking. Tableau simplifies data visualization and presentation. Together, these tools allow for the quick and simple creation of benchmarking dashboards that can be used in both policy analysis and planning, and also deployed for self-service utilization. This session will demonstrate how data in the IPEDS Data Center can be used to build comparative dashboards in Tableau. The session will include a step-by-step walk-through of a basic dashboard, as well as examples of how to use more advanced techniques such as calculated fields, parameters, and dashboard actions to enhance benchmarking visualization.
Topic Area: IR Technologies   |   Format: Speaker Session
Data Governance at APUS: A Play in Three Acts by Elizabeth Wallace, Dave Becher, Geoff Koch
Consistency, integrity, transparency and cohesiveness are four philosophies that are integral to the culture of the Office of Institutional Research at American Public University System (APUS) and are applicable to Institutional Research at any university. APUS implemented the Data Cookbook to support these four philosophies. By making data definitions easily accessible and incorporating them into systems and processes, institutional terms are better documented and understood throughout the organization. Data definitions may be searched through the school’s intranet page and specific definitions are embedded in reports and dashboards. An agile process has resulted in over 2,000 terms being defined, reviewed, and approved in the Data Cookbook. This presentation will provide attendees with a technical, interactive demonstration of how Tableau and the Data Cookbook have been integrated to provide consistent and clear documentation of dashboards.
Topic Area: IR Technologies   |   Format: Speaker Session
Data Mining Demystified: Techniques and Information for Model Improvement by Nora Galambos
The presentation will focus on various data mining techniques to improve modeling results. An understanding of the diagnostic output is key to the modeling process. As such, examples of the various graphic and statistical output important to comparing modeling results, like receiver operating characteristic (ROC) curves, lift, Gini, gain, and captured response are presented and defined. Since data mining accommodates large numbers of predictors, variable selection methods such as LASSO and clustering will be discussed. For model improvement, bagging and boosting are explained along with how to avoid overfitting the model. The methods will be accompanied by examples using actual higher education data to demonstrate their correct use. The presentation will conclude with a discussion of a data mining model that incorporates a wide range of measures including transaction data, like course management system logins and advising visits, utilizing the presented techniques.
Topic Area: IR Technologies   |   Format: Speaker Session
Developing a New Website for Data Transparency: Vision to Implementation by Julio E. Moreno, Paul Rusinko, Jennifer Anderson
This session is a case study of how one Institutional Effectiveness department at a large, urban community college implemented a new website for the purpose of data transparency, in order to positively inform and impact strategic decision-making among key stakeholders across the institution. The session will include an overview of each step -- and the associated, departmental consideration and thought process -- that was required for project completion, from visioning through implementation, as well as details on the challenges that needed to be overcome for implementation.
Topic Area: IR Technologies   |   Format: Speaker Session
Developing a Peer Comparison Database to Support Strategic Planning by Michael Bolen, Jennifer McCarthy, Karina Pineda
This presentation will discuss how and why an IR office at a large university successfully developed a comprehensive database of data about peer institutions for quickly reporting actionable data to university leaders. Significant consideration will be given to navigating the unforeseen problems surrounding this proactive initiative as well as the technological requirements necessary for implementation. Attendees will be encouraged to apply the lessons learned and techniques employed at their institutions.
Topic Area: IR Technologies   |   Format: Speaker Session
Different Questions, Different Views: A Guide to Selecting a Visualization by Lauren Young, Craig Abbey
Effective data visualization, a key tool in any institutional researcher’s toolbox, depends on a match between the questions asked and the types of visualizations produced. The right match makes data highlights leap off the page, but the wrong visualization can obscure key points. Our presentation will outline the main types of questions asked of institutional researchers, identify the best visualizations for each type, and explain the aspects of each visualization that make it the right tool for a particular job.
Topic Area: IR Technologies   |   Format: Speaker Session
Engaging Campus Constituents with Infographics and Streamlined Reports by Angela Henderson, Resche Hines, Patti Sanders
In order to provide useful data for audiences who increasingly expect immediate access to talking points in lieu of comprehensive data reports, IR must embrace new technologies and data designs. To be effective in this endeavor, IR needs to provide administrators with easily digestible data ready for distribution to broader audiences. To achieve this goal, IR must function as data analyst, interpreter, and translator, ingesting extensive datasets and distilling them into concise infographics and reports. This session will discuss one institution’s experience with implementing infographics and simplified visual reports as a means to foster data engagement across campus.
Topic Area: IR Technologies   |   Format: Speaker Session
Excel Dashboards: You name it, we have it! by Ozlem Kacira, David Purkiss
Data extraction, transformation, and presentation is an essential factor in how Institutional Research Departments function. Especially, data visualization which allows users to identify patterns and anomalies and understand the big picture. This presentation will Introduce and discuss how the visualization process affects data and its perception, the main benefits to IR offices and overall effectiveness in data management and reporting. The discussion will be guided by demonstration of how to create Excel dashboards and discuss what is important when mining unstructured data to use in these dashboards. Example reports will be presented along with time-saving tips and tricks in Excel. Also, how the use of dashboards has significantly improved our reporting practices and meet end users’ expectations will also be discussed.
Topic Area: IR Technologies   |   Format: Speaker Session
Going Digital: Building an Accountability Dashboard at a University System by David Blough, Heather Kim, Todd Bailey
After more than two decades of print-based accountability reports, the University of Wisconsin System transitioned reporting of key performance indicators into the digital age with an online, interactive accountability dashboard. Learn how our team responded to several opportunities and challenges during dashboard development and updates, including: balancing stakeholder needs and expectations, getting consensus on accountability measures, adapting to changing external requirements, building skills and capacity in dashboard technology (Tableau), maximizing available resources, and altering workflows to efficiently and accurately create and update content. A brief demonstration of the dashboard will be included, along with statistics on web traffic. We’ll share lessons learned and tips for others developing public-facing dashboards.
Topic Area: IR Technologies   |   Format: Speaker Session
How and Why IR Can (and Should) Build a Data Warehouse by Elizabeth Barlow, Seth Ovadia, Timothy Wasserman
Institutional Research offices can and should be the primary business owner of data warehouses containing institutional data, but the skills required can make this a challenge. This session will outline the staffing and skill level changes one IR office had to make as it initiated and oversaw the development of a dimensionally modeled data warehouse. Emphasis will be placed on the roles and skills needed to sustain a data warehouse when IR is the primary business owner and plays the key data governance role.
Topic Area: IR Technologies   |   Format: Speaker Session
Improving learning outcomes and program design using multi-modal analytics by Karen Matheson, Catherine Mutti-Driscoll, Stephen Kerr
With the relatively recent growth of online education programs within the US University system, it is essential for faculty, program administrators, student support staff, and analysts to assess student outcomes and program success. The University of Washington started its first fully online undergraduate program in the fall of 2013 and, as part of this effort, allocated resources towards developing a set of analytics that could improve student retention and assess program outcomes. In this session, we will outline the different types of analytic data used, along with results and lessons learned along the way. Analytic data presented will include student survey data and online course metrics.
Topic Area: IR Technologies   |   Format: Speaker Session
Informing Academic Program Decisions with Labor Market Information by Elizabeth Riesser, Brian Perry, Alicia Crouch
Preparing students to enter the workforce is an important aspect of higher education. As administrators seek to make decisions about the best ways to educate and prepare their students, labor market data can provide valuable information on academic program viability. Data points such as employment growth, median wages, and job openings are all indicators of whether an occupation is growing and in demand in the region. This presentation will demonstrate an online program alignment tool developed by a community college system that connects academic programs to labor market information.
Topic Area: IR Technologies   |   Format: Speaker Session
Instant Comparison: IPEDS Survey Files Working for You by Emma Morgan
IPEDS provides useful longitudinal and peer comparison data, but it can be time consuming to sift through all of the available data to select institutions and variables of interest. This session will describe how one IR office used R to automatically shape and merge full IPEDS survey files into a usable format that both provides peer and longitudinal comparison data and is compatible with Tableau. While some code will be shown for demonstration, attendees are not expected to have prior exposure to R.
Topic Area: IR Technologies   |   Format: Speaker Session
Maximizing Fiscal and Physical Efficiencies in Space Management by William Nunez, Jennifer Dam Shewchuk
At a time when fiscal and physical resources in higher education are already at a threshold, decision-makers are consistently being asked for more. While many tools and resources for decision makers are available related to student and financial data, one of the most important and costly institutional resources - space - is a topic that receives less focus and discussion. This presentation will focus on strategic space management and provide a perspective of best practices, how space has been integrated in university reporting and analytics, and how enhanced space management has enhanced the fiscal return to the university.
Topic Area: IR Technologies   |   Format: Speaker Session
Presentation matters: Creating visually appealing graphs, tables, and text by Anne Marie Karlberg
This interactive session provides tips from renowned experts (Edward Tufte, Stephen Few, Presentation Zen, etc.) on effective data presentation strategies of text-based, tabular/numerical and graphical information. When these tips and principles are applied to the presentation of information in documents and formal presentations, it can make these presentations and documents informative, clear, accessible and thought-provoking, so that meaningful conversations are initiated, producing actions with results. As a result of participating in this session, participants will improve their graphical, tabular and numerical, and text-based presentations and documents.
Topic Area: IR Technologies   |   Format: Speaker Session
Providing What Students Need to Know: A Student Dashboard System by Takeshi Matsuda, Yuki Watanabe, Hiroshi Kato, Katsusuke Shigeta
This poster session addresses an online dashboard system for students in Japanese Universities. The focus of this system is data-driven support for academic success in students’ selecting subjects. Among student’s criteria of selecting classes, the degree of difficulty to get credits contains complicated elements. The dashboard, therefore, indicates there determining factors in the difficulties: (1) difficulty level of learning contents; (2) learning activities and the readiness required in the class; (3) tendency for the teacher in charge to over-/under-mark. After confirming figures and facts, a student can decide whether he or she would apply for that class or other with referring to additional information.
Topic Area: IR Technologies   |   Format: Poster Presentation
Ready, set...Wait! Navigating a bumpy ride to online course evaluations by Tafaya Ransom, Lan Ma
Morehouse College took the leap into online course evaluations to reap some of their known benefits in the spring of 2014, charging a new IR office of one (eventually two) with developing an online course evaluation system using existing campus resources. IR and assessment professionals will learn specific pros and cons of building a campus-wide course evaluation process around a common cloud-based enterprise survey tool, Qualtrics. This presentation will unpack three aspects of our experience with online course evaluations: the importance of institutional context, specific system/process details, and the challenges – from technical to interpersonal – we encountered with implementation. Our current approach offers a low-cost, workable online course evaluation solution that is particularly well-suited for decentralized course evaluations and feasible for centralized evaluations in similarly resource-strapped environments.
Topic Area: IR Technologies   |   Format: Speaker Session
Research metadata analysis to promote interdisciplinary collaboration by Heather Lent, Lydia Snover
This discussion will address tools and best practices used to analyze institutional research metadata to promote interdisciplinary research collaboration across programs, departments, campuses, disciplines, and institutions.
Topic Area: IR Technologies   |   Format: Discussion Group
SAS Codes Used in the Quadrant Analysis of Student Satisfaction Survey by Robert Zhang
Quadrant analysis is one of the popular methodologies used in analyzing the student satisfaction survey. A quadrant chart is designed to help the audience understand relationships between two factors – satisfaction and importance – affecting an issue. Technically speaking, there are 4 steps in quadrant analysis. They are a) obtain mean scores of importance and satisfaction for each question or issue, b) obtain median scores of importance means and satisfaction means, c) generate a quadrant chart, and d) perform quadrant analysis based on the quadrant chart. This poster will demonstrate SAS codes used in the first three steps, as well as the final quadrant analysis.
Topic Area: IR Technologies   |   Format: Poster Presentation
Survey technologies: Dealing with mobile device impacts on assessment data by Steve Wygant, Danny Olsen
Use of mobile devices (especially smart-phones) to complete assessment surveys has significant, and largely negative, impacts on both the quantity and quality of survey data collected. Given their popularity, we can be sure that some percentage of students will respond to any survey we send them using a smart-phone, whether we intend them to or not. Considering these trends, it behooves us to make our assessment surveys more amenable to smart-phone usage. This presentation details some differences between data collected via desktop/laptop computers, tablets and smart-phones and presents several strategies for making surveys more mobile-friendly.
Topic Area: IR Technologies   |   Format: Speaker Session
The critical role of IR teams in enhancing business intelligence capability by Anne Young
There is a growing interest in business intelligence (BI) in higher education to support strategic planning, performance and IR. While the focus in the early phases of developing a BI program of work is primarily on the selection and use of BI software, promoting engagement in BI by staff across the organization is critical to ensure that the information is actively sought and easily interrogated to support decision making. This session will present a case study of how the University of Newcastle, Australia, has enhanced its BI maturity, particularly using visual and predictive analytics. The session will also highlight the value of a BI benchmarking exercise conducted among 28 Australian and New Zealand universities that enabled institutions to assess their current BI capability compared with their peers and develop their BI roadmap. The role of IR teams in building capability across the institution in using BI tools and developing outputs will also be discussed.
Topic Area: IR Technologies   |   Format: Speaker Session
The impact of invitation wording on web survey response rates by Tim Stanley, Angela Ward
Utah Valley University conducts a large scale survey of students every semester. Building on existing literature, we took this opportunity to conduct a randomly controlled experiment on the impact of various differences in wording between email invitations. Four variables were tested, creating an array of 16 groups representing every possible combination of wording differences. The tested variables include personalizing the (1) salutation and (2) closing, (3) the specificity of topics addressed by the survey, and (4) focusing the application of the results on the future or past. Data on response was analyzed using simple multiple logistic regression. This allowed us to estimate the impact to response odds for each variable and the interactions between variables. Findings will be valuable in improving response rates in future surveys.
Topic Area: IR Technologies   |   Format: Poster Presentation
The Use of Visualization to Support a Data Culture for Student Success by Laurie Heacock, Bill Schneider, Daryl Davis, Emily Baranello
Many institutions lack both the skills in data analysis and the tools that are required to leverage data as a strategic asset. Data visualization utilizes technology to contribute to our data sense making so we can weave the data we collect into the fabric of understanding. The panelists represent a network, a college, a system and a technology solution provider and share insights on how to use a data informed approach to improve student outcomes. Attendees will learn why it is important for colleges to consider an analytics solution and how visual analytics moves beyond standard BI. Attendees will see a demo of the visual dashboard used by Valencia College to manage enrollment and student progression and how requests for ad hoc data reports were reduced by 60%. A second demo illustrates the visual dashboard used by the North Carolina Community College System for performance management. The panelists will share best practices, lessons learned and tips for colleges just getting started.
Topic Area: IR Technologies   |   Format: Panel Session
University Innovation Alliance Members Discuss Predictive Analytics by Sandra Archer, Julie Carpenter-Hubin, Brent Drake, M. Paige Borden, Salvador Castillo, Bridget Burns, Bethan Cantwell
Corporations have long mined data to predict and influence customer decisions. Universities are now adapting this technology to gather data on students and use the data to keep students on track to graduation. The University Innovation Alliance (UIA) is a consortium of 11 large public research universities committed to making high-quality college degrees accessible to a diverse body of students. UIA institutions with established predictive analytics initiatives are serving as mentors to the remaining members, who are in various stages of implementation. Representatives from several UIA institutions will serve as panel members with the goal of bringing our experiences, results, and recommendations to the broader higher education community. Topics will include: selecting a predictive analytics software solution, start-up time and resources, data preparation, necessary campus culture changes, and the successes, challenges, and lessons learned along the way.
Topic Area: IR Technologies   |   Format: Panel Session
Using Data Visualization Software to Develop an Interactive Factbook by Christopher Pena
The field of institutional research promotes the use of institutional data in strategic planning and program development at higher education institutions. However, as the demand for institutional data increases, institutional research offices must find new and innovative ways to deliver complex information to diverse audiences. This session presents our experience using Tableau to transition our factbook from a collection of static PDF reports to an interactive self-service reporting tool with dynamic tables and engaging visualizations. We will discuss the benefits of developing an interactive factbook and how we leveraged growing institutional interest in analytics and data literacy to gain executive sponsorship for our project. We will also review the challenges we encountered and how we overcame them, such as balancing competing reporting and analytics needs in a public factbook and adapting to unexpected system and data limitations.
Topic Area: IR Technologies   |   Format: Speaker Session
Using Excel Dashboards to Monitor, Manage, and Report on Strategy Execution by John Hofmann
Monitoring a strategic plan, assessing its impact, and communicating results to stakeholders remain some of the most important but also vexing challenges institutions face while executing their strategy. This session will review an Excel-based project dashboard that on one page communicates the University’s key initiatives, completion timelines, and budget metrics, summarizes accomplishments, and outlines next steps. Besides reviewing the components of this dashboard home page, attendees will learn how this tool drills-down into mirror-like tactical work plans for each initiative that describe tasks and assignment leads, target completion dates, key performance measure, and include standardized progress status reports templates.
Topic Area: IR Technologies   |   Format: Speaker Session
Using Tableau Mapping Features to Build a Variety of Interactive Dashboards by Mark Leany, Tim Stanley
Tableau has built-in mapping functionality that allows you to create dashboards and interactive visualizations based on common geographic elements such as Country, State, City or ZIP Code (i.e. you can build maps without having to geocode your data). Whether you own a Tableau license or are considering the free public version, you can take advantage of this mapping. In a step-by-step format, we will show: (1) quick and easy creation of a basic map of institutional data; (2) different fields used to map coordinates; (3) different ways to display mapped information; (4) correction of non-standard or unrecognized geographic names; (5) coloring methods (including using categories vs. sums); (6) creation of time-lapse maps and (7) some other unique variations that allow you to create a more complete result. No Tableau experience is necessary to appreciate this, but interest in dashboards and mapping is required.
Topic Area: IR Technologies   |   Format: Speaker Session
Visibility in Response Options: Considerations for Web Survey Design by Luanne Holden, Elin Trollang, Daniel Sturtevant
Our poster presentation will highlight the importance of question format in web survey design. We will show that radio buttons and drop boxes are not equivalent measurement tools and that visible presentation of options has an effect on how respondents answer. A brief survey will include two questions as part of the study. We will survey three groups of students where one group receives, for the first question in the study, a radio button format, the second group will be presented with a drop box where part of the options are initially displayed, and the third group will receive a drop box where none of the items are visible. The second question for each group will be the same format with the options randomized. We expect to replicate the primacy effects from previous studies and show that respondents are more likely to choose one of the visible options than an option that is initially hidden.
Topic Area: IR Technologies   |   Format: Poster Presentation
What you should know before using—Insights from first-time users by Daniel Regnier, Nicole Holland, Sun No
The demand for data from Institutional Research offices seems only to be increasing from both internal and external sources. However, it can take a great deal of time for staff to produce all this data, which in turn, reduces the efficiency of their office. This poster outlines the presenters’ insights as first-time users of Tableau to not only increase office efficiency, but also produce more dynamic reports for stakeholders.
Topic Area: IR Technologies   |   Format: Poster Presentation

Reporting and Transparency

Benefit or Burden: External Data Reporting by Kristina Powers, Julie Carpenter-Hubin, Yvonne Kirby, Eric Atchison, Yanli Ma
There is little to no debate from institutional researchers that the scope of reporting to external agencies has increased in volume, quantity, and complexity. However, there is some contention as to whether external reporting represents benefit, burden, or both. This panel brings together a set of authors from a volume entitled Benefit or Burden: External Data Reporting in New Directions for Institutional Research (released in spring 2016). Panelists will discuss key issues in external reporting, including: Transparency, Relationships, Optional Reporting and Surveys, Benchmarking, and the Future of Reporting. Audience members will have an opportunity to ask questions to the panelists.
Topic Area: Reporting and Transparency   |   Format: Panel Session
CDS- Annual Update, Use in Rankings and Proposed New Outcomes Measures by Robert Morse, Stephen Sauermelch, Teri Hinds, Matthew Gazda
Every year hundreds of IR offices fill out the Common Data Set (CDS) and the U.S. News Best Colleges, College Board and Peterson’s surveys. For the first time at an AIR forum U.S. News will explain in detail how the Common Data Set (CDS) is used in the Best Colleges rankings. This sessions will describe how these U.S. News, College Board and Peterson’s and CDS surveys overlap and how IR offices can most efficiently fulfill these important data requests and reduce their survey burden. The Association of Public and Land-grant Universities will explain why the Student Achievement Measure is a comprehensive measure of completion and why the CDS should add it in order to go beyond the traditional student six-year gradation rate the CDS is collecting now. The session will also discuss the basics of the CDS and what changes are being considered for future CDS editions.
Topic Area: Reporting and Transparency   |   Format: Panel Session
College Portraits At A Glance: a New Tool for Evidence-Based Storytelling by Teri Hinds, Christine Keller
As policymakers from the White House to state and local boards increasingly call for greater accountability, campuses need better tools to help present data embedded within appropriate institutional context. This practice of evidence-based storytelling has long been a part of the VSA's College Portraits, helping institutions tell their stories about student learning, participation in high impact practices, and student success. College Portraits At A Glance, a new tool that launched in fall 2015, allows institutions to create custom pages with snapshots of their College Portrait data tailored for a particular issue or theme, viewable either online or as hard copy, with the information needed to support an institution’s story in an integrated, easily understood, graphic format. Participants of this session will learn how to use College Portraits At A Glance at their own institutions.
Topic Area: Reporting and Transparency   |   Format: Speaker Session
Designing a Taxonomy for Data Reporting across Institutions and Agencies by Emily Burke, Christine Plepys
This session will show how to influence data collection and reporting standards across multiple stakeholders by outlining the creation of an Academic Public Health Taxonomy and discussing how to gain buy-in from accreditors and other agencies and organizations. This will be followed by a discussion of the practical application of the Taxonomy by presenting a case study on conducting data reporting to the accreditor and the membership association in tandem. The case study will highlight how the Taxonomy provides a framework leading to greater alignment during data collection and accuracy when reporting.
Topic Area: Reporting and Transparency   |   Format: Poster Presentation
How IR Can Support the State Higher Education Policy Environment by Kathleen Zaback, Andy Carlson, Dustin Weeden
As state funding for higher education continues to decline and higher education outcomes become an increasing part of the public dialogue good data to inform policy has never been more important. Institutional research professionals are an essential to improving student outcomes at the institutional level and as states focus on this issue it’s important for these same professionals to lend their voice to state level policy conversations. Still political decision making is different than institutional decision-making and both often are at odds with the research methods valued by researchers. This panel is designed to give insight to IR professionals about how state policy makers work together to use data in decision-making. The State Higher Education Executive Officers, National Conference of State, and Complete College America will provide insight into the needs of their membership and ways they are able to bridge research with the policy needs of their constituents.
Topic Area: Reporting and Transparency   |   Format: Panel Session
Improving the Value of Data for Internal and External Stakeholders by Godfrey Noe
This session will start by reviewing the national, political, and regulatory landscape regarding expectations for higher education data on quality and student progress. In addition, the session will explore the current state of data available to the public, from higher education institutions, on quality and student learning. Finally, participants will discuss and formulate practical and effective strategies that institutional researchers can use to ensure that student achievement information provided by their schools meet stakeholder expectations.
Topic Area: Reporting and Transparency   |   Format: Discussion Group
International Rankings Go Deeper and Wider: More Subjects to be Ranked by Yang Zhang, Robert Morse, Baerbel Eckelmann, Yan Wu, Gero Federkeil, Phil Baty
Following our 2015 panel presentation on international university rankings in general, this year we shift attention to subject rankings. Different from overall university rankings, subject rankings position institutional performance in specific academic disciplines or fields, and provide valuable supplemental information to the university’s overall rankings. Led by an institutional researcher, a few of the best-known international ranking organizations will introduce the purposes, methodologies, and trends behind their subject rankings, followed by a discussion on how to better utilize ranking data for campus decision making support, from the perspectives of both institutional research and international rankings.
Topic Area: Reporting and Transparency   |   Format: Panel Session
IPEDS Update: Data Dissemination and Use by Gigi Jones
This session will discuss new ways to get IPEDS data, including the downloadable database and mapping tool, as well as updates to the website and existing tools. In addition, it will discuss IPEDS new position pieces which are being created to improve understanding of IPEDS data.
Topic Area: Reporting and Transparency   |   Format: Speaker Session
New Frontiers: Third Party Industry Certification Data Exchange Project by Scott Parke, Gretchen Koch, Vladimir Bassis
Accessing detailed student records about credentials that are not awarded by educational institutions – third party industry certifications – is outside established higher education data sharing initiatives. However, it is a topic worthy of further investigation. According to the US Census Bureau (2014), one-quarter of adults in the US hold a non-degree credential and those possessing industry credentials working full-time, had higher median earnings than those without them. This session highlights current outcomes from a precedent setting unit record data sharing project between a consortium of states (CA, FL, IL, IA, OK and NC) and CompTIA – the leading third party vendor-neutral standards based information technology industry certifier. Planned project expansion will be discussed.
Topic Area: Reporting and Transparency   |   Format: Panel Session
Onboarding Institutions to a State-Wide Higher Education Data Warehouse. by Tan Tran, Angela Bell
An obstruction in access to a centralized, system-level data warehouse (DW) produced a strong appeal from campus users for access, fueled a lack of confidence in the data integrity, and stopped an important feedback loop for data quality. The determined solution was to create on onboarding process similar to onboarding an organization into a transactional enterprise resource planning (ERP) software. DW access is a key step to creating intelligence out of data along with integrity, analysis, and use. The onboarding project addresses both theoretical and practical aspects for creating a healthy DW community.
Topic Area: Reporting and Transparency   |   Format: Discussion Group
Putting the "Integrated" Back Into IPEDS: Improving an Established System by Jamey Rorison
IPEDS is a comprehensive and trusted data source. However, many institutional researchers find current IPEDS reporting requirements to be overly burdensome, while policymakers frequently criticize its inability to adapt to different student characteristics and behaviors and policy interests. IPEDS serves many purposes, most of which go beyond its original purpose of reporting national statistics on higher education. There are many ways in which IPEDS can be improved, and these improvements could greatly enhance data availability to policymakers and researchers. That said, institutional reporting burden must be commensurate with value. The most viable improvements would link IPEDS to other existing data systems, add key new elements, remove outdated data, and feed data back to institutions. This session will explore potential improvements to IPEDS while giving participants an opportunity to share their perspectives on ways to generate better data while alleviating reporting burden.
Topic Area: Reporting and Transparency   |   Format: Speaker Session
Reframing Reporting: From Information Access to Engaged Communication by Natasha Jankowski, Timothy Cain
This session presents a re-conceptualizing of reporting efforts from meeting disclosure requirements and providing comparable data to effectively communicating audience-specific and contextualized information to multiple stakeholders. This approach of focusing on communication allows colleges and universities to meet accountability requirements while crafting an evidence-based story about their priorities and outcomes. The session provides resources to help institutional researchers think about communication strategies that can meet audience specific needs. The objective of the session is to move from compliance reporting towards effective communication, and to provide resources that can help improve communication of institutional data.
Topic Area: Reporting and Transparency   |   Format: Speaker Session
The National Faculty Data Pool: methods, challenges and perspectives by Meghan Edwards, James MacLean
The National Faculty Data Pool (NFDP) collects salary and demographic data on individual faculty members from over 60 institutions across Canada in order to aid in institutional planning and salary negotiations. The collection and comparison of faculty salaries is of the utmost importance to faculty bargaining and university financial and budget planning. In the course of collecting national level data, the NFDP has created numerous tools and processes to ensure data integrity and privacy and to aid member institutions in determining who their comparator universities are. Participants will learn about national-level data collection, including processes and challenges, maintaining data integrity and confidentiality, and what tools and aids can be developed to aid in large-scale data collection.
Topic Area: Reporting and Transparency   |   Format: Poster Presentation
The Student Achievement Measure (SAM): More Outcomes for More Students by Teri Hinds
The federal graduation rate, the most common measure of student attainment, does not count students who transfer, often leading to the mistaken portrayal of those students as failures. The Student Achievement Measure (SAM) reports more outcomes for more students by tracking both full- and part-time student movement within and across postsecondary institutions. Now in its third year of implementation, SAM has been recognized by the Department of Education, which has committed to adding links to institutional SAM pages from the new College Scorecard. This session will share the characteristics of the two SAM models and plans for future development. Planned features for 2016 include the ability to add SAM outcomes additional institutionally defined cohorts such as Pell recipients or students receiving veterans benefits and the addition of an Associates Degree model that tracks student outcomes past transfer or who enroll for additional education after receipt of their associate degree.
Topic Area: Reporting and Transparency   |   Format: Speaker Session