Learning Together: Perceived Cognitive and Behavioral Outcomes of Learners in a Nontraditional, Professional Doctoral Program - Hank Radda, Ted Cross & Rick Holbeck

About the Authors

Hank Radda
Hank Radda, Ph.D., is the Dean/Senior Vice President of the College of Doctoral Studies at Grand Canyon University. He holds a Master's of Science in Community-School Psychology from the College of New Rochelle and a Doctorate of Philosophy in Clinical Psychology from the Union Institute.

Ted Cross
Ted Cross earned a Master of Science in Education from the University of Pennsylvania in 2009. Ted also received both a Master and Bachelor of Arts in English from Arizona State University in 2007 and 2005, respectively. Ted Cross is a Manager of Full-time Online Faculty as well as an instructor at Grand Canyon University, teaching a wide range of classes from University Success to Communications.

Rick Holbeck
Rick Holbeck earned a Bachelor's Degree in Secondary Music Education from Bemidji State University (Minnesota) and a Master's Degree in Educational Leadership from Southwest Minnesota State University. Rick has also finished the coursework for a PhD in Educational Leadership at Walden University and is waiting to begin the dissertation process. Currently, Rick holds the position of Manager of Full Time Online faculty at Grand Canyon University as a part of the Academic Operations Department.



In recent years nontraditional professional doctoral programs have been on the rise. Fueled by the demand of the new "knowledge-based" economy, these doctoral programs are becoming increasingly important in training professionals to become experts in their fields. As such, data on the learning outcomes of nontraditional programs becomes useful in understanding educational effectiveness as well as creating feedback for future program design. In this study over 500 doctoral learners in an online doctoral program with residency requirements participated in structured discussions about the perceived outcomes of their doctoral journey. Qualitative analysis using grounded theory suggests that students perceived certain behavioral and cognitive dispositions during the doctoral program. Discussion reveals that there is a fertile area for ongoing investigation on the perceived outcomes of nontraditional doctoral programs in terms of both academic learning and social-emotional development.

Over past decades nontraditional professional doctoral programs have steadily grown in numbers (Archbald, 2011). Having first appeared in 1921 at Harvard University, professional doctoral programs were the precursor to the newer, nontraditional professional programs, such as online, cohort, and hybrid programs. Today, nontraditional professional doctorates are being awarded in greater numbers than ever before. In fact, some scholars report that the number of nontraditional doctorates awarded each year exceeds the number of traditional doctorates, which are mostly earned by full-time students (Archbald, 2011). Interestingly, nontraditional degrees now make up the majority of doctoral degrees awarded in the United States (Archbald, 2011). These nontraditional programs are often seen as being "new and different" as well as a "departure" from traditional brick and mortar archetypal, or traditional doctoral programs (Archbald, 2001). There are several reasons for these distinctions. Besides their unique instructional delivery methods, these programs often have a different student base than traditional programs. For instance, nontraditional doctoral students are typically older than their traditional counterparts (National Opinion Research Center, 2006). Also, while traditional doctoral students are more likely to be male and single, nontraditional doctoral students are just as likely to be female, married, and have children. Thus, nontraditional doctoral students are characterized as "...being older, more engaged in family and work life, financially independent, and studying part-time" (Offerman, 2011, p. 26).

In many nontraditional programs, doctoral learners are often employed professionals with decades of work experience, who wish to improve their skills and competencies and have a vision of improving their organizations. They reenter academia with diverse backgrounds, professional experience, and a sense of direction regarding the outcomes they envision for their doctoral journeys. For the purpose of this study, the term "nontraditional" referred to an online doctoral program with residency requirements.

In response to this different student population, nontraditional doctoral programs often offer their course work in flexible schedules via blended learning, both on and off the Internet (Grable, 2011; Singleton & Sessions, 2011). In this sense many nontraditional doctoral programs appeal to adult learners as they offer convenience and flexibility (Pappas & Jerman, 2011). However, it is this convenience and flexibility that often results in stigma (Grable, 2011). Montel (2003) noted that many traditional academics still question the quality of nontraditional degree programs, especially online programs. While this stigma continues to exist among traditional academics, examining learning outcomes of nontraditional programs may help bolster their perceived validity.

Despite perceived stigmas, nontraditional doctoral programs aim to fill the growing need of practitioner researchers in public and private institutions (Servage, 2009). Thus, as the new knowledge-based economy increases its focused search for individuals prepared to do both research and practitioner work, it is likely that these programs will continue to grow (Archbald, 2001). Consequently, in the wake of this demand, it may be useful to begin examining the learning outcomes of professional, nontraditional doctoral programs to better understand the perception of students in relation to the skills and knowledge they acquire during their tenure in these programs.

As such, the purpose of the current investigation is to examine the perceived learning outcomes of nontraditional, professional doctoral learners. Understanding student perceptions of learning serves as a starting point from which to begin the dialogue on learning outcomes of professional, nontraditional doctoral learners. In this way, emergent data collected from current students can help inform theories and practices to increase student learning and match that learning with professional practice. In this realm of thought, this study sought to understand the following research question:

What are the perceived cognitive and behavioral learning outcomes of nontraditional doctoral students in an online program with residency requirements?

Through the examination of both perceived cognitive and behavioral learning outcomes, the purpose of this study is to enhance the evaluation of nontraditional professional programs. Of course, traditional university and programmatic assessments need to occur; however, for a richer dialogue, this investigation was conducted from the perspective of learners who had between one and three years of doctoral experience in the program. Implicit in this exploration is the goal of understanding how nontraditional, professional doctoral programs prepare students for further work as scholar-practitioners.

Due to the exploratory nature of this study, a qualitative, emergent research design was implemented. Creswell (2007) described qualitative research as a process that flows from "philosophical assumptions, to world-views, into a theoretical lens and onto the procedures involved" (p. 37). Similarly, Creswell (2007) noted that qualitative research includes the use of natural settings, the researcher as gatherer of data, multiple sources of data, inductive analysis and the consideration of participant meanings (p. 37-39). Within these criteria this study followed a qualitative method emerging from a social constructivist worldview and employed a grounded theory approach (Creswell, 2007). Specifically, the methods involved in this study were the formation of open-ended interview questions, the formation of focus groups, the interviewing of focus groups through structured, open-ended questions, the collection of interview response data, the development of codes, emerging categories, operational definitions, the analysis of the frequency of those categories, and the interpretation of the final set of resultant themes. 

Participant s and Procedure
Participants for this study were selected from several cohorts of online doctoral students with residency requirements at a private Christian University in the southwestern United States. In July of 2011, at two residency events (Week 1 July and Week 2 July) all learners attending the residency were seated at tables of 8 to 10. The Week 1 July group was comprised of 270 participants, and the Week 2 July group amounted to 256 participants. Nominal demographic data was collected from each participant through the completion of a questionnaire identifying his or her program of study and years of study in the program.

Next, an orientation to the research study process was provided. The work was conceptualized as a conversation centered on the current perceived learning of students, their future individual learning goals, and the development of a doctoral culture at the University. The learners were informed that this would be part of an ongoing dialogue between them and the university. In this way, the context was set for an ongoing development of a doctoral culture. This study focused on two of the questions and discussions that ensued during the previously mentioned residencies.

After the orientation to the process, the moderator explained the first question as an interactive focus group - "What have you learned cognitively?" In other words, "What do you perceive you learned most during your doctoral experience so far?" Each learner was given approximately 5 minutes to identify the top three cognitive learning outcomes, individually. Next, the group was given 10 minutes to discuss individual answers and reach consensus on the top three responses for the table.

This process was then reiterated with the question, - "What have you learned behaviorally?" Put another way, "What learning outcomes have you learned that show up in your behavior?" Similar time was given for individual responses, and then group consensus on the top three answers per table was reached. In this way, the learners at each table came up with several answers to the open-ended interview questions that reflected the consensus of the group.

Last, the groups recorded their answers on large sheets of paper. The papers were collected at the end of the session and stored in a secure location. Subsequently, a student worker merged each table's data into one master spreadsheet using Excel. This spreadsheet was then coded following a four-step, grounded theory process of open coding, axial coding, selective coding, and visual portrayal (Creswell, 2007). 

During the open coding and axial coding process, the researchers analyzed each set of data separately. First, the data gathered in response to the question, "What did you learn cognitively?" was analyzed. The researchers sifted through the data coming to consensus on six emergent categories.  Next, through the process of axial coding, the researchers coded each piece of data with the appropriate category. In other words, each piece of data was assigned one of the six emerging themes. Later,  this coded data was then assigned a numerical code associated with each of the six themes. Last, the numerically coded data was sorted from least to greatest usage in order to group the data in its appropriate emergent categories. This same process was repeated for the second open-ended question, "What did you learn behaviorally?"

After the data was numerically sorted, it was then sorted into tables that labeled each of the emerging themes or categories with its associated frequency (Figures 1 and 2). Last, the data was analyzed through the selective coding process in order to create connections between each category (Creswell, 2007) and identify dominant themes that emerged.

The results of this study are divided into two categories: perceived cognitive learning outcomes and perceived behavioral learning outcomes. Cognitive outcomes include what students believed they had learned at this point in their doctoral program, whereas behavioral outcomes include actions that students believe were strengthened during the doctoral program. The cognitive category describes things students perceive they had learned from the content of their programs, while the behavioral category represents perceptions of applied skills. 

The data revealed seven areas of perceived growth: time management, interpersonal skills, research and writing skills, leadership, critical thinking, perseverance, and other. Both the cognitive and behavioral data shared all of these categories.

Along with these subsets by which the data was coded, operational definitions were also established to create consistency in the coding process. The definitions are:

1. Time Management- ideas surrounding the ability or recognition of the need to schedule, plan, or otherwise structure time. This included responses of time management, organization, and planning.

2. Interpersonal Skills- notions centered on the ideas of interacting in positive ways with others. This included responses of communication, collaboration, and networking.

3. Research and Writing Skills- perceptions in or around the writing or research process not including the areas of data gathering and analysis. This included responses of scholarly writing, data analysis, and research skills.

4. Leadership Skills- ideas concerning the learning and practices of leadership and its associated responsibilities. This included responses of leadership styles and leadership theories.

5. Critical Thinking- notions centered on problem-solving in its various forms. This included responses of critical thinking, reflection, and application of theory.

6. Perseverance- the idea and qualities needed to push through obstacles or barriers. This included responses of commitment, discipline, and endurance.

7. Other- data that did not readily fit into one of the definitions above. This included responses of honesty, integrity, and values.

In examining the data from the students' perceived cognitive learning outcomes, the three most prevalent categories that emerged were: Research and Writing Skills (38.4%), Critical Thinking (25.79%), and Leadership Skills (20%). Figure 1 shows the total breakdown of the perceived cognitive learning outcomes.

The three most prevalent categories related to perceived behavioral learning were: Time Management (26.42%), Perseverance (23.11%), and Interpersonal Skills (18.87%). Figure 2 shows the total breakdown of the perceived behavioral outcomes.

Andragogy posits that adult learners are characterized by their ability to self-direct their learning, access a depth of personal experiences, desire learning in relation to societal roles, and are interested in applications of knowledge to solve problems (Knowles, 1970). The results of this study show several similarities between these characteristics of adult learners and the perceptions of student learning outcomes in this particular population. While this study focused on developing an understanding of the perceptions of doctoral student learning, it illustrates what the students believed they learned and reveals insights that are consistent not only with adult learning theory, but also with theories of emotional intelligence. This may suggest important insights for these learners and their program of study thus far. 

First, over 60% of what students perceived that they learned cognitively consisted of academic and problem-solving skills. These results are consistent with Knowles' (1970) assertion that adult learners are interested in learning what will help them problem-solve and fulfill certain societal roles. It may be that these doctoral students view academic and problem- solving skills as directly relevant to their current tasks as doctoral students such as completing papers and conducting research. However, it may also be true that these learners view these skills as important for their current careers and lives. Thus, according to their own perceptions, these learners have increased their knowledge of how to be doctoral level researchers and writers as well as learning problem-solving skills.

While these findings are encouraging, they are based upon student perceptions. In other words, these students may have placed emphasis on these skills simply because they felt that they were important. Thus, while the students may have increased their knowledge in the area of academics skills and problem- solving, it may also be true that these students, because of the nature of their own perceptions, merely believed that is what they learned the most. This emphasis could be a result of the students' perceived importance of the area of problem- solving and academic skills. However, because these results are based on student perceptions, it does not decrease the utility of such data in understanding how students feel in relation to their doctoral education.

In short, when adding in the category of "Leadership Skills" over 80% of the results of the perceived cognitive learning outcomes fall into the general area of practical academic skills. This general category is consistent with the application of knowledge that Knowles (1970) expressed as an important aspect of adult learning. This alignment between what students perceive to learn and the theoretical underpinnings of adult learning is significant because it highlights a congruence between what students may be learning in relation to what adult learners need, at least in reference to andragogy. Therefore, this program of study is not only preparing doctoral learners for research, problem-solving, and application of content, but it may also be better serving nontraditional students who fit Knowles' (1970) learning needs.

Next, when examining the findings from the behavioral set of data, a different picture emerged. The major themes from this data were: Time Management (26.42%), Perseverance (23.11%), and Interpersonal Skills (18.87%). Interestingly, these categories seem to coincide well with Daniel Goleman's (1995) conception of emotional intelligence. The first two categories, Time Management and Perseverance fall under Goleman's (2005) idea of self-regulation, and the last category of Interpersonal Skills fits closely with Goleman's (2005) notion of social skill. Thus, it appears that one of the major perceived learning outcomes of these doctoral students was a self-professed increase in emotional intelligence. This becomes an important outcome for nontraditional students because of their learning goals as well as circumstances.

For example, because nontraditional students are often juggling career, family duties, and schooling (Offerman, 201) an increase in emotional intelligence may allow for better coping with the myriad demands of life. Strikingly, the students' highest rated perceived learning outcome, Time Management, a skill that falls under Goleman's (2005) notion of self-regulation, becomes part and parcel of balancing the competing demands noted above.

Also, because nontraditional students are often employed and seeking doctoral education in order to enhance their careers, emotional intelligence becomes important to help produce competent practitioners. In this way the theme of Interpersonal Skills that the students identified as one of their most highly rated perceived learning outcomes, becomes an indicator of what Goleman (2005) called social skill. These social skills can be important for these students as they interact with colleagues at work as well as within their academic program.

In sum, the results from both the cognitive and the behavioral perceived learning outcomes can fit into two categories: Academic Skills and Emotional Intelligence (Goleman, 1995; 2005). Academic Skills fits well with Knowles' (1970) construct of adult learning and may represent indications that this nontraditional doctoral program is aligned to teach adult learners. Next, Emotional Intelligence encompasses the perceived behavioral learning outcomes and thus may signify skills important for developing practitioner researchers. Therefore, a new assertion for testing emerged from this study: Can nontraditional doctoral programs better serve the needs of adult learners than traditional brick -and mortar programs?


Future Research
This study is an initial step in studying the perceived learning outcomes of nontraditional doctoral learners. As such, there is much more that can be gleaned from the data gathered for this study. Ongoing investigations will examine data according length of time in the program, to see if the perceived learning outcomes differ throughout the course of their doctoral journey. Additionally, the data from the learners' perceptions could be compared to the University's assessment of the cognitive skills in parallel with those identified by the learners as a means of triangulating institutional data concerning program effectiveness.

Further, another area of study may come from seeking to understand the behavioral outcomes of Perseverance, Time Management, and Interpersonal skills. In short, the research question may be: Does a doctoral program help learners develop these skills, or is this a part of the hidden curriculum of doctoral studies? In response, the perceived behavioral outcomes could be triangulated with entrance letters of intent, a normed instrument that measures perseverance, and student pass rates over three classes. This would allow for data to be gathered on student perceptions, student measures of persistence, and actual outcomes of persistence between what students perceive and what is actually measured.

Last, this study, through its emergent design, revealed important insights into how nontraditional doctoral programs serve adult students. If the perceived learning outcomes can be verified through further research as mentioned above, it may point to measureable learning outcomes that could show that nontraditional doctoral programs are aligned with adult learning theory and create practical skills that can be useful in practitioner and research settings. This emergent study showed that new insights into the learning outcomes and effectiveness of nontraditional doctoral programs can be verified through further investigation. These investigations could add more evidence to support the indications of this study that nontraditional doctoral students feel that they are increasing in academic skills, emotional intelligence, and may be better served through nontraditional programs. Further, that nontraditional doctoral programs may meet the needs of adult learners as well as prepare students for the world of work.  

Author Biographies
Hank Radda, Ph.D., is the Dean/Senior Vice President of the College of Doctoral Studies at Grand Canyon University. He holds a Master's of Science in Community-School Psychology from the College of New Rochelle and a Doctorate of Philosophy in Clinical Psychology from the Union Institute.

Ted Cross is a native of Scottsdale and a fifth generation Arizonian. He is a lifelong learner with a multidisciplinary academic and professional background. Ted has worked in investment advising, secondary education, and higher education. He holds the following credentials: B.A. English Brigham Young University; M.A. English Arizona State University; M.S.ed Education The University of Pennsylvania; Post Bac. Cert. Human Resource Management The Wharton School of Business; Ed.D. Organizational Leadership Pepperdine University (in-progress, expected graduation-2013). Ted is interested in applying his deep academic background to teaching and leading across disciplines. Through inspiring others he aims to help others reach their full potential. Ted is currently a Manager of Fulltime Online Faculty at Grand Canyon University working for the Department of Academic Operations.

Rick Holbeck earned a Bachelor's Degree in Secondary Music Education from Bemidji State University (Minnesota) and a Master's Degree in Educational Leadership from Southwest Minnesota State University. He has also finished the coursework for a PhD in Educational Leadership at Walden University and is waiting to begin the dissertation process. In the meantime, Rick is also working on a second Master's Degree in Curriculum and Instruction with an emphasis on Technology from Grand Canyon University. Rick began teaching adjunct classes for Grand Canyon University in February of 2010 and moved to a Full Time Online Faculty position in August, 2010. Currently, Rick holds the position of Manager of Full Time Online faculty at Grand Canyon University as a part of the Academic Operations Department.


Archbald, D. (2011). The emergence of the nontraditional doctorate: A historical overview. New Directions for Adult and Continuing Education, 2011(129), 7-19. doi:10.1002/ace.396

Creswell, J. W. (2007). Qualitative inquiry & research design: Choosing among five approaches (2nd ed.). Thousand Oaks, CA: Sage Publications.

Gardner, S. K. (2009). Conceptualizing success in doctoral education: Perspectives of faculty in seven disciplines. Review of Higher Education, 32(3), 383-406.

Goleman, D. (1995). Emotional intelligence. New York: Bantam Books.

Goleman, D. (2004). What makes a leader? Harvard Business Review, 82(1),82-91.

Grable, J. E. (2011). Innovation in doctoral degrees designed for adult learners: A hybrid model in personal financial planning. New Directions for Adult and Continuing Education, Spring 2011(129), 43-51. doi:10.1002/ace.399

Knowles, M. S. (1970). The modern practice of adult education: Andragogy versus pedagogy. New York: Association Press.

Montell, G. (2003, August 6). Battling the stigma of nontraditional credentials. The Chronicle of Higher Education.. Retrieved from http://chronicle.com/article/Battling-the-Stigma-of/45160/

National Opinion Research Center (2006). Doctorate recipients from United States universities summary report 2005 (SED Report). Retrieved from the National Science Foundation website: http://www.nsf.gov/statistics/doctorates/pdf/sed2005.pdf

Offerman, M. (2011). Profile of the nontraditional doctoral degree student. New Directions for Adult and Continuing Education, 2011(129), 21-30. doi:10.1002/ace.397

Pappas, J. P., & Jerman, J. (2011). Topics for current and future consideration. New Directions for Adult and Continuing Education, 2011(129), 95-103. doi:10.1002/ace.404

Servage, L. (2009). Alternative and professional doctoral programs: What is driving the demand? Studies in Higher Education, 34(7), 765-779. doi:10.1080/03075070902818761

Singleton, H. W., & Session, C. L. (2011). Faculty concerns related to distance learning within nontraditional doctoral programs. New Directions for Adult and Continuing Education, 129(129), 31-41. doi:10.1002/ace.398