Running Head: Agronomy Curricular Feedback
Agronomy Curricular Feedback from Alumni, Employers, and Faculty – Initial Report to the Teaching Panel
Outcomes Assessment Committee
Michelle D. Cook
May 7, 2004
Agronomy Curricular Feedback from
Alumni, Employers, and Faculty
Introduction
The purposes of this study were to 1) gather curricular feedback from alumni, employers, and undergraduate teaching faculty on the importance of the departmental learner outcomes, assess strength of initial alumni performance in these areas, and identify potential learner outcomes critical in the future; 2) gather alumni and employer perceptions of alumni knowledge foundation, aptitude for career advancement, and ability to adapt to change; 3) assemble alumni post-graduation professional development activity; 4) collect employer expectations related to professional development; 5) solicit undergraduate teaching faculty opinions on percentage of the most recent Agronomy B.S. graduates they would feel comfortable hiring and a description of reasons for hiring or not hiring recent graduates.
Assessment Questions
In this study, the following assessment questions were explored:
1) Which learner outcomes
a) Do alumni perform well?
b) Do alumni perform poorly?
c) Are important?
d) Are unimportant?
e) Are important and alumni performance is high?
f) Are important and alumni performance is lagging?
g) Are unimportant and alumni performance is high?
h) Are unimportant and alumni performance is lagging?
2) In addition to the current list of learner outcomes, which knowledge, abilities/skills, and perspectives will be critical in the future?
3) How strong is alumni knowledge foundation, aptitude for career advancement, and ability to adapt to change?
4) What degrees, certifications, and professional development opportunities have alumni pursued since graduation?
5) What expectations do employers have regarding employee professional development?
6) What percentage of Agronomy B.S. graduates would faculty feel comfortable hiring? For what reasons would faculty hire or not hire recent graduates?
Method
Participants
This study included three samples: alumni, employers, and undergraduate teaching faculty. Each sample will be described briefly.
Alumni. The alumni sample was comprised of all Agronomy majors who received a bachelor’s of science degree from fall 2000 to spring 2002. Alumni names and addresses were obtained from the Alumni Association. The initial alumni sample included 109 participants. Three participants had to be excluded for lack of current home address. The final alumni sample size was 106. Of the 106 alumni, 47.2%, or 50 individuals completed and returned the survey. Of the 50 alumni respondents, two-thirds (n = 34 or 68%) are employed in production agriculture, three alumni (6%) are employed in agribusiness industry, seven alumni (14%) are employed in agriculture government/regulation, one alumni (2%) is not involved in agriculture, and five alumni (10%) did not respond to this item.
Employers. The
employer sample included all first-employer post-graduation supervisors of the
alumni sample. Names and contact information were obtained through department
records and phone, e-mail, and U.S. Mail contact with both employers and
alumni. Six of the alumni were enrolled in graduate studies in the Agronomy
Department at
Undergraduate teaching faculty. The third sample included all Iowa State Department of Agronomy faculty who teach one or more undergraduate classes. Of the 18 undergraduate teaching faculty, 55.6%, or 10 individuals completed and returned the faculty survey.
Measures
The instruments employed in this studied were developed by first, reviewing the list of 43 departmental learner outcomes developed in 1995. Because many of the outcomes were double-barreled (i.e., contain more than one learner outcome per statement) or not measurable as written, the primary investigator set out to parse out embedded outcomes and re-write, for the purpose of assessment, a few outcomes that were not measurable as written. For example, the following learner outcome, “Be able to recognize common biotic stresses, their potential effects on plants at various stages of crop development, and options for amelioration of stresses with minimal disturbance to the environment. Understand the concepts of weed threshold populations and economic injury levels for insects, nematodes and diseases,” was broken out into five items: 1) recognize common biotic stresses, 2) recognize the potential effects of biotic stresses on plants at various stages of crop development, 3) recognize options for amelioration of stresses with minimal disturbance to the environment, 4) understand weed threshold populations, 5) understand economic injury levels for insects, nematodes, and diseases. As a result of the review of the learner outcomes, the 43 outcomes became 86 individual items.
In order to have faculty involved and invested in the process, a brief description of the measures was sent via email through the Teaching Panel list-serve to invite faculty feedback for the purposes of identifying topics for the survey as well as any possible methodological concerns. This issue was also included as a Teaching Panel discussion item. As a result of the Teaching Panel discussion, learner outcomes were referred to as knowledge, skills/abilities, and perspectives to avoid the use of educational jargon.
Each measure was divided into two parts. The first section was the same for each sample. Using a seven-point Likert scale, participants were asked to assess 1) the importance of each of the learner outcomes and 2) the strength of initial alumni performance in these areas. Importance was assessed using the following scale: 7 = Very Important; 6 = Mostly Important; 5 = Somewhat Important; 4 = Neither Important nor Unimportant; 3 = Somewhat Unimportant; 2 = Mostly Unimportant; 1 = Very Unimportant. Alumni performance was assessed using the following scale: 7 = Very Strong; 6 = Mostly Strong; 5 = Somewhat Strong; 4 = Neither Strong nor Weak; 3 = Somewhat Weak; 2 = Mostly Weak; 1 = Very Weak. Employers and faculty were also given the option to select Not Observed (NO), if for any reason they did not feel they had adequate experience to make a judgment on a specific alumni assessment item.
The second section differed based on group membership. Common items included identifying important intended learner outcomes not currently being addressed in the curriculum and recording additional comments or feedback. In addition, alumni and employers were both asked to assess alumni knowledge foundation, aptitude for career advancement, and ability to adapt to change and to categorize their business type. Alumni provided information on their post-graduation professional experience; current professional status; and degrees, certifications, and professional development. Employers were asked to describe expected employee continuing education. Faculty were asked to identify outcomes that were not measurable as written, indicate what percentage of the most recent Agronomy B.S. graduates they would feel comfortable hiring, and describe reasons for hiring or not hiring recent graduates.
The final instruments, Agronomy Alumni Survey, Agronomy Employer Survey, and Undergraduate Teaching Faculty Survey are included as Appendix A, B, and C respectively.
Procedures
Alumni and employers. For some alumni, employer and supervisor data were not available. In those cases (n = 35), alumni were asked to either deliver the employer survey directly to their first supervisor after graduation or to provide supervisor contact information with their completed survey. It is important to note that none of the employer surveys sent directly to alumni were completed and returned.
Dillman’s (2000) Tailored Design Method was employed for the survey administration. For all alumni and employers with contact information, pre-survey letters (See Appendix D and E) were mailed on Monday, July 28, 2003. The purpose of the pre-survey letters was to alert participants to the study and increase the likelihood of their timely response. Surveys and cover letters were mailed on Friday, August 1, 2003. Reminder postcards were mailed on Friday, August 15, 2003. A second wave of surveys was mailed to non-respondents on Friday, August 29, 2003.
Undergraduate teaching
faculty. Faculty surveys were administered via email on July 2. Returned
completed surveys were collected over the summer until early October.
Analysis
Quantitative data were analyzed using SPSS for Windows version 11.5. First, basic descriptive statistics, frequencies, means, and standard deviations, were computed with all three groups combined and then each group was looked at separately. Gap scores were computed by subtracting the importance value from the performance value.
The interpretation of results focused on all three scores: importance, performance, and gap scores. Of special interest were those outcomes that have a high importance and high gap score indicating the knowledge, skill/ability, or perspective is regarded as highly important, yet graduate performance is lagging. Qualitative data were entered into a spreadsheet. The responses were then sorted into common themes and categories by group.
Results
The following sections summarize results including performance, importance, and gap score descriptive information. Tables are located at the end of the document.
Alumni, employers, and faculty
were each asked to assess alumni performance for each of the outcomes.
Generally speaking, both alumni and employers tended to rate the majority of
the outcomes as somewhat to mostly important and also rated alumni performance
of a majority of the outcomes as somewhat to mostly strong. Faculty rated
three-quarters of the outcomes as mostly to very important and the most common
rating for alumni performance was neither strong nor weak to somewhat strong.
Which learner outcomes are important and alumni performance is high? The following learner outcomes were rated important with strong alumni performance: access and use electronic mail; communicate effectively using electronic media; organize, manage, interpret, and communicate information using computer tools (four learner outcomes combined); access and use databases; understand basic soil principles; present an effective oral report; and perform mathematical calculations appropriate to your profession (See Table 5). Additional learner outcomes with high importance and high performance are reported by group below:
Alumni
Ø Prepare a resume (F)
Ø Understand the soil as a resource upon which ecosystems, agriculture and other land uses depend (F)
Ø Interpret graphical and tabular information (E)
Ø Write a concise, grammatically correct report
Ø Appreciate the family as an interdependent, supportive unit (E)
Ø Understand basic technical principles and methods relating to conservation tillage methods and seedbed preparation
Ø Demonstrate commitment to effective time management
Employers
Ø Appreciate the family as an interdependent supportive unit (A)
Ø Acknowledge differences in political, cultural, religious, and ethnic beliefs and practices
Ø Interpret graphical and tabular information (A)
Ø Understand basic climate principles
Faculty
Ø
Prepare a resume (A)
Ø
Understand the scope of agronomy and its
relationships to other disciplines and professions
Ø
Understand basic technical principles and
methods relating to production of important crops
Ø
Recognize common biotic stresses
Ø
Quantify amounts, rates, and relationships using
measurements basic to agronomy, such as yield per unit, bulk density, relative
humidity, etc.
Ø
Understand basic technical principles and
methods relating to efficient and environmentally sound fertilization and weed control
strategies
Ø
Understand the soil as a resource upon which
ecosystems, agriculture, and other land uses depend (A)
Ø
Use electronic repositories in support of
further personal and professional growth
Ø
Work effectively in a team situation as a
participant
Which learner outcomes are important and alumni performance is lagging? The ten learner outcomes rated important with weak alumni performance were as follows: recognize a conflict of interest involving oneself and one’s client or employer, propose and evaluate alternative solutions to a problem based on the resources of the client, identify resources needed to solve a problem, assess and evaluate the credibility and biases of different sources of information, debate issues in a professional manner, answer oral questions extemporaneously and understandably, work effectively in a team situation as a leader, define a problem, demonstrate high standards of achievement, analyze and interpret simple research data (See Table 6). Additional learner outcomes with high importance and lower performance are reported by group below:
Alumni
Ø Understand basic technical principles and methods related to cultivar and hybrid selection in relation to soil type and climate
Ø Recognize the potential effects of biotic stresses on plants at various stages of crop development
Ø Recognize common biotic stresses
Ø Understand basic technical principles and methods relating to seeding times and methods
Ø Understand basic plant principles
Employers
Ø
Write a concise grammatically correct report (F)
Ø
Understand basic technical principles and
methods relating to efficient and environmentally sound fertilization and weed
control strategies
Ø
Motivate and organize a group in defining and
solving a problem
Faculty
Ø Use libraries in support of further personal and professional growth
Ø Perform mathematical calculations appropriate to your profession
Ø Write a concise, grammatically correct report (E)
Ø Appreciate the responsibility of the individual to society in sustainable management of energy resources
Ø Interpret graphical and tabular information
Ø Be aware of the global extent of agriculture
Importance, Performance, and Gap Means of Outcomes Rated Important With Strong Alumni Performance
|
|
Importance |
Performance |
Gap |
|
Learner Outcome |
M |
M |
M |
|
Access and use electronic
mail |
6.32 |
6.53 |
0.16 |
|
Communicate effectively using electronic media |
6.28 |
5.97 |
-0.38 |
|
Communicate information using computer tools |
6.31 |
5.95 |
-0.42 |
|
Interpret graphical and
tabular information |
6.11 |
5.72 |
-0.44 |
|
Access and use databases |
6.11 |
5.72 |
-0.49 |
|
Organize information using
computer tools |
6.38 |
5.95 |
-0.49 |
|
Understand basic soil
principles |
6.08 |
5.73 |
-0.54 |
|
Present an effective oral
report |
6.18 |
5.66 |
-0.59 |
|
Manage information using
computer tools |
6.39 |
5.85 |
-0.60 |
|
Perform mathematical calculations appropriate to
your profession |
6.44 |
5.88 |
-0.61 |
aOn some items, total number of responses may not equal
98 due to non-responses.
bThe scale for these items is as follows: 1=Very Unimportant; 2=Mostly Unimportant; 3=Somewhat Unimportant; 4=Neither Important nor Unimportant; 5=Somewhat Important; 6=Mostly Important; 7=Very Important.
cThe scale for these items is as follows: 1=Very Weak;
2=Mostly Weak; 3=Somewhat Weak; 4=Neither Strong nor Weak; 5=Somewhat Strong;
6=Mostly Strong; 7=Very Strong; NO=Not Observed.
dThe n for these items ranges from 89 to 96.
Importance,
Performance, and Gap Means of Outcomes Rated Important With Weak Alumni
Performance
|
|
Importance |
Performance |
Gap |
|
Learner Outcome |
M |
M |
M |
|
Recognize a conflict of interest involving oneself
and one’s client or employer |
6.13 |
5.10 |
-1.13 |
|
Propose and evaluate alternative solutions to a
problem based on the resources of the client |
6.12 |
5.18 |
-1.08 |
|
Identify resources needed to solve a problem |
6.41 |
5.42 |
-1.04 |
|
Assess and evaluate the credibility and biases of
different sources of information |
6.18 |
5.23 |
-1.00 |
|
Debate issues in a
professional manner |
6.15 |
5.22 |
-0.99 |
|
Answer oral questions extemporaneously and
understandably |
6.40 |
5.51 |
-0.97 |
|
Work effectively in a team situation as a leader |
6.26 |
5.33 |
-0.95 |
|
Define a problem |
6.38 |
5.51 |
-0.94 |
|
Demonstrate high standards
of achievement |
6.51 |
5.70 |
-0.88 |
|
Analyze and interpret
simple research data |
6.36 |
5.62 |
-0.88 |
aOn some items, total number of responses may not equal
98 due to non-responses.
bThe scale for these items is as follows: 1=Very Unimportant; 2=Mostly Unimportant; 3=Somewhat Unimportant; 4=Neither Important nor Unimportant; 5=Somewhat Important; 6=Mostly Important; 7=Very Important.
cThe scale for these items is as follows: 1=Very Weak; 2=Mostly Weak; 3=Somewhat Weak; 4=Neither Strong nor Weak; 5=Somewhat Strong; 6=Mostly Strong; 7=Very Strong; NO=Not Observed.
dThe n for these items ranges from 85 to 95.