Researching the Process and Outcomes of Electronic Portfolio Development

in a Teacher Education Program - DRAFT

Ann H. McCoy, Ph.D.

Assistant Professor Elementary Education

College of Education

University of Alaska Anchorage

USA

afahm@uaa.alaska.edu

Abstract: This paper will present the results of a preliminary study of teacher candidates¹ electronic portfolio development efforts, student attitudes and skills developed during the process, and some preliminary conclusions. In 1999 and again in 2000, this university received two PT3 grants along with a Title II Partnership Grant to restructure our teacher education program. A major element of the Student Development component of these PT3 grants was the implementation of electronic portfolios to support documentation of professional growth and provide a framework for career-long professional development. A variety of strategies were used to collect data about student attitudes toward the electronic portfolios that were developed in the program. This paper will present a summary of the results collected from four cohorts of students, ³lessons learned² and new directions.


For the last five years, the University of Alaska Anchorage College of Education (COE), along with partner school districts and businesses, has redesigned components of its teacher education program with support from a Federal Title II Partnership Grant and two Federal Preparing Tomorrows Teachers to Use Technology (PT3) grants.

The Alaska PT3 grant has actively supported the integration of technology throughout the program design process by addressing four specific goals which lead to the outcome that COE graduates are well prepared, technology proficient educators. A major element of the Student Development component of the grant was the implementation of electronic portfolios to support documentation of professional growth. Formative and summative assessment activities have occurred throughout the grant period. A variety of strategies were used to collect data about student attitudes and skills toward the electronic portfolios that were developed in the program. Faculty were also surveyed about their use and support for portfolios. Preliminary results from surveys and interviews are explored and changes to the programs resulting from responsive activities will be shared.

Teacher Education Program Redesign

Intense partnership activities in 1999-2000 resulted in the creation of the Post-baccalaureate Teacher Education Program (PTEP). This intensive, one-year, postgraduate, internship program was based on best practice. Technology was an integral part of the program rather than simply a stand-alone course. Basic technology skills in the PTEP program were incorporated at the beginning of the program in education foundations classes. More advanced technology experiences were designed to be integrated throughout the program. Electronic portfolio development was a ongoing activity throughout the program.

The three College of Education (COE) teacher preparation programs, all based on the PTEP model, have emerged as the program redesign proceeded. All three programs are committed to the integration of issues related to diversity and technology across the curriculum. The Secondary MAT Program, Elementary Education Program, and Early Childhood Education Program provide a high quality professional preparation for beginning teachers, with an emphasis on content preparation, a year-long internship experience, and a commitment to a partnership model.

Electronic Portfolio Creation Process

Numerous research papers and books support using portfolios for demonstration of competency. As Dr. Helen Barrett states "Portfolios have become a widely used assessment method in American education-particularly as a device for representing and developing teacher knowledge." NCATE defines portfolios "as a means of gathering candidate performance data" that are "used in evaluation of competencies" (p. 14). All teacher education students complete standards-based electronic portfolios, the details of which continue to evolve based on student and faculty feedback. The portfolio used in UAA teacher education programs is based on the Alaska State Teaching Standards, the NETS for Teachers Technology Standards, and national content standards.

Interns began creating their portfolios in the summer at the beginning of the program. They receive instructions on a variety of technology applications periodically throughout the internship year. Interns received handouts with a development calendar that included which skills were expected by which review. They also received handouts for self-instruction on a variety of topics. They are continually reminded to save everything in electronic format so they will have it if needed later for their portfolios.

The technology skills are usually taught by a technology instructor as part of the internship seminar. The content of the portfolio and process for reflection has been specified by program faculty and partners. The interns were able to personalize the portfolios since they are the ones who select the artifact that best demonstrated their proficiency in each standard. 

Working portfolios received formative and summative reviews by interns, faculty, and mentor teachers periodically during the program. The Intern Self-Review Form and Intern Continuous Assessment of Progress Form were used to facilitate the review. Interns completed the Intern Self-Review Form prior to the portfolio review conference. The School University Liaison completed the Intern Continuous Assessment of Progress Form following the conference.

By the end of their internship experiences, interns were responsible for creating a standards-based, electronic teaching portfolio that demonstrated their ability to make a difference in student learning. The final capstone course provided students with the opportunity to reflect on their learning for the year, culminating in a technology-supported summary presentation which included highlights from the portfolio.

Electronic Portfolios Data Collection

Students from all four cohorts from the PTEP program completed electronic portfolios. Data collection included analyzing a sample of the 2000-2001 and the 2001-2002 portfolios for proficiency in technology standards, surveying interns upon completion of the program, conducting focus groups during and at completion of the internship year, and interviews of a random sample of 2002-2003 interns.

The analysis of a sample of 2001-2002 portfolios showed all demonstrated proficiency at the Student Teacher/Intern Level for ISTE Technology Standard I, and the majority met Standards II, III, V, and VI. Many students in the first cohort, 2000-2001, erroneously thought they had addressed Standard IV (Assessment and Evaluation) by using an electronic grade book to record student evaluation results. Analysis of portfolios from subsequent cohorts indicates students' proficiency increased.

The portfolio development process has evolved as the programs were further developed and refined, Even so focus group data showed the students continue to report frustration with the portfolio development process. Each year additional changes were made to address this problem.

Survey data collected by Helen Barrett (2002) from interns in the first cohort, 2000-2001, led to the redesign of the portfolio development process. Since many of the needed technology skills were not developed until the final course, a new model for electronic portfolio development was created. The new model emphasized the need to develop multimedia skills during each semester. Dr. Barrett developed an Electronic Portfolio Handbook which included Step-By-Step tutorials to provide support for self-directed learning of skills. The handbook was given to the second cohort of students near the beginning of their program in July, 2001.

Results from the survey of the second cohort show similar patterns. How-to handouts and Internet tutorials were not useful to most of the students. Only one-half of the respondents found one-on-one meetings with the instructor helpful. The instructor may have been the technology or the content faculty so it is difficult to tell what type of support the interns were seeking.

An item was added to the 2002 survey to collect data about the Electronic Portfolio Handbook. Two respondents found the handbook to be essential or very useful and nine found it to be somewhat useful. The remaining eleven found it to be not useful or not used.

Results from the 2002-2003 cohort show this trend continued. Responses indicated that students came into the program with more technology knowledge and skills than the prior cohorts. They still found printed tutorials and material to be of limited use to at least half the cohort. The Electronic Portfolio Handbook was found to be useful by one, somewhat useful by three, and not useful or not used by thirteen respondents. This could be a result of changing to a newer version of the operating system so the tutorial information was not current. Program expectations for the portfolio were changed as well and the information was distributed as part of the program handbook.

A random sample of interns, interviewed in June 2003 by Ann McCoy, identified questions and concerns related to the format of portfolio, content and process, purpose, technical assistance, and confusing expectations. Intern interviews unanimously identified a lack of faculty technology skills as a barrier to students' portfolio development. Some common themes emerged. They recommended more frequent planned visits to the portfolio, scheduled lessons/workshops throughout the year, built into the time for peer and mentor feedback and more technology support throughout the year. Interns reported that different people in the program, faculty, school university liaisons, and mentors, had differing expectations for how the portfolio was to be completed and it's purpose. Students would like instructors who are comfortable with technology and share a common vision of portfolio purpose and requirements.

Twenty-four College of Education faculty were surveyed in Spring 2003 about use of technology and portfolios requirements in their courses. The survey instrument was distributed to faculty on PDAs at the monthly faculty meeting. Table x: Faculty Use of Technology and Table x: Faculty Expectations for Students to Use Technology show the results.

Table x: Faculty Use of Technology 2002-2003

 

Email

Blackboard

Internet

Word Processing

Spreadsheet

Presentation Software

Graphics Software

Digital Pictures or Video

Handhelds

Subject Specific Software

Number

23

20

23

23

16

17

14

18

8

11

Percent

95.8%

83.3%

95.8%

95.8%

66.7%

70.8%

58.3%

75.0%

33.3%

45.8%

n=24

                   

Table x: Faculty Expectations for Students to Use Technology 2002-2003

 

Email

Blackboard

Internet

Word Processing

Spreadsheet

Presentation Software

Graphics Software

Digital Pictures or Video

Handhelds

Subject Specific Software

Number

19

15

20

20

9

7

8

10

3

9

Percent

79.2%

62.5%

83.3%

83.3%

37.5%

29.2%

33.3%

41.7%

12.5%

37.5%

n=24

Results related to portfolio requirements are shown in Table x: Faculty Requirements for Electronic Portfolios by Programs. PTEP faculty reported mixed requirements for the portfolio. Since portfolios are required for the program, it makes sense that they may not necessarily require a portfolio specifically for the courses they teach. However, only three of the five require interns to produce artifacts for the portfolio in their classes. Lack of faculty understanding and use of portfolios was a consistent finding from the intern interview in June 2003.

Table x: Faculty Requirements for Electronic Portfolios by Programs

Program Faculty

Portfolio Required for Course?

Portfolio Artifacts for Program?

Elementary Education 1

No

No

Elementary Education 2

Yes, Non-Electronic

Yes, Non-Electronic

Elementary Education 3

Yes, Electronic

Yes, Electronic

Early Childhood Education 1

No

Yes, Electronic

Early Childhood Education 2

Yes, Electronic

No

Early Childhood Education 2

No

Yes, Non-Electronic

Early Childhood Education 4

Yes, Non-Electronic

Yes, Electronic

Ed Leadership 1

Yes, Electronic

Yes, Electronic

Ed Leadership 2

Yes, Non-Electronic

Yes, Electronic

Ed Leadership 3

No

No

Other 1

Yes, Electronic

Yes, Electronic

Other 2

No

No

Other 3

Yes, Electronic

Yes, Electronic

Other 4

Yes, Electronic

No

Other 5

Yes, Non-Electronic

No

PTEP 1

No

Yes, Electronic

PTEP 2

No

No

PTEP 3

Yes, Electronic

Yes, Electronic

PTEP 4

Yes, Electronic

Yes, Electronic

PTEP 5

No

No

Special Education 1

Yes, Electronic

Yes, Electronic

Special Education 2

No

No

Special Education 3

Yes, Electronic

Yes, Electronic

Special Education 4

Yes, Non-Electronic

Yes, Non-Electronic

n=24

Each year the data received from various sources has been used to inform changes to portfolio development process. Results of surveys and portfolio review indicate interns would like to receive consistent directions and support for portfolio development.

Lessons Learned and New Directions

To address these suggestions and concerns, faculty from the three teacher education programs are participating in an AK PT3 Electronic Portfolio Symposium during the 2003-2004 year. The symposium gives them a forum to discuss issues of portfolio design and assessment. It also creates an environment conducive to continued renewal of the programs. All participants are in the process of creating their own standards-based electronic portfolios. Symposium activities have generated discussion about developing and assessing portfolios, clarifying expectations, and a much greater awareness of the purpose and process of portfolio development. Positive changes to the programs have occurred.

Faculty from the Department of Teaching and Learning along with teachers from partner districts are in the process of designing a continuum of practice.

For the last three years PTEP faculty and mentor teachers have been continuing to improve our support structures for and assessment of interns as relates to their development and attainment of the AK standards for teachers. While we have developed some tools and forms, we have still struggles with helping interns and mentors get a sense of "what it should look like in practice" and "what it should look like when."  Based on feedback this year it seems we have one last design element we need to work on, a developmental continuum that we can use as a rubric for assessing their progress during the internship. The continuum would also be a great support for our professional development work with inservice teachers and mentor teachers. (Unpublished email to the author from Janet Steinhauser, (June 2003)

The PTEP/MAT program faculty have clarified expectations for portfolio content and process. They are developing strategies for ensuring consistency. The elementary education faculty are in the midst of working with the first cohort of interns. Their expectations and processes are based on the PTEP model. In an effort to provide more useful support for interns, lab aides' roles have been clarified and communicated to the interns. A technology instructor had developed different handouts to try to find something that will support more of the interns. However, it may be that not all of the interns have a learning style that makes these resources helpful. Conversations from the faculty e-portfolio symposium have already led to positive changes in the programs. Additional data will be collected this summer.

References

Barrett, H. (1999). Electronic teaching portfolios. Published in SITE Annual Proceedings, AACE

Barrett, H. (2002). Researching the process and outcomes of electronic portfolio development in a teacher education program. Published in SITE Annual Proceedings, AACE

McCoy, A. and O'Dell, K. (July 2003). Supporting development of standards based electronic portfolio in teacher education, PT3 Grantees' Showcase, National Education Computing Conference.

NCATE. (2002). Professional standards for the accreditation of schools, colleges, and departments of education(2002 edition). Washington, DC: National Council for the Accreditation of Teacher Education.