Assessing Study Skill Needs for Information Technology and Computer Science Students in Technical and Vocational Universities.

Not a great deal is known about what study skills are essential for success in the science, technology, engineering, and mathematics' disciplines, particularly in information technology (IT) and computer science (CS) programs in the technical and vocational education (TVE) in Taiwanese higher education. Since TVE accounts for more than half of the post-secondary enrolments in the country, and with the increasing demand for IT and CS talents, we studied perceptions of the importance and competency of such skills to identify the students' needs and demographic factors influencing them. A survey was administered to 1398 students in IT and CS programs in Taiwanese TVE universities. General skills were seen as the most important one among the study skills examined, and students felt competent using them. The needs to manage time, perform quantitative/mathematical tasks, and delegate were identified and these needs were affected by institutional quality, gender, and academic achievement. The results might be useful for further investigation in this area and guiding future plan to improve student performance in TVE.

Further considerations of evaluation competencies in Taiwan.

A list of evaluator competencies (Stevahn, King, Ghere, & Minnema, 2005) was adapted to fit the Taiwanese context by Lee, Altschuld, & Lee (2012). It was studied as to how it generalized to a large sample in Taiwan. Likert and Fuzzy surveys with needs assessment formats (importance and competence) were mailed via random assignment to two groups of participants. The questions for the study were: do the modified competencies relate country-wide to Taiwan, did the investigation uncover training needs for evaluators, and were there convergent rating patterns across the two forms of the instrument? The results supported a fit of the modified competencies to the context and convergent validity was observed but strong competency needs were not apparent. Reasons for the findings and implications for future research are discussed.

Challenges in needs assessment: the Head Start needs assessment national analysis.

Needs assessment (NA) is conducted to identify discrepancies that guide the development and improvement of services. In this article we focus on methodological issues in a secondary analysis of a multi-site, national needs assessment. Understanding challenges in such a context is important for needs assessors. The Head Start needs assessment project is a good illustration of the problems that can be encountered. Lessons learned will be useful for advancing NA practice in the future.

Exploring training needs of educational program evaluators in the Asia-Pacific region.

Training is critical for the growth of the evaluation profession as more qualified and competent individuals will be required to meet the increased demand for evaluation. This paper deals with the training needs of professionals and is based on a part of a study on the status and challenges of educational program evaluation in the Asia-Pacific region. A web-based Delphi was used with a needs assessment framework for examining the current nature of training, what should be done, and discrepancies (gaps) between the two. The results should be of value for academic training programs and those in non-academic settings.

Essential competencies for program evaluators in a diverse cultural context.

Essential evaluator competencies as identified by Stevahn, King, Ghere, and Minnema (2005) were studied in regard to how well they generalize to an Asian (Taiwan) context. A fuzzy Delphi survey with two iterations was used to collect data from 12 experts. While most competencies fit Taiwan, there were a number of unique ones. A complete set of results is provided along with the implications of the findings and what they might mean for evaluation in Taiwan particularly in relationship to the professionalization of evaluation.

Understanding the "what should be condition" in needs assessment data.

In needs assessment (N/A), the calculation of discrepancies is based on the assumption the "what should be" condition is a reasonable representation of respondent perceptions. That assumption may be erroneous and requires a closer inspection. This paper examines the use of importance scores in NA and some of the problems that can arise when they are used as a proxy to measure the "what should be" condition. A review of the literature and ways of dealing with importance scores are presented, followed by a discussion of the problems and issues that can arise. Some solution strategies are offered along with recommendations for practice and research. The paper provides guidance for others interested in improving needs assessment procedures.

Practices and challenges in educational program evaluation in the Asia-Pacific region: results of a Delphi study.

While educational program evaluation has become more important in recent years because of increasing governmental demands for accountability, little is known about the development of and issues in regard to this topic in the Asia-Pacific region. The findings from a Delphi study conducted in a number of relevant countries are provided in this article. Thirty panelists participated in three rounds of iterations to identify current status, challenges, and concerns in the near future. The results should be of value to educational evaluators, researchers, and decision makers.

Evaluating minority retention programs: problems encountered and lessons learned from the Ohio science and engineering alliance.

The retention rates for African-Americans, Hispanics, and Native-Americans in science, technology, engineering, and mathematics (STEM) are lower than those of White or Asian college students. In response, the National Science Foundation formed statewide partnerships of universities to develop programs to address this disparity. The deliberations and experiences in evaluating one such partnership are retrospectively reviewed. Problems and issues encountered during conceptualization and implementation are presented. Lessons learned from this endeavor should generalize to similar situations and provide guidance for others new to or interested in evaluating STEM retention programs as well as those evaluating collaborative endeavors.

Methodological and conceptual issues confronting a cross-country Delphi study of educational program evaluation.

Although the Delphi is widely used, research on certain methodological issues is somewhat limited. After a brief introduction to the strengths, limitations, and methodological challenges of the technique, we share our experiences (as well as problems encountered) with an electronic Delphi of educational program evaluation (EPE) in the Asia-Pacific region. The study is described followed by a discussion of the difficulties in participant recruitment and selection, sample size, instrumentation, data collection and analysis and attempts to resolve them. Some of these problems are generic to the Delphi whereas others related to the specifics of the investigation. What we learned should be useful for future Delphis with a similar focus.

Effects of multiple stakeholders in identifying and interpreting perceived needs.

The participation of diverse groups is advocated for planning and implementing needs assessment (NA) procedures. While the involvement of varied constituencies is important, obtaining it requires more effort from the needs assessor and therefore is less commonly employed [Witkin, B.R. (1994). Needs assessment since 1981: The state of the practice. Evaluation Practice, 15(1), 17-27]. In this paper the perspectives held by two groups of stakeholders in an NA were obtained, compared, and when disparate, the groups were queried as to why they were different. A mixed-method design, a quantitative approach followed by a qualitative one, was utilized. Self-report data were collected from the two groups. Group effects, although not strong, were apparent for some items. Understanding the reasons for such differences contributes to a fuller and more meaningful interpretation of needs.

Problems in needs assessment data: discrepancy analysis.

Needs assessment (NA) is generally based on the discrepancy between two conditions-the desired and present states. To date, there has not been an extensive research regarding a number of subtle problems in discrepancy analysis. One such example is missing data for one or both the two states. This leads to highly varied item n's for calculating discrepancy scores. Concerns like this arose in a NA study of minority students in science, technology, engineering, and mathematics programs in universities. A number of problems observed in this context are discussed as well as possible solutions for them. The results should be valuable to needs assessors and evaluators responsible for assessing needs.