An Own Office for Joint Medical Education and Training Campus?

The Commission on Base Realignment and Closure recommended the creation of a joint medical training and education center to promote the exchange of medical personnel and units, interoperability among the different services, and cost and efficiency gains. In accordance with this recommendation, such a campus has been established at Fort Sam Houston, Texas.

RAND Health and the RAND National Defense Research Institute were engaged to provide technical and scientific assistance in several areas related to MET committee implementation, including examining the need for and feasibility of establishing a MET committee research and evaluation unit. This capability would normally be within the Office of Institutional Research (OIR).

What is the MET Committee and what does it do?

The METC is a multifaceted set of military trainings. It brings together most of the medical training currently conducted at the various military institutions. When fully operational, METC will train over 100 military medical specialties and become one of the largest medical training institutions in the world, with an annual capacity of over 24,500 students, an average of over 8,000 students per day, and a total of 1,400 faculty and staff. Its goal is to adopt best practices, improve training and become an effective “learning” organization.

Key Results

• The U.S. Department of Defense concentrates most of the military medical training for soldiers at Fort Sam Houston, Texas.
• The two main goals of the Joint University for Medical Education are to become an institution of excellence and an accredited university.
• An institutional research office would assist the JMEU in achieving its organizational goals and should be established.

Does METC need research and higher evaluation capabilities?

RAND researchers used two approaches to answer this question. One approach was to examine it in light of the organization’s goal of becoming a high-performing learning organization. The second was to assess the need in light of accreditation requirements.

High-performing organizations often adopt organizational development models to guide their strategic actions and focus on results. Most high-performing organizations use an organizational development model or methodology, such as Malcolm Baldrige’s National Quality Award program, total quality management, lean manufacturing, or Six Sigma. All of these models focus on measuring and analyzing organizational performance and using the results to improve the organization.

To support its goal of becoming a high-performing organization, METC must develop and maintain the ability to collect, organize, analyze, and use a wide range of process and performance data to support innovation and excellence. In addition, these data and analysis metrics and systems must be regularly reviewed and adapted to new or changing environments and stakeholder needs. RTI provides METC with such a capability.

Accreditation is an important indicator that the institution is providing quality and legal education and training. Accreditation may also be granted to an institution’s individual training programs, which demonstrates that the training program has met certain quality standards. Many METC programs require program accreditation. In addition, METC goals include formal accreditation of a higher education provider.

Accrediting agencies are increasingly requiring programs and institutions to develop and implement quality improvement plans and learning objectives, and to provide reliable evidence of the added value of student learning and subsequent workplace outcomes. The three accrediting agencies reviewed in this study identified a number of quality indicators that can be used to evaluate VET programs. These indicators include graduation or completion rates, employment or job placement rates, occupational qualification pass rates, employer satisfaction, participant satisfaction, and assessment of occupational skills and knowledge.

In particular, several of these indicators (license exam pass rates, employer satisfaction, job placement rates) require follow-up by program completers and supervisors. Rules regarding administrative structure, program duration, and trainer qualification may pose significant challenges to the Market Economy Treatment Committee. Regardless, if METC seeks accreditation in the future, it will need research and evaluation capacity to meet the accreditation requirements for institutional improvement plans, integrated assessment, and tracking of various indicators. Thus, METC’s two primary goals support the creation of the RTI.

What can we learn from institutions with a similar mission?

To gain insight into the types of studies and evaluations that could be conducted by a regional implementing agency, researchers examined organizations with similar missions to METC and drew appropriate conclusions. Similar organizations include corporate universities, which are separate entities within companies; METC’s British equivalent, the Defense Medical Education and Training Agency (DMETA); and federal agencies focused on training and development, such as the Veterans Health Administration (VHA).

The results show that while corporate universities vary in size and function, measurement and evaluation of program effectiveness is still a key component, and corporate training leaders devote significant resources and attention to evaluation. They also build evaluation into their training programs from the start and pay particular attention to evaluation issues when designing and planning programs. In addition, best practice organizations focus their evaluation activities on clients.

DMETA, on the other hand, emphasizes continuous evaluation throughout the training and development cycle. It also coordinates the higher-level evaluations for which the various UK departments are responsible. It also collects data and reports annually on performance indicators as part of the Defense Balanced Scorecard.

The VHA has invested significant time and effort in transforming itself into an effective learning organization. It has firmly embraced the need for continuous assessment, feedback, and change in all of VHA’s training and development programs and has invested significant resources in assessment, performance measurement, and metrics to improve the organization.

Recommendations for the Market Economy Steering Committee

The research and evaluation capacity of the SEM Committee is clearly needed and must be established. The following recommendations will be helpful to the SEM in establishing its own regional foundation. The office should be located so that it reports directly to the Executive Board and its director should be a member of the Executive Board.

METC should consider establishing a central repository to track students and indicators of their learning and progress, to collect and report data for a balanced scorecard, and to transform these results for use in organizational improvement, to collect, analyze, and report baseline data about the institution that may be needed for external reporting, and to plan and evaluate educational programs.