Description of Assessment Tools

Before we describe the assessment and evaluation plan for program outcomes, we briefly review the methods used for the assessment of program outcomes:

1. 
   Computer Engineering Exit Exam: The computer engineering exit exam is designed based on the core Computer engineering courses in which all the program outcomes are covered. This includes the courses: COE 202, COE 205, COE 305, COE 308, COE 341, COE 344, COE 360, and COE 400. The exam characteristics are as follows:
   

• 
  60 questions (multiple choice format) in total
  
• 
  Two hours exam setting
  
• 
  Each question should take an average of 2 minutes to answer
  
• 
  Each question should map into one of the outcomes (a through n) covered by the course.
  

The COE exit exam focuses on fundamentals and provides a good feedback in identifying some areas of improvements in the program. Specifically, the exit exam addresses the program outcomes (a), (c), (e), (k), (l), and (n). The COE exit exam is conducted every semester.

2. 
   Seminar Course (COE 390): This course is used in the program outcomes assessment as it addresses some program outcomes with greater emphasis compared to other courses. In this course, effective presentation skills are covered and students are given the chance to give two presentations. In the first presentation, students will be coached on the areas that need improvement and will receive feedback from both the instructor and other students based on the use of oral presentation assessment rubric. The second presentation has a high weight in the course and the presentations skills of the students will be assessed to determine their ability in achieving this outcome. The first presentation is based on a topic related to computing ethics and the second is based on a technical computer engineering topic. Professional code of ethics and computing ethics issues are also covered in this course. In addition, aspects related to contemporary issues and the impact of engineering solutions in a global and societal context are addressed in the course through presentation by invited speakers. Students are asked to assess the achievement of the course outcomes at the end of the semester and these results are used as part of the assessment of related program outcomes.
   

3. 
   Embedded Systems (COE 400): The purpose of this course is to integrate student's knowledge of hardware and software in the design, implementation, debugging, and documentation of a major system. The twin learning experience of making hardware versus software decisions, and participating in a structured design are integrated into the same design exercise.
   

4. 
   Senior Design Project (COE 485): The purpose of this course is to integrate students' knowledge of hardware and software in the design. This course is designed to give students the experience of tackling a realistic engineering problem. The intent is to show how to put theoretical knowledge gained into practical use by starting from a word description of a problem and proceeding through various design phases to end up with a practical engineering solution. Various projects are offered by COE faculty in their respective specialization areas. The project advisor guides the student in conducting a feasibility study, preparing specifications, and adapting design methodology. Detailed design and implementation of the project are carried out followed by testing, debugging, and documentation. An oral presentation and a final report are required at the end of the semester.
   

5. 
   Cooperative Work (COE 350/351): This course is taken by all computer engineering students in the COOP program. It is based on a continuous period of 28 weeks spent in industry with the purpose of acquiring practical experience in various areas of computer engineering. During these activities, a faculty member and a company supervisor supervise the students. During this period, the student is exposed to the profession of computer engineering by working in the field. Students are required to submit a final report and give a presentation about their experience and the knowledge gained during their Cooperative work. Sample of COOP reports and presentations are collected every semester and evaluated using appropriate rubrics. Company supervisors are asked to evaluate the performance of the students during these activities. Industry supervisors of COOP students are also surveyed regarding the achievements of the POs.
   

6. 
   Summer Training (COE 399): The aim of the summer training program in industry is to provide students with direct on-the-job experience working with professionals in the field. This training provides an opportunity to expose students to the reality of professional practice. Students are required to submit a report and make a presentation on their summer training experience and the knowledge gained. The summer Training coordinator assigns faculty for attending student presentations and for grading their summer training report. The faculty carries out the rubrics assessment based on student presentation, training report, and COOP supervisor evaluation sheet. The rubric data contributes together with other rubric assessment data toward the assessment of some program outcomes.
   

7. 
   Graduate Exit Survey: The graduate exit survey is conducted every semester. The survey asks Computer Engineering graduates to assess their ability in achieving each of the program outcomes and provide feedback on which outcome they think needs improvement and greater emphasis in the program.
   

8. 
   Employer and Alumni Survey: Employers of COE graduates are surveyed once every three years as part of the indirect assessment process of the program educational objectives and program outcomes achievement. Employer surveys are sent to companies who hire our COE graduates. A similar process is adapted for surveying the COE Alumni.
   

9. 
   Industrial Advisory Committee (IAC): The industrial advisory committee consists of members from industrial organizations, which hire a good percentage of our graduates. The IAC members are asked to provide their input on the program learning outcomes. They are also asked to provide feedback on the achievement of program outcomes by our graduates and on which outcomes need to be improved through the program. Whenever possible, they will also be asked to evaluate some of students' work, especially the senior design project and Cooperative work, through which many of the program outcomes can be assessed. Two meetings (2007 and 2009) were arranged with IAC. Minutes of the IAC meetings and discussion sessions will be part of the display material. Starting from 2009, a meeting with the IAC is be arranged once a year.
   

In this section we describe how each program outcome is assessed. For each program outcome, an assessment and evaluation plan is developed that consists of the following elements:

1. 
   Assessment and Evaluation Methods: This describes what assessment methods are used to collect data and how the data is evaluated and interpreted.
   

2. 
   Performance Criteria: This determines the criteria used to indicate whether an outcome has been achieved with satisfactory levels or it needs improvement.
   

3. 
   Logistics: This indicates when the data is collected and who will collect it, interpret it, and report the results.
   

The assessment and evaluation plan for the program outcomes is shown in Table ‎3.6 -17. For each program outcome, the assessment and evaluation methods are given. Scoring rubrics designed by the undergraduate committee are used for the assessment of the outcomes. For each assessment method, the used performance criteria enables determining when an outcome is considered achieved. Finally, the frequency of assessment of each program outcome is also indicated.

The objective of this sub-section is to describe how actual learning by students is judged in the courses used in the rubric assessment. The basis for rating the learning achieved using direct measures is presented. For each rubric (program outcome) the rating culminates into determining the level of achievement of the corresponding program outcome. Support to the above course learning and its rating is presented in the next sub-section in form of student-by-student performance tables within the rubric assessment.

The rubric-based assessment uses three COE courses for assessing most of the technical program outcomes. These courses are: (1) the Digital Systems Design (COE 400), (2) the Senior Design Project (COE 485), and to some lesser degree (3) the Cooperative Work (COE 350/351). Course organization is summarized as follows:

1. 
   The Embedded Systems (COE 400)
   

This course is a project-oriented embedded systems course. The project takes over 70 % of the time and effort in this course. At the beginning of the semester, few weeks are taken to teach the students the theoretical background of embedded systems development. During this period, basics of embedded systems are introduced followed by more detailed hardware and software related material. Hands-on experience is taught during lab sessions. In the lab sessions, students are asked to reproduce known experiments then modify the experiments in order to achieve new goals and features. The project is introduced slowly through few brainstorming sessions where students propose ideas for projects. Following numerous discussions, one project is selected through voting. The selected project must require enough work and diversity of aspects to be distributed among all the students in the course. Students are individually asked to translate business-level requirements, provided by the instructor, into technical specifications. Following a discussion session then unifies the various ideas expressed into the specification and produces a single specification report. Students are then divided into groups where every group is responsible for one aspect/part of the project. Students are taught that decision making is performed through consensus among the members of the group and in consultation and coordination with other groups when necessary. The instructor acts as a technical advisor/expert for any group. After several weeks of work, the different groups are required to produce a design report in which they describe the various aspects of the part of the project assigned to them. The implementation phase then starts in earnest. At the end of the semester, a combined final report is produced by all the groups as a collective effort.

2. 
   The Senior Design Project COE 485
   

1. 
   Although there is one regular weekly class, office hours and appointments are utilized by students to obtain all necessary information and guidance towards meeting project objectives.
   
2. 
   Email and WebCT are the main form of communication outside the classroom as required by the project progress.
   
3. 
   The Internet (company websites, newsgroups, forums, etc.) and the KFUPM library are the main source for information regarding the proposed projects. Bookstores are also a good source for books unavailable at KFUPM.
   
4. 
   Projects are student-driven; the instructor will provide description of the final outcome and set guidelines for the development process. Students are responsible for carrying out the intended tasks. The projects require excellent programming skills as well as hardware interfacing in addition to self motivation and persistence.
   
5. 
   Students are required to read the rubrics (design, teamwork, and lifelong learning) and pay attention to the following:
   
   1. 
      They should understand the objective of the project and identify and formulate the problem in their project before they start the designing phase.
      
   2. 
      They should list different solutions or design alternatives and choose the appropriate solution using technical and economic evaluation methods.
      
   3. 
      After selecting the best solution (design), they should break down the design into tasks and steps to achieve their goal and identify the role of each one student in the group.
      
   4. 
      During the course of the senior project, students alternate in assuming the role of the team leader to test their ability to function as a team leader and a team member.
      
   5. 
      During the course of designing and implementing their projects the instructor tests their behavior in dealing with these challenges, how they make decisions, and check their level of independency.
      
   6. 
      A demonstration of their work is required at the end of the project as well as the submission of a technical report.
      
6. 
   Work will be carried out by teams (3 or 4 students per team) – all members of a team should contribute equally to the project
   
7. 
   Although cooperation amongst all students is encouraged, teams must work independently. Deliverables (designs, background material, code, report, etc.) submitted by different teams that are similar will receive an F grade.
   
8. 
   It is prohibited to copy or paste text, figures, diagrams, or plots from other sources (books, articles, etc.) in the progress/final report or presentations without referencing the original source. Student may refer once to figures, diagrams, or plots that appear in other sources, then student should include clear reference to the source in the caption. An ‘F’ grade will be given to students who do not adhere to the above rule.
   

For COE 400 the Rubrics Rating of Student Learning is based on the Student Design Report which includes problem definition, analysis of alternate solutions, selection of solutions, implementation, debugging, testing, and documenting the project. It also includes work habits and team work aspects and many other soft aspects.

1. 
   Project Implementation
   

• 
  Engineering approach: Specification, System design, critical examination of different approaches and justification for the selected approach(s), and utilization of basic engineering science in the design.
  
• 
  Completion of the design and the demonstration.
  
• 
  Design verification and testing: Simulations, modeling, emulation, prototyping (where appropriate) and testing.
  

2. 
   Work Habits
   

• 
  Interview with the student to evaluate his involvement in the project
  
• 
  Peer evaluation where each group member evaluates every other member in the group
  
• 
  Motivation, organization, self-reliance, planning, and critical thinking.
  

3. 
   Project Documentation and Final Report
   

• 
  Compliance with the report writing guidelines
  
• 
  Clarity of the problem description and proposed solution
  
• 
  System design, approach selection and design segmentation
  
• 
  Implementation/Testing report and any product manuals required
  

4. 
   Final Presentation
   

• 
  Clarity of stated problem and solution
  
• 
  Quality of presentation (organization, body language …etc)
  
• 
  Discussion (how the student answers the questions which demonstrate his understanding of the project and its socio-economical aspects)