The three regional programs evaluate the projects that were given Grants based on the following criteria. The criteria is structured by twelve categories disaggregated in sub-categories that correspond either to the moment when proposal was approved or the Final Reports submitted by the Grant Recipients. Each sub-categories have differential scores theoretically defined according to their importance and contribution to their respective global category. Finally, weighted averages are calculated for each category and sub-category in order to present the final score percentage for comparative purposes.
A score card for each supported project can be seen when hovering over the Supported Projects map as well as the individual pages for each project.
| CATEGORY WEIGHT | PROPOSAL RELATIVE WEIGHT | REPORT RELATIVE WEIGHT | CATEGORY |
|---|---|---|---|
| 8 | PROJECT CONSISTENCY | ||
| 20 | Does not apply to report score | The objectives are clearly specified | |
| 20 | Does not apply to report score | There is a well-known methodology | |
| 20 | Does not apply to report score | The resources provided are sufficIent and appropriate in terms of quantity and quality | |
| 40 | Does not apply to report score | The link between objectives, methodology and resources are clearly demonstrated | |
| 100 | Subtotal | ||
| 7 | REGIONAL RELEVANCE OF THE ISSUE DESCRIBED | ||
| 30 | Does not apply to report score | A regional problem has been clearly identified as an issue to be solved | |
| 30 | Does not apply to report score | The problem identified is an ICT issue | |
| 40 | Does not apply to report score | Importance of the issue for the region has been clearly demonstrated | |
| 100 | Subtotal | ||
| 10 | ACHIEVABLE AND MEASURABLE OBJECTIVES | ||
| 40 | Does not apply to report score | Performance indicators have been clearly elaborated | |
| 20 | Does not apply to report score | The objectives are achievable | |
| 20 | Does not apply to report score | The objectives are measurable | |
| 20 | Does not apply to report score | The objectives are verifiable | |
| 100 | Subtotal | ||
| 9 | INNOVATIVE CHARACTER | ||
| 25 | Does not apply to report score | There is an innovative aspect in the project | |
| 25 | Does not apply to report score | There is an innovation in the field of ICT | |
| 25 | Does not apply to report score | This is a new initiative in the region | |
| 25 | Does not apply to report score | The technology has never been developed | |
| 100 | Subtotal | ||
| 10 | EXECUTION CAPACITY | ||
| 30 | Does not apply to report score | There is a clear project management plan | |
| 20 | Does not apply to report score | The project management plan is consistent with the objectives | |
| 25 | Does not apply to report score | The project team has all the knowledge to implement the activities | |
| Does not apply to proposal score | 25 | The plan has strictly been followed during the implementation stage | |
| 100 | Subtotal | ||
| 8 | BUDGET | ||
| 50 | Does not apply to report score | A financial plan has been submitted | |
| Does not apply to proposal score | 50 | The expenses have been made accordingly to the plan | |
| 100 | Subtotal | ||
| 8 | MONITORING AND EVALUATION PLAN | ||
| 40 | Does not apply to report score | A monitoring and evaluation plan exists and has been submitted to the ISIF program team | |
| Does not apply to proposal score | 30 | Monitoring and evaluation actions have been performed during the implementation stage | |
| Does not apply to proposal score | 30 | Both report contains a chapter on the monitoring and evaluation actions performed during the project | |
| 100 | Subtotal | ||
| 8 | CAPACITY BUILDING | ||
| 40 | Does not apply to report score | A capacity building plan has been identified by the project team | |
| 20 | Does not apply to report score | The project team is one of the beneficiaries of the capacity building plan | |
| 40 | Does not apply to report score | The capacity building plan includes more people than just the project team | |
| 100 | Subtotal | ||
| 8 | REPLICABILITY | ||
| 15 | 15 | There is an interest in other countries for the same project | |
| 35 | Does not apply to report score | The proposal demonstrated the feasibility of the project in other countries | |
| Does not apply to proposal score | 35 | The report proposed some means to replicate the project in other countries | |
| 100 | Subtotal | ||
| 7 | IMPACT | ||
| 50 | Does not apply to report score | Impact of the project has been clearly identified in the proposal | |
| Does not apply to proposal score | 50 | Reports show the the contribution of the project through the achievement of the impact | |
| 100 | Subtotal | ||
| 9 | SUSTAINABILITY | ||
| Does not apply to proposal score | 30 | Reports clearly demonstrate sustainability of the project beyond the funding period | |
| Does not apply to proposal score | 40 | Project team put in place innovative means to ensure sustainability of the project beyond the funding period | |
| Does not apply to proposal score | 30 | External partners demonstrate interest in financing the project beyond the funding period | |
| 100 | Subtotal | ||
| 8 | REPORTS | ||
| Does not apply to proposal score | 15 | Technical reports have been submitted | |
| Does not apply to proposal score | 15 | Financial reports have been submitted | |
| Does not apply to proposal score | 30 | Reports have been submitted on time | |
| Does not apply to proposal score | 40 | Both Report follows the requirements in terms of quality | |
| 100 | Subtotal | ||
| 100 | OVERALL SCORES |
As the table shows, each category adds up to 100 and is weighted according to theoretical definitions. In order to make the calculations, a simple sum is made inside of each category. Lastly, a weighted sum is calculated across all of the categories to reach a final score with 100 as the maximum. The formula expressed in Excel format is the following:
RW: RELATIVE WEIGHT
CW: CATEGORY WEIGHT
=((SUM(RWi:RWn)*CWi)+……+(SUM(RWi:RWn)*CWn) )/100
This method uses a maximum of 100 points to both allow for easier comparisons among projects, and at the same time enable the use of a five-level scale. Said scale stems from the division of the final scores (OVERALL SCORE) into five equal parts. Each of the classes is then assigned a number of stars, as can be seen in the following table:
| STARS | OVERALL SCORE (PERCENTAGE) |
| 1 | 0% y 20% |
| 2 | 20% y 40% |
| 3 | 40% y 60% |
| 4 | 60% y 80% |
| 5 | 80% y 100% |
To understand more about the methodology, download the Seed Alliance – Evaluation Methodology Final Report that FOCUS, our evaluation consultants , prepared in November 2017.
LinkedIn
Twitter
Facebook