Results Assessment Framework (RAF)

During the three years that the Seed Alliance has operated, the team has used the Results Assessments Framework (RAF) to monitor the three programs. The RAF supported the understanding on where the project started, the work done and where the project is heading, as well as how the goals and objectives intertwined.

What We Expected

External Expectations:

  • Receive high quality applications coming from as many of the economies as we serve
  • The grantees use their funding efficiently and in line with their proposed activities
  • The grantees use using training provided to enhance their capacity in business management M&E, communications and networking
  • The projects funded are enabled to move forward on their own innovation path (from idea to realization)
  • Recipients make use of networking and mentoring opportunities

The Seed
Alliance offers opportunities to project teams developing innovative Internet solutions addressing social and economic
challenge

3 pillars

3

Networking opportunities

2

Capacity building (training, mentoring and coaching)

1

Access to much needed funding (grants and awards)

What we are expecting to see (the changes we want) for those we support (external level)

Before Oct 2012

FIRE did not exists prior, no baseline available.

Oct 2012 – Nov 2014

14 economies (7.84% of economies covered) received 22 grants between 2012-2014.

8 economies (4.48 %) received 8 awards between 2012-2014.

1 regional award in 2012.

Oct 2014 – Oct 2015

No Grant in 2015.

3 economies (10.71% of economies covered) received 3 awards in October 2015.

Before Oct 2012

14 economies (39.4% of economies covered and Jamaica) received 56 grants between 2004-2009.

7 economies (21.2%) received 15 awards between 2010-2011.

Oct 2012 – Nov 2014

13 economies (39.4% of economies covered and Jamaica) received 19 grants, including scale-up funds, between 2012-2014.

9 economies (27.3%) received 16 awards between 2012-2014.

3 new economies (9.1%) received regional awards.

Oct 2014 – Oct 2015

2 economies received 2 grants in 2015. Only 1 economy covered (3%). Second grant awarded to Jamaica.

2 economies (6.6% of economies covered) received 2 awards in 2015.

No new economies received regional awards in 2015.

Before Oct 2012

11 economies (19.64% of economies covered) received 19 grants between 2004-2009.

5 economies (8.92%) received 4 awards in 2011.

Oct 2012 – Nov 2014

17 economies (30.35% of economies covered) received 22 grants between 2012-2014.

6 economies (10.71%) received 15 awards between 2012-2014.

Oct 2014 – Oct 2015

4 grants allocated to 6 different economies (one grant with regional coverage) 10.7% coverage of AP region.

4 awards allocated to 3 economies 5.3% coverage of AP region.

Before Oct 2012

FIRE did not exists prior, no baseline available.

Oct 2012 – Nov 2014

FIRE implemented the template.

The template was reviewed annually and incorporated recipients’ feedback from all 3 regions. Surveys conducted by every regional program about grants and awards application processes and general management of the grants programs.

Oct 2014 – Oct 2015

Annual reviews conducted, reporting templates updated and surveys conducted. To facilitate the assessment of all grant supported projects, a scoring tool was designed based on the following criteria and individual scores for projects supported can be found on the Impact section of this website, under the supported projects map.

Before Oct 2012

Basic reporting templates were available for grant recipients to share progress and final results. FRIDA and ISIF Asia secretariats communicated fluidly with grantees to assess project implementation.

Surveys were conducted as a mechanism to incorporate feedback to the following round of grants.

Oct 2012 – Nov 2014

FRIDA and ISIF Asia reviewed their existing templates and a combined template was produced.

Template reviewed by the evaluation consultants to improve monitoring and better capture project results.

Oct 2014 – Oct 2015

Annual reviews conducted, reporting templates updated and surveys conducted. To facilitate the assessment of all grant supported projects, a scoring tool was designed based on the following criteria and individual scores for projects supported can be found on the Impact section of this website, under the supported projects map.

Before Oct 2012

Basic reporting templates were available for grant recipients to share progress and final results. FRIDA and ISIF Asia secretariats communicated fluidly with grantees to assess project implementation.

Surveys were conducted as a mechanism to incorporate feedback to the following round of grants.

Oct 2012 – Nov 2014

FRIDA and ISIF Asia reviewed their existing templates and a combined template was produced.

Template reviewed by the evaluation consultants to improve monitoring and better capture project results.

Oct 2014 – Oct 2015

Annual reviews conducted, reporting templates updated and surveys conducted. To facilitate the assessment of all grant supported projects, a scoring tool was designed based on the following criteria and individual scores for projects supported can be found on the Impact section of this website, under the supported projects map.

Before Oct 2012

FIRE did not exists prior, no baseline available.

Oct 2012 – Nov 2014

9 events.

14 travel grants supported.

Oct 2014 – Oct 2015

3 events.

29 Travel Grants supported.

Before Oct 2012

No opportunities for capacity building support were available before 2012.

Oct 2012 – Nov 2014

8 events.

31 travel grants supported.

Oct 2014 – Oct 2015

2 events.

8 Travel Grants supported.

Before Oct 2012

1 event for prospective applicants 2009.

1 event for 2010 grant recipients on reporting.

Oct 2012 – Nov 2014

10 events.

26 travel grants supported.

Oct 2014 – Oct 2015

11 events.

58 Travel Grants supported.

60 seats available for online mentoring for start-ups.

Before Oct 2012

FIRE did not exists prior, no baseline available.

Oct 2012 – Nov 2014

14 economies (7.84% of economies covered) received 22 grants between 2012-2014.

8 economies (4.48 %) received 8 awards between 2012-2014.

1 regional award in 2012.

Oct 2014 – Oct 2015

No Grant in 2015.

3 economies (10.71% of economies covered) received 3 awards in October 2015.

Before Oct 2012

14 economies (39,4% of economies covered and Jamaica) received 56 grants between 2004-2009.

7 economies (21.2%) received 15 awards between 2010-2011.

Oct 2012 – Nov 2014

13 economies (39.4% of economies covered and Jamaica) received 19 grants, including scale-up funds, between 2012-2014.

9 economies (27.3%) received 16 awards between 2012-2014.

3 new economies (9.1%) received regional awards.

Oct 2014 – Oct 2015

2 economies received 2 grants in 2015. Only 1 economy covered (3%). Second grant awarded to Jamaica.

2 economies (6.6% of economies covered) received 2 awards in 2015.

No new economies received regional awards in 2015.

Before Oct 2012

11 economies (19.64% of economies covered) received 19 grants between 2004-2009.

2 economies (6.6% of economies covered) received 2 awards in 2015.

5 economies (8.92%) received 4 awards in 2011.

Oct 2012 – Nov 2014

17 economies (30.35% of economies covered) received 22 grants between 2012-2014.

6 economies (10.71%) received 15 awards between 2012-2014.

Oct 2014 – Oct 2015

4 grants allocated to 6 different economies (one grant with regional coverage) 10.7% coverage of AP region.

4 awards allocated to 3 economies 5.3% coverage of AP region.

Before Oct 2012

FIRE did not exists prior, no baseline available.

Oct 2012 – Nov 2014

14 recipients had the opportunity to attend 9 events.

1 event for prospective applicants in 2013.

1 event for prospective applicants in 2014.

Assessment in progress.

Oct 2014 – Oct 2015

29 recipients had the opportunity to attend 2 events.

1 event for the 2013 and 2014 Grant recipients on Intellectual Property.

1 event for the 2013 and 2014 Grant recipients on Business Development skills – Part 1.

1 event for the 2013 and 2014 Grant recipients on Business Development skills – Part 2.

Before Oct 2012

No opportunities for networking and mentoring were available before 2012.

Oct 2012 – Nov 2014

31 recipients had the opportunity to attend 8 events.

Oct 2014 – Oct 2015

8 recipients had the opportunity to attend 2 events.

Before Oct 2012

1 event for prospective applicants 2009.

1 event for 2010 grant recipients on reporting.

Remote support via email and Skype.

Oct 2012 – Nov 2014

26 recipients had the opportunity to attend 10 events.

Oct 2014 – Oct 2015

11 events.

81 Travel Grants supported.

60 seats available for online mentoring for start-ups.

Internal Expectations:

  • Increasing inter-RIR collaboration around grant making mechanisms such as funding, capacity building, mentoring, evaluation, communication and networking.
  • Regional programs are recognized by staff and members as an integral part of the organizations strategic objectives
  • Successful replication (knowledge transfer) of FRIDA and ISIF Asia in the African region

The Seed
Alliance offers opportunities to project teams developing innovative Internet solutions addressing social and economic
challenge

3 pillars

3

Networking opportunities

2

Capacity building (training, mentoring and coaching)

1

Access to much needed funding (grants and awards)

What we are expecting to see (the changes we want) for our organization (internal level)

Before Oct 2012

FIRE did not exists prior, no baseline available.

Oct 2012 – Nov 2014

Public website integrated with online application system and a combination of online and offline evaluation system.

6-12 month grants, awards based in categories, community vote included in 2014.

2 Proposals development workshops.

3 capacity building sessions.

Oct 2014 – Oct 2015

1 Business Development Skills workshop – Part 1.

1 Intellectual Property Rights Protection workshop.

1 workshop on Business Development skills – Part 2.

Development of 5 webinars to support grant recipients and applicants.

Before Oct 2012

Public website.

Online application system.

Offline evaluation system.

Funding mechanisms available were grants (12-24 months) and awards.

No training or networking opportunities.

Oct 2012 – Nov 2014

Public website integrated with online application system and a combination of online and offline evaluation system.

Funding mechanisms reviewed and new ones added (12 months grants, awards+, Scale-ups) based in categories, votes and selection committee assessments.

2 online workshops for proposals development.

3 capacity building sessions.

Oct 2014 – Oct 2015

Public website with online application system used in 2015 application cycle. Scale ups maintained but no projects selected under said category. Online workshops for proposal development held with support of LACNIC’s Workshop Coordinator and Webinars developed for Seed Alliance to support future applicants and grantees. Travel support for capacity building offered to 8 award recipients and grantees.

Development of 5 webinars to support grant recipients and applicants.

Before Oct 2012

Public website separate from online application and evaluation system using online tools plus FtF meeting.

Funding mechanisms available were grants (12 months) and awards.

Limited training and networking opportunities were available.

Oct 2012 – Nov 2014

Public website integrated with online application and selection system for a fully online evaluation process.

3-12 months grants, awards, incorporated community vote.

1 FtF proposals development workshop.

3 capacity building sessions.

Oct 2014 – Oct 2015

Public website integrated with online application and selection system for a fully online evaluation process.

And online community vote.

3 UFE&ResCom mentoring FtF events.

3 JFDI.Asia pre-accelerator courses run for 60 projects.

Development of 5 webinars to support grant recipients and applicants.

Before Oct 2012

FIRE did not exists prior, no baseline available.

Oct 2012 – Nov 2014

Support from Senior management and Executive Council for staffing, selection and promotion.

Anecdotal evidence suggests lack of understanding from staff about FIRE reporting requirements for different donors.

AFRINIC financial policies and procedures in use to report management of external funding sources.

Oct 2014 – Oct 2015

Support from Senior management for selection and promotion.

Communication to STAFF about the Programme and invitation to vote during the “Public Voting Phase”.

Before Oct 2012

Full support from Senior management for staffing and promotion process.

Anecdotal evidence suggests lack of understanding from staff about FRIDA objectives, activities and results.

LACNIC Financial policies and procedures in use to report management of external funding sources.

Oct 2012 – Nov 2014

Support from Senior management and Board for staffing and promotion.

Anecdotal evidence suggests lack of understanding from staff about FRIDA reporting requirements for different donors.

LACNIC Financial policies and procedures used to report management of external funding sources.

Oct 2014 – Oct 2015

Support from Senior management for selection and promotion.

LACNIC Financial Policies and procedures used to report management of external funding sources.

Before Oct 2012

Full support from Senior management and Executive Council for funding contributions, staffing and selection process.

Anecdotal evidence suggests lack of understanding from staff about ISIF objectives.

APNIC Financial policies and procedures in use to report management of external funding sources.

Oct 2012 – Nov 2014

Documented correspondence evidence full support from Senior management and Executive Council for funding allocation, staffing, selection and promotion. Internal re-structure process to align ISIF with a new Development Area.

Anecdotal evidence suggests lack of understanding from staff about ISIF reporting requirements for different donors.

Financial policies reviewed and processes streamlined to report management of external funding sources.

Oct 2014 – Oct 2015

Documented correspondence evidence full support from Senior management and Executive Council for funding allocation, staffing, selection and promotion. Internal re-structure process to align ISIF with a new Development Area.

Anecdotal evidence suggests lack of understanding from staff about ISIF reporting requirements for different donors.

Financial policies reviewed and processes streamlined to report management of external funding sources.

Before Oct 2012

During 2010, LACNIC, APNIC, AFRINIC and IDRC discussed the establishment of the Seed Alliance and the launch of the FIRE program.

Oct 2012 – Nov 2014

ISIF and FRIDA shared with FIRE all original contracts, reporting templates (technical and financial), forms, evaluation tools, mentoring areas, networking opportunities as well as descriptions of funding mechanisms, systems, processes and procedures they had used until 2011.

Since then, FIRE, FRIDA and ISIF team have actively shared online (via email, mailing lists, file sharing, and skype) as well as during the IGF events so that all 3 programs benefit from the lessons learn from the others.

A continuous improvement methodology is in use so that changes to the programing and planning are implemented, when appropriate.

Oct 2014 – Oct 2015

FIRE shared with ISIF Asia and FRIDA the Evaluation Scoring Matrix template prepared with the help of the FIRE Evaluator.

Exchanges and Cooperation between the 3 program are constant via email, mailing lists and during Seed Alliance Meetings (Paris – IGF Brazil).

Before Oct 2012

Prior to 2004, LACNIC, APNIC and IDRC senior management exchanged information about program operations, mechanisms, funding sources and strategies around the ICT4D R&D fund that operated in Asia.

From 2008 to 2010, LACNIC and APNIC shared information about FRIDA operations to support the establishment of ISIF Asia.

During 2010, LACNIC, APNIC, AFRINIC and IDRC discussed the establishment of the Seed Alliance and the launch of the FIRE program.

Oct 2012 – Nov 2014

ISIF and FRIDA shared with FIRE all original contracts, reporting templates (technical and financial), forms, evaluation tools, mentoring areas, networking opportunities as well as descriptions of funding mechanisms, systems, processes and procedures they had used until 2011.

Since then, FIRE, FRIDA and ISIF team have actively shared online (via email, mailing lists, file sharing, and skype) as well as during the IGF events so that all 3 programs benefit from the lessons learn from the others.

A continuous improvement methodology is in use so that changes to the programing and planning are implemented, when appropriate.

Oct 2014 – Oct 2015

FIRE shared with ISIF Asia and FRIDA the Evaluation Scoring Matrix template prepared with the help of the FIRE Evaluator.

Exchanges and Cooperation between the 3 program are constant via email, mailing lists and during Seed Alliance Meetings (Paris – IGF Brazil).

Before Oct 2012

Prior to 2004, LACNIC, APNIC and IDRC senior management exchanged information about program operations, mechanisms, funding sources and strategies around the ICT4D R&D fund that operated in Asia.

From 2008 to 2010, LACNIC and APNIC shared information about FRIDA operations to support the establishment of ISIF Asia.

During 2010, LACNIC, APNIC, AFRINIC and IDRC discussed the establishment of the Seed Alliance and the launch of the FIRE program.

Oct 2012 – Nov 2014

ISIF and FRIDA shared with FIRE all original contracts, reporting templates (technical and financial), forms, evaluation tools, mentoring areas, networking opportunities as well as descriptions of funding mechanisms, systems, processes and procedures they had used until 2011.

Since then, FIRE, FRIDA and ISIF team have actively shared online (via email, mailing lists, file sharing, and skype) as well as during the IGF events so that all 3 programs benefit from the lessons learn from the others.

A continuous improvement methodology is in use so that changes to the programing and planning are implemented, when appropriate.

Oct 2014 – Oct 2015

FIRE shared with ISIF Asia and FRIDA the Evaluation Scoring Matrix template prepared with the help of the FIRE Evaluator.

Exchanges and Cooperation between the 3 program are constant via email, mailing lists and during Seed Alliance Meetings (Paris – IGF Brazil).

Share this article