•
Description of the technical approach and method-ology, work plan, and organization and staffing proposed by the consultants to perform the assign-ment (see para. 3.4 [c][i] of the ITC)•
Proposed professional staff (see para. 3.4 [d] of the ITC)•
Estimates of the staff input needed to carry out the assignment (see para. 3.4 [e] of the ITC)•
Curricula vitae (CVs) of the proposed professional staff (see para. 3.4 [f] of the ITC)•
Description of the proposed methodology and staffing for training, if the training is a specific component of the assignment (see para. 3.4 [g] of the ITC)The FTP is recommended when
(a) the assignment is likely to have significant down-stream impacts;
(b) the assignment is large and complex and the TOR require consultants to carry out activities that differ substantially from each other or activities whose contents may vary, depending on the results of other previous activities to be carried out under the same assignment (as may be the case, for example, with a phased assignment or an assignment requiring the comparison of dif-ferent alternatives);
(c) the assignment is new and unusual, making it difficult to specify, in the TOR, the tasks that the consultants will have to complete, or alter-native methods can be used to achieve the as-signment objectives and required outputs;
(d) the characteristics of the assignment require the evaluation of specific aspects of the consultants’
experience, in addition to those already consid-ered by the Borrower during the preparation of the short list (the FTP [and not the STP] pro-vides a specific section in which consultants are requested to outline their organization and
re-cent experience on assignments similar to the one under consideration); and
(e) the capacity-building program is a specific com-ponent of the assignment, because the FTP (and not the STP) provides not only a specific section in which consultants are requested to lay out their proposed methodology and staffing for ca-pacity building, transfer of knowledge, and train-ing but also a specific criterion (and subcriteria, if needed) for the Borrower to evaluate the suit-ability of the proposed program.
The FTP offers consultants the possibility of de-scribing their proposed methodology without limit to the number of pages, while the STP recommends lim-iting it to 10 pages (see para. 12.2.2). The FTP also allows the Borrower to use subcriteria to evaluate (a) the adequacy of the proposed methodology and work plan (see para. 12.3.1) and (b) the capacity-building pro-gram, if requested by the TOR.
12.2.2 The Simplified Technical Proposal (STP)
The STP is a new, less-detailed format of technical pro-posal introduced in the May 2004 edition of the Bank SRFP. Different from the FTP, the STP comprises the following four sections:
•
A description of the proposed technical approach and methodology, work plan, organization, and staffing by the consultants undertaking the assign-ment (this section could also include consultants’comments on the TOR and counterpart staff and fa-cilities and should normally comprise 10 pages, in-cluding, if needed, charts and diagrams) (see para.
3.4 [c][i] and 3.4 [c] [ii] of the ITC)
•
The proposed professional staff (see para. 3.4 [d] of the ITC)•
Estimates of the staff input needed to carry out the assignment (see para. 3.4 [e] of the ITC)•
Curricula vitae (CVs) of the proposed professional staff (see para. 3.4 [f] of the ITC)The STP should be used when
(a) the assignment is unlikely to have significant downstream impacts;
(b) the assignment is of a routine, straightforward, or simple nature and when the TOR already de-fine in detail the tasks the consultants will have to complete, the assignment objectives, and re-quired outputs (therefore, [i] the description of
the proposed technical approach and method-ology, work plan, organization, and staffing are relatively uncomplicated and may be reasonably limited to 10 pages, and [ii] the Borrower can evaluate the technical proposals without using subcriteria);
(c) the characteristics of the assignment do not re-quire the evaluation of specific aspects of the consultants’ previous work experience, in addi-tion to those already considered by the Borrower for the short list; and
(d) capacity building is not a specific component of the assignment, and consequently it does not require a separate evaluation.
When one or more of the above conditions are not fulfilled, the FTP should be used instead of the STP.
The STP reduces the time and cost required not only by the consultants to prepare their proposals but also by the Evaluation Committee to evaluate them.
12.3 Evaluation Criteria of Technical Proposals
12.3.1 Evaluation Criteria for the FTP
The following criteria shall be used as a basis for eval-uation of the FTP:•
Specific experience of the consultants relevant to the assignment•
Adequacy of the proposed methodology and work plan in responding to the TOR•
Key professional staff qualifications and compe-tence for the assignmentDepending on the particular objectives of the as-signment, two additional criteria may be required:
•
Suitability of the transfer-of-knowledge (capacity-building) program•
Participation by national consultants among pro-posed key staffThe first of these two optional criteria should be included and the capacity-building program should be clearly mentioned in the TOR when a transfer-of-knowledge (capacity-building) program is required by the nature of the assignment (see chapter 6). The sec-ond optional criterion, participation of national con-sultants, should be included when necessary for the assignment or desirable because the Borrower wants to encourage the participation of national consultants in its activities as a matter of policy.
SETTING THE EVALUATION CRITERIA 57
Evaluation Criteria for the FTP 1 2 . 3 . 1
The Data Sheet contained in the ITC of the actual RFP will indicate the points allocated to each of the cri-teria and subcricri-teria (see Paragraph Reference 5.2 (a) of the Data Sheet). Table 12.1 shows the range of points that may be allocated to each criterion on a scale of 1 to 100 in accordance with Bank Guidelines. The actual points must be adjusted to the specific characteristics and circumstances of the assignment, including the rel-ative importance of each criterion or the Borrower’s preferences. A very good practice is to have those who prepared the TOR advise on the choice of criteria and, if needed, subcriteria and the points given to each.
The criterion “adequacy of the proposed method-ology and work plan in responding to the TOR” is divided into the following three subcriteria (see Paragraph Reference 5.2 [a][ii] of the Data Sheet of the ITC):
•
Technical approach and methodology•
Work plan•
Organization and staffingThe criterion “key professional staff qualifications and competence for the assignment” is divided into the following three subcriteria (see para. 2.17 of the Consultant Guidelines and Paragraph Reference 5.2 [a][iii] of the Data Sheet of the ITC):
•
General qualifications•
Adequacy for the assignment•
Experience in region and languageThe other three evaluation criteria,
•
specific experience of the consultants relevant to the assignment,•
suitability of the transfer-of-knowledge (capacity-building) program, and•
participation by national consultants among pro-posed key staffmight also be divided into subcriteria. However, given their relatively small assigned weights (in most cases, a maximum of 10 points each out of 100), adoption of subcriteria is not recommended, and the suggested ap-proach for evaluating an FTP under these three criteria is to assess the substantial responsiveness of the proposal to the aspects listed for each of them in chapter 17.
When the capacity-building program is a particu-larly important component of the assignment, or even its main objective, this fact should be indicated in the TOR; in such case, subject to the Bank’s prior clearance, more than 10 points may be allocated to this criterion and subcriteria may be provided (see para. 12.8).
Because subcriteria and their weighting often de-termine the outcome of the evaluation, they must be derived from the aspects that are critical to the success of the assignment. Evaluation criteria and subcriteria, associated points, and the rating system form a numeric model to assess the technical merit of each proposal.
The more accurate the model, the more reliable the evaluation and the greater the likelihood that the Borrower will select the proposal of the consultants best suited for the assignment.
All adopted subcriteria should be specified in the RFP as follows (see Paragraph Reference 5.2 (a) of the Data Sheet):
•
Under the criterion “adequacy of the proposed tech-nical approach, methodology, and work plan in re-sponding to the TOR,” Borrowers must indicate the points assigned to each one of the three subcriteria:“technical approach and methodology,” “work plan,” and “organization and staffing.”
•
Under the criterion “key professional staff quali-fications and competence for the assignment,”Borrowers must indicate the percentage weights as-signed to each one of the three subcriteria: “general qualifications,” “adequacy for the assignment,” and
“experience in region and language.” Under this cri-terion, Borrowers should also indicate, in addition to the team leader, whose position is mandatory, each one of the key experts responsible for the main activ-58 SETTING THE EVALUATION CRITERIA
Evaluation Criteria for the FTP 1 2 . 3 . 1
Table 12.1 Point Distribution of Evaluation Criteria for the FTP
Evaluation criteria Points
Specific experience of the 0–10 consultants
Adequacy of the proposed 20–50 methodology and work plan
Key professional staff 30–60
qualifications and competence
Suitability of the transfer-of- Normally 0–10 knowledge (training)
program—optional
Participation by national 0–10 consultants among proposed
key staff—optional
Total 100
ities or technical disciplines of the assignment and the points assigned to each of them.
•
When more than 10 points are allocated to “suit-ability of the transfer-of-knowledge program,” it is recommended that the points allocated to the three subcriteria spelled out in the Paragraph Reference 5.2 (a)(iv) of the Data Sheet be disclosed in the RFP, because it increases the transparency of the evalu-ation. The Borrower may adopt subcriteria that differ from those indicated in the Data Sheet if they do not adequately meet the characteristics of the capacity-building program. If points allocated to subcriteria are not disclosed in the RFP, the Evaluation Committee should allocate them shortly before proposal submission to reduce the risk of manipulation during the evaluation process.•
The points allocated to each subcriterion should vary according to its relative importance for the consultant assignment. It is recommended that no fewer than 3 points be allocated to each subcrite-rion. Allocation of fewer than 3 points would imply that the subcriterion is of only minor importance to the overall evaluation.Defining whether the methodology and work plan or the quality of staff is more important depends on the stage of the project, the nature of the assignment, and the preferences of the Borrower. On the one hand, be-cause the methodology and work plan are usually less important in the final stages of the project (such as con-struction supervision) and deserve fewer points, weight given to key staff may be higher. On the other hand, be-cause the methodology and work plan are more im-portant in the initial stages of a project (such as master plans and feasibility studies) and deserve more points, fewer points are left to key staff (see table 12.6). A Borrower with a strong sense of ownership over the intellectual work requested of the consultant may dictate its own approach in the TOR and even in the methodology, in which case it may value the key staff of the consultants more than their firm’s experience or methodology.
(Paras. 12.4 to 12.8 suggest proven practices for selecting subcriteria and allocating points [weights] to criteria and subcriteria for the evaluation of quality.)
12.3.2 Evaluation Criteria for the STP
The following criteria shall be used as a basis for eval-uation of the STP:•
Adequacy of the proposed technical approach, methodology, and work plan in responding to the TOR•
Key professional staff qualifications and compe-tence for the assignmentThe Data Sheet contained in the ITC of the RFP discloses the points allocated to each of the evaluation criteria and subcriteria (see Paragraph Reference 5.2 [b]
of the Data Sheet). Table 12.2 shows the range of points that may be allocated to each of the two criteria on a scale of 1 to 100, in accordance with Bank Guidelines.
The actual distribution should depend on the type of assignment, the relative importance of the two criteria for the success of the assignment, and the preferences of the Borrower. A very good practice is to have those who prepared the TOR advise on the selection of sub-criteria and related points.
The criterion “key professional staff qualifications and competence” is divided into the following three subcriteria (see para. 2.17 of the Consultant Guidelines and Paragraph Reference 5.2 [b][ii] of the Data Sheet of the ITC):
•
General qualifications•
Adequacy for the assignment•
Experience in the regionBecause the rationale of the STP is to provide a for-mat that is less time-consuming, less costly to prepare, and simpler to evaluate than the FTP, it is suggested that no subcriteria for the criterion “adequacy of the proposed technical approach, methodology, and work plan in responding to the TOR” be adopted, but that the criterion be evaluated as a whole (see para. 17.3.4 of this Manual).
Under the criterion “key professional staff qualifi-cations and competence for the assignment,” Borrowers must indicate the percentage weights assigned to each one of the three subcriteria “general qualifications,”
SETTING THE EVALUATION CRITERIA 59
Evaluation Criteria for the STP 1 2 . 3 . 2
Table 12.2 Point Distribution of Evaluation Criteria for the STP
Evaluation criteria Points
Adequacy of the proposed 20–40 methodology and work plan
Key professional staff 60–80
qualifications and competence
Total 100
“adequacy for the assignment,” and “experience in re-gion and language.” Under this criterion, Borrowers should also indicate, in addition to the team leader, whose position is mandatory, each key staff responsible for the main activities or technical disciplines of the as-signment and the points assigned.
(Para. 12.7 recommends “best” practices for allo-cating points (weights) to criteria and subcriteria for quality evaluation of the proposed key professional staff.)