What is an evaluation strategy?
An evaluation strategy is the documented process a procuring agency will use to evaluate proposals received in response to an RFP or similar procurement method where factors other than cost are considered. The procuring agency should develop an evaluation strategy in conjunction with the procurement plan.
The goal of an evaluation strategy is to ensure that the evaluation committee conducts an evaluation and selection of a product or service in an impartial, fair, efficient, transparent and non-discriminatory manner. To facilitate this, the procurement team must establish and include objective criteria, and any subjective criteria subject to a particular standardized scoring method that helps provide guidance to evaluation committee members as they apply independent judgment to assess such criteria, for evaluation in the solicitation:
- To determine whether a particular proposal is responsive, that is, whether it materially meets the requirements of the solicitation.
- To assign scores or ratings to each of the evaluation factors established in the solicitation and responded to by a proposer.
- To record all determinations of the evaluation process in writing along with the justification for determination.
An agency should be as specific as possible in documenting its evaluation strategy during the planning phase. To frame the development of its evaluation strategy, an agency should consider the following key elements of the strategy:
- Description of planned evaluation process.
- Mandatory requirements.
- Evaluation criteria important to the procurement (including cost).
- Relative importance (in percentages) of each criterion identified.
Unless an agency has delegated procurement authority to conduct a Competitive Sealed Proposal, it must provide this information with a statement of work, specifications, and other checklist items to DAS Procurement Services to coordinate the development of the evaluation strategy.
When to develop an evaluation strategy
An evaluation strategy should be developed during the planning phase after an agency has determined its method of procurement (refer to Determine Procurement Method
). The procurement team should develop an evaluation strategy if using a Competitive Sealed Proposal (RFP) method or another related procurement method that evaluates proposals on mandatory and desirable requirements in addition to cost.
Key information from the evaluation strategy, including the criteria and scoring methodology, is described in an RFP. This information must be final and documented in the evaluation strategy prior to completing the solicitation documents.
How to develop an evaluation strategy
During the Plan stage and prior to completing a solicitation document, the procurement team must determine and document a strategy that describes how the evaluation committee will evaluate proposals. The primary purpose of an evaluation strategy is to document the evaluation criteria, the scoring applied to each criterion, and the process for determining the scores for each criterion. This process will also help the procurement professional to develop templates and forms that will guide the proposal submission format.
The organization of an evaluation strategy may vary, but should address the following topics:
|The procurement professional identifies the roles and potential members
of the evaluation committee, which may include members of the
procurement team. Depending on the nature and complexity of the
solicitation, the procurement professional may establish a committee
that includes both voting and non-voting members. The procurement
professional serves as a facilitator and advisor and is not a voting
member of the committee.Responsibilities of evaluation committee members require that they:
The criteria for evaluation committee membership may include
representation by area of expertise, job function, relationship to the
end user organization, or as stakeholder outside of the organization. A
best practice is to include a minimum of three agency staff with
sufficient qualifications in the procurement’s product or service. Individuals
within the private sector who have subject matter knowledge and
expertise in the procurement may participate on an evaluation committee,
with or without compensation and as non-voting advisors. A procurement
professional should ensure private sector participants do not have a
conflict of interest and agree to keep the evaluation and all
information they view confidential.
- Read, sign and submit a conflict of interest form to the procurement professional.
- Read the RFP and all addenda.
- Rate each proposal independently.
- Keep rating confidential during individual scoring.
- Do not discuss proposals outside of evaluation committee meetings.
- Do not permit personal knowledge and bias towards a firm or product,
and evaluate on material provided in, or directly related to, the
- Score consistently and do not change evaluation standards across proposals.
- Are able to attend all meetings and demonstrations.
Resource: Log in to ORPIN to access the approved Evaluation Committee Instructions and Conflict of Interest Certification forms to manage the process of selecting and commissioning an evaluation committee.
The evaluation strategy should describe the specific process for how the committee will evaluate solicitation responses.
The evaluation process typically includes four phases:
Depending on the agency’s procurement method and approach during the
evaluation and selection processes, the agency may choose to add an
additional phase to identify a competitive range.
- Evaluation of mandatory requirements.
- Evaluation of proposals.
- Evaluation of cost.
- Evaluation scoring, ranking and selection.
The following sections provide an overview of each phase of the typical evaluation process.
Evaluation of mandatory requirements
A responsive proposal must satisfy all mandatory requirements of a
solicitation. A procuring agency determines whether a proposal is
responsive by evaluating each of the mandatory requirements on a
pass-fail basis. This evaluation assesses whether a proposal materially
conforms to the minimum qualifications and submission requirements and
determines whether a proposal moves forward in the evaluation process.
evaluation strategy should identify all mandatory requirements
evaluated in this phase. Developing an administrative review checklist
that includes mandatory requirements used in performing this phase of
evaluation is a best practice. This assures that the evaluation is
consistent across all proposals.
Mandatory requirements can include, but are not limited to the following:
If a proposal fails any of the mandatory requirements, it is considered
non-responsive and is not evaluated further. In some cases if the
failure is determined to be minor or immaterial in nature, the procuring
agency may choose to work with the proposer to resolve the issue and
allow the proposal to progress in the evaluation process.
- Proposal submitted before opening date.
- Proposal submitted in proper format.
- Proper number of proposals submitted.
- Proposal signed by authorized person.
- Submitting required forms and information.
- Minimum qualifications.
- Minimum experience.
- Meets specified methodology.
- Meets specified functionality.
- Minimum price.
Evaluation of proposal
Committee members evaluate the scored criteria on each proposal that
meets mandatory requirements. Each evaluator should use standardized
tools and forms to assign scores to each component of the evaluation
criteria. Based on findings by the evaluation committee, any proposal
found to be incomplete, inconsistent, or inaccurate may be deemed as
non-responsive and can be rejected.
In some cases if the failure
is determined to be minor or immaterial in nature, the procuring agency
may choose to work with the proposer to resolve the issue and allow the
proposal to progress in the evaluation process.
Evaluation of cost
The procurement professional will evaluate the cost proposal for each
proposal that meets mandatory requirements. Cost proposal scoring is
based on the methodology described in the solicitation document. Refer
to topic #4, titled "Evaluation criteria weighting and calculations", for the section on objective criteria that includes examples of scoring methods.
no time should the evaluation committee ever see or be informed of the
cost proposals or scores received for proposals under evaluation while
performing its evaluation of proposals.
Evaluation scoring, ranking, and selection
After the evaluation committee has evaluated and scored each of the
proposals, those scores are combined with the cost evaluation scores
computed by the procurement professional to calculate a final score for
each proposal. Ranking these scores allows for supplier selection based
on best value, typically the highest scoring proposer.
Identification of competitive range
In certain circumstances, a process may require, or the procurement team
may determine that it is beneficial to establish a competitive range
for proposers, following the scoring process. Both the use of, and the
approach that will be used for, the competitive range should be
identified as part of the evaluation strategy. In addition, if the
procurement professional plans to use a competitive range, the
solicitation document must communicate this fact to potential proposers.
competitive range is the establishment of a threshold for scores that
will be acceptable to the agency for continued consideration in the
procurement process or for negotiations. Acceptable options for
determining a threshold include using a formula or using the outcome of
the evaluation. In either case, the goal is to limit the proposals in
the award selection pool.
Evaluation criteria are the measure used to assess each proposal in a
fair and consistent way. Criteria consist of the factors and sub-factors
that reflect the areas of importance to an agency in its selection
decision. Using evaluation criteria, the procuring agency can assess the
extent to which a proposal is responsive to a solicitation’s
specifications or scope of work. The assessment also helps make a sound
Evaluation criteria provide committee members
with a clear and concise method to identify responsive proposals, allow
the procuring agency to rank the proposals and ultimately select the
proposal representing best value to the state.
evaluation criteria, the procurement team should begin by making a list
of the most important aspects of the product or service required,
including cost. From this list, the team should group and then
prioritize the factors by category in order of importance. These
categories represent the evaluation criteria for the product or service.
This process will help to determine the most appropriate weighting for
each criteria based on its relative importance. The most important items
will naturally be evaluated heavier and have more points available.
is no right or wrong evaluation criteria for a solicitation. Evaluation
criteria should be tailored to each procurement and include only those
factors having an impact on the selection. The nature and types of
evaluation criteria applied to a procurement are within the broad
discretion of the procuring agency.
Evaluation criteria weighting provides a means for the procuring agency
to communicate the relative importance of each evaluation criterion to
potential proposers. The procuring agency must include the evaluation
criteria and the weighting of those criteria in the solicitation
document. The following table provides an example of a solicitation document evaluation criteria weighting:
There are two types of evaluation criteria – subjective criteria and
objective criteria. The evaluation and final score calculation are
different for the two criteria types.
|Qualifications, prior experience and references
Subjective criteria are criteria that require an evaluation committee
member to make an independent judgment of the proposal to assign a
score. Examples of subjective criteria include:
The procurement team will need to develop a standardized scoring method
to provide guidance to evaluation committee members in assessing
subjective criteria. The formats used can vary; however, standardized
scoring methods typically establish ranked levels with associated
scoring for each. The following tables provides examples of a standardized scoring method:
- Technical approach.
- Management approach.
- Project plans.
|Does not meet/no response
While the standardized
scoring method above is sufficient to perform an evaluation, the
procurement team will also need to define each ranking to remove
ambiguity and promote a uniform approach to the evaluation process.The table below provides an example of each rank and its definition:
|Does not meet/no response
Establishing a standard scoring method promotes objectivity and
consistency in assessing subjective criteria. This approach will derive a
best value selection for the state and limit the likelihood of
||Response meets all the requirements and
has demonstrated in a clear and concise manner a thorough knowledge and
understanding of the subject matter and project. The proposer provides
insight into its expertise, knowledge, and understanding of the subject
||Response provides useful information,
while showing experience and knowledge within the category. Response
demonstrates above average knowledge and ability with no apparent
||Response meets all requirements in an
adequate manner. Response demonstrates an ability to comply with
guidelines, parameters, and requirements with no additional information
put forth by the proposer.|
||Response meets all requirements; however, it demonstrates marginal expertise or knowledge of the subject matter.|
||Response meets minimum requirements, but does not demonstrate sufficient knowledge of the subject matter.|
|Does not meet/no response
||An unacceptable response that does not
meet the requirements ddescribed in the RFP. Proposer has not
demonstrated knowledge of the subject matter.|
Objective criteria rely on formulas to calculate score, and do not
require independent judgment. Typically, the only criterion of an
evaluation that is objective is the cost evaluation, but in some cases,
other criteria may be able to be objectively scored.A standard
approach to computing the score for objective criteria is to establish a
formula based on aspects of the criteria. For cost, the ratio method
formula is commonly used:Cost Score = (Lowest Cost Proposal / Proposal Cost) x Total PointsThe
following provides an example computation for a cost evaluation using
the ratio method formula and a maximum total point possible of 200:
In situations where cost may include several components, or where the
evaluation committee may need to consider total cost of ownership
including additional costs beyond the cost provided by the proposer, a
more detailed formula may be required to compute the actual cost. The
following provides an example using the procurement of a vehicle as an
example of computing total cost of ownership:
|Proposer 1 - lowest cost proposal at $26,000
||Calculation - (26000/26000)x200|
Points received = 200
|Proposer 2 - next lowest proposal at $28,400
||Calculation - (26000/28400)x200|
Points received = 183
|Proposer 3 - highest cost proposal at $40,000
||Calculation - (26000/40000)x200|
Points received = 130
VEHICLE COST (per Cost Proposal) + (EPA estimated Annual Fuel Cost x 5
years) + (Estimated Annual Maintenance Cost x 5 years) = TOTAL COST OF
OWNERSHIP (TCO)Regardless of the formula used to
calculate cost, the final score assigned to each cost proposal is based
on the ratio of lowest cost proposal to the proposal cost.