Develop Evaluation Strategy

What is an Evaluation Strategy?

An evaluation strategy is the documented process a procuring agency will use to evaluate proposals received in response to an RFP or similar procurement method where factors other than cost are considered. The procuring agency should develop an evaluation strategy in conjunction with the procurement plan.

The goal of an evaluation strategy is to ensure that the evaluation committee conducts an evaluation and selection of a product or service in an impartial, fair, efficient, transparent and non-discriminatory manner. To facilitate this, the procurement team must establish and include objective criteria, and any subjective criteria subject to a particular standardized scoring method that helps provide guidance to evaluation committee members as they apply independent judgment to assess such criteria, for evaluation in the solicitation:

To determine whether a particular proposal is responsive, that is, whether it materially meets the requirements of the solicitation.
To assign scores or ratings to each of the evaluation factors established in the solicitation and responded to by a proposer.
To record all determinations of the evaluation process in writing along with the justification for determination.

An agency should be as specific as possible in documenting its evaluation strategy during the planning phase. To frame the development of its evaluation strategy, an agency should consider the following key elements of the strategy:

Description of planned evaluation process.
Mandatory requirements.
Evaluation criteria important to the procurement (including cost).
Relative importance (in percentages) of each criterion identified.

Unless an agency has delegated procurement authority to conduct a Competitive Sealed Proposal, it must provide this information with a statement of work, specifications, and other checklist items to DAS Procurement Services to coordinate the development of the evaluation strategy.

When to Develop an Evaluation Strategy

An evaluation strategy should be developed during the planning phase after an agency has determined its method of procurement (refer to Determine Procurement Method). The procurement team should develop an evaluation strategy if using a Competitive Sealed Proposal (RFP) method or another related procurement method that evaluates proposals on mandatory and desirable requirements in addition to cost.

Key information from the evaluation strategy, including the criteria and scoring methodology, is described in an RFP. This information must be final and documented in the evaluation strategy prior to completing the solicitation documents.

How to Develop an Evaluation Strategy

During the Plan stage and prior to completing a solicitation document, the procurement team must determine and document a strategy that describes how the evaluation committee will evaluate proposals. The primary purpose of an evaluation strategy is to document the evaluation criteria, the scoring applied to each criterion, and the process for determining the scores for each criterion. This process will also help the procurement professional to develop templates and forms that will guide the proposal submission format.

The organization of an evaluation strategy may vary, but should address the following topics:

The procurement professional identifies the roles and potential members of the evaluation committee, which may include members of the procurement team. Depending on the nature and complexity of the solicitation, the procurement professional may establish a committee that includes both voting and non-voting members. The procurement professional serves as a facilitator and advisor and is not a voting member of the committee.
Responsibilities of evaluation committee members require that they:

Read, sign and submit a conflict of interest form to the procurement professional.

Read the RFP and all addenda.
Rate each proposal independently.
Keep rating confidential during individual scoring.
Do not discuss proposals outside of evaluation committee meetings.
Do not permit personal knowledge and bias towards a firm or product, and evaluate on material provided in, or directly related to, the proposal only.
Score consistently and do not change evaluation standards across proposals.
Are able to attend all meetings and demonstrations.

The criteria for evaluation committee membership may include representation by area of expertise, job function, relationship to the end user organization, or as stakeholder outside of the organization. A best practice is to include a minimum of three agency staff with sufficient qualifications in the procurement’s product or service.

Individuals within the private sector who have subject matter knowledge and expertise in the procurement may participate on an evaluation committee, with or without compensation and as non-voting advisors. A procurement professional should ensure private sector participants do not have a conflict of interest and agree to keep the evaluation and all information they view confidential.

Resource: Log in to OregonBuys (account required, documents located in News, Links to Templates and Tools) to access the approved Evaluation Committee Instructions and Conflict of Interest Certification forms to manage the process of selecting and commissioning an evaluation committee.

The evaluation strategy should describe the specific process for how the committee will evaluate solicitation responses.

The evaluation process typically includes four phases:

Evaluation of mandatory requirements.
Evaluation of proposals.
Evaluation of cost.
Evaluation scoring, ranking and selection.

Depending on the agency’s procurement method and approach during the evaluation and selection processes, the agency may choose to add an additional phase to identify a competitive range.

The following sections provide an overview of each phase of the typical evaluation process.

Evaluation of mandatory requirements

A responsive proposal must satisfy all mandatory requirements of a solicitation. A procuring agency determines whether a proposal is responsive by evaluating each of the mandatory requirements on a pass-fail basis. This evaluation assesses whether a proposal materially conforms to the minimum qualifications and submission requirements and determines whether a proposal moves forward in the evaluation process.

The evaluation strategy should identify all mandatory requirements evaluated in this phase. Developing an administrative review checklist that includes mandatory requirements used in performing this phase of evaluation is a best practice. This assures that the evaluation is consistent across all proposals.

Mandatory requirements can include, but are not limited to the following:

Proposal submitted before opening date.
Proposal submitted in proper format.
Proper number of proposals submitted.
Proposal signed by authorized person.
Submitting required forms and information.
Minimum qualifications.
Minimum experience.
Meets specified methodology.
Meets specified functionality.
Minimum price.

If a proposal fails any of the mandatory requirements, it is considered non-responsive and is not evaluated further. In some cases if the failure is determined to be minor or immaterial in nature, the procuring agency may choose to work with the proposer to resolve the issue and allow the proposal to progress in the evaluation process.

Evaluation of proposal

Committee members evaluate the scored criteria on each proposal that meets mandatory requirements. Each evaluator should use standardized tools and forms to assign scores to each component of the evaluation criteria. Based on findings by the evaluation committee, any proposal found to be incomplete, inconsistent, or inaccurate may be deemed as non-responsive and can be rejected.

In some cases if the failure is determined to be minor or immaterial in nature, the procuring agency may choose to work with the proposer to resolve the issue and allow the proposal to progress in the evaluation process.

Evaluation of cost

The procurement professional will evaluate the cost proposal for each proposal that meets mandatory requirements. Cost proposal scoring is based on the methodology described in the solicitation document. Refer to topic #4, titled "Evaluation criteria weighting and calculations", for the section on objective criteria that includes examples of scoring methods.

At no time should the evaluation committee ever see or be informed of the cost proposals or scores received for proposals under evaluation while performing its evaluation of proposals.

Evaluation scoring, ranking, and selection

After the evaluation committee has evaluated and scored each of the proposals, those scores are combined with the cost evaluation scores computed by the procurement professional to calculate a final score for each proposal. Ranking these scores allows for supplier selection based on best value, typically the highest scoring proposer.

Identification of competitive range

In certain circumstances, a process may require, or the procurement team may determine that it is beneficial to establish a competitive range for proposers, following the scoring process. Both the use of, and the approach that will be used for, the competitive range should be identified as part of the evaluation strategy. In addition, if the procurement professional plans to use a competitive range, the solicitation document must communicate this fact to potential proposers.

A competitive range is the establishment of a threshold for scores that will be acceptable to the agency for continued consideration in the procurement process or for negotiations. Acceptable options for determining a threshold include using a formula or using the outcome of the evaluation. In either case, the goal is to limit the proposals in the award selection pool.

Evaluation criteria are the measure used to assess each proposal in a fair and consistent way. Criteria consist of the factors and sub-factors that reflect the areas of importance to an agency in its selection decision. Using evaluation criteria, the procuring agency can assess the extent to which a proposal is responsive to a solicitation’s specifications or scope of work. The assessment also helps make a sound selection decision.

Evaluation criteria provide committee members with a clear and concise method to identify responsive proposals, allow the procuring agency to rank the proposals and ultimately select the proposal representing best value to the state.

To establish evaluation criteria, the procurement team should begin by making a list of the most important aspects of the product or service required, including cost. From this list, the team should group and then prioritize the factors by category in order of importance. These categories represent the evaluation criteria for the product or service. This process will help to determine the most appropriate weighting for each criteria based on its relative importance. The most important items will naturally be evaluated heavier and have more points available.

There is no right or wrong evaluation criteria for a solicitation. Evaluation criteria should be tailored to each procurement and include only those factors having an impact on the selection. The nature and types of evaluation criteria applied to a procurement are within the broad discretion of the procuring agency.

Evaluation criteria weighting provides a means for the procuring agency to communicate the relative importance of each evaluation criterion to potential proposers. The procuring agency must include the evaluation criteria and the weighting of those criteria in the solicitation document.

The following table provides an example of a solicitation document evaluation criteria weighting:

Evaluation criteria	Points
Technical proposal	400
Qualifications, prior experience and references	400
Cost	200
Total points	1000

There are two types of evaluation criteria – subjective criteria and objective criteria. The evaluation and final score calculation are different for the two criteria types.

Subjective criteria

Subjective criteria are criteria that require an evaluation committee member to make an independent judgment of the proposal to assign a score. Examples of subjective criteria include:

Technical approach.
Management approach.
Qualifications.
Experience.
References.
Project plans.

The procurement team will need to develop a standardized scoring method to provide guidance to evaluation committee members in assessing subjective criteria. The formats used can vary; however, standardized scoring methods typically establish ranked levels with associated scoring for each.

The following tables provides examples of a standardized scoring method:

Rank	Score
Excellent	100-81
Good	80-61
Satisfactory	60-41
Below average	40-21
Unsatisfactory	20-1
Does not meet/no response	0

Rank	Score
Excellent	5
Good	4
Satisfactory	3
Below average	2
Unsatisfactory	1
Does not meet/no response	0

While the standardized scoring method above is sufficient to perform an evaluation, the procurement team will also need to define each ranking to remove ambiguity and promote a uniform approach to the evaluation process.

The table below provides an example of each rank and its definition:

Rank	Definition of rank
Excellent	Response meets all the requirements and has demonstrated in a clear and concise manner a thorough knowledge and understanding of the subject matter and project. The proposer provides insight into its expertise, knowledge, and understanding of the subject matter.
Good	Response provides useful information, while showing experience and knowledge within the category. Response demonstrates above average knowledge and ability with no apparent deficiencies noted.
Satisfactory	Response meets all requirements in an adequate manner. Response demonstrates an ability to comply with guidelines, parameters, and requirements with no additional information put forth by the proposer.
Below average	Response meets all requirements; however, it demonstrates marginal expertise or knowledge of the subject matter.
Unsatisfactory	Response meets minimum requirements, but does not demonstrate sufficient knowledge of the subject matter.
Does not meet/no reponse	An unacceptable response that does not meet the requirements described in the RFP. Proposer has not demonstrated knowledge of the subject matter.

Establishing a standard scoring method promotes objectivity and consistency in assessing subjective criteria. This approach will derive a best value selection for the state and limit the likelihood of potential protests.

Objective criteria

Objective criteria rely on formulas to calculate score, and do not require independent judgment. Typically, the only criterion of an evaluation that is objective is the cost evaluation, but in some cases, other criteria may be able to be objectively scored.

A standard approach to computing the score for objective criteria is to establish a formula based on aspects of the criteria. For cost, the ratio method formula is commonly used:

Cost Score = (Lowest Cost Proposal / Proposal Cost) x Total Points

The following provides an example computation for a cost evaluation using the ratio method formula and a maximum total point possible of 200:

Sample cost	Cost score (lowest cost proposal/proposal cost) x total points
Proposer 1 - lowest cost proposal at $26,000	Calculation - (26000/26000)x200 Points received = 200
Proposer 2 - next lowest proposal at $28,400	Calculation - (26000/28400)x200 Points received = 183
Proposer 3 - highest cost proposal at $40,000	Calculation - (26000/40000)x200 Points received = 130

In situations where cost may include several components, or where the evaluation committee may need to consider total cost of ownership including additional costs beyond the cost provided by the proposer, a more detailed formula may be required to compute the actual cost. The following provides an example using the procurement of a vehicle as an example of computing total cost of ownership:

TOTAL VEHICLE COST (per Cost Proposal) + (EPA estimated Annual Fuel Cost x 5 years) + (Estimated Annual Maintenance Cost x 5 years) = TOTAL COST OF OWNERSHIP (TCO)

Regardless of the formula used to calculate cost, the final score assigned to each cost proposal is based on the ratio of lowest cost proposal to the proposal cost.