Why Selection Committees Are Often Right for the Wrong Reasons
In a typical decision-making meeting, after three presentations, participants often utter phrases like, "I was impressed by them," or "They seemed the most professional." Sometimes the choice is correct. The problem is there’s no way to know—and no way to explain the decision a year from now, when the project inevitably hits complications.
A scorecard isn't meant to replace judgment. It’s designed to ensure that all bidders are evaluated on the same criteria, that the evidence for each score is documented, and that what the committee deemed important before seeing the presentations remains the guiding factor afterward.
Seven Dimensions of Evaluation
1. Understanding the Problem. Does the proposal address your specific processes and volumes, or is it generic? Did the bidder identify a contradiction or gap in the Request for Proposal (RFP)?
2. Quality of Architectural Proposal. Is there a diagram? Are truth sources defined? Were alternatives considered, and was it explained why they were rejected?
3. The Actual Team. Who is leading the project? How much time are they dedicating? Who is performing the work, and what is the proportion of senior professionals involved in key decision-making stages?
4. Relevant Experience. Not just the number of projects, but their similarity in terms of industry, integration complexity, scale, and regulatory environment.
5. Operating Model and Governance. The frequency of demos, decision management, risk management, and how changes are handled.
6. Knowledge Transfer and Autonomy. Is there an explicit plan that will enable you to maintain the system without their ongoing involvement?
7. Commercials. Normalized price, engagement model, contractual flexibility, and willingness to include protective clauses.
Sample Weights – and How to Adjust Them
| Dimension | New Project in Organization with No Team | Rescue Project | Expansion in Organization with Strong Team |
|---|---|---|---|
| Understanding the Problem | 20% | 25% | 15% |
| Architecture | 20% | 20% | 25% |
| Actual Team | 15% | 20% | 15% |
| Relevant Experience | 10% | 10% | 10% |
| Operating Model & Governance | 10% | 10% | 10% |
| Knowledge Transfer | 10% | 5% | 5% |
| Commercials | 15% | 10% | 20% |
This table illustrates a principle: weights are not fixed but derive from the dominant risk. In an organization without an internal team, knowledge transfer holds more weight. For a project rescue, understanding the situation and the team's identity are more crucial.
Establish these weights before receiving proposals and publish them in your RFP, as detailed in our Salesforce RFP guide.
Scoring Scale with Required Evidence
The problem with a 1–5 scale is that everyone tends to score a 4. The solution is to associate each level with specific evidence:
| Score | Meaning | Required Evidence |
|---|---|---|
| 1 | Not addressed | The topic does not appear in the proposal |
| 2 | Generic | Template text with no specific reference to the organization |
| 3 | Adequate | Correct reference but without depth or alternatives |
| 4 | Good | Specific reference with justification and example |
| 5 | Excellent | Alternatives considered, risk identified, and recommendation against something you requested |
The evidence for a score of 5 is what makes this model useful: a vendor who says, "You shouldn't build this part right now," demonstrates an understanding that cannot be faked.
Disqualification Conditions – Before Scoring
Some factors are not worth weighting because they are grounds for disqualification:
- Refusal to assign ownership of deliverables and documentation to the organization.
- Refusal to name team members and their allocation percentages.
- Non-compliance with mandatory regulatory or information security requirements.
- A proposal that does not adhere to the defined pricing structure, after being given an opportunity to correct it.
- Unwillingness to include a basic exit clause.
Define these upfront. A disqualification made retrospectively always appears to be unfairly targeting a specific bidder.
Scoring Process to Minimize Bias
- Each committee member scores independently before the joint discussion.
- The score is accompanied by a brief comment referencing its source in the proposal.
- Discussion focuses only on significant discrepancies between scorers—that’s where the valuable information lies.
- Pricing is revealed at this stage and not before, if the process allows for it.
- The final score is documented along with its rationale.
The fourth step has the most significant impact. A committee that has seen prices before scoring quality will, almost unconsciously, score quality in alignment with the price.
Illustrative Example: Commercial Security Company
This scenario is hypothetical and illustrative. A selection committee scored four bidders. The proposal that received the highest score for "Relevant Experience" received the lowest score for "Understanding the Problem" because the document it submitted was almost identical to one submitted for another project, including an irrelevant industry name.
During the discussion, an argument was made that the experience compensates for this. The committee referred back to the weights it had set two months prior, where "Understanding the Problem" was given twice the weight of "Relevant Experience." The decision remained unchanged.
What the model prevented here was not necessarily a wrong choice, but a change of the rules after the outcome was already known.
What to Do with the Outcome
The score is not the decision itself. It's a document that facilitates a productive conversation: where are the significant gaps between bidders, what is missing from the leading proposal, and what risks remain open. Often, the most beneficial outcome is a list of contractual conditions, rather than just a choice between providers.
The commercial aspect is normalized separately before scoring, as detailed in our Salesforce proposal comparison guide, and the relationship between cost structure and the commercial score is explained in our Salesforce implementation cost guide.
Integration with the Professional Interview
Scoring based solely on documents has limitations. A crucial complement is a meeting where you ask the vendor open-ended questions and observe their real-time thought process. A comprehensive set of questions is available in our guide on questions to ask before choosing a Salesforce integrator, and general criteria for evaluating a company can be found in our guide to choosing a Salesforce implementation company.
Next Steps
Put your weighted criteria on paper before you read the first proposal, and have each committee member score independently. These two steps, taking a combined hour, improve the quality of the decision more than any additional round of presentations.