After you issue an RFP and proposals arrive from several firms, many buyers hit the same wall: “I don’t know how to compare these.” Each firm’s proposal has a different structure and level of detail, and it is not unusual for the quoted amounts to differ by more than a factor of two. This article explains how to build your evaluation axes and how to read an estimate.
Decide your evaluation axes before you issue the RFP
As a fundamental rule, evaluation axes are not something you think about after proposals arrive. If you build them afterward, you end up with “axes tailored to the firm that made a good impression.” If you state the evaluation points in the RFP, vendors can also submit sharper, better-targeted proposals.
An example of standard evaluation axes (adjust the weighting to your objectives):
- Understanding of requirements (25 pts) — Do they correctly understand your operations and problems? Be wary of a proposal that merely parrots the wording of your RFP
- Soundness of the approach (20 pts) — Are the reasons for the technology choices explained? Or is it just a string of buzzwords?
- Team and track record (20 pts) — Who will actually do the hands-on work? Do they have experience with similar projects?
- Price (20 pts) — Evaluate “transparency of the breakdown” rather than the figure itself
- Maintenance and operations (15 pts) — The team, cost, and response hours after delivery. A proposal that leaves this blank will cause trouble later
Items you must always check in an estimate
| Item to check | What to look for |
|---|---|
| Assumptions | The list of “X to be supplied by the client” and “Y not included.” This is where additional costs catch fire |
| Effort breakdown | The split across design, implementation, and testing. An estimate where testing is under 10% carries quality risk |
| Team rates | Who (PM / senior / junior), and how many person-months. A lump-sum estimate showing only a total cannot be compared |
| Acceptance conditions | What counts as “complete.” Left vague, acceptance becomes a dispute |
| Change management | How requirement changes are handled. Are the unit price and procedure written down? |
| Maintenance fee | Monthly amount and scope of coverage. Even if the first year is free, check the figure from year two onward |
Three patterns hidden in a “cheap estimate”
1. They read the scope narrowly
Estimating the ambiguous parts of an RFP with the minimum interpretation lowers the figure. This is the pattern where “that isn’t included” begins after the contract is signed, and the final total often exceeds the higher of the original estimates. Because the ambiguity is partly the RFP’s own responsibility, resolving it through Q&A before contracting is the defense.
2. They lower the rate with a junior-heavy team
Cheap rates show up in the team chart. Always confirm who leads the design, how experienced they are, and what percentage of their time they will commit to this project.
3. A strategic loss-leader bid
The model of taking the first project cheaply and recouping through maintenance and additional development. That is not inherently wrong, but you need to check the maintenance rate and lock-in conditions (can it be handed off to another firm? Are the source code and documentation delivered?) before signing.
The Q&A period is a free skills test
The pre-proposal Q&A is where a vendor’s true ability shows most clearly. A firm that asks high-quality questions is a firm trying to understand your operations. Conversely, treat a firm that asks zero questions and submits a proposal that simply follows the RFP as carrying a high risk of misalignment surfacing after the contract is signed.
Summary | Making things comparable is the buyer’s job
The success of a competitive bid is largely decided before the proposals arrive—in the quality of the RFP and the design of the evaluation axes. At SHANNON, drawing on our sense of market rates as a development firm, we help buyers design evaluation axes, check the validity of estimates, and handle technical Q&A with vendors on the client’s side. When our own firm would be a candidate, we step out of the evaluation, so you can also use us as a neutral second opinion.