Side-by-side evaluation plan
Documentfinal Etowah Lane Equipment Rental · onboarding_plan · final · 2026-06-02
Final version one of an evaluation plan, issued 2 Jun 2026, comparing BB-Demo Enterprise with the group-wide analytics suite on the same exports and twenty-five customer questions, scored by the customer. It carries no price or service terms.
What it says
What this says
Current to 13 Oct 26Final version one of an evaluation plan, issued 2 Jun 2026, comparing BB-Demo Enterprise with the group-wide analytics suite on the same exports and twenty-five customer questions, scored by the customer. It carries no price or service terms.
What it is: an onboarding plan prepared by Hannah Lowe for Etowah Lane Equipment Rental. It sets out how BB-Demo Enterprise is compared with "the group-wide analytics suite it already knows" on the same data and questions.
Commits: both tools get the same three months of exports, and "DeShawn's team scores every answer". BB-Demo does not score its own results, and nothing is tuned after the questions are frozen.
Unusual: an answer that is "confident but wrong is scored lower than one that says it cannot tell". Set-up effort is judged separately from answer quality.
Not settled: the final question wording, the exports and the evaluation dates are not stated. No price, term or contract is set, and Jordan Pike follows up commercial terms only if Etowah Lane wishes to continue.
As found in this document
The document
Body
Side-by-side evaluation plan
Prepared for: Etowah Lane Equipment Rental Prepared by: Hannah Lowe, Solutions Engineer, BB-Demo Customer lead: Carlos Medina, Director of Data and Analytics BB-Demo account owner: Jordan Pike Issued: Tuesday 2 June 2026 Status: Final, version one
Purpose
Etowah Lane Equipment Rental is evaluating the BB-Demo Enterprise tier against the group-wide analytics suite it already knows. The aim is for branch, national-accounts and credit teams to see each contractor account in one view. This plan sets out how the two tools will be compared fairly, on the same data and the same questions, and how the result will be judged.
The short version: the national-accounts team supplies twenty-five questions they genuinely ask, both tools are run on the same three months of exports, and DeShawn's team scores the answers.
Principles
Read the whole document (6,202 characters)
- Same data, same period. Both tools receive the same three months of exports. Neither tool gets extra sources the other does not have.
- Questions come from the business. The questions are the ones the national-accounts team asks in their normal week, not questions chosen to suit either product.
- Scoring is done by the customer. DeShawn's team scores every answer. BB-Demo does not score its own results.
- Answers are written down. Each answer is recorded as given, so it can be re-read and challenged later.
- Nothing is tuned after the questions are frozen. Once the question set is agreed, neither side rewrites a question to improve a result.
Scope
| In scope | Out of scope |
|---|---|
| BB-Demo Enterprise, running in the customer's own Snowflake account | Production rollout across every branch |
| The exports the national-accounts team agrees with Carlos | Sources that are not part of the agreed exports |
| Twenty-five questions from the national-accounts team | Questions added or changed after the set is frozen |
| Scoring by DeShawn's team | Scoring by BB-Demo or by the vendor of the other tool |
Question set
The question set is built by the national-accounts team and held by Carlos. It is expected to cover the themes below. The examples are illustrations of the kind of question, not the final wording.
| Theme | Example of the kind of question |
|---|---|
| Account overview | What is the current picture for this contractor across all the branches it uses? |
| Rental activity | Which contractors have changed how much equipment they rent compared with the same period last year? |
| Credit and payment | Which contractor accounts show a change in payment behaviour that credit should know about? |
| Branch coordination | Which national accounts are being served by several branches, and who is the point of contact in each? |
| Contract and terms | Which accounts are on terms that differ from the standard national-accounts terms? |
| Risk and follow-up | Which accounts need attention from the national-accounts team this month, and why? |
Each question is written once, in plain English, and is put to both tools in exactly the same words.
Method
Phase one: agree the inputs
- Carlos confirms which exports make up the three months of data.
- DeShawn's team confirms the twenty-five questions and freezes the list.
- Hannah confirms how the exports will be loaded so that both tools see identical content.
Phase two: load and check
- The same exports are loaded into both tools.
- Hannah and Carlos check record counts and a small sample of accounts against the source, so that any loading problem is found before scoring starts.
- Any difference in what each tool received is logged and fixed before questions are asked.
Phase three: ask and record
- Each of the twenty-five questions is put to both tools in the same words.
- Each answer is captured as returned, with the sources the tool cites where it gives them.
- Answers are shared with DeShawn's team without saying which tool produced which answer, wherever practical.
Phase four: score and review
- DeShawn's team scores each answer.
- Carlos and Hannah review the scores together, including any answer where the scorers disagreed.
- A short write-up records the result, the questions each tool handled well, and the questions each tool handled badly.
Scoring
Each answer is judged on four things, each on a simple three-level scale agreed with DeShawn's team before the run starts:
| Measure | What the scorers ask |
|---|---|
| Correct | Does the answer match what the team knows to be true from the source? |
| Complete | Does it cover every part of the question, or leave out something that matters? |
| Traceable | Can the team see where the answer came from and check it? |
| Useful | Would a national-accounts manager act on it without redoing the work by hand? |
An answer that is confident but wrong is scored lower than one that says it cannot tell. We want that to be visible in the result.
Roles
| Person | Responsibility |
|---|---|
| Carlos Medina | Owns the question set and the exports; customer lead for the evaluation |
| DeShawn's team | Scores every answer |
| Hannah Lowe | Runs BB-Demo in the customer's Snowflake account, checks the load, and explains any answer on request |
| Jordan Pike | BB-Demo account owner; handles commercial questions once the result is known |
Assumptions and open points
- The exports are available to Carlos's team and can be shared into the evaluation environment without further approvals.
- Both tools are judged on what they return, not on how much set-up effort either needed. Set-up effort is noted separately in the write-up.
- If a question cannot be answered by either tool because the data is not in the exports, it is marked as a data gap and not as a failure of the tool.
What happens after scoring
Once scoring is complete, Carlos and Hannah review the write-up together. If Etowah Lane wishes to continue, Jordan will follow up on commercial terms for the Enterprise tier, and Hannah will cover the security review and connector design. If the result does not favour BB-Demo, the write-up still stands as an honest record of where each tool did better.
Thanks, Hannah
Unusual terms
Current to 9 Oct 26- customer scores all answers
- confident but wrong scored lower than cannot tell
- answers shared without saying which tool produced them where practical