Which Records Does the Technician Need Together?
A pump screen, a recent-reading lookup and a site report ask different questions of the same records. Start with those requests before choosing how to store the data.
13 sections · 4 questions · 1 coding challenge
The arrow keys move between sections too.
A Shopping Cart Had to Survive a Busy Service
BOTIE
Our invented maintenance service is much smaller, but we can ask the same first question: what operation must remain practical? The pump screen supplies an asset ID and wants its visits. Grouping visits under that identity makes the requested association visible. We still have to decide how writes keep the grouping current.
Sources for this history
The Pump Screen Keeps Asking for the Same Visits
Botie already has an asset register and a table of inspections. A technician opens P1 and expects its recent visits. Another screen needs the latest temperature for P1. The weekly report needs totals across every pump at a site. Those requests use the same information in different ways.
Before choosing another database, write down exactly what each request knows and what it must return. That gives us something to judge a design against. A convenient record layout for one screen can make another question harder.
These maintenance records are invented. The operations model a lookup, not a database benchmark.
Step 1 of 2
One record set, different questions
Write the request beside the information it already knows.
| Screen | Known on arrival | Needed result |
|---|---|---|
| Pump history | asset_id P1 | Every visit for P1 |
| Latest reading | site north and asset_id P1 | One current reading |
| Site report | site north and a date range | Totals across many assets |
Pump history
- Known on arrival
- asset_id P1
- Needed result
- Every visit for P1
Latest reading
- Known on arrival
- site north and asset_id P1
- Needed result
- One current reading
Site report
- Known on arrival
- site north and a date range
- Needed result
- Totals across many assets
Step 2 of 2
A repeated asset ID is useful information
These are visits, so a pump may occur more than once.
| Visit | Asset | Minutes |
|---|---|---|
| V1 | P1 | 12 |
| V2 | P2 | 8 |
| V3 | P1 | 5 |
V1
- Asset
- P1
- Minutes
- 12
V2
- Asset
- P2
- Minutes
- 8
V3
- Asset
- P1
- Minutes
- 5
The request determines the useful grouping
The grouping preserves every visit; the key selects the requested history.
Keep every visit under its asset
Build a keyed collection, look up the requested asset, and supply an empty list when it has no visits.
BOTIE
Visits returned to the pump screen
[
{
"asset_id": "P1",
"visit": "V1"
}
]Follow the operations
1. Inspection records
[ { "asset_id": "P1", "visit": "V1" }, { "asset_id": "P2", "visit": "V2" } ]2. Collect records under a key
{ "P1": [ { "asset_id": "P1", "visit": "V1" } ], "P2": [ { "asset_id": "P2", "visit": "V2" } ] }3. Requested asset ID
P1
4. Look up one exact key
{ "found": true, "value": [ { "asset_id": "P1", "visit": "V1" } ] }5. Empty history
[]
6. Use a lookup or its fallback
[ { "asset_id": "P1", "visit": "V1" } ]7. Result
[ { "asset_id": "P1", "visit": "V1" } ]
The browser runs an explicit model of the mechanism. It does not connect to MongoDB or Redis, simulate their query planners, or establish their concurrency guarantees.
1. Describe the read before naming a database
An access pattern is a particular way the application reads or changes data. “Show recent visits for this pump” tells us the lookup identity and the records we need together. Add the required ordering, the largest plausible result and how fresh it must be.
Ask about writes too. A new inspection arrives while somebody is viewing the history. Does the next refresh need to show it immediately? Can the screen tolerate a short delay? Those are application requirements, and a data model cannot choose them for us.
A useful design note names an operation and its constraints. “We need something scalable” leaves the important questions unanswered.
2. Keep the meaning of a record intact
The asset table has one row per asset. The inspection table has one row per visit. If we create a dictionary keyed by asset ID and assign a single visit to each entry, a later visit replaces the earlier one. The code has changed the meaning of the data.
A bucket of visits is different: the asset ID selects a list, and that list retains separate visit identities. This gives the pump screen what it asked for without pretending each pump has only one inspection.
Row grain from SQL still matters when the container is a dictionary or a document.
3. Build a lookup while keeping repeated keys
A Python dictionary maps a key to a value. Here each value is a list. Read a visit, find or create its asset’s list, and append the visit. We perform the grouping once in this demonstration, then retrieve a selected list.
Constructing the grouping scans the input. Keeping an equivalent structure in a real service also costs storage and update work. We have made the read simple by doing work elsewhere.
This dictionary is an inspectable model of grouping, not an implementation of distributed storage.
A repeated pump adds to its list
Use setdefault to create each list only when its key first appears.
visits = [("P1", "V1"), ("P2", "V2"), ("P1", "V3")]
by_asset = {}
for asset, visit in visits:
by_asset.setdefault(asset, []).append(visit)
print(by_asset["P1"])
print(by_asset.get("P9", []))['V1', 'V3']
[]The first lookup retains both visit identities. The second has no bucket and returns the application’s chosen empty result.
Check yourself
Questions on Parts 1 to 3
Question 1 of 2
Which value belongs under key P1?
Use the information below.
Visits are V1 for P1, V2 for P2 and V3 for P1. The screen requests every P1 visit.4. Compare the families by the operations they expose
A key-value store starts with a key and returns its associated value. A document database can inspect fields inside a stored record. Wide-column systems organise records around keys and related columns, often with query patterns tied closely to their partitioning. Graph databases make relationships and traversals central to their interfaces. Products vary within each family.
Relational databases can also store JSON and maintain indexes for targeted lookups. A JSON-shaped request alone does not establish that we need a document database. Compare the integrity rules and operational burden of the actual candidates.
NoSQL names a broad collection of approaches. It does not promise a particular consistency model, transaction boundary or performance result.
5. Account for the copies a faster read creates
Suppose visits remain in the SQL system of record while a service keeps a pump-history document. Now an inspection write may need to reach both representations. A failed second write can leave them disagreeing. We need a rebuildable projection, a reliable change feed, or another explicit update process.
A screen-specific copy is useful when its freshness contract permits that process. The same copy is a poor authority for a decision that requires the newest committed safety status. Say which store owns each fact and how other representations catch up.
Before adding a second representation, write down what happens when only one copy changes.
6. Test the request when the data stops being convenient
The workshop first contains one visit per pump, so assigning a single record might appear sufficient. Then a second visit for the requested pump arrives. Your grouping must retain both. A request for an unseen asset needs an explicit empty response.
Use this failure to challenge a proposed model before measuring speed. Once the answers are correct, investigate the workload with a real database and realistic record sizes. Our in-memory counts cannot predict network latency or server capacity.
The next lesson follows the lookup key itself. Two sites can both call their pump P1.
Check yourself
Questions on Parts 1 to 6
Question 1 of 2
What should we establish before choosing a key-only lookup?
Use the information below.
The input is a site ID and a time range; the result is a total across many pumps.Try it with Botie
Build with Botie
Write and run Python