How Much Urgent Work Was Hidden Inside the Pump Documents?
Follow documents through filtering, array expansion and grouping. Keep track of what one row represents so the site total counts visits exactly once.
15 sections · 4 questions · 1 coding challenge
The arrow keys move between sections too.
Document Applications Needed Reporting Operations Too
BOTIE
A pipeline expresses those changes as ordered stages. That makes our maintenance report readable as a sequence of record transformations. It also makes a mistake inspectable: after expanding an array, the rows represent visits, so repeating a pump-level quantity on each row can inflate a later sum.
Sources for this history
The Site Report Needs Visits, but the Records Contain Pumps
The maintenance screen stores each pump with a small list of recent visits. The supervisor asks how many minutes of urgent work occurred at north. A pump can contain both urgent and routine visits. Adding its whole history after finding one urgent visit would overstate the requested work.
We can transform the records in stages. First choose the site. Then expose individual visits, keep urgent ones, and collect their minutes. At each stage we should be able to point to one output row and say what it represents.
The local tools model these explicit transformations. They do not execute a MongoDB aggregation pipeline or reproduce server optimisations.
Step 1 of 2
A qualifying pump can contain non-qualifying visits
North is the selected site.
| Asset / visit | Site | Kind | Minutes |
|---|---|---|---|
| P1 / V1 | north | urgent | 12 |
| P1 / V2 | north | routine | 30 |
| P2 / V3 | north | urgent | 8 |
| P3 / V4 | south | urgent | 99 |
P1 / V1
- Site
- north
- Kind
- urgent
- Minutes
- 12
P1 / V2
- Site
- north
- Kind
- routine
- Minutes
- 30
P2 / V3
- Site
- north
- Kind
- urgent
- Minutes
- 8
P3 / V4
- Site
- south
- Kind
- urgent
- Minutes
- 99
Step 2 of 2
Watch the meaning of one row change
Each stage receives the previous stage’s output.
- 1
Site match
One row per north pump - 2
Array expansion
One row per visit of a north pump - 3
Visit match
One row per urgent visit - 4
Group by site
One total record per represented site
Name the grain after every stage
Each surviving visit contributes once; parent selection alone cannot exclude routine members.
Follow urgent visits into the site total
Filter the site, expand its visits, retain urgent members and sum their minutes by site.
BOTIE
Urgent visit minutes by site
[
{
"key": "north",
"total": 20
}
]Follow the operations
1. Pumps with recent visits
[ { "asset_id": "P1", "site": "north", "visits": [ { "kind": "urgent", "minutes": 12 } ] }, { "asset_id": "P2", "site": "north", "visits": [ { "kind": "urgent", "minutes": 8 } ] } ]2. Requested site
north
3. Keep matching document values
[ { "asset_id": "P1", "site": "north", "visits": [ { "kind": "urgent", "minutes": 12 } ] }, { "asset_id": "P2", "site": "north", "visits": [ { "kind": "urgent", "minutes": 8 } ] } ]4. Make one row per array member
[ { "asset_id": "P1", "site": "north", "visits": { "kind": "urgent", "minutes": 12 } }, { "asset_id": "P2", "site": "north", "visits": { "kind": "urgent", "minutes": 8 } } ]5. Visit kind to count
urgent
6. Keep matching document values
[ { "asset_id": "P1", "site": "north", "visits": { "kind": "urgent", "minutes": 12 } }, { "asset_id": "P2", "site": "north", "visits": { "kind": "urgent", "minutes": 8 } } ]7. Sum values under a group key
[ { "key": "north", "total": 20 } ]8. Result
[ { "key": "north", "total": 20 } ]
The browser runs an explicit model of the mechanism. It does not connect to MongoDB or Redis, simulate their query planners, or establish their concurrency guarantees.
1. Write the report’s population in ordinary words
We want urgent visit minutes for the requested site. The unit being counted is a visit minute. We are not counting pumps, summing every visit on a pump that once needed urgent work, or combining sites.
The fixture has bounded recent visits embedded in pump documents. A production report over a full history might read a separate visit collection instead. The pipeline should follow the actual source representation and retention policy.
The report description tells us which records must enter the final aggregate.
2. Filter parent information before expanding children
Site belongs to the parent pump, so we can reject other sites before touching their visit arrays. This reduces the input to later stages without changing the selected visit population.
In a real MongoDB pipeline, a suitable early $match may use an index and the optimiser may move eligible filters. Our model states the order explicitly so we can inspect correctness first. It does not promise a particular execution plan.
A filter can move earlier only when the fields and meaning it needs are already available.
3. Expand the array and keep its parent identity
An unwind step produces an output for each array member while retaining the parent’s other fields. P1 therefore produces separate rows for its urgent and routine visits. The visit field now contains one member instead of a list.
Our local tool requires an array on every document and emits no rows for an empty array. MongoDB $unwind has additional behaviour and options, including preserving certain empty, null or missing cases. Read those rules before assuming the teaching model covers every stored value.
After expansion, one row represents a visit. Repeated parent fields are context, not additional pumps.
4. Express the transformations as an aggregation pipeline
MongoDB accepts an ordered array of stage objects. The first $match selects site. $unwind exposes visits. The next $match keeps urgent members. $group collects their minutes, and an explicit sort gives the result a presentation order.
A pipeline ordinarily returns computed results without changing the input collection. Stages such as $merge or $out can write results and need their own permissions and deployment decisions; this example uses neither.
The following JSON is the pipeline argument passed to an aggregate operation.
Run in your own environment
Aggregate urgent minutes for north
This is real MongoDB pipeline syntax. Its group key appears as _id in the server result.
Use aggregate with a disposable collection containing the shown pump documents. The browser’s Python solution implements the stated mechanism directly without parsing this JSON.
[{"$match":{"site":"north"}},{"$unwind":"$visits"},{"$match":{"visits.kind":"urgent"}},{"$group":{"_id":"$site","total":{"$sum":"$visits.minutes"}}},{"$sort":{"_id":1}}]The two urgent north visits contribute to the total; routine visits on the same pump are excluded.
Aggregate urgent minutes for north. Documentation checked .
Check yourself
Questions on Parts 1 to 4
Question 1 of 2
Which values reach the group stage?
Use the information below.
North P1 contains urgent 12 and routine 30 minutes; north P2 contains urgent 8. We want urgent minutes only.5. Group only the values belonging to the new grain
After filtering, add visits.minutes under each site key. The routine visit no longer exists in the input to this sum. If no urgent visits remain, this pipeline has no group and returns an empty result. An explicit zero row would require a separate preserved site population or another stated default.
Likewise, if a parent carries an annual budget, summing that budget after unwind repeats it once per visit. Reduce at the correct grain or keep that parent-level branch separate.
The field’s location in a repeated row does not change what it measures.
6. Inspect a complete small Python reduction
The dictionary below accumulates minutes for the surviving visit rows. It is a model of the group stage; its input has already passed site and kind checks. Sorting the keys makes output order deterministic.
The later challenge includes the earlier filtering and array steps, so a correct sum alone will not solve it.
Each input contributes exactly once to its group.
Collect already selected visits
The example starts after filtering to show just the grouping operation.
rows = [{"site":"north","minutes":12},{"site":"north","minutes":8},{"site":"west","minutes":3}]
totals = {}
for row in rows:
totals[row["site"]] = totals.get(row["site"], 0) + row["minutes"]
print(sorted(totals.items()))[('north', 20), ('west', 3)]The result is grouped by site, not by pump. No database or query planner runs in this program.
7. Budget for the expanded intermediate result
A small number of parent documents can contain many array members. Expansion increases rows for later filtering and grouping. Grouping and sorting can also need substantial working memory. Use bounded records, useful early predicates and measured server plans to control the workload.
MongoDB has stage-specific resource behaviour and deployment options. Treat those as implementation details to inspect for your version. A fast run on four invented visits does not establish the capacity of a production report.
Count intermediate records as well as final groups when diagnosing the expensive stage.
8. Use the changed history to expose an early parent match
The first workshop contains only urgent visits, so summing every visit after choosing the site happens to work. The changed fixture inserts a large routine visit into P1. The construction must exclude that member even though P1 still has urgent work.
Then try an empty visit array and a site with no urgent visits. Decide whether the report should contain no group or a preserved zero row before writing a test for it. This lesson deliberately returns no group.
Keep the output contract visible as the pipeline grows. Later stages cannot repair minutes already counted from the wrong population.
Check yourself
Questions on Parts 1 to 8
Question 1 of 2
What result follows?
Use the information below.
No urgent visits survive. The pipeline only groups surviving visit rows and has no preserved site list.Try it with Botie
Build with Botie
Write and run Python