Field notes

Things we notice at the boundaries.

Short, practical pieces on moving data between systems, teams and organisations.

Spotting the hidden boundaries in your data

Ask one question of every report you rely on: who touched this before me? Then ask the same of them. The answer is rarely a single system. It is usually a system, then a person with a spreadsheet, then an email, then another system. Each of those arrows is a boundary. The ones nobody mentions without prompting, such as "oh, Sam always removes the test accounts first", are the ones most worth writing down.

Why dates break more handoffs than anything else

A date like 04/05/2026 means the fourth of May in Britain and the fifth of April elsewhere. Spreadsheet software may silently convert it when a file is opened, and save it back in a different form. The safest handoffs use year-month-day, state it in the agreement, and are not opened and re-saved by hand between steps. If a human must look, they should look at a copy.

Agreeing a layout with an external partner

Start with what they already produce, not what you would like. Mark the columns you truly need, agree on dates and empty values, and ask how much notice they can give before changing anything. Write it down on one page and send it back for a yes. The page will be more useful than any number of meetings.

How long should staged data stay?

Staging exists to bridge a gap in time between sender and receiver. Once the receiver has collected and confirmed, most staged data has done its job. Keep a short grace period in case a collection fails, and keep run records longer than data. If you need longer retention for your own reasons, decide it with your advisers and we will implement it.

Naming files so flows can find them

  • Include the date in year-month-day form.
  • Use one fixed prefix per flow.
  • Avoid spaces and characters that some systems reject.
  • Never reuse a name for a different layout.

Six words we use precisely

Flow
A repeatable route from a source to a receiver.
Handoff
A boundary where data changes owner or shape.
Transformation
One written step that changes data.
Staging
Temporary hosted storage between steps.
Run
One execution of a flow.
Record
The account of what one run did.