Learning platforms rarely own their roster. Students come from the MIS, staff from the directory, timetables from a scheduling package. During term this is stable. In August every one of those changes simultaneously, and the platform has to be right on day one.
Sync is reconciliation, not import
The naive job reads the upstream file and writes rows. It handles additions fine and everything else badly: a student who left is still present, a renamed class creates a duplicate, a teacher who changed surname becomes two people.
Treat each run as a three-way comparison — what upstream says now, what we have, and what upstream said last time — and classify every difference before applying any of it:
sync_changes
run_id
entity_kind -- student | staff | cohort | timetable_slot
external_id
change_kind -- create | update | deactivate | conflict
before, after -- jsonb
applied_at -- null until committed
Deletion is the dangerous one
An upstream export that omits a student usually means they left. It can also mean the export was filtered wrong, or truncated, or the query ran mid-transaction.
Never hard-delete on absence. Mark inactive, and refuse the whole run if the deactivation count exceeds a threshold — 5% is a reasonable default. A run proposing to deactivate 400 of 900 students should stop and ask, not proceed.
That single guard prevents the worst incident this integration can have.
Dry run is the feature
The August run should be reviewable before it commits. Produce the change list, show it to a human, let them apply it. Once the shape is trusted, it can run unattended during term — but the first run of the year is the one that needs eyes.
External ids, not names
Match on the upstream identifier, always. Matching on name plus date of birth works until two students share both, and in a large intake they will. Store the external id, index it, and treat a missing one as a conflict for a human rather than guessing.
Idempotent by construction
The same export applied twice must produce no second change. That is what makes a failed run safe to rerun at 6am on the first day of term, which is when you will be doing it.