Two threads run through this module
The business thread is a legal obligation with a data problem hiding inside it. Reporting is mandatory. A late, missing, or wrong report is a conduct breach that draws fines, censure, and remediation orders. But the obligation is not really discharged by lawyers — it is discharged by getting a hundred or more fields, sourced from dozens of systems, correct at the scale of millions of trades a day. The legal framing is the why; the data framing is the how, and the how is where almost all the effort goes.
The technology thread is a pipeline that is never finished. A trade must be captured, enriched with identifiers, tested for eligibility, assembled into a schema, submitted before a deadline, reconciled against the counterparty, and corrected when it breaks — continuously, for the life of the trade. And the rules themselves are owned by an external party, the regulator, who keeps changing them. So reporting is not a system you build once. It is a standing capability you fund forever. Follow this thread and you understand why banks staff reporting as a permanent run-and-change function rather than a one-off project.
How to read this module
Read it front to back the first time; use the cheat sheet and glossary as references afterwards. The argument is cumulative — Part 1 defines the landscape, Part 2 shows how a report is actually made and reconciled, Part 3 names the actors and the people who do the work, Part 4 opens the systems and data, and Part 5 turns to risk and transformation. Every fictional figure is flagged as illustrative; every real fact is presented as such. The coloured boxes carry specific freight:
• Key insight boxes crystallise the one idea a section is built around.
• IT and data lens boxes translate a concept into systems, data, and integration terms.
• Worked example boxes follow a single running trade — a ten-year euro interest-rate swap — from execution to reconciled report.
• Watch out boxes flag the traps that produce most real-world breaches.
• Going deeper boxes add mechanism for readers who want the full picture.
• In a nutshell boxes recap each part; Check your understanding boxes let you test yourself.
Five things you will be able to do
You will understand regulatory reporting end to end, from a booked trade to a reconciled report
You will be able to trace a single trade through capture, enrichment, eligibility, build, submission, and reconciliation, and explain what can go wrong at each step and why the deadline is the one irreversible moment.
You will be able to distinguish the three regimes and say which one bites when
You will know cold what EMIR, MiFIR RTS 22, and SFTR each cover, who must report under each, where each report goes, and why the same trade can generate several reports to several bodies at once.
You will understand why reporting is a data-sourcing problem before it is a legal one
You will be able to explain why the reporting engine invents nothing, why most of the work is finding and joining fields across siloed systems, and why "completeness and accuracy" is the phrase you will hear every day.
You will be able to read the identifiers that hold the whole system together
You will know what the LEI, UTI, UPI, ISIN, and MIC are, why identifier hygiene is core data-quality work, and why the shared UTI is the coordination problem at the heart of dual-sided reporting.
You will recognise the risk, control, and transformation landscape
You will be able to tell a well-run reporting estate from a fragile one, name the failure modes that produce regulatory findings, and judge honestly where standardisation, controls, and artificial intelligence do and do not change the economics.
1The Reporting Landscape
What regulatory reporting is, why it exists, the three regimes that shape it, the identifiers that hold it together, and the infrastructure that carries it.
Regulators demanded to see the market, and reporting is how banks are made visible
Start with the problem the 2008 crisis exposed: nobody could see the risk building up. When Lehman Brothers failed in September 2008, regulators found they had no map of who was exposed to whom. The most dangerous instruments — over-the-counter derivatives, traded privately between two firms rather than on an exchange — were invisible by construction. A swap booked between two banks left no public trace. Multiply that by hundreds of thousands of contracts across the system, and the authorities were, in a precise sense, blind: they could not answer the one question a crisis demands, which is where does the exposure sit.
The response was to force the market to describe itself. In September 2009, the G20 leaders met in Pittsburgh and agreed a set of commitments that reshaped derivatives markets. One of them was blunt: all over-the-counter derivative contracts should be reported to trade repositories. From that political commitment flowed the European rulebooks this module is about. The idea then widened beyond derivatives — if the point is to see and police the market, then transaction data across other instruments should flow to regulators too, and financing transactions that move collateral around the system should be visible as well. Reporting became the mechanism by which a private market is made legible to the state.
Regulatory reporting is the obligation to send regulators structured data about the trades you do. That is the one-sentence definition, and it is worth reading slowly. Structured means a fixed set of fields in a fixed format, not a narrative. The trades you do means the population is defined by rules, not by choice. Send to regulators means there is a destination and a deadline. Put together, reporting is the machinery that lets an authority reconstruct the market from the outside — who traded what, with whom, when, at what price, and on what terms.
Three questions every reporting regime is built to answer
Each regime exists to answer one of three supervisory questions, and knowing which is which tells you how each regime behaves. The first question is systemic: who holds what exposure, and to whom? This is the post-Lehman question, and it is why derivative and financing exposures must be reported — so an authority can aggregate positions and spot a dangerous concentration before it detonates. The second question is about conduct: is anyone abusing the market? This is why transaction data must reach regulators — so they can run surveillance for insider dealing and manipulation. The third question is about fairness: is the market transparent? That last one is answered by a related but separate mechanism, price and volume publication, which we distinguish carefully below.
These three questions are not academic — they determine the shape of each report. A regime built to see systemic risk cares about exposures, valuations, and collateral, so it demands those fields and insists both sides report. A regime built for surveillance cares about who decided and who executed, down to the individual trader or algorithm, so it demands identity fields and microsecond timestamps. Keep the purpose in view and the strange, specific demands of each regime stop looking arbitrary.
What these trades actually are, built from zero
Before you can report a derivative, a financial instrument, or a securities-financing transaction, you have to know what each one is — so we build them from nothing. The three regimes are defined by the kind of trade they capture, so the whole edifice rests on three product categories. A reader who has never priced one should still be able to follow, so here is each from first principles, defined once and then used freely.
A derivative is a contract whose value derives from something else — a rate, a price, an index — rather than from owning the thing itself. Take our running example, an interest-rate swap. Two parties agree to exchange interest payments on a notional amount for ten years: one pays a fixed rate, the other pays a floating rate that resets periodically. Nobody lends the notional; it is just the number the payments are calculated on. The swap has value because interest rates move — if rates rise, the party paying fixed is better off, and the contract is worth something to them. That is the essence of a derivative: a bet or a hedge on how a reference value will move, packaged as a contract between two parties. Derivatives come in a few families — swaps, forwards, futures, and options — but they share this one feature: their value is derived, not intrinsic.
The crucial split for reporting is between over-the-counter and exchange-traded derivatives. An exchange-traded derivative is standardised and bought or sold on an exchange, like a share — the exchange sees every trade. An over-the-counter, or OTC, derivative is negotiated privately between two parties, tailored to their needs, and traded off any exchange. Our ten-year swap is OTC: its exact terms suit these two dealers and exist nowhere else. This is precisely the instrument that made regulators blind in 2008, because an OTC trade leaves no public trace. EMIR's whole existence is a response to that blindness — it forces every derivative, OTC and exchange-traded alike, into a report so the private trades become visible.
A financial instrument, in MiFIR's sense, is the broad category of tradable things — shares, bonds, and derivatives among them. Where EMIR carves out derivatives specifically, MiFIR reaches across the instrument universe. When you buy a thousand shares of a listed company, that is a transaction in a financial instrument, and MiFIR wants it reported so a surveillance system can watch for insider dealing or manipulation. The reach is defined by whether the instrument is traded on a venue, a concept we return to in Part 2, because deciding what counts is one of the regime's hardest judgement calls.
A securities-financing transaction is a trade where securities are lent, borrowed, or pledged to raise or deploy short-term financing. The cleanest example is a repurchase agreement — a repo. One party sells a security now and agrees to buy it back later at a slightly higher price; economically, it has borrowed cash and posted the security as collateral, and the price difference is the interest. Securities lending is the mirror image — lending a security to someone who needs it, against collateral. These trades move enormous quantities of collateral around the financial system every day, quietly, which is why the 2008 aftermath treated them as a systemic blind spot and why SFTR exists to make them visible. The reporting difficulty follows directly from the economics: because these trades are about collateral, the collateral itself must be reported in detail.
Reporting sits at the end of the chain but depends on the very beginning
Reporting is downstream of everything, which is the single most important structural fact about it. A trade is executed and booked. It is enriched with identifiers and reference data. A reporting engine assembles, validates, and formats the report. The report is submitted to a repository or regulator. The repository reconciles it against the other side. Each step depends on the one before, and reporting is last. That position has a brutal consequence: the reporting engine does not invent data — it assembles what upstream systems already hold. If the booking is wrong, the counterparty identifier is stale, or a trade key is missing, the report is wrong too. Nothing downstream can fix what was broken upstream.
This is why "garbage in, garbage out" is the whole problem, not a slogan. Most reporting effort is not spent formatting messages. It is spent sourcing data — chasing a hundred or more fields across front-, middle-, and back-office systems, none of which was built to feed reporting. The reporting layer must reach into all of them and join their data reliably, trade after trade, day after day. A data error at capture becomes a reporting error, and a reporting error becomes a regulatory breach. The chain is unforgiving because it is a chain: the weakest link sets the quality of the whole.
Think of the reporting stack as a pipeline with a source, a series of transforms, and a sink. The source is the bank's transactional systems — booking platforms, reference-data stores, collateral systems. The transforms are enrichment (adding identifiers), eligibility (deciding what is reportable), and formatting (building the schema). The sink is the trade repository or regulator. In pipeline terms, reporting is a classic extract-transform-load problem with a hard service-level agreement: the load must complete by the next working day, every field must validate, and the whole thing must be auditable years later. The difficulty is never the sink. It is joining messy data from systems that key the same counterparty or instrument differently.
Two report families that are easy to confuse and must be kept apart
There are two distinct things a bank sends about its trades, and they travel down different pipes. The first is transaction and trade reporting: per-trade records sent after execution to a repository or regulator, describing what was traded. EMIR, MiFIR RTS 22, and SFTR are all of this kind, and they are the subject of this module. The second is trade transparency: near-real-time publication of prices and volumes to the market, so that other participants can see what is trading. Transparency is a MiFIR obligation too, handled through a different piece of infrastructure — an Approved Publication Arrangement — and its purpose is public price formation, not private supervision.
The distinction matters because the two are confused constantly, including by people who work near them. Transaction reporting is a private data feed to an authority, on a T+1 clock, for surveillance and systemic oversight. Transparency is a public broadcast to the market, in near-real-time, for price discovery. Same underlying trade, two entirely separate obligations, two separate pipes, two separate destinations. This module is about the first. Whenever a colleague says "reporting," it is worth a half-second check of which pipe they mean.
Three regimes, one bank, and the same trade reported several ways
A single bank runs three overlapping European reporting regimes at once, and the overlap is the source of much of the cost. EMIR governs derivatives. MiFIR governs transactions in financial instruments generally. SFTR governs securities financing. They were written as separate legal texts, with separate technical standards, at different times, for different supervisory purposes. They share identifiers and a common deadline, but they differ on who reports, where the report goes, and what it contains. The consequence is that one economic event can trigger several reports. A single interest-rate swap between two banks generates an EMIR report from each side and a MiFIR transaction report — three records for one trade.
The three regimes are best learned side by side, because their differences are exactly what you must keep straight. The table below is the one a business analyst keeps on the wall. Read down each column and the personality of each regime emerges: EMIR and SFTR are dual-sided, so both parties report and their reports must reconcile at a repository; MiFIR is single-sided, so only the executing firm reports, to its regulator, for surveillance.
| Dimension | EMIR | MiFIR / MiFID II (RTS 22) | SFTR |
|---|---|---|---|
| What it covers | OTC and exchange-traded derivatives | Transactions in financial instruments (equities, bonds, derivatives...) | Securities financing: repo, sec-lending, buy/sell-back, margin lending |
| Who reports | Both counterparties (dual-sided) | The investment firm that executed (single-sided) | Both counterparties (dual-sided) |
| Reports to | A Trade Repository (TR) | The national regulator (NCA), via an ARM or direct | A Trade Repository (TR) |
| Deadline | T+1 (the working day after) | T+1 | T+1 |
| Main purpose | Systemic risk and exposure view | Market-abuse surveillance and integrity | Shadow-banking and collateral visibility |
| Roughly how many fields | ~200 after REFIT | ~65 | ~150 |
Read the table as a set of consequences, not a set of facts. Because EMIR and SFTR are dual-sided, your report must agree with a counterparty's report you do not control — which makes reconciliation the hard part of those regimes. Because MiFIR is single-sided and surveillance-driven, its distinctive difficulty is identifying the people and algorithms behind a trade, not agreeing with anyone. Because all three run on a T+1 clock, none of them can be a leisurely batch job. And because the same trade can fall under more than one regime, a bank needs one place that decides which regimes apply to each event, or it will both over-report and under-report at once.
EMIR reports every derivative, from both sides, for its whole life
EMIR is the European Market Infrastructure Regulation, and its reporting arm requires that every derivative contract be reported to a trade repository. That covers OTC derivatives — swaps, forwards, options traded privately — and exchange-traded derivatives too. Four features define how EMIR behaves. It is dual-sided: both counterparties report their own side, and the two reports are then paired and matched at the repository. It is lifecycle-based: not just the new trade, but every modification, valuation, collateral movement, and termination must be reported, sometimes for years. It is keyed on a shared identifier, the Unique Trade Identifier, so the repository can link the two sides. And it runs on a T+1 deadline.
EMIR is distinctively hard because two independent firms must produce data that agrees. Dual-sided reporting means your data must match the counterparty\'s; where they differ, the repository records a reconciliation break. Lifecycle reporting means the obligation never closes on a live contract — you report events on it for as long as it lives. Valuation and collateral reporting means a heavy, recurring data feed: daily mark-to-market and collateral figures for many trades. And the shared trade key must be generated by one side and transmitted to the other in time for both to report — a coordination problem we return to in Part 2. EMIR is the reference regime for this module because it exhibits every difficulty at once.
EMIR is three obligations, and reporting is only one
A common and costly misconception is that EMIR means reporting; in truth reporting is one of three obligations EMIR imposes, and keeping them distinct clarifies the whole regime. EMIR rests on three pillars. The clearing obligation requires that standardised OTC derivatives above certain thresholds be cleared through a central counterparty — a clearing house that steps between the two sides and guarantees the trade, mutualising the risk. The risk-mitigation obligation covers derivatives that are not centrally cleared: they must be margined with collateral, confirmed promptly, and reconciled between the parties, so that even a bilateral trade carries protection. And the reporting obligation, the subject of this module, requires every derivative to be reported to a trade repository. The three pillars interlock — what is cleared, how it is margined, and how it is reported all derive from the same trade — but they are separate legal duties owned by different teams.
Who must report under EMIR depends on what kind of counterparty you are
EMIR sorts every entity into categories, and the category decides the weight of the obligation — so classification is the first real decision, not a formality. A financial counterparty, or FC, is a regulated financial firm: a bank, an investment firm, an insurer, a fund. A non-financial counterparty, or NFC, is everyone else — typically a corporate using derivatives to hedge its business. NFCs are split again by whether their speculative derivative activity crosses a clearing threshold: above it they are NFC+, treated much like an FC with clearing and heavier duties; below it they are NFC-, with lighter obligations. The classification determines who must clear, who must margin, and — the part that concerns us — who carries the reporting burden for a given trade.
EMIR REFIT went further and made the stronger party responsible by law, which quietly enlarged the bank's obligation. Since REFIT, a financial counterparty is solely responsible and legally liable for reporting on behalf of an NFC- counterparty: the corporate no longer reports its own side, and the bank reports both. This is mandatory delegation — imposed by the rule, not chosen commercially — and it is different in kind from the voluntary delegated reporting we meet in Part 3. For a CIB desk facing corporates, the consequence is concrete and uncomfortable: your reporting population now includes trades where you are accountable for data you must source from a counterparty who no longer has any legal obligation to help you get it right. Classification, in other words, is not paperwork. It decides whose problem the data quality is, and REFIT made more of it yours.
MiFIR RTS 22 feeds surveillance, and cares about people, not just trades
Under MiFID II and MiFIR, investment firms must report transactions in financial instruments to their regulator, and RTS 22 is the technical standard that defines the fields. It is single-sided: the executing investment firm reports, not both parties. It goes to the national competent authority, usually through an Approved Reporting Mechanism. It carries roughly sixty-five fields — buyer and seller, the investment decision-maker, the trader or algorithm, price, quantity, venue, and timestamps. And its purpose is market-abuse surveillance: detecting insider dealing and manipulation.
What makes RTS 22 special is that it is about actors, not only trades. To trace abuse to a person, the report must identify the investment decision-maker and the executor — including, where an algorithm decided or executed, the algorithm itself. Those identities are captured as short codes that must map to real identities, which is a common data-quality trap. Deciding what is even reportable is itself subtle and error-prone: the scope turns on concepts like whether an instrument is traded on a venue, and getting eligibility wrong produces both over-reporting and under-reporting. And because surveillance depends on reconstructing sequence, timestamps must be synchronised and reported to the microsecond. RTS 22 is less about agreeing with a counterparty and more about precision on who did what, when.
SFTR reports securities financing, and drowns in collateral data
SFTR is the Securities Financing Transactions Regulation, and it covers the trades where securities are lent, borrowed, or pledged as collateral for financing. That means repo and reverse repo — sell a security now, buy it back later — securities and commodities lending, buy/sell-back and sell/buy-back arrangements, and margin lending in prime brokerage. Like EMIR, SFTR is dual-sided, reports to a trade repository on a T+1 clock, is keyed on a shared trade identifier, and demands heavy collateral reporting.
SFTR is its own beast because these trades are fundamentally about collateral, so the collateral itself must be reported in detail. You report the collateral posted, its valuation, the haircut applied, and whether it can be reused — a large extra data set that the other regimes do not demand at the same depth. It has the richest field set of the three, around a hundred and fifty fields, split across counterparty, loan, collateral, and margin data. Collateral is often allocated after the trade on a net or pool basis, so reporting has to handle later updates rather than a clean one-shot record. And the data lives in many silos — repo desks, securities-lending desks, triparty agents, and prime-brokerage systems each hold pieces — so sourcing is unusually fragmented. SFTR is where the data-sourcing problem is at its most acute.
Why three regimes and not one
A newcomer's first fair question is why Europe built three overlapping regimes instead of one clean reporting rule, and the answer is that each was written for a different purpose at a different time. EMIR came first, as the direct European implementation of the G20's derivatives commitment, so it is shaped entirely around systemic risk in derivatives — hence dual-sided reporting to a repository and heavy valuation and collateral data. MiFIR's transaction reporting came from a different lineage: the markets-in-financial-instruments framework, whose purpose is market conduct and surveillance, so it is single-sided, regulator-facing, and obsessed with who decided and who executed. SFTR came later still, closing the securities-financing blind spot that the first two regimes did not cover. Three purposes, three lineages, three rulebooks.
The overlap is a feature of how regulation accretes, not a design anyone chose, and it is exactly why a bank needs one place that decides which regimes apply. Nobody sat down and designed a system in which one interest-rate swap produces an EMIR report from each side and a MiFIR report as well. It emerged because the derivatives rule and the transaction-reporting rule were written by different processes for different reasons and happen to catch the same trade. This is why the eligibility engine in Part 4 is so important: without a single brain that reads each event and decides its full set of obligations, a bank confronts three rulebooks trade by trade and inevitably gets some combinations wrong. The three regimes are the reason eligibility is hard, and eligibility is the reason the overlap is manageable.
It is tempting to imagine the regimes will eventually merge into one. They will not, at least not soon, and the reason is instructive. Each regime is embedded in its own legal text, its own supervisory purpose, and now its own EU-versus-UK version, so unifying them would require reopening several pieces of primary law simultaneously across two jurisdictions. What is happening instead is quieter convergence at the data layer: ISO 20022 as a common message standard, and globally harmonised Critical Data Elements so that a notional or a timestamp means the same thing in every regime. The rulebooks stay separate; the data underneath them slowly speaks one language. For a technologist, that is the more useful kind of convergence anyway, because it is the data model, not the legal text, that the pipeline actually implements.
The same trade can be reportable in a dozen jurisdictions at once
Europe's three regimes are one corner of a global map, and a bank that trades across borders reports under many rulebooks for the same activity. The G20 commitment was global, so most major jurisdictions built their own derivative and transaction reporting regimes. The table names the ones a CIB desk actually meets. The point is not to memorise each — it is to grasp that "reporting" for a global bank is a portfolio of parallel regimes, each with its own fields, formats, deadlines, and supervisor, all converging slowly on the same global identifiers and the ISO 20022 data model.
| Jurisdiction | Regime | Roughly analogous to |
|---|---|---|
| EU | EMIR, MiFIR, SFTR | (the module's focus) |
| UK | UK EMIR, UK MiFIR, UK SFTR | onshored EU regimes, now diverging |
| US | CFTC (Dodd-Frank) swap reporting; SEC SBSR | EMIR, for derivatives |
| Switzerland | FinfraG | EMIR |
| Australia | ASIC derivative reporting | EMIR |
| Singapore / Hong Kong | MAS / HKMA reporting | EMIR |
| EU (money markets) | MMSR | transaction reporting for money-market instruments |
The global patchwork multiplies everything the module describes. One interest-rate swap between a London desk and a New York desk can require a UK EMIR report and a US CFTC report, and — depending on the venues and entities involved — more. Each is a separate obligation to a separate body in a separate format. The shared global identifiers (LEI, UTI, UPI) and the convergence on ISO 20022 exist precisely to make this patchwork survivable, but the operational reality is that a global bank runs many reporting pipelines in parallel, and cross-jurisdiction consistency of the same trade is its own emerging supervisory concern. This module teaches the EU and UK regimes in depth because the mechanics transfer directly; a professional who masters EMIR reporting can pick up CFTC or ASIC reporting quickly, because it is the same discipline in a different rulebook.
Identifiers are the join keys that make the whole system reconcile
Reports are linked, paired, and matched by identifiers, so getting them wrong means nothing reconciles. Under the surface, a reporting estate is an identifier-management problem. A dual-sided report only pairs if both sides use the same trade key. A report only makes sense if the parties, the instrument, and the venue are named in a way the regulator recognises. The record you would see in a real transaction report is, in large part, a string of codes. Learn the codes and you understand what most of a report actually is.
| Identifier | What it names | Where it bites |
|---|---|---|
| LEI — Legal Entity Identifier | A 20-character global code for a legal entity (a firm, fund, or counterparty) | Must be present and active; a lapsed LEI breaks reports |
| UTI — Unique Trade Identifier | A code identifying one trade, shared by both sides | Someone must generate it and share it in time (EMIR, SFTR) |
| UPI — Unique Product Identifier | A code for the product type of an OTC derivative | Introduced with EMIR REFIT; standardises "what kind of trade this is" |
| ISIN — International Securities Identification Number | A 12-character code for a specific instrument | The "what" for on-venue instruments; sourced from static data |
| MIC — Market Identifier Code | A 4-character code for a trading venue | Says where the trade happened; off-venue uses XOFF or XXXX |
| Short / long codes | Firm-internal codes for traders, clients, and algorithms | Must map to real identities; a frequent data-quality failure point |
The identifiers divide neatly into who, what, and where, and each answers a different question in the report. The LEI answers who is a party. The ISIN and UPI answer what was traded — the ISIN for an on-venue instrument, the UPI for the product type of an OTC derivative. The MIC answers where. And the UTI is the special one: it does not describe the trade at all, it links the two sides of it, which is why it is a coordination problem rather than a lookup. Short and long codes sit underneath, mapping a firm's internal names for its own people and algorithms to the real-world identities a regulator can act on.
Reports do not go straight to the regulator — they flow through specialised infrastructure
Between the bank and the authority sit two kinds of regulated intermediary: trade repositories and approved reporting mechanisms. A trade repository is a regulated central store for derivative and financing reports. It receives reports from both counterparties, validates them against the schema and rules with an accept-or-reject verdict, pairs and matches the two sides using the shared trade key, publishes break statistics, and gives regulators access to the aggregated data. A bank chooses a repository and connects to it. The dominant names are DTCC's Global Trade Repository, REGIS-TR, and UnaVista.
An approved reporting mechanism, or ARM, plays the equivalent role for MiFIR transaction reports. It takes the firm's report, validates it, forwards it to the national regulator, and handles the acknowledgements and rejection feedback that come back. Like a repository, it is itself authorised and supervised. Some large firms report directly to their regulator, but many route through an ARM. There is a third piece of infrastructure worth naming so you never confuse it: an Approved Publication Arrangement, or APA, which publishes the transparency data — the public price-and-volume broadcast — down that separate pipe we distinguished earlier.
Connecting to this infrastructure is itself a decision with cost and control implications, not a mere plug-in. A firm must choose which trade repository to use for EMIR and SFTR, and whether to reach its regulator directly or through an ARM for MiFIR. Each connection is a piece of engineering — a certified interface, a message format, a testing regime — that must be built and then maintained as the infrastructure's own rules evolve. A firm active across the European Union and United Kingdom typically needs connections to infrastructure in both, doubling the connectivity burden. And the choice is not purely technical: a repository's reconciliation tooling, its break reporting, and its fees all differ, so selecting one is partly an operations decision about how easy it will be to work breaks day to day. The plumbing has to be chosen, built, paid for, and kept current, which is why connectivity is a standing line in every reporting budget.
The infrastructure exists because reconciliation and validation are jobs no single bank should do alone. A repository can pair two banks' reports precisely because both banks send to it; neither could reconcile with the other directly at scale. Validation against a common schema is more consistent when a central utility does it. And regulators would rather supervise a handful of repositories and ARMs than connect to every bank individually. The plumbing, in other words, is not overhead — it is what makes dual-sided reporting and centralised surveillance possible in the first place.
The rules never stop moving, which makes reporting a permanent programme
Regulatory reporting has a cycle, and it is a regulatory cycle rather than a market one. Other CIB businesses are cyclical because markets rise and fall. Reporting is cyclical because the rules are rewritten on the regulator's clock. The arc so far runs from the post-2008 mandate, through the first generation of European rules in the mid-2010s, to a major rewrite that landed in 2024, and onward to a widening split between the European Union and the United Kingdom. Each turn of the cycle is a full delivery programme: remap fields, upgrade schemas, re-test end to end, migrate open trades.
The arc from crisis to divergence is worth seeing laid out, because it shows a discipline defined by external events rather than internal choices. Each date below was set by a regulator or a political event, not by any bank, and each one triggered a delivery programme across the whole industry. This is the visible shape of the regulatory cycle.
| When | Event | What it did to reporting |
|---|---|---|
| 2008 | Financial crisis | Exposed that regulators could not see OTC derivative exposures |
| 2009 | G20 Pittsburgh | Committed to reporting all OTC derivatives to trade repositories |
| ~2014 | EMIR reporting go-live | First mandatory derivative reporting in the EU |
| 2018 | MiFID II / MiFIR and SFTR | Transaction reporting and securities-financing reporting begin |
| 2020 | Brexit transition ends | UK onshores EMIR, MiFIR, SFTR; two rulebooks begin |
| 2024 | EMIR REFIT | ISO 20022 XML, ~200 fields, mandatory UPI; EU and UK staggered |
| Ongoing | MiFIR review, EU/UK divergence | The next migrations already in the pipeline |
The 2024 rewrite of EMIR — known as REFIT — was the biggest reporting change in a decade, and it points where the whole estate is heading. It mandated a single structured message format, ISO 20022 XML, in place of older CSV and proprietary formats. It expanded the field set to around two hundred, added the Unique Product Identifier, and tightened validation and global data harmonisation. Every firm had to remap fields, rebuild message generation, re-test, and migrate its live trades to the new format on a hard cut-over date. We treat REFIT in depth in Part 2; here it matters as the clearest evidence that reporting requirements are owned externally and revised continuously.
Why did the 2008 crisis lead to mandatory reporting, and what was the G20's specific response?
Regulators were blind to risk building up in opaque OTC derivatives; the G20 (Pittsburgh, 2009) agreed that all OTC derivatives be reported to trade repositories, and reporting later widened across markets.
Which of the three regimes are dual-sided and which is single-sided, and why does that distinction matter?
EMIR and SFTR are dual-sided (both parties report, so reports must reconcile at a repository); MiFIR RTS 22 is single-sided (only the executing firm reports, to its regulator for surveillance). Dual-sided regimes make reconciliation the hard part; single-sided reporting makes identifying actors the hard part.
EMIR is often equated with reporting — why is that wrong, and what are its three pillars?
Reporting is only one pillar. EMIR also imposes a clearing obligation (standardised OTC derivatives cleared through a CCP) and risk-mitigation duties for uncleared derivatives (margin, confirmation, portfolio reconciliation). The three are separate legal duties owned by different teams.
How does counterparty classification decide who reports under EMIR, and what did REFIT change?
Entities are FC (financial), NFC+ (non-financial above the clearing threshold), or NFC- (below it). REFIT made the financial counterparty solely responsible and legally liable for reporting on behalf of NFC- counterparties — mandatory delegation, which enlarges the bank's obligation and data-quality risk.
What does it mean that reporting sits at the end of the trade chain?
The reporting engine invents no data; it assembles what upstream systems already hold. So an error at capture becomes a reporting error and then a breach — reporting is a data-sourcing problem before a legal one.
Why is the UTI different in kind from the other identifiers?
The LEI, ISIN, UPI, and MIC describe a trade (who, what, where); the UTI links the two sides of a dual-sided report. It is a coordination problem — one side must generate it and share it in time — not a lookup.
Why is EMIR only one corner of a global map?
The G20 commitment was global, so most jurisdictions built their own regimes (CFTC/Dodd-Frank in the US, ASIC in Australia, FinfraG in Switzerland, MAS in Singapore, plus MMSR in the EU). A global bank reports the same trade under several parallel rulebooks — the mechanics transfer, but the operational burden multiplies.
1
Which regime is single-sided?
Right. EMIR and SFTR are dual-sided and must reconcile.
Dual-sided.
No.
Dual-sided.
What are EMIR's three pillars?
Right.
No.
No.
A common mistake.
What did the G20 decide in Pittsburgh in 2009?
Right.
No.
No.
No.
2How a Report Is Actually Made
The reporting lifecycle end to end: from a booked trade to a reconciled report. Where the data comes from, how the report is built and validated, why it gets rejected, how the two sides are matched, and what happens when it all goes wrong.
Reporting is a lifecycle, not a single send
A report is not a one-off act — it is a stream of events, each of which must be reported and confirmed. People new to reporting picture it as a moment: the trade is done, you send a message, you are finished. That picture is wrong in a way that causes most of the operational pain. A single trade sets off a sequence — the event occurs, it is checked for eligibility and enriched, the report is built and validated, it is submitted before the deadline, rejections are repaired, the two sides are reconciled, and every later event on the trade is reported until it terminates. The obligation is continuous, event-driven, and unforgiving of the deadline. This is why reporting needs a controlled, monitored pipeline, not a batch job someone runs when they remember.
The six stages below are the spine of this part, and each has its own failure modes. The overview table gives the shape; the sections that follow open each stage, and along the way take the deep dives that separate someone who has heard of reporting from someone who can run it.
| Stage | What happens | Timing | Who leads |
|---|---|---|---|
| 1. Execution and capture | The trade is agreed and booked; the source record is created | Trade time | Front office, trade capture |
| 2. Eligibility and enrichment | Decide it is reportable and under which regime; add identifiers and reference data | Same day | Data engineering, reference data |
| 3. Build and validate | Assemble the fields, run internal checks, format to the schema | Same day | Reporting engine, ops |
| 4. Submit to TR / ARM | Send by T+1; receive acknowledgement or rejection | By T+1 | Operations |
| 5. Pair, match, reconcile | The repository pairs the two sides and matches fields; breaks surface | T+1 / T+2 onward | Operations, reconciliation |
| 6. Repair and back-report | Fix rejections and breaks; correct history when a defect is found | Ongoing | Ops, data, business analysis |
The effort concentrates around the deadline and the reconciliation that follows
The pressure in reporting is not evenly spread — it piles up at submission and reconciliation. Capture and enrichment are mostly automated and quiet. But the T+1 deadline is a wall, and everything must clear it: eligibility decided, fields sourced, report built, validation passed, submission acknowledged. A rejection discovered at the deadline is a scramble, because a rejected report is an unmet obligation until it is repaired. Then, after the deadline, the second wave arrives — reconciliation breaks, where a report that was accepted still fails to agree with the counterparty's. The intensity curve below peaks exactly there, at the deadline and the reconciliation immediately after it, because those are the moments with no second chance and the most parties involved.
Stage 1 — Capture is where every reportable fact is born
The trade is executed and booked, and that booking is the source of every fact the report will ever contain. When our two dealers agree the swap, front-office capture creates the source record: the economics — notional, fixed rate, effective and maturity dates, the floating index — and the parties. Nothing in the eventual report exists that was not first written here or in a system the report will later reach into. This is the practical meaning of "downstream": the quality ceiling of the report is set at capture. If the desk books the notional in the wrong units, or omits a field that reporting will later need, no amount of downstream cleverness recovers it.
Capture systems were built to book and risk-manage trades, not to feed reporting — and that mismatch is the root of the whole sourcing problem. A booking system cares that the trade's economics are right so the desk's risk and profit-and-loss are right. It does not care whether an algorithm identifier or a decision-maker code was recorded, because those fields have no bearing on risk. So fields that reporting needs are often simply absent at source, or captured inconsistently across desks. You cannot report what was never recorded. A large part of reporting change work is going back into capture systems to make them record fields they were never designed to hold.
Stage 2 — Sourcing the hundred-plus fields is the real work
The core difficulty of reporting is not the format — it is assembling correct data from many disconnected systems. A single report pulls fields from across the whole bank. The economics — price, notional, dates, product — come from the front office and booking. The identifiers — LEIs, ISINs, UPIs, instrument details — come from reference and static data. The counterparty's identity, classification, and decision-maker come from client and know-your-customer systems. Collateral, haircuts, and valuations come from collateral and margin systems. The venue, timestamps, trading capacity, and trader or algorithm codes come from execution systems. None of these was built to feed reporting. The reporting layer must reach into all of them and join their data reliably.
Four specific problems make this the hard part rather than a routine lookup. The systems do not agree with each other: the same counterparty or instrument may be keyed differently in each, so joins fail or mismatch. Fields are missing at source: an algorithm identifier that was never captured cannot be conjured. Timing and latency intrude: some data, especially collateral and valuations, arrives after the trade, so the pipeline must wait for it and still hit T+1. And every field needs one agreed authoritative source — a golden source — because if two systems disagree about a counterparty's LEI, someone must decide which one wins. Deciding and enforcing golden sources is a major, unglamorous business-analysis workstream, and it is where reporting programmes spend much of their time.
The UTI is a coordination problem, solved by a waterfall
In a dual-sided regime, both sides must report the same trade identifier, so exactly one party must generate it and share it in time. The repository pairs the two reports using the shared UTI. If the two sides use different UTIs, the reports never pair — an instant break, before any field is even compared. So one, and only one, party must generate the UTI and get it to the other before both must report by T+1. Left to negotiation, this would produce endless arguments about who owns the key. So the regulators define a waterfall: a fixed priority order that decides, for any given trade, who is responsible for generating the UTI.
The waterfall resolves the "who" but leaves the "how" as a real engineering problem. If a central counterparty clears the trade, the clearing house generates the UTI. Failing that, if it traded on a venue, the venue generates it. Failing that, if it was confirmed on a matching platform, that platform generates it. Only if none of those applies do the counterparties fall back to agreeing it between themselves, with the seller or a determined party generating it. The European Union and United Kingdom both follow the same globally agreed waterfall, so this rule is broadly harmonised. But knowing who must generate the UTI does not build the plumbing to receive it, or to generate and transmit it, before the deadline. That operational sharing — actually getting the key to a counterparty in time, especially a less-automated one — is where firms still struggle.
Stage 3 — Building the report means speaking the regulator's exact language
Once the fields are sourced, the reporting engine assembles them, runs internal validations, and formats the report to the required schema. The engine checks the schema, the mandatory fields, the allowed values, and the business rules before sending — catching errors cheaply, while they are still yours to fix quietly. Then it formats the report into the message standard the regime demands. For EMIR since REFIT, that standard is ISO 20022 XML, a structured, richly typed format that both the repository and the counterparty interpret identically. The format is not cosmetic: a common, machine-validatable schema is the foundation of clean pairing and matching, because both sides and the repository read every field the same way.
EMIR REFIT rewrote the format and enlarged the field set
EMIR REFIT, live in 2024, was the biggest reporting change in a decade, and it standardised the message while expanding what must be reported. It mandated ISO 20022 XML in place of older CSV and proprietary formats — a single, standardised format. It expanded the field set from around a hundred and thirty to roughly two hundred, with new and redefined data points. It made the Unique Product Identifier mandatory, classifying each OTC derivative. And it tightened validation, aligning European data with a global set of Critical Data Elements so that firms everywhere report the same core fields the same way.
REFIT helped matching but cost a fortune to implement, and that tension is the story of every reporting change. On the benefit side, ISO 20022 plus global Critical Data Elements means both sides speak the same language, which improves pairing and matching and reduces breaks. On the cost side, every firm had to remap fields, rebuild message generation, and re-test end to end — a multi-year programme. New identifiers like the UPI had to be wired into enrichment, creating new data dependencies. And existing live trades had to be upgraded to the new format on go-live, a mass migration event with a hard cut-over date and no grace for late or wrong reports. SFTR and, through the MiFIR review, MiFIR are on similar journeys toward ISO 20022 and global data alignment — the whole estate is slowly converging on one message standard.
The same swap as an EMIR ISO 20022 message, element by element
Post-REFIT, our swap's EMIR report is an ISO 20022 XML message — the derivatives trade report, known by its message name auth.030 — split into a counterparty block and a common block. The table flattens the message into fields, with the one column that matters most for reconciliation: does this element have to match the dealer's independently produced report?
| XML element | Illustrative value | Block — must it match the dealer? |
|---|---|---|
| RptgCtrPty (reporting LEI) | 5493001KJTIIGC8Y1R12 | Counterparty block — yours alone |
| OthrCtrPty (other LEI) | 213800X1YZ8H7K2M4C33 | Counterparty block — yours alone |
| Drctn (direction) | SLLR | Counterparty block — the mirror of the dealer |
| UnqTxIdr (UTI) | E12A93...UTI...0007F4 | Common block — must match exactly |
| UPI | QZ4CP1J2KNRG | Common block — must match exactly |
| NtnlAmt (notional) | 50,000,000 EUR | Common block — must match |
| Pric (rate) | 3.4750 | Common block — must match |
| FctvDt / XprtnDt (dates) | 2026-07-16 / 2036-07-16 | Common block — must match |
| ExctnTmStmp (execution time) | 2026-07-16T09:32:14Z | Common block — must match |
| Vltn (valuation) | -124,500 EUR | Common block — matched within tolerance |
| Coll (collateral status) | UNCL / FULL | Common block — must match |
The two blocks are the visual proof of why reconciliation is the hard part of EMIR. The counterparty block is yours alone — you can never disagree with the dealer about it, because they never see it. The common block must agree, element for element, with a report the dealer produced independently from their own systems. Every value in that common block is a place the two sides can diverge, and each divergence beyond tolerance is a break. The richly typed, machine-validated ISO 20022 schema exists precisely so that both firms and the repository read each of these elements identically; before REFIT, looser formats let the same economic value be represented two ways, which seeded breaks the schema now prevents.
MiFIR transaction reporting hides its difficulty in judgement calls
RTS 22 looks like a form to fill in, but its real difficulty is a series of judgement calls where errors hide. The scope question comes first and is genuinely hard: a transaction is reportable if the instrument is admitted to trading or traded on an EU venue, or if its underlying is — the "traded on a venue" concept — plus certain related instruments, with carve-outs for things like some transfers and collateral moves. Deciding whether a given event is reportable at all is subtle, and getting it wrong produces both over-reporting and under-reporting. Then, for chains, allocations, and agency trades, correctly identifying the buyer, the seller, and each party's capacity is genuinely difficult. Reporting the investment decision-maker and the executor — person or algorithm — through short codes is a common failure. And the trading capacity flag — dealing on own account, matched principal, or any other capacity — changes the report and is easy to mislabel.
| Judgement call | The question | Where the error hides |
|---|---|---|
| Eligibility (ToTV) | Is this instrument or activity reportable at all? | Over- and under-reporting both start here |
| Buyer / seller | Who are the parties in a chain or agency trade? | Mis-identified parties in complex flows |
| Decision-maker vs executor | Who decided, who executed (person or algo)? | Short-code mapping failures |
| Trading capacity | Own account, matched principal, or other? | Easy to mislabel; changes the report |
A MiFIR RTS 22 record, read field by field
Here is a single illustrative transaction report — flagged illustrative only — so you can see what the data actually looks like rather than only read about it. Every value is fictional, but the structure is exactly what travels to the regulator.
| Field | Illustrative value | What it is / where it comes from |
|---|---|---|
| Report status | NEWT | New, cancel, or correct — pipeline logic |
| Transaction reference no. | TRN-2026-0000123456 | Unique id for this report — reporting engine |
| Executing entity (LEI) | 5493001KJTIIGC8Y1R12 | Who executed — reference data |
| Buyer (LEI) | 213800X1YZ8H7K2M4C33 | The client — reference / client system |
| Seller | Own account | Our side of the trade |
| Investment decision-maker | FR19740823JEAN#DUPONT | National ID of the decider — short-code map |
| Execution within firm | GBR7429XTRADER014 | Who or what executed — short-code map |
| Instrument (ISIN) | FR0000131104 (BNP Paribas) | What was traded — instrument static |
| Price / currency | 62.480 / EUR | Economics — booking |
| Quantity | 15000 | Economics — booking |
| Venue (MIC) | XPAR (Euronext Paris) | Where — execution system |
| Trading date-time (UTC) | 2026-07-16T09:32:14.482913Z | When, to the microsecond — venue |
| Trading capacity | DEAL (own account) | Regulatory flag — booking logic |
| Short-selling indicator | SELL / SESH | Regulatory flag — booking logic |
Read the record and the module's whole thesis appears in one glance: it is mostly identifiers. LEI, ISIN, MIC, national IDs, short codes — the substance of the report is codes, each sourced from a different system and each needing to be current. The price and quantity are the only real economics; everything else is identity and classification. Get any single code wrong and the report is rejected or, worse, accepted and silently wrong. This is the concrete reason reporting is an identifier-management and data-sourcing problem before it is anything else.
RTS 22 is single-sided, which changes where its risk lives. Because only the executing firm reports, there is no counterparty to reconcile against, so RTS 22 does not generate pairing breaks the way EMIR does. Instead, its risk is concentrated in eligibility and accuracy: reporting the wrong population, or reporting the right trades with a wrong actor or capacity. A regulator running surveillance on a wrongly populated or mislabelled feed draws wrong conclusions, which is exactly the harm the regime exists to prevent. So RTS 22 quality is measured less by "did it pair" and more by "was the right set of trades reported, with the right people attached."
SFTR's difficulty is the collateral dimension and its awkward timing
An SFTR report is organised into four data blocks, and the collateral block is what makes the regime heavy. The counterparty block names the parties, their side, sector, and the broker or agent chain. The loan block describes the securities financing transaction itself — principal, rate, term, and the securities lent or borrowed. The collateral block describes what collateral was posted, its value, its haircut, and whether it can be reused. The margin block, for cleared transactions, covers margin posted and received. That is roughly a hundred and fifty fields, and the collateral and margin blocks are data that the other regimes simply do not demand at the same depth.
| Data block | What it holds | Why it is hard |
|---|---|---|
| Counterparty | Parties, side, sector, broker/agent chain | Chains and agents complicate who is who |
| Loan | Principal, rate, term, securities lent/borrowed | The core economics, usually cleanest |
| Collateral | What was posted, value, haircut, reusability | Allocated late; sourced from other systems |
| Margin | Margin posted and received (cleared SFTs) | Extra feed for cleared transactions |
SFTR is painful because collateral is often assigned after the trade and lives in other people's systems. On net or pool bases, collateral is allocated after execution, so you report an update once it is known — reporting cannot wait for a clean single record. Lifecycle events are frequent: rate changes, collateral substitutions, partial returns, and reuse all trigger reports over the life of the transaction. Much collateral data sits with triparty agents and lending agents, so you must ingest their feeds rather than owning the data yourself. And, being dual-sided like EMIR, both sides must reconcile — with far more fields on which to disagree. SFTR is where the data-sourcing problem, the timing problem, and the reconciliation problem all peak at once.
Stage 4 — The deadline is a wall, and rejection is an unmet obligation
The report is submitted to the repository or ARM by T+1, and what comes back is an acknowledgement or a rejection with error codes. This is the moment the intensity curve peaks, because the deadline is real and a rejection means nothing was reported. The repository or ARM validates the submission against the regulator's rules and returns one of two verdicts. An acknowledgement means the report was accepted into the system. A rejection — a NACK — means it failed validation and, in the eyes of the regulator, does not exist. A rejected report sitting unrepaired past the deadline is a breach in exactly the same way a missing report is.
Rejection repair is a same-day discipline, because the deadline does not move for your error. When a report is rejected, operations must read the error code, investigate, fix the data at source or in the report, and resubmit — ideally still inside the deadline window. This is why reporting operations is a live, monitored function rather than an overnight batch: someone must be watching the acknowledgements and rejections as they come back, ready to repair within hours. A firm that only checks its submissions the next morning has already missed the window on anything that rejected the night before.
The submission layer is a message-passing system with guaranteed delivery and acknowledgement semantics, and it must be monitored like one. Every report sent must be matched to an ACK or a NACK; an unanswered submission is as dangerous as a rejected one, because silence might mean the report never arrived. Mature shops run a dashboard that tracks, in near-real-time, how many reports were sent, how many acknowledged, how many rejected, and how many are still outstanding against the deadline clock. The exception queue — the list of rejections waiting for repair — is the operational heartbeat of a reporting desk. When it ages, the desk is losing.
Rejections come in a few recognisable kinds, and reading the error code is a skill
A rejection is not a single thing — it is one of a handful of error types, and diagnosing which one is the first move in any repair. Schema errors mean the message is malformed — a field in the wrong place, a value in the wrong type — and they point at the reporting engine's formatting. Mandatory-field errors mean a required field is missing, and they point back at sourcing: something was never captured or never joined. Allowed-value errors mean a field holds a value the regulator's rulebook does not permit — a country code that does not exist, a capacity flag that is not on the list — and they usually point at a mapping. Business-rule errors are the subtlest: the message is well formed and every field is individually valid, but the combination violates a rule, such as an expiry date before an effective date. An operator who can read an error code and instantly know whether to send the break to the engine team, the data team, or the mapping owner is worth a great deal, because the deadline is running while the report sits rejected.
Stage 5 — Acceptance is not correctness, and reconciliation proves it
Even an accepted report can fail to agree with the counterparty's, and catching that is what reconciliation does. For dual-sided regimes, the repository does two things in sequence. First it pairs: did both sides' reports find each other at all? They pair on the shared UTI, supported by the LEIs. No pair means one side is missing or used a different key. Then, once paired, it matches: do the key economic fields agree? The repository compares a defined set of matchable fields — notional, price, dates, direction, and more — within tolerances, and returns statistics on what paired, what matched, and which fields differ. Those statistics are the raw material of the breaks queue.
Breaks happen for a small number of recurring reasons, and working them is a daily discipline. The two sides may never have agreed the identifier — a pairing failure whose root is broken UTI sharing. They may have booked the trade slightly differently — a notional convention, a date roll — producing a field-level matching break. Or one side reported late or amended, so the snapshots differ until both catch up. Operations triage breaks by materiality, work out which side is wrong, correct it, and resubmit. This is not a project that ends; it is an ongoing queue that must be worked down faster than it fills up, because ageing breaks are exactly what draws supervisory attention.
Matching happens on a defined set of fields, within tolerances
Not every field has to agree for two reports to match — the repository compares a defined set of matchable fields, some within tolerances — and knowing which fields those are tells you where breaks come from. The economic fields that must agree are the ones that describe the trade's substance: notional, price or rate, effective and maturity dates, direction, and the shared identifiers. Some are matched exactly, because there is one right answer — the UTI either agrees or it does not. Others are matched within a tolerance, because two firms can represent the same economic value slightly differently — a valuation to the nearest euro versus the nearest cent, or a rate to a different number of decimal places. The tolerances exist precisely because perfect agreement between two independent systems is unrealistic, and a regime with no tolerance would drown in trivial breaks.
The tightening of tolerances over time is a deliberate supervisory lever, and it quietly raises the bar every few years. As reporting data matures, regulators narrow the tolerances, so differences that once passed now register as breaks. This is not bureaucratic fussiness; it is how the authority ratchets up data quality without rewriting the rules. A firm that was comfortably inside the tolerances last year can find itself generating breaks this year on exactly the same booking conventions, because the acceptable gap shrank. Reporting teams therefore watch tolerance changes as closely as field changes, because a tolerance that tightens is a new source of work that no rule text announced.
The obligation continues for the life of the trade
A new trade is only the first report — the obligation continues through every event on the trade until it terminates, and each event is its own report. Our swap lives for ten years, and over that decade it generates a stream of reports beyond the original. Each amendment — a change to terms — must be reported. Each daily valuation must be reported for EMIR, a heavy recurring feed. Each collateral movement must be reported. And when the swap is finally terminated or matures, that too is a reportable event. The mental model of "report the trade once" is wrong in a way that produces under-reporting: a firm that captures new trades cleanly but misses lifecycle events has an invisible completeness gap that grows with every live trade on its books.
The three questions a mature reporting shop actually measures
Acceptance, correctness, and completeness are three different things, and most regulatory findings are about the last two. It is tempting to treat a low rejection rate as success. It is not. A report can be accepted and still be wrong — wrong price, wrong flag, wrong party — which validation alone will never catch. And a trade can simply never have produced a report at all, which no acceptance statistic can reveal, because you cannot see the absence of something you never sent. A mature shop therefore measures three separate things, and the diagram below names them.
Completeness is the hardest failure to see, which is why it is the most dangerous. Under-reporting — a reportable trade that produced no report, or a lifecycle event that was missed — means the regulator never saw the trade at all. It is invisible unless you reconcile the population of trades in your books against the reports you produced; the gap is the under-reporting. Over-reporting — a non-reportable trade that was reported, or a trade reported twice — pollutes the regulator's data and is a breach in its own right. Mis-reporting — the report exists, was accepted, but a field is wrong — is silent inaccuracy. Finding what is not there requires reconciling to source, treating eligibility as a tested control, sampling reports and independently re-deriving fields from source, and monitoring for the sudden volume drop or value spike that flags a broken feed before the regulator does.
Stage 6 — When you find an error, you must correct history
Reporting errors are rarely one trade — a broken feed or a wrong rule affects thousands of reports over months before anyone notices. This is the uncomfortable arithmetic of a data pipeline: a defect does not damage one report, it damages every report that flowed through the defect for as long as it was live. So when the notional scaling bug on our swap's product family is found, the question is not "fix this report" but "how many reports, over what period, are wrong?" Remediation then means four things in order: quantify the affected population and period; fix the root cause so it stops recurring; back-report — resubmit corrected or missing reports for the whole affected history; and notify the regulator, because many breaches must be self-reported.
Remediation is judged not on the patch but on the control you add to stop it happening again. Reprocessing months of history through the pipeline is a large, risky data-migration exercise in its own right. Firms are generally expected to notify the regulator of material errors — hiding them makes everything far worse. And supervisors want the cause fixed and the controls strengthened, with evidence, not just the reports patched. Repeated remediation signals weak controls and erodes supervisory trust; a clean, well-run reporting estate is, conversely, a trust asset that buys a firm the benefit of the doubt. The lesson that runs through every reporting incident is the same: a reconciliation break is a detector, and the correct response is trace-to-source, fix the pipeline, back-report the history, and add a control.
Controls turn a pipeline into assurance the regulator will accept
Regulators expect a documented control framework, because reporting is now governed, evidenced, and owned rather than merely performed. Six control layers turn a pipeline into something defensible. Reconciliation controls — report-to-source for completeness, repository pairing and matching for consistency, and accuracy sampling — are run and evidenced regularly, with breaks tracked to closure. Eligibility and rule governance means a documented, version-controlled eligibility engine and field logic, with change control when rules move, so scope decisions are consistent and auditable. Reference-data quality keeps LEIs active, instrument and UPI static maintained, and short-code mappings current. Monitoring and management information means dashboards of volumes, reject rates, match rates, and break ageing, with thresholds and escalation. Independent assurance brings second-line and internal audit review, and often specialist accuracy testing that re-derives fields from source. And clear ownership means a named owner per regime, a reporting policy, and senior accountability.
One trade, several regimes, and the coordination cost of running them together
A single economic event can spawn several reports, in several formats, to several bodies — all of which must be consistent. An OTC derivative triggers EMIR, from both sides to a repository, and MiFIR, from the executing firm to its regulator. A repo triggers SFTR from both sides to a repository. And a trade with an EU and a UK dimension may need EU and UK versions of each regime, to two different destinations. Our swap, if the dealer were in the United Kingdom, would need an EU EMIR report to an EU repository and a UK EMIR report to a UK repository, under two diverging rulebooks. This cross-border doubling, driven by Brexit, is one of the biggest sources of reporting cost and complexity for CIB desks.
The way to manage the multi-report burden is to source once and fan out, not to build a separate machine per regime. Four disciplines contain the cost. Source and enrich the data once in a shared layer, then format it into each regime's schema — never duplicate the sourcing logic per regime. Use the same identifiers across regimes, so the reports agree with each other and with the counterparty. Run one eligibility engine — one "brain" — that decides which regimes apply to each event, reducing both over- and under-reporting. And reconcile across regimes: check that the same trade tells a consistent story under EMIR, MiFIR, and SFTR, an emerging supervisory expectation. The architecture insight is that the expensive, firm-specific part is the shared sourcing and enrichment layer; the per-regime formatting is comparatively cheap, which is exactly why buying a reporting engine and owning the data feeding it is the dominant pattern, as Part 4 explains.
Why is reporting a lifecycle rather than a single send?
A trade sets off a continuous, event-driven sequence — capture, enrich, build, submit, reconcile, repair — and every later event on the trade (amendment, valuation, collateral, termination) must be reported until it closes. It needs a monitored pipeline, not a batch job.
Why is field sourcing, not formatting, the hard part?
A report pulls 100+ fields from many systems that were never built to feed reporting; they key data differently, some fields are missing at source, some arrive late, and each field needs one agreed golden source. Joining that reliably at scale is the real work.
What problem does the UTI waterfall solve, and what does it leave unsolved?
It fixes who must generate the shared trade key (CCP, then venue, then matching platform, then the counterparties). It does not solve actually transmitting the UTI to the other side in time — the operational sharing where pairing failures usually originate.
What did EMIR REFIT change, and why was it costly?
It mandated ISO 20022 XML, expanded fields to ~200, made the UPI mandatory, and tightened validation with global Critical Data Elements. It was costly because every firm had to remap fields, rebuild message generation, re-test end to end, and migrate live trades on a hard cut-over.
Distinguish pairing from matching, and give one cause of each kind of break.
Pairing asks whether both sides' reports found each other, on the shared UTI (a pairing break is caused by different UTIs). Matching asks whether the paired reports' fields agree within tolerance (a matching break is caused by, e.g., a notional convention or date-roll difference).
Why is "acceptance is not correctness" the key insight of this part?
A report can be accepted by the repository and still be wrong (mis-reporting), and a reportable trade can produce no report at all (under-reporting) — neither is visible in acceptance statistics. A mature shop measures completeness, validity, and accuracy separately; most fines are about completeness and accuracy.
When a systematic error is found, why is "just fix the report" the wrong response?
A defect usually spans thousands of reports over months. The right response is to quantify the affected population, fix the root cause, back-report the corrected history, self-report to the regulator, and add a control — remediation is judged on the control added, not the patch.
Why does running three regimes push firms toward a shared sourcing layer?
One trade can trigger EMIR, MiFIR, and SFTR (and EU/UK versions), so duplicating sourcing per regime multiplies cost and inconsistency. Sourcing and enriching once, then fanning out to each schema with consistent identifiers and one eligibility engine, contains the burden.
2
Why is reporting a lifecycle, not a single send?
Right.
No.
No.
No.
What is the hard part of reporting?
Right. Not formatting.
Bought commodity.
No.
No.
What does the UTI waterfall leave unsolved?
That is what it solves.
Right. It only fixes who generates it.
No.
No.
3Actors and the People Who Do the Work
Who is involved in getting a trade reported: the chain of firms and authorities outside the bank, the counterparties whose data you depend on, and the ops, data, controls, and change people inside the bank who actually run reporting.
A reportable trade passes through a chain of specialised firms before it is seen
Reporting is an ecosystem, not a single firm sending a message into the void. A trade the bank does must travel through a defined chain of parties and authorities before a regulator can be said to have seen it. Six kinds of actor make up that chain, and each holds part of the truth. Understanding who they are — and, crucially, that your report depends on parties you do not control — is the key to understanding why reconciliation and coordination dominate the work.
The reporting firm is the centre of gravity, but it is not the whole story. The reporting firm — the investment firm or CIB desk that trades and must report — owns data sourcing, report building, submission, and repair. But the counterparty or client on the other side of the trade matters enormously: for dual-sided regimes they report too, so your data must reconcile with theirs, and some clients delegate their reporting to the bank. The trade repository is the regulated central store for EMIR and SFTR reports that validates, pairs, matches, and gives regulators access. The approved reporting mechanism submits MiFIR transaction reports to the regulator on the firm's behalf. The national competent authority — the regulator such as the AMF and ACPR in France, BaFin in Germany, or the FCA in the United Kingdom — receives, supervises, and enforces. And above them sit the standards bodies: ESMA in the European Union and the FCA in the United Kingdom set the technical standards, GLEIF governs the LEI system, and the Derivatives Service Bureau issues the UPI.
Post-Brexit, the top of the stack doubles, and that doubling is a structural cost, not a detail. An EU trade faces an EU repository or ARM, ESMA-set standards, and an EU regulator. A UK trade faces a UK repository or ARM and FCA-set standards. A CIB active in both must maintain connectivity, rulebooks, and controls for both ecosystems in parallel. This is the same divergence we met in Parts 1 and 2, seen now from the actor side: it is not just two sets of rules, it is two sets of relationships with two sets of infrastructure and two sets of supervisors.
Every interaction is a data or instruction flow, and the table names them
The actor map becomes concrete when you list who sends what to whom, and at which stage. The hub-and-spoke diagram shows the parties; the table below shows the flows between them. Read it as the wiring diagram of the ecosystem — each row is a place where data or an instruction crosses an organisational boundary, and therefore a place where coordination can fail.
| From | To | What flows | Stage |
|---|---|---|---|
| Counterparty | Reporting firm | Agreed UTI, trade confirmation | Enrichment |
| Reporting firm | Counterparty | Reconciliation data, break resolution | Reconcile |
| Reporting firm | Trade repository | EMIR / SFTR reports and lifecycle events | Submit |
| Trade repository | Reporting firm | ACK / NACK, pairing and matching statistics | Submit / reconcile |
| Reporting firm | ARM | MiFIR transaction reports | Submit |
| ARM | National regulator | Forwarded transaction reports | Submit |
| National regulator | Reporting firm | Validation rejects, rules, findings, fines | Ongoing |
| Standards bodies (ESMA / FCA / GLEIF / DSB) | Reporting firm | Schemas, technical standards, identifiers | Ongoing |
| Triparty / lending agents | Reporting firm | Collateral and margin data (SFTR) | Enrichment |
The table makes one point vividly: the reporting firm sits on almost every flow, but controls almost none of the other parties. You depend on the counterparty to agree the UTI and to report data that matches yours. You depend on the repository to pair and match honestly. You depend on standards bodies to publish schemas you must implement, and on agents to feed you collateral data. Reporting is therefore an exercise in managing dependencies you cannot command — which is exactly why it lives in operations and data teams that are good at chasing, reconciling, and coordinating across boundaries.
Counterparties are not all alike, and the differences shape your reporting risk
Just as a deal desk must understand the different kinds of investor, a reporting shop must understand the different kinds of counterparty — because your reporting quality depends on theirs. In a dual-sided regime, your report has to agree with the other side's, so the sophistication, automation, and behaviour of the counterparty is not their problem, it is yours. A break caused by a poorly automated counterparty ages on your queue and draws supervisory attention to your estate. So the counterparty taxonomy below is a map of where reconciliation risk concentrates.
| Counterparty type | Reporting behaviour | Data-quality risk | Why it matters to you |
|---|---|---|---|
| Another dealer bank | Reports its own side, usually well automated | Low — but conventions can still differ | Cleanest reconciliation; breaks are usually genuine field differences |
| Buy-side client (fund, asset manager) | Reports its own side, variable automation | Medium — timing and mapping breaks | May be slower to share UTIs and resolve breaks |
| Client that delegates | You report on their behalf | Concentrated — you own both sides' quality | Delegation removes a break source but adds obligation and liability |
| Non-financial counterparty (corporate) | Often least automated, may delegate | High — missing data, slow response | A frequent source of ageing, hard-to-clear breaks |
Delegated reporting moves the work, not the accountability
Delegated reporting is one party reporting on another's behalf, and it is common where one side is far more capable than the other. A large bank routinely reports for its smaller or less-automated clients, especially corporates. Delegation is attractive because it removes a reconciliation counterparty — if you report both sides, the two reports come from one source and cannot disagree with each other in the way two independent firms can. It simplifies the client relationship and concentrates control. But delegation does not move the ultimate accountability entirely off the delegating party, and it loads the operational obligation and a share of the liability onto the delegate. So a bank offering delegated reporting is taking on other firms' reporting risk as a service — a real business decision with a real risk attached, not a favour.
REFIT added a second, mandatory kind of delegation that is not a commercial choice at all. As Part 1 noted, since EMIR REFIT a financial counterparty is legally responsible for reporting on behalf of its NFC- (smaller corporate) counterparties, whether it wants to be or not. It is worth holding the two kinds apart. Voluntary delegated reporting is a service a bank offers and a client accepts, governed by a contract that can allocate responsibility. Mandatory delegation is imposed by the rule: the bank must report both sides for an NFC-, and the legal liability sits with the bank regardless of any agreement. The operational consequence is the same in both cases — the bank sources and reports data for a counterparty it does not control — but the mandatory version removes the bank's choice and concentrates the liability by law. For a desk facing corporates, a large slice of the reporting population is now trades where the counterparty has no obligation to help and the bank carries the full accountability.
Reporting is run by ops, data, controls, and change — almost never the front office
The most important fact about who does reporting is who does not: the trader. Reporting is overwhelmingly a middle- and back-office, data, and technology discipline. The front-office trader executes the trade and moves on; everything downstream — sourcing the fields, deciding eligibility, building and submitting the report, working the breaks, proving accuracy — is done by other people entirely. This is not a slight on the trader; it is a description of where the work lives. And it is precisely why reporting is such a natural, durable home for business analysts, data engineers, operations professionals, and change specialists. Four functions carry the load, and each maps to specific stages and systems.
Regulatory reporting operations run the daily cycle
Operations, sitting in the middle office, own the heartbeat: getting today's reports out and reconciled. They monitor submissions, chase rejections, and meet the T+1 deadline. They work reconciliation breaks with counterparties and repositories, and escalate ageing items. They own the exception queues and the completeness check — the "did everything that should have reported actually report?" question. Operations is where the pipeline meets the clock, and where a quiet, disciplined daily routine is the difference between a clean estate and a growing backlog. In our stage model, operations own stages four, five, and six.
Data sourcing and reference data feed the machine
Data engineering and reference-data teams solve the sourcing problem that Part 2 identified as the real work. They source the hundred-plus fields from front-, middle-, and back-office systems, and they build and run the enrichment pipelines that feed the reporting engine. They maintain the identifiers everything joins on — LEIs, ISINs, UPIs, MICs, and the short- and long-code mappings. When a golden source must be chosen or a new field wired in, this is the team that does it. They own stage two and the reference-data spine that runs through all the others.
Controls and assurance prove the estate is trustworthy
Controls and assurance — spanning first line, second line, compliance, and audit — design and run the checks that make reporting defensible. They build and operate the completeness, accuracy, and timeliness controls, run independent assurance and specialist testing that re-derives fields from source against an oracle, and own the control framework, the attestations, and the interaction with the regulator. Their independence is the point: a control run by the same people who run the pipeline proves less than a control run by someone with the standing to say the pipeline is wrong. This is the three-lines-of-defence idea applied to reporting, and it is the closest thing reporting has to the information-barrier principle from a trading floor — a deliberate separation of the doers from the checkers.
Business analysis, IT, and change are the permanent engine room
Business analysts and delivery teams translate the never-ending stream of rule changes into working code, and they never run out of work. They turn new rules — REFIT, ISO 20022, the MiFIR review — into requirements and field mappings, deliver schema upgrades and new validation logic, and run the go-lives. Because the requirements are owned by an external party who keeps changing them, this team is not a project that finishes but a standing capability. They interpret rule changes into working systems, and then do it again next year. In career terms, this is the durable heart of reporting: the skills transfer directly between firms and between the EU and UK regimes.
| Function | Owns which stages | Core deliverable | Primary systems |
|---|---|---|---|
| Operations / middle office | Submit, reconcile, repair | Today's reports out and reconciled | Connectivity, exception workflow |
| Data engineering / reference data | Eligibility and enrichment | Sourced, enriched, flagged records | Reference-data store, enrichment pipelines |
| Controls and assurance | Wraps all stages | Evidenced completeness, accuracy, timeliness | Reconciliation and control layer, accuracy testing |
| Business analysis / IT / change | Delivers changes to all stages | Rule changes turned into working systems | Eligibility engine, reporting engine, test harness |
Map the four functions onto the pipeline and the operating model becomes clear. Data engineering owns the sourcing and enrichment block. Business analysis owns the eligibility logic and its change control. The reporting engine — build, validate, format — is run by IT and operated by ops. Connectivity and the exception queue are operations. And wrapped around all of it, independent of the pipeline, sits controls and assurance with its own reconciliation, sampling, and monitoring. No single team owns reporting end to end; it is a relay, and the handoffs between teams are themselves a place where quality is won or lost. A firm that treats reporting as "an IT system" rather than a cross-functional operating model will find the handoffs are where its breaks come from.
The three lines of defence, and why independence is the point
The controls-and-assurance function is organised around a model called the three lines of defence, and it is the closest thing reporting has to a trading floor's information barrier. The first line is the pipeline itself — the operations and data teams who run reporting and own its controls day to day. The second line is independent risk and compliance, who set the control standards and check that the first line is meeting them. The third line is internal audit, independent of both, who periodically assess whether the whole framework works. The point of the layering is independence: a control checked only by the people who run the pipeline proves little, because they are marking their own homework. A control checked by someone with the standing and independence to declare the pipeline wrong is worth far more.
Independence is expensive and occasionally adversarial, and that is exactly why it matters. The second and third lines exist to say uncomfortable things — that a control is weak, that a break has aged too long, that a rule change was mis-scoped — and their value depends on their freedom to say those things without the first line being able to overrule them. This is the same principle that separates the functions on a trading floor: a check is only a check if the checker is independent of the checked. In reporting, where the failures are silent and slow, an independent second and third line are often the only mechanism that finds a decade-long defect before a supervisor does. Firms that collapsed these lines into the operational team, or let the first line grade its own accuracy, are heavily represented in the enforcement record.
A day in the life of the reporting cycle
The abstract stages become vivid when you watch a single working day, because reporting is one of the few bank functions with a daily, non-negotiable rhythm. The cycle turns over every day the market is open, and each function has its window. Overnight and into the early morning, the previous day's trades flow from capture into enrichment, and the pipeline builds the reports. Through the morning, operations watch the submissions go out and the acknowledgements come back, working any rejections while there is still time before the deadline. The trade repository, meanwhile, runs its pairing and matching on the prior day's population and publishes the break statistics, which land on the reconciliation team's queue. By the afternoon, attention has split: operations are closing today's submissions while the reconciliation team works yesterday's breaks and chases counterparties for the ones that are not theirs to fix. The change team, on a different clock entirely, is building next quarter's rule change. Four functions, three time-horizons — today's deadline, yesterday's breaks, next quarter's migration — all running at once.
This rhythm is why reporting operations is a discipline of vigilance rather than heroics. The estate does not need someone to perform a brilliant rescue once a quarter; it needs someone to watch the acknowledgements every single morning and clear the exception queue before it ages. The failures in Part 5 all share a signature: nobody was watching the completeness, or the accuracy, or the breaks, day after day, until years had passed. A reporting shop is judged over a long run of ordinary days, and the ordinary day is where it is won or lost.
Name the six kinds of actor in the reporting ecosystem and the reporting firm's relationship to each.
Reporting firm (owns sourcing/build/submit/repair), counterparty/client (reports its own side or delegates), trade repository (validates, pairs, matches EMIR/SFTR), ARM (submits MiFIR to the NCA), national regulator (supervises and enforces), standards bodies — ESMA, FCA, GLEIF, DSB (set schemas and issue identifiers).
Why does the type of counterparty affect your reporting risk?
In dual-sided regimes your report must agree with the counterparty's, so a poorly automated counterparty (often a non-financial corporate) produces ageing, hard-to-clear breaks on your queue. Your quality is capped by your least sophisticated counterparty.
What does delegated reporting move, and what does it not move?
It moves the operational work and part of the liability onto the delegate (and removes a reconciliation counterparty), but it does not fully move ultimate accountability off the delegating party. A bank offering it is taking on others' reporting risk as a service.
Which internal function owns each of: field sourcing, the daily deadline, independent accuracy testing, and turning REFIT into code?
Data engineering/reference data owns field sourcing; operations owns the daily deadline and breaks; controls and assurance owns independent accuracy testing; business analysis/IT/change owns turning rule changes into working systems.
Why is the front-office trader absent from the reporting operating model, and why does that matter for a career?
Reporting is a middle/back-office, data, and technology discipline — the trader executes and moves on. That makes reporting a durable home for BAs, data engineers, ops, and change professionals, with skills that transfer between firms and between EU and UK regimes.
Why is independence the point of the three lines of defence?
A control checked only by the people who run the pipeline proves little (they mark their own homework). Independent second and third lines can declare the pipeline wrong without being overruled, which — given reporting's silent, slow failures — is often the only mechanism that finds a defect before the regulator does.
3
What does delegated reporting not move?
No.
Right.
It removes one.
It moves that.
Why does counterparty type affect your reporting risk?
Right.
Not necessarily.
No.
It does in dual-sided regimes.
What does a trade repository do?
No.
No.
Right. An ARM submits MiFIR reports.
No.
4Systems, Data, and the Reporting Pipeline
The architecture that turns a booked trade into an accepted, reconciled report: the functional blocks, the vendors who fill them, how data flows front to back, the anatomy of a report, the identifiers, the integration patterns, the message standards, and the resilience that the deadline demands.
Ask what functions any reporting estate needs before naming a single vendor
The cleanest way to understand reporting systems is to ask what any reporting estate must do, regardless of who built it. Strip away the vendor names and every estate performs the same four functions in sequence, wrapped in a set of cross-cutting capabilities. Source and enrich the data. Decide eligibility. Build and validate the report. Connect to the repository or ARM. Around those four sit the shared services that every block relies on. This functional view is worth holding firmly, because vendors and build decisions change, but the functions do not.
| Functional block | What it does | Owns |
|---|---|---|
| Data sourcing and enrichment | Pull trade and reference data from many systems; add LEI, UPI, ISIN, valuations, collateral | The join problem — the hard, firm-specific part |
| Eligibility logic | Decide if and under which regime(s) each event is reportable | The rules brain |
| Reporting engine | Assemble, validate, and format each report to the required schema | Schema compliance and internal validation |
| TR / ARM connectivity | Transmit, receive ACK/NACK, feed rejections and breaks back | The message layer and its acknowledgements |
The pipeline is a cycle, not a one-way street, and the feedback loop is the part beginners miss. Rejections and reconciliation breaks flow backward from connectivity to the engine and to sourcing, because that is where they must be fixed. A report is not fired and forgotten; its failure returns upstream to be repaired at the point where the error was made. Around the four blocks sit the cross-cutting systems: a reference-data store as the golden source for LEIs, instruments, and calendars; an exception and workflow tool that queues rejections and breaks for operations; a reconciliation and control layer that runs independent completeness and accuracy checks; and an immutable audit store that records what was reported, when, and why.
Most firms buy the engine and own the data feeding it
Few firms build the whole stack, and the buy-versus-build line falls in a very consistent place. Most banks buy a vendor reporting engine — eligibility, build, and connectivity — and wrap it with their own sourcing, enrichment, and controls. The reason is a clean map of competitive advantage: the engine is a commodity, because every firm needs the same ISO 20022 output to the same repositories, so a vendor spreads that cost across many clients and keeps up with rule changes better than any single bank could. But getting clean data into the engine is firm-specific — it depends on the bank's own tangle of booking systems, reference data, and desks — so no vendor can do it for you. The hard, differentiating part is almost always the sourcing, not the engine.
| Layer | Typical choice | Why |
|---|---|---|
| Reporting engine (build, format, connect) | Buy | Commodity output to shared infrastructure; vendor absorbs rule changes |
| Eligibility logic | Buy or partner | Complex, rule-driven, benefits from a specialist's rule library |
| Sourcing and enrichment | Build | Firm-specific; depends on the bank's own systems and golden sources |
| Controls and assurance | Build or independent specialist | Must be independent of the pipeline; often an outside oracle |
The vendor landscape has a small set of names worth knowing
A technologist or change professional working in reporting will meet the same infrastructure and RegTech names repeatedly. On the infrastructure side, DTCC's Global Trade Repository is the dominant trade repository for EMIR and SFTR reporting worldwide; REGIS-TR is a major European repository, a joint venture strong in continental Europe; and UnaVista, part of the London Stock Exchange Group, is a major ARM for MiFIR as well as offering repository services. On the RegTech and solutions side, Bloomberg offers a reporting hub and entity and reference data that converts trades into regime-ready reports and manages LEIs; Kaizen Reporting is an assurance specialist that independently tests fields against the rulebook; Cappitech, part of S&P, is a multi-regime reporting platform with analytics and reconciliation dashboards; and Droit provides a computational eligibility and rule engine that codifies "is this reportable, and where?" as executable logic. These are illustrative market examples for orientation only, not endorsements, and the landscape shifts through consolidation and new entrants.
Follow one report front to back and the cascade principle becomes concrete
Data lineage is the discipline of following one object from origination to regulatory reporting, and it reveals why upstream errors are uncontainable. Trace our swap's report backward and you pass through every system that touched it: the booking system created the trade record with its economics and parties; enrichment added the LEIs, ISIN or UPI, UTI, and MIC; the eligibility engine flagged it as an EMIR derivative; the reporting engine built the ISO 20022 XML; connectivity submitted it and received the acknowledgement; and reconciliation paired and matched it. Each stage created a data object that the next stage consumed.
The cascade principle is simply this: an error at the top of the chain reaches every system below it. Because each object depends on an upstream decision, a wrong value at capture is wrong in enrichment, wrong in the built report, wrong at the repository, and wrong in the regulator's aggregated data. Lineage is not a documentation exercise for its own sake; it is the tool that lets you answer the question every remediation asks — where did this bad value enter, and everything downstream of that point is suspect. A firm that cannot trace lineage cannot size a defect, and a firm that cannot size a defect cannot remediate it credibly.
| Stage | System | Data object created |
|---|---|---|
| Capture | Booking / front office | Trade record (economics, parties) |
| Enrichment | Reference-data store | Enriched trade (+ LEI, ISIN, UPI, UTI, MIC) |
| Eligibility | Eligibility engine | Reportable flag and regime(s) |
| Build | Reporting engine | ISO 20022 XML report |
| Submit | TR / ARM connectivity | Submitted report + ACK/NACK |
| Reconcile | Reconciliation layer | Pairing and matching status, breaks |
The anatomy of a report, field by field
A report is a data object with a defined set of fields, and knowing what each field drives downstream is what separates a data professional from a form-filler. Take the MiFIR transaction report as the worked anatomy. Its report status says whether this is a new record, a cancellation, or a correction — which determines how the regulator's system treats it. The transaction reference number uniquely identifies this report. The executing-entity LEI, submitting-entity LEI, buyer, and seller identify the parties, sourced from the reference-data store. The investment decision-maker and the execution-within-firm fields identify the people or algorithms, sourced from short-code mappings. The instrument ISIN identifies what was traded, from instrument static. Price, quantity, currency, and the venue MIC describe the trade economics and location. The trading date-time, to the microsecond, comes from the execution venue. The trading capacity — dealing on own account, matched principal, or any other capacity — and the short-selling indicator are regulatory flags that change the meaning of the report.
| Field | What it holds | Sourced from | Failure if wrong |
|---|---|---|---|
| Report status | New, cancel, or correct | Pipeline logic | Regulator mishandles the record |
| Transaction reference | Unique ID for this report | Reporting engine | Cannot trace or correct the report |
| Executing-entity LEI | Who executed | Reference data | Rejection; wrong attribution |
| Buyer / seller | The parties | Reference data, client systems | Surveillance sees the wrong actors |
| Investment decision-maker | Who decided (person or algo) | Short-code mapping | Abuse cannot be traced to an actor |
| Execution within firm | Who executed (person or algo) | Short-code mapping | Broken accountability chain |
| Instrument ISIN | What was traded | Instrument static | Wrong instrument; rejection |
| Price / quantity / currency | The economics | Booking | Mis-stated exposure |
| Venue MIC | Where it traded | Execution systems | Wrong venue; on/off-venue error |
| Trading date-time | When, to the microsecond | Execution venue | Broken sequence for surveillance |
| Trading capacity | Own account, matched principal, other | Booking logic | Mislabelled role; common error |
| Short-selling indicator | Regulatory flag | Booking logic | Mis-reported regulatory status |
Every field is a data-sourcing task, and getting any one wrong can reject the whole report or, worse, make it silently wrong. Notice how much of the record is identifiers — LEIs, ISIN, MIC, national identifiers, short codes. Regulatory reporting is, in large part, an identifier-management problem, which is why so much of the engineering effort goes into keeping those identifiers correct and current. The fields divide into three kinds: identifiers that must be looked up and kept fresh, economics that must be captured accurately at source, and flags that must be derived by correct logic. Each kind fails differently, and each needs its own control.
The identifiers, seen as data to be managed rather than codes to be looked up
From a systems perspective, the identifiers are reference data with lifecycles, ownership, and failure modes, not static lookups. The LEI must be present and active, and it expires — it is renewed annually, so a reporting estate must monitor LEI validity and refresh mappings before they lapse. The UPI is issued by the Derivatives Service Bureau and must be sourced and cached. The ISIN comes from instrument static data that changes as instruments are issued and mature. The MIC is a stable code but must be applied correctly, including the off-venue conventions. And the short and long codes are firm-internal data that must be mapped to real identities and kept current as traders and algorithms change. Treating identifiers as living reference data, with monitoring and refresh, is the difference between an estate that reconciles and one that quietly rots.
Reference data is a living system, and the exception queue is where the estate is run
The reference-data store is not a filing cabinet — it is a living system that must be fed, refreshed, and reconciled, because the identifiers it holds decay. The golden source for LEIs, instruments, and calendars sits at the centre of every enrichment, so its quality sets a ceiling on the quality of every report. And its contents change constantly: LEIs are renewed annually and lapse if they are not, instruments are issued and mature, calendars shift with holidays, and short-code mappings change as traders and algorithms come and go. A reference-data store that is not actively maintained does not fail loudly; it rots quietly, and the reports built from it drift wrong one stale record at a time. This is why reference-data management is a standing function, not a setup task, and why so many real breaches trace back to a single identifier nobody was keeping current.
The exception and workflow tool is where the abstract pipeline becomes a place people actually work, and its design decides whether the estate scales. Every rejection and every reconciliation break lands in a queue that operations must work. A good exception tool does more than list problems: it groups related breaks so a single root cause is worked once rather than a thousand times, it ages each item so nothing sits ignored, it routes each error type to the right team, and it records what was done for the audit trail. A reporting estate can be judged surprisingly well by the state of its exception queue — a clean, ageing-controlled queue is a well-run estate, and a growing backlog of stale, ungrouped breaks is a fine in slow motion. The queue is the operational conscience of the whole system.
Think of the exception queue as the reporting estate's equivalent of a bug tracker, and apply the same discipline. Items must be triaged by severity, grouped by root cause, assigned to an owner, and closed with a record of the fix — and the health of the whole system is read from the queue's ageing profile, not from any single item. The difference from a software bug tracker is the clock: a reporting break has a regulatory ageing dimension, because an unresolved break that sits for weeks is not just untidy, it is evidence of a control weakness that a supervisor can act on. The best reporting operations run their exception queue with the rigour of a well-run engineering team running its incident backlog, because that is functionally what it is.
How the systems talk to each other decides how fresh the data is
The pipeline's blocks must exchange data, and the pattern they use — batch, API, or event streaming — quietly determines the estate's latency and its appetite for reconciliation. This is one of those architectural choices that sounds like plumbing and turns out to be strategy. Three patterns dominate, and they form a ladder from the legacy default to the modern target.
Batch is the legacy default: systems exchange large files on a schedule, usually overnight, and it is where most latency and reconciliation live. In a batch world, the booking system writes a file of the day's trades, and the reporting pipeline picks it up hours later. It is simple and robust, but it has two weaknesses. It is slow — a trade booked in the morning may not reach the pipeline until the nightly run, eating into the T+1 window. And because each system holds its own copy of the data between runs, the copies drift apart, so you must reconcile them constantly to catch the drift. Batch does not just move data slowly; it manufactures the reconciliation problem by letting copies diverge.
An API — an application programming interface — lets one system ask another for data on demand, which removes the wait but not the copies. Instead of receiving a nightly file, the pipeline calls the reference-data service the moment it needs an LEI and gets the current value back. This is the modernisation step: data is fresh at the point of use, and the pipeline no longer works from stale overnight snapshots. APIs are how a cloud-based, on-demand architecture replaces brittle batch joins, and they are the direction most enrichment layers are heading.
Event streaming is the furthest rung: systems publish changes as they happen, and downstream systems consume the stream in near-real-time — which removes the copies, and with them much of the reconciliation. In a streaming model, the booking system publishes each new trade as an event the instant it occurs, and the reporting pipeline consumes it immediately. There is no nightly file and no on-demand pull; there is one flowing source of truth that every consumer sees the same way. This is more than a speed upgrade, and that is the point worth dwelling on.
The migration from batch to streaming is often sold as "faster," but its deeper value is that it removes the need for reconciliation. Reconciliation exists because two systems hold two copies of the same data that can drift apart; you reconcile to catch the drift. If there are no separate copies — if every system consumes the same event stream as the single source of truth — then there is nothing to drift, and a whole category of internal reconciliation simply disappears. That is why event streaming is strategic rather than cosmetic in reporting: it attacks the root of the data-quality problem, which is duplicated state, instead of treating the symptom. A reporting estate that moves from nightly batch to event streaming is not just shaving hours off its latency; it is dissolving the internal breaks that batch created in the first place. The external reconciliation with counterparties remains — you cannot stream your way out of the fact that the other firm keeps its own books — but the internal drift can be designed away.
The audit store is immutable, and the eligibility engine is the one thing you always own
Compliance-critical records must be append-only, because the whole point of an audit trail is that it cannot be quietly rewritten. The audit store records what was reported, when, and why — and it must be immutable, so that a report's history can be reconstructed years later for a regulator or an auditor. You do not edit an audit record; you append a correction that references the original. This append-only discipline is what lets a firm prove, after the fact, exactly what it did and did not report, which is precisely the evidence supervisors now demand. Governance platforms exist to hold these records and manage the attestations and control evidence around them.
The eligibility engine — the rules brain — is the one component that is effectively always proprietary in its configuration, even when the platform is bought. Deciding whether an event is reportable, and under which regimes, encodes the firm's specific interpretation of the rules, its product set, and its booking model. Even firms that buy a computational eligibility platform must configure it to their own trades and defend those decisions to the regulator. Eligibility is where over- and under-reporting are decided, so it is the component that most rewards careful version control, testing, and documentation. The engine may be bought, but the rules it runs are the firm's own, and the firm owns the consequences of getting them wrong.
Two message worlds: the front-office capture layer and the reporting layer
Reporting sits at the meeting point of two messaging worlds, and confusing them is a classic beginner's error. The front-office layer speaks FIX — the Financial Information eXchange protocol — a fast, session-based messaging standard that carries orders and executions between trading systems and venues in real time. FIX is where a trade's economics and timestamps are born; it is the fire hose of execution data that feeds the capture systems reporting later reaches into. The reporting layer speaks a different language entirely: ISO 20022 XML for EMIR reports, structured messages to the repository, on a T+1 clock rather than in real time. FIX is front-office, fast, and about doing the trade; ISO 20022 reporting is post-trade, deadline-bound, and about describing the trade to an authority.
| Layer | Standard | Carries | Timescale |
|---|---|---|---|
| Front-office / execution | FIX | Orders, executions, venue messages | Real-time |
| Reporting to repository / regulator | ISO 20022 XML | Structured trade reports (EMIR REFIT) | T+1 |
| Legacy reporting formats (pre-REFIT) | CSV / proprietary | Older report submissions | Being retired |
ISO 20022 matters because a single, richly typed, validated schema means everyone interprets every field identically — the foundation of clean matching. Before REFIT, EMIR reporting used older CSV-style and proprietary formats that different systems could interpret slightly differently, which seeded matching breaks. ISO 20022 replaces that with a structured schema where the repository and both firms read each field the same way. The direction of travel is unmistakable: the whole reporting estate is converging on this one message standard, with SFTR and MiFIR following EMIR, and with global Critical Data Elements ensuring the core fields mean the same thing everywhere. Standardised data leads to better pairing and matching, which leads to fewer breaks — but only after a multi-year re-plumbing to get there.
The concrete plumbing: message names and how reports reach a repository
Once you know reporting speaks ISO 20022, the next practical question a technologist asks is which messages, and over what wire — and both have concrete answers. Under EMIR REFIT the derivatives trade report is carried in the ISO 20022 message known as auth.030, with related messages for the repository's feedback and reconciliation status; SFTR uses its own family of auth-series messages for securities-financing reports. These message names are not trivia — they are what a build team implements, tests against, and version-controls, and a rule change often means moving to a new version of a named message. Knowing the message is the difference between talking about reporting and building it.
The wire itself is more prosaic than people expect, and that is the point. Reports typically reach a trade repository or ARM by one of a few connectivity mechanisms: secure file transfer, where batches of ISO 20022 messages are dropped onto an endpoint on a schedule; a message queue for higher-volume, more continuous flow; or increasingly an API for on-demand submission and status retrieval. Whatever the channel, the same discipline applies — every message sent must be matched to an acknowledgement or rejection, connectivity must be monitored and redundant, and the whole exchange must be logged for audit. The unglamorous truth is that a large reporting estate is, at the wire level, a high-reliability file-and-message transfer system with a legal deadline, and its engineering is judged on exactly the properties that implies.
How independent accuracy testing actually works: the oracle
The strongest accuracy control is to rebuild the report a second time, independently, and compare — and that independent rebuild is called an oracle. Acceptance is not correctness: a report can pass every validation and still carry a wrong value, and fixed rules cannot catch that, because the value is well-formed. Only re-deriving the expected report from source data and comparing, field by field, reveals it. An oracle is exactly that — a separate, independent implementation of the reporting logic, built by a different team or a specialist vendor, that takes the same source data and computes what each report should say. Where the oracle and the production report disagree, one of them is wrong, and the disagreement is a candidate accuracy defect to investigate.
The oracle is powerful precisely because it does not trust the pipeline it is checking. If the production engine and the checking engine shared code, they would share bugs and agree with each other while both being wrong. Independence is the whole value: a second implementation, ideally reading the rules afresh from the regulator's text, will make different mistakes, so the places where the two agree are trustworthy and the places where they diverge are where the truth is contested. This is the technical shape of the specialist accuracy-testing that supervisors now expect, and it is why "we validated the report" and "we independently re-derived the report from source" are two very different claims — only the second is real assurance.
Testing a reporting change before it goes live
Because a reporting change lands on a hard regulatory deadline with no grace, it is tested more like an aircraft system than a typical software release. Trade repositories and ARMs publish their validation rules and provide test environments — sandboxes — where a firm can submit sample messages and see the accept-or-reject verdict before production. A change programme works through several layers: unit tests on the new field logic, schema validation against the published rules, submission to the repository's test environment to confirm the messages are accepted, and reconciliation testing with counterparties in a user-acceptance environment to confirm the two sides still pair and match. The most important layer is the dual run — for a period, the firm produces reports under both the old and the new logic and compares them, so any divergence is caught before the cut-over rather than after.
The reason for this rigour is that a reporting go-live cannot be quietly rolled back. If a change breaks reporting on its first live day, the deadline still passes and the broken or missing reports are a breach, so there is no equivalent of pulling a bad release the next morning with no harm done. Testing therefore has to prove not just that the new logic works, but that it works at production volume, produces accepted messages, and reconciles with counterparties — before a fixed date set by a regulator who will not move it. This is why change and testing are a permanent, well-resourced function in reporting, not an afterthought bolted onto delivery.
The deadline makes resilience non-negotiable
Reporting has an irreversible moment — the T+1 deadline — and that changes what resilience means. Some systems can tolerate an outage because the work can be redone later. Reporting cannot, because the deadline does not move for your outage. If the pipeline is down on the evening a day's trades must report, the deadline passes and the missing reports become a breach regardless of the reason. So the resilience bar for reporting is set not by convenience but by an external clock that will not wait. This is the same principle that makes a deal's pricing night irreversible, applied to a discipline that has an irreversible night every single day.
| Concept | What it guarantees | Why it matters on the deadline |
|---|---|---|
| Redundancy | A spare component ready if one fails | A single failure does not stop submission |
| Automatic failover | Switchover to the spare without manual intervention | No human delay while the clock runs down |
| Disaster recovery | The estate can be rebuilt at a second site | A site loss does not mean a day of missed reports |
| Business continuity | Documented, rehearsed processes to keep running | People know what to do when systems degrade |
The regimes, reread as system specifications
Each regulation is best read by a technologist not as a legal text but as a specification for the pipeline, and five constraints shape every system in the chain. EMIR specifies dual-sided reporting to a repository, keyed on a shared UTI, with lifecycle, valuation, and collateral events — so the system must generate and share keys, hold trades open for later events, and feed reconciliation. MiFIR RTS 22 specifies single-sided, surveillance-driven reporting with actor identification and microsecond timestamps — so the system must map short codes to identities and synchronise clocks precisely. SFTR specifies dual-sided reporting with a heavy collateral dimension and post-trade allocation — so the system must ingest triparty and agent feeds and handle late collateral updates. The technical standards — ISO 20022 schemas and validation rules under REFIT — specify the exact message format and allowed values, so the engine must produce schema-perfect, validatable output. And data-protection law, GDPR, specifies how the personal data inside reports — national identifiers, decision-maker identities — must be handled, retained, and protected, so the audit and reference-data stores must respect retention and access rules.
Reading the regimes as specifications turns compliance into engineering, which is the mindset the whole module is arguing for. A legal summary tells you the rule exists; a specification tells you what the system must do. The productive question is never "what does this regulation say?" but "what must the pipeline do differently because of it?" That translation — from rule to requirement to field mapping to validation logic — is the daily work of the business analysts and engineers who run reporting, and it is why the discipline rewards people who can hold both the regulatory intent and the system behaviour in mind at once.
What are the four functional blocks of a reporting estate, and what is the feedback loop?
Sourcing and enrichment, eligibility logic, the reporting engine, and TR/ARM connectivity. The feedback loop is rejections and reconciliation breaks flowing backward to the engine and sourcing to be repaired — the pipeline is a cycle, not one-way.
Why do firms buy the engine but build the sourcing?
The engine is a commodity — every firm needs the same ISO 20022 output to the same infrastructure, so a vendor spreads the cost and absorbs rule changes. Sourcing is firm-specific, depending on the bank's own booking systems, reference data, and golden sources, so no vendor can do it for you. The hard, differentiating part is the data.
What is the cascade principle, and why is data lineage the tool for it?
An error at the top of the chain is wrong in every system below it, because each object depends on an upstream decision. Lineage lets you find where a bad value entered and treat everything downstream as suspect — you cannot size or remediate a defect without it.
Why is deduplication mission-critical?
The same trade can appear in multiple systems or be picked up twice, producing over-reporting (a breach); over-aggressive deduplication suppresses a needed report, producing invisible under-reporting. Deciding when two records are "the same trade" is genuinely hard because of allocations, amendments, and give-ups.
Distinguish FIX from ISO 20022 reporting, and describe the batch-to-streaming direction of travel.
FIX is the real-time front-office execution protocol where a trade's economics are born; ISO 20022 XML is the structured, T+1 reporting format (mandated by EMIR REFIT) used to describe the trade. Integration is moving from overnight batch (which manufactures reconciliation by letting copies drift) through on-demand APIs to event streaming (one source of truth, which dissolves internal reconciliation).
Why is redundancy alone insufficient for reporting resilience?
A spare that must be switched on by hand is useless if a failure hits late at night with the deadline approaching and no one watching. Reporting's daily irreversible T+1 moment requires automatic failover with no human in the critical path, plus rehearsed disaster recovery — a far higher bar than having spares.
What does it mean to read a regulation as a system specification?
Instead of asking what the rule says, ask what the pipeline must do differently because of it — e.g. EMIR means generate and share UTIs and hold trades open for lifecycle events; RTS 22 means map short codes and synchronise clocks. It turns compliance into engineering requirements, field mappings, and validation logic.
4
Why buy the engine but build the sourcing?
Right.
No.
No.
Reversed.
Why is data lineage the tool for the cascade principle?
No.
Right.
No.
It is.
What is the feedback loop of a reporting estate?
No.
No.
Right.
No.
5Risks, Failures, and Transformation
The ways reporting fails and who owns each risk, how to tell a well-run estate from a lucky one, three real fines that map to three failure modes, the transformation agenda, an honest look at artificial intelligence, and the six ideas to carry out of the module.
Every way reporting fails is a data-and-controls problem, not a market one
Reporting has a small, complete set of failure modes, and naming them precisely is the first step to controlling them. Unlike a trading desk, a reporting estate is not exposed to market risk or pricing risk. Its risks are all variations on a single theme: the data was wrong, incomplete, late, inconsistent, or ungoverned. The six risk types below are mutually exclusive and collectively exhaustive — every real reporting incident is one of them — and each has a natural owner and a primary defence.
| Risk type | What it is | Who owns it | Primary defence |
|---|---|---|---|
| Completeness | A reportable trade produced no report | Operations, data | Reconcile reports to source population |
| Accuracy | A report was accepted but a field is wrong | Data, controls | Independent accuracy sampling from source |
| Timeliness | A report missed the T+1 deadline | Operations | Deadline monitoring, automatic failover |
| Reconciliation | Our data disagrees with the counterparty's | Operations | Break management, counterparty outreach |
| Regulatory / conduct | Fines, findings, and the self-reporting duty | Compliance, senior management | Control framework, self-reporting, remediation |
| Change / delivery | A rule or system change breaks a feed | Business analysis, IT | Change control, dual-run, regression testing |
Telling a good reporting shop from a lucky one
A low rejection rate is the metric everyone quotes and the one that hides the most, so a mature shop measures five things instead. It is tempting to judge a reporting estate by how few of its reports the repository rejects. But rejections are the easy, visible failure; the dangerous failures are invisible in a rejection statistic. The five indicators below, read together, distinguish an estate that is genuinely well run from one that has simply not been caught yet.
| Indicator | What it measures | Why it matters |
|---|---|---|
| Completeness rate | Share of reportable trades that produced a report | Catches under-reporting — the invisible, worst failure |
| Acceptance / reject rate | Share of reports the TR/ARM accepts | The visible baseline; necessary but not sufficient |
| Match rate | Share of dual-sided reports that pair and match | Measures consistency with counterparties |
| Break ageing | How long reconciliation breaks stay open | Ageing breaks are what draw supervisory attention |
| Accuracy sample pass rate | Share of sampled reports that re-derive correctly from source | Catches silent mis-reporting validation misses |
The maturity ladder from reactive to assured
Reporting estates sit on a ladder from reactive to assured, and knowing which rung you are on tells you what your next risk is. The ladder is a useful diagnostic because firms rarely leap; they climb. At the bottom is the reactive estate, which reports what it can and fixes what the repository rejects, measuring only its reject rate — precisely the posture that produced the fines below. A rung up is the controlled estate, which reconciles reports to source for completeness and works its breaks to closure, so under-reporting can no longer hide. Higher still is the assured estate, which independently re-derives fields from source to test accuracy, evidences every control, and names an owner per regime. At the top is the optimised estate, which uses standardised data, straight-through processing, and analytics to prevent errors rather than catch them, and treats artificial intelligence as an assistant to its controls.
The value of the ladder is that each rung neutralises a specific failure mode, so it doubles as a to-do list. If you only measure reject rates, your live risk is under-reporting and mis-reporting, because nothing you do can see them — climb to completeness reconciliation and accuracy testing. If you have controls but no independent assurance, your risk is that your controls are marking their own homework — bring in a second line and an outside oracle. If you are assured but still fighting the same breaks every month, your risk is that you are catching errors instead of preventing them — invest in standardisation and straight-through processing. The three fines below are all firms that sat on the bottom rung for years, measuring the one thing that could not save them, and the transformation agenda that follows is simply the climb up the ladder made concrete.
Three real fines, three distinct failure modes
The regulatory record contains a clean set of examples, and they are not random — they are the failure modes of the process made concrete. Three Financial Conduct Authority enforcement actions, all real, illustrate the three ways a reporting estate fails: it can miss reports entirely, it can send reports that are wrong, and it can let a control weakness run undetected for years. Each maps to a stage of the lifecycle and carries a lesson.
The completeness failure — Merrill Lynch International, EMIR, 2017
Merrill Lynch International was fined £34,524,000 by the FCA in October 2017 for failing to report 68.5 million exchange-traded derivative transactions between February 2014 and February 2016. This was the first FCA enforcement action for failing to report exchange-traded derivatives under EMIR. The regulator found the firm had inadequate oversight arrangements, did not test its reporting adequately, and did not allocate enough staff to meet its obligations. The failure was not that reports were wrong — it was that reports were absent. This is the completeness failure in its purest form: trades that should have produced reports produced none, and nobody was reconciling the population of trades against the reports to notice the gap. It maps to stage four, submission, and its lesson is that completeness must be a tested control, because a missing report is invisible unless you look for its absence.
The accuracy failure — UBS AG, MiFID transaction reporting, 2019
UBS AG was fined £27,599,400 by the FCA in March 2019 for failings across 135.8 million transaction reports between November 2007 and May 2017. The breakdown is instructive: around 3.65 million transactions were not reported at all, roughly 83 million were reported inaccurately, and about 49.1 million were reported that should not have been. The dominant failure was accuracy — reports that were accepted but wrong, driven substantially by reference-data errors, the stale and incorrect identifiers that Part 1 warned about. The regulator's concern was not that the reports failed validation; most did not. It was that the accepted reports were unreliable as surveillance data. This maps to the accuracy control, and its lesson is that acceptance is not correctness: without independent accuracy testing that re-derives fields from source, wrong reports flow for a decade looking perfectly valid.
The control failure — Goldman Sachs International, MiFID transaction reporting, 2019
Goldman Sachs International was fined £34,344,700 by the FCA in March 2019 for failings across 220.2 million transaction reports over roughly a decade, from November 2007 to March 2017. The scale tells the story: around 9.5 million transactions not reported, some 204 million reported inaccurately, and about 6.6 million reported that should not have been. All three error types were present, running simultaneously for ten years. This is not a single defect but a control-framework failure — the estate lacked the change control, testing, and independent assurance that would have caught any of the three problems earlier. It maps not to one stage but to the governance wrapper around all of them, and its lesson is the hardest: reporting failures compound silently at the scale of millions of records, so the absence of a real control framework is itself the risk. A decade of luck ended with a fine.
| Case | Failure type | Stage it maps to | Lesson |
|---|---|---|---|
| Merrill Lynch (EMIR, 2017) | Completeness — missing reports | Submission | Completeness must be a tested control; absence is invisible |
| UBS (MiFID, 2019) | Accuracy — accepted but wrong | Accuracy control | Acceptance is not correctness; sample from source |
| Goldman Sachs (MiFID, 2019) | Control failure — all modes, undetected | Governance | No control framework is itself the risk; luck expires |
Why reporting costs what it does, and why the cost is structural
Reporting is expensive in a way that surprises people, and understanding why explains the whole operating model. The cost is not in the reporting engine, which is a bought commodity. It is in three places that are structural rather than incidental. First, the sourcing and enrichment layer must reach into dozens of systems and keep those connections alive as each of those systems changes — a standing integration burden. Second, the daily operation must be staffed for vigilance every working day, plus the reconciliation and remediation that never fully clear. Third, and largest over time, the permanent change programme must interpret and deliver a stream of externally owned rule changes, often twice over for the EU and UK. None of these is a one-off. A firm cannot buy its way to a finished reporting estate, because two of the three costs are recurring by nature and the third grows as the rulebooks diverge.
The divergence between the European Union and the United Kingdom is the cost driver that is getting worse rather than better. A bank active in both jurisdictions may, in a single year, run an EU REFIT release, a UK REFIT release, a MiFIR-review migration, and a data-quality remediation — four change programmes for what is conceptually one reporting obligation. Every rule change potentially lands twice, in two slightly different forms, against two sets of infrastructure. This is why the change backlog for reporting is effectively permanent and why firms staff reporting as a run-and-change function rather than a project with an end date. For the institution it is a cost; for the professionals who do the work it is durable, transferable demand.
The transformation agenda fixes the plumbing before it reaches for intelligence
Five transformation trends are reshaping reporting, and they have a deliberate order: standardise and control first, automate second. Each attaches to a specific part of the lifecycle, and the sequencing matters — you cannot automate your way out of ungoverned, unstandardised data. The "boring but powerful" levers come before the exciting ones.
| Trend | Attaches to | Why it matters |
|---|---|---|
| ISO 20022 standardisation | Build and matching | Common machine-validatable schema cuts ambiguity and breaks |
| Control frameworks (completeness, accuracy, timeliness) | Reconcile and assurance | Formal, owned controls with KPIs that satisfy the regulator |
| RegTech assurance | Accuracy control | Independent field-by-field testing finds accepted-but-wrong errors |
| Cloud and API-first architecture | Sourcing and integration | On-demand data joins and elastic scale replace brittle batch |
| Straight-through processing | The whole pipeline | Automated, exception-based flow removes manual handoffs |
Standardisation and controls deliver most of the quality gain and are prerequisites for anything cleverer. ISO 20022 gives a common structured data model across regimes and infrastructure, so machine-validatable schemas cut ambiguity and rejection rates and enable straight-through processing and cross-regime reuse. Control frameworks turn quality from a hope into a measured discipline: formal completeness, accuracy, and timeliness controls with named owners and metrics, independent reconciliation against an oracle recreated from source data, and the attestation and evidence that satisfy audit and the regulator. RegTech assurance — specialist tools that test every field against the rulebook — finds the silent accepted-but-wrong errors that validation alone misses. Cloud and API-first architecture replaces brittle overnight batch joins with on-demand, elastic data access. And straight-through processing removes the manual handoffs between the four functions that are themselves a source of error.
Apply a single filter to every proposed transformation: does it change the economics or the risk, or does it just change the appearance? An ISO 20022 migration changes the risk — cleaner matching, fewer breaks, better evidence — so it earns its cost. An independent accuracy-testing capability changes the risk — it finds errors a regulator would otherwise find first — so it earns its cost. A slick new dashboard that reorganises the same unreconciled data changes only the appearance, and it should be treated with suspicion no matter how good the demo looks. The discipline that survives scrutiny asks of every initiative whether it moves a real risk or a real cost. Reporting is full of expensive changes that moved neither, and the filter is how you tell them apart before you fund them.
Data and artificial intelligence help around the pipeline, not inside it
Artificial intelligence genuinely helps reporting, but its right place is around the deterministic pipeline, not as the reporting engine itself. There are real, specific gains. Machine learning can run automated quality checks, flagging outliers and implausible field combinations before submission, beyond what fixed rules catch. It can make reconciliation smarter, using fuzzy matching and clustering to group related breaks and suggest likely root causes to operations. It can do anomaly detection, learning the normal shape of a day's reporting and alerting on volume or pattern shifts that signal a missed population or a broken feed. And large language models can assist field mapping, drafting mapping specifications from dense rule text as a first draft for a business analyst to verify. Each of these is an assistant to a human, applied to detection and drafting, where a wrong suggestion is caught before it does harm.
| Use of GenAI | The gain | The governing risk |
|---|---|---|
| Draft mapping specs from rule text | Faster interpretation of new rules; a BA's first draft | Hallucination — the spec must be verified line by line |
| Anomaly and outlier detection | Catches broken feeds and missed populations early | Explainability — alerts must be traceable to be actionable |
| Break triage and root-cause suggestion | Speeds reconciliation; clusters related breaks | Model risk — suggestions guide but do not decide |
| Confidentiality-sensitive drafting | Productivity on research and documentation | MNPI and confidentiality — trade data must not leak to a model |
The honest caveats are what keep artificial intelligence in its place, and they are non-negotiable in a regulated pipeline. Reporting must be exact and explainable — a black-box model that is usually right is not acceptable to a regulator who expects deterministic, reproducible reports. Artificial intelligence assists detection while humans keep accountability: it surfaces candidates, people decide and sign off, and the human in the loop is a control, not a limitation. Garbage in still wins — a model applied to ungoverned data just produces wrong reports faster, which is why standardisation and controls come first. And any model in the chain is itself governed, tested, and documented under model-risk rules, because a model is not a way to escape governance; it is one more thing to govern. Material non-public information and confidentiality set a hard boundary: the trade data that flows through reporting is sensitive, and it must not leak into a model that was not built to hold it.
5
Which fine illustrates “acceptance is not correctness”?
Completeness.
Right. Merrill Lynch is completeness; Goldman Sachs is absent controls.
Control framework failure.
No.
Why is a low rejection rate a misleading metric?
Right.
It is.
No.
No.
What is the right order for transformation?
No.
No.
There is one.
Right.
Where does AI belong in reporting?
Right.
No.
No.
It helps around the pipeline.
The six ideas that matter most
Idea 1 — The regime defines the pipeline, not the volume.
Whether a trade is a derivative, a financial instrument, or a securities-financing transaction determines which regime bites, who reports, where, and what fields are needed. Understand the regime and you understand the machine; the number of trades is just throughput.
Idea 2 — Data cascades, so errors are never contained.
Reporting sits at the end of the trade chain and invents nothing. A field wrong at capture is wrong in every downstream report and in the regulator's data, which is why lineage, golden sources, and deduplication — the anatomy of the enriched record — are the highest-leverage places to invest.
Idea 3 — Reconciliation is where consistency is actually made or lost.
For dual-sided regimes your report must agree with a counterparty's you do not control, so pairing on the shared UTI and matching on economic fields is the hard part, and your quality is capped by your least sophisticated counterparty.
Idea 4 — Controls are core architecture, not an afterthought.
Acceptance is not correctness; a mature estate measures completeness, validity, and accuracy separately, evidences them, and names an owner. Supervisors now inspect the control framework, and the message standard and resilience that support it, as much as the reports.
Idea 5 — The permanent change programme quietly sets the cost.
Because the requirements are owned by an external party who keeps changing them — REFIT, ISO 20022, the MiFIR review, EU/UK divergence — reporting is never finished. It is funded as a standing run-and-change capability, and that structural fact, not any single rule, is the dominant cost driver.
Idea 6 — Real transformation moves cost or risk, including artificial intelligence with its governance.
Standardisation, control frameworks, and independent assurance deliver most of the quality gain and come first; artificial intelligence helps around the deterministic pipeline, on detection and drafting, governed as tightly as it is useful. Everything else is appearance.
Why are reporting risks described as controllable, and what does that imply for accountability?
Every risk (completeness, accuracy, timeliness, reconciliation, conduct, change) has a defence that is a buildable, evidenced control — unlike market risk you can only hedge. Because the risk is controllable, the regulator holds the firm personally accountable when it materialises; there is no "market moved against us" excuse for a missing report.
Why is a low rejection rate a misleading measure of a reporting estate?
Rejections only measure whether the reports you sent were well formed. They are blind to under-reporting (trades that produced no report) and mis-reporting (accepted-but-wrong fields). A shop that celebrates its reject rate while never reconciling to source or sampling for accuracy is lucky, not well run.
How do the three FCA cases map to the three failure modes?
Merrill Lynch (EMIR, 2017) = completeness, 68.5m missing reports; UBS (MiFID, 2019) = accuracy, ~83m of 135.8m reported inaccurately; Goldman Sachs (MiFID, 2019) = control-framework failure, all three error types over a decade. Each ran undetected for years — reporting fails silently at scale.
Why must standardisation and controls come before artificial intelligence in the transformation agenda?
AI applied to ungoverned, unstandardised data just produces wrong reports faster. ISO 20022 and control frameworks deliver most of the quality gain and are prerequisites; you cannot automate your way out of a data mess.
What is the right role for artificial intelligence in reporting, and how is it governed?
As an assistant around the deterministic pipeline — anomaly detection, break triage, and mapping-spec drafts — not as the reporting engine. It is governed by keeping humans accountable (a control, not a limitation), verifying outputs for hallucination, protecting MNPI and confidentiality, and treating any model as itself subject to model-risk governance. Regulators expect deterministic, reproducible reports.
State the module's through-line in one sentence.
Regulatory reporting turns a trade into a data record and delivers it, complete and correct, to the right regulator on time — a permanent discipline of sourcing, identifiers, controls, and reconciliation rather than a system you finish.
Reporting Cheat Sheet — The Whole Lifecycle on One Page
A single reference for the entire regulatory reporting flow — the stages, the numbers, and the one sentence that holds it together.
The lifecycle at a glance
| Stage | Timing | Core activity | Key output | Lead and systems |
|---|---|---|---|---|
| 1. Execution and capture | Trade time | Book the trade | Source trade record | Front office, booking |
| 2. Eligibility and enrichment | Same day | Decide regime(s); add identifiers | Enriched, flagged record | Data engineering, ref-data, eligibility engine |
| 3. Build and validate | Same day | Assemble and format to schema | ISO 20022 XML report | Reporting engine, ops |
| 4. Submit to TR / ARM | By T+1 | Transmit; receive ACK/NACK | Accepted or rejected report | Operations, connectivity |
| 5. Pair, match, reconcile | T+1 / T+2 on | Repository pairs and matches | Pairing/matching status, breaks | Operations, reconciliation |
| 6. Repair and back-report | Ongoing | Fix rejections, breaks, and history | Corrected reports, added controls | Ops, data, business analysis |
The three regimes in one look
| Regime | Covers | Sided | To | Fields |
|---|---|---|---|---|
| EMIR | Derivatives | Dual | Trade Repository | ~200 (post-REFIT) |
| MiFIR RTS 22 | Financial instruments | Single | NCA via ARM | ~65 |
| SFTR | Securities financing | Dual | Trade Repository | ~150 |
Key numbers and rules of thumb
| Metric | Rule of thumb |
|---|---|
| Reporting deadline | T+1 — the working day after the trade |
| Fields per report | 100+ (EMIR ~200, SFTR ~150, MiFIR ~65) |
| LEI length | 20 characters; renewed annually; must be active |
| ISIN length | 12 characters |
| MIC length | 4 characters (XPAR = Euronext Paris; XOFF/XXXX off-venue) |
| EMIR REFIT go-live | EU April 2024; UK September 2024 (staggered, not identical) |
| Message standard | ISO 20022 XML (EMIR REFIT); SFTR and MiFIR converging |
| Dual-sided key | Shared UTI, generated per the CPMI-IOSCO waterfall |
| The three quality questions | Completeness, validity (acceptance), accuracy |
| Most common pairing-break root cause | UTI not shared or shared in a different form |
| Where the effort peaks | The T+1 deadline and the reconciliation just after it |
| Cross-border multiplier | EU + UK = two rulebooks, two destinations, two go-lives |
The whole thing in one sentence
Regulatory reporting turns a trade into a data record and delivers it, complete and correct, to the right regulator on time — a permanent discipline of sourcing, identifiers, controls, and reconciliation.