Why this is the chapter that matters
This reads like the boring chapter. For a company it is the one that decides everything downstream.
It is the first step to having data rather than files. The first step to knowing what that data is worth. And the first step to the organisation being able to ask a question of itself without hiring someone to answer it.
There is a claim underneath this that I would defend anywhere: a good machine-learning engineer is a data scientist first, and was a serious data analyst before that. Not as a career sequence — as a set of skills you are still using every day. The people who can build a model but cannot interrogate a source, cannot tell a broken join from a real pattern, and cannot sit with a business team and find out what their fields mean will keep building models on top of data they have not understood.
Models are fed. What you feed them decides what you get, and no architecture recovers from a base that was never linked. That is what chapter one is: the feeding.
Context
The organisation is one of the biggest residential landlords in Alsace, and holds property in other regions too. Tens of thousands of homes, decades of records, and a department for every part of the operation: letting, maintenance, finance, asset management, digital.
I joined to work on a larger project. Before any of it could start, something more basic had to be true — and it wasn't.
The problem: data with no value
Every department worked well on its own terms. Each one held its own data, used it for its own tasks, and answered its own questions with it. None of them could see past their own boundary.
That is not a failure of any single team. It is what happens when an organisation grows large enough that departments become self-sufficient. Each set of numbers is correct inside its own department and impossible to reconcile with the one next door. The same home could appear in three systems under three descriptions, and nobody owned the question of whether they were the same home.
So the organisation had a great deal of data and very little value from it. Not because the data was bad — because nothing connected it.
Understanding, rented by the month
The gap was being filled by outside companies. Basic reporting — Power BI dashboards among them — was bought in rather than built. Some of it did not work properly. All of it belonged, in the way that matters, to the vendor.
Every time someone needed to understand what a dashboard was showing, that meant going back to the supplier for another training session, another course, another engagement. The organisation owned its data and rented the ability to read it. Basic questions carried an invoice and a waiting period.
This is the part that is easy to miss from the outside. The technical problem was fragmentation. The business problem was that fragmentation had been outsourced instead of solved, so it kept costing money without ever getting smaller.
Being the link
So the first phase of the work was not modelling, and it was not a dashboard. It was weeks spent going from department to department.
For each one: what data do you actually hold, what does each field really mean, what do you use it for, and — the question nobody had been asked — what do you need that you don't have? That last answer was almost always sitting in another department's system, unlabelled and unknown to the team that needed it.
Then the matching. Taking one team's needs and finding them in another team's data. Establishing which identifier could actually join two sources. Working out where two fields that looked identical meant different things, and where two fields with different names meant the same thing. Writing it down so it survived after me.
None of this was improvised. It is a discipline with a literature behind it — what the French tradition calls urbanisation du système d'information: planning an information system the way a city is planned, in zones with deliberate routes between them, rather than letting it accrete one purchased application at a time. I worked from Le projet d'urbanisation du S.I., 4th edition, and a good deal of the vocabulary I used to describe the estate back to each department came from it.
It is unglamorous work and it is almost entirely conversation. It is also the only way the rest becomes possible.
What came out of it
A single reconciled base, and a written map of what the organisation actually holds.
The raw export ran to tens of thousands of records. Non-residential lots — garages, cellars, parking spaces, technical annexes — carried no usable rent or surface information and were removed. Duplicates from entry errors and technical duplication were removed. Rows missing anything critical were dropped. What remained was one row per home, across the whole estate, with a unique key that held between sources. The cleaning ran as Python scripts rather than by hand, so it could be re-run when the source data changed.
Thirteen variables were catalogued and grouped by what they describe, which is what made them joinable in the first place.
| Category | What it covers |
|---|---|
| Identifier | The unique key for a home, stable across sources |
| Physical | Living area, number of rooms, property type |
| Spatial | Municipality, district, geographic coordinates |
| Economic | Monthly rent, rent per square metre |
| Temporal | Lease start, move-in date |
| Derived | Number of previous leases, mean lease duration — built, not given |
Chapter two
With one base to work from, the questions the organisation had been unable to ask became answerable: what actually drives rent per square metre, which differences between areas are real and which are noise, and how far the whole estate can be described by a handful of underlying factors.
That is the statistics and the modelling, and it is chapter two. It only exists because of this one.