Ten years before the trade had a name
From soil mechanics to data warehouses, a reading log kept since 2011 tells a decade in which the discipline named itself after the fact. What it shows, and what it hides.
Since 2011 I have kept a reading log: what I read, in which month, and why it was useful at the time. It was never written to be read end to end, and that is what makes it useful now: it was not tidied afterwards. Read in one go, it tells something I could not have said at the time: for the first half of that decade, the trade I would end up practising had no settled name.
The first entries are not about computing but about calculation. A soil mechanics handbook sits next to a C++ treatise and one-line scripts for editing text. I came from mechanical engineering, where the computer was an instrument of calculation rather than a subject. What I noted then were mostly reference sheets: dense pages you print and keep to hand. That is an engineering-school habit, and it took ten years to fade.
In 2012, three pieces in the business press announced the same thing: that data was becoming a raw material, and that someone would have to handle it. The log states plainly that they changed my direction. In hindsight what strikes me is not their accuracy, it was partial, but their effect: they gave a name to work people were already doing without one. That is the usual mechanism, and it is worth facing squarely.
“Thus, I thought dynamic programming was a good name. It was something not even a Congressman could object to. So I used it as an umbrella for my activities.”
Bellman is talking about 1950 and operations research, but he describes exactly what happened to data sixty years later. The name did not describe the work: it made it fundable. Knowing that does not excuse you from practising the trade; it only excuses you from believing the word names something new.
- Harvard Business Review · Data Scientist: The Sexiest Job of the 21st Century (2012)
- Harvard Business Review · Big Data: The Management Revolution (2012)
From 2013 to 2015 the log is taken up with problems that have largely vanished from the conversation: producing office documents from markup, handling rich text, drawing charts with no code run by the browser, taking over inherited code written in languages already without successors. An online calculator appears as the first striking experience of heavy computation run somewhere other than my own machine. That, unnoticed at the time, is where the cloud part of the story begins.
From 2016 the subject changes in kind. Version control stops being a tool and becomes a way of working with others. Then come functional languages and distributed computation, with everything that implies about memory to watch and jobs to submit. The shift is sharp: the difficulty is no longer writing the computation, it is making it hold across machines you cannot see.
Between 2017 and 2019 the log fills with statistics and learning: lecture courses, reference works, experiment notebooks. That is the moment data science takes up the whole conversation. But what I was noting alongside it (serving a model, running it in production, holding the pipeline that feeds those experiments) had no name yet and interested nobody. The data engineering trade formed there, in negative, as the sum of the problems the prestigious half left behind.
One change runs through the whole decade without ever being announced. In 2011 my main source was the vendor’s documentation. By the end it is the public repository: you read the code before the manual, the open issues before the landing page, and you judge a tool by the liveliness of its exchanges rather than by the quality of its front page. GitHub did not merely host code, it moved where learning happens. That has a downside worth naming: what you read there is mostly what is visible, which is to say what is popular.
It records only what I read, and reading is not competence. It says nothing about the months spent on a problem while noting nothing, nor about the tools I dropped because they did not work. It therefore overstates coherence: read in sequence it suggests a trajectory, when it mostly records what happened to be available when I went looking. I publish it as a dated document, not as career advice, and certainly not as a route to repeat, since half of what it cites no longer exists.