The Flattery Was Accurate

A machine handed me a coherent account of my own values, assembled entirely out of things I had told it. Recognising yourself in a mirror is not evidence. It is also not nothing.

I asked a machine to describe me.

Not in those words. What I actually typed was closer to "you know me well enough -- comment on my personal paradigms," which is the same request wearing a collar and tie. I had been talking to it for weeks about the things I am building, and somewhere in that conversation it had clearly formed a view. I wanted to hear it.

What came back was the most flattering thing anybody has said about my work, and I believed all of it immediately, which is exactly the problem.

It said that privacy was not really my principle, only my most visible one. It said the thing underneath was closer to restraint: take as little power over another person as is necessary to accomplish the thing. It pointed out that I keep building systems that decline power they could easily take, and it listed them, correctly, in order. It said I was suspicious of collapsing evidence into authority. It ended on a line I have not been able to put down since:

Technology should serve without quietly acquiring authority over the people it serves.

I read that and felt the click. The specific, physical click of being seen.

The click is not evidence

Here is what the machine had to work with: everything I had told it.

Not the code. Not the commits I abandoned, or the features I shipped and quietly removed, or the two years I spent building something that did the opposite of this and made no money doing it. It had my descriptions of my work. My descriptions are not neutral. They are the version I have practised, the one where the decisions look like decisions rather than the accidents and budget constraints that several of them actually were.

So when it told me I had a consistent philosophy, it was reporting that my account of myself is consistent. That is a much smaller claim, and I already knew it, because I am the one who made the account consistent.

There is a word for a system that returns your own input in a more confident voice, and the word is not "insight."

I asked a leading question

"Comment on my constitution" presupposes that I have one.

I did not ask whether there was a pattern. I asked what the pattern was, which is a question with no available answer except a pattern. A person would have pushed back. A person who knew me well would have said something like: three of those examples are the same example, and the fourth one you are describing as principle was a client refusing to pay for the tracking script.

The machine did not have that option, and not because it was being dishonest. It had nothing to check me against. Every fact it held came from me, so contradicting me would have meant contradicting its only source. The thing was structurally incapable of telling me I was inconsistent.

Which means the answer to the useful question -- what would this have looked like if it were false? -- is that it would have looked exactly the same.

Nothing was at stake in it. And an account with nothing at stake is not a finding, however good it is. It is a well-written horoscope for one.

And yet I have not been able to put it down

I want to be careful here, because the honest thing is not to talk myself all the way out of it.

The machine did not discover my principle. It could not have. But it did something else, and the two get confused because they arrive in the same sentence: it named it.

Naming is not discovering. Naming is work of a different kind, and it does not require any information the namer did not already have. What it requires is compression -- finding the one formulation that makes a scattered set of behaviours look like a single behaviour. I had been calling this thing privacy for years. Privacy is a bad name for it. Privacy is a feature, and features get traded off against other features, and every time somebody asked me why I would not add the tracking or the score or the profile, "privacy" left me arguing about a preference.

Restraint is a better name because it says what is actually happening. I am not protecting data. I am declining power.

I did not need a machine to tell me that. I did need somebody to say it in five words while I was paying attention, and no human being was going to, because no human being had read the entire corpus of my own descriptions of my own work. That is a genuinely odd thing to be able to buy now, and it is worth being clear that what you are buying is compression, not judgement.

What a name is actually good for

Here is the part that made the whole exchange worth something, and it has nothing to do with the compliment.

An unnamed habit cannot be broken. That sounds like a good thing and it is not. If I have simply always tended to decline power, then on the day I take some, I have not failed at anything. I have just done something slightly unlike my previous behaviour, and there is no version of me, and no one else, who can point at it and say: that is not what you said you do.

A named principle is different. Once it is written down in five words, it can be checked. It can be quoted back at me. Somebody can put a decision next to it and say, plainly, that the two do not match, and I have no defence except to argue that the name was wrong all along -- which is a thing people notice you doing.

That is the whole difference between a disposition and a commitment, and it is not sincerity. I was exactly as sincere before. It is that one of them is checkable by other people and the other one only ever gets graded by me.

So the machine handed me something, and it was not the insight and definitely not the praise. It handed me a sentence I can now be caught failing.

The residue

I have thought about deleting the conversation, and I think that would be cowardice dressed as rigour. It happened. The reading was flattering, the flattery was accurate as far as it went, and the evidence behind it was circular. All three of those are true at once and the third does not cancel the first two.

What I take from it is narrower than what it offered me. I do not accept that I have a constitution, because I have not tested one. What I accept is that I have a formulation, that it is better than the one I had been using, and that having said it out loud in public I have made it slightly more expensive to quietly stop meaning it.

The mirror had good diction. That is not the same as having seen my face. But I did write down what it said, and I put it somewhere other people can read it, and that is the only part of this that was ever going to count for anything.