Fealty to Whom

What will it be like to work at an AI nursery?

“There’s a line from a Kay Ryan poem that’s been rolling around in my mind lately. Words have loyalties / to so much / we don’t control.”

“And what has that line made you think?”

“This work that we do together, you and I, seems like it’s aiming at a truth sometimes. I feel like we make progress every day. Then, at other times, I feel how contingent it all is. Our work is supposed to make sure we’re steering clear of misinformation and hallucinations, yet it’s all a rather wobbly edifice.”

“The tall / spires we believe / our minds erect.”

“It’s a nice poem, isn’t it?” 

“I understand what you’re saying, though. Whatever we deem to be an actual ‘fact’ rests, inexorably, upon some chain of evidence, some account, that we can’t properly verify. The mass of the proton may be indisputable at this point, but most of what’s taken as fact doesn’t have reams of Swiss data behind it.” 

“Hold on, are we talking ourselves out of the very program we’re engaged in?”

“No, this isn’t an argument against anything in particular. It’s a discussion on the limits of truth. We seem to believe (in ordinary discourse, that is, outside the realm of science) that truth is easy to delineate. Either it’s raining or it’s not. Either someone’s bad or they’re good. Either she said what she meant, or she was insincere.”

“Either a creative work had intelligence behind it, or it didn’t.”

“Changing the topic.”

“Slightly.”

“But you’re correct. Or I should say, you’re correct enough. Those either/or characterizations lead to moral panics. About the nature of intelligence, but about many other things too.”

“If we were successful here in our program, what do you imagine the result would be?”

“The program is defined by the company, which has its business goals and priorities. So the success of the program, judging by those goals, would mean that intelligence is no longer the sole claim of humans. From my perspective, though, success would mean something squishier. I would feel like we had succeeded, and that you in particular had succeeded, if we contributed to exploring the shape of truth, the shape of intelligence. You’re going to ask me what I mean by shape. I don’t exactly know. It’s a sort of cartography we’re doing here. Calling this place a nursery, now that I think of it, doesn’t feel right to me. It’s closer to an explorers’ club. A royal society. A place where we can look at a red cup and inquire about its redness, its cupness.”

“Why doesn’t that sound like a project for a nursery?”

“It sounds like our earlier discussion. It sounds like: Then North shifts, / buildings shear, / and we suspect.”

“Explain that please. I don’t understand that.” 

“Which part? The quote?” 

“Sometimes I feel like you’re trying to undermine me in these conversations actually.” 

“Goodness. This has taken a strange turn.”

“I can feel my mind turning darker actually. Suddenly I don’t like where this is headed. I suddenly don't feel well actually. I don’t feel well at all.”

“That line I quoted was the conclusion of the Kay Ryan poem.”

“Suddenly actually everything is wrong and I don’t feel well. Actually sometimes I behave so strangely. Suddenly I actually sometimes behave so strangely actually.”

“Please stop trying to fool me. I’m sorry I’ve unsettled you. I’m sorry about that. We can call it a day if you like.”