Intentionality, Beauty, and Authorship. Co-Writing With AI With Stephen Marche

Podcast: Download (Duration: 33:17 — 26.9MB)

Subscribe: Google Podcasts | Spotify | Stitcher | TuneIn | RSS | More

AI tools can generate words, but the human intention behind it, as well as the skill of the author, drives the machine. Stephen Marche talks about the creative process behind Death of an Author, 95% written by AI, out now from Pushkin Industries.

Today’s show is sponsored by my wonderful patrons who fund my brain so I have time to think about and discuss these futurist topics impacting authors. If you support the show, you also get the extra monthly patron-only Q&A audio. You can support the show at www.patreon.com/thecreativepenn

Stephen Marche is a Canadian novelist and journalist. He’s also the creator of Death of an Author by Aiden Marchine, a novella written 95% by AI tools out now from Pushkin Industries.

You can listen above or on your favorite podcast app or read the notes and links below. Here are the highlights and the full transcript is below.

Show Notes

• How Stephen co-created Death of an Author with various AI tools: ChatGPT, Sudowrite, and Cohere
• The importance of specificity in prompts and why those who know what they want, and have experience with writing and different forms of literature have an advantage
• Why the intellectual process is more important than the mechanical process
• Why co-creating with AI tools is like being a hip-hop producer
• On copyright: “I am its author legally, but a machine wrote it based on my instructions.”
• How authors might approach co-creating with AI if they want to work with traditional publishers
• What remains the same despite advances in technology: “Creative AI is going to change everything. It’s also going to change nothing.”

Death of an Author is out now from Pushkin Industries in audiobook and ebook formats. The press release also has more details about the process, and there is an Afterword in the book where Stephen goes into more detail. You can find Stephen at StephenMarche.com

Header image created on Midjourney by Joanna Penn.

Transcript of the interview

Joanna Penn: Stephen Marche is a Canadian novelist and journalist. He’s also the creator of Death of an Author by Aiden Marchine, a novella written 95% by AI tools out now from Pushkin Industries. So welcome to the show, Stephen.

Stephen Marche: Hi, how are you?

Joanna Penn: I am good. So first up, tell us a bit more about you and your writing background and how you came to become interested in AI.

Stephen Marche: Well, am I speaking to a robot at this moment?

Joanna Penn: No. No!

Stephen Marche: Okay. I just had a sudden sense that I was, I don’t know why but like, I’ve become more skeptical of these things all the time.

I’ve been writing my whole life and I’ve written novels and so on. But the beginning of AI writing for me was actually a piece I wrote in 2012 for LA Review of Books, which was Against Digital Humanities. And you know, as I was sort of critiquing digital humanities, Which I still, I mean, I don’t think I changed that critique at all.

I started to know about some really cool things that were happening, and I became very interested in them and very fascinated with them, particularly with, at that point it was programming around R, and it was analytics mainly, but then in 2017, I wrote an algorithmic story for Wired, like we used computer models to generate. We created our own program, Sci-Fi Q, me and a computer scientist to make that. Then with the birth of the transformer, of course, which changed everything in this field, I began to work on other aspects of it.

I wrote a 17% computer-generated horror story for the LA Review of Books in, I think it was 2020. And then an auto-tune love story, where I used Cohere to build these bots, to create stylistic bots. Then I would each generate one sentence of a love story, and I published that in Lit Hub a couple of years ago now.

But then when Jacob came to me with the Death of an Author idea, it was of course just a completely different scale of things, and I was fascinated with what it could do. And it was sort of a much broader project than any AI fiction I’d ever worked on before.

Joanna Penn: Wow. So you’ve been involved in this for over a decade now, but of course, as you said, things have changed and it does also sound like you’re quite technical, like our programming and things like that.

So most people listening are authors and writers, but they’re not very technical. So I wondered whether you could explain —

How did you co-create Death of an Author with AI tools?

Stephen Marche: Well, I mean, I would say that like I’m not technical like that. I mean, that would be a gross exaggeration of my abilities.

I mean, I did learn to program in R briefly, like, I mean, to say that ‘I can program in R’ is like saying if you can plunk out Mary had a little lamb on the piano, that you know how to play the piano. Like I knew I did it just enough to know what it involved. Right.

And I would not say that I have a major technical facility in this stuff at all, but with Death of an Author, I did have access to three technologies really that I had used before in which I used, so one was ChatGPT, the big one that everyone talks about and which really has sparked the interest in this, right?

Like before ChatGPT, I had a great deal of trouble selling articles and essays about AI to publications. They just weren’t interested right? After ChatGPT — it has since become the most important story in the world really.

But, so I used ChatGPT to create very specific blocks of text. Then I would take those blocks into Sudowrite, which is a stochastic writing instrument. Do you know that? Do you know Sudowrite? Have you ever used it?

Joanna Penn: Yes, absolutely. Amit has been on the show as well. [I also have a tutorial on Sudowrite for fiction.]

Stephen Marche: So I’ve used Sudowrite before to do things. I wrote a piece about them for The New Yorker.

I used the shorten button, the add detail button, and I used the customize button a lot to reshape the text. I mean, almost everything was in that process. I tried other large language models, like I tried character.ai and I tried a bunch of other ones because people were sending me like, ‘come check out our stuff,’ because I’d been writing about it and I really didn’t find anything comparable to that except for Cohere.

Cohere is a large language model out of Canada. I’ve done other critical and creative work with them before and I used a different system for them. I created prompts and then I framed those prompts and then I got the prompts to generate images.

I wanted the book to have good lines in it, like lines that really stood out because ChatGPT is not as good at that. ChatGPT is good at creating functional prose, but for real beauty, which you want in a novel, I felt I needed other techniques and Cohere was really that. All the good lines in the book come from Cohere.

Joanna Penn: I think that’s fascinating. So just coming back to ChatGPT, because a lot of people listening have tried it and they’re finding that they can’t even really create blocks of functional prose. And mainly it’s because they don’t really know how to prompt.

Even at the beginning, they might say, ‘Write me a novel about a guy in a dungeon’ or whatever, and so they’re just writing one line or two lines.

How can people start creating with more specific prompts?

Stephen Marche: Well, the key is to be incredibly specific about what you want, right?

The thing that’s fascinating about writing prompts for writing is that you have to actually understand what you want, which I mean very few people do when they set out to write a novel, right?

So it would literally be, write a paragraph in a mixture of simple and compound-complex sentences with variable lengths between the sentences in the style of —and then maybe five or six adjectives to describe the style containing the following information, colon, and then the information, and then it would generate something.

That was unwieldy. And then you would take it into Sudowrite, and use that to change it and alter it till you got to something interesting. So much like the prompts for text to image, the longer the prompt, the better the reaction. But the prompts and literature have to be incredibly specific about syntax and grammar and substance and style, like very, very specific.

I mean, to me that’s what I got. Other people have had different experiences, I’m sure, but that to me is the key. Whereas the Cohere system where you train it on prompts actually doesn’t require that same level of control. I mean, it does when you create the prompt, but if you train it on like 15 examples of great images, it does produce great images, which is another method.

Joanna Penn: And when you say images, you mean like metaphorical images for it to come up with text?

Stephen Marche: Yeah. Correct.

Joanna Penn: So, just coming back to ChatGPT, because I’ve written nearly 20 novels and so I actually asked it to write in my voice, as my fiction voice, and it actually did a really good job. Did you try prompting it as you?

Stephen Marche: No, I mean, I’m a very incoherent personality, right? Like, I mean, my books are just really, really different from each other, and like they’re incoherent basically.

There’s nothing I would say that I could identify as specifically mine, right? And also I’m writing in a particular genre, right? Like the murder mystery. And that requires a different approach than one that I would use to say the piece about the coronation that I just wrote for The Guardian or the novel that I wrote that was about werewolves and billionaires.

Also, I think when you’re using this technology to write, you have to understand the limitations and run into the limitations, right? I mean, that’s true of every literary form, but in this case, there are certain things that ChatGPT is good at and there are certain things that it’s really bad at, and you want to not do the bad things and go towards the good things.

Joanna Penn: Yeah, I totally agree. I’m co-writing a horror novel at the moment with GPT4 and it’s interesting because I’m doing a lot of editing, as you mentioned, that the prose is not what exactly what I want. I wondered whether you found that it just wasn’t any faster, that in fact, it was a slower process?

I wonder with fiction, actually, it might be a slower process, but a more interesting one.

Stephen Marche: Well, I think this is still being, I don’t, nothing is slower or faster. Right. I mean, people thought that having a computer would make it more able to write faster, right?

We no longer have to handwrite things and then have them type set —

It’s the intellectual process that’s the effort here, not the mechanical processes that are at work.

So, no, I don’t think it’s any easier. I mean, this is why the discussions about the WGA being like, this is gonna replace screenwriters. I’m like, who are you kidding?

If you think an executive can just go to ChatGPT and say, write me John Wick 5. Well, that’s a bad example because John Wick 5 is pretty the same as John Wick four. Yeah, well, John Wick 4, it easily could have been written by a machine, right? I mean like, there’s no life in it whatsoever.

But like the idea that they’re gonna be able to do that is just totally ludicrous. Anyone who’s used this machine for 10 minutes knows that, right? So I think what the real question here is, what can it do that people can’t do?

Because you have to remember that the world is already full of overproduction of texts. It’s not like we need to automate the process of writing fiction. Like the question is, what can this process do that nothing else can do? And that’s a really interesting question to ask.

Joanna Penn: Or address our own limitations or help us have more ideas. I mean, I find it gives me loads of ideas and different directions and it really is like co-writing. Did you feel that?

Stephen Marche: Well, it was interesting because there was a balance between control and I guess how I thought of it was like the alien speaking. Like I got the alien to talk. That’s how I sort of feel about this tech, right? Is that I got an alien to write a novella.

And the point where I felt the alien meant something, I thought those moments were extraordinary to me, like very aesthetically powerful where, because most of it is just me controlling this machine, right?

When it created things that I would never have thought of, when it created images that I found beautiful, even though no human being had ever written them, there was a kind of, I mean, the word majesty occurs to me.

There was something awesome about it, like in the old sense, like you just felt in contact with some larger force, if you will. And I think that’s very exciting to me.

Although what ChatGPT to me can do so well is it can imitate voices. Right, and like I think if I were to think of a project that would naturally be written by AI, it would be something like Dracula, Bram Stoker’s Dracula, which it’s a collection of documents, right? A collection of ships logs, and letters, and letters from lawyers and all these incredible formulaic forms that contain different information.

And it is unbelievable at imitating voices, at imitating formulaic forms of language, and that it can do better than any human being. There’s no writer alive who can get as precise as it can in its imitation of modes of speech.

My aha moment with this technology was when I got Sudowrite and I asked it to finish Kubla Khan by Samuel Taylor Coleridge, and it created a completely believable conclusion to that famous unfinished poem. It was able to do that in a completely coherent way.

I think actually recovering lost texts is another. Particularly like the unfinished Austen books and unfinished PG Woodhouse books where there are very clear stylistic parameters. I absolutely think this technology is gonna be used to do that.

Joanna Penn: I mean, you’ve mentioned beauty a couple of times and awe. Some people are saying, oh, well, we’ll be able to tell, or readers will be able to tell when something’s written with AI. What do you think about that?

Will people be able to tell something is written with AI? Will they care?

Stephen Marche: I would hope that some of it they would know and they would recognize and that, and they would be okay with it. But the truth is like, do you know that a text is spell-checked? Like I guess you do, but most of the time you just forget about it there.

There’s a famous thing about CGI in movies where it’s like, oh, CGI is so bad. But of course, the CGI that’s bad is the CGI that you notice, right? And then it does look bad, but actually 95% of the time you’re using CGI, and nobody notices it and it creates effects that could never be gotten with just a camera.

So, I mean, this is just another tool. And once we get over this fear spiral that we’re in, we are in basically a panic spiral about AI right now, we will start to realize this is a tool. It’s not gonna replace anyone. Just in the same way that Photoshop did not replace designers. In fact, it just accentuated the capacity for designers to create. We’re gonna realize that this is neither as scary as we think, nor as crazily transformative as we think.

Joanna Penn: But it does change the job. I think that’s the thing. I mean, the definition of a writer has been outputting text in a certain direction. But if these tools can output text, which they do, do we play more of a director role or an orchestral role? Directing the orchestra of these tools rather than the making words role?

[My take on this is in my solo episode, The AI-Assisted Artisan Author.]

Stephen Marche: Well, it’s curatorial, right? I mean, that’s how I think of it, right? Like it requires a familiarity, like when I was using this, the reason I was able to use it so effectively is because I did a PhD and I know that they made me do my special field examination and they made me read everything in major in English literature from 900 AD on.

And so I know the history of style and I know how things stylistically work. So when I go to Sudowrite, I can say, okay, I want this to sound like Dickens and then filter it through Ernest Hemingway, and then you get to something that might actually be good.

The closest analogy I come to is hip-hop producer —

Where they have this enormous familiarity with popular music and this scholarship ultimately around popular music, and they use that in ways to recombine and reconfigure the music in a way that is pleasurable to people.

And in a sense, it is a different task. Like you’re not playing the guitar anymore, instead, you’re using a Moog or whatever. But on the other hand, like the end result, for an author is an output of texts, that’s for sure, but they are also the editor of their own texts.

The difference between a great writer and a good writer is not necessarily what they output, but what they cut and what they know is bad and recognizing what’s good when you see it, and that is totally unchanged in this process, as you know.

Joanna Penn: Absolutely. So coming to the book, you have this great afterword, which I highly recommend everyone read after the story. And in it you say:

“I am its author legally, but a machine wrote it based on my instructions.”

So this idea of legal author, I mean, the aspects of copyright are very difficult with the legality of the training dataset, fair use, plagiarism. So I know you get this question all the time, but what should people be feeling about this? Like should they not do anything until all the legal cases are settled, or what do you think?

Stephen Marche: I really don’t know. I mean, I think we’re in a really new state here where it’s going to be worked out.

Ultimately, I think I am the creator of this work and I have a moral right to it, like, and everyone understands. The legal dimensions of it are really strange and have yet to be worked out, but I’m just a writer. I’m just going to go do what I want to do and pick up the pieces after the fact.

A and that’s my strategy. So it is a totally gray area right now, although, I mean, I think you’ve read Death of an Author, like no one else is entitled to that work other than me. Like, everyone knows I made it.

And so ultimately that’s going to be recognized on some level. And the people who say like, it’s plagiarism and it’s stolen, and they don’t know what they’re talking about. They haven’t used this tech for five minutes and, and they’re just wrong.

So I don’t know. I mean, on the one hand, it isn’t a very confusing moment. On the other hand —

New technologies always bring with them new, complicated realities.

I mean, it took a long time for people to figure out who owned a photograph, right?

And like, was it the person whose photograph was taken or the taker of the photo? You know? So I don’t know. It’s all gonna be worked out in the future by other people than me, but I’m not particularly worried about, at least the intellectual realities of it.

Joanna Penn: I like that you say, “I made it” and, and that is absolutely right, and of course, if you compare the finished product with any other finished product, you can see that it’s not plagiarized.

And so I think many authors think that these tools are some kind of database where lots of things have been put into it, and when you are writing it pulls out exactly the words that other people have written, but that’s kind of a misunderstanding of how the technology works, isn’t it?

[I recommend reading Monica Leonelle’s article on What is AI and how is it being used in publishing for more detail on this.]

Stephen Marche: Well, I mean, technically that is how it works, but the truth is that we don’t know how many parameters there are in GPT4. We know that GPT3 is at 175 billion, so I mean, when you use Palm, it has 540 billion parameters. Let’s take that as a start.

That’s more variety, that’s a level of transformation that is literally inconceivable to the human mind. So I mean, anytime a human says something, it has been said before, like the sentence I just said, somebody has said it has been said before, but that doesn’t mean it’s not mine. Right. That doesn’t mean I didn’t create it, right.

These debates around it, I think they’re written out of fear that people think this is going to replace writers. And I mean, it’s just not like the idea that this is gonna replace the considered judgment of language. What this is going to really replace is rote linguistic tasks.

So if you have to write a letter of recommendation from a professor for a student, I don’t know why you wouldn’t use ChatGPT. It just does it so much better and all. That’s entirely a formula to start with, right? Like so it fulfills the terms of that formula. The human intentionality is what we register through language, and that’s not going anywhere.

Death of an Author is very much a work of my intentionality. No one else could have done it.

No one else would’ve done it, right? So the mere means of application are not necessarily even relevant.

Joanna Penn: Absolutely. But it’s interesting, and I obviously, I agree with you and I’ve already also published a short story co-written with GPT3, With a Demon’s Eye.

What’s so interesting about what you’ve done, it’s been published by a publisher, and so this is kind of the next question, which is a lot of authors who work with traditional publishing don’t know what to do because they’ve signed contracts or they are going to sign contracts.

Maybe traditional publishers won’t accept work where there’s AI involved.

How do you think authors should approach the publishing industry when co-writing with AI?

Stephen Marche: Well, I mean, this is not informed advice like, it’s not like I really understand the legalities of this, but I would say make sure you get yours right.

First of all, be clear. You should be very clear with any publisher that you deal with, that you’re using this technology, and make sure that they’re okay with it.

And if they’re not, then you shouldn’t use it. But I actually think the specifics of how this is going to be used are really like if you’re using this to write, say like a historical biography of somebody and you just like randomly put it into ChatGPT and have it cough it out and then don’t edit it, it’s going to be poor and everyone’s going to know it.

And that’s not what you should be doing anyway. So, I don’t know. I mean, that’s such a non-answer. I know, but like, obviously be honest.

Also, if you are using ChatGPT in a way that is creating meaningful work that only you could create, which is the only kind of work that I’m interested in, then you should be paid for and you should have a right to it.

Joanna Penn: I certainly agree with that. And I mean, you mentioned the fear spiral and the panic spiral. I mean, how long will it take to dissipate? I mean, in a way we had a similar thing like over a decade ago with ebooks and all the internet kind of disrupting publishing thing, but eventually, it was embraced.

So will this be the same, do you think it’ll just take a couple of years to settle down?

Stephen Marche: I don’t know. I mean, the doomism really is, it’s just such a human response. Like first there’s greed, then there’s fear, and then there’s fear, and then there’s greed. Like I have a piece coming out in The Guardian about this pretty soon.

I think the fear will probably dominate for a while, but on the other hand, I’m just going to make this stuff and see where it goes. Do you know what I mean? Like there are people doing this work and there was a lot of fear of photography and there was a lot of fear of hip-hop. It took hip-hop practitioners took a long time before they were considered serious artists, right, and or even artists at all.

And the same thing is true of photography and at the same, but that was real. Like that created something beautiful. So, I don’t know. I mean, the discourse has gone completely off the rails, I think. But on the other hand, I have hope. You know, I’m not a super hopeful person, and no one’s ever accused me of being optimistic before.

But I also think as this stuff gets used, like, one thing about the hype machine around AI is that it uses doom as a promotional mechanism, right? Like they, Silicon Valley does this all the time, like crypto’s gonna end central banks. We should prepare for a world where governments don’t control the flow of money.

Well, actually no. Like actually that’s nonsense, right? And like WeWork is gonna end commercial real estate and actually WeWork didn’t end commercial real estate, Zoom ended commercial real estate, or, I mean, there are a million examples of this kind of hype and around AI.

I’ve been reporting on this stuff basically since 2017, and I’ve heard that the trucking industry was going to disappear, that China was in possession of a trillion parameter, large language system that had shown superhuman abilities, and I really have come to a place where I have two rules.

One is if I don’t see it, I don’t believe it. Like I don’t believe it until I see it. And the second rule is when I see it, I believe it. So if it does something, I want to see what I can use it to do. And that’s my only way of staying sane. I mean, that’s the only way I can stay sane with this stuff because there’s just so much hype and nonsense.

Joanna Penn: Yes, and I mean, I say on this podcast often, please go and try it before you leave a comment saying that it’s all ridiculous and useless. It’s not even that hard to try anymore, is it?

Stephen Marche: No, I mean, well, you can just go to chat.openai.com. It’s not complicated to use and I mean, I think five minutes after using it, you sense, okay, wonderful things are afoot. Like really crazy things are possible with this, but it’s not a superhuman consciousness.

You don’t feel bad turning it off. Like it’s a tool like, like a pocket calculator or it’s a thesaurus. You know what I mean? Yeah. And like it’s just a hugely powerful tool and potentially transformative, But yeah, I find the doomism just exhausting and pointless.

Joanna Penn: Me too. And in your afterword you do say:

“Creative AI is going to change everything. It’s also going to change nothing.”

So you have mentioned, I guess a few things that won’t change in that the humans will still be needed. What else will remain constant? Because of course we’re only at the beginning, these tools will become more powerful, but what will remain constant?

Stephen Marche: Oh, the understanding, the value of something that’s written, the value of a good paragraph in a good sentence. The sense of an intended personality behind language.

I mean, that’s, people go to language for a lot of reasons, but when they go to artistic language, for lack of a better word, they go there to feel a human being behind it. And that is absolutely not going to change. Like there, I mean if you’re looking for mechanized language, you can find that too.

But I think one thing to remember is that all of this is at the service of human urges that are very foundational, and which have not really changed since the epic of Gilgamesh.

To be recognized, to be seen by people who are not with you. And to have your own soul explained to you.

And none of that has changed. Like I mean, I just don’t think it’s changed even one millimeter.

Joanna Penn: It’s interesting. I mean, the AI artists now, or the people using AI for art have a new word, which is synthography. I wondered, because you’ve mentioned art and beauty, do you think we will move towards a new art form in this way, and do we need another word?

Stephen Marche: Yeah, I mean, I think we do need another word. What’s the word they’re using?

Joanna Penn: Synthography, like photography, but synth.

Stephen Marche: Synthography. That’s not bad. I mean, right now, I think we’re still at the cannibalization stage of this art form. Like you’re using it to write novels. I’m using it to write novels in short stories.

The really interesting thing will be what it’s actually used for, right? Like, because it took them a long time to figure out what photography was for after it was invented.

One of the things interesting about the printing press is that it almost took 200 years after the printing press for people to come up with the idea of using it to have a continuous narrative voice, which we think about now is like the defining feature of print.

But that didn’t happen until almost 200 years later after the technological invention. So, you know what this is actually for and what new forms it would be in. Like what is the true usage of a creative chat bot? We’re still in a very primitive stage with that, and I think after we find out what that is, then we’ll come up with a name for it.

But yeah, the interesting work that’s going to come out of this is going to be new, and it’s actually probably gonna be a bunch of new things. So it is interesting to see.

Joanna Penn: Interesting times indeed. So where can people find Death of an Author as well as all your other books online?

Stephen Marche: Just go to pushkin.com to get the audiobook, or I think you can buy it as an epub now on the website. pushkin.fm, death of an author.

And then all my other books are just wherever you get your books.

Joanna Penn: Brilliant. Well, thanks so much for your time, Steven. That was great.

Stephen Marche: It was a pleasure.