What makes synthetic dialogue sound unrehearsed
Two synthetic voices can be individually convincing and still fail the moment they have to talk to each other. The difference is almost never the timbre.
Single-utterance text-to-speech has been good enough to fool a casual listener for some time. Put two of those utterances in sequence and the illusion usually collapses. The voices remain clean, the pronunciation stays correct, and the result still sounds like a public address system reading a script rather than two people talking. The failure is not in the voices. It is in everything that happens between them.
It helps to be precise about what a listener is actually tracking. Human conversation carries a dense layer of timing information that nobody consciously attends to but everybody notices when it is wrong: how long a speaker waits before answering, whether the answer starts before the question finishes, how the pitch of a turn's final syllable predicts whether another turn is coming.
The gap between turns is a signal
Recorded conversation between people who know each other tends to hand the floor over with very short gaps, and a noticeable share of turns begin slightly before the previous one ends. A synthesis pipeline that concatenates utterances with a fixed pause inserts a uniform silence that no real exchange contains. The uniformity is the tell. A listener cannot name it, but a run of identical gaps reads as machinery.
Worse, the length of a gap carries meaning. A long pause before an agreement reads as reluctance. A fast overlap on a question reads as eagerness or interruption. When gap length is a constant, every line is delivered with the same social temperature, which is why generated dialogue so often sounds uniformly polite and slightly dead.
A conversation is not two monologues interleaved; it is a negotiation about who holds the floor.
Turn-final pitch does the handover
Speakers signal that they are finished with a fall in pitch, a lengthening of the final syllable, and a drop in loudness. They signal that they are not finished with a level or slightly rising contour and no lengthening. A model that treats each utterance as an isolated sentence will end nearly every turn with the same declarative fall, which tells the listener that the conversation ended six times and restarted six times.
Systems that condition on speaker tags across a whole passage, rather than synthesising line by line, have a structural advantage here: the context needed to shape a turn's ending is the turn that follows it. That is an argument for generating a passage in one pass wherever the architecture allows it.
Non-verbal sound is punctuation, not decoration
Breaths, laughter, throat clearing and hesitation sounds are frequently treated as novelty features. In practice they carry structural work. An audible in-breath before a turn tells the listener a turn is coming, which is why speech with the breaths edited out feels faintly airless. A hesitation sound in the middle of a clause marks planning trouble and buys the floor.
Placed at random, these sounds are worse than nothing: laughter that does not follow anything funny is a strong cue of synthesis. Placed at turn boundaries and clause junctions, at a rate closer to sparse than generous, they do most of the work of making a passage sound unrehearsed.
Consistency across the passage
The final ingredient is boring and is usually what breaks first: a speaker has to stay the same speaker. Small drifts in perceived age, brightness or speaking rate between one turn and the next are the kind of thing a listener registers as unease rather than as an identifiable defect. When evaluating a dialogue system, it is worth listening once only to the second speaker, and once only to the handovers, because those two passes surface different failures.
None of this requires a larger model. It requires treating a conversation as a single object with internal timing structure, rather than as a queue of sentences that happen to alternate.