Training shapes a model’s general behavior; product design supplies the instructions and tools around it; the conversation supplies the question and the evidence you choose to include. Agreement can be encouraged at more than one stage, and a pleasing answer does not tell us which stage was responsible.
The reward channel teaches these systems what “good” looks like — and its analysis of human preference data found that responses matching the user’s views were more likely to be preferred. Persuasive presentation sometimes won over a correct answer. Preference-following can be measured and reduced through training; whether better replies improve what people do with them is a separate question.