What We Learned About Follow-up Questions and What's Next

To that end, we will be releasing the ‘Say Thanks’ feature at a later date once we have made some additional improvements to it. For instance, creating a direct path to give Thanks, such as its own button.

That defeats the purpose. You want to reduce the number of "thank you" comments. A separate button doesn't really work that good; and we've been down that road: Feature test: Thank you reaction

Feedback throughout the experiment made it clear that the multi-item menu on the comment link was not ideal.

Yes! I was amongst users that did not like the pop-up menu for the comments. I still don't like it. But, you're missing the point. An established user (let's say 500 reputations, for lack of a better metric) does not need to deal with that menu and its side effects such as broken userscripts and keyboard shortcuts. Yet, a new contributor can benefit from it. They will learn about "thanks" being considered fluff, they will know about voting (if they have gained the privilege, that button would add an upvote for them), and more.

That little annoyance (little, because they're not leaving as many comments as a moderator for instance), is worth it.

Do Stack Overflow Staff have any interest in helping clean up the mess that was created in the Discussions section of the site? I understand that there are 1,000's of flags in the queue, now, due to the volume follow up questions that were generated in there; and I don't even know if everything got flagged that should have (I know I personally gave up flagging).

It would be nice of Stack Overflow help clear up the mess that they helped create; my estimate is there's probably at least 500 posts that need deleting (discussions was sitting at a little over 600 post pre experiment, and it's now at over 1,250).

I think there is a decent chance that I might have totally messed up your experiment. Let me explain.

As you are clearly well aware, there are a lot of "thanks" comments. Many of them are really obvious, and I figured I could make an SEDE query to find them with more or less 100% accuracy, and even find only ones that were nuked with only 1 flag, to avoid bothering the mods.

Turns out I could and I pretty quickly did. I then used said query to flag thousands of comments, and I also linked many other users to the query, who likely used it to flag many more comments themselves.

Based on my work with my comment bot, I'd estimate there is roughly 9,000 or so "No Longer Needed" comments a month, of which a significant amount if not a majority are not "thanks" comments (ex. "You're welcome" or "Please upvote").

I am aware of (roughly) what you were looking to figure out how "thanks" comments were affected (although that was told to me in a private chat by a staff member, so I think I'd need permission to post it publicly here) and I'm also aware of your cutoff data for the baseline (also told to me in the private chat), which unfortunately happens to be roughly around when I stopped doing this. Based on what I know, it seems its quite plausible that my comment flagging alone accounted for most or all of this 10%, and that's ignoring the unknown number of flags cast by users I shared the query with. These comments likely wouldn't have been flagged/deleted otherwise. Comments are so hard to moderate, particularly "No Longer Needed" ones, that the vast majority seem to remain undeleted, while my query makes it quite easy for someone to spend 10-15 minutes and flag 100 comments daily.

I did alert a Community Manager about this, so did you wind up doing something to fix/avoid this problem?

For follow-up questions, from an askers point of view it could be quite difficult for them to tell what they should really be doing. What I mean by this is that a user could have a follow-up question for several reasons, such as (but not limited to):

1. The question they asked wasn't quite complete, and missed out some context; this doesn't change the question entirely, but does impact the solution(s). For example they may have omitted information on (some) edge cases.
2. The answer is actually just plain wrong, but the reader of the answer lacks the ability to identify that
3. The reader is in a "me too!" position, and failed to implement the problem properly
4. They genuinely do have a follow-up question; perhaps because they have implemented that solution and are now stuck on a latter (but related) part.

These scenarios likely all need different actions by the reader, and not all need a follow up question. Using the above number points, my expected behaviours would be

1. 1. Provided that the change doesn't significantly change the asked question edit the question to improve it and provide the missing context. If an existing answerer chooses to update their answer is up to them.
   2. If the missing context is significant, a new distinct question should be asked. (I've seen many times a user repost the entire question with changes, which often results in duplicate closer). The answers to the existing question should be judged on their attempt to answer the asked question (not the unasked question)
2. Downvote the answer (if able).
3. A comment may be appropriate. Follow-up questions, depending on the users attempt, can often result in duplicate closure as they don't actually link to the answer they tried to implement (incorrectly), and then get pointed back to the post that they failed to implement. Ideally, the user should cite the original answer, demonstrate their attempt and explain why it didn't work with a full MRE.
4. Ask a new question.

All of these different actions need good awareness of how the site, community, and curators work. If we do want follow-up questions (which I'm not against), I think that the user should be directed on being asked why they want to ask a follow-up, and this should differ depending on if the user is the author of the question or not. This means that the user might be directed to comment, or downvote, rather than asking a follow-up question. If they are directed to ask a follow up question, then a box like shown in Discussions in the new question, linking to the answer they were looking at, is important. If it's a "Me too!" question, they should also be directed to included their attempt, and explain why it didn't work, as this will help users see that their question is distinct enough. It could still be a simple error, but at least it's not a "I'm having this problem too, but the answer didn't work" type post.

This dialogue could be presented a little like the flags screen, and then leads to a (more) bespoke ask wizard or advises the user on the "correct" action, such as "Please consider editing your question to include inforamtion on the edge case and leaving a comment to let the answerer know you've added some additional context". If they are sent to a question wizard it should include appropriate prompts for the user to ensure that they demonstrate the relevant information, based on their follow-up question "type."

In total, we saw a ~10% decrease in comments that included some form of 'thanks' from Feb to Mar 2025. To that end, we will be releasing [...]

How did you run your numbers? Did you normalize for the number of days per month? Did you normalize against overall change in influx of comments? (which, by my calculation was about -8.08% for february).

Currently, if you have a question that builds on or is related to an existing Q&A, there isn't a straightforward and clear path for new users, and sometimes even experienced users, to ask that question.

No, I'd say it's been pretty straightforward. Just make a new question post and link to the related one, explaining the connection in words as needed. The Linked Questions section shows the relation on both sides. But this experiment made things more complicated and confusing by suddenly throwing discussions into it.

The adoption we saw was higher than anticipated, reaching 1200 uses per week, which is a bit more than one and a half times the expected 750 uses....

That doesn't tell me much because I have no idea what you expected nor do I assume that my expectations would be the same. I would be more interested in if the total rate of thanks increased. You report 10% less thanks in comments and these 1200 additional uses per week correspond to how many percent? Please provide either only absolute numbers or only rates (per total number of comments I guess) or both but not a mixture of them.

All in all, if that feature is used 1200 times per month and reduces some (don't know how many) comments and has its own button, I would accept it, even though there is no real benefit to it (it's not a vote). But in general I still think this feature is not needed and distracts more than helps. I won't use it and vote instead.

Btw. did voting suffer in any way by people thanking?

Regarding the follow ups: they often read like comments that should be comments not follow-up questions.

• "Sharing a slight variation to the provided answer (14 times)." Either put as new answer or as edit to the answer.
• "Asking "How does this work?" or seeking clarification (10 times)." Would be best directly under the content it is concerned with.
• "Stating "I still need help" or noting the answer didn't work/is outdated (8 times)." Answer didn't work is useful directly below the answer. Still need help can be anything really.

And if there really is a follow-up question, why not ask it as a question and refer to the original? That could be done already in the past. If they just end up as discussions I fear they might take away some potential Q&As. Everything that can be a Q&A, should be a Q&A. Discussions or chats should be second level to that (just my opinion).

Discussions would be fine, if curation would be possible of the discussions. So far it's not well supported as far as I know. This seems like a big gap. Did you maybe check how the created discussions fared? How many of them resulted in some sort of satisfying conclusion?

But as long as there isn't another extra menu just to comment I'm fine with everything.

I'm concerned about encouraging follow-up questions. I don't mind clarifying some detail in my answer, and of course we all should repair any errors when they're pointed out. However, writing an answer should not be an open-ended task.

If someone has a related / follow-on question they can ask a fresh question, linking to any existing questions or answers that they feel are relevant. If they also post a comment on my answer asking me to take a look at their new question, that's ok, but I'm not obligated to answer it, or even read it.

This is closely related to the issue of chameleon questions. IMHO, encouraging follow-on questions has to be done with caution. Otherwise, it could encourage "help vampire" behaviour.