Evolving comments: An experiment to encourage engagement and follow-up questions

Despite how the current chat system is implemented, this should be handled as a feature of the "third space", ref. Do Trilogy Sites Need a "Third Place"? instead of only working on the "comments space"

The "comments space" is intended to improve posts. It might be convenient to have a specific post from the staff bout this space name and purpose.

Anything else beyond improving a question or answer, or asking for clarification, should not invade the "comments space". However, there should be a way to "move" or refer a misplaced post from one space to another.

Reference

• How do comments work?

Has any thought gone into how this new design will affect the handling and perception of duplicates?

I would generally expect dupe handling to not change due to this, but it’s certainly possible that people will use “it’s not an exact dupe but a follow up Q&A on this other question answers your question” as a means of closing questions. There’s also the possibility of follow up questions themselves being duplicates, which may need their own process of being handled.

How does this integrate with chat?

Right now, comments and chat are sort-of one thing. Comment enough, and the system suggests moving to chat. However, chat's threading works by having messages begin with special :nnnn codes. On the other hand, chat messages are quite limited when it comes to content (though not as limited as current comments). Are you intending to integrate this properly?

Per How might Chat evolve? Help us identify problems and opportunities, you want to increase the visibility of chat. However, comments-moved-to-chat is one of the few things that expose Chat to the common user. Removing this integration would be a loss for Chat.

Collapsing comments (and answers?)

I think all comments should be collapsed by default irrespective of this experiment. Comments make it more difficult to get from question to first answer, and from first answer to second answer. If this is the only outcome of the experiment, I would be happy. Come to think of it, maybe in cases where there are more than 5 answers, answers should be collapsed as well. For this post, there was no efficient way to scan all the answers.

Visibility of posts and comments

Right now, every time a new answer is posted, the question moves to the top of the queue. If someone asks a follow-up question to an answer in a comment according to the new model, only the author of the answer (and the OP?) sees it.

I would encourage users to ask a new question, referencing the old answer, to get access to the entire community expertise. If they want to, they can add a comment to the old answer to specifically alert the answerer. However, the person who answered does not have a special responsibility to address follow-up questions for the next 10 years.

If you were contemplating to have new comments move questions to the top of the queue, that would probably bury new questions, so it would be a bad idea.

As an aside, sometimes questions move up the queue because someone posts a spam answer. After the answer gets deleted by moderators, the question remains high in the queue. It shouldn't, though, because there is nothing new to see.

Turning comments into new questions

How about a mechanism that detects follow-up questions. It would open a dialog "Would you like to ask a new question". It could auto-generate a link to the answer, and auto-generate a comment below the answer linking to the new question. This would be similar to the moderator turning a short non-answer answer into a comment, but in this case, the user would make their own determination and save some typing time.

Our primary metric will be an increase in constructive engagement within comments. We will also closely monitor comment flags and deletion rates to understand the impact on content quality and moderation load.

I think it's pretty obvious that this metric will improve with the new system, so it seems that by using it you are not really constructing a fair experiment. Will the number of answers decrease as the number of comments go up, with answers being replaced by comments which are harder to moderate?

A metric that will probably increase (but is not easy to measure) is the 'average time spent reading before I get a useful answer'.

As this feature got rolled out to me just now, the comments are by default expanded not collapsed - in contrast to what the post states (bold is my emphasis):

Please be aware that the screenshot below is a preliminary view and the final version may differ. The comments are expanded here just for demonstration purposes. Default would be to have them collapsed.

So they are not just expanded for demonstration but also for testing purposes?

My gut reaction to the feature is disgust, mostly because of the amount of space it takes. There might be good things hidden, I can't tell because of the overwhelming negative emotion.

Currently, there is an "Add a comment" link for the comment feature, intended to ask for clarification and help improve a question and answer.

For this experiment, instead of replacing the "Add a comment" link and mixing it with other types of content that will be allowed during the experiment, consider adding a new link next to it. Make it clear that it is part of an experiment and will follow the experiment content policies, along with details about what those policies are.

The regular comments should keep the "No longer need" flag.

If the above is not possible, please don't allow replies to old comments.

A specific usage scenario comes to mind that doesn't seem to have a good solution. Take the following example of a response to a question about turning on a compiler's optimizations:

The answer contained an error, which was pointed out in a comment. The error was corrected in a reply to the comment, but not edited into the answer itself (common practice on discussion forums).

This leaves both the question asker and other readers in a confusing position about what to do next:

• Upvote the question/mark as answer - seems wrong because the answer itself is demonstrably incorrect and doing so would be misleading or harmful to future readers.
• Downvote or ignore the answer - also seems wrong because there is indeed a correct answer here, if you dig far enough down to find it.
• Upvote the comment but not the question - makes it harder to find accurate information. Should an answer with 30 upvotes rank higher than a 35-upvote comment on a 1-upvote answer? This gets complicated very quickly.
• Post a new answer that contains the corrected information - would essentially be stealing someone else's content and is liable to get flagged/deleted.
• Edit the correction into the answer, flag and delete the now-obsolete comments, and then upvote the question - lots of extra work for editors/moderators.

The last option is the only one where the question and answer end up in the desired state. It's also the least likely to be done since it's a multi-step process that takes time to complete. There's nothing preventing a user from doing any of the other options, and some of those can leave information landmines for future readers.

Comments already have this problem somewhat, but this change in its current form will make it much worse. Discussion forums typically frown upon editing existing posts (to avoid distorting the discussion) and encourage updates to be posted as conversational replies. People coming from those worlds will find it much easier to use this site incorrectly unless additional guardrails are put into place.

Also consider how this will impact SEO. Currently, when I do a general web search for a programming-related question and a StackOverflow page is in the top 3 results, DuckDuckGo will show a sidebar with a preview of the top-rated answer. This is by far my most common entry point to SO, and thanks to that helpful preview I'm significantly more likely to click that SO link than I am some other site where I have no idea if the page contains the answer I need. When meaningful content gets buried in discussion threads, these sorts of previews become difficult. Discussion requires context to understand. Machines aren't good at handling that, and neither are humans who are using a language they're not fluent in.

I don't know what the best way to reconcile this would be. At a minimum, we'd want a new way to flag a question as a "landmine" (meaning there's something buried in the comments that could be dangerous if a reader doesn't know about it). Setting this flag would need to do a couple of things:

• Send a special type of notification to the answer author politely asking them to edit the information into the answer. You'd need to have some way to specify a comment so that the answer author knows where to look.
• Provide a streamlined workflow for editing the question and cleaning up the obsolete comments all at once and without moderator involvement
• Add some sort of obvious visual styling change to the answer, alerting readers that this answer is not what it seems
• Add the answer to a separate "landmine" review queue so that high-rep users can easily find and fix them

That only helps fix these problems after they happen, though. It would be far better if we could prevent them from happening in the first place.

What I don't understand here is... Why are we changing the UX here at the same time that we are experimenting with allowing longer-form comments?

UX changes are always going to get blowback and negative feedback because either they break someone's workflow, make information someone thinks is more important harder to access, or people just don't like the new look and think it's crowded. But the whole point of what you're doing here doesn't even need a UX change, you could have experimented with expanding the content allowed in comments without moving to this new UX at all.

These several initiatives did not need to be combined and likely would have been received/responded to better if they weren't. You could have simply replaced the commenting editor with the stacks editor, increased the allowed length, then when displaying comments implement a "show more" button that becomes available when a comment takes up more than a set height. The curation buttons didn't need to move, we didn't need a new vote button, we didn't need more whitespace, we didn't strictly need threaded functionality (but that too could be tested without the UX update,) it just feels like you're trying to do too much.

Soon we will be launching an experiment on Stack Overflow to enhance the quality and usefulness of comments.

That's good... I guess? I recall that the previous recent comment experiment went really poorly. Given the feedback for example here, the only sensible thing is to delete that whole previous experiment, since every single thing about it was poorly designed by people who don't even understand how the site is used.

So that doesn't exactly build confidence in a larger experiment. Please make sure that this is an opt-out experiment.

threaded replies and code formatting

That's great and long overdue!

This experiment represents a significant shift from the long-standing philosophy that comments are ephemeral and intended solely for post improvement or clarification.

That's probably ok but maybe we could at least design in some mechanism against what common misuses of comments: detailed follow-up questions or "I'm having this problem too". Perhaps a feature can be made so that you can click on some comment option "ask a follow-up question" or "I'm having this problem too" and then it takes you to ask a new question. With a link back to from where the question originated.

Please be aware that the screenshot below is a preliminary view

That looks pretty good! But I'm missing a down vote button or another way to deal with blatantly incorrect, harmful or misleading comments without flagging for moderator attention. Moderators have better things to do and often lack the domain knowledge to deal with such flags anyway.

Overall I feel like a comment system overhaul could create more moderator work, so maybe there needs to be a mechanism for trusted users to deal with bad/incorrect comments, like a 3 user consensus delete vote (15k or 20k rep required, something like that?). Or maybe related to tag badges.

This would mainly focus on incorrect comments. Things like CoC abuse should obviously be dealt with by moderators still.

Asking specific follow-up questions about the post.

Again, I don't think this should take place in the comment field, but through a link to a new question, with some manner of relation between the "parent question" and the "child follow-up question".

Overall, I really like the premise of this idea. Long overdue, in my opinion.

• Asking specific follow-up questions about the post.

We have a line to walk here, because I believe the phrase "follow-up question" can be ambiguous. A follow-up question could be asking for clarification about post (the original intent of comments). A follow-up question could be interpreted by people as the equivalent of clicking the "Ask Question" button on the SO home page. We saw LOTS of this in the Discussions experiment where discussions were started that should have been legit Q&A posts on SO.

I still think "ask a follow-up question" is a missing workflow linking a new question to an old question or answer. I think a lot of noise will be introduced with this aspect of comments.

• Seeking clarification on how an answer works or why it might not work for you.

Yes! Needed for a long time! Good stuff to have. Again, the challenge is in the presentation to the user. How do we, as people seeking the answer to a question, cut right to the chase — how do we go past the comments to get to the answer? This is a UX issue, in my opinion, and not a fundamental flaw with the premise of this experiment. UX issues can be sussed out with future revisions to this feature.

• Sharing variations or related experiences pertinent to the Q&A.

This is an unheralded path to getting an answer. Countless times I've stumbled across a Q&A where the comments actually answered my question, because someone posted saying the answer almost worked for a similar problem.

• Engaging in constructive, technical discussion sparked by a question or answer, even if it explores associated concepts.

This is the "community aspect" of comments. I like this idea. Again, though, we need to present the Q&A first. The UX treatment for comment threads should be easily accessible, but downplayed. I still want the Q&A, but I also want a quicker and easier segway into these deeper discussions which are interesting and can be insightful in unpredictable ways.

What will happen to the content when the question or answer is deleted?

The normal behavior is:

• Deleting a question will also delete its answers and comments.
• An answer creator can delete the answer unless it is the accepted answer.
• This might be an edge case, but it's plausible that the question creator deletes the question even if it has answers, if the answers don't get upvotes.

References

• How does deleting work? What can cause a post to be deleted, and what does that actually mean? What are the criteria for deletion?

Markdown

With this fancier formatting, please add a "view source" button, or something like that, to view a comment's Markdown. That will enable converting comments to answers more easily. Currently, with the simpler formatting, it's easy enough to do it by hand, but with fancy formatting it would be more difficult.

You could even add a "copy this comment to an answer" button, or something like that, which could automatically copy the Markdown, put it in a blockquote, and add a link to the comment and the commenter's name (in order to fulfill the attribution requirements). In practice that could look like:

Comment:

Comment with Markdown – Username May 4 at 17:03

Copied to answer:

Comment with Markdown

– comment by Username, May 4 at 17:03

This is similar to "promote comment to answer" like Mad Scientist suggested.

Background

I do this pretty often on Stack Overflow when I see someone who should know better posting an answer as a comment. I'll copy the text verbatim, add the formatting, put it in a blockquote, and add the attribution below, then post it as a community wiki answer. If it needs any edits, I'll wait 5 minutes to let the grace period expire then edit it; sometimes if the edits are different enough, I'll remove the blockquote and move the attribution into the edit summary.

Sometimes I use Paste to Markdown to automate copying the formatting, but it's not perfect.

Another suggestion: reply-to commenting, basically automating copying the user's ID into an @ reference at the start of a comment. Yes auto complete helps but is still a more manual task than is really necessary. Would require adding a "reply" link to the comment's attribution, which some may consider excessive additional bulk.

(Comment threading might also be good, but would require more changes to the system.)

As pointed out the design has the issue of taking up quite some space vertically. As scrolling should be minimized maybe comments could be moved to the right side?

The hastily put together mockup below shows how this might look like:

In this design I respected the content-boundary set for general content. Perhaps the whitespace to the right couldn't be used - in the desktop configuration at least? Nested comment threads might become unreadable otherwise.

Code block formatting is long overdue, especially for Python (which uses significant whitespace). Until now, we've had to use creative workarounds, like, say this is the code I want to post in a comment:

from sys import version_info as v
if v.major > 2:
    print(v.minor)
print(v.micro)

This can't be put in one line, so the only option is to break it up in a comment, which is ugly, e.g.

from sys import version_info as v then if v.major > 2: print(v.minor) then print(v.micro)

And there's at least one syntax difference between comments and posts (namely how to write a single code-formatted backslash: \), so hopefully this does away with it.

Not quite sure if this is the place to report issues with the new comments but I have attached a screenshot of a formatting issue with code in a comment. For any unable to see the image, new lines seem to be showing as tabs, and the code block itself extends beyond the borders of the comment and into the "Hot Questions" list on the right side of the page. Honestly it makes the code harder to read than just having it as plaintext in my opinion.