"model agnostic" is not something valuable to users, it's valuable to you only in being able to up or downgrade quality as you see fit. Transparency about the exact model used would be a minimum value.

It would also be valuable to know if and what kind of fine-tuning was applied and for what purpose.

RAG is a powerful technique to reduce hallucinations, keep answers up to date with recent news, it's again unclear why you'd present that as a quality rather than a caveat.

Who is the president of the United States of America As of my last update, the President of the United States is Joe Biden, who took office on January 20, 2021...

Lack of RAG is not the mark of quality.

Also transparency would require being open about usage quotas, as nobody can afford to offer unlimited queries for free.

Additionally, this limited release will help us ensure our tools effectively detect and manage unrelated, inappropriate, or harmful content

Didn't you just say the product is model agnostic and uses an off-the-shelf model? What tools could you hope to validate and how? Do you seek red-teaming (https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/red-teaming), or do you just want to know if asking a question about how to implement a CSS animation feature leads to instructions on how to manufacture a car bomb? What degree of confidence are you opting for?

Import chat history - Developers can pick up right where they left off in another AI tool to get unstuck on stackoverflow.ai.

That would assume some developer comes from another AI tool that was even worse than stackoverflow.ai having an old model with a small token window, no RAG, no local source code context, no reasoning, no agentic features (not trying to bash you here, just looking at the obvious advantages of the competition)? In your market study, what other AI tool would qualify for that scenario? ELIZA?

"model agnostic" is not something valuable to users, it's valuable to you only in being able to up or downgrade quality as you see fit. Transparency about the exact model used would be a minimum value.

It would also be valuable to know if and what kind of fine-tuning was applied and for what purpose.

Another very important aspect to know is whether you store and share any of the conversations between your users and the chatbots. If I ask it whether my passwords are secure, are my passwords going to surface in the future?

RAG is a powerful technique to reduce hallucinations, keep answers up to date with recent news, it's again unclear why you'd present that as a quality rather than a caveat.

Who is the president of the United States of America As of my last update, the President of the United States is Joe Biden, who took office on January 20, 2021...

Lack of RAG is not the mark of quality.

Also transparency would require being open about usage quotas, as nobody can afford to offer unlimited queries for free.

Additionally, this limited release will help us ensure our tools effectively detect and manage unrelated, inappropriate, or harmful content

Didn't you just say the product is model agnostic and uses an off-the-shelf model? What tools could you hope to validate and how? Do you seek red-teaming (https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/red-teaming), or do you just want to know if asking a question about how to implement a CSS animation feature leads to instructions on how to manufacture a car bomb? What degree of confidence are you opting for?

Import chat history - Developers can pick up right where they left off in another AI tool to get unstuck on stackoverflow.ai.

That would assume some developer comes from another AI tool that was even worse than stackoverflow.ai having an old model with a small token window, no RAG, no local source code context, no reasoning, no agentic features (not trying to bash you here, just looking at the obvious advantages of the competition)? In your market study, what other AI tool would qualify for that scenario? ELIZA?