Assist Service: Responsible AI FAQ

What is Assist Service?

Assist Service is designed to be an intuitive, resourceful, and personal productivity guide and assistance. It helps answer questions users may have on how to use products discussed in articles published on support.microsoft.com by offering curated and authoritative content. The system aims to assist users in completing tasks more efficiently through real-time and interactive guidance that adapts to different work environments.

What can Assist Service do?

Assist Service uses a large language model (LLM) grounded only in articles published on support.microsoft.com. The system prompts and classifiers prevent users from asking questions unrelated to products discussed in articles published on support.microsoft.com. The expected inputs for the Assist Service are text queries from the user about using products discussed in articles published on support.microsoft.com. The expected outputs are tailored responses, on-screen guided instructions, and diverse help content (videos, images, and text), all of which help users efficiently complete their tasks, discover new features, and solve problems.

What are Assist Service’s intended uses?

Assist Service can be used in multiple scenarios. The system’s intended uses are question-answering scenarios all grounded in articles published to support.microsoft.com. The following list isn’t comprehensive, but it illustrates the diversity of questions that can be supported by Assist Service.

• Discovery: Users can ask Assist Service about new product features discussed in articles published on support.microsoft.com and the service will provide an answer that may explain what the feature is, how to use it, and how to access it.

• Troubleshooting: Users can ask Assist Service about how to solve an issue for a product discussed in articles published on support.microsoft.com such as “how to fix printer connection issues” and the service will provide troubleshooting instructions.

• How to: Users can ask Assist Service how to perform a specific task in a product discussed in articles published on support.microsoft.com such as “how to summarize a Word document with Copilot” and the service will provide steps to complete the task.

How was Assist Service evaluated? What metrics are used to measure performance?

Assist Service was evaluated through extensive manual testing on top of Microsoft internal usage data. Additional evaluation was performed over custom datasets for offensive and malicious prompts (user questions). Further, Assist Service is continuously evaluated with user online feedback.

Assist Service's accuracy and performance have been assessed through human evaluations, resulting in an ACRUE score of 3.47 with a margin of error of ±0.11. The ACRUE score is rated on a scale of 1 to 5, with 5 indicating the highest level of accuracy and performance. This evaluation provides evidence of system reliability. However, the extent to which these results generalize across use cases that were not part of the evaluation may vary depending on factors such as data diversity.

What are the limitations of Assist Service? How can users minimize the impact of Assist Service’s limitations when using the system?

• Language Support: Assist Service currently is only available in English. More languages will be added in the future.

• Content Scope: Assist Service is limited to providing guidance on Microsoft products discussed in articles published on support.microsoft.com and cannot answer questions unrelated to this scope.

• Response Accuracy: Assist Service generates content that is fluent and grammatically correct, but it may still be inaccurate or inappropriate. The system cannot understand meaning or evaluate accuracy, so it is important to review what it produces and use your judgment.

• Over Reliance: While Assist Service includes features to avoid sharing unexpected offensive content and takes steps to prevent displaying potentially harmful content, you may still encounter unexpected results. We are constantly working to improve our technology to proactively address issues in line with our responsible AI principles. As with any AI-generated content, Assist Service helps users in completing tasks more efficiently, but it is crucial that you verify the answers it provides.

How do I provide feedback on Assist Service?

Your feedback is important for helping us improve the Assist Service. If you encounter harmful, inaccurate, or otherwise unexpected outputs from Assist Service, please use the thumbs-up and thumbs-down icons and comment box to share your feedback or report your concerns.