State Department’s AI Chatbot Success Hinges on Collaboration

The State Department’s journey in developing and deploying its internal artificial intelligence (AI) chatbot underscores the importance of collaboration, as noted by agency officials.

Matthew Graviss, the State Department’s Chief Data and AI Officer, and Gharun Lacy, Deputy Assistant Secretary and Assistant Director of the Diplomatic Security Service for Cyber and Technology Security, discussed the AI-powered chatbot’s role in streamlining operations during a briefing with Nextgov/FCW. Both emphasized that the collaborative approach between various departments was critical to the chatbot's progress.

Graviss highlighted the partnership as a model for other agencies. “We’ve had the heads of cyber, diplomatic technology, and analytics working closely together, meeting regularly to ensure our modernization efforts with AI are a team effort,” Graviss said. “AI modernization is a collective endeavor.”

The State Department’s chatbot journey has spanned over 18 months—a relatively long process for launching an AI tool. Initially used as a research resource, the chatbot has evolved into a generative AI-powered assistant after several AI models were tested. The team ultimately settled on using Microsoft Azure’s OpenAI platform.

To further advance the chatbot’s capabilities, other software providers such as Palantir, Deloitte, and Bright Star have been contracted. Palantir is focused on enhancing the user interface, Deloitte assists with analyzing chatbot responses, and Bright Star conducts independent verification and validation audits.

With 10,000 department employees already using the chatbot, Graviss anticipates its features will continue to expand, benefiting the department's 270 global mission areas, including diplomatic efforts.

Graviss explained, “Our diplomats must be comfortable using AI daily, as it enhances our primary function—engaging with words. AI aligns perfectly with diplomacy, a field that relies heavily on communication and documentation.”

One of the chatbot's primary goals is to consolidate the vast amounts of research produced annually by State analysts, covering reports on human trafficking, human rights, and Congressional summaries. The AI tool summarizes and translates these documents, enabling diplomats to access relevant data more efficiently.

As AI becomes increasingly integrated into operations, Lacy stressed the importance of balancing cybersecurity and emerging technology adoption. State’s subagencies, including in-house cybersecurity experts, have played a crucial role in ensuring the secure deployment of AI tools.

“We’re showing that early involvement of security teams accelerates the business process, rather than hindering it,” Lacy noted.

Ensuring a strong cybersecurity approach is vital, given the sensitive nature of the data handled by the chatbot. The department’s internal teams continue to oversee end-to-end software evaluations and monitor the tool’s performance in real-world scenarios.

Graviss and Lacy both credited Secretary of State Antony Blinken’s support for AI adoption as a driving force behind the technology’s rapid development. Regular collaboration between leadership teams has allowed the State Department to quickly implement updates and integrate emerging AI tools into its operations.

“Leadership backing is key to accelerating adoption,” Graviss concluded. “With regular meetings involving key decision-makers, we’ve been able to make meaningful progress swiftly.”