As the world grapples with the accelerating pace of artificial intelligence development, a significant, albeit unofficial, step towards global AI governance is on the horizon. Tomorrow, November 19, 2025, experts from the United States and China are expected to converge in Hong Kong, aiming to establish a crucial consensus on limiting the use of AI in the defense sector. This anticipated agreement, while not a binding governmental treaty, signifies a pivotal moment in the ongoing dialogue between the two technological superpowers, highlighting a shared understanding of the inherent risks posed by unchecked AI in military applications.
The impending expert consensus builds upon a foundation of prior intergovernmental talks initiated in November 2023, when US President Joe Biden and Chinese President Xi Jinping first agreed to launch discussions on AI safety. Subsequent high-level dialogues in May and August 2024 laid the groundwork for exchanging views on AI risks and governance. The Hong Kong forum represents a tangible move towards identifying specific areas for restriction, particularly emphasizing the need for cooperation in preventing AI's weaponization in sensitive domains like bioweapons.
Forging Guardrails: Specifics of Military AI Limitations
The impending consensus in Hong Kong is expected to focus on several critical areas designed to establish robust guardrails around military AI. Central to these discussions is the principle of human control over critical functions, with experts advocating for a mutual pledge ensuring affirmative human authorization for any weapons employment, even by AI-enabled platforms, in peacetime and routine military encounters. This move directly addresses widespread ethical concerns regarding autonomous weapon systems and the potential for unintended escalation.
A particularly sensitive area of focus is nuclear command and control. Building on a previous commitment between Presidents Biden and Xi Jinping in 2024 regarding human control over nuclear weapon decisions, experts are pushing for a mutual pledge not to use AI to interfere with each other's nuclear command, control, and communications systems. This explicit technical limitation aims to reduce the risk of AI-induced accidents or miscalculations involving the most destructive weapons. Furthermore, the forum is anticipated to explore the establishment of "red lines" – categories of AI military applications deemed strictly off-limits. These taboo norms would clarify thresholds not to be crossed, thereby reducing the risks of uncontrolled escalation. Christopher Nixon Cox, a board member of the Richard Nixon Foundation, specifically highlighted bioweapons as an "obvious area" for US-China collaboration to limit AI's influence.
These proposed restrictions mark a significant departure from previous approaches, which often involved unilateral export controls by the United States (such as the sweeping AI chip ban in October 2022) aimed at limiting China's access to advanced AI hardware and software. While those restrictions continue, the Hong Kong discussions signal a shift towards mutual agreement on limitations, fostering a more collaborative, rather than purely competitive, approach to AI governance in defense. Unlike earlier high-level talks in May 2024, which focused broadly on exchanging views on "technical risks of AI" without specific deliverables, this forum aims for more concrete, technical limitations and mutually agreed-upon "red lines." China's consistent advocacy for global AI cooperation, including a July 2025 proposal for an international AI cooperation organization, finds a specific bilateral platform here, potentially bridging definitional gaps concerning autonomous weapons.
Initial reactions from the AI research community and industry experts are a blend of cautious optimism and urgent calls for stability. There is a broad recognition of AI's inherent fragility and the potential for catastrophic accidents in high-stakes military scenarios, making robust safeguards imperative. While some US chipmakers have expressed concerns about losing market share in China due to existing export controls – potentially spurring China's domestic chip development – many experts, including former (Alphabet (NASDAQ: GOOGL)) CEO Eric Schmidt, emphasize the critical need for US-China collaboration on AI to maintain global stability and ensure human control. Despite these calls for cooperation, a significant lack of trust between the two nations remains, complicating efforts to establish effective governance. Chinese officials, for instance, have previously viewed US "responsible AI" approaches with skepticism, seeing them as attempts to avoid multilateral negotiations. This underlying tension makes achieving comprehensive, binding agreements "logically difficult," as noted by Tsinghua University's Sun Chenghao, yet underscores the importance of even expert-level consensus.
Navigating the AI Divide: Implications for Tech Giants and Startups
The impending expert consensus on restricting military AI, while a step towards global governance, operates within a broader context of intensifying US-China technological competition, profoundly impacting AI companies, tech giants, and startups on both sides. The landscape is increasingly bifurcated, forcing strategic adaptations and creating distinct winners and losers.
For US companies, the effects are mixed. Chipmakers and hardware providers like (NVIDIA (NASDAQ: NVDA)) have already faced significant restrictions on exporting advanced AI chips to China, compelling them to develop less powerful, China-specific alternatives, impacting revenue and market share. AI firms developing dual-use technologies face heightened scrutiny and export controls, limiting market reach. Furthermore, China has retaliated by banning several US defense firms and AI companies, including TextOre, Exovera, (Skydio (Private)), and (Shield AI (Private)), from its market. Conversely, the US government's robust support for domestic AI development in defense creates significant opportunities for startups like (Anduril Industries (Private)), (Scale AI (Private)), (Saronic (Private)), and (Rebellion Defense (Private)), enabling them to disrupt traditional defense contractors. Companies building foundational AI infrastructure also stand to benefit from streamlined permits and access to compute resources.
On the Chinese side, the restrictions have spurred a drive for indigenous innovation. While Chinese AI labs have been severely hampered by limited access to cutting-edge US AI chips and chip-making tools, hindering their ability to train large, advanced AI models, this has accelerated efforts towards "algorithmic sovereignty." Companies like DeepSeek have shown remarkable progress in developing advanced AI models with fewer resources, demonstrating innovation under constraint. The Chinese government's heavy investment in AI research, infrastructure, and military applications creates a protected and well-funded domestic market. Chinese firms are also strategically building dominant positions in open-source AI, cloud infrastructure, and global data ecosystems, particularly in emerging markets where US policies may create a vacuum. However, many Chinese AI and tech firms, including (SenseTime (HKEX: 0020)), (Inspur Group (SSE: 000977)), and the Beijing Academy of Artificial Intelligence, remain on the US Entity List, restricting their ability to obtain US technologies.
The competitive implications for major AI labs and tech companies are leading to a more fragmented global AI landscape. Both nations are prioritizing the development of their own comprehensive AI ecosystems, from chip manufacturing to AI model production, fostering domestic champions and reducing reliance on foreign components. This will likely lead to divergent innovation pathways: US labs, with superior access to advanced chips, may push the boundaries of large-scale model training, while Chinese labs might excel in software optimization and resource-efficient AI. The agreement on human control in defense AI could also spur the development of more "explainable" and "auditable" AI systems globally, impacting AI design principles across sectors. Companies are compelled to overhaul supply chains, localize products, and navigate distinct market blocs with varying hardware, software, and ethical guidelines, increasing costs and complexity. The strategic race extends to control over the entire "AI stack," from natural resources to compute power and data, with both nations vying for dominance. Some analysts caution that an overly defensive US strategy, focusing too heavily on restrictions, could inadvertently allow Chinese AI firms to dominate AI adoption in many nations, echoing past experiences with Huawei.
A Crucial Step Towards Global AI Governance and Stability
The impending consensus between US and Chinese experts on restricting AI in defense holds immense wider significance, transcending the immediate technical limitations. It emerges against the backdrop of an accelerating global AI arms race, where both nations view AI as pivotal to future military and economic power. This expert-level agreement could serve as a much-needed moderating force, potentially reorienting the focus from unbridled competition to cautious, targeted collaboration.
This initiative aligns profoundly with escalating international calls for ethical AI development and deployment. Numerous global bodies, from UNESCO to the G7, have championed principles of human oversight, transparency, and accountability in AI. By attempting to operationalize these ethical tenets in the high-stakes domain of military applications, the US-China consensus demonstrates that even geopolitical rivals can find common ground on responsible AI use. This is particularly crucial concerning the emphasis on human control over AI in the military sphere, especially regarding nuclear weapons, addressing deep-seated ethical and existential concerns.
The potential impacts on global AI governance and stability are profound. Currently, AI governance is fragmented, lacking universally authoritative institutions. A US-China agreement, even at an expert level, could serve as a foundational step towards more robust global frameworks, demonstrating that cooperation is achievable amidst competition. This could inspire other nations to engage in similar dialogues, fostering shared norms and standards. By establishing agreed-upon "red lines" and restrictions, especially concerning lethal autonomous weapons systems (LAWS) and AI's role in nuclear command and control, the likelihood of accidental or rapid escalation could be significantly mitigated, enhancing global stability. This initiative also aims to foster greater transparency in military AI development, building confidence between the two superpowers.
However, the inherent dual-use dilemma of AI technology presents a formidable challenge. Advancements for civilian purposes can readily be adapted for military applications, and vice versa. China's military-civil fusion strategy explicitly seeks to leverage civilian AI for national defense, intensifying this problem. While the agreement directly confronts this dilemma by attempting to draw lines where AI's application becomes impermissible for military ends, enforcing such restrictions will be exceptionally difficult, requiring innovative verification mechanisms and unprecedented international cooperation to prevent the co-option of private sector and academic research for military objectives.
Compared to previous AI milestones – from the Turing Test and the coining of "artificial intelligence" to Deep Blue's victory in chess, the rise of deep learning, and the advent of large language models – this agreement stands out not as a technological achievement, but as a geopolitical and ethical milestone. Past breakthroughs showcased what AI could do; this consensus underscores the imperative of what AI should not do in certain contexts. It represents a critical shift from simply developing AI to actively governing its risks on an international scale, particularly between the world's two leading AI powers. Its importance is akin to early nuclear arms control discussions, recognizing the existential risks associated with a new, transformative technology and attempting to establish guardrails before a full-blown crisis emerges, potentially setting a crucial precedent for future international norms in AI governance.
The Road Ahead: Challenges and Predictions for Military AI Governance
The anticipated consensus between US and Chinese experts on restricting AI in defense, while a significant step, is merely the beginning of a complex journey towards effective international AI governance. In the near term, a dual approach of unilateral restrictions and bilateral dialogues is expected to persist. The United States will likely continue and potentially expand its export and investment controls on advanced AI chips and systems to China, particularly those with military applications, as evidenced by a final rule restricting US investments in Chinese AI, semiconductor, and quantum information technologies that took effect on January 2, 2025. Simultaneously, China will intensify its "military-civil fusion" strategy, leveraging its civilian tech sector to advance military AI and circumvent US restrictions, focusing on developing more efficient and less expensive AI technologies. Non-governmental "Track II Dialogues" will continue to explore confidence-building measures and "red lines" for unacceptable AI military applications.
Longer-term developments point towards a continued bifurcation of global AI ecosystems, with the US and China developing distinct technological architectures and values. This divergence, coupled with persistent geopolitical tensions, makes formal, verifiable, and enforceable AI treaties between the two nations unlikely in the immediate future. However, the ongoing discussions are expected to shape the development of specific AI applications. Restrictions primarily target AI systems for weapons targeting, combat, location tracking, and advanced AI chips crucial for military development. Governance discussions will influence lethal autonomous weapon systems (LAWS), emphasizing human control over the use of force, and AI in command and control (C2) and decision support systems (DSS), where human oversight is paramount to mitigate automation bias. The mutual pledge regarding AI's non-interference with nuclear command and control will also be a critical area of focus.
Implementing and expanding upon this consensus faces formidable challenges. The dual-use nature of AI technology, where civilian advancements can readily be militarized, makes regulation exceptionally difficult. The technical complexity and "black box" nature of advanced AI systems pose hurdles for accountability, explainability, and regulatory oversight. Deep-seated geopolitical rivalry and a fundamental lack of trust between the US and China will continue to narrow the space for effective cooperation. Furthermore, devising and enforcing verifiable agreements on AI deployment in military systems is inherently difficult, given the intangible nature of software and the dominance of the private sector in AI innovation. The absence of a comprehensive global framework for military AI governance also creates a perilous regulatory void.
Experts predict that while competition for AI leadership will intensify, there's a growing recognition of the shared responsibility to prevent harmful military AI uses. International efforts will likely prioritize developing shared norms, principles, and confidence-building measures rather than binding treaties. Military AI is expected to fundamentally alter the character of war, accelerating combat tempo and changing risk thresholds, potentially eroding policymakers' understanding of adversaries' behavior. Concerns will persist regarding operational dangers like algorithmic bias and automation bias. Experts also warn of the risks of "enfeeblement" (decreasing human skills due to over-reliance on AI) and "value lock-in" (AI systems amplifying existing biases). The proliferation of AI-enabled weapons is a significant concern, pushing for multilateral initiatives from groups like the G7 to establish global standards and ensure responsible AI use in warfare.
Charting a Course for Responsible AI: A Crucial First Step
The impending expert consensus between Chinese and US experts on restricting AI in defense represents a critical, albeit foundational, moment in the history of artificial intelligence. The key takeaway is a shared recognition of the urgent need for human control over lethal decisions, particularly concerning nuclear weapons, and a general agreement to limit AI's application in military functions to foster collaboration and dialogue. This marks a shift from solely unilateral restrictions to a nascent bilateral understanding of shared risks, building upon established official dialogue channels between the two nations.
This development holds immense significance, positioning itself not as a technological breakthrough, but as a crucial geopolitical and ethical milestone. In an era often characterized by an AI arms race, this consensus attempts to forge norms and governance regimes, akin to early nuclear arms control efforts. Its long-term impact hinges on the ability to translate these expert-level understandings into more concrete, verifiable, and enforceable agreements, despite deep-seated geopolitical rivalries and the inherent dual-use challenge of AI. The success of these initiatives will ultimately depend on both powers prioritizing global stability over unilateral advantage.
In the coming weeks and months, observers should closely monitor any further specifics emerging from expert or official channels regarding what types of military AI applications will be restricted and how these restrictions might be implemented. The progress of official intergovernmental dialogues, any joint statements, and advancements in establishing a common glossary of AI terms will be crucial indicators. Furthermore, the impact of US export controls on China's AI development and Beijing's adaptive strategies, along with the participation and positions of both nations in broader multilateral AI governance forums, will offer insights into the evolving landscape of military AI and international cooperation.
This content is intended for informational purposes only and represents analysis of current AI developments.
TokenRing AI delivers enterprise-grade solutions for multi-agent AI workflow orchestration, AI-powered development tools, and seamless remote collaboration platforms.
For more information, visit https://www.tokenring.ai/.
