Cointime

Download App
iOS & Android

Microsoft CEO: Treat Advanced Models as Internal Risks, Control Must Be External to Models

Microsoft CEO Satya Nadella published a discussion on the trust framework in the era of superintelligence. He stated that traditional software allows for tracing behavior back to specific code paths, whereas current superintelligent systems lack this mechanistic understanding and cannot attribute model behavior and outputs to specific training data or weight configurations. These systems are being deployed and accessing sensitive data, executing critical actions on behalf of users. He emphasized that responsibility for intelligent behavior cannot be outsourced to model providers, nor can superintelligence be accepted or rejected as a nested black box based solely on its suggestions and actions. Nadella proposed separating the supply of intelligence from the authorization of intelligence, surrounding nondeterministic models with deterministic system design, human control, and operational procedures, and establishing industry standards where existing standards are insufficient. He suggested managing cutting-edge models with closed-source and open-source weights as internal risks, not because they are inherently malicious, but because entities with sufficient capabilities that can access important systems may make mistakes or be compromised. Relevant practices include establishing identities, restricting permissions, recording activities, and setting isolation boundaries, with transparency in the model's reasoning chain as a non-negotiable starting point. He further stated that permission and action control must be external to the model, separating the model, orchestration layer, and action space. Principles include avoiding reliance on a single model for important outcomes or self-verification, ensuring key actions leave tamper-proof and human-readable evidence, continuously testing for failures, attacks, and edge cases, allowing organizations to independently decide access and action permissions, validating independently of the intelligence being verified, presuming models have been compromised and supporting mid-course pauses or shutdowns, and timely disclosure and sharing of lessons within the industry in case of failure or compromise. He concluded that the most trustworthy superintelligent systems are those that require the least trust in the model.

Comments

All Comments

Recommended for you

  • BTC Surpasses $82,000

    Market data shows that BTC has surpassed $82,000, currently priced at $82,038.65, with a 24-hour increase of 0.31%. The market is experiencing significant volatility, so please ensure proper risk management.
  • ETH Surpasses $2400

    Market data shows that ETH has surpassed $2400, currently priced at $2400.26, with a 24-hour decline of 1.33%. The market is experiencing significant volatility, so please ensure proper risk management.
  • BTC Surpasses $81,000

    Market data shows that BTC has surpassed $81,000, currently priced at $81,016.34, with a 24-hour decline of 1.53%. The market is experiencing significant volatility, so please ensure proper risk management.
  • BTC Falls Below $80,000

    Market data shows that BTC has fallen below $80,000, currently priced at $79,940.09, with a 24-hour decline of 3.44%. The market is experiencing significant volatility, so please ensure proper risk management.
  • Philadelphia Semiconductor Index Set to Lose Over $2 Trillion in Market Value in Seven Trading Days

    The Philadelphia Semiconductor Index is poised to evaporate over $2 trillion in market value within seven trading days.
  • Philadelphia Semiconductor Index Drops 4% to Lowest Level in Nearly a Month

    The Philadelphia Semiconductor Index fell by 4.01% during the day, currently reported at 11,215.37 points. Nvidia dropped by 3.25%, Broadcom fell by 2.87%, Micron decreased by 5.79%, Intel declined by 5.81%, AMD dropped by 6.28%, Qualcomm rose by 0.66%, TSMC fell by 4.37%, and ASML decreased by 3.34%.
  • Philadelphia Semiconductor Index Falls 4% to Lowest Level in Nearly a Month

    The Philadelphia Semiconductor Index dropped 4.01% during the day, currently reported at 11,215.37 points. Nvidia fell by 3.25%, Broadcom by 2.87%, Micron by 5.79%, Intel by 5.81%, AMD by 6.28%, Qualcomm rose by 0.66%, TSMC fell by 4.37%, and ASML dropped by 3.34%.
  • Ripple Prime Launches Perpetual Contract Trading Services for Institutional Clients

    According to Bloomberg, Ripple's main brokerage division, Ripple Prime, is launching perpetual contracts for real-world assets (RWA) aimed at institutional clients, covering over 100 reference markets including stocks, indices, ETFs, and commodities. This product is offered through Trade.xyz on Hyperliquid, continuing Ripple Prime's previous initiative to provide clients with access to cryptocurrency perpetual contract trading.
  • NYC Comptroller Urges Anthropic to Pause IPO Plans

    On October 15, New York City Comptroller Mark Levine stated in an interview that he has written to Anthropic CEO Dario Amodei, urging the company to suspend its planned initial public offering (IPO). In the letter, he called for Anthropic to pause its IPO plans until it can fully and transparently address concerns related to AI safety, governance, and development that the company itself has raised. Levine wrote that the company should focus on mitigating these risks rather than advancing the IPO. Currently, New York City has an investment exposure of less than $10 million in Anthropic through the private market.
  • US September PPI Rises 6.3% Year-on-Year, Exceeding Market Expectations

    The US Producer Price Index (PPI) for September recorded a year-on-year increase of 6.3%, the highest since May 2026, surpassing market expectations of 5.3%. The month-on-month PPI for September was reported at 1.3%, significantly higher than the market forecast of 0.5%. Additionally, US retail sales for September showed a month-on-month increase of 0.8%, also exceeding market expectations.