Cointime

Download App
iOS & Android

Revolution in AI Evaluation! Artificial Analysis Launches Industry Capability Index: Claude Fable 5 Dominates, Open Source GLM-5.2 Approaches Closed Source Frontier

According to monitoring by Dongcha Beating, traditional AI model evaluations often rely on generic academic datasets, leading to discrepancies between high test scores and actual business performance. Evaluation agency Artificial Analysis has launched six industry-specific indices to assess real business performance, building on existing general skill indices for intelligence and programming (covering finance, law, healthcare, operations, engineering, and economics). Unlike general skill indices, industry indices are customized with weighted distributions based on the O*NET work activity classification, simulating the frequency of tasks in real job positions. It matches the underlying capability tests of large models (such as HLE reasoning, LCR long text processing, GDPval agent work, etc.) with the specialized knowledge base of each industry (AA-Omniscience). In the initial testing phase, the closed-source model Claude Fable 5 topped all eight indices, while the open-source GLM-5.2 secured five first places in six industry indices, scoring 53 in the engineering index, close to Claude Sonnet 5's 55. The evaluation also revealed a significant performance gap: in the operations index, the single-task cost of Claude Fable 5 ($3.48) is over a hundred times that of the open-source DeepSeek V4 Pro ($0.03); in the legal index, the preview version of Gemini 3.1 Pro is nearly seven times faster than Claude Fable 5, with a score only 11 points lower. Weighing performance and cost-effectiveness based on task scenarios is becoming a new benchmark for AI selection in vertical fields.

Comments

All Comments

Recommended for you

  • BTC Surpasses $64,000

    Market data shows that BTC has surpassed $64,000, currently priced at $64,018, with a 24-hour decline narrowed to 0.29%. The market is experiencing significant volatility, so please ensure proper risk management.
  • US Central Command: No US Troops Captured or Killed Recently

    On July 17, the US Central Command stated that Iranian forces claimed to have attacked US troops stationed in Syria's Tanf and captured or killed American soldiers during the incident. This claim is false, and there have been no US personnel killed or captured in the region recently. (Jinshi)
  • SpaceX's Market Value Drops by Nearly $1 Trillion

    On July 17, SpaceX (SPCX.O) saw its stock price decline by 38% from its peak, resulting in a market value loss of approximately $1 trillion from its peak.
  • US Military Night Raid in Iran Destroys Multiple Bridges

    On July 17, according to CCTV International News, a video filmed by residents of the port city of Hamir Port in Hormozgan Province, southern Iran, shows significant damage to the Kahureshtan Bridge due to a US military night airstrike, reportedly resulting in at least 4 deaths.
  • Philadelphia Semiconductor Index Opens Down Over 3%

    On July 17, the Philadelphia Semiconductor Index opened down 3.24% in the U.S. stock market. Among the component stocks, Nvidia's share price fell by 2.52%, TSMC's share price dropped by 4.77%, Broadcom's share price decreased by 1.92%, Micron Technology's share price declined by 2.78%, Advanced Micro Devices' share price was down by 4.02%, and ASML's share price fell by 2.74%.
  • SK Hynix ADR (SKHY.O) Falls Below IPO Price of $149 for the First Time

    SK Hynix ADR (SKHY.O) has fallen below its initial public offering price of $149 for the first time.
  • Nasdaq 100 Index Drops Nearly 3% to Intraday Low

    The Nasdaq 100 Index fell nearly 3% to an intraday low.
  • U.S. Stock Indices Open Lower; Semiconductor Sector Continues Adjustment

    All three major U.S. stock indices opened lower, with the Dow Jones Industrial Average down 1.03%, the S&P 500 down 1.19%, and the Nasdaq Composite down 1.80%. The semiconductor sector continues its adjustment. Nvidia's stock price fell by 2.07%, TSMC's stock price dropped by 4.49%, Broadcom's stock price decreased by 1.62%, SK Hynix's stock price declined by 2.33%, Micron Technology's stock price fell by 3.19%, Advanced Micro Devices' stock price dropped by 3.86%, ASML's stock price decreased by 2.46%, Intel's stock price fell by 3.69%, Lam Research's stock price dropped by 4.62%, and ARM's stock price declined by 4.77%.
  • Philadelphia Semiconductor Index Falls Over 3%, Approaching Bear Market

    The Philadelphia Semiconductor Index has fallen over 3%, approaching a bear market.
  • Apple Reclaims Top Spot in Global Market Value, Surpassing Nvidia After 3 Months

    On July 17, Nvidia's stock fell by 3.2%, further reducing its market value to approximately $4.86 trillion, while Apple's market value rose to about $4.90 trillion due to a recent strong rally. Apple has regained the position of the world's most valuable company. Notably, Apple's cumulative increase since the beginning of the year has reached 22.8%, with an 11.2% rise in the past 20 days, showing a significantly stronger performance than the market; Nvidia has been nearly flat over the past 20 days (-3%), indicating a clear short-term momentum weakness compared to Apple. This marks Apple's return to the top of the global market value rankings after several months. Market analysts point out that Apple's recent surge is boosted by expectations for the implementation of AI features, coupled with continued institutional buying, which has provided ongoing support for its stock price; Nvidia, on the other hand, is facing profit-taking pressure due to its high valuation.