en.Wedoany.com Reported - The China Academy of Information and Communications Technology (CAICT) released the latest large model Token service performance monitoring results at the "High-Quality Token Service Seminar" held on June 16, 2026. Qin Sisi, Senior Business Manager of the Platform and Engineering Department at the CAICT Artificial Intelligence Research Institute, presented and interpreted the "Large Model Token Service Performance Monitoring Report (2026 Public Cloud)." The monitoring shows that in 2026, the overall call success rate of domestic public cloud large model Token services remained stable, with both Tokens Per Second (TPS) and Time to First Token (TTFT) increasing simultaneously, and service prices showing a significant rebound.

Since launching large model Token service performance monitoring at the end of 2024, CAICT has established an integrated monitoring system encompassing "standards, indicators, data, and platforms." Based on relevant standards such as the "Test Method for Large Model Inference Service Performance" (ITU-T F.PEM-LLM, 2026-0478T-YD, AIAT/T 0221-2025), this system designs a comprehensive set of performance indicators and multi-dimensional test datasets, conducting round-the-clock performance monitoring via servers located in different geographical regions. As of May 2026, over a hundred public cloud large model Token services have been tested, covering original large model Token services from domestic and international providers, as well as the engineering performance of open-source models such as DeepSeek, Qwen, GLM, and Kimi on various platforms. In terms of monitoring frequency, requests are continuously sent from servers in different geographical locations at different times each day. Since late May 2026, the monitoring frequency for call success rate has been updated to once every five minutes to more sensitively capture service fluctuations.

The monitoring data reveals three key observations. First, large model Token service throughput and latency are increasing simultaneously. Domestically, from January to April, the call success rate for original large models exceeded 99.9%, and the proportion of Token services achieving a 100% call success rate rose from 75% to 83%. In May, due to the increase in monitoring sampling frequency from 15 times per day to once every five minutes, more sporadic unavailability incidents were captured, causing the overall call success rate to drop to 99.79% and the proportion of 100% success rate to fall to 39%.

Regarding the TPS indicator, the average value for domestic models from January to May steadily increased, with a 73.12% improvement in May compared to the full year of 2025. Two models achieved a TPS exceeding 100 tokens/second, with the highest surpassing 200 tokens/second. The average TPS was 66.22 tokens/second, and the median was 53.68 tokens/second. Overall, TPS for foreign models remained at a relatively high level, with Gemini-3.1-pro-preview reaching 158.37 tokens/second and GPT-5.5 reaching 64.08 tokens/second. The average TPS for foreign models in May was 99.03 tokens/second, still significantly higher than domestic models.

In terms of Time to First Token (TTFT), the average TTFT for domestic models from January to May increased from 0.8 seconds to 1.09 seconds, a rise of 36.25%. In May, the lowest TTFT was 0.4 seconds, and the highest was 1.64 seconds, with 47% of models having a TTFT below 1 second. The TTFT for three foreign large-parameter models all exceeded 1 second, averaging 1.8 seconds, with Gemini-3.1-pro-preview having the longest TTFT exceeding 2 seconds. The report notes that the increase in TTFT is not simply a performance degradation but a result of multiple factors, including increases in model parameter size, context length, and call volume.

Second, large models are evolving towards "value regression" and "capability leap." In terms of market pricing, Token service prices rebounded significantly in 2026. The normalized price per million Tokens in the second half of 2025 was 6.8 RMB, which rose to 10.3 RMB in 2026, an increase of 51%, but still far below the price level of approximately 120 RMB for mainstream foreign models. The report emphasizes that actual usage costs need to consider both unit price and Token consumption efficiency. Foreign models consume fewer Tokens during inference, but due to high prices, the overall expenditure remains relatively high, with GPT and Claude being the two most expensive models. Domestic models generally have higher Token consumption, but due to lower prices, the overall expenditure is relatively low.

Model context length continues to expand towards the million-token scale. In May, 35.3% of domestic models had a context length of 1M (one million Tokens), and a total of 88.3% had a context length of 200K or more, an increase of over 60% compared to the end of 2025. The latest model series from foreign companies like OpenAI, Anthropic, and Google have all achieved context lengths of 1M or more. Domestic open-source large models have become the mainstream choice for many global platforms. Among the 22 domestic and international MaaS platforms surveyed, the deployment rate for DeepSeek and Qwen series models reached 100%, while the deployment rates for GLM, Kimi, and MiniMax series models reached 91%, 86%, and 82%, respectively.

Third, the inference performance of open-source models varies significantly across different MaaS platforms. The performance of the same model fluctuates considerably across platforms, indicating substantial differences in platform-side engineering optimization. The TPS for four domestic open-source models on different platforms is concentrated around 50 tokens/second, and the TTFT is concentrated around 1 second. The overall call success rate approaches 100%, but the call success rate for individual models on some platforms remains below 90%, indicating room for further optimization in platform service stability.

CAICT stated that the next phase of Token service performance monitoring will be deepened in four areas. First, optimize the monitoring system by adding performance indicators for multimodal large models, shortening the report release cycle, and improving the platform's automated analysis and early warning functions. Second, gradually open monitoring data to provide data support for scientific research, industry analysis, and application selection. Third, build regional monitoring platforms to offer customized services for regions and groups, serving as the service access benchmark for Token factories. Fourth, strengthen service quality supervision and anomaly early warning mechanisms, conducting in-depth problem analysis and quality supervision for abnormal situations.

CAICT will continue to collaborate with various industry sectors, relying on the Token Service Working Group (under preparation) of the Artificial Intelligence Industry Alliance (AIIA), to deepen and advance efforts in standard development, the Climbing Plan, research report compilation, and compliance evaluation, accelerating the construction of a high-quality Token service ecosystem.