During the National Day holiday, have you grabbed a ticket on 12306? Although complaints such as "It's so hard to buy high-speed rail tickets" and "Why do you have to wait for a ticket after it's issued?" are common during holidays, the China Railway Customer Service Center 12306 platform has been unknowingly going through 13 years since its launch on June 12, 2011, and has become the preferred ticket purchasing platform for most passengers traveling by high-speed rail. Looking back at the development history of 12306, it can be said that it is an "epic" that involves a battle of wits and courage with ticket-grabbing software.
On National Day 2024, railways usher in peak passenger traffic ICPhoto
Everything has to start from the time when 12306 was established. With the sale of the first Beijing-Tianjin intercity railway e-ticket on June 12, 2011, China officially kicked off ticket sales on the railway network. In just a few months, the scope of online ticket sales has rapidly expanded to include Beijing-Shanghai, Wuhan-Guangzhou, Zhengzhou-Xi'an and other high-speed railway lines. As of September 30, high-speed rail tickets (prefixed with G and D) nationwide can be purchased on 12306. By the end of 2011, nationwide train tickets (prefixed with Z, T, K) can be purchased on 12306. In just half a year, ticket purchasing on China's railway network has made a leap from "from scratch" to "comprehensive popularization"!
But the novelty of buying tickets on the railway network has not faded, and the 2012 Spring Festival has quietly arrived. The Spring Festival, a rare mass migration event in the world, brings unprecedented pressure to China's transportation industry every year. It can be said to be a "big test" for the transportation industry. And the first system crash of 12306 came beyond everyone’s expectations.
Starting from January 5, 2012, the 12306 website received more than 1 billion hits for five consecutive days, and the number of visits surged more than 10 times month-on-month. Among them, the number of hits exceeded 1.4 billion on January 9, making it one of the busiest websites in the world - you must know that there were only 500 million Internet users in China at the end of 2011.
Such access pressure was not taken into consideration at the beginning of the construction of the platform. At the beginning of the construction of the 12306 system, based on the core system architecture of passenger tickets and the characteristics of Internet applications, the developers designed multiple relatively independent business partitions for the system such as caching services, user management, ticket inquiry, order and electronic ticket processing, as well as three-level network security domains, namely the external network, the intranet and the passenger ticket network. In the stress test before going online, a process included user login, ticket inquiry, order placement and payment. The system's maximum transaction capacity was 34 tickets/second. Calculated based on a peak period of 10 hours, the ticket sales volume could reach the design capacity of 1.2 million tickets/day.
However, the stress test missed an "uninvited guest" - ticket-grabbing software. Ticket-grabbing software is a browser-based plug-in that uses the machine's rapid response to help users complete the actions of constantly querying and refreshing the page. It uses high-frequency query for remaining tickets for trains, automatically fills in personal information, and operates multiple accounts at the same time, thereby completing the ticket purchase process faster than users who purchase tickets manually.
Passengers who suffer from "no tickets" on the platform can only spend money to seek help from ticket grabbing software in order to ensure everything is safe.
Due to the lack of sufficient estimation of the huge traffic caused by ticket-grabbing software, 12306 was vulnerable to a large number of ticket-grabbing software - the 12306 website was paralyzed, a large number of users were unable to log in, and the page refresh time took up to 30 minutes. After booking and paying, tickets could not be purchased. Many ticket buyers helplessly denounced the 12306 website as "in name only".
However, there was no turning back, so the 12306 ticketing team had no choice but to fight: in response to the problem of insufficient bandwidth, the 12306 ticketing team made a prompt decision and quickly increased the bandwidth from 600 MB to 1000 MB, and soon increased it to 1500 MB. After monitoring and analyzing the online ticketing data, the ticketing team found that very few users purchased a large number of tickets online, and the total number of tickets purchased in a single day exceeded 100. In order to ensure fair ticket purchasing, starting from January 5, the 12306 Ticketing Team has adjusted the online ticket purchasing process. After the user successfully purchases the ticket, the system will force the user to log out, and they need to log in again when purchasing tickets again.
The former Ministry of Railways also thought of inviting "foreign aid". At that time, among all the websites, the only ones with rich experience in "not collapsing despite the surge in traffic in a short period of time" were Tmall and Taobao, which were in full swing during the "Double Eleven". Therefore, Alibaba Group sent 17 technical elites to form a project team to assist the 12306 website in optimizing and improving it. The two teams worked side by side to greatly improve the user experience of the 12306 website. The system's daily ticket sales increased from 650,000 in early January to more than 1 million, and set a record of 1.192 million daily ticket sales on January 20.
Tmall Taobao Mall’s “Double 11 Global Shopping Carnival” advertising dominates the Shanghai Xujiahui subway station to compete for user traffic ICPhoto
After this "extreme test" of the Spring Festival travel less than a year after the launch, discussions and controversies about the 12306 architecture also appeared on the Internet. The Ministry of Railways at that time carefully listened to the opinions of all parties and carefully sorted out the causes of the problem, and concluded that it was mainly due to ticket inquiries and reservations. The single/electronic ticket business partition has insufficient processing capacity, resulting in long response times under high concurrent access requests during peak periods. In addition, each business partition cannot be well isolated, resulting in an "avalanche" effect from the inside to the outside of the system, causing website congestion and affecting the user's ticket purchasing experience.
In response to the above problems and reasons, the developers focused on architecture optimization and reconstruction ideas to improve the response speed of ticket inquiry and transaction processing, improve the scalability of the backend system, change the online queuing method, and reduce the bandwidth pressure caused by centralized ticket release during peak hours. At the same time, separate core businesses as much as possible to reduce strong correlations between business links. Specific contents include:
First of all, the concurrent query capability must be greatly improved. 12306 uses an in-memory computing database to replace the traditional database, increasing the response speed of ticket queries from less than 1,000 times/second to more than 20,000 times/second, and reducing the response time from the original 1s to 10ms, allowing users to quickly obtain train numbers and remaining tickets.
Secondly, it is necessary to allow users to "queue to place orders without congestion" during the peak period. To this end, 12306 has built a transaction processing queuing system. The queue's order request reception capacity exceeds 100,000 orders per second. Users can quickly complete the order operation during the peak period of ticket sales and wait for the system to process it in sequence. During the waiting process, the queuing status (waiting time for processing) can be queried. In-memory computing databases are also used in queuing systems.
Third, orders/e-tickets are transformed into nodes, databases, and tables, and the original 1 node, 1 database, and 1 table are split into 3 nodes, 30 databases, and 30 tables. Online related operations are dispersed to each node and database table. In this way, users' ticket booking requests on the website can be responded to and processed faster.
Finally, the business separation of ticket booking and ticket collection operations is carried out, and different business nodes (ticket sales nodes and ticket collection nodes) carry the online ticket sales and offline ticket collection services; the reading and writing of order/e-ticket generation and query are separated, and the in-memory computing database is used to centrally store orders/e-tickets. The response speed of order queries is increased from about 200 times/second to more than 5000 times/second, which greatly improves the order/e-ticket query efficiency.
In the stress test before going online, the system after the optimized architecture achieved a maximum transaction capacity of 300 tickets/second, which can meet the business needs of a daily ticket sales volume of 5 million. During the 2013 Spring Festival, the highest daily ticket sales of the 12306 website after the optimized structure reached 3.64 million, accounting for 40% of the total ticket sales. The ticket sales were more than three times the peak of the 2012 Spring Festival (1.19 million).
On February 14, 2013, parents and their children were waiting for their return journey at Huaibei Railway Station in Anhui Province. ICPhoto
However, the demand for the 12306 platform is still growing rapidly, and the original improvements are quickly reaching their limits. During the National Day Golden Week in 2013, 12306 Internet ticket sales reached 4.6 million, which was once again close to the upper limit of system processing. Moreover, due to the limitations of the 3G network (4G network was first year but not yet popular) and the rapid growth of the number of Internet users, the bandwidth of the external network entrance was tight during the peak period, which could no longer meet the needs of further increase in Internet ticket sales. In addition, as the main channel for railway ticket sales, the single-center operation model of the Internet ticketing system can no longer meet the needs of business security and reliability.
To this end, the second round of structural optimization of the 12306 website has been launched since the end of 2013:
Migrate services such as user login and frequently used contact inquiries to the memory database, improving the processing performance and reliability of related services.
The second production center of the Academy of Railway Sciences was built to achieve "double active" with the existing first production center of China State Railway Group Co., Ltd. to improve the security and reliability of the website and double the processing capacity of the order/e-ticket cluster. The order/e-ticket cluster has been expanded to 10 groups of nodes, 100 libraries, and 100 tables.
Deploy the ticket inquiry service on the public cloud. Through policy configuration, the ticket inquiry traffic can be diverted to the public cloud at any time to alleviate the pressure on the processing resources and bandwidth of the website during peak ticket sales periods.
The stress test before going online verified that the system could meet the designed ticket sales capacity of 10 million tickets per day. During the peak period of Spring Festival travel in 2015, the actual ticket sales speed exceeded 1,000 tickets per second (approximately 3.6 million tickets per hour). The public cloud diverted up to 75% of query requests during the Spring Festival in 2015, and the website's external ticket query service capacity increased three times. The 12306 website processed more than 18 billion ticket inquiry services during the peak period of Spring Festival travel in 2015, with an average response speed of more than 300,000 times/second.
As 12306's carrying capacity for the "extreme pressure" of Spring Festival travel has doubled, the "ticket-grabbing plug-in" that once caused headaches for the design and operation teams has become the biggest factor of instability. Although the 12306 team has repeatedly protested against the use of ticket grabbing software when purchasing tickets, there are still endless small software on the Internet that rely on automatically and frequently refreshing ticket information and automatically filling in reports to grab tickets. This type of software improves the success rate of ticket grabbing by quickly refreshing ticket information, aiming to help users grab tickets as soon as possible. Although it meets the market demand, it also seriously interferes with the public who buy tickets manually and affects the fairness of ticket purchasing. In order to improve the success rate, users often need to enter the personal information of the passengers into the system in advance. In the anxiety of "grabbing tickets", they personally leak the personal information.
Against this background, the 12306 team must also make targeted changes - in addition to continuously optimizing the system's peak processing capacity, it also introduced real-name systems, complex verification codes, mobile phone verification codes and other functions. However, it was still unable to curb the rampant use of ticket-grabbing software, resulting in the phenomenon of poor network ticket purchasing during the Spring Festival and the National Day Golden Week.
Until the Spring Festival in 2019, 12306 piloted the wait-list ticket purchase service function. This function puts manual user orders and ticket-grabbing software user orders into the same "prize pool", allowing everyone to compete fairly, and the system randomly selects "winners" to complete the subsequent ticket purchase process. In this way, no matter how fast the screen refreshes, the ticket-grabbing software has no advantage at all, and this has gradually curbed the popularity of ticket-grabbing software.
As the saying goes, "The devil is as good as the road." The waitlist ticket model is a heavy blow for 12306 against ticket grabbing software. However, driven by interests, ticket grabbing software will definitely continue to compete unfairly against this model, such as increasing the "winning probability" of ticket grabbing software orders in the waitlist ticket prize pool. I believe that our 12306 team can overcome difficulties and maintain the ticketing system with the largest number of visits and transactions in the world. The battle of wits and courage between the two parties is still evolving along with China's ever-developing railway network and residents' travel needs.