Optimizing Origin Servers: Load Balancing Strategies

With the explosive growth of the Internet, increasing traffic and concurrency have placed tremendous pressure on the origin servers of many enterprises. Moreover, the varying performance of these origin servers and their differing capacities to handle business loads have led to numerous issues, such as:

• Uneven traffic distribution, causing some servers to slow down or even fail due to overload.
• Cross-network or cross-region origin fetches resulting in slow access speeds.
• Inability to promptly detect and switch to alternate servers when the origin server is down.

These problems have significantly impacted business operations, where business continuity and high availability are crucial concerns. This is particularly true in industries like finance and e-commerce, where interruptions or delays in service can lead to substantial losses. CDNetworks’ origin load balancing function is designed to address these issues by helping enterprises with multiple origin servers balance services between them, alleviate the pressure on their origin servers, and ensure they can provide normal and stable business processing services as much as possible.

CDNetworks’ origin load balancing automatically switches incoming requests to backup origins when an origin server anomaly is detected, thereby avoiding losses due to single-origin failures. Additionally, it can route user requests to different origin servers based on the region, preventing requests from traversing regions unnecessarily and thereby improving the availability of the origin servers and enhancing end-user satisfaction.

CDNeworks Implementation method of origin load balancing

The main types of origin server load balancing include the following 4 types:

1. Origin server Round Robin polling

Round Robin polling refers to the method by which CDNetworks, when an enterprise has multiple origin servers, can configure the distribution of origin requests in an equal polling manner to ensure load balancing among multiple origin servers. For example, the load balancer will assign the first request to the first server, then the next request to the second server, and so on, until completing one round and returning to the beginning to assign requests to the first server again. This method is suitable for situations where the processing capabilities of each server are similar and each server handles a similar amount of business.

2. Origin server weighted round-robin

Weighted polling refers to the method by which CDNetworks, when an enterprise has multiple origin servers, can assign a weight to each origin server and then distribute origin requests according to this weight ratio to ensure load balancing among multiple origin servers.

The CDNetworks CDN system will distribute back-to-origin request in a polling manner based on the set weight. The higher the weight, the more requests each origin server will receive per polling cycle. Weighted polling adds weight-based request allocation on top of Round Robin polling. For example, if there are three origin servers A, B, and C, with weights set as A:B:C=1:2:3, then under normal circumstances, the allocation of the first 6 requests will follow the ratio of A:B:C=1:2:3. This means that server C, with the highest weight, will receive the most requests. Under normal circumstances, after each polling cycle, the request ratio between servers will be maintained according to the set weight ratio.

3. Primary and backup origin

The primary and backup origin refers to when an enterprise has two or more origin servers, CDNetworks can configure one or more of them as the primary origin and the others as a backup origin. This strategy monitors the primary and backup origins, ensuring that when the primary origin malfunctions, it can quickly switch to the backup origin to avoid losses caused by a single origin failure. Additionally, after the primary origin recovers, it automatically switches back to the primary origin to continue providing services. The primary and backup origin switching cycle is within 1 to 3 minutes.

4. Region-based origin fetching

Region-based origin fetching refers to when an enterprise has multiple origin servers in different regions, CDNetworks can divide user requests back to different origins based on user’s location to avoid requests crossing regions, ensuring proximity in response.

Suppose the enterprise provides origins in Japan and America, and configures where requests from Asia go back to the Japan origin, while requests from Europe and the United States go back to the US origin. When the website’s Asian and European-American user requests are close to a 1:1 ratio, this configuration ensures that the ratio of requests returned to the Japanese and US origins is close to 1:1, effectively maintaining load balance between the Japanese and US origins.

Key Benefits of CDNetworks’ Origin Load Balancing

CDNetworks’ Origin Load Balancing offers several crucial advantages that enhance the overall performance and reliability of enterprise origin servers. Here are the three main benefits:

Balance services between origin servers, alleviate the pressure on enterprise origin servers, improve stability and availability.

Timely detection of origin server abnormalities, swift switching between primary and backup origins, enhancing user experience.

Shorten response time by Region-based origin fetching, responding locally, and avoiding requests crossing regions.