Recently, 10 Gbps or higher speed links are being widely deployed in data centers. Novel high-speed packet I/O frameworks have emerged to keep pace with such high-speed links. These frameworks mainly use techniques, such as memory preallocation, busy polling, zero copy, and batch processing, to replace costly operations (e.g., interrupts, packet copy, and system call) in native OS kernel stack. For high-speed packet I/O frameworks, costs per packet, saturation throughput, and latency are performance metrics that are of utmost concern, and various factors have an effect on these metrics. To acquire a comprehensive understanding of high-speed packet I/O, we propose an analytical model to formulate its packet forwarding (receiving-processing-sending) flow. Our model takes the four main techniques adopted by the frameworks into consideration, and the concerned performance metrics are derived from it. The validity and correctness of our model are verified by real system experiments. Moreover, we explore how each factor impacts the three metrics through a model analysis and then provide several useful insights and suggestions for performance tuning.

Tunnel crossing problem is unavoidable in the development of the topic crawler. To solve this problem, a self-decision topic crawler algorithm based on Boyd loop (FCIDOL) was proposed. The algorithm took the Boyd loop as the basic framework and formed a closed loop according to the principle of “observation-assessment-decision-action”. According to the work completed by the crawler, which refers to memory, the algorithm evaluated the current state observed to generate decisions of radical or conservative strategies, guiding the crawler to search for new theme-relevant web pages or to focus on the actions of short-term benefits. The role of memory was to provide training materials for the assessment network, thus realizing the online training of the network to meet the cold start of the crawler. The experiment shows that compared with various topic crawler algorithms in different topic environments, FCIDOL achieves an improvement of over 7.8% in harvest rate, and the number of duplicate links is reduced by more than 15.6%.