Accurate and reliable production forecasting is a critical component for the efficient development of oil and gas fields and supports informed scientific decision-making. Although machine learning methods have achieved significant progress in this domain, existing models are typically trained from scratch using limited historical production data, making it difficult to effectively capture the complex nonlinear dynamics, long-term temporal dependencies, and high-dimensional interactions among variables inherent in production time series. This often leads to insufficient generalization capacity and limited predictive robustness. To address these challenges, this study proposes a novel gas well production forecasting method based on large language models (LLMs). The approach builds upon a pre-trained GPT-2 architecture and incorporates several key adaptations to enable effective time-series prediction. First, the input data—including daily gas production rate, tubing pressure, casing pressure, and production time—are subjected to instance normalization to facilitate knowledge transfer. Second, a trainable embedding layer is designed to map numerical time-series data into the semantic embedding space of the LLM, thereby achieving cross-modal alignment between continuous signals and the discrete representation format required by the model. Third, a parameter-efficient transfer learning strategy combining freezing and fine-tuning is implemented: the core self-attention and feed-forward network layers of the LLM are frozen to preserve general-purpose knowledge acquired during pre-training, while the positional encoding and layer normalization modules are selectively fine-tuned to enhance the model’s ability to characterize temporal patterns specific to production dynamics. The resulting model, termed GPT4TS, is systematically evaluated on real-world production data from a marine carbonate gas reservoir in the Sichuan Basin. Experimental results show that for wells with long production histories, GPT4TS significantly outperforms the conventional LSTM model. Under univariate input, the mean absolute percentage error (MAPE) is reduced by 18.573% on average; under multivariate input, the MAPE reduction reaches 35.610%, demonstrating its superior capability in modeling complex trends and leveraging multi-variable synergies. However, for newly commissioned wells with short production histories, insufficient data hinders effective fine-tuning, leading to lower prediction accuracy compared to LSTM. This study not only validates the potential of large language models in petroleum production forecasting but also highlights their strong dependence on historical data length, providing both theoretical insights and practical guidance for model selection in real-world engineering applications.