One key challenge in materials informatics is how to effectively use the material data of small size to search for desired materials from a huge unexplored material space. We review the recent progress on the use of tools from data science and domain knowledge to mitigate the issues arising from limited materials data. The enhancement of data quality and amount via data augmentation and feature engineering is first summarized and discussed. Then the strategies that use ensemble model and transfer learning for improved machine learning model are overviewed. Next, we move to the active learning with emphasis on the uncertainty quantification and evaluation. Subsequently, the merits of the combination of domain knowledge and machine learning are stressed. Finally, we discuss some applications of large language models in the field of materials science. We summarize this review by posing the challenges and opportunities in the field of machine learning for small material data.