As a new trend in Natural Language Processing (NLP), prompt tuning has been explored to provide a reliable answer without requiring massive labeled samples and training learning in sentiment and emotion detection tasks. However, how to effectively encode the commonness and uniquesness across difference affections into prompts sets a limit to the potential of multi-affection joint detection. To fill this gap, we propose a multi-affection prompt (MAP) learning framework that takes both the commonness of multiple affections and the uniqueness of specific affection into consideration. More specifically, two different prompt encoders are first proposed to elaborate the multi-task shared prompt and the task-specific prompt, respectively. Second, a multi-task prompt interaction learning layer is proposed to capture the correlation between the multi-task and task-specific prompts. MAP adopts separate multi-task and task-specific prompts to learn different vectors for different affection tasks, thus mitigating the affection discrepancy of the [MASK] token in the masked language modeling task. Extensive experiments on two benchmark datasets show that our proposed method can significantly improve the multi-task generalization capability of PLMs, and yield better results than other state-of-the-art (SOTA) baselines, by the margin of 2.7% and 3.4%.