Journal Home > Volume 2 , Issue 2
Complex System Modeling and Simulation 2022, 2(2): 113-129 https://doi.org/10.23919/CSMS.2022.0002
Open Access | Issue | Published: 30 June 2022

Q-Learning-Based Teaching-Learning Optimization for Distributed Two-Stage Hybrid Flow Shop Scheduling with Fuzzy Processing Time

Show Author's Information Bingjie Xi1,  Deming Lei1( )
School of Automation, Wuhan University of Technology, Wuhan 430070, China
Keywords:
teaching-learning based optimization, Q-learning algorithm, two-stage hybrid flow shop scheduling, fuzzy processing time
Cite this article:
Xi B, Lei D. Q-Learning-Based Teaching-Learning Optimization for Distributed Two-Stage Hybrid Flow Shop Scheduling with Fuzzy Processing Time. Complex System Modeling and Simulation, 2022, 2(2): 113-129. https://doi.org/10.23919/CSMS.2022.0002
Download citation
EndNote(RIS)
BibTeX

779

Views

55

Downloads

Citations

22

Crossref

N/A

WoS

18

Scopus

N/A

CSCD

Abstract Full text About this article

Two-stage hybrid flow shop scheduling has been extensively considered in single-factory settings. However, the distributed two-stage hybrid flow shop scheduling problem (DTHFSP) with fuzzy processing time is seldom investigated in multiple factories. Furthermore, the integration of reinforcement learning and metaheuristic is seldom applied to solve DTHFSP. In the current study, DTHFSP with fuzzy processing time was investigated, and a novel Q-learning-based teaching-learning based optimization (QTLBO) was constructed to minimize makespan. Several teachers were recruited for this study. The teacher phase, learner phase, teacher’s self-learning phase, and learner’s self-learning phase were designed. The Q-learning algorithm was implemented by 9 states, 4 actions defined as combinations of the above phases, a reward, and an adaptive action selection, which were applied to dynamically adjust the algorithm structure. A number of experiments were conducted. The computational results demonstrate that the new strategies of QTLBO are effective; furthermore, it presents promising results on the considered DTHFSP.


menu
Abstract
Full text
Outline
About this article

Q-Learning-Based Teaching-Learning Optimization for Distributed Two-Stage Hybrid Flow Shop Scheduling with Fuzzy Processing Time

Show Author's information Bingjie Xi1, Deming Lei1( )
School of Automation, Wuhan University of Technology, Wuhan 430070, China

Abstract

Two-stage hybrid flow shop scheduling has been extensively considered in single-factory settings. However, the distributed two-stage hybrid flow shop scheduling problem (DTHFSP) with fuzzy processing time is seldom investigated in multiple factories. Furthermore, the integration of reinforcement learning and metaheuristic is seldom applied to solve DTHFSP. In the current study, DTHFSP with fuzzy processing time was investigated, and a novel Q-learning-based teaching-learning based optimization (QTLBO) was constructed to minimize makespan. Several teachers were recruited for this study. The teacher phase, learner phase, teacher’s self-learning phase, and learner’s self-learning phase were designed. The Q-learning algorithm was implemented by 9 states, 4 actions defined as combinations of the above phases, a reward, and an adaptive action selection, which were applied to dynamically adjust the algorithm structure. A number of experiments were conducted. The computational results demonstrate that the new strategies of QTLBO are effective; furthermore, it presents promising results on the considered DTHFSP.

Keywords: teaching-learning based optimization, Q-learning algorithm, two-stage hybrid flow shop scheduling, fuzzy processing time

References(49)

1

H. Allaoui and A. Artiba, Scheduling two-stage hybrid flow shop with availability constraints, Comput. Oper. Res., vol. 33, no. 5, pp. 1399–1419, 2006.
DOI Google Scholar
2

J. Yang, Minimizing total completion time in two-stage hybrid flow shop with dedicated machines, Comput. Oper. Res., vol. 38, no. 7, pp. 1045–1053, 2011.
DOI Google Scholar
3

S. Wang and M. Liu, A heuristic method for two-stage hybrid flow shop with dedicated machines, Comput. Oper. Res., vol. 40, no. 1, pp. 438–450, 2013.
DOI Google Scholar
4

S. J. Wang, X. D. Wang, F. Chu, and J. B. Yu, An energy-efficient two-stage hybrid flow shop scheduling problem in a glass production, Int. J. Prod. Res., vol. 58, no. 8, pp. 2283–2314, 2020.
DOI Google Scholar
5

X. Feng, F. Zheng, and Y. Xu, Robust scheduling of a two-stage hybrid flow shop with uncertain interval processing times, Int. J. Prod. Res., vol. 54, no. 12, pp. 3706–3717, 2016.
DOI Google Scholar
6

S. Wang and M. Liu, A genetic algorithm for two-stage no-wait hybrid flow shop scheduling problem, Comput. Oper. Res., vol. 40, no. 4, pp. 1064–1075, 2013.
DOI Google Scholar
7

Y. Tan, L. Mönch, and J. W. Fowler, A hybrid scheduling approach for a two-stage flexible flow shop with batch processing machines, J. Scheduling., vol. 21, no. 2, pp. 209–226, 2017.
DOI Google Scholar
8

B. Fan, W. Yang, and Z. Zhang, Solving the two-stage hybrid flow shop scheduling problem based on mutant firefly algorithm, J. Amb. Intel. Hum. Comp., vol. 10, no. 3, pp. 979–990, 2019.
DOI Google Scholar
9

G. M. Komaki, E. Teymourian, and V. Kayvanfar, Minimising makespan in the two-stage assembly hybrid flow shop scheduling problem using artificial immune systems, Int. J. Prod. Res., vol. 54, no. 4, pp. 963–983, 2016.
DOI Google Scholar
10

S. Wang and M. Liu, Two-stage hybrid flow shop scheduling with preventive maintenance using multi-objective tabu search method, Int. J. Prod. Res., vol. 52, no. 5, pp. 1495–1508, 2014.
DOI Google Scholar
11

S. W. Liu, J. Pei, H. Cheng, X. B. Liu, and P. M. Pardalos, Two-stage hybrid flow shop scheduling on parallel batching machines considering a job-dependent deteriorating effect and no-identical job sizes, Appl. Soft Comput., vol. 84, p. 105701, 2019.
DOI Google Scholar
12

O. Kheirandish, R. Tavakkoli-Moghaddam, and M. Karimi-Nasab, An artificial bee colony algorithm for a two-stage hybrid flowshop scheduling problem with multilevel product structures and requirement operations, Int. J. Comput. Integ. M., vol. 28, no. 5, pp. 437–450, 2015.
DOI Google Scholar
13

M. Rabiee, R. S. Rad, M. Mazinani, and R. Shafaei, An intelligent hybrid meta-heuristic for solving a case of no-wait two-stage flexible flow shop scheduling problem with unrelated parallel machines, Int. J. Adv. Manuf. Tech., vol. 71, nos. 5–8, pp. 1229–1245, 2014.
DOI Google Scholar
14

S. Lang, T. Reggelin, J. Schmidt, M. Müller, and A. Nahhas, NeuroEvolution of augmenting topologies for solving a two-stage hybrid flow shop scheduling problem: A comparison of different solution strategies, Expert Syst. Appl., vol. 172, p. 114666, 2021.
DOI Google Scholar
15

D. M. Lei and T. Wang, Solving distributed two-stage hybrid flowshop scheduling using a shuffled frog-leaping algorithm with memeplex grouping, Eng. Optimiz., vol. 52, no. 9, pp. 1461–1474, 2020.
DOI Google Scholar
16

J. C. Cai, R. Zhou, and D. M. Lei, Fuzzy distributed two-stage hybrid flow shop scheduling problem with setup time: Collabortive variable search, J. Intell. Fuzzy Syst., vol. 38, no. 3, pp. 3189–3199, 2020.
DOI Google Scholar
17

C. R. Vela, S. Afsar, and J. J. Palacios, Evolutionary tabu search for flexible due-date statisfaction in fuzzy job shop scheduling, Comput. Oper. Res., vol. 119, p. 104931, 2020.
DOI Google Scholar
18

Y. Dorfeshan, R. Tavakkoli-Moghaddam, S. M. Mousavi, and B. Vahedi-Nouri, A new weighted distance-based approximation methodology for flow shop scheduling group decisions under the interval-valued fuzzy processing time, Appl. Soft Comput., vol. 91, p. 106248, 2020.
DOI Google Scholar
19

L. Sun, L. Lin, M. Gen, and H. J. Li, A hybrid cooperative coevolution algorithm for fuzzy flexible job shop scheduling, IEEE T. Fuzzy Syst., vol. 27, no. 5, pp. 1008–1022, 2019.
DOI Google Scholar
20

J. Lin, Backtracking search based hyper-heuristic for the flexible job-shop scheduling problem with fuzzy processing time, Eng. Appl. Artif. Intell., vol. 77, pp. 186–196, 2019.
DOI Google Scholar
21

K. Z. Gao, P. N. Suganthan, Q. K. Pan, and M. F. Tasgetiren, An effective discrete harmony search algorithm for flexible job shop scheduling problem with fuzzy processing time, Int. J. of Prod. Res., vol. 53, no. 19, pp. 5896–5911, 2015.
DOI Google Scholar
22

J. Q. Li and Q. K. Pan, Chemical-reaction optimization for solving fuzzy job-shop scheduling problem with flexible maintenance activities, Int. J. of Prod. Econ., vol. 145, no. 1, pp. 4–17, 2013.
DOI Google Scholar
23

L. Wang, G. Zhou, Y. Xu, and M. Liu, A hybrid artificial bee colony algorithm for the fuzzy flexible job-shop scheduling problem, Int. J. of Prod. Econ., vol. 51, no. 12, pp. 3593–3608, 2013.
DOI Google Scholar
24

D. M. Lei, Co-evolutionary genetic algorithm for fuzzy flexible job shop scheduling, Appl. Soft Comput., vol. 12, no. 8, pp. 2237–2245, 2012.
DOI Google Scholar
25

K. Li, J. F. Chen, H. Fu, Z. H. Jia, and W. Z. Fu, Uniform parallel machine scheduling with fuzzy processing times under resource consumption constraint, Appl. Soft Comput., vol. 82, p. 105585, 2019.
DOI Google Scholar
26

Z. S. Shao, W. S. Shao, and D. C. Pi, Effective heuristics and metaheuristics for the distributed fuzzy blocking flow-shop scheduling problem, Swarm and EComput., vol. 59, p. 100747, 2020.
DOI Google Scholar
27

J. Zheng, L. Wang, and J. -J. Wang, A cooperative coevolution algorithm for multi-objective fuzzy distributed hybrid flow shop, Knowl. Based Syst., vol. 194, p. 105536, 2020.
DOI Google Scholar
28

R. V. Rao, V. J. Savsani, and D. P. Vakharia, Teaching-learning-based optimization: A novel method for constrained mechanical design optimization problems, Computer-Aided Design, vol. 43, no. 3, pp. 303–315, 2011.
DOI Google Scholar
29

W. W. Lin, D. Y. Yu, C. Y. Zhang, X. Liu, S. Q. Zhang, Y. H. Tian, S. Q. Liu, and Z. P. Xie, A multi-objective teaching-learning-based optimization algorithm to scheduling in turning processes for minimizing makespan and carbon footprint, J. Clean. Prod., vol. 101, pp. 337–347, 2015.
DOI Google Scholar
30

J. Q. Li, Q. K. Pan, and K. Mao, A discrete teaching-learning-based optimisation algorithm for realistic flowshop scheduling problems, Eng. Appl. Artif. Intell., vol. 37, pp. 279–292, 2015.
DOI Google Scholar
31

D. M. Lei, L. Gao, and Y. L. Zheng, A novel teaching-learning-based optimization algorithm for energy-efficient scheduling in hybrid flow shop, IEEE T. Eng. Manage., vol. 65, no. 2, pp. 330–340, 2018.
DOI Google Scholar
32

Y. Xu, L. Wang, S. Y. Wang, and M. Liu, An effective teaching-learning-based optimization algorithm for the flexible job-shop scheduling problem with fuzzy processing time, Neurocomputing, vol. 148, pp. 260–268, 2015.
DOI Google Scholar
33

W. S. Shao, D. C. Pi, and Z. S. Shao, A hybrid discrete teaching-learning based meta-heuristic for solving no-idle flow shop scheduling problem with total tardiness criterion, Comput. Oper. Res., vol. 94, pp. 89–105, 2018.
DOI Google Scholar
34

A. Mishra and D. Shrivastava, A TLBO and a Jaya heuristics for permutation flow shop scheduling to minimize the sum of inventory holding and batch delay costs, Comput. Ind. Eng., vol. 124, pp. 509–522, 2018.
DOI Google Scholar
35

R. Buddala and S. S. Mahapatra, Two-stage teaching-learning-based optimization method for flexible job-shop scheduling under machine breakdown, Int. J. of Adv. Manuf. Tech., vol. 100, nos. 5–8, pp. 1419–1432, 2019.
DOI Google Scholar
36

D. M. Lei, B. Su, and M. Li, Cooperated teaching-learning-based optimisation for distributed two-stage assembly flow shop scheduling, Int. J. of Prod. Res., vol. 59, no. 23, pp. 7232–7245, 2020.
DOI Google Scholar
37

R. Chen, B. Yang, S. Li, and S. Wang, A self-learning genetic algorithm based on reinforcement learning for flexible job-shop scheduling problem, Comput. Ind. Eng., vol. 149, p. 106778, 2020.
DOI Google Scholar
38

Z. Cao, C. Lin, M. Zhou, and R. Huang, Scheduling semiconductor testing facility by using cuckoo search algorithm with reinforcement learning and surrogate modeling, IEEE T. Autom. Sci. Eng., vol. 16, no. 2, pp. 825–837, 2018.
DOI Google Scholar
39

Z. Cao, C. Lin, and M. Zhou, A knowledge-based cuckoo search algorithm to schedule a flexible job shop with sequencing flexibility, IEEE T. Autom. Sci. Eng., vol. 18, no. 1, pp. 56–69, 2019.
DOI Google Scholar
40
H. Öztop, M. F. Tasgetiren, L. Kandiller, and Q. K. Pan, A novel general variable neighborhood search through Q-learning for no-idle flowshop scheduling, in Proc. IEEE Congress on Evolutionary Computation (CEC), Glasgow, UK, 2020, pp. 1–8.https://doi.org/10.1109/CEC48606.2020.9185556
DOI
41

J. Wang, D. M. Lei, and J. C. Cai, An adaptive artificial bee colony with reinforcement learning for distributed three-stage assembly scheduling with maintenance, Appl. Soft. Comput., vol. 117, p. 108371, 2021.
DOI Google Scholar
42

L. Wang, Z. X. Pan, and J. J. Wang, A review of reinforcement learning based intelligent optimization for manufacturing scheduling, Complex. Syst. Mod. Sim., vol. 1, no. 4, pp. 257–270, 2021.
DOI Google Scholar
43

M. Karimi-Mamaghan, M. Mohammadi, P. Meyer, A. M. Karimi-Mamaghan, and E. -G. Talbi, Machine learning at the service of meta-heuristics for solving combinatorial optimization problems: A state-of-the-art, Eur. J. Oper. Res., vol. 296, no. 2, pp. 393–422, 2022.
DOI Google Scholar
44

Q. L. Zhou, A novel movies recommendation algorithm based on reinforcement learning with DDPG policy, Int. J. Intell. Comput. Cybernetics, vol. 13, no. 1, pp. 67–79, 2020.
DOI Google Scholar
45

M. H. Pandit, R. N. Mir, M. A. Chishti, Adaptive task scheduling in IoT using reinforcement learning, Int. J. Intell. Comput. Cybernetics, vol. 13, no. 2, pp. 261–282, 2020.
DOI Google Scholar
46

J. C. H. Watkins and P. Dayan, Q-learning, Mach. Learn., vol. 8, no. 3, pp. 279–292, 1992.
DOI Google Scholar
47

K. C. Ying and S.W. Lin, Minimizing makespan for the distributed hybrid flowshop scheduling problem with multiprocessor tasks, Expert. Syst. Appl., vol. 92, pp. 132–141, 2018.
DOI Google Scholar
48

Y. L. Li, X. Y. Li, L. Gao, and L. L. Meng, An improved artificial bee colony algorithm for distributed heterogeneous hybrid flowshop scheduling problem with sequence-dependent setup times, Comput. Ind. Eng., vol. 147, p. 106638, 2020.
DOI Google Scholar
49

J. Q. Li, J. K. Li, L. J. Zhang, H. Y. Sang, Y. Y. Han, and Q. D. Chen, Solving type-2 fuzzy distributed hybrid flowshop scheduling using an improved brain storm optimization algorithm, Int. J. Fuzzy Syst., vol. 23, pp. 1194–1212, 2021.
DOI Google Scholar
Publication history
Copyright
Rights and permissions

Publication history

Received: 21 October 2021
Revised: 12 January 2022
Accepted: 19 January 2022
Published: 30 June 2022
Issue date: June 2022

Copyright

© The author(s) 2022

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return