Abstract

In modern manufacturing processes, ensuring the precision of 3D profiles of products is crucial. Nonetheless, achieving this accuracy is challenging due to the complex interactions between process inputs and the data structure of the 3D profile data. Our solution, a 3D profile-based control framework, addresses this challenge by actively adapting and controlling the manufacturing process to enhance 3D shape accuracy. 3D profile scans represent the ultimate measure of desired part quality. Therefore, utilizing them as the system responses for control purposes yields the most direct and effective feedback. We leverage recent advancements from Koopman operator theory to create an effective model-based control strategy. Initially, we estimate the process model by exploring the relationship between 3D profiles and heterogeneous process inputs. Then, we formulate an online model predictive control law. Challenges include dealing with unstructured, high-dimensional 3D point cloud data, capturing spatial and temporal structures, and integrating heterogeneous, high-dimensional process input data into the control model. To overcome these challenges, we introduce RETROFIT, a solution designed for the real-time control of time-dependent 3D point cloud profiles. Unlike traditional models, RETROFIT is not bound by linear assumptions and can handle unstructured 3D point cloud data directly. We demonstrate its effectiveness through a wire arc additive manufacturing case study, highlighting its potential to enhance 3D profile accuracy in manufacturing processes.

Graphical Abstract Figure
Graphical Abstract Figure
Close modal

References

1.
Biehler
,
M.
,
Lin
,
D.
, and
Shi
,
J.
,
2023
, “
DETONATE: Nonlinear Dynamic Evolution Modeling of Time-Dependent 3-Dimensional Point Cloud Profiles
,”
IISE Trans.
,
56
(
5
), pp.
541
558
.
2.
Zhen
,
Z.
,
Paynabar
,
K.
, and
Shi
,
J.
,
2023
, “
Image-Based Feedback Control Using Tensor Analysis
,”
Technometrics
,
65
(
3
), pp.
305
314
.
3.
Rawlings
,
J. B.
,
Mayne
,
D. Q.
, and
Diehl
,
M.
,
2017
,
Model Predictive Control: Theory, Computation, and Design
,
Nob Hill Publishing
,
Madison, WI
.
4.
Jin
,
J.
, and
Shi
,
J.
,
1999
, “
State Space Modeling of Sheet Metal Assembly for Dimensional Control
,”
ASME J. Manuf. Sci. Eng.
,
121
(
4
), pp.
756
762
.
5.
Williams
,
R. L.
, and
Lawrence
,
D. A.
,
2007
,
Linear State-Space Control Systems
,
John Wiley & Sons
,
Hoboken, NJ
.
6.
Berber
,
R.
, and
Kravaris
,
C.
,
2012
,
Nonlinear Model Based Process Control
,
Springer Science & Business Media
,
Berlin, Germany
.
7.
Landers
,
R. G.
,
Barton
,
K.
,
Devasia
,
S.
,
Kurfess
,
T.
,
Pagilla
,
P.
, and
Tomizuka
,
M.
,
2020
, “
A Review of Manufacturing Process Control
,”
ASME J. Manuf. Sci. Eng.
,
142
(
11
), p.
110814
.
8.
Sikder
,
S.
,
Mukherjee
,
I.
, and
Panja
,
S. C.
,
2020
, “
A Synergistic Mahalanobis–Taguchi System and Support Vector Regression Based Predictive Multivariate Manufacturing Process Quality Control Approach
,”
J. Manuf. Syst.
,
57
, pp.
323
337
.
9.
Babu
,
A.
,
Yurtdas
,
K. Y.
,
Koch
,
C. E. S.
, and
Yüksel
,
M.
,
2019
, “
Trajectory Following Using Nonlinear Model Predictive Control and 3D Point-Cloud-Based Localization for Autonomous Driving
,”
2019 European Conference on Mobile Robots (ECMR)
,
Prague, Czech Republic
,
Sept. 4–6
,
IEEE
, pp.
1
6
.
10.
Ruiz
,
C.
,
Bhatt
,
P. M.
,
Gupta
,
S. K.
, and
Huang
,
Q.
,
2023
, “
Process-Informed Segmentation of Dense Point Clouds for Layer Quality Assessment in Large-Scale Metal Additive Manufacturing
,”
2023 IEEE 19th International Conference on Automation Science and Engineering (CASE)
,
Auckland, New Zealand
,
Aug. 26–30
,
IEEE
, pp.
1
6
.
11.
Shen
,
H.
,
Jin
,
J.
,
Liu
,
B.
, and
Zhou
,
Z.
,
2021
, “
Measurement and Evaluation of Laser-Scanned 3D Profiles in Wire Arc Hybrid Manufacturing Processes
,”
Measurement
,
176
, p.
109089
.
12.
Tang
,
S.
,
Wang
,
G.
, and
Zhang
,
H.
,
2019
, “
In Situ 3D Monitoring and Control of Geometric Signatures in Wire and Arc Additive Manufacturing
,”
Surf. Topogr.: Metrol. Prop.
,
7
(
2
), p.
025013
.
13.
Wu
,
R.
,
Yu
,
Z.
,
Ding
,
D.
,
Lu
,
Q.
,
Pan
,
Z.
, and
Li
,
H.
,
2021
, “
OICP: An Online Fast Registration Algorithm Based on Rigid Translation Applied to Wire Arc Additive Manufacturing of Mold Repair
,”
Materials
,
14
(
6
), p.
1563
.
14.
Biehler
,
M.
,
Yan
,
H.
, and
Shi
,
J.
, “
ANTLER: Bayesian Nonlinear Tensor Learning and Modeler for Unstructured, Varying-Size Point Cloud Data
,”
IEEE Trans. Autom. Sci. Eng.
,
21
(
1
), pp.
402
415
.
15.
Proctor
,
J. L.
,
Brunton
,
S. L.
, and
Kutz
,
J. N.
,
2016
, “
Dynamic Mode Decomposition With Control
,”
SIAM J. Appl. Dyn. Syst.
,
15
(
1
), pp.
142
161
.
16.
Williams
,
M. O.
,
Kevrekidis
,
I. G.
, and
Rowley
,
C. W.
,
2015
, “
A Data-Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
,”
J. Nonlinear Sci.
,
25
(
6
), pp.
1307
1346
.
17.
Yeung
,
E.
,
Kundu
,
S.
, and
Hodas
,
N.
,
2019
, “
Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems
,”
2019 American Control Conference (ACC)
,
IEEE
, pp.
4832
4839
.
18.
Schmid
,
P. J.
,
2022
, “
Dynamic Mode Decomposition and Its Variants
,”
Annu. Rev. Fluid Mech.
,
54
(
1
), pp.
225
254
.
19.
Lusch
,
B.
,
Kutz
,
J. N.
, and
Brunton
,
S. L.
,
2018
, “
Deep Learning for Universal Linear Embeddings of Nonlinear Dynamics
,”
Nat. Commun.
,
9
(
1
), p.
4950
.
20.
Morton
,
J.
,
Jameson
,
A.
,
Kochenderfer
,
M. J.
, and
Witherden
,
F.
,
2018
, “
Deep Dynamical Modeling and Control of Unsteady Fluid Flows
,”
Thirty-Second Annual Conference on Neural Information Processing Systems (NIPS)
,
Montreal, Canada
,
Dec. 2–8
.
21.
Otto
,
S. E.
, and
Rowley
,
C. W.
,
2019
, “
Linearly Recurrent Autoencoder Networks for Learning Dynamics
,”
SIAM J. Appl. Dyn. Syst.
,
18
(
1
), pp.
558
593
.
22.
Achlioptas
,
P.
,
Diamanti
,
O.
,
Mitliagkas
,
I.
, and
Guibas
,
L.
,
2018
, “
Learning Representations and Generative Models for 3D Point Clouds
,”
International Conference on Machine Learning
,
Stockholm, Sweden
,
July 20–25
, PMLR, pp.
40
49
23.
Brunton
,
S. L.
,
Budišić
,
M.
,
Kaiser
,
E.
, and
Kutz
,
J. N.
,
2022
, “
Modern Koopman Theory for Dynamical Systems
,”
SIAM Rev.
,
64
(
2
), pp.
229
340
.
24.
Korda
,
M.
, and
Mezić
,
I.
,
2018
, “
Linear Predictors for Nonlinear Dynamical Systems: Koopman Operator Meets Model Predictive Control
,”
Automatica
,
93
, pp.
149
160
.
25.
Proctor
,
J. L.
,
Brunton
,
S. L.
, and
Kutz
,
J. N.
,
2018
, “
Generalizing Koopman Theory to Allow for Inputs and Control
,”
SIAM J. Appl. Dyn. Syst.
,
17
(
1
), pp.
909
930
.
26.
Williams
,
M. O.
,
Hemati
,
M. S.
,
Dawson
,
S. T.
,
Kevrekidis
,
I. G.
, and
Rowley
,
C. W.
,
2016
, “
Extending Data-Driven Koopman Analysis to Actuated Systems
,”
IFAC-PapersOnLine
,
49
(
18
), pp.
704
709
.
27.
Bartlett
,
P. L.
,
Foster
,
D. J.
, and
Telgarsky
,
M. J.
,
2017
, “
Spectrally-Normalized Margin Bounds for Neural Networks
,”
The Thirty-First Annual Conference on Neural Information Processing Systems (NIPS)
,
Long Beach, CA
,
Dec. 4–9
.
28.
Mezić
,
I.
,
2022
, “
On Numerical Approximations of the Koopman Operator
,”
Mathematics
,
10
(
7
), p.
1180
.
29.
Virmaux
,
A.
, and
Scaman
,
K.
,
2018
, “
Lipschitz Regularity of Deep Neural Networks: Analysis and Efficient Estimation
,”
Thirty-Second Annual Conference on Neural Information Processing Systems (NIPS)
,
Montreal, Canada
,
Dec. 2–8
.
30.
Bullo
,
F.
, and
Lewis
,
A. D.
,
2019
,
Geometric Control of Mechanical Systems: Modeling, Analysis, and Design for Simple Mechanical Control Systems
,
Springer
,
New York
.
31.
Heinrich
,
L.
,
Feldhausen
,
T.
,
Saleeby
,
K. S.
,
Saldana
,
C.
, and
Kurfess
,
T. R.
,
2022
, “
Analysis of Conduction Cooling Strategies for Wire Arc Additive Manufacturing
,”
International Manufacturing Science and Engineering Conference
, Vol.
85819
,
American Society of Mechanical Engineers
, p.
V002T05A005
.
32.
Huang
,
Q.
,
Wang
,
Y.
,
Lyu
,
M.
, and
Lin
,
W.
,
2020
, “
Shape Deviation Generator – A Convolution Framework for Learning and Predicting 3-D Printing Shape Accuracy
,”
IEEE Trans. Autom. Sci. Eng.
,
17
(
3
), pp.
1486
1500
.
33.
Ruiz
,
C.
,
Jafari
,
D.
,
Venkata Subramanian
,
V.
,
Vaneker
,
T. H.
,
Ya
,
W.
, and
Huang
,
Q.
,
2022
, “
Prediction and Control of Product Shape Quality for Wire and Arc Additive Manufacturing
,”
ASME J. Manuf. Sci. Eng.
,
144
(
11
), p.
111005
.
34.
Lam
,
T. F.
,
Xiong
,
Y.
,
Dharmawan
,
A. G.
,
Foong
,
S.
, and
Soh
,
G. S.
,
2020
, “
Adaptive Process Control Implementation of Wire arc Additive Manufacturing for Thin-Walled Components With Overhang Features
,”
Int. J. Adv. Manuf. Technol.
,
108
, pp.
1061
1071
.
35.
Dharmawan
,
A. G.
,
Xiong
,
Y.
,
Foong
,
S.
, and
Soh
,
G. S.
,
2020
, “
A Model-Based Reinforcement Learning and Correction Framework for Process Control of Robotic Wire Arc Additive Manufacturing
,”
2020 IEEE International Conference on Robotics and Automation (ICRA)
,
Paris, France, May 31–Aug. 31
,
IEEE
, pp.
4030
4036
.
36.
Shi
,
J.
,
2022
, “
In-Process Quality Improvement: Concepts, Methodologies, and Applications
,”
IISE Trans.
,
55
(
1
), pp.
2
21
.
37.
Xia
,
C.
,
Pan
,
Z.
,
Polden
,
J.
,
Li
,
H.
,
Xu
,
Y.
,
Chen
,
S.
, and
Zhang
,
Y.
,
2020
, “
A Review on Wire arc Additive Manufacturing: Monitoring, Control and a Framework of Automated System
,”
J. Manuf. Syst.
,
57
, pp.
31
45
.
38.
Xiong
,
J.
, and
Zhang
,
G.
,
2014
, “
Adaptive Control of Deposited Height in GMAW-Based Layer Additive Manufacturing
,”
J. Mater. Process. Technol.
,
214
(
4
), pp.
962
968
.
39.
Doumanidis
,
C.
, and
Kwak
,
Y.-M.
,
2001
, “
Geometry Modeling and Control by Infrared and Laser Sensing in Thermal Manufacturing With Material Deposition
,”
ASME J. Manuf. Sci. Eng.
,
123
(
1
), pp.
45
52
.
40.
Xiong
,
J.
,
Yin
,
Z.
, and
Zhang
,
W.
,
2016
, “
Closed-Loop Control of Variable Layer Width for Thin-Walled Parts in Wire and Arc Additive Manufacturing
,”
J. Mater. Process. Technol.
,
233
, pp.
100
106
.
41.
Bonaccorso
,
F.
,
Cantelli
,
L.
, and
Muscato
,
G.
,
2011
, “
An Arc Welding Robot Control for a Shaped Metal Deposition Plant: Modular Software Interface and Sensors
,”
IEEE Trans. Ind. Electron.
,
58
(
8
), pp.
3126
3132
.
42.
Yang
,
J.
,
Nguyen
,
M. N.
,
San
,
P. P.
,
Li
,
X. L.
, and
Krishnaswamy
,
S.
,
2015
, “
Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition
,”
Twenty-Fourth International Joint Conference on Artificial Intelligence
,
Buenos Aires, Argentina
,
July 25–31
, pp.
3995
4001
.
43.
Valdarrama
,
S.
Convolutional Autoencoder for Image Denoising. https://keras.io/examples/vision/autoencoder/, Accessed February 2, 2024.
You do not currently have access to this content.