非凸不可分离问题的广义交替方向乘子法的收敛性

相关文章推荐

骑白马的毛巾 · paper 110：凸优化和非凸优化- ...· 1 年前 ·

骑白马的毛巾 · 凸优化和非凸优化-CSDN博客· 1 年前 ·

骑白马的毛巾 · 凸和非凸的理解_非凸函数-CSDN博客· 1 年前 ·

骑白马的毛巾 · 凸优化和非凸优化的区别-腾讯云开发者社区-腾讯云· 1 年前 ·

骑白马的毛巾 · 信息学院王浩团队在非凸优化算法研究领域发表重 ...· 1 年前 ·

交替方向乘子法（ADMM）是求解大规模优化问题和非凸非光滑问题的一种有效的方法，但当目标函数为非凸非光滑的情况时，原始ADMM算法的收敛性无法保证，且若目标函数中存在耦合函数，则算法的收敛性证明将更为复杂。在现实生活中存在的很多问题，其本质都是非凸的。因此，本文提出了一种改进的ADMM算法。与原始ADMM算法相比，该算法引入了一个松弛因子 $\alpha $ ，构造了一种广义交替方向乘子法（GADMM）来求解具有线性约束的非凸不可分离优化问题。在一定的假设条件下，通过假设增广拉格朗日函数满足K-L不等式，证明了当惩罚参数足够大时，算法生成的序列收敛到增广拉格朗日函数的稳定点。关键词 : 广义交替方向乘子法 K-L不等式非凸最优化不可分离问题收敛性分析 Abstract : The alternating direction method of multiplies (ADMM) is an effective method for solving large-scale optimization and nonconvex non-smooth objective. However, the convergence cannot be guaranteed when the objective function is nonconvex and non-smooth. Moreover, the proof of the convergence is more complex when the objective contains coupled function. Many problems in real life are nonconvex in nature, so the research on nonconvex optimization problems is particularly important. Therefore, an improved ADMM algorithm is proposed in this paper. Compared with the original ADMM algorithm, a relaxation factor $\alpha $ is introduced and a generalized alternating direction multiplier method (GADMM) is constructed to solve the non-convex nonseparable optimization problem with linear constraints. Under certain assumptions, by assuming that the augmented Lagrangian function satisfies the K-L inequality, it is proved that the sequence generated by the algorithm converges to the critical point of the augmented Lagrangian function when the penalty parameter is sufficiently large. Key words : generalized alternating direction multiplier method K-L inequality nonconvex optimization nonseparable problem convergence analysis $ \begin{array}{l}\mathrm{min}\;g\left(x,y\right)+f\left(x\right)+h\left(y\right)\\ {\rm{s.t.}}\;\;{\boldsymbol A}x+{\boldsymbol B}y=0\end{array} $ $ \begin{array}{l} \min \; f\left( x \right) + h\left( y \right) \\ {\rm{s.t.}}\; \; {\boldsymbol A}x + {\boldsymbol B}y = 0 \\ \end{array} $ $ \left\{\begin{array}{l}{x}^{k+1}\in \mathrm{arg}\;{\mathrm{min}}_{x}\left\{f\left(x\right)-\langle \lambda ,{\boldsymbol A}x\rangle +\dfrac{\beta }{2}{\Vert {\boldsymbol A}x+{\boldsymbol B}{y}^{k}\Vert }^{2}\right\}\\ {y}^{k+1}\in \mathrm{arg}\;{\mathrm{min}}_{y}\left\{g\left(y\right)-\langle \lambda ,{\boldsymbol B}y\rangle +\dfrac{\beta }{2}{\Vert {\boldsymbol A}{x}^{k+1}+{\boldsymbol B}y\Vert }^{2}\right\}\\ {\lambda }^{k+1}={\lambda }^{k}-\rho \left({\boldsymbol A}{x}^{k+1}+{\boldsymbol B}{y}^{k+1}\right)\end{array}\right. $

ADMM算法通过引入一个新的辅助变量，将原问题改写为一个目标函数可分离且辅助变量与原变量是线性约束的形式，通过交替更新原变量、辅助变量和对偶变量来迭代求解问题的最优解。通过引入合适的辅助变量，每个迭代步骤可以变成非常简单的子问题，通常可以收敛到稳定点或者被并行求解。这使得ADMM算法适用于求解大规模的优化问题。

$ f $ $ h $ 都是凸函数的情况，ADMM（式（3））的收敛性得到了很好的证明，并且对其进行了收敛率分析 ^{[

1

-

3

]} 。在没有凸性的假设下，更难以证明ADMM的收敛性。在这方面的研究取得了一些进展 ^{[

4

-

8

]} 。Guo等 ^{[

4

]} 利用经典ADMM算法求解非凸多块可分离最优化问题，证明了其收敛性，并且提出了一些充分条件，保证了算法的超线性和线性收敛速率。Li等 ^{[

5

]} 提出了一种近似ADMM算法来解决非凸非光滑优化问题，证明了当罚参数足够大且生成的序列有聚点时，算法所产生迭代点列收敛到稳定点。

目前的研究大多数考虑如下的问题：

$ \begin{array}{l} \min \; g\left( {x,y} \right) + f\left( x \right) + h\left( y \right) \\ {\rm{s.t.}}\; \; {\boldsymbol{A}}x + y = b \\ \end{array} $ 是二次函数的情况下，分析了ADMM解决问题（4）的收敛性。Liu等 ^{[

11

]} 提出了线性化ADMM来解决非凸非光滑的目标问题，通过将目标中的可微项以及增广项线性化，证明了算法的收敛性，然后将问题推广到多块并行的ADMM算法中。以上研究都是矩阵 ${\boldsymbol B}$ 为单位阵的情况。在耦合函数缺失的情况下，Wang等 ^{[

12

]} 研究了用Bregman距离修饰ADMM算法（BADMM算法），分别分析了矩阵 ${\boldsymbol B}$ 单射和非单射情况下算法的收敛性，证明了BADMM算法生成的序列收敛到稳定点。Guo等 ^{[

13

]} 提出了一种广义的ADMM算法来求解问题（4）。

本文研究广义ADMM算法的收敛性，通过假设矩阵 ${\boldsymbol B}$ 列满秩以及增广拉格朗日函数满足K-L不等式，当罚参数足够大时，证明了算法产生的序列收敛到其稳定点，从而证明了算法的收敛性。

2 预备知识

现给出理论分析所需要的概念和性质。

$x \in {{\bf{R}}^n}$ 的极小值点的必要条件是 $0 \in \partial f\left( x \right)$ ，满足这个条件的点称为稳定点，函数的稳定点集记作 ${\rm{crit}}\; f$

定义1　令 $f:{{\bf R}^n} \to R \cup \left\{ { + \infty } \right\}$ 为正常的下半连续函数。

a . Fréchet次微分。

$x \in {\rm {dom}}\; f$ 的Fréchet次微分，定义为满足下列关系 ${x^*} \in {{\bf R}^n}$ $\left\| {f\left( x \right) - f\left( y \right)} \right\| \leqslant L\left\| {x - y} \right\|$ 满足Lipschitz连续条件，为Lipschitz常数。

$ h:{{\bf{R}}^m} \to R $ 是连续可微函数，且 $ \nabla h $ 是关于常数 L 利普希茨连续的，那么，对于任意的 $ x,y \in {{\bf R}^n} $

d . 对所有的 $x \in U \cap \left[ {f\left( {{x^*}} \right) < f < f\left( {{x^*}} \right) + \eta } \right]$ ，有Kurdyka-Lojasiewicz不等式

$x \in \left\{ {x \in {{\bf{R}}^n}:d\left( {x,\varOmega } \right) < \varepsilon } \right\} \cap \{ f\left( {\tilde x} \right) < f < f\left( {\tilde x} \right) + \eta \}$ $ \left\{ \begin{array}{l} {x^{k + 1}} \in \arg {\min _x}\Biggr\{ g\left( {x,{y^k}} \right) + f\left( x \right) - \left\langle {{\gamma ^k},{\boldsymbol{A}}x} \right\rangle + \dfrac{\beta }{2}{{\left\| {{\boldsymbol{A}}x + {\boldsymbol{B}}{y^k}} \right\|}^2} \Biggr\} \\ {y^{k + 1}} \in \arg {\min _y}\Biggr\{ g\left( {{x^{k + 1}},y} \right) + h\left( y \right) - \left\langle {{\gamma ^k},{\boldsymbol{B}}y} \right\rangle +\dfrac{\beta }{2}{{\left\| {\alpha {\boldsymbol{A}}{x^{k + 1}} - \left( {1 - \alpha } \right){\boldsymbol{B}}{y^k} + {\boldsymbol{B}}y} \right\|}^2} \Biggr\} \\ {\gamma ^{k + 1}} = {\gamma ^k} - \beta \left( \alpha {\boldsymbol{A}}{x^{k + 1}} + \left( {1 - \alpha } \right){y^k} + {\boldsymbol{B}}{y^{k + 1}} \right) \end{array} \right. $ $ \left\{ \begin{array}{l} 0 \in \partial f\left( {{x^{k + 1}}} \right) + {\nabla _x}g\left( {{x^{k + 1}},{y^k}} \right) - {{\boldsymbol{A}}^{\rm T}}{\gamma ^k} + \\ \qquad \beta {{\boldsymbol{A}}^{\rm T}}\left( {{\boldsymbol{A}}{x^{k + 1}} + {\boldsymbol{B}}{y^k}} \right) \\ 0 = \nabla h\left( {{y^{k + 1}}} \right) + {\nabla _y}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) - {{\boldsymbol{B}}^{\rm T}}{\gamma ^k} + \\ \qquad {{\boldsymbol{B}}^{\rm T}}\beta \left( {\alpha {\boldsymbol{A}}{x^{k + 1}} - \left( {1 - \alpha } \right){\boldsymbol{B}}{y^k} + {\boldsymbol{B}}{y^{k + 1}}} \right) \\ {\gamma ^{k + 1}} = {\gamma ^k} - \beta \left( {\alpha {\boldsymbol{A}}{x^{k + 1}} + \left( {1 - \alpha } \right){y^k} + {\boldsymbol{B}}{y^{k + 1}}} \right) \end{array} \right. $ $ \left\{ \begin{array}{l} {{\boldsymbol{A}}^{\rm T}}{\gamma ^k} - \beta {{\boldsymbol{A}}^{\rm T}}\left( {{\boldsymbol{A}}{x^{k + 1}} + {\boldsymbol{B}}{y^k}} \right) - {\nabla _x}g\left( {{x^{k + 1}},{y^k}} \right) \in\\ \qquad\quad\partial f\left( {{x^{k + 1}}} \right) \\ \nabla h\left( {{y^{k + 1}}} \right) = {{\boldsymbol{B}}^{\rm T}}{\gamma ^{k + 1}} - {\nabla _y}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) \\ {\gamma ^{k + 1}} = {\gamma ^k} - \beta \left( {\alpha {\boldsymbol{A}}{x^{k + 1}} - \left( {1 - \alpha } \right){y^k} + {\boldsymbol{B}}{y^{k + 1}}} \right) \end{array} \right. $

a . 矩阵 B 是列满秩的，且 ${\rm{Im}} \left( {\boldsymbol{A}} \right) \subset {\rm{Im}} \left( {\boldsymbol{B}} \right)$ $\left\| {{\nabla _y}g\left( {x,y} \right) - {\nabla _y}g\left( {x,\tilde y} \right)} \right\| \leqslant {L_1}\left( x \right)\left\| {y - \tilde y} \right\|,\forall y,\tilde y \in {{\bf{R}}^n} $ $ \left\| {{\nabla _x}g\left( {x,y} \right) - {\nabla _x}g\left( {\tilde x,y} \right)} \right\| \leqslant {L_2}\left( y \right)\left\| {x - \tilde x} \right\|,\forall x,\tilde x \in {{\bf{R}}^n} $ $\nabla g$ ${{\bf{R}}^n} \times {{\bf{R}}^m}$ 的有界子集上是利普希茨连续的，即对于每个有界子集 $ {B_1} \times {B_2} \subseteq {{\bf{R}}^n} \times {{\bf{R}}^m} $ $M > 0$ ，使得对于所有的 $\left( {{x_i},{y_i}} \right) \in {B_1} \times {B_2},i = 1,2$ $\| {\nabla _x}H\left( {{x_1},{y_1}} \right) - $ $ {\nabla _x}H\left( {{x_2},{y_2}} \right)，{\nabla _y}H\left( {{x_1},\;{y_1}} \right) - {\nabla _y}H\left( {{x_2},{y_2}} \right) \| \leqslant M\| ( {x_1} - {y_1}$ ${x_2} - {y_2} ) \|$ $ \begin{array}{l} \tilde \beta : = \max \left\{ {\dfrac{{\alpha {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}\left( {{L_3} + \omega } \right) + \sqrt {{\alpha ^2}{{\left( {{L_3} + \omega } \right)}^2} + 16\alpha \mu {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}{M^2}} }}{{2\alpha \mu {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}} \right. \\ \qquad \left. {\dfrac{{\alpha {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}\left( {{L_1} + {L_2}} \right) + \sqrt {{\alpha ^2}\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}^2{{\left( {{L_1} + {L_2}} \right)}^2} + 16\left( {2 - \alpha } \right)\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}^2\left( {L_1^2 + {M^2}} \right)} }}{{2\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}^2\left( {2 - \alpha } \right)}}} \right\} \end{array} $ $\begin{aligned} {\widehat L_\beta }\left( {x,y,\gamma ,\alpha ,\vartheta } \right) =& g\left( {x,y} \right) + f\left( x \right) + h\left( y \right) - \left\langle {\gamma ,{\boldsymbol{A}}x + {\boldsymbol{B}}y} \right\rangle + \\ &\frac{\beta }{2}{\left\| {\alpha {\boldsymbol{A}}x - \left( {1 - \alpha } \right){\boldsymbol{B}}\vartheta + {\boldsymbol{B}}y} \right\|^2} \end{aligned} $ ${\rm{Im}}\; \left( {\boldsymbol{B}} \right) = {\rm{Im}} \;\left( {{{\boldsymbol{Q}}^{\rm T}}} \right)$ ${\gamma ^l} - {\gamma ^k} \in {\rm{Im}}\; \left( {{{\boldsymbol{Q}}^{\rm T}}} \right)$ $\|{\gamma ^l} - {\gamma ^k} \|^2 = $ $ {\left\| {{\boldsymbol{Q}}\left( {{\gamma ^l} - {\gamma ^k}} \right)} \right\|^2}$ 。由此得到

$ \begin{aligned} {\left\| {{{\boldsymbol{B}}^{\rm T}}\left( {{\gamma ^l} - {\gamma ^k}} \right)} \right\|^2} =\ & {\left\| {{\boldsymbol{PQ}}\left( {{\gamma ^l} - {\gamma ^k}} \right)} \right\|^2} \geqslant \\ &{\lambda _{{{\boldsymbol{P}}^{\rm T}}{\boldsymbol{P}}}}{\left\| {{\boldsymbol{Q}}\left( {{\gamma ^l} - {\gamma ^k}} \right)} \right\|^2} \geqslant {\lambda _{{{\boldsymbol{P}}^{\rm T}}{\boldsymbol{P}}}}{\left\| {{\gamma ^l} - {\gamma ^k}} \right\|^2} \end{aligned} $ 的定义，有 $ {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}} = {\lambda _{{\boldsymbol{P}}{{\boldsymbol{P}}^{\rm T}}}} $ ，又因为线性代数的常见结论 $ {\lambda _{{{\boldsymbol{P}}^{\rm T}}{\boldsymbol{P}}}} = {\lambda _{{\boldsymbol{P}}{{\boldsymbol{P}}^{\rm T}}}} $ ${\lambda _{{{\boldsymbol{P}}^{\rm T}}{\boldsymbol{P}}}} = $ $ {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}$

引理5　令 $ \left\{ {\left( {{x^k},{y^k},{\gamma ^k}} \right)} \right\} $ 由算法GADMM式（式（5））产生，则有

$ {L_\beta }\left( {{\omega ^k}} \right) - {L_\beta }\left( {{\omega ^{k + 1}}} \right) \geqslant \delta {\left\| {{v^{k + 1}} - {v^k}} \right\|^2} $ $ \begin{split} {L_\beta }\left( {{\omega ^k}} \right) - {L_\beta }\left( {{\omega ^{k + 1}}} \right) = &{L_\beta }\left( {{x^k},{y^k},{\gamma ^k}} \right) - {L_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^{k + 1}}} \right) = {\widehat L_\beta }\left( {{x^k},{y^k},{\gamma ^k},\alpha,y^k } \right) - {\widehat L_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^{k + 1}},\alpha,y^k } \right) =\\ &\left({\widehat L_\beta }\left( {{x^k},{y^k},{\gamma ^k},\alpha,y^k } \right) - {\widehat L_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha,y^k } \right)\right) + \left({\widehat L_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha,y^k } \right) - \right. \\ &\left.{\widehat L_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha ,{y^k}} \right)\right) + \left({\widehat L_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha ,{y^k}} \right) - {\widehat L_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^k},\alpha ,{y^k}} \right)\right) + \\ &\left({\widehat L_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^k},\alpha ,{y^k}} \right) - {\widehat L_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^{k + 1}},\alpha,y^k } \right)\right) \end{split} $ $ \begin{split} &{\widehat L_\beta }\left( {{x^k},{y^k},{\gamma ^k},\alpha,y^k } \right) - {\widehat L_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha,y^k } \right) = \\ &\qquad f\left( {{x^k}} \right) - f\left( {{x^{k + 1}}} \right) + \left\langle {{\gamma ^k},A{x^{k + 1}} - A{x^k}} \right\rangle +\\ &\qquad \beta \left\langle {{\boldsymbol{A}}{x^{k + 1}} + {\boldsymbol{B}}{y^k},{\boldsymbol{A}}{x^k} - {\boldsymbol{A}}{x^{k + 1}}} \right\rangle + \\ &\qquad\frac{\beta }{2}{\left\| {{\boldsymbol{A}}{x^k} - {\boldsymbol{A}}{x^{k + 1}}} \right\|^2} + g\left( {{x^k},{y^k}} \right) - g\left( {{x^{k + 1}},{y^k}} \right) \end{split} $ $\begin{aligned} f\left( {{x^k}} \right) \geqslant &f\left( {{x^{k + 1}}} \right) + \left\langle {{\boldsymbol{A}}^{\rm T}}{\gamma ^k} - \beta {{\boldsymbol{A}}^{\rm T}}\left( {{\boldsymbol{A}}{x^{k + 1}} + {\boldsymbol{B}}{y^k}} \right) -\right.\\ &\left.{\nabla _x}g\left( {{x^{k + 1}},{y^k}} \right),{x^k} - {x^{k + 1}} \right\rangle - \frac{\omega }{2}{\left\| {{x^{k + 1}} - {x^k}} \right\|^2}\end{aligned} $ $\begin{split}{\widehat L_\beta }\left( {{x^k},{y^k},{\gamma ^k},\alpha,y^k } \right)- &{\widehat L_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha,y^k } \right) \geqslant \\ &\frac{{\beta \mu - {L_3}\left( {{y^k}} \right) - \omega }}{2}{\left\| {{x^{k + 1}} - {x^k}} \right\|^2}\end{split} $ $ \begin{split} &{\widehat L_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha ,{y^k}} \right) - {\widehat L_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^k},\alpha ,{y^k}} \right) =\\ &\quad h\left( {{y^k}} \right) - h\left( {{y^{k + 1}}} \right) + g\left( {{x^{k + 1}},{y^k}} \right) - g\left( {{x^{k + 1}},{y^{k + 1}}} \right) +\\ &\quad \left\langle {{\gamma ^k},{\boldsymbol{B}}\left( {{y^{k + 1}} - {y^k}} \right)} \right\rangle - \alpha \beta \left\langle {A{x^{k + 1}} + {\boldsymbol{B}}{y^k},{\boldsymbol{B}}\left( {{y^{k + 1}} - {y^k}} \right)} \right\rangle -\\ &\quad \dfrac{\beta }{2}{\left\| {{\boldsymbol{B}}{y^{k + 1}} - {\boldsymbol{B}}{y^k}} \right\|^2} = h\left( {{y^k}} \right) - h\left( {{y^{k + 1}}} \right) + g\left( {{x^{k + 1}},{y^k}} \right) -\\ &\quad g\left( {{x^{k + 1}},{y^{k + 1}}} \right) + \; \left\langle {{\gamma ^{k + 1}},{\boldsymbol{B}}\left( {{y^{k + 1}} - {y^k}} \right)} \right\rangle +\\ &\quad\dfrac{\beta }{2}{\left\| {{\boldsymbol{B}}{y^{k + 1}} - {\boldsymbol{B}}{y^k}} \right\|^2}\\[-15pt] \end{split} $ $ \begin{aligned}g\left( {{x^{k + 1}},{y^k}} \right) -& g\left( {{x^{k + 1}},{y^{k + 1}}} \right) \geqslant \left\langle {{\nabla _y}g\left( {{x^{k + 1}},{y^{k + 1}}} \right),{y^k} - {y^{k + 1}}} \right\rangle -\\ &\frac{{{L_2}\left( {{x^{k + 1}}} \right)}}{2}{\left\| {{y^{k + 1}} - {y^k}} \right\|^2}\end{aligned} $ $ \begin{split}&{\widehat L_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha ,{y^k}} \right) - {\widehat L_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^k},\alpha ,{y^k}} \right) \geqslant \\ &\qquad\frac{{\beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}} - {L_2}\left( {{x^{k + 1}}} \right) - {L_1}}}{2}{\left\| {{y^{k + 1}} - {y^k}} \right\|^2}\end{split} $ $ \begin{aligned} &\left( {{{\widehat L}_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha,y^k } \right) - {{\widehat L}_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha ,{y^k}} \right)} \right) + \\ &\qquad\left( {{\widehat L}_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^k},\alpha ,{y^k}} \right) -\right. \\ &\qquad\left. {{\widehat L}_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^{k + 1}},\alpha,y^k} \right) \right) \end{aligned} $ $ \begin{split} &{\widehat L_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha,y^k } \right) - {\widehat L_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha ,{y^k}} \right) = \\ &\qquad\frac{\beta }{2}{\left\| {{\boldsymbol{A}}{x^{k + 1}} + {\boldsymbol{B}}{y^k}} \right\|^2} - \frac{\beta }{2}{\left\| {\alpha \left( {{\boldsymbol{A}}{x^{k + 1}} + {\boldsymbol{B}}{y^k}} \right)} \right\|^2} \end{split} $ $ \begin{split} &{\widehat{L}}_{\beta }\left({x}^{k+1},{y}^{k+1},{\gamma }^{k},\alpha ,{y}^{k}\right)-{\widehat{L}}_{\beta }\left({x}^{k+1},{y}^{k+1},{\gamma }^{k+1},\alpha,y^k \right) =\\ &\quad\langle {\gamma }^{k+1}-{\gamma }^{k},{\boldsymbol{A}}{x}^{k+1}+{\boldsymbol{B}}{y}^{k+1}\rangle +\dfrac{\beta }{2}\Vert \alpha \left({\boldsymbol{A}}{x}^{k+1}+{\boldsymbol{B}}{y}^{k}\right)+\\ &\quad B\left({y}^{k+1}-{y}^{k}\right)\Vert ^{2}-\dfrac{\beta }{2}{\Vert {\boldsymbol{A}}{x}^{k+1}+{\boldsymbol{B}}{y}^{k+1}\Vert }^{2} =\\ &\quad\langle {\gamma }^{k+1}-{\gamma }^{k},{\boldsymbol{A}}{x}^{k+1}+{\boldsymbol{B}}{y}^{k+1}\rangle +\dfrac{\beta }{2}{\Vert \alpha \left({\boldsymbol{A}}{x}^{k+1}+{\boldsymbol{B}}{y}^{k}\right)\Vert }^{2}+\\ &\quad\alpha \beta \langle {\boldsymbol{A}}{x}^{k+1}+{\boldsymbol{B}}{y}^{k},{\boldsymbol{B}}\left({y}^{k+1}-{y}^{k}\right)\rangle -\dfrac{\beta }{2}{\Vert {\boldsymbol{A}}{x}^{k+1}+{\boldsymbol{B}}{y}^{k}\Vert }^{2}-\\ &\quad\beta \langle {\boldsymbol{A}}{x}^{k+1}+{\boldsymbol{B}}{y}^{k},{\boldsymbol{B}}\left({y}^{k+1}-{y}^{k}\right)\rangle \\[-12pt] \end{split} $ $ \begin{split} &\left({\widehat{L}}_{\beta }\left({x}^{k+1},{y}^{k},{\gamma }^{k},\alpha,y^k \right)-{\widehat{L}}_{\beta }\left({x}^{k+1},{y}^{k},{\gamma }^{k},\alpha ,{y}^{k}\right)\right)+\\ &\quad \left({\widehat{L}}_{\beta }\left({x}^{k+1},{y}^{k+1},{\gamma }^{k},\alpha ,{y}^{k}\right)-{\widehat{L}}_{\beta }\left({x}^{k+1},{y}^{k+1},{\gamma }^{k+1},\alpha,y^k \right)\right) =\\ &\quad \langle {\gamma }^{k+1}-{\gamma }^{k},{\boldsymbol{A}}{x}^{k+1}+{\boldsymbol{B}}{y}^{k+1}\rangle +\alpha \beta \langle {\boldsymbol{A}}{x}^{k+1}+\\ &\quad {\boldsymbol{B}}{y}^{k},{\boldsymbol{B}}\left({y}^{k+1}-{y}^{k}\right)\rangle -\beta \langle {\boldsymbol{A}}{x}^{k+1}+{\boldsymbol{B}}{y}^{k},{\boldsymbol{B}}\left({y}^{k+1}-{y}^{k}\right)\rangle =\\ &\quad \langle {\gamma }^{k+1}-{\gamma }^{k},-\dfrac{1}{\alpha \beta }\left({\gamma }^{k+1}-{\gamma }^{k}\right)-\dfrac{1-\alpha }{\alpha }{\boldsymbol{B}}\left({y}^{k+1}-{y}^{k}\right)\rangle +\\ &\quad \dfrac{1-\alpha }{\alpha }\langle {\gamma }^{k+1}-{\gamma }^{k}+\beta \left({\boldsymbol{B}}{y}^{k+1}-{\boldsymbol{B}}{y}^{k}\right),{\boldsymbol{B}}\left({y}^{k+1}-{y}^{k}\right)\rangle =\\ &\quad -\dfrac{1}{\alpha \beta }{\Vert {\gamma }^{k+1}-{\gamma }^{k}\Vert }^{2}+\dfrac{\left(1-\alpha \right)\beta }{\alpha }{\Vert {\boldsymbol{B}}\left({y}^{k+1}-{y}^{k}\right)\Vert }^{2}\\[-15pt]\end{split} $ $ \begin{aligned} & {\left\| {{{\boldsymbol{B}}^{\rm T}}{\gamma ^{k + 1}} - {{\boldsymbol{B}}^{\rm T}}{\gamma ^k}} \right\|^2} =\left\| \nabla h\left( {{y^{k + 1}}} \right) + {\nabla _y}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) -\right.\\ &\quad\left.\nabla h\left( {{y^k}} \right) - {\nabla _y}g\left( {{x^k},{y^k}} \right) \right\|^2 \leqslant 2\left\| \nabla h\left( {{y^{k + 1}}} \right) -\right.\\ &\quad\left. \nabla h\left( {{y^k}} \right) \right\|^2 + 2{\left\| {{\nabla _y}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) - {\nabla _y}g\left( {{x^k},{y^k}} \right)} \right\|^2} \leqslant\\ &\quad 2L_1^2{\left\| {{y^{k + 1}} - {y^k}} \right\|^2} + 2{M^2}{\left\| {{x^{k + 1}} - {x^k}} \right\|^2} + 2{M^2}{\left\| {{y^{k + 1}} - {y^k}} \right\|^2} =\\ &\quad\left( {2L_1^2 + 2{M^2}} \right){\left\| {{y^{k + 1}} - {y^k}} \right\|^2} + 2{M^2}{\left\| {{x^{k + 1}} - {x^k}} \right\|^2} \end{aligned} $ $\begin{split} {\left\| {{\gamma ^{k + 1}} - {\gamma ^k}} \right\|^2} \leqslant& \frac{1}{{{\lambda _{{B^{\rm T}}B}}}}\left( \left( {2L_1^2 + 2{M^2}} \right){{\left\| {{y^{k + 1}} - {y^k}} \right\|}^2} +\right.\\ & \left. 2{M^2}{{\left\| {{x^{k + 1}} - {x^k}} \right\|}^2} \right)\end{split} $ $ \begin{split} &\left( {{{\widehat L}_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha,y^k } \right) - {{\widehat L}_\beta }\left( {{x^{k + 1}},{y^k},{\gamma ^k},\alpha ,{y^k}} \right)} \right) + \\ &\qquad\left( {{{\widehat L}_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^k},\alpha ,{y^k}} \right) - {{\widehat L}_\beta }\left( {{x^{k + 1}},{y^{k + 1}},{\gamma ^{k + 1}},\alpha,y^k } \right)} \right) \geqslant \\ &\qquad- \frac{1}{{\alpha \beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}\left( \left( {2L_1^2 + 2{M^2}} \right){\left\| {{y^{k + 1}} - {y^k}} \right\|}^2 +\right.\\ &\qquad \left.2{M^2}{{\left\| {{x^{k + 1}} - {x^k}} \right\|}^2} \right) + \frac{{\left( {1 - \alpha } \right)\beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}{\alpha }{\left\| {{y^{k + 1}} - {y^k}} \right\|^2} = \\ &\qquad \left( {\frac{{\left( {1 - \alpha } \right)\beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}{\alpha } - \frac{{2L_1^2 + 2{M^2}}}{{\alpha \beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}} \right){\left\| {{y^{k + 1}} - {y^k}} \right\|^2} -\\ &\qquad\frac{{2{M^2}}}{{\alpha \beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}{\left\| {{x^{k + 1}} - {x^k}} \right\|^2} \\[-15pt] \end{split} $ $ \begin{aligned} {L_\beta }\left( {{\omega ^k}} \right) -\ & {L_\beta }\left( {{\omega ^{k + 1}}} \right) \geqslant \left( {\frac{{\beta \mu - {L_3}\left( {{y^k}} \right) - \omega }}{2} - \frac{{2{M^2}}}{{\alpha \beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}} \right)\times\\ &{\left\| {{x^{k + 1}} - {x^k}} \right\|^2}+ \left( \frac{{\beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}} - {L_2}\left( {{x^{k + 1}}} \right) - {L_1}}}{2}+\right.\\ &\left.\frac{{\left( {1 - \alpha } \right)\beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}{\alpha } - \frac{{2L_1^2 + 2{M^2}}}{{\alpha \beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}} \right){\left\| {{y^{k + 1}} - {y^k}} \right\|^2} \geqslant \\ &\left( {\frac{{\beta \mu - {L_3} - \omega }}{2} - \frac{{2{M^2}}}{{\alpha \beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}} \right){\left\| {{x^{k + 1}} - {x^k}} \right\|^2}+\\ &\left( \frac{{\beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}} - {L_2} - {L_1}}}{2}+\frac{{\left( {1 - \alpha } \right)\beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}{\alpha } -\right. \\ &\left.\frac{{2L_1^2 + 2{M^2}}}{{\alpha \beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}} \right){\left\| {{y^{k + 1}} - {y^k}} \right\|^2} \geqslant \delta {\left\| {{v^{k + 1}} - {v^k}} \right\|^2} \end{aligned} $ $ \begin{aligned}\delta : =\ & \min \left\{ \frac{{\beta {\mu _1} - {L_3} - \omega }}{2} - \frac{{2{M^2}}}{{\alpha \beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}},\frac{{\beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}} - {L_2} - {L_1}}}{2}+\right.\\ &\left.\frac{{\left( {1 - \alpha } \right)\beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}{\alpha } - \frac{{2L_1^2 + 2{M^2}}}{{\alpha \beta {\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}} \right\}\end{aligned} $ $ \sum\limits_{k = 0}^{ + \infty } {{{\left\| {{\omega ^{k + 1}} - {\omega ^k}} \right\|}^2} < + \infty } $ $ {\left\{ {{L_\beta }\left( {{\omega ^{{k_j}}}} \right)} \right\}_{j \in N}} $ 有下界，又由式（17）可知 $ {\left\{ {{L_\beta }\left( {{\omega ^{{k_j}}}} \right)} \right\}_{j \in N}} $ 单调递减，所以， $ {\left\{ {{L_\beta }\left( {{\omega ^{{k_j}}}} \right)} \right\}_{j \in N}} $ 收敛，此外， $ {\left\{ {{L_\beta }\left( {{\omega ^k}} \right)} \right\}_{j \in N}} $ 也是收敛的，且 $ {L_\beta }\left( {{\omega ^k}} \right) \geqslant {L_\beta }\left( {{\omega ^*}} \right) $ 。由引理4可得

$\begin{aligned} \sum\limits_{k = 0}^m \delta {{\left\| {{v^{k + 1}} - {v^k}} \right\|}^2} \leqslant & \ {L_\beta }\left( {{\omega ^0}} \right) - {L_\beta }\left( {{\omega ^{m + 1}}} \right) \leqslant \\ &{L_\beta }\left( {{\omega ^0}} \right) - {L_\beta }\left( {{\omega ^*}} \right) < + \infty \end{aligned} $ $ \sum\limits_{k = 0}^{ + \infty } {{{\left\| {{x^{k + 1}} - {x^k}} \right\|}^2}} < + \infty ,\ \ \sum\limits_{k = 0}^{ + \infty } {{{\left\| {{y^{k + 1}} - {y^k}} \right\|}^2}} < + \infty $ $ d\left( {0,\partial {L_\beta }\left( {{\omega ^{k + 1}}} \right)} \right) \leqslant \xi \left\| {{v^{k + 1}} - {v^k}} \right\| $ $ \left\{ \begin{array}{l} {\partial _x}{L_\beta }\left( {{\omega ^{k + 1}}} \right) = \partial f\left( {{x^{k + 1}}} \right) + {\nabla _x}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) - \\ \qquad\qquad\qquad{{\boldsymbol{A}}^{\rm T}}{\gamma ^{k + 1}} + \beta {{\boldsymbol{A}}^{\rm T}}\left( {{\boldsymbol{A}}{x^{k + 1}} + {\boldsymbol{B}}{y^{k + 1}}} \right) \\ {\partial _y}{L_\beta }\left( {{\omega ^{k + 1}}} \right) = \nabla h\left( {{y^{k + 1}}} \right) + {\nabla _y}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) -\\ \qquad\qquad\qquad{{\boldsymbol{B}}^{\rm T}}{\gamma ^{k + 1}} + \beta {{\boldsymbol{B}}^{\rm T}}\left( {{\boldsymbol{A}}{x^{k + 1}} + {\boldsymbol{B}}{y^{k + 1}}} \right) \\ {\partial _\gamma }{L_\beta }\left( {{\omega ^{k + 1}}} \right) = - \left( {{\boldsymbol{A}}{x^{k + 1}} + {\boldsymbol{B}}{y^{k + 1}}} \right) \end{array} \right. $ $ \left\{ \begin{array}{l} {{\boldsymbol{A}}^{\rm T}}\left( {{\gamma ^k} - {\gamma ^{k + 1}}} \right) + \beta {{\boldsymbol{A}}^{\rm T}}{\boldsymbol{B}}\left( {{y^{k + 1}} - {y^k}} \right) +\\ \quad {\nabla _x}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) - {\nabla _x}g\left( {{x^{k + 1}},{y^k}} \right) \in {\partial _x}{L_\beta }\left( {{\omega ^{k + 1}}} \right) \\ \dfrac{1}{\alpha }{{\boldsymbol{B}}^{\rm T}}\left( {{\gamma ^k} - {\gamma ^{k + 1}}} \right) + \dfrac{{\left( {1 - \alpha } \right)\beta }}{\alpha }{\boldsymbol{B}}\left( {{y^k} - {y^{k + 1}}} \right) \in\\ \quad{\partial _y}{L_\beta }\left( {{\omega ^{k + 1}}} \right)\\ \dfrac{1}{{\alpha \beta }}\left( {{\gamma ^{k + 1}} - {\gamma ^k}} \right) + \dfrac{{1 - \alpha }}{\alpha }{\boldsymbol{B}}\left( {{y^{k + 1}} - {y^k}} \right) \in\\ \quad {\partial _\gamma }{L_\beta }\left( {{\omega ^{k + 1}}} \right) \end{array} \right. $ $ \begin{aligned} &x_{k + 1}^* = {{\boldsymbol{A}}^{\rm T}}\left( {{\gamma ^k} - {\gamma ^{k + 1}}} \right) + \beta {{\boldsymbol{A}}^{\rm T}}{\boldsymbol{B}}\left( {{y^{k + 1}} - {y^k}} \right) +\\ &\qquad\quad{\nabla _x}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) - {\nabla _x}g\left( {{x^{k + 1}},{y^k}} \right)\\ &y_{k + 1}^* = \frac{1}{\alpha }{{\boldsymbol{B}}^{\rm T}}\left( {{\gamma ^k} - {\gamma ^{k + 1}}} \right) + \frac{{\left( {1 - \alpha } \right)\beta }}{\alpha }{\boldsymbol{B}}\left( {{y^k} - {y^{k + 1}}} \right)\\ &\gamma _{k + 1}^* = \frac{1}{{\alpha \beta }}\left( {{\gamma ^{k + 1}} - {\gamma ^k}} \right) + \frac{{1 - \alpha }}{\alpha }{\boldsymbol{B}}\left( {{y^{k + 1}} - {y^k}} \right) \end{aligned} $ $\left( {x_{k + 1}^*,y_{k + 1}^*,\gamma _{k + 1}^*} \right) \in {L_\beta }\left( {{\omega ^{k + 1}}} \right)$ ${\xi _1},{\xi _2},{\xi _3} > 0$ $ \begin{split} &\left\| {x_{k + 1}^*,y_{k + 1}^*,\gamma _{k + 1}^*} \right\| \leqslant {\xi _1}\left\| {{y^{k + 1}} - {y^k}} \right\| + {\xi _2}\left\| {{\gamma ^{k + 1}} - {\gamma ^k}} \right\| + \\ &\qquad {\xi _3}\left\| {{\nabla _x}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) - {\nabla _x}g\left( {{x^{k + 1}},{y^k}} \right)} \right\| \\[-12pt] \end{split} $ $ \left\| {{\nabla _x}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) - {\nabla _x}g\left( {{x^{k + 1}},{y^k}} \right)} \right\| \leqslant M\left\| {{y^{k + 1}} - {y^k}} \right\| $ $ \begin{split}\left\| {{\gamma ^{k + 1}} - {\gamma ^k}} \right\| \leqslant& \frac{1}{{\sqrt {{\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}} }}\left( \sqrt {\left( {2L_1^2 + 2{M^2}} \right)} \left\| {{y^{k + 1}} - {y^k}} \right\| + \right.\\ &\left.\sqrt 2 M\left\| {{x^{k + 1}} - {x^k}} \right\| \right)\\[-12pt]\end{split} $ $ \begin{aligned} &d\left( {0,\partial {L_\beta }\left( {{\omega ^{k + 1}}} \right)} \right) \leqslant \left\| {x_{k + 1}^*,y_{k + 1}^*,\gamma _{k + 1}^*} \right\| \leqslant\\ &\qquad\left( {{\xi _1} + \sqrt {\frac{{\left( {2L_1^2 + 2{M^2}} \right)}}{{{\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}} + M{\xi _3}} \right)\left\| {{y^{k + 1}} - {y^k}} \right\| + \\ &\qquad \sqrt {\frac{2}{{{\lambda _{{{\boldsymbol{B}}^{\rm T}}{\boldsymbol{B}}}}}}} M\left\| {{x^{k + 1}} - {x^k}} \right\| \end{aligned} $ $ \begin{split} &\mathop {\lim \sup }\limits_{j \to + \infty } {L_\beta }\left( {{x^{{k_j} + 1}},{y^{{k_j}}},{\gamma ^{{k_j}}}} \right) = \\ &\quad \mathop {\lim \sup }\limits_{j \to + \infty } {L_\beta }\left( {{x^{{k_j} + 1}},{y^{{k_j} + 1}},{\gamma ^{{k_j} + 1}}} \right) \leqslant{L_\beta }\left( {{x^*},{y^*},{\gamma ^*}} \right)\end{split} $

由引理6可得， $\displaystyle\sum\limits_{k = 0}^{ + \infty } {{{\left\| {{\omega ^{k + 1}} - {\omega ^k}} \right\|}^2} < + \infty } $ $ \mathop {\lim \;\inf }\limits_{j \to + \infty } {L_\beta }\left( {{x^{{k_j} + 1}},{y^{{k_j} + 1}},{\gamma ^{{k_j} + 1}}} \right) \geqslant {L_\beta }\left( {{x^*},{y^*},{\gamma ^*}} \right) $ $ \mathop {\lim }\limits_{j \to + \infty } f\left( {{x^{{k_j} + 1}}} \right) = f\left( {{x^*}} \right) $ $\nabla h,\nabla {g_x}\left( { \cdot , \cdot } \right),\nabla {g_y}\left( { \cdot , \cdot } \right)$ 的连续性，在式（6）中对序列 $\left\{ {\left( {{x^{{k_j}}},{y^{{k_j}}},{\gamma ^{{k_j}}}} \right)} \right\}$ 取极限，可得

$ \left\{ \begin{array}{l} {{\boldsymbol{A}}^{\rm T}}{\gamma ^k} - {\nabla _x}g\left( {{x^{k + 1}},{y^k}} \right) \in \partial f\left( {{x^*}} \right) \\ {{\boldsymbol{B}}^{\rm T}}{\gamma ^{k + 1}} - {\nabla _y}g\left( {{x^{k + 1}},{y^{k + 1}}} \right) = \nabla h\left( {{y^*}} \right) \\ {\boldsymbol{A}}{x^*} + {\boldsymbol{B}}{y^*} = 0 \end{array} \right. $ ${L_\beta }\left( {{\omega ^{{k_0}}}} \right) = {L_\beta }\left( {{\omega ^*}} \right)$ ，由引理5可知，

$ \delta {\left\| {{v^{k + 1}} - {v^k}} \right\|^2} \leqslant \xi \left\| {{v^k} - {v^{k - 1}}} \right\| {\varDelta _{k,k + 1}} $ ${L_\beta }\left( {{\omega ^k}} \right) \to {L_\beta }\left( {{\omega ^*}} \right)$ ，那么，对于任意的 $\varepsilon ,\kappa > 0$ $\tilde k > 0$ 使得对于所有的 $k > \tilde k$ $\begin{aligned} {L_\beta }\left( {{\omega ^k}} \right) -\ & {L_\beta }\left( {{\omega ^{k + 1}}} \right)=\left( {{L_\beta }\left( {{\omega ^k}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right) -\\ &\left( {{L_\beta }\left( {{\omega ^{k + 1}}} \right) - {L_\beta }\left( {{\omega ^*}} \right)} \right) \end{aligned}$ $\begin{aligned} & \varphi \left( {{L_\beta }\left( {{\omega ^k}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right) - \varphi \left( {{L_\beta }\left( {{\omega ^{k + 1}}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right) \geqslant \\ &\qquad \varphi '\left( {{L_\beta }\left( {{\omega ^k}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right)\left( {{L_\beta }\left( {{\omega ^k}} \right) - {L_\beta }\left( {{\omega ^{k + 1}}} \right)} \right)\end{aligned}$

因此，结合引理7以及 $ \varphi '\left( {{L_\beta }\left( {{\omega ^k}} \right) \ - \ {L_\beta }\left( {{\omega ^{*}}} \right)} \right) \ > \ 0 $ $ \begin{aligned} &{L_\beta }\left( {{\omega ^k}} \right) - {L_\beta }\left( {{\omega ^{k + 1}}} \right) \leqslant \\ &\qquad\frac{{\varphi \left( {{L_\beta }\left( {{\omega ^k}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right) - \varphi \left( {{L_\beta }\left( {{\omega ^{k + 1}}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right)}}{{\varphi '\left( {{L_\beta }\left( {{\omega ^k}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right)}} \leqslant \\ & \qquad\xi \left\| {{v^k} - {v^{k - 1}}} \right\|\left\| \varphi \left( {{L_\beta }\left( {{\omega ^k}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right) -\right.\\ &\qquad\left.\varphi \left( {{L_\beta }\left( {{\omega ^{k + 1}}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right) \right\| = \xi \left\| {{v^k} - {v^{k - 1}}} \right\|{\varDelta _{k.k + 1}} \end{aligned} $

利用不等式 $2\sqrt {\alpha \beta } \leqslant \alpha + \beta \left( {\forall \alpha ,\beta > 0} \right)$ $ \varphi \left( {{L_\beta }\left( {{\omega ^{m + 1}}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right) > 0 $ $m \to + \infty $ ，上式变为

$ \begin{split}&\sum\limits_{k = \tilde k + 1}^{ + \infty } \left\| {{v^{k + 1}} - {v^k}} \right\| \leqslant \left\| {{v^k} - {v^{k - 1}}} \right\| +\\ &\qquad \frac{\xi }{\delta }\varphi \left( {{L_\beta }\left( {{\omega ^{\tilde k + 1}}} \right) - {L_\beta }\left( {{\omega ^{*}}} \right)} \right) \end{split} $ $\displaystyle\sum\limits_{k = 0}^{ + \infty } {\left\| {{v^{k + 1}} - {v^k}} \right\|} < + \infty $ $ \begin{aligned} &\left\| {{\omega ^{k + 1}} - {\omega ^k}} \right\| = \left( {{\left\| {{x^{k + 1}} - {x^k}} \right\|}^2} + {{\left\| {{y^{k + 1}} - {y^k}} \right\|}^2} +\right.\\ &\qquad\quad\left.{{\left\| {{\gamma ^{k + 1}} - {\gamma ^k}} \right\|}^2} \right)^{\frac{1}{2}} \leqslant \left\| {{x^{k + 1}} - {x^k}} \right\| +\\ &\qquad\quad\left\| {{y^{k + 1}} - {y^k}} \right\| + \left\| {{\gamma ^{k + 1}} - {\gamma ^k}} \right\| \end{aligned} $ BOLEY D. Local linear convergence of the alternating direction method of multipliers on quadratic or linear programs[J]. SIAM Journal on Optimization, 2013, 23(4): 2183-2207. DOI:10.1137/120878951 HAN D R, YUAN X M. Local linear convergence of the alternating direction method of multipliers for quadratic programs[J]. SIAM Journal on Numerical Analysis, 2013, 51(6): 3446-3457. DOI:10.1137/120886753 HE B S, YUAN X M. On the O (1/ n ) convergence rate of the Douglas-Rachford alternating direction method [J]. SIAM Journal on Numerical Analysis, 2012, 50(2): 700-709. DOI:10.1137/110836936 GUO K, HAN D R, WANG D Z W, et al. Convergence of ADMM for multi-block nonconvex separable optimization models[J]. Frontiers of Mathematics in China, 2017, 12(5): 1139-1162. DOI:10.1007/s11464-017-0631-6 LI G Y, PONG T K. Global convergence of splitting methods for nonconvex composite optimization[J]. SIAM Journal on Optimization, 2015, 25(4): 2434-2460. DOI:10.1137/140998135 GUO K, HAN D R, WU T T. Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints[J]. International Journal of Computer Mathematics, 2017, 94(8): 1653-1669. DOI:10.1080/00207160.2016.1227432 蒋峰, 党亚峥. 求解凸优化问题的改进对称交替方向乘子法[J]. 上海理工大学学报, 2020, 42(3): 269-274. 王欣, 郭科. 一类非凸优化问题广义交替方向法的收敛性[J]. 应用数学和力学, 2018, 39(12): 1410-1425. GAO X, ZHANG S Z. First-order algorithms for convex optimization with nonseparable objective and coupled constraints[J]. Journal of the Operations Research Society of China, 2017, 5(2): 131-159. DOI:10.1007/s40305-016-0131-5 CHEN C H, LI M, LIU X, et al. Extended ADMM and BCD for nonseparable convex minimization models with quadratic coupling terms: convergence analysis and insights[J]. Mathematical Programming, 2019, 173(1/2): 37-77. LIU Q H, SHEN X Y, GU Y T. Linearized ADMM for nonconvex nonsmooth optimization with convergence analysis[J]. IEEE Access, 2019, 7: 76131-76144. DOI:10.1109/ACCESS.2019.2914461 WANG F H, XU Z B, XU H K. Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems[J]. Eprint Arxiv, 2014, 2014: 1−17. GUO K, WANG X. Convergence of generalized alternating direction method of multipliers for nonseparable nonconvex objective with linear constraints[J]. Journal of Mathematical Research with Applications, 2018, 38(5): 523-540.