-
A Model-Free Stealthy Attack for Cyber-Physical Systems Based on Deep Reinforcement Learning
Qirui ZhangSiqi MengWei DaiZhenxing XiaChunyu YangXuesong Wang
Keywords:Deep reinforcement learningDetectorsNonlinear systemsConvergenceTrajectorySystem dynamicsEntropyTechnological innovationOptimizationLyapunov methodsDeep LearningDeep Reinforcement LearningCyber-physical SystemsStealthy AttacksSystem DynamicsValue FunctionSystem StateLinear SystemNonlinear SystemsLyapunov FunctionMarkov Decision ProcessReinforcement Learning AlgorithmTheoretical GuaranteesAction-value FunctionDeep Neural NetworkHidden LayerControl PerformanceLagrange MultiplierKullback-LeiblerOptimal PolicyAttack PerformanceModel-based AlgorithmDetection PrincipleReplay BufferPolicy ImprovementLinear Quadratic GaussianTraining EpisodesPolicy EvaluationCritic NetworkAttack DetectionCyber-physical systems (CPSs)deep reinforcement learningmodel-freestealthy attack
Abstracts:This article, from the attacker’s standpoint, develops a model-free stealthy attack that can steer the system state to the predefined target value and evade detection, without prior knowledge of the system dynamics. A constrained Markov decision process (CMDP) is first modeled to characterize the objective of the stealthy attack. On the basis of the established CMDP, an actor-critic reinforcement learning algorithm is proposed to train the attacker’s policy. Furthermore, by introducing a Lyapunov function constructed from the action value function to the algorithm, convergence of the attacked system’s state to the target is theoretically guaranteed. Differing from existing model-free stealthy attacks which are only suitable for linear systems, the proposed approach guarantees the applicability to nonlinear systems. A linear numerical example and a nonlinear example of flotation industrial system are provided to validate the effectiveness of our proposed stealthy attack.
-
Obstacle Avoidance and Safe Coverage of Moving Domains for Multiagent Systems via Adaptive Control Barrier Function
Shuxuan WuLijun Long
Keywords:Multi-agent systemsSafetySwitchesLogicCollision avoidanceUncertain systemsTransient analysisUncertaintyAdaptive controlVectorsAdaptive ControlMulti-agent SystemsObstacle AvoidanceControl Barrier FunctionsParameter UncertaintyModel IdentificationDistributed ControlQuadratic ProgrammingSwitching MechanismTransient PerformanceCoverage ProblemNominal ControlDistributed Control StrategySwitching LogicParameter EstimatesUnknown ParametersAdaptive AlgorithmPositive ConstantSafety SystemsCompact SetSafety ControlVelocity Of AgentSafety ConstraintsPosition Of AgentN-dimensional SpaceSafe SetIth AgentVoronoi DiagramEquilibrium RelationshipNeighboring AgentsAdaptive controllogic switchingmultiagent systemsparameter uncertaintiessafe coverage
Abstracts:This article investigates the problem of safe coverage for multiagent systems with parameter uncertainties. A novel distributed control strategy is proposed to simultaneously guarantee safety and coverage for multiagent systems based on adaptive artificial potential function (AAPF) and adaptive control barrier function (ACBF). In particular, a logic switching mechanism based on multiple identification models is integrated into a coverage controller while the transient performance of multiagent systems is effectively improved. Also, a framework for safe coverage in multiagent systems is presented in the context of parametric uncertainties. In this framework, a nominal controller is obtained by using the AAPF method. Furthermore, the nominal controller undergoes modification through the application of ACBF based on quadratic programs to achieve safe coverage. Finally, simulation results are presented to demonstrate the effectiveness of the proposed control strategy.
-
Output Regulation Based on Zero-Sum Game for Discrete-Time System Driven by Exogenous Signal
Ruizhuo SongGaofu YangFrank L. Lewis
Keywords:RegulationQ-learningNoiseOptimal controlRiccati equationsHeuristic algorithmsGame theoryOutput feedbackGamesAsymptotic stabilityZero-sumExogenous SignalsOutput RegulationValue FunctionSystem StateStability Of SystemProblem Of SystemsControl InputRiccati EquationInput Output DataQ-learning AlgorithmGrid-connected InverterLeast-squaresSystem DynamicsMeasurement DataOptimal ControlLess Than Or EqualFeedback ControlSystem OutputEigenvalues Of MatrixUnknown StateOptimal Control LawNull SpaceAugmented StateOutput Feedback ControlPositive Definite MatrixAugmented SystemSignal TracksPractical ScenariosUnit CircleAdaptive dynamic programmingdiscrete-time systemsgame theoryoutput feedbackreinforcement learning
Abstracts:This article proposes a novel Q-learning algorithm that relies solely on input-output data to address the output regulation control problem of complex discrete-time systems affected by exogenous signals. Unlike traditional methods, this algorithm does not require detailed system information, state knowledge, or data about external systems or exogenous signals. Additionally, the control strategy does not depend on state information, but on input-output data processed by a set of filters. We provide upper and lower bounds on the discount factor, eliminating the need to solve the Riccati equation. These bounds ensure that the value function remains finite, and we prove the stability of the system when using control inputs derived from the value function with the given discount factor. Furthermore, the Q-learning algorithm, when applied with input data containing probing noise, is shown to yield Q-function estimates that are independent of the probing noise. Finally, a simulation involving a grid-connected inverter is presented, demonstrating the effectiveness of the proposed algorithm in a practical setting.
-
Optimal Sequential-Parallel Test Strategy Generation Method for Complex Systems
Jingyuan WangZhen LiuJiahong WangMin WangBorui GuYuhua Cheng
Keywords:Flow graphsSwitchesHeuristic algorithmsTestingEstimationNeural networksInformation entropyCyberneticsCostsClassification algorithmsOptimal TestNeuroanatomicalCybersecurityTest SequencesHeuristic AlgorithmTest DesignNeural AlgorithmParallel TestNeural NetworkPopulation SizeComputation TimeEstimated ValuesMutation RateProbability Density FunctionParallelizationInformation GainFuzzy SetProportion TestIntelligence AlgorithmsDiscrete MatrixMultiple DistributionsInformation EntropySimple DistributionOriginal GraphComplex systemmultisignal flow graph modelprobability heuristic functionsequential-parallel test strategy (SPTS)testability
Abstracts:One of the core tasks of design for testability (DFT) is to generate an optimal test strategy based on the test mode, to isolate faults quickly and accurately. There are currently two modes: 1) sequential test mode (STM) and 2) parallel test mode (PTM). For complex systems, limited testing resources are difficult to meet parallel test conditions, so STM is mostly used. The multisignal flow graph is a widely used model for generating optimal sequential test strategy (STS) in DFT. However, this STM-based model overlooks the possibility of conducting some tests in parallel, resulting in lengthy test time and greatly affecting the reliability and security of the systems. To solve this problem, an optimal sequential-parallel test strategy (SPTS) generation method is proposed. First, a new test mode of global sequential testing and local parallel testing is proposed to generalize the original model. Second, to overcome the combinatorial explosion caused by the new model, we approximate the discrete model to continuous and derive a probability heuristic function. Then, a neural network-intelligent algorithm structure is established to simplify the complex recursion of the heuristic function. Finally, this heuristic function is used to guide the generation of SPTS, which has a shorter test time than STS. Simulation results show that the reduction in time is related to the type and number of locally parallel tests, and reaches 39.5% in a real case.
-
Event-Triggered Robust Adaptive Fault-Tolerant Tracking and Vibration Control for the Rigid-Flexible Coupled Robotic Mechanisms With Large Beam-Deformations
Xingyu ZhouHaoping WangKe WuYang TianGang Zheng
Keywords:DeformationVibrationsActuatorsDeformable modelsEvent detectionAdaptation modelsStrainRobotsFault tolerant systemsFault toleranceAdaptive ControlTracking ControlRobust Adaptive ControlDynamicalHorizontal PlaneRobust ControlTracking ErrorCompact SetAngular PositionAdaptive LawVirtual WorkFault-tolerant ControlVirtual InputAdaptive Control ApproachHorizontal AxisVibrational ModesLarge DeformationLyapunov FunctionExternal DisturbancesFlexible LinkerActuator FaultsSmall DeformationFlexible ModesEvent-triggered MechanismVibration ReductionVirtual ForceDisturbance ObserverVibration SuppressionActuator FailuresFlexible BeamActuator faultsdisturbance observerevent-triggered controllarge deformationsrigid-flexible coupled robotic mechanisms (RFCRMs)vibration suppression
Abstracts:A detailed modeling approach that utilizes the virtual work idea is developed for modeling the dynamical formulas of the rigid-flexible coupled robotic mechanisms (RFCRMs) with large beam-deformations across the horizontal plane. To follow the required angular positions of RFCRMs, a virtual robust linear quadratic state feedback (RLQSF) input is constructed using the converted full-actuated model in conjunction with an event-triggered robust adaptive fault-tolerant control (ETRAFTC) approach. The integration of virtual input and the proposed RLQSF law design enables simultaneous angular tracking and vibration elimination. To make up for the defective actuators with part loss of efficacy and evaluate the unknown fault parameters, an adaptive estimation law with a projection mapping operator is adopted. With the help of the Lyapunov direct approach, the angular position tracking errors and the flexible vibration of RFCRMs are demonstrated to converge to a tiny confined compact set with fewer communications. At last, the performance of the designed ETRAFTC is presented via three numerical scenarios.
-
Filter-Based Intelligent Output-Constrained Control of Uncertain MIMO Nonlinear Systems With Sensor and Actuator Faults
Ning ZhouCanyang ZhaoXiaodong ChengYuanqing Xia
Keywords:ActuatorsAdaptive systemsFault tolerant systemsFault toleranceRobot sensing systemsBacksteppingUncertaintyNonlinear dynamical systemsUpper boundElectronic mailNonlinear SystemsUncertain SystemsIntelligent ControlUncertain Nonlinear SystemsActuator FaultsSensor FaultsMIMO Nonlinear SystemsNeural NetworkNew VariablesRadial Basis FunctionTracking ErrorCoordinate TransformationIterative DesignTracking ProblemHigher-order DerivativesNonlinear FunctionControl DesignAdaptive ControlPositive ParameterSimulation ExampleBackstepping DesignAdaptive Fuzzy ControlBarrier Lyapunov FunctionFault-tolerant ControlAdaptive CompensationFuzzy ControlDynamic Surface ControlTracking ControlLyapunov CandidateMultiple DefectsAdaptation mechanismcoordinate transformationfilter-based adaptationintelligent constraint controlmultiple sensor and actuator faults
Abstracts:This article studies the tracking problem for a class of strict-feedback uncertain multi-input-multi-output (MIMO) nonlinear systems, considering both the output constraints and multiple sensor/actuator faults. A novel control approach, named adaptive-neural-backstepping fault-tolerant constrained (ANBFTC) algorithm, is proposed, which incorporates the dynamic surface analysis into the iterative design. A filter-based adaptation coordinate transformation (FBACT) is introduced to define new backstepping iteration variables, eliminating the need for fault amplitudes and bias information. To further address the nonlinear uncertainties inherent in the system, we employ a learning approach, specifically utilizing radial basis function neural networks (RBFNNs), to approximate the uncertainty dynamics. This methodology not only mitigates the computational challenges typically associated with high-order derivatives in iterative designs but also ensures the convergence of tracking errors while adhering to output constraints, even in the presence of multiple sensor/actuator faults. Finally, numerical simulation results are presented to demonstrate the feasibility of the ANBFTC approach.
-
Fixed-Time Event-Triggered Bipartite Consensus of Multiagent Systems Under Time-Varying Disconnected Topologies
Xiaoyang LiuHaibin HeZhuyan JiangLu FanWenwu Yu
Keywords:TopologyEvent detectionNetwork topologySensorsPartitioning algorithmsMulti-agent systemsControl systemsProtocolsCyberneticsTime-varying systemsMulti-agent SystemsConsensus Of Multi-agent SystemsBipartite ConsensusEnergy FunctionConnection TopologyZeno BehaviorUpper BoundControl InputNonlinear DynamicsExternal DisturbancesNonlinear TermsImpulse ControlDirac DeltaSettling TimeLaplacian MatrixTrajectories Of SystemMinimum IntervalType Of PositionTriggering ConditionAverage Dwell TimeEvent-triggered MechanismBipartite consensusdisconnected topologyfixed-time event-triggered controlmultiagent systems
Abstracts:This article focuses on the fixed-time bipartite consensus problems of multiagent systems under time-varying disconnected topologies. Different from the jointly connected topology, an enhanced fixed-time local pinning algorithm is proposed to overcome challenges posed by disconnected signed networks without introducing additional topological assumptions. Especially, the motion tendencies of isolated agents in cooperative-competitive networks are thoroughly discussed. Event-triggered control with the impulsive effect is utilized to achieve the bipartite consensus of MASs with minimal energy consumption, where the total energy function does not need to be monotonic, and the Zeno behavior can be avoided. Finally, the efficacy of the designed protocol is demonstrated through two numerical examples.
-
Competition and Cooperation of Multiagent System for Moving Target Defense With Dynamic Task-Switching
Haiyan ZhaoRongxin CuiWeisheng YanLepeng Chen
Keywords:Decision makingConvergenceProtocolsResource managementPlanningMulti-agent systemsCostsRobustnessDynamicsCollaborationMulti-agent SystemsTarget DefenseMoving Target DefenseClosed-loop SystemCooperative BehaviorCooperative StrategySecond-order DynamicsAllianceCost FunctionAutonomic SystemOptimal ControlControl ParametersControl InputPositive MatrixTransformation MatrixFinite TimeConvex HullQuadratic ProgrammingTask SwitchingCompetitive BehaviorExponential ConvergenceWireless Local Area NetworkAuxiliary VectorPlanning CoordinationCoordination ControlSingle Point Of FailureCompetition and cooperationmoving target defensemultiagent systemtask-switchingwheeled mobile robot (WMR)
Abstracts:In this study, we present a coordinated protocol for a multiagent system (MAS) in competitive and cooperative manners for moving target defense with dynamic task-switching. The protocol comprises three components. First, we design a distance-based competitive distributed decision algorithm within an improved k-Winner-Take-All (k-WTA) framework. This algorithm generates dynamic binary task-driven signals for each agent, enabling near-optimal online grouping of MAS with arbitrary proportions. Second, we introduce a cooperative strategy that employs a shared decision-making mechanism and utilizes feedback linearization without global position information. This strategy generates motion planning signals to coordinate the agents’ actions, achieving overall cooperative behaviors such as tracking, capturing, and intercepting. Finally, we incorporate an adaptive sliding mode technique based on second-order nonlinear dynamics to enhance robustness against disturbances, ensuring uniformly ultimately boundedness (UUB) of the closed-loop system. In addition, simulations and experiments with wheeled mobile robots (WMRs) validate the effectiveness of our method.
-
Finite-Time Synchronization of Complex Dynamic Networks via Pinning Hybrid Control With Stochastic Disturbances
Bo ZhangLiang ChenShengli XieFeiqi Deng
Keywords:SynchronizationStochastic processesStability criteriaVectorsNumerical stabilityFeedback controlSwitchesCouplingsCostsAdaptive controlSynchronization Of NetworksComplex Dynamical NetworksFinite-time SynchronizationNumerical SimulationsFeedback ControlImpulse ControlSettling TimePresence Of DisturbancesFinite-time ControlFinite-time StabilityStochastic TheoryStochastic StabilityShorter TimeComplex NetworkConvergence RateSystematic ErrorsControl InputGain ControlFinite TimeComplex ScenariosConvergence TimeJensen’s InequalityTime-varying DelaysLyapunov FunctionFeedback StrengthExponential StabilityCost ControlEquilibrium Of SystemDirac DeltaNonnegative ConstantsComplex dynamic networks (CDNs)finite-time stability theoryfinite-time synchronization (FTS)pinning hybrid controlstochastic disturbances
Abstracts:This article presents a novel theoretical framework for the finite-time synchronization (FTS) of complex dynamic networks (CDNs) under stochastic disturbances. Few studies have explored the combination of pinning impulsive control and pinning finite-time feedback control, with most finite-time feedback controls being designed globally rather than locally. Our approach integrates both pinning impulsive and pinning finite-time feedback strategies to achieve FTS of CDNs. We introduce a new impulse-type stochastic finite-time stability theory to demonstrate FTS in the presence of disturbances. Additionally, we propose criteria to ensure FTS and provide an explicit expression for the settling time, which is shown to be shorter than those in previous works. A numerical simulation is presented to validate the proposed methodology.
-
State Estimation of Stochastic Boolean Networks Based on Event-Triggered Sampling
Zibo WeiYong DingYuqian GuoWeihua Gui
Keywords:VectorsState estimationEvent detectionEstimation errorUncertaintyStochastic processesNoiseLogic functionsIterative methodsEscherichia coliEvent-triggered SamplingStochastic Boolean NetworksEscherichia ColiError Of The MeanEstimation ErrorSampling RateGene Regulatory NetworksOptimal StateOptimal EstimationOutput SamplesLac OperonTrigger ThresholdWorst-case EstimateWorst-case ErrorOne-step PredictionOptimization ProblemAverage ErrorMeasurement NoiseControl SequenceOutput MeasurementsAverage Estimation ErrorObservational DatasetsProcess NoiseEvent-triggered MechanismBoolean LogicState Estimation ProblemEvent-triggered SchemeChemical TestsOutput SequenceOptimal Value Of ProblemEvent-triggered samplingsemi-tensor product (STP)state estimationstochastic Boolean networks (SBNs)
Abstracts:A stochastic Boolean network (SBN) emerges as a more realistic model for gene regulatory networks than a deterministic Boolean network (BN). In order to reduce output sampling while ensuring a given estimation accuracy, this article proposes an event-triggered sampling strategy for the state estimation of SBNs. Under this strategy, the output is sampled when the one-step prediction mean error exceeds a prespecified threshold. An iterative algorithm for the state probability distribution is proposed based on the algebraic form of SBNs, which determines the optimal state estimation. A matrix inequality method is proposed to calculate the worst-case mean estimation error based on its monotonicity with time. Then, the range of sampling triggering thresholds that minimize the worst-case mean estimation error is obtained. This article demonstrates that the event-triggered sampling strategy can make a tradeoff between estimation error and sampling rate. It explains that the full sampling estimator is a special event-triggered sampling estimator. Finally, the proposed method is applied to BN models of the lac operon in Escherichia coli to analyze the relationship among the sampling triggering threshold, the sampling rate, and the estimation error.