Cooperative Anti-Jamming and Interference Mitigation for UAV Networks: A Local Altruistic Game Approach

Yueyue Su ,Nan Qi,* ,Zanqi Huang ,Rugui Yao ,Luliang Jia

1 College of Electronic Information Engineering,Nanjing University of Aeronautics and Astronautics,Nanjing 211106,China

2 National Mobile Communications Research Laboratory,Southeast University,Nanjing 210096,China

3 School of Electronics and Information,Northwestern Polytechnical University,Xi’an 710072,China

4 School of Space Information,Space Engineering University,Beijing 101416,China

Abstract: To improve the anti-jamming and interference mitigation ability of the UAV-aided communication systems,this paper investigates the channel selection optimization problem in face of both internal mutual interference and external malicious jamming.A cooperative anti-jamming and interference mitigation method based on local altruistic is proposed to optimize UAVs’ channel selection.Specifically,a Stackelberg game is modeled to formulate the confrontation relationship between UAVs and the jammer.A local altruistic game is modeled with each UAV considering the utilities of both itself and other UAVs.A distributed cooperative anti-jamming and interference mitigation algorithm is proposed to obtain the Stackelberg equilibrium.Finally,the convergence of the proposed algorithm and the impact of the transmission power on the system loss value are analyzed,and the anti-jamming performance of the proposed algorithm can be improved by around 64% compared with the existing algorithms.

Keywords: channel selection;cooperative antijamming and interference mitigation;local altruistic game;Stackelberg game;unmanned aerial vehicle(UAV)

I.INTRODUCTION

Since the low-altitude unmanned aerial vehicle(UAV)can establish short range line-of-sight(LoS)links with the ground units and is faster to deploy,it is often used to assist information dissemination and collection[1].As one of modern high-tech equipment,UAVs are widely adopted in agriculture,firefighting,and military departments [2-4] with admirable performance.However,the UAV is limited by payload during mission,poor flight capability and insufficient data processing equipment [1].With these inevitable shortcomings,a single UAV communication device is difficult to cope with the challenges of complex tasks and a harsh environment.Multiple UAVs are more capable of processing tasks than a single UAV.Therefore,it usually takes a group of UAVs to accomplish a mission.

The communication system of UAVs is vulnerable to malicious jammers.Especially when the UAVs perform tasks in clusters,there is not only external malicious jamming but also serious internal interference in the environment.Recent works related to optimizing the anti-jamming performance of the UAV communication system mainly focus on three areas:power control,spectrum allocation and path planning [5-7].Specifically,in[5],the incomplete information and the co-channel mutual interference were considered,and a Bayesian Stackelberg game was modeled to formulate the relation between UAVs and the jammer.With the advent of 6G technology [8],channel resource allocation is becoming increasingly urgent.There is no information exchange among users in[6],and a channel selection algorithm was proposed to improve the throughput and interference mitigation performance.Moreover,a path planning method of jammer evasion was proposed to guarantee communication quality in[7].But the cooperative anti-jamming and coordination capabilities among UAVs are not considered in such literature,which limits the application in multiuser systems.

With the growth of jamming attackers’capabilities,the smart jammer can sense and analyze the behavior of communication networks [9].The traditional individual anti-jamming methods are difficult to deal with intelligent complex and strong jamming.Moreover,due to the limited resources that a single UAV can be equipped with,its anti-jamming performance is often difficult to meet real needs.However,the multi-UAV intelligent cooperative anti-jamming method is superior to traditional methods [10].The difference between traditional anti-jamming methods and cooperative ones is that the former means the legitimate nodes only care about their own utilities and separately occupy spectrum resources.But the latter means resources sharing,not“selfish”,and cares about the interests of themselves and other nodes simultaneously.When the legal communication unit is subjected to malicious jamming from the jammer,it can avoid interference by adjusting its own channel selection.And when mutual interference occurs due to communication requirements,multiple UAVs can choose different communication channels to improve reliability and effectiveness.

In recent years,several pieces of literature about cooperative anti-jamming have been proposed to address the traditional shortcomings.The existing cooperative anti-jamming methods mainly include four categories: cooperative transmission[11,12],cooperative anti-jamming strategy [13,14],cooperative information interaction[15,16]and cooperative system framework[17].Cooperative transmission is one of the most common cooperative anti-jamming methods.In[11],a collaborative uncoordinated frequency hopping based(CUB)broadcast scheme was proposed to improve the communication efficiency against the smart jammer.And it was performed over universal software radio peripherals in[12].The CUB allows nodes to relay received packets to other nodes so that the communication efficiency and anti-jamming performance can be improved.Cooperative anti-jamming strategy refers to the system reducing the interference by using honeypots[13]or other ways.An idea named“No Pains No Gains” was designed to trap the jammer by sacrificing parts of users’ benefits in [14].Cooperative information interaction means interactive cooperation among users and improves the anti-jamming capability of the system.A collaborative multi-agent antijamming algorithm was proposed in [15],and users realized coordination by Q values exchange to obtain the optimal anti-jamming strategy.In[16],users interact with the neighbor node to determine the communication payoff,according to which the channel access probability is updated.The Cooperative system framework uses the different perceptions,computing abilities,and communication resources among communication nodes to achieve intelligent collaboration.In[17],cooperative cognitive radio networks can switch among three kinds of architectures autonomously and flexibly,which can deal with electromagnetic interference.It can be observed that all these cooperative antijamming works focus on the jamming from malicious jammers and co-channel users to improve the defense performance.However,users in these studies pay little attention to the local utility of the system.It is a fundamental and important problem to study the optimal channel selection under the local impact of users.

To tackle the above problems,this paper considers the scenario where the UAVs perform telemetry tasks and send information to the fusion center (FC)along different routes.UAVs are affected by both malicious intelligent interference and co-channel interference of all other UAVs.Inspired by [18],each UAV considers the payoffs of itself and all other UAVs.UAVs choose different channels to induce interference with each other.To obtain the optimal channel selection of UAVs,a distributed cooperative antijamming and interference mitigation channel selection algorithm is proposed based on stochastic learning automata (SLA).UAVs and the jammer can learn from the utility function and adjust their channel selection strategy flexibly.

To summarize,the major contributions of this paper are given as follows:

1) The confrontation relationship between UAVs and the jammer is modeled as a Stackelberg game,where the jammer is the leader and the UAVs are the followers.The cooperative anti-jamming and interference mitigation problem among UAVs is modeled as a local altruistic game.The sum payoff of itself and all other UAVs is considered by each UAV.

2)The proposed game is proven to be an exact potential game.The existence of the Stackelberg equilibrium(SE)is also proved according to the properties of the exact potential game and the limited strategic game.

3)A distributed cooperative anti-jamming and interference mitigation channel selection algorithm based on local altruistic is proposed to obtain the final SE.Simulation results show that the algorithm has a fast convergence rate and is close to the optimal Nash equilibrium (NE).The proposed method achieves better performance compared with non-cooperative ones.

The rest of this paper is organized as follows.The system model and game model are described in Section II.The proposed algorithm of cooperative antijamming and interference mitigation is developed in Section III.In Section IV,the simulation results are shown and analyzed.Finally,some conclusions are drawn in Section V.

Related notations are shown in Table 1.

Table 1.Notations.

II.SYSTEM MODEL AND PROBLEM FORMULATION

We consider that a cluster of UAVs is executing telemetry tasks and sending information to the FC,as is shown in Figure 1.To prevent collisions,the flight altitude of UAVnis set asHn.The trajectory from the beginning to the destination is divided intoZperiods,and each period has timeT.The coordinate of periodz ∈{1,2,...,Z}is defined as (xn,z,yn,z,Hn).The intelligent jammer located on(xj,yj)sends jamming signals to the FC on (x0,y0).Suppose that the locations of the UAVs and the jammer are available to each other.The location of the jammer and UAVs can be estimated with high accuracy by the localization techniques [19-21].It can sense the channel available to UAVs,and adaptively adjust its strategy to achieve the maximum jamming effect.On the other hand,UAVs are intelligent and choose channels flexibly to minimize external interference and co-channel interference received.

Figure 1.System model.

The UAVs set is denoted asN={1,2,···,N},and the available channels set is denoted asM={1,2,···,M}.The channel selection strategy of all UAVs in periodzis set asaz={a1,z,a2,z,···,aN,z},an,z ∈M,The channel selection strategy of all UAVs except UAVnis expressed asa-n,z={a1,z,···,an-1,z,an+1,z,···,aN,z}.The available channels of jammer is set asC={c1,c2,···,cJ},and the jamming channel iscj,z ∈C.The information transmission process of UAVs is shown in Figure 2.At millimeter wave frequency bands [22],the UAVs and FC can obtain the perfect channel state information(CSI)through channel estimation during the channel coherence time[23].Therefore,it is supposed that the CSI is known.Channel selection can be made quickly,and the complexity is given in Section III.The channel estimation and selection are completed in a short period defined asτ.After that,data is transmitted inT-τ.There is a period of idle time after the data transmission is complete.Note that the idle duration can vary from period to period,depending on the channel state,jamming power,etc.

Figure 2.Process of information transmission in time domain.

Due to the moving transmitter and varying channel characteristics of UAVs,the UAV-to-FC channel is different from traditional channels[24,25].Assume that the wireless channel experiences block fading [26],that is,in one period,the characteristics of the channel remain constant and change randomly in the next period.The channel model between UAVnand the FC is given by.

It is considered that the channel between the jammer and the FC follows the Rayleigh fading model.Because the jammer and the FC cannot conduct channel training,the channel models established by the two sides are different.Without loss of generality,the jammer-to-FC smallscale fading gain modeled by the jammer and the FC are set asga={ga,1,ga,2,···,ga,s,···,ga,S}andgj={gj,1,gj,2,···,gj,s,···,gj,S},respectively,wheres ∈ {1,2,···,S}is the sample index,gj,sandga,sare the small-scale fading gains following the probabilitiesPra={Pra,1,Pra,2,···,Pra,s,···,Pra,S},andPrj={Prj,1,Prj,2,···,Prj,s,···,Prj,S},The expected jammer-to-FC channel gains modeled by the jammer and the FC are given as

In periodz,the signal to interference plus noise ratio(SINR)of UAVnto the FC is expressed as

wherepnis the transmission power of the UAVn;N0is the noise power spectral density;the sum mutual interference caused by all other UAVs is denoted asand the malicious interference iswith the jamming powerpj.

To improve the network performance,effective antijamming channel selection is faced with the following challenges.That is,it is essential to reduce malicious jamming as well as mitigate co-channel interference among UAVs.Inspired by[27],we consider that less interference of the UAV is equivalent to higher throughput.

Due to the limited UAVs’ payload,the battery capacity is finite[28].Therefore,we consider minimizing the loss value under a certain flight energy cost.The loss of the UAVnin periodzis described as expected weighted sum interference (i.e.,the first term and the second term in (4)) plus flight energy consumption (i.e.,the third term in (4)) which is nonnegligible for UAVs,as follows,

whereCfis the flight energy consumption per unit distance;C0is a constant factor to balance the weighted interference and jamming with the flight cost;dn,z,z-1is the flight distance of UAVnin periodz;andf(·)is an indicator function as

Based on the above analysis,the objective of UAVs is to optimize the channel selection strategies in each period to minimize the total system loss.Therefore,the optimization problem can be described as

whereaoptis the optimal combination of strategies for all UAVs.This problem is a combinatorial optimization problem,the computational complexity of centralized exhaustive method is too high,and the convergence of heuristic method is difficult to ensure.Therefore,we model the scene as a Stackelberg game,and design an effective distributed method based on local altruistic to obtain the game equilibrium.

2.1 Stackelberg Game Formulation

Inspired by [29],a cooperative anti-jamming and interference mitigation channel selection method based on a local altruistic game is proposed to solve the above problem.The game model is represented asG={N,J,M,C,un,z,uj,z},whereNandJare the UAV set and jammer,respectively,MandCare the strategy space of UAVs and the jammer,respectively,un,zanduj,zare the corresponding utilities.Denote UAVs as followers and the jammer as the leader in the Stackelberg game.

Consider local altruistic behavior among UAVs,in which each UAV considers the utility of both itself and all other UAVs.The utility function of UAVnin periodzis defined as

whereWis a predefined positive constant to guarantee that the utility is not negative,En,zis the loss value of UAVn,is the sum loss value of all other UAVs.Therefore,the optimization problem of UAVnis as

The subgame of the leader is defined as

whereNis the UAV set,andMis the strategy set of UAVs.Similarly,the utility function and the optimization problem of the jammer are respectively

The subgame of the follower is defined as

where theJis the jammer,Cis jammer’s strategy set with a cardinality ofM.

2.2 Analysis of Stackelberg Equilibrium

Definition 1.The Stackelberg Equilibriumis defined as follows[30],

Definition 2.A game G is an exact potential game if there exists a potential function such that[31]

whereis the changed action of UAV n.

The exact potential game has two most important properties,as follows:

1)There is at least a pure strategy NE;

2)The global or local optimal solution of the potential function is a NE.

Theorem 1.For a given jammer’s strategy,the subgame of the follower is an exact potential game,and there is at least one pure strategy NE.The optimal solution to the problem of minimizing the total loss of the system is the pure strategy NE of the game.Proof.See APPENDIX.

Theorem 2.There exists a jammer’s stationary policy and a user’s NE that constitute a SE in the proposed game.

Proof.Based on Theorem 1,the subgame of the follower is an exact potential game,and there is at least a pure strategy NE.For the subgame of the leader,it is a finite strategy game.According to[32],if a game is a finite strategy game,there must be a mixed strategy equilibrium.Therefore,there exists a stationary SE in the proposed game.Inspired by[33],given the jammer’s strategy,there always existsNE(cj,z)in the proposed game.The jammer’s stationary strategy can be expressed as

According to [32],every finite strategy game has a mixed strategy equilibrium.Thereforeconstitutes a stationary equilibrium.

The game model proposed in this paper is an exact potential game and has good convergence behaviors.In other words,since the utility function changes by the same amount as the potential function,when one player adjusts its strategy in the direction of increasing the utility function,this adjustment also increases the potential function and eventually converge to the pure strategy NE of the game in a finite number of iterations.

III.DISTRIBUTED ALGORITHM

3.1 Algorithm Description

To solve the proposed game,the cooperative antijamming and interference mitigation channel selection game is extended to a mixed strategy form.UAVn’s mixed strategy in slottis defined asθn(t)=(θn1(t),···,θnm(t),···,θnM(t)),The probability that UAVnchooses channelmin slottis set asθnm(t).Similarly,the jammer’s mixed strategy in epochkis defined asθj(k)=(θj1(k),···,θjm(k),···,θjM(k)),The probability that the jammer chooses channelmin epochkis set asθjm(k).

Suppose the channel selection strategy of all UAVs in slottisa(t)={a1(t),a2(t),···,aN(t)}and the jammer’s channel iscj(t).The utility function of UAVnin slottcan be formulated as (17),

whereWis a positive constant to guaranteeun(t) is not negative.

The utility function of the jammer in epochkis defined as

Details of cooperative anti-jamming and interference mitigation algorithm based on SLA are given in Algorithm 1.The termination condition of the algorithm is to reach the maximum number of iterations or the channel selection probability satisfies the conditionθjm(k)＞Q,?m ∈MwhereQis a convergence threshold.

3.2 Analysis of Algorithm

Theorem 3.With sufficiently small step sizes b1and b2,Algorithm 1 converges to a NE in the follower subgame.

Proof.According to Theorem 1,the follower subgame is an exact potential game.Then by following Theorem 6 in [18],the SLA based learning algorithm can be proven to converge towards NE.

Remarkably,the channel selection probability is updated based on the payoff feedback.The probability of choosing the channel increases if the payoff is received positively.In addition,the smaller the value of step sizesb1andb2are,the more accurate it is.But meanwhile the calculation speed decrease.We can get the appropriate values through simulation experiments.

The analysis of the computational complexity of the proposed algorithm is provided in Table 2.

Table 2.Complexity of proposed algorithm.

1)Denote the channel training and CSI broadcasting process byC1,which is a small constant.The complexity isO(C1).

2)Secondly,UAVs sense their aggregated weighted interference simultaneously in a small sensing durationC2.The complexity of this period can be donated asO(C2).

3)Then the complexity of computing UAVs’respective payoffs can be expressed asO(C3),whereC3is also a short duration.

4) After computing payoffs,UAVnupdates its channel selection probability immediately,and the complexity isO(C4).

5) Finally,UAVnselects the channel according to the converged probability for one time slot.

To sum up,the total complexity of SLA-based learning algorithm is aboutO(C1)+O(C2)+O(C3)+O(C4)+1.

IV.SIMULATION RESULTS AND DISCUSSIONS

To verify the proposed method,simulation results are given and discussed in this section.Some basic settings are as follows.As depicted in Figure 3,the mission area is 200m×100m with six UAVs and one jammer distributed in it.UAVs keep flying at different altitudes which constitute a vector H=[100,110,120,130,140,150] m.Let UAVs take off from their respective starting points to destinations.The overall flight duration is discretized into 6 periods of the same length.The FC and the jammer are located at [100,140] m and [120,70] m on the ground,respectively.The parameters are set as Table 3.The parametersb1,b2,Cf,C0andWare set according to simulation experience.

Figure 3.Scenarios.

Table 3.Parameters.

4.1 Convergence Performance

The resulting channel selection policies of the UAVs and the jammer in each period are shown in Figure 4.It can be seen that UAVs who are close to each other,such as UAV 1 and 6,select different channels in periods 1-6,while UAVs who are far away,such as UAV 1 and 5,share the same channels in periods 1-3.This is because the closer the UAVs are,the stronger the interference is;otherwise,the weaker the interference is.On the other hand,since the jammer makes the decision after the user,to maximize the jamming effect,the jammer chooses the channel which is accessed by the most UAVs that are close to the jammer.This verifies the intelligence of the jammer.

Figure 4.Channel selection of UAVs and the jammer.

The convergence of channels choice probability of the jammer in different periods is shown in Figure 5,whereqj(z,c)is the probability of the jammer choosing channelcin periodz.Iteration is stopped if a preset objective function tolerance(i.e.,the probability of choosing the channel＞0.999)or the maximum number of iterations 500 is met.The jammer selects channels 4,3,4,4,3,and 4 in turn in 6 periods,which is consistent with the channel selection of the jammer in Figure 4.The jammer’s channel selection probabilities can converge in less than 290 epochs,which indicates that the proposed algorithm has a fast convergence rate.

Figure 5.Convergence of channel selection probability of the jammer.

To analyze the convergence of UAVs’ channel selection,take UAV 2 as an example.Figure 6 shows the convergence curve of UAV 2’s channel selection probability in different periods,whereq2(z,c) is the probability of UAV 2 choosing channelcin periodz.The maximum number of iterations is 300.UAV 2 selects channels 1,4,4,4,3,and 4 in turn in 6 periods,which is consistent with the channel selection of UAV 2 in Figure 4.The UAV 2’s channel selection probabilities can converge in less than 100 slots,which indicates that the proposed algorithm has a fast convergence rate.

Figure 6.Convergence of channel selection probability of UAV 2.

4.2 Performance Comparison

To evaluate the performance of the proposed algorithm,the comparison curves of total system loss with the proposed algorithm,the best NE,the worst NE,and the random selection are presented in Figure 7.As the number of channels increases,the total system loss of these algorithms all decreases.This is because the mutual interference between users decreases.Compared with the random selection algorithm,the total network loss of the proposed algorithm decreases obviously,especially when the number of channels is 6,the anti-jamming performance increases by about 64%.Additionally,when the number of channels is 2,the anti-jamming performance of our proposed algorithm is about 32%better than that of non-cooperative ones,which shows the superiority of the cooperative algorithm.The proposed algorithm is close to the best NE,which indicates the reliability of the proposed algorithm.

Figure 7.The loss of the total system of different algorithms.

Figure 8 shows the impact of transmission power of the jammer and UAVs on the total system loss.As the power of the jammer increases,the interference becomes more serious,leading to the increase of the total loss of the system.At the same time,the increase of users’power also leads to the aggravation of UAVs’interference and the increase of total interference.

Figure 8.The total loss of the system versus transmission power of the jammer and UAVs.

As shown in Figure 9,we change the accuracy of channel estimation of UAV 1.As can be seen,the loss value of the system increases with the increase of the modulus of the channel estimation error.This is because the larger the error,the more serious the cognitive bias of the UAV to the optimal channel.Moreover,the loss value of the system also increases when there exists errors in the channel estimation of more UAVs.This is reasonable and advocates the institution.

Figure 9.System loss under different channel estimation accuracy.

V.CONCLUSION

In this paper,a cooperative anti-jamming and interference mitigation channel selection problem was studied in the multi-UAV communication scenario.The impact of malicious jamming and co-channel interference on system performance was considered.The problem was modeled as a Stackelberg game,and the local altruistic game was established among UAVs.In our game model,the UAV pursued the maximization of the utilities of itself and all other UAVs,and the jammer pursued the maximization of its own utility.To obtain the game equilibrium,a distributed cooperative anti-jamming and interference mitigation algorithm based on local altruism was proposed.Finally,the simulation results showed that the proposed algorithm satisfied good convergence,and significantly improved the anti-jamming performance.

ACKNOWLEDGEMENT

This work was supported in part by the National Natural Science Foundation of China (No.62271253,61901523,62001381),Fundamental Research Funds for the Central Universities (No.NS2023018),the National Aerospace Science Foundation of China under Grant 2023Z021052002,the open research fund of National Mobile Communications Research Laboratory,Southeast University(No.2023D09),and Postgraduate Research &Practice Innovation Program of NUAA(No.xcxjh20220402).

APPENDIX

Here,we provide the proof for the Theorem 1.Given the jammer’s strategy,the potential function of the subgame of the follower can be constructed as(A.1),

Notice that Φ3(dn,z,z-1)=CfC0dn,z,z-1,a function about distance,is irrelevant toan,z.Thus,when UAVnunilaterally changes actionan,zto ?an,z,the change in utility function is as (A.5).

The change in potential function is written as(A.7).

Therefore,we have

(A.8) suggests that the subgame of the follower follows an exact potential game and there exists at least one NE.

It can be seen from(A.8)that the unilateral strategy change of any UAV can lead to the change of potential function that equals to the change of UAV utility function.Considering the positive linear correlation between the potential function and the total loss of the system,the optimal pure strategy NE is the global optimal solution of the problem[36].The proof is done.