arXiv is now an independent nonprofit! Learn more
License: arXiv.org perpetual non-exclusive license
arXiv:2211.12988v1 [cs.MA] 23 Nov 2022

A Secure and Intelligent Data Sharing Scheme for UAV-Assisted Disaster Rescue

Yuntao Wang    Zhou Su    Qichao Xu    Ruidong Li    Tom H. Luan       Pinghui Wang Thanks: Y. Wang, Z. Su, and T. H. Luan are with the School of Cyber Science and Engineering, Xi’an Jiaotong University, Xi’an, China (Corresponding author: Zhou˜Su). Thanks: Q. Xu is with the School of Mechatronic Engineering and Automation, Shanghai University, China. Thanks: R. Li is with the Department of Electrical and Computer Engineering, Kanazawa University, Japan. Thanks: P. Wang is with MOE KLINNS Laboratory, Xi’an Jiaotong University, China
Abstract

Unmanned aerial vehicles (UAVs) have the potential to establish flexible and reliable emergency networks in disaster sites when terrestrial communication infrastructures go down. Nevertheless, potential security threats may occur on UAVs during data transmissions due to the untrusted environment and open-access UAV networks. Moreover, UAVs typically have limited battery and computation capacity, making them unaffordable for heavy security provisioning operations when performing complicated rescue tasks. In this paper, we develop RescueChain, a secure and efficient information sharing scheme for UAV-assisted disaster rescue. Specifically, we first implement a lightweight blockchain-based framework to safeguard data sharing under disasters and immutably trace misbehaving entities. A reputation-based consensus protocol is devised to adapt the weakly connected environment with improved consensus efficiency and promoted UAVs’ honest behaviors. Furthermore, we introduce a novel vehicular fog computing (VFC)-based off-chain mechanism by leveraging ground vehicles as moving fog nodes to offload UAVs’ heavy data processing and storage tasks. To offload computational tasks from the UAVs to ground vehicles having idle computing resources, an optimal allocation strategy is developed by choosing payoffs that achieve equilibrium in a Stackelberg game formulation of the allocation problem. For lack of sufficient knowledge on network model parameters and users’ private cost parameters in practical environment, we also design a two-tier deep reinforcement learning-based algorithm to seek the optimal payment and resource strategies of UAVs and vehicles with improved learning efficiency. Simulation results show that RescueChain can effectively accelerate consensus process, improve offloading efficiency, reduce energy consumption, and enhance user payoffs.

Index Terms: 
Unmanned aerial vehicle, blockchain, vehicular fog computing, deep reinforcement learning, disaster rescue.

I Introduction

Natural disasters, such as earthquakes, bushfires, floods, often inflict devastating losses in lives and property [1, 2, 3]. In disaster areas, reliable and resilient emergency communications are pivotal for quick damage assessment and effective disaster rescue [4]. However, owing to the destruction or inefficiency of terrestrial network infrastructures (e.g., Wi-Fi access points and cellular base stations), unmanned aerial vehicles (UAVs) based communication approach may be the only plausible solution in such situations so far due to their fast deployment and flexible mobility [5, 6, 7]. UAVs can not only be dispatched and deployed quickly to establish the emergency communication infrastructure on the air, but also help information dissemination (e.g., rescue commands, maps of affected areas, and survivors’ locations) in disaster relief networks [8, 9, 10].

There are however several fundamental challenges underlying the UAV-aided disaster relief networks (UDRNs). Since UAVs need to be open to all nodes, including malicious entities, in the disaster area to maximize the life-saving potential, the network is vulnerable to various attacks (e.g., spoofing attack and DDoS attack) perpetrated by adversaries [4]. The UAVs may also be compromised to forge, remove, and replace the transmitted data, and even inject malwares and viruses to mislead or interfere with the rescue operations. In addition, the onboard battery and computing capabilities of UAVs are typically limited, whereas the heavy computation missions in affected areas such as human detection and video recognition often exceed UAVs’ local processing capabilities [11, 12]. Consequently, the time required to execute search and rescue operations may be overextended and the efficiency of disaster rescue may be degraded. Therefore, a secure data transmission scheme with efficient computation and storage offloading should be devised for UAVs in UDRNs.

The emerging blockchain holds numerous potentials to build trust among various collaborative entities; by offering decentralized hash-linked ledgers with time-stamped data and behavior records, the blockchain prevents fraud in a reliable and distributed manner [13, 14, 15]. Participants can share and retrieve the desired data on the basis of blockchain which is featured with immutability, transparency, and auditability. For example, in [16], a decentralized data collection mechanism is designed based on blockchain to safeguard data delivery in UAV-assisted IoT. In [17], a blockchain-based group key distribution mechanism is developed to build trust among UAVs and ensure the security of UAVs’ sensory data.

Existing blockchain approaches for UAVs heavily depend on the availability of infrastructures for security-critical operations (e.g., consensus management and ledger maintenance). A fully distributed approach tolerable to UAV failures is more suitable for practical disaster scenarios. Meanwhile, the compute-intensive data processing and consensus operations along with heavy storage requirements in blockchain-enabled UDRNs often occupy a large amount of constrained computation and storage for UAVs. Existing works mainly focus on cloud or edge based approaches for UAVs’ computation and storage offloading [18, 19, 20]; nonetheless, due to the long distance of remote clouds and the insufficiency or unavailability of edge servers under disasters, the stringent quality of service (QoS) requirements of UAVs (e.g., completing missions before expiration) may fail to meet in the current cloud or edge based offloading approaches. Hence, it is still an open and vital issue to secure data sharing by deploying a lightweight and robust blockchain system in disaster areas while efficiently offloading UAVs’ heavy computation and storage tasks.

In this paper, we develop RescueChain, a novel secure and energy-efficient data sharing scheme for UDRNs. We first propose a lightweight and infrastructure-free blockchain-based framework to safeguard data sharing and immutably trace misbehaving entities in disaster sites. We then devise a reputation-based Tendermint consensus protocol to efficiently and robustly reach consensus under weak network connections while encouraging UAVs’ legitimate behaviors in the network. Afterwards, by exploiting ground vehicles as moving fog nodes, a novel vehicular fog computing (VFC)-based off-chain computation and storage mechanism is presented to collaboratively offload UAVs’ data processing and security provisioning missions to moving vehicles. The Stackelberg game model is formulated to model the interactions between UAVs and vehicles under VFC and the Stackelberg equilibrium (SE) of the static Stackelberg game is derived to stimulate vehicles’ participation in computing resources sharing. As accurate network parameters and the private user cost model are not readily available in realistic offloading applications, a dynamic Stackelberg game is presented, and its near-optimal solution is sought by using an intelligent learning algorithm based on deep reinforcement learning (DRL) techniques. The contribution of this paper is three-fold as follows.

  • Framework: We present RescueChain, which is low-cost, infrastructure-free, and robust. We make two improvements in the implementation of blockchain in UDRNs: 1) a green consensus protocol with non-mining bookkeeping, low communication complexity, and high robustness under partially synchronous environment; 2) a VFC-based off-chain data storage and computation mechanism to efficiently move the heavy data processing and storage missions from resource-constrained UAVs to nearby cooperative ground vehicles.

  • Algorithm: Due to sparse network connections in UDRNs, we develop an energy-efficient and partition-tolerant consensus algorithm for battery-limited UAVs based on Tendermint via signature aggregation and reputation evaluation. Besides, nodes’ benign and malicious behaviors can be immutably traced and recorded on the redesigned blockchain ledgers for reputation computing. To promote vehicles’ collaboration under VFC in the fast-changing environment, a learning-based algorithm in the dynamic Stackelberg game is designed to intelligently schedule the optimal resource sharing and pricing strategies for ground vehicles and UAVs. To cope with the large space size and address the curse of dimensionality in learning, we also exploit DRL techniques with two tiers for efficient state space compression and accelerated convergence rate.

  • Validation: We evaluate the effectiveness of RescueChain through extensive simulations. It is demonstrated that our RescueChain can attain better payoffs for ground vehicles and UAVs, reduced data delivery latency and UAVs’ energy consumption, improved offloading efficiency, and enhanced consensus efficiency in blockchain, by comparing with other existing schemes.

The remainder of the work is organized as follows. Related works are reviewed in Section II. The system model is elaborated in Section III. The design of RescueChain system is presented in Section IV. The optimal offloading policies in static and dynamic games are given in Section V. Section VI evaluates the proposed scheme and Section VII closes this paper with conclusions. For reader’s convenience, the main research contents and their organization structure are illustrated in Fig. 1.

Fig. 1: Main research contents and their organization structure.

II Related Work

In this section, we review the related works on blockchain-based information sharing approaches and computation offloading methods for UAVs.

II-A Blockchain-Based Data Sharing

The emerging blockchain technologies in wireless networks for secure data sharing have attracted wide attention from both academia and industry. Liang et al. [21] design a permissioned blockchain-based electricity trading platform in industrial Internet of things (IoT) to address the high management cost, ensure reliable data transmission, and build trust among grid operators and electricity consumers. By leveraging the consortium blockchain, Chen et al. [22] develop a trustful on-chain and off-chain data sharing framework in vehicular networks, where edge nodes (i.e., roadside units (RSUs)) are responsible for blockchain maintenance by operating the practical Byzantine fault tolerance (PBFT) consensus protocol. Jiang et al. [23] investigate a blockchain-based reliable artificial intelligence (AI) model sharing mechanism for object detection in autonomous driving with cross-domain adaptation, where mobile edge computing (MEC) nodes run the delegated proof-of-stake (DPoS) algorithm for efficient ledger management. By deploying a vehicular blockchain network with homomorphic cryptosystem, Kong et al. [24] propose a secure and verifiable sensory information collection and sharing scheme in fog computing-enabled IoV, where the consensus process in blockchain is managed by RSUs with PBFT protocol. One can observe that most of the existing blockchain systems heavily depend on communication infrastructures for block building and consensus management and cannot be directly applied for disaster areas with severely disrupted or unavailable infrastructures.

II-B Computation Offloading for UAVs

Recently, many works have been reported on computation offloading for UAVs with constrained onboard resources. Bai et al. [18] design an MEC-based energy-efficient task offloading framework for UAVs in wireless networks with consideration of active and passive eavesdroppers and time-duration constraints. By exploiting the edge infrastructures, Callegaro et al. [25] propose an edge computing-based computation offloading scheme and design the optimal offloading policies for UAVs with joint consideration of network and computation load of MEC nodes. Chen et al. [26] present a hybrid edge/cloud computing-based offloading scheme to efficiently offload UAVs’ real-time computation tasks to proximal edge servers or remote cloud servers to reduce latency and improve energy efficiency. Liu et al. [20] devise an online multi-hop trajectory scheduling and computation task assignment algorithm for UAVs based on Lyapunov optimization and Markov approximation approaches for joint computing delay and cost minimization in the edge-cloud environment. However, as network infrastructures can be unavailable or insufficient in disaster scenarios, conventional cloud or edge based offloading mechanisms are difficult to meet the QoS requirements of UAVs. Moreover, different from the assumption in most of the existing works, it is not readily available for both resource consumers and resource contributors to acquire the accurate cost model parameters to determine the optimal offloading policies in practice.

In this work, distinguished from existing works, we study a lightweight and green consensus protocol for disaster scenarios with weak network connections to facilitate the implementation of blockchain into UDRNs. Besides, we devise an intelligent DRL-based optimal offloading algorithm under the promising VFC paradigm by exploiting the idle computation and storage resources shared by ground vehicles. Compared with our previous work [1], new contributions of this paper include: (1) we design a misbehavior tracing mechanism and give the property analysis of our RescueChain system; (2) we derive the SE for the general case of VFC-based offloading with multiple UAVs and vehicles with continuous strategies; and (3) we develop a two-tier deep Q-network (DQN)-based offloading mechanism to enhance the performance of Q-learning based offloading mechanism in [1].

III System Model

In this section, we introduce the system model including the network model, mobility model, VFC model, communication model, and adversary model, respectively. A summary of notations used in the paper is presented in Table I.

Refer to caption
Fig. 2: System model of RescueChain in a disaster area [1].
TABLE I: Summary of Notations
Notation Description Notation Description
\mathcal{I} Set of ground vehicles. ZZ Number of validators in the blockchain.
𝒥\mathcal{J} Set of UAVs. Ψ\Psi Number of level-1 validators.
j[tn]{\mathcal{I}_{j}}[t_{n}] Set of vehicles at UAV jj’s coverage at nn-th time slot. 𝒯1\mathcal{T}_{1} Merkle tree of time-stamped transactions in the block body.
𝒵\mathcal{Z} Set of elected validators in the blockchain. 𝒯2\mathcal{T}_{2} Merkle tree of serialized chunks of the raw block.
𝒰\mathcal{U} Set of full nodes in the blockchain. 𝒯3\mathcal{T}_{3} Merkle tree of signed evidences for misbehavior forensics.
NN Total number of time slots. r,hr,h Round/height in the consensus process.
RjA2G,RjA2AR_{j}^{A2G},R_{j}^{A2A} Radius of A2G/A2A communication range of UAV jj. \hbar The leader designated from level-1 validators.
𝐥j[tn]\mathbf{l}_{j}[t_{n}] Horizontal location of UAV jj at nn-th time slot. Υ\Upsilon_{\hbar} Proof-of-lock (PoL) of the leader {\hbar}.
HjH_{j} Flying altitude of UAV jj. σ,σz\sigma_{\hbar},\sigma_{z} Signature of leader {\hbar}/validator zz.
vj[tn],vjmaxv_{j}[t_{n}],v_{j}^{\max} Velocity/maximum velocity of UAV jj at nn-th time slot. txdatatx_{\mathrm{data}} Off-chain storage transaction.
aj[tn]a_{j}[t_{n}] Acceleration of UAV jj at nn-th time slot. txreptx_{\mathrm{rep}} Report transaction of nodes’ misbehaviors.
v¯GV\overline{v}_{GV} Average vehicular velocity. LastProofLastProof A script for misbehavior forensics.
vGVmax,vGVmin{v_{GV}^{\max}},{v_{GV}^{\min}} Maximum/minimum vehicle velocity. LastCommitLastCommit Precommits making the prior block at height hh committed.
χ,χmax\chi,{\chi_{\max}} Traffic density/maximum traffic density of vehicles. LastVotesLastVotes Votes justifying precommits at all rounds for height hh.
KjK_{j} Total number of UAV jj’s tasks to be offloaded. un\Im_{u}^{n} Reputation value of full node uu.
Γj,k\Gamma_{j,k} kk-th computation task offloaded by UAV jj. ¯un{{\overline{\Im}_{u}^{n}}} Normalized reputation value of full node uu.
Dj,kD_{j,k} Data size of task Γj,k\Gamma_{j,k}. uini\Im_{u}^{\mathrm{ini}} Initial reputation value of full node uu.
θi,j,k\theta_{i,j,k} Required CPU cycles to process one bit of task Γj,k\Gamma_{j,k}. Δb\Delta_{b} Reputation reward/penalty for behavior bb.
Tj,kmaxT_{j,k}^{\max} Time-to-live (TTL) of task Γj,k\Gamma_{j,k}. xi,j,k{x_{i,j,k}} Amount of computing resource (AoCR) in doing task Γj,k\Gamma_{j,k}.
αj,k\alpha_{j,k} Urgency degree of task Γj,k\Gamma_{j,k}. yi,j,k{y_{i,j,k}} Payment of task Γj,k\Gamma_{j,k}.
μj,k\mu_{j,k} Output/input ratio of task Γj,k\Gamma_{j,k}. Sj,k(xi,j,k)S_{j,k}(x_{i,j,k}) Satisfaction function of UAV jj.
Ej[tn]E_{j}[t_{n}] UAV jj’s remaining battery energy at nn-th time slot. Cj,k(xi,j,k,yi,j,k)C_{j,k}(x_{i,j,k},y_{i,j,k}) Cost function of UAV jj.
EjminE_{j}^{\min} Minimum energy reserve of UAV jj. πj(𝐱j,𝐲j)\pi_{j}\left(\mathbf{x}_{j},\mathbf{y}_{j}\right) Payoff function of UAV jj.
CjC_{j} Battery energy capacity of UAV jj. πi(xi,j,k,yi,j,k){\pi_{i}}({x_{i,j,k}},y_{i,j,k}) Payoff function of ground vehicle ii.
Pjfly[tn]P_{j}^{\mathrm{fly}}[t_{n}] Flying or propulsion power of UAV jj at nn-th time slot. λp,λc,λe\lambda_{p},\lambda_{c},\lambda_{e} Positive adjustment parameters.
PiTX,PjTXP_{i}^{\mathrm{TX}},P_{j}^{\mathrm{TX}} Transmit power of vehicle ii/UAV jj. Ψi(xi,j,k)\Psi_{i}(x_{i,j,k}) Cost function of vehicle ii.
Ei,j,kvfcE_{i,j,k}^{\mathrm{vfc}} Energy consumption of vehicle ii in task execution. 𝐬j(n)\mathbf{s}_{j}^{(n)} State of UAV jj at time slot nn.
Ei,j,kflyE_{i,j,k}^{\mathrm{fly}} Energy consumption of UAV jj in movement. s~i,j,k(n){\tilde{s}}_{i,j,k}^{(n)} State of vehicle ii at time slot nn.
Ei,j,kA2GE_{i,j,k}^{\mathrm{A2G}} Energy consumption of UAV jj in A2G transmission. Q(𝐬j(n),𝐲j(n))Q(\mathbf{s}_{j}^{(n)},\mathbf{y}_{j}^{(n)}) Q-function of UAV jj for state-action pair (𝐬j(n),𝐲j(n))(\mathbf{s}_{j}^{(n)},\mathbf{y}_{j}^{(n)}).
φi,j\varphi_{i,j} Channel gain between UAV jj and vehicle ii. Q~(s~i,j,k(n),xi,j,k(n))\tilde{Q}({\tilde{s}}_{i,j,k}^{(n)},{x}_{i,j,k}^{(n)}) Q-function of vehicle ii for state-action pair (s~i,j,k(n),xi,j,k(n))({\tilde{s}}_{i,j,k}^{(n)},{x}_{i,j,k}^{(n)}).
di,j[tn]d_{i,j}[t_{n}] Euclidean distance between UAV jj and vehicle ii. ςj,ςi\varsigma_{j},\varsigma_{i} Discount factor of UAV jj/vehicle ii.
γi,jG2A,γi,jA2G{\gamma_{i,j}^{\mathrm{G2A}}},{\gamma_{i,j}^{\mathrm{A2G}}} Transmission rate of G2A uplink/A2G downlink. 𝜸j(n),𝜸i(n){\bm{\gamma}}_{j}^{(n)},{\bm{\gamma}}_{i}^{(n)} State sequence of UAV jj/vehicle ii in DQN.
BiUL,BjDLB_{i}^{\mathrm{UL}},B_{j}^{\mathrm{DL}} Uplink/downlink bandwidth. ϕj(n),ϕi(n){\bm{\phi}}_{j}^{(n)},{\bm{\phi}}_{i}^{(n)} Interaction experience of UAV jj/vehicle ii in DQN.
Φi,j,k\Phi_{i,j,k} Total offloading latency of task Γj,k\Gamma_{j,k}. Ξj,Ξi\Xi_{j},\Xi_{i} Replay memory of UAV jj/vehicle ii in DQN.
ti,j,kvfct_{i,j,k}^{\mathrm{vfc}} Execution time of task Γj,k\Gamma_{j,k}. ϵj,ϵi\epsilon_{j},\epsilon_{i} Parameter of ϵ\epsilon-greedy policy of UAV jj/vehicle ii.
ti,j,kA2Gt_{i,j,k}^{\mathrm{A2G}} A2G transmission time of task Γj,k\Gamma_{j,k}. WW Number of UAV’s payment levels in DQN.
ti,j,kG2At_{i,j,k}^{\mathrm{G2A}} G2A transmission time of task Γj,k\Gamma_{j,k}. VV Number of vehicle’s AoCR levels in DQN.

III-A Network Model

Fig. 2 depicts a typical UAV-aided disaster rescue network (UDRN), which is composed of a group of UAVs, ground vehicles, ground stations, and a permissioned blockchain. More basic information about UDRNs can be found in works [10, 8].

In a given investigated disaster area, a set of vehicles, denoted as ={1,,i,,I}\mathcal{I}=\{1,\cdots,i,\cdots,I\}, are to perform search and rescue missions, where part of roads and network infrastructures are damaged. To facilitate data transmissions among ground rescue vehicles, a fleet of UAVs, denoted as 𝒥={1,,j,,J}\mathcal{J}=\{1,\cdots,j,\cdots,J\}, are dispatched to establish emergency communications by forming flexible aerial subnetworks. Each UAV j𝒥j\in\mathcal{J} is equipped with multiple sensors (e.g., thermometer, infrared camera, and GPS) to perceive its surroundings. On one hand, to save UAV’s limited battery energy, the sensory data (e.g., images, videos, and audios) collected by UAVs can be transmitted to ground vehicles for processing (referred to as VFC) such as pattern recognition and survivor detection via aerial-to-ground (A2G) links. On the other hand, a group of UAVs can serve as aerial communication relays via aerial-to-aerial (A2A) links to enhance network connectivity. Let RjA2GR_{j}^{A2G} and RjA2AR_{j}^{A2A} denote the radiuses of A2G and A2A communication ranges of UAV jj, respectively. Due to the destruction of network infrastructures, the emergency control centers such as ground stations are deployed to coordinate both aerial and ground subnetworks by scheduling UAVs and vehicles. To better adapt to the affected areas, the ground station is hosted by an emergency communication vehicle (ECV) [27] with powerful communication and computing capabilities.

The blockchain contains a growing sequence of hash-chained blocks and each block \mathcal{B} has two parts: the block body bodybody and the block header headerheader. The block body includes a set of time-stamped records (e.g., sensory data, computing results, rescue commands, and node misbehaviors) which are compressed into a Merkle tree 𝒯1\mathcal{T}_{1}. To alleviate the heavy storage burden of blockchain, only the data pointers are recorded on-chain while the source data are moved to an off-chain data repository. The block header consists of the metadata of block such as the hashes of parent and current blocks, the Merkle root of 𝒯1\mathcal{T}_{1}, the block height, the signature of block creator, aggregated signatures of block validators, and a script LastProofLastProof for misbehavior tracing (defined in IV-D). In RescueChain, only authorized entities can participate in the permissioned blockchain network after registration at the certification authority (CA) using their true identities. Let ={1,,m,,M}\mathcal{M}=\{1,\cdots,m,\cdots,M\} be the set of authorized nodes in the blockchain. Three different roles of participants are considered in the blockchain as follows:

  • Full nodes store the copy of all blocks and are candidates of validators.

  • Lightweight nodes only need to store the block headers and can receive blockchain services from nearby full nodes. They can not participate in the consensus process for ledger maintenance.

  • Validators are part of full nodes and serve as consensus nodes in the blockchain that are responsible for consensus management by executing consensus protocols.

In the system, each ECV acts as a full node. Each authorized UAV can opt to be a lightweight node or full node based on its computation and storage capacities.

III-B Mobility Model

For efficient flying trajectory modeling, the total time period TT is evenly divided into NN time slots with an interval of Δt=TN\Delta_{t}\!=\!\frac{T}{N}. As each time slot can be sufficiently small, UAV jj’s instant location at nn-th time slot can be roughly fixed. The horizontal location of UAV jj at nn-th time slot is denoted as 𝐥j[tn]=(xj[tn],yj[tn])\mathbf{l}_{j}[t_{n}]=(x_{j}[t_{n}],y_{j}[t_{n}]). To avoid frequent ascending and descending and maintain continuous flight over the air, the hovering altitude of UAV jj is considered to be fixed at Hj{H}_{j} during executing a rescue task [5, 28, 29, 30, 31], which varies for different tasks. In practice, the fixed hovering altitude refers to the lowest altitude at which the UAV can avoid any terrain or building obstructions in the task area, hence reducing the additional energy loss caused by frequent height changes. Let EjminE_{j}^{\min} be the minimum energy reserve of UAV jj to prolong its battery life. Then, UAV jj’s remaining battery energy at nn-th time slot is constrained by EjminEj[tn]CjE_{j}^{\min}\leq E_{j}[t_{n}]\leq C_{j}, where CjC_{j} is the battery energy capacity of UAV jj. As UAV jj flies at a constant height, according to [29, 30, 31], its flying power Pjfly[tn]P_{j}^{\mathrm{fly}}[t_{n}] to propel its mobility and keep it aloft at nn-th time slot is positively related with both the velocity vj[tn]v_{j}[t_{n}] and acceleration aj[tn]a_{j}[t_{n}], i.e.,

Pjfly[tn]=λ1(vj[tn])3+λ2vj[tn](1+(aj[tn])2g2),\displaystyle P_{j}^{\mathrm{{fly}}}[t_{n}]=\lambda_{1}(v_{j}[t_{n}])^{3}+\frac{\lambda_{2}}{v_{j}[t_{n}]}\left(1+\frac{(a_{j}[t_{n}])^{2}}{g^{2}}\right), (1)

where gg is gravitational acceleration with nominal value 9.8m/s29.8\,\mathrm{m/s^{2}}. λ1\lambda_{1} and λ2\lambda_{2} are parameters depending on UAV’s weight (containing its payload), air density, wing area, UAV type (e.g., fixed-wing or rotary-wing), etc. The flying velocity vj[tn]v_{j}[t_{n}] of UAV jj is:

vj[tn]=𝐥j[tn+1]𝐥j[tn]Δtvjmax,1n<N,\displaystyle v_{j}[t_{n}]=\frac{\mathbf{l}_{j}[t_{n+1}]-\mathbf{l}_{j}[t_{n}]}{\Delta_{t}}\leq v_{j}^{\max},1\leq n<N, (2)

where vjmaxv_{j}^{\max} is the maximum velocity of UAV jj.

For ground vehicles, the fluid traffic model [32] is adopted to capture the relationship between the traffic density χ\chi and average vehicular velocity v¯GV\overline{v}_{GV}. We have:

v¯GV=max{vGVmin,vGVmax(1χ/χmax)},\displaystyle\overline{v}_{GV}=\max\left\{{{v_{GV}^{\min}},{v_{GV}^{\max}}\left({1-{{\chi}}/{{\chi_{\max}}}}\right)}\right\}, (3)

where χmax{\chi_{\max}} means the maximum traffic density. vGVmax{v_{GV}^{\max}} and vGVmin{v_{GV}^{\min}} are the maximum and minimum vehicle velocities, respectively. The number of ground vehicles entering the communication coverage of UAV jj at nn-th time slot can be computed as ϕj[tn]=χv¯GVΔt\phi_{j}[t_{n}]={\chi}\overline{v}_{GV}\Delta_{t}. Let oj[tn]o_{j}[t_{n}] denote the ratio of ground vehicles leaving the coverage of UAV jj at nn-th time slot. Based on [11], the number of vehicles in the communication range of UAV jj at nn-th time slot is calculated as:

Ij[tn]={ϕj[t1](1oj[t1]),n=1;(ϕj[tn]+Ij[tn1])(1oj[tn]),<nN.\displaystyle{I_{j}}[t_{n}]\!=\!\begin{cases}{\phi_{j}}[t_{1}](1-{o_{j}}[t_{1}]),&n=1;\\ \left({\phi_{j}}[t_{n}]+{I_{j}}[t_{n-1}]\right)\left(1-{o_{j}}[t_{n}]\right),&1\!<\!n\!\leq\!N.\end{cases} (4)

III-C Vehicular Fog Computing Model

The limited communication and battery resources of UAVs may be unaffordable to execute compute-intensive and delay-critical rescue tasks. Consequently, the executing time for rescue missions may be prolonged and the recharging interval of UAVs may be shortened. Under the VFC paradigm, the UAV’s heavy data computation and storage missions can be efficiently offloaded to ground vehicles in its communication coverage by collaboratively contributing their idle resources. Let j[tn]={1,,i,,Ij[tn]}\mathcal{I}_{j}[t_{n}]=\{1,\cdots,i,\cdots,I_{j}[t_{n}]\} be the set of cooperative vehicles in UAV jj’s coverage at nn-th time slot. Each task of UAV jj to be offloaded to a vehicle ij[tn]i\in\mathcal{I}_{j}[t_{n}] can be denoted as a 4-tuple, i.e.,

Γj,k=Dj,k,θi,j,k,Tj,kmax,αj,k,1kKj,\displaystyle\Gamma_{j,k}=\left\langle D_{j,k},\theta_{i,j,k},T_{j,k}^{\max},\alpha_{j,k}\right\rangle,1\leq k\leq K_{j}, (5)

where Dj,kD_{j,k} (in bits) denotes the data size of task. θi,j,k\theta_{i,j,k} (in CPU cycles/bit) indicates the required CPU cycles to process one bit. Tj,kmaxT_{j,k}^{\max} (in seconds) means the time-to-live (TTL) of task. αj,k[0,1]\alpha_{j,k}\in[0,1] is the urgency degree for mission processing, and a higher αj,k\alpha_{j,k} implies a higher urgency degree. KjK_{j} is the total number of UAV jj’s tasks to be offloaded.

In VFC, when all Dj,kD_{j,k} bits of task data are transmitted to a ground vehicle ij[tn]i\!\in\!\mathcal{I}_{j}[t_{n}] through A2G communication, the data computation process is executed immediately on the vehicle, and then μj,kDj,k\mu_{j,k}D_{j,k} bits of the processed results are sent back to the UAV jj through ground-to-aerial (G2A) communication. Here, μj,k(0,1)\mu_{j,k}\!\in\!(0,1) denotes the output/input ratio. Let xi,j,kx_{i,j,k} be the amount of computing resource (AoCR) contributed by vehicle ii in performing mission Γj,k\Gamma_{j,k}, which is measured by CPU cycles per second. According to [33], the execution time of mission Γj,k\Gamma_{j,k} in vehicle ii can be calculated as ti,j,kvfc=θi,j,kDj,kxi,j,kt_{i,j,k}^{\mathrm{vfc}}=\frac{\theta_{i,j,k}D_{j,k}}{x_{i,j,k}}. Besides, the energy consumption of vehicle ii in performing mission Γj,k\Gamma_{j,k} is:

Ei,j,kvfc=κi(xi,j,k)3ti,j,kvfc=κiθi,j,kDj,k(xi,j,k)2,\displaystyle E_{i,j,k}^{\mathrm{vfc}}=\kappa_{i}(x_{i,j,k})^{3}\cdot t_{i,j,k}^{\mathrm{vfc}}=\kappa_{i}\theta_{i,j,k}D_{j,k}(x_{i,j,k})^{2}, (6)

where κi\kappa_{i} is the effective switched capacitance [34].

III-D Communication Model

For A2G/G2A communications, the line-of-sight (LoS) link is presumed to be dominant [33], and the channel gain between UAV jj and ground vehicle ii can be modeled by the free-space path loss model, i.e., φi,j[tn]=φ0(di,j[tn])μ\varphi_{i,j}[t_{n}]={\varphi_{0}}\left(d_{i,j}[t_{n}]\right)^{-\mu}, where φ0{\varphi_{0}} is the channel gain at the reference distance d0=1d_{0}=1, μ>1\mu>1 means the path loss exponent, and di,j[tn]d_{i,j}[t_{n}] represents the Euclidean distance between UAV jj and ground vehicle ii at nn-th time slot. The frequency division multiple access (FDMA) protocol is applied for bandwidth sharing among ground vehicles in transmitting computation outcomes to the UAV [34]. Then, the available G2A uplink transmission rate from vehicle ii to UAV jj is obtained as:

γi,jG2A[tn]=BiULlog2(1+PiTXφ0(di,j[tn])μBiULσ02),\displaystyle{\gamma_{i,j}^{\mathrm{G2A}}}[t_{n}]=B_{i}^{\mathrm{UL}}{\log_{2}}\bigg({1+\frac{{P_{i}^{\mathrm{TX}}{\varphi_{0}}\left(d_{i,j}[t_{n}]\right)^{-\mu}}}{B_{i}^{\mathrm{UL}}\sigma_{0}^{2}}}\bigg), (7)

where BiULB_{i}^{\mathrm{UL}} is the assigned uplink bandwidth to vehicle ii, PiTXP_{i}^{\mathrm{TX}} means vehicle ii’s transmit power, and σ02\sigma_{0}^{2} is the power of additive white Gaussian noise. Moreover, the A2G downlink transmission rate from UAV jj to vehicle ii is denoted as:

γi,jA2G[tn]=BjDLlog2(1+PjTXφ0(di,j[tn])μBjDLσ02),\displaystyle{\gamma_{i,j}^{\mathrm{A2G}}}[t_{n}]=B_{j}^{\mathrm{DL}}{\log_{2}}\bigg({1+\frac{{P_{j}^{\mathrm{TX}}{\varphi_{0}}\left(d_{i,j}[t_{n}]\right)^{-\mu}}}{B_{j}^{\mathrm{DL}}\sigma_{0}^{2}}}\bigg), (8)

where BjDLB_{j}^{\mathrm{DL}} is UAV jj’s downlink bandwidth, and PjTXP_{j}^{\mathrm{TX}} is UAV jj’s transmit power.

III-E Adversary Model

Three kinds of adversaries considered during information sharing in UDRNs are listed as follows.

  • Malicious Insider UAVs. Due to the open-access features of UAV networking to maximize the life-saving potential under disasters, any entities including malicious ones can connect to UAVs freely. Consequently, UAVs may be compromised and controlled by them to tamper with the sensory data, disseminate falsified information, and produce fake blocks.

  • Threats to Data Dependability. Owing to the unreliable wireless environment, the disseminated data (e.g., sensory data, processed results, and scheduling commands) may be forged, replaced, or deleted during multi-hop data transmissions[35]. Moreover, it is difficult to transparently audit the delivered information to enforce accountability when disputes occur.

  • Threats to Misbehavior Traceability. Traditional centralized misbehavior tracing mechanisms usually lack transparency and auditability and may suffer from the single point of failure (SPoF) and DDoS attack, causing difficulty in trustworthy, auditable, and immutable misbehavior tracing and punishment enforcement.

IV Design of RescueChain

In this section, we present the detailed design of RescueChain, which is a blockchain-based system for secure and efficient information sharing in UDRNs.

IV-A Overview of RescueChain

In UDRNs, owing to the sparse network environment and possible poor connections, network partitions may be frequent, resulting in the risks of blockchain forking. Besides, the limited resource of UAVs in terms of battery, computation, and storage restrict the adoption of conventional resource-hungry blockchains. Accordingly, our RescueChain should be low-cost, infrastructure-free, and robust in UDRNs. To achieve the three goals, we make two improvements: one is the VFC-based off-chain mechanism to offload the heavy data computation and storage tasks from UAVs; the other is the reputation-based Tendermint consensus protocol to robustly reach consensus in the weakly connected environment with enhanced safety. In particular, as shown in Fig. 3, the following three phases need to be undertaken:

  • Entity registration and key distribution. In this phase, after registration at CA, each authorized entity obtains its key pair and opts its role in the blockchain system.

  • VFC-based off-chain storage and computation. This phase performs off-chain data computation and storage for UAVs under the VFC paradigm.

  • Reputation-based Tendermint consensus process. In this phase, each entity runs the reputation-based Tendermint protocol to reach consensus on the new transactions to be added to the blockchain via validator election, two-phase voting with locking, and reputation assessment.

Fig. 3: Workflow of the proposed Rescuechain.

IV-B Entity Registration and Key Distribution

The system parameters are selected by CA based on the Boneh-Lynn-Shacham (BLS) short signature scheme [36]. A multiplicative bilinear map e:𝔾1×𝔾2𝔾3e:\mathbb{G}_{1}\times\mathbb{G}_{2}\rightarrow\mathbb{G}_{3} is chosen by CA, where g1g_{1} and g2g_{2} are generators of groups 𝔾1\mathbb{G}_{1} and 𝔾2\mathbb{G}_{2}, and qq is the prime order of 𝔾k(k=1,2,3)\mathbb{G}_{k}(k=1,2,3). Two hash functions 𝐇0:{0,1}𝔾2\mathbf{H}_{0}:\{0,1\}^{*}\rightarrow\mathbb{G}_{2} and 𝐇1:{0,1}q\mathbf{H}_{1}:\{0,1\}^{*}\rightarrow\mathbb{Z}_{q} are chosen by CA. Then, CA broadcasts the system parameters parmparm to the network:

parm=(e,𝔾1,𝔾2,𝔾3,q,g1,g2,𝐇0,𝐇1).\displaystyle parm=(e,\mathbb{G}_{1},\mathbb{G}_{2},\mathbb{G}_{3},q,g_{1},g_{2},\mathbf{H}_{0},\mathbf{H}_{1}). (9)

Each authorized node mm\in\mathcal{M} registers itself at CA using the real identity RIDmRID_{m} and receives a group of LL private/public key pairs {skml,pkml}l=1L\{{sk}_{m}^{l},{pk}_{m}^{l}\}_{l=1}^{L}, wallet addresses, and certificates in the blockchain from CA. Here, skml𝑆q{sk}_{m}^{l}\xleftarrow{S}\mathbb{Z}_{q}, pkml=g2skml𝔾2{pk}_{m}^{l}=g_{2}^{{sk}_{m}^{l}}\in\mathbb{G}_{2}, and 𝑆\xleftarrow{S} indicates randomly sampling.

IV-C VFC-Based Off-Chain Storage and Computation

Under VFC, UAVs’ compute-intensive missions (e.g., human detection and multi-object tracking) on sensory data can be offloaded to ground vehicles to save their constrained battery energy. To alleviate the overhead of blockchain and improve system scalability, the VFC-based off-chain approach is adopted by moving both sensed and processed data to off-chain repositories while retaining a data pointer of source data on the blockchain. The set of data to be disseminated in UDRNs is denoted as 𝒟\mathcal{D}. Each data d𝒟d\in\mathcal{D} is composed of two parts, i.e., the raw sensory data drawd_{\mathrm{raw}} and the computation outcome doutd_{\mathrm{out}}. In RescueChain, the InterPlanetary File System (IPFS) [37] serves as the distributed off-chain data store, where each data dd is stored in the format of distributed files in IPFS and is uniquely addressed by its hash pointer 𝐇0(d)={𝐇0(draw),𝐇0(dout)}\mathbf{H}_{0}(d)=\{\mathbf{H}_{0}(d_{\mathrm{raw}}),\mathbf{H}_{0}(d_{\mathrm{out}})\}. The ground vehicles serve as the distributed storage nodes in IPFS. After VFC-based computation offloading process for task Γj,k\Gamma_{j,k}, UAV jj sends an off-chain storage transaction txdatatx_{\mathrm{data}} to the network as:

txdata=\displaystyle tx_{\mathrm{data}}=

pkjl||{pkil}i=1Ij||𝐇0(d)||descd||tStamp||σj||σk||certDS,\bigm\langle{pk}_{j}^{l}||\{{pk}_{i}^{l}\}_{i=1}^{I_{j}}||\mathbf{H}_{0}(d)||desc_{d}||tStamp||\sigma_{j}||\sigma_{k}||cert_{DS}\bigm\rangle,

(10)

where pkjl{pk}_{j}^{l} is the public key of UAV jj. {pkil}i=1Ij\{{pk}_{i}^{l}\}_{i=1}^{I_{j}} is the group of public keys of vehicles involved in task Γj,k\Gamma_{j,k}. descddesc_{d} is the description of data dd. tStamptStamp is the timestamp of txdatatx_{\mathrm{data}} generation. σj=𝐇0(txdata)skjl\sigma_{j}=\mathbf{H}_{0}(tx_{\mathrm{data}})^{{sk}_{j}^{l}} means the signature of UAV jj. σk=i=1Ijσ¯i\sigma_{k}=\prod_{i=1}^{I_{j}}\bar{\sigma}_{i} is the BLS multi-signature of Ij{I_{j}} vehicles, where σ¯i=𝐇0(txdata)ωiskil\bar{\sigma}_{i}=\mathbf{H}_{0}(tx_{\mathrm{data}})^{\omega_{i}\cdot{sk}_{i}^{l}} and ωi=𝐇1(pkil,{pk1l,,pkIjl})\omega_{i}=\mathbf{H}_{1}({pk}_{i}^{l},\{{pk}_{1}^{l},\cdots,{pk}_{I_{j}}^{l}\}). certDScert_{DS} is the unique certificate issued by the IPFS data store.

IV-D Reputation-Based Tendermint Consensus Protocol

In RescueChain, the Tendermint protocol [38] is adopted to reach consensus among distrustful UAVs due to its high energy efficiency and partial partition tolerance than its alternatives. To be adaptive to the sparse network environment in UDRNs, the multi-signature method [36] is applied in the consensus phase for bandwidth saving. To further enhance the safety of consensus and speed up the consensus process, our RescueChain incorporates reputation assessment into the Tendermint for validator election and block bookkeeping. In the proposed reputation-based Tendermint protocol, the following steps are included:

Step 1: Validator election. The consensus process is carried out by an elected validator committee, denoted as 𝒵={1,,z,,Z}\mathcal{Z}=\{1,\cdots,z,\cdots,Z\}. Let 𝒰={1,,u,,U}\mathcal{U}=\{1,\cdots,u,\cdots,U\} be the set of full nodes. Every full node u𝒰u\in\mathcal{U} can vote for a delegate and its voting weight is determined by its stake, namely, the reputation value ¯u{\overline{\Im}_{u}}. The top ZZ delegates with the highest votes are selected to form the validator committee 𝒵\mathcal{Z}, which can be further classified into two kinds: level-1 validators and level-2 validators. The top Ψ\Psi validators with the highest votes serve as level-1 validators which can propose and validate new blocks, while the remaining ZΨZ-\Psi validators are level-2 validators that can only perform block verification.

Step 2: Consensus process. The consensus process is composed of three phases: propose, prevote, and precommit. To eventually reach consensus in weakly connected UDRNs, the network is assumed to be partially synchronous, and the partial synchrony model [39] is applied in protocol design. An upper bound Δ\Delta of transmission delay and an unknown global stabilization time (GST) are defined in UDRN. In the partial synchrony model, there exist either a known upper bound Δ\Delta after the unknown GST or an unknown upper bound Δ\Delta on the transmission latency of messages. It means that the transmission between two correct entities in UDRN will eventually arrive within 1) the unknown timeout Δ\Delta, or 2) the known timeout Δ\Delta beginning with an unknown GST. At each round of consensus process for a certain height, a small fixed increment of the timeout Δ\Delta is added than its previous round [40]. Besides, the internal clocks of non-Byzantine validators are assumed to be sufficiently accurate during a short time period until reaching consensus on the next block. In the following, we elaborate the detailed process to reach consensus in our RescueChain.

1) Propose phase. A leader \hbar is designated from Ψ\Psi level-1 validators in a round-robin fashion. At its round rr, the leader \hbar collects a batch of recent transactions from its local memory pool, compresses them into a Merkle tree 𝒯1\mathcal{T}_{1}, and packages them into a block \mathcal{B} which includes a hash linked to the previous block. Then, a signed proposal is gossiped to the network, i.e.,

proposal=Proposal||h||r||Υσ,\displaystyle{proposal}=\left\langle{{\mathrm{Proposal}}||h||r||\mathcal{B}||\Upsilon_{\hbar}||\sigma_{\hbar}}\right\rangle, (11)

where Υ\Upsilon_{\hbar} is the proof-of-lock (PoL) indicating the locked block of the leader {\hbar}, and σ\sigma_{\hbar} is its signature. A PoL for block \mathcal{B} (or nilnil) means that a leader receives at least 23Z\frac{2}{3}Z prevotes for block \mathcal{B} (or nilnil) at round rr for height hh [40]. Note that the block \mathcal{B} can be quite large, and can be a network bottleneck in disseminating it to other validators. To alleviate the communication burden of leader {\hbar} in disseminating block \mathcal{B}, we design a block partition method for efficient proposal transmission. Specifically, the raw block is serialized and split into multiple chunks with appropriate size, and all the chunks are hashed into a Merkle tree 𝒯2\mathcal{T}_{2}. A Merkle root rootroot_{\mathcal{B}} together with the signature of leader {\hbar} are included in the proposal. Once a validator receives all the chunks, it performs the block deserialization operation and validates the correctness of the received proposal by checking the Merkle root rootroot_{\mathcal{B}}.

2) Two-phase voting with locking. In two-phase voting, each validator sends its prevote result of the proposal to the network in the first phase, and tells its precommit result of other validators’ claims regarding the proposal in the second phase. Intuitively, it can be ensured that the result of the first phase has been witnessed by enough validators in the second phase to offer Byzantine tolerance. Besides, the following two rules are defined in the locking mechanism to ensure blockchain safety:

  • Locking rule. A validator zz is locked on {\mathcal{B}} (or nilnil) if it receives over 23Z\frac{2}{3}Z prevotes for block \mathcal{B} (or nilnil) at round rr for height hh. Then, at round r>rr^{\prime}>r for height hh, it should prevote for the locked block, and propose it if it is the leader. This can prevent validators from prevoting different blocks in different rounds, thereby protecting safety as conflicting blocks may be committed at the same height.

  • Unlocking rule. A validator can only unlock the block {\mathcal{B}} which is locked at round rr if there exist a PoL for another block {\mathcal{B}}^{\prime} (or nilnil) at a higher round rr^{\prime}, where r>rr^{\prime}>r. This allows validators to be unlocked and precommit another block that the rest of the network intend to commit, thereby protecting liveness.

If the validator z𝒵z\in\mathcal{Z} is locked on a block proposal at a previous round, it broadcasts a prevote for the locked block with its signature σz\sigma_{z}. Otherwise, if a valid proposal is received for the current round after verification, validator zz broadcasts a prevote for that block with its signature. If no proposal or a invalid one is received within the timeout proposeTimeoutproposeTimeout, validator zz sends a special prevote for nilnil instead. The form of prevote message is:

prevote=prevote||hr𝐇0(block)||σz,\displaystyle{prevote}=\left\langle{{\mathrm{prevote}}||h||r||\mathbf{H}_{0}(block)||\sigma_{z}}\right\rangle, (12)

where blockblock is the prevoted block or nilnil. proposeTimeoutproposeTimeout is the maximum time that a validator stays in the propose phase, which is initialized as Δpropose\Delta_{propose} at the beginning of a height hh and is incremented every time proposeTimeoutproposeTimeout expires.

If there exists a PoL for block \mathcal{B}, the validator zz locks on block \mathcal{B}, releases the prior locked block, and broadcasts a precommit for \mathcal{B}. If there exists a PoL for nilnil, the validator zz unlocks, and it broadcasts a precommit for nilnil. Otherwise, the validator zz precommits nilnil. The explicit form of precommit message is

precommit=precommit||hr𝐇0(block)||σz.\displaystyle{precommit}=\left\langle{{\mathrm{precommit}}||h||r||\mathbf{H}_{0}(block)||\sigma_{z}}\right\rangle. (13)

The same prevote or precommit messages signed by distinct validators can be aggregated by BLS multi-signatures for compressed signature size. If validator zz receives over two-thirds of precommits for a specific block, it simply sets its commit time commitTimecommitTime to the current time and moves on to round 00 at the next height h+1h+1. Then, the specific block is eventually committed. Otherwise, it moves to the next round r+1r+1 at current height hh. Then, a new leader is designated and the above process is repeated. Before entering the next height h+1h+1, validators wait until some fixed time duration past commitTimecommitTime to include more commits from validators with slower network connections.

Step 3: Misbehavior tracing. Malicious validators may conduct various kinds of misbehaviors to crash the consensus process and fork the blockchain. A malicious leader may conduct conflicting proposals (cp) misbehavior by disseminating conflicting block proposals to different validators within a round, and malicious validators may perform conflicting votes (cv) misbehavior by giving prevotes or precommits to conflicting proposals. A malicious leader can conduct wrong block creation (wbc) misbehavior by producing invalid block or true block at false round and height, and non-block creation (nbc) misbehavior if the block proposal is not generated on time or it is offline or partitioned. The above misbehaviors (i.e., cpcp, cvcv, wbcwbc, and nbcnbc) can be identified via their signatures attached on the proposals or votes, thereby the corresponding misbehaving entities can be detected. Moreover, malicious validators can perform violation of locking (vol) misbehaviors by violating the locking and unlocking rules to make the network commit on two different blocks at a height. In this case, if a prevote or precommit resulted from a volvol misbehavior affects the final commit, at least one honest validator must have been received it. As honest validators should broadcast all their received votes at each round of a height, the evidences of violations of the locking rule can be collected by stitching all votes and matching every prevote with the most recent precommit of the same validator. The violations of unlocking rule can be detected similarly via matching every precommit with the PoL that justifies it.

For efficient and reliable misbehavior forensics and tracing, a script denoted by LastProofLastProof is included in the next block at height h+1h+1, which contains the set of signed precommits that make the prior block at height hh committed (i.e., LastCommitLastCommit), together with all votes justifying the precommits at all rounds for height hh (i.e., LastVotesLastVotes). As such, the unjustified votes and the misbehaving entities can be identified and traced. For alleviated blockchain burden, all the votes in LastVotesLastVotes are moved to an off-chain data store instead, and only a Merkle tree 𝒯3\mathcal{T}_{3} constructed from LastVotesLastVotes are included in the LastProofLastProof with a root rootVroot_{V}. Furthermore, to accelerate misbehavior detection, nodes can form informer groups to report misbehaviors by collecting the cryptography trails as evidence and generating a report transaction as:

txrep=pkul||{pkul}u=1Nrep||Evidence||fee||tStamp||σrep,tx_{\mathrm{rep}}=\bigm\langle{pk}_{u^{\prime}}^{l}||\{{pk}_{u}^{l}\}_{u=1}^{N_{rep}}||Evidence||fee||tStamp||\sigma_{rep}\bigm\rangle,

(14)

where pkul{pk}_{u^{\prime}}^{l} is the public key of accused node uu^{\prime}. {pkul}u=1Nrep\{{pk}_{u}^{l}\}_{u=1}^{N_{rep}} and σrep\sigma_{rep} are the public keys and BLS multi-signature of the Nrep{N_{rep}} informers, respectively. feefee is the fixed report fee to prevent DDoS attack and is evenly divided among them. EvidenceEvidence is the collected evidence such as the digital signatures on votes or proposals at different rounds of a height.

Step 4: Reputation evaluation. The reputation of each full node is calculated based on two aspects: behavior effect and time fading effect. Commonly, node’s reputation can be gradually improved by benign behaviors and significantly decreased by misbehaviors; meanwhile, latest behaviors are of more significance than previous ones in determining current reputation. The following benign behaviors are considered in blockchain: the successful block creation (sbc) behavior of the leader and the successful block verification (sbv) behavior of validators. Both the sbcsbc and sbvsbv behaviors can be detected by the signatures on block proposal and precommits included in the LastCommitLastCommit. Let Δb\Delta_{b} be the reputation reward or penalty for behavior bb. A binary variable βb={1,1}\beta_{b}=\{-1,1\} is defined, where βb=1\beta_{b}=1 if bb is a benign behavior; and βb=1\beta_{b}\!=\!-1 if bb is a misbehavior. Besides, if txreptx_{\mathrm{rep}} is verified as true and recorded in blockchain, all members within the informer group are rewarded with a reputation increase Δrep\Delta_{rep}, as well as a reputation decrease Δacc\Delta_{acc} for the accused entity. Otherwise, the expended report fee is in vain. Here, the evidence collection and misbehavior forensics are out of scope for this paper, and readers can refer to existing works [41, 42]. At each time slot, each full node uu’s reputation can be computed as:

un={uini,n=1;b=1NunβbΔb+eηun1,1<nN,\displaystyle\Im_{u}^{n}=\begin{cases}\Im_{u}^{\mathrm{ini}},&n=1;\\ \sum\nolimits_{b=1}^{N_{u}^{n}}{\beta_{b}\Delta_{b}}+e^{-\eta}\Im_{u}^{n-1},&1<n\leq N,\end{cases} (15)

where uini\Im_{u}^{\mathrm{ini}} is the initial reputation of full node uu, Nun{N_{u}^{n}} is the number of recorded behaviors of full node uu at nn-th time slot, and η>0\eta>0 is the exponential decay factor. The normalized reputation of each full node can be formulated as ¯un=(1+eun)1{{\overline{\Im}_{u}^{n}}}=\left(1+e^{-\Im_{u}^{n}}\right)^{-1}. The reputation values of full nodes are updated after each round of consensus according to the time-stamped behavior records stored in the blockchain.

IV-E Analysis of RescueChain

Let PbP_{b} denote the ratio of Byzantine validators in RescueChain, and PpP_{p} be the ratio of validators that are offline or partitioned in UDRNs. According to [40, 38], under partial synchrony assumption of the network, our RescueChain system is resilient up to Nfault=(Z1)/3N_{fault}=\lfloor\left(Z-1\right)/3\rfloor Byzantine validators. Next, we analyze the finality, network availability, and communication complexity of RescueChain in the following Theorems 1–3, respectively.

Theorem 1 (Finality): With assumptions of partial synchrony and Pb1/3P_{b}\leq 1/3, by taking multiple rounds at a given height to commit a block via two-phase voting with locking, our RescueChain can reach deterministic finality, where a deadlock that different blocks are locked by part of honest validators at different rounds can be avoided.

Proof:

We prove the theorem by contradiction. Suppose that RescueChain does not satisfy finality, where more than one block are committed at the same height. Without loss of generality, we consider two committed blocks \mathcal{B} and \mathcal{B}^{\prime} of height hh at rounds rr and rr^{\prime}, respectively. There exist the following two cases: r=rr^{\prime}=r and rrr^{\prime}\neq r. In the first case that r=rr^{\prime}=r, both blocks \mathcal{B} and \mathcal{B}^{\prime} receive over two thirds of precommits at round rr. Obviously, at least 1/31/3 Byzantine validators has precommitted for both blocks at round rr. In the other case that rrr^{\prime}\neq r, we assume r>rr^{\prime}>r without loss of generality. As over two thirds of validators precommit for \mathcal{B} at round rr, they get locked on \mathcal{B} and must prevote for \mathcal{B}. To precommit for block \mathcal{B}^{\prime}, it requires over two-thirds to prevote for \mathcal{B}^{\prime}, which means that at least 1/31/3 Byzantine validators violate the locking and unlocking rules. ∎

Theorem 2 (Availability): In face of network asynchrony, our system will never halt if 1PbPp(1Ppb)>2/31-P_{b}-P_{p}\cdot\left(1-P_{pb}\right)>2/3, where PpbP_{pb} is the ratio of offline or partitioned Byzantine validators to the total number of offline or partitioned validators.

Proof:

Let PnrP_{n}^{r} denote the ratio of remaining non-Byzantine validators that are not offline or partitioned in UDRNs. We have Pnr=1PbPp(1Ppb)P_{n}^{r}=1-P_{b}-P_{p}\cdot\left(1-P_{pb}\right). When Pnr>2/3P_{n}^{r}>2/3, as analyzed in Theorem 1, our system will never halt and the unambiguous consensus can be reached. ∎

Remark: If over a third of validators get offline or partitioned (i.e., Pp>1/3P_{p}>1/3), the network will halt since no block can receive two thirds or more votes from non-Byzantine validators (i.e., Pnr<2/3P_{n}^{r}<2/3). The unambiguous consensus can be restored until the network connection recovers.

Theorem 3 (Complexity): Compared with the 𝒪(Z3)\mathcal{O}(Z^{3}) view-change overhead in PBFT, the total communication complexity is reduced to 𝒪(Z2)\mathcal{O}(Z^{2}) in our RescueChain.

Proof:

In the propose phase, the new block proposal needs to be delivered to other Z1Z\!-\!1 validators, and the communication complexity is 𝒪(Z)\mathcal{O}(Z). In the two-phase voting phase, each validator needs to collect the voting messages from others to make progress, and the communication complexity is 𝒪(Z2)\mathcal{O}(Z^{2}). Therefore, the total communication complexity is 𝒪(Z2)\mathcal{O}(Z^{2}). ∎

Remark: Based on the proposed block partition method in the propose phase and the BLS signature aggregation in two-phase voting, the delivery latency of block proposal and the signature size in voting can be reduced, thereby alleviating the communication burden in consensus process.

Next, we give the security analysis of our RescueChain. Our paper mainly focuses on the blockchain-based solution to defend against attacks defined in Sect. III-E. The authentication and access control, which are also essential to ensure network security in UDRNs, are out of scope for this paper, and readers can refer to existing cryptographic approaches [43, 44].

  • Thanks to the decentralized ledgers and special data structure of blockchain (i.e., transactions are time-stamped, signed, and recorded in the Merkle tree structure in hash-chained blocks), the metadata of shared data can be immutably recorded on the blockchain to prevent data tampering attacks and guarantee the integrity and dependability of delivered information; meanwhile, the recorded node misbehaviors on blockchain ledgers can be transparently traced back to the source to ensure the traceability of misbehaviors. Besides, the defense against compromised insider UAVs is analyzed in the simulation section.

  • Since only registered nodes are authorized for network access, the air traffic scheduling of UAVs can be facilitated while preventing identity thefts. By dynamically altering the public keys in transactions via existing pseudonymity mechanisms in [45], the true identities can be hidden and remain unlinkable to preserve nodes’ identity privacy.

V Optimal Offloading in Static and Dynamic Games

In this section, the payoff functions and optimization problems of UAVs and ground vehicles in VFC-based offloading are first formulated. Then, we analyze the optimal offloading strategies of vehicles and the optimal payment strategies of UAVs under a static Stackelberg game model with one interaction and a dynamic Stackelberg game with repeated interactions, respectively.

V-A Optimization Problem

Payoff function of UAV. During VFC-based offloading process, to stimulate ground vehicles’ participation and high amount of resource sharing, each UAV jj needs to determine the payment vector 𝐲j={yi,j,k}k=1Kj{\bf{y}}_{j}=\left\{{y_{i,j,k}}\right\}_{k=1}^{K_{j}} for each task kk to compensate the cost of ground vehicles in contributing computation resources. Let 𝐱j={xi,j,k}k=1Kj{\bf{x}}_{j}=\left\{{x_{i,j,k}}\right\}_{k=1}^{K_{j}} denote the AoCR vector of ground vehicles that participate in the offloading process for UAV jj. The payoff function of UAV jj is defined as the difference between the satisfaction and its cost, i.e.,

πj(𝐱j,𝐲j)=k=1Kji=1Ijβi,j,k[Sj,k(xi,j,k)Cj,k(xi,j,k,yi,j,k)],{\pi_{j}}({\bf{x}}_{j},{\bf{y}}_{j})\!=\!\sum\limits_{k=1}^{{K_{j}}}{\sum\limits_{i=1}^{{I_{j}}}{\beta_{i,j,k}\left[S_{j,k}\left(x_{i,j,k}\right)\!-\!C_{j,k}\left(x_{i,j,k},y_{i,j,k}\right)\right]}},

(16)

where βi,j,k={0,1}\beta_{i,j,k}\!=\!\{0,1\} is a binary variable which equals to zero if vehicle ii does not participate in the offloading of task Γj,k\Gamma_{j,k}, otherwise it equals to one. Sj,k(xi,j,k)S_{j,k}\left(x_{i,j,k}\right) is the satisfaction function of UAV jj with the obtained AoCR xi,j,k{x_{i,j,k}} of each task Γj,k\Gamma_{j,k}. Cj,k(xi,j,k,yi,j,k)C_{j,k}\left(x_{i,j,k},y_{i,j,k}\right) is the cost function of UAV jj during offloading. Here, the logarithmic function, which is widely employed in resource allocation mechanisms [46], is used to model the satisfaction. We have

Sj,k(xi,j,k)=ρjαj,klog(1+xi,j,k),\displaystyle S_{j,k}\left(x_{i,j,k}\right)={\rho_{j}}{\alpha_{j,k}}\log\left({1+{x_{i,j,k}}}\right), (17)

where ρj{\rho_{j}} means the satisfaction parameter of UAV jj, and αj,k{\alpha_{j,k}} is the urgency degree of task Γj,k\Gamma_{j,k} defined in Eq. (5). The cost function Cj,k(xi,j,k,yi,j,k)C_{j,k}\left(x_{i,j,k},y_{i,j,k}\right) is composed of two parts: the payment for shared computation resource (i.e., λpxi,j,kyi,j,k{\lambda_{p}{x_{i,j,k}}{y_{i,j,k}}}) and the time delay to obtain the processing results (i.e., Φi,j,k\Phi_{i,j,k}). We have

Cj,k(xi,j,k,yi,j,k)=ϖpλpyi,j,kxi,j,k+(1ϖp)Φi,j,k,C_{j,k}\left(x_{i,j,k},y_{i,j,k}\right)={{\varpi_{p}}\lambda_{p}{y_{i,j,k}}{x_{i,j,k}}+\left({1-{\varpi_{p}}}\right)\Phi_{i,j,k}},

(18)

where ϖp{\varpi_{p}} means the weight factor, and λp\lambda_{p} is the price adjustment parameter. Next, we analyze the total delay and energy consumption in task offloading.

1) Delay analysis. The total offloading latency Φi,j,k\Phi_{i,j,k} for every task Γj,k\Gamma_{j,k} includes three parts: the A2G transmission time ti,j,kA2Gt_{i,j,k}^{\mathrm{A2G}}, the task execution time ti,j,kvfct_{i,j,k}^{\mathrm{vfc}}, and the G2A transmission time ti,j,kG2At_{i,j,k}^{\mathrm{G2A}}. The A2G transmission time is denoted as ti,j,kA2G=Dj,kγi,jA2Gt_{i,j,k}^{\mathrm{A2G}}=\frac{{{D_{j,k}}}}{{\gamma_{i,j}^{\mathrm{A2G}}}}. As UAVs and ground vehicles are highly mobile, vehicle ii may leave the communication range of UAV jj and drive into the coverage of another UAV jj^{\prime}, jjj^{\prime}\neq j. Without loss of generality, we denote tkt_{k} as the time slot that UAV jj begins to offload mission Γj,k\Gamma_{j,k}. Two cases of G2A transmission latency are considered as below:

ti,j,kG2A={μj,kDj,kγi,jG2A,ifij[tk+ti,j,kA2G+ti,j,kvfc];μj,kDj,kγi,jG2A+μj,kDj,kγj,jA2A,otherwise,\displaystyle t_{i,j,k}^{\mathrm{G2A}}=\begin{cases}\frac{{\mu_{j,k}{D_{j,k}}}}{{\gamma_{i,j}^{\mathrm{G2A}}}},&\resizebox{9043830}{}{${\rm{if}}~i\in{\mathcal{I}_{j}}\left[{t_{k}}+t_{i,j,k}^{\mathrm{A2G}}+t_{i,j,k}^{\mathrm{vfc}}\right]\!;\!$}\\[5.12128pt] \frac{{\mu_{j,k}{D_{j,k}}}}{{\gamma_{i,{j^{\prime}}}^{\mathrm{G2A}}}}+\frac{{\mu_{j,k}{D_{j,k}}}}{{\gamma_{{j^{\prime}},j}^{\mathrm{A2A}}}},&{\rm{otherwise}},\end{cases} (19)

where γj,jA2A{\gamma_{{j^{\prime}},j}^{\mathrm{A2A}}} is the A2A transmission rate between UAV jj and UAV jj^{\prime}. Hence, we have Φi,j,k=ti,j,kvfc+ti,j,kA2G+ti,j,kG2A\Phi_{i,j,k}=t_{i,j,k}^{\mathrm{vfc}}+t_{i,j,k}^{\mathrm{A2G}}+t_{i,j,k}^{\mathrm{G2A}}.

2) Energy consumption analysis. The energy consumption of UAV jj during task offloading includes two parts: the A2G transmission energy and the flying energy. UAV jj’s energy consumption during A2G transmission is denoted as Ei,j,kA2G=PjTXti,j,kA2GE_{i,j,k}^{\mathrm{A2G}}=P_{j}^{\mathrm{TX}}\,t_{i,j,k}^{\mathrm{A2G}}. The flying energy of UAV jj can be attained as Ei,j,kfly=PjflyΦi,j,kE_{i,j,k}^{\mathrm{fly}}=P_{j}^{\mathrm{fly}}\Phi_{i,j,k}.

The target of each UAV j𝒥j\in\mathcal{J} is to maximize its payoff function by deciding the optimal payment strategy 𝐲j{{\bf{y}}_{j}}, whereby its optimization problem P1P1 is formulated as:

max𝐲jπj(𝐱j,𝐲j)\displaystyle\mathop{\max}\limits_{{\bf{y}}_{j}}\,\pi_{j}\left(\mathbf{x}_{j},\mathbf{y}_{j}\right) =k=1Kji=1Ijπj(xi,j,k,yi,j,k)\displaystyle=\sum\limits_{k=1}^{K_{j}}{\sum\limits_{i=1}^{I_{j}}{\pi_{j}\left({x}_{i,j,k},{y}_{i,j,k}\right)}}
=k=1Kji=1Ijβi,j,k{ρjαj,klog(1+xi,j,k)\displaystyle=\sum\limits_{k=1}^{K_{j}}{\sum\limits_{i=1}^{I_{j}}{\beta_{i,j,k}\left\{\rho_{j}\alpha_{j,k}\log\left(1+x_{i,j,k}\right)\right.}}
[ϖpλpyi,j,kxi,j,k+(ϖp)Φi,j,k]}\displaystyle\left.\!-\!\left[\varpi_{p}\lambda_{p}y_{i,j,k}x_{i,j,k}+\left(1\!-\!\varpi_{p}\right)\Phi_{i,j,k}\right]\right\} (20)
s.t.\displaystyle{\rm{s.t.}} ti,j,kvfc+ti,j,kA2G+ti,j,kG2ATj,kmax,\displaystyle t_{i,j,k}^{\mathrm{vfc}}+t_{i,j,k}^{\mathrm{A2G}}+t_{i,j,k}^{\mathrm{G2A}}\leq T_{j,k}^{\max},\hfill (21)
s.t.\displaystyle{\rm{s.t.}} Ej[tk]Ei,j,kflyEi,j,kA2GEjmin,\displaystyle{E_{j}}[{t_{k}}]-E_{i,j,k}^{\mathrm{fly}}-E_{i,j,k}^{\mathrm{A2G}}\geq E_{j}^{\min},\hfill (22)
s.t.\displaystyle{\rm{s.t.}} 0yi,j,kyj,kmax.\displaystyle 0\leq{y_{i,j,k}}\leq y_{j,k}^{\max}.\hfill (23)

Constraint (21) indicates that the total time delay of each task Γj,k\Gamma_{j,k} should be less than or equal to the task TTL Tj,kmaxT_{j,k}^{\max}. Constraint (22) implies that the remaining battery energy of UAV jj after task execution should not less than its minimum energy reserve EjminE_{j}^{\min}. Constraint (23) means that the payment should be constrained by the lower bound 00 and the upper bound yj,kmaxy_{j,k}^{\max}.

Payoff function of ground vehicle. In VFC-based task offloading, the payoff function of each ground vehicle is associated with the payment provided by the UAV and its cost of sharing computing resources to the UAV. As such, the payoff function of ground vehicle ii, ij\forall i\in\mathcal{I}_{j} is the difference between the payment and the cost in sharing computing resource to UAV jj, i.e.,

πi(xi,j,k,yi,j,k)\displaystyle{\pi_{i}}({x_{i,j,k}},y_{i,j,k}) =βi,j,k[λpyi,j,kxi,j,kΨi(xi,j,k)],\displaystyle=\beta_{i,j,k}\left[\lambda_{p}y_{i,j,k}x_{i,j,k}-\Psi_{i}\left({x_{i,j,k}}\right)\right], (24)

where Ψi(xi,j,k)\Psi_{i}\left({x_{i,j,k}}\right) is the cost function, which contains two parts: the cost for sharing xi,j,k{x_{i,j,k}} amount of computation resources and the energy consumption in data computing and transmission. According to the quadratic cost model [47], the cost of resources sharing can be formulated as a quadratic function of the contributed resources. As such, Ψi(xi,j,k)\Psi_{i}\left({x_{i,j,k}}\right) can be defined as

Ψi(xi,j,k)=λcψi(xi,j,k)2+λe(Ei,j,kvfc+PiTXti,j,kG2A).\displaystyle\Psi_{i}\left({x_{i,j,k}}\right)=\lambda_{c}{\psi_{i}}\left({x_{i,j,k}}\right)^{2}\!+\!{\lambda_{e}}\left(E_{i,j,k}^{\mathrm{vfc}}+P_{i}^{\mathrm{TX}}\,t_{i,j,k}^{\mathrm{G2A}}\right). (25)

The first term in formula (25) represents the quadratic cost in sharing xi,j,k{x_{i,j,k}} amount of computation resources, and the second term indicates the total energy consumption in computing (i.e., Ei,j,kvfcE_{i,j,k}^{\mathrm{vfc}}) and data transmission (i.e., PiTXti,j,kG2AP_{i}^{\mathrm{TX}}t_{i,j,k}^{\mathrm{G2A}}). Here, λc\lambda_{c} and λe{\lambda_{e}} are positive adjustment parameters. ψi{\psi_{i}} is the unit cost of vehicle ii’s computation resource.

The target of each vehicle iji\in\mathcal{I}_{j} is to maximize its payoff function by deciding the optimal AoCR strategy 𝐱j{{\bf{x}}_{j}}, whereby its optimization problem P2P2 is formulated as:

maxxi,j,k\displaystyle\mathop{\max}\limits_{x_{i,j,k}}\, πi(xi,j,k,yi,j,k)=βi,j,k{λpyi,j,kxi,j,k\displaystyle{\pi_{i}}({x_{i,j,k}},y_{i,j,k})=\beta_{i,j,k}\left\{\lambda_{p}y_{i,j,k}x_{i,j,k}\right.
[λcψi(xi,j,k)2+λe(Ei,j,kvfc+PiTXti,j,kG2A)]}\displaystyle\left.-\left[\lambda_{c}{\psi_{i}}\left({x_{i,j,k}}\right)^{2}\!+\!{\lambda_{e}}\left(E_{i,j,k}^{\mathrm{vfc}}+P_{i}^{\mathrm{TX}}\;t_{i,j,k}^{\mathrm{G2A}}\right)\right]\right\} (26)
s.t.0xi,j,kximax.\displaystyle{\rm{s.t.}}~0\leq{x_{i,j,k}}\leq x_{i}^{\max}. (27)

Constraint (27) means that the AoCR of vehicle ii in executing task Γj,k\Gamma_{j,k} should be constrained by the lower bound 00 and the upper bound ximaxx_{i}^{\max}.

V-B Static Stackelberg Game Analysis

During VFC-based offloading process, both UAVs and ground vehicles are rational and selfish and they are aimed to maximize their own profits [45]. Specifically, the UAV intends to enjoy high amount of computing resource with low payments, while each ground vehicle hopes the payment can be as high as possible. Based on the existing work [46], the competitive interactions between a UAV and ground vehicles can be formulated as a one-leader and multiple-followers Stackelberg game, i.e.,

𝔾={(j,1,2,,Ij);(𝐱j,𝐲j);(πj,π1iIj)}.\displaystyle\mathbb{G}=\left\{{\left({j,1,2,\cdots,I_{j}}\right);\left(\mathbf{x}_{j},\mathbf{y}_{j}\right);\left({{\pi_{j}},{\pi_{1\leq i\leq{{I}_{j}}}}}\right)}\right\}. (28)

In the two-stage game 𝔾\mathbb{G}, the UAV jj first determines its optimal payment strategy 𝐲j\mathbf{y}_{j}^{*} in stage I to optimize its payoff, and then each vehicle iji\in\mathcal{I}_{j} selects its optimal AoCR strategy xi,j,k{x}_{i,j,k}^{*} in stage II to maximize its benefit. Here, 𝐲j=(yi,j,k)1kKj\mathbf{y}_{j}^{*}=\left({y}_{i,j,k}^{*}\right)_{1\leq k\leq{K_{j}}} is a solution of problem P1P1, and 𝐱j=(xi,j,k)1iIj\mathbf{x}_{j}^{*}=\left({x}_{i,j,k}^{*}\right)_{1\leq i\leq{I_{j}}} is a solution of problem P2P2. The solution of game 𝔾\mathbb{G} is the Stackelberg equilibrium (SE) defined in the following Definition 1, which means neither the UAV nor the ground vehicles can improve their payoffs by deviating it.

Definition 1

The point (𝐱j,𝐲j)\left(\mathbf{x}_{j}^{*},\mathbf{y}_{j}^{*}\right) is a SE of the proposed game 𝔾\mathbb{G}, if the following conditions hold:

πj(𝐱j,𝐲j)πj(𝐱j,𝐲j),\displaystyle~~~~\ \pi_{j}\left(\mathbf{x}_{j}^{*},\mathbf{y}_{j}^{*}\right)\geq\pi_{j}\left(\mathbf{x}_{j}^{*},\mathbf{y}_{j}\right), (29)
πi(xi,j,k,yi,j,k)πi(xi,j,k,yi,j,k),ij.\displaystyle{\pi_{i}}({x_{i,j,k}^{*}},y_{i,j,k}^{*})\geq{\pi_{i}}({x_{i,j,k}},y_{i,j,k}^{*}),\forall i\in{{\cal I}_{j}}. (30)

In the static Stackelberg game with one interaction, the parameters of the game are public known by all players. To find the SE of the static Stackelberg game, the backward induction method is employed. Specifically, we first investigate the decision process of every follower (i.e., vehicle) in stage II to attain its optimal AoCR strategy. Then, we analyze the optimal payment strategy of the leader (i.e., UAV) in stage I. Every ground vehicle iji\in\mathcal{I}_{j} determines its optimal AoCR strategy xi,j,k{x_{i,j,k}^{*}} to maximize its payoff πi(xi,j,k,yi,j,k){\pi_{i}}({x_{i,j,k}},y_{i,j,k}) according to the following theorem.

Theorem 4: The optimal AoCR strategies of vehicle ii in performing task kk of UAV jj is

xi,j,k={ximax,2λcψiximaxλpyi,j,kyj,kmax;λpyi,j,k2λcψi,0<yi,j,k<2λcψiximaxλp;0,yi,j,k=0.\displaystyle x_{i,j,k}^{*}=\left\{\begin{array}[]{ll}x_{i}^{\max},&\frac{2\lambda_{c}\psi_{i}x_{i}^{\max}}{\lambda_{p}}\leq y_{i,j,k}\leq y_{j,k}^{\max};\\[4.26773pt] \frac{\lambda_{p}y_{i,j,k}}{2\lambda_{c}\psi_{i}},&0<y_{i,j,k}<\frac{2\lambda_{c}\psi_{i}x_{i}^{\max}}{\lambda_{p}};\\[3.41418pt] 0,&y_{i,j,k}=0.\\ \end{array}\right.
Proof:

Refer to Appendix A

Given the optimal AoCR strategy of vehicles in Eq. (V-B), UAV jj determines its optimal payment strategy 𝐲j\mathbf{y}_{j}^{*} for tasks to maximize its payoff πj(𝐱j,𝐲j)\pi_{j}\left(\mathbf{x}_{j}^{*},\mathbf{y}_{j}\right) based on the following theorem.

Theorem 5: The optimal payment strategy of UAV jj on task kk for vehicle ii is

yi,j,k={2λcψiximaxλp,Θi,j,k0;Ωi,j,kϖpλcψiϖpλp,Θi,j,k<0.\displaystyle y_{i,j,k}^{*}=\left\{\begin{array}[]{ll}\frac{2\lambda_{c}\psi_{i}x_{i}^{\max}}{\lambda_{p}},&\Theta_{i,j,k}\geq 0;\\[2.84544pt] \frac{\sqrt{\Omega_{i,j,k}}-\varpi_{p}\lambda_{c}\psi_{i}}{\varpi_{p}\lambda_{p}},&\Theta_{i,j,k}<0.\\ \end{array}\right.

where

Θi,j,k\displaystyle\Theta_{i,j,k} =ρjαj,k4ϖpλcψiximax(1+ximax),\displaystyle=\rho_{j}\alpha_{j,k}-4\varpi_{p}\lambda_{c}\psi_{i}x_{i}^{\max}\left(1+x_{i}^{\max}\right), (36)
Ωi,j,k\displaystyle\Omega_{i,j,k} =ϖp2λc2ψi2+ϖpλcψiρjαj,k.\displaystyle=\varpi_{p}^{2}\lambda_{c}^{2}\psi_{i}^{2}+\varpi_{p}\lambda_{c}\psi_{i}\rho_{j}\alpha_{j,k}. (37)
Proof:

Refer to Appendix B

Based on above analysis, the SE of the Stackelberg game 𝔾\mathbb{G} can be derived as:

(xi,j,k,yi,j,k)\displaystyle\left(x_{i,j,k}^{*},y_{i,j,k}^{*}\right)

={(ximax,2λcψiximaxλp),Θi,j,k0;(Ωi,j,kϖpλcψi2λcψi,Ωi,j,kϖpλcψiϖpλp),Θi,j,k<0.=\left\{\begin{array}[]{l}\left(x_{i}^{\max},\frac{2\lambda_{c}\psi_{i}x_{i}^{\max}}{\lambda_{p}}\right),~~~~~~~~~~~~~~~~~~~~~\Theta_{i,j,k}\geq 0;\\[4.26773pt] \left(\frac{\sqrt{\Omega_{i,j,k}}-\varpi_{p}\lambda_{c}\psi_{i}}{2\lambda_{c}\psi_{i}},\frac{\sqrt{\Omega_{i,j,k}}-\varpi_{p}\lambda_{c}\psi_{i}}{\varpi_{p}\lambda_{p}}\right),~\Theta_{i,j,k}<0.\end{array}\right.

Remark: In the static Stackelberg game-based offloading process where the parameters of the game are public knowledge, both UAVs and ground vehicles can apply the above SE to decide their optimal strategies to gain maximized payoffs.

V-C DQN-Based Offloading for Dynamic Stackelberg Game

In a realistic VFC-based offloading application, the parameters of user payoff model (e.g., satisfaction, cost, and network parameters) are usually private and the parameters of network model are time-varying [7]. As a consequence, these parameters cannot be readily available for all participants. Both UAVs and ground vehicles can conduct multiple interactions and employ reinforcement learning technologies to find the optimal payment and AoCR strategies via trials, respectively, without fully knowing the accurate parameters of the network model and payoff model. The repeated sequential interactions between a UAV and ground vehicles can be formulated as a dynamic Stackelberg game.

DQN-based payment strategy of UAV. A high payment of UAV for can decrease its immediate payoff, but it stimulates more vehicles’ participation and their higher AoCR contribution in the future. Therefore, the current payment strategy of the UAV influences the long-term benefits. The payment decision of UAV jj in the dynamic game can be formulated as a finite Markov decision process (MDP) [4], and reinforcement learning algorithms can be exploited to achieve the optimal payment strategy. In particular, UAV jj observes the previous AoCR sequences of corresponding ground vehicles and formulates the current system state. The state of UAV jj at nn-th time slot (or interaction) is 𝐬j(n)={si,j,k(n)}k=1Kj\mathbf{s}_{j}^{(n)}=\{s_{i,j,k}^{(n)}\}_{k=1}^{K_{j}}, and we have 𝐬j(n)=𝐱j(n1)\mathbf{s}_{j}^{(n)}=\mathbf{x}_{j}^{(n-1)}. The action of UAV jj at time slot nn is 𝐲j(n)\mathbf{y}_{j}^{(n)}. For simplicity, the feasible payments of UAV jj are uniformly discretized into WW levels, i.e., yi,j,k(n)𝒲={wW1yj,kmax}0wW1y_{i,j,k}^{(n)}\in\mathcal{W}=\{\frac{w}{W-1}\cdot y_{j,k}^{\max}\}_{0\leq w\leq W-1}. Let Q(𝐬j(n),𝐲j(n))Q(\mathbf{s}_{j}^{(n)},\mathbf{y}_{j}^{(n)}) denote the Q-function of UAV jj with state-action pair (𝐬j(n),𝐲j(n))(\mathbf{s}_{j}^{(n)},\mathbf{y}_{j}^{(n)}), which means the expected long-term discounted reward. Note the size of state space of UAV jj, i.e., (Ij)V(I_{j})^{V} [48], increases with both the number of involved vehicles (i.e., IjI_{j}). Due to the presence of large state space, traditional Q-learning approaches may suffer the curse of dimensionality and cause slow learning speed and long convergence time.

In the DQN-based pricing process, to address the curse of dimensionality and accelerate the convergence rate of Q-learning, the convolutional neural network (CNN) is exploited for efficient state space compression and Q-value estimation. Specifically, the output of the CNN is the Q-value for each payment level, i.e.,

Q(𝐬j(n),𝐲j(n))=𝔼𝐬j(n+1)[πj(𝐱j(n),𝐲j(n))+ςjmax𝐲jQ(𝐬j(n+1),𝐲j)],Q\left(\mathbf{s}_{j}^{(n)},\mathbf{y}_{j}^{(n)}\right)\!=\!\mathbb{E}_{\mathbf{s}_{j}^{(n+1)}}\left[\pi_{j}\left(\mathbf{x}_{j}^{(n)},\mathbf{y}_{j}^{(n)}\right)\!+\!\varsigma_{j}\underset{\mathbf{y}_{j}^{\prime}}{\max}\,Q\left(\mathbf{s}_{j}^{(n+1)},\mathbf{y}_{j}^{\prime}\right)\right]\!,\!

(40)

where ςj[0,1]\varsigma_{j}\in[0,1] is the discount factor implying the myopic view of UAV jj about the future return. 𝐬j(n+1)\mathbf{s}_{j}^{(n+1)} is the new state transited from 𝐬j(n)\mathbf{s}_{j}^{(n)} with action 𝐲j(n)\mathbf{y}_{j}^{(n)}. The payoff πj(𝐱j(n),𝐲j(n))\pi_{j}(\mathbf{x}_{j}^{(n)},\mathbf{y}_{j}^{(n)}) represents the immediate reward of UAV jj, which is rewritten as πj(n)\pi_{j}^{(n)}. In DQN, the value of Q-function is estimated by a nonlinear neural network function approximator realized by CNN, which includes 22 convolutional (Conv) layers and 22 fully connected (FC) layers. The first Conv layer involves 2020 filters and each of them has size 3×33\times 3 and stride 11. The second Conv layer contains 4040 filters and each of them has size 2×22\times 2 and stride 11. The widely used rectified linear unit (ReLU) [49] is adopted as the activation function in both Conv layers. The first FC layer uses 180180 ReLUs, while the second FC layer has WW ReLUs for each participating vehicle. The architecture parameters in CNN are summarized in Table II.

TABLE II: Architecture Parameters in the CNN of UAV and ground vehicle
Layer Input Filter Size Stride # Filters Activation Output
Conv 1 6×66\times 6 3×33\times 3 1 20 ReLU 4×4×204\times 4\times 20
Conv 2 4×4×204\times 4\times 20 2×22\times 2 1 40 ReLU 3×3×403\times 3\times 40
FC 1 360360 // // 180 ReLU 180180
FC 2
of UAV
180180 // // WW ReLU WW
FC 2
of Vehicle
180180 // // VV ReLU VV

To efficiently learn from past experiences and smooth the learning, the state sequence 𝜸j(n){\bm{\gamma}}_{j}^{(n)} is constructed for UAV jj in DQN, which consists of the current state and prior A1A_{1} states, i.e.,

𝜸j(n)={𝐬j(nA1),𝐬j(nA1+1),,𝐬j(n)}.\displaystyle{\bm{\gamma}}_{j}^{(n)}=\left\{\mathbf{s}_{j}^{(n-A_{1})},\mathbf{s}_{j}^{(n-A_{1}+1)},\cdots,\mathbf{s}_{j}^{(n)}\right\}. (41)

The state sequence 𝜸(n){\bm{\gamma}}^{(n)} is then reshaped into a 6×66\times 6 matrix and input to the CNN. We rewrite πj(𝐱j(n),𝐲j(n))\pi_{j}(\mathbf{x}_{j}^{(n)},\mathbf{y}_{j}^{(n)}) as πj(n)\pi_{j}^{(n)} for simplicity. The interaction experience that UAV jj learned at time slot nn is denoted as

ϕj(n)={𝜸j(n),𝐲j(n),πj(n),𝜸j(n+1)},\displaystyle{\bm{\phi}}_{j}^{(n)}=\left\{{\bm{\gamma}}_{j}^{(n)},\mathbf{y}_{j}^{(n)},\pi_{j}^{(n)},{\bm{\gamma}}_{j}^{(n+1)}\right\}, (42)

and is stored into a replay memory, denoted as Ξj={ϕj(nd+1),,ϕj(n)}\Xi_{j}=\left\{{\bm{\phi}}_{j}^{(n-d+1)},\cdots,{\bm{\phi}}_{j}^{(n)}\right\}. Here, only the latest dd related experiences are stored to save memory space and restrain UAV jj from focusing on the immediate interaction experience. Let 𝜽(n){\bm{\theta}}^{(n)} denote the filter weights of UAV jj in the CNN. Based on the experience replay method, 𝜽(n){\bm{\theta}}^{(n)} is updated for D1D_{1} times at every time slot by randomly selecting an experience from Ξj\Xi_{j} to minimize the mean-squared error of the target optimal Q-function. According to [49], the loss function can be defined as:

(𝜽(n))=𝔼ϕj(n)[(πj(n)+ςjmax𝐲jQ(𝜸j(n+1),𝐲j,𝜽(n1))Q(γj(n),𝐲j(n),𝜽(n)))2].\mathscr{L}({\bm{\theta}}^{(n)})\!=\!\mathbb{E}_{{\bm{\phi}}_{j}^{(n)}}\left[\left(\pi_{j}^{(n)}\!+\!\varsigma_{j}\underset{\mathbf{y}_{j}^{\prime}}{\max}\,Q({\bm{\gamma}}_{j}^{(n+1)},\mathbf{y}_{j}^{\prime};{\bm{\theta}}^{(n-1)})\!-\!Q(\gamma_{j}^{(n)},\mathbf{y}_{j}^{(n)};{\bm{\theta}}^{(n)})\right)^{2}\right]\!.\!

(43)

The stochastic gradient descent (SGD) algorithm is adopted to update 𝜽(n){\bm{\theta}}^{(n)} for alleviated computational cost in learning via mini-batch updates. Based on the CNN model and current system state 𝐬j(n)\mathbf{s}_{j}^{(n)}, UAV jj applies the ϵ\epsilon-greedy policy to choose its action 𝐲j(n)\mathbf{y}_{j}^{(n)} for a better tradeoff between exploration and exploitation. More specifically, the greedy payment vector that maximizes its Q function is chosen with a high probability ϵj\epsilon_{j}, and other actions are randomly selected with a very small chance 1ϵj1-\epsilon_{j}, i.e.,

Pr[𝐲j(n)=𝐲j]={ϵj,𝐲j=argmax𝐲jQ(𝐬j(n),𝐲j);ϵj,otherwise.\displaystyle\Pr\left[\mathbf{y}_{j}^{(n)}\!=\!\mathbf{y}_{j}^{*}\right]\!=\!\left\{\begin{array}[]{ll}\epsilon_{j},~~~~\;\mathbf{y}_{j}^{*}\!=\!\arg{\max\limits_{\mathbf{y}_{j}^{\prime}}}\,Q(\mathbf{s}_{j}^{(n)},{\mathbf{y}_{j}^{\prime}});\\ {1\!-\!\epsilon_{j}},~{\rm{otherwise}}.\end{array}\right.

The detailed procedure of DQN-based payment for UAV jj is shown in Algorithm 1.

Algorithm 1 DQN-Based Optimal Payment Strategy
1: Initialize: θ(0){\bm{\theta}}^{(0)}, ςj\varsigma_{j}, Ξj=\Xi_{j}=\emptyset, 𝐬j(0)\mathbf{s}_{j}^{(0)}, A1A_{1}, D1D_{1}, 𝒲\mathcal{W}
2: for n=1,2,,Nn=1,2,\cdots,N do
3:    Set 𝐬j(n)=𝐱j(n1)\mathbf{s}_{j}^{(n)}=\mathbf{x}_{j}^{(n-1)}.
4:    if nA1n\leq A_{1} then
5:     Select 𝐲j(n)𝒲Ij\mathbf{y}_{j}^{(n)}\in\mathcal{W}^{I_{j}} at random.
6:    else
7:     Build 𝜸j(n)={𝐬j(nA1),𝐬j(nA1+1),,𝐬j(n)}{\bm{\gamma}}_{j}^{(n)}=\{\mathbf{s}_{j}^{(n-A_{1})},\mathbf{s}_{j}^{(n-A_{1}+1)},\cdots,\mathbf{s}_{j}^{(n)}\}.
8:     Obtain CNN output Q(𝐬j(n),𝐲j)Q(\mathbf{s}_{j}^{(n)},\mathbf{y}_{j}), 𝐲j𝒲Ij\forall\mathbf{y}_{j}\in\mathcal{W}^{I_{j}}, as Q-values with input 𝜸j(n){\bm{\gamma}}_{j}^{(n)} and 𝜽(n){\bm{\theta}}^{(n)}.
9:     Select 𝐲j(n)\mathbf{y}_{j}^{(n)} via ϵ\epsilon-greedy policy in Eq. (V-C) and send yi,j,k(n){y}_{i,j,k}^{(n)} to vehicle iji\in\mathcal{I}_{j}.
10:    end if
11:    Observe and evaluate the AoCR 𝐱j(n){\bf{x}}_{j}^{(n)}.
12:    Compute payoff πj(𝐱j(n),𝐲j(n))\pi_{j}(\mathbf{x}_{j}^{(n)},\mathbf{y}_{j}^{(n)}) via Eq. (V-A).
13:    Update Ξj{𝜸j(n),𝐲j(n),πj(n),𝜸j(n+1)}Ξj\Xi_{j}\leftarrow\{{\bm{\gamma}}_{j}^{(n)},\mathbf{y}_{j}^{(n)},\pi_{j}^{(n)},{\bm{\gamma}}_{j}^{(n+1)}\}\cup\Xi_{j}.
14:    for d=1,2,,D1d=1,2,\cdots,D_{1} do
15:     Select {𝜸j(d),𝐲j(d),πj(d),𝜸j(d+1)}Ξj\{{\bm{\gamma}}_{j}^{(d)},\mathbf{y}_{j}^{(d)},\pi_{j}^{(d)},{\bm{\gamma}}_{j}^{(d+1)}\}\in\Xi_{j} at random.
16:     Update 𝜽(n){\bm{\theta}}^{(n)} via mini-batch SGD.
17:    end for
18: end for
Algorithm 2 DQN-Based Optimal AoCR Strategy
1: Initialize: θ~(0)\tilde{{\bm{\theta}}}^{(0)}, ςi\varsigma_{i}, Ξi=\Xi_{i}=\emptyset, s~i,j,k(0){\tilde{s}}_{i,j,k}^{(0)}, A2A_{2}, D2D_{2}, 𝒱\mathcal{V}
2: for n=1,2,,Nn=1,2,\cdots,N do
3:    for i=1,2,,Iji=1,2,\cdots,I_{j} do
4:     Set s~i,j,k(n)=yi,j,k(n1){\tilde{s}}_{i,j,k}^{(n)}={y}_{i,j,k}^{(n-1)}.
5:     if nA1n\leq A_{1} then
6:      Select xi,j,k(n)𝒱{x}_{i,j,k}^{(n)}\in\mathcal{V} at random.
7:     else
8:      Build 𝜸i(n)={s~i,j,k(nA2),s~i,j,k(nA2+1),,s~i,j,k(n)}{\bm{\gamma}}_{i}^{(n)}=\{{\tilde{s}}_{i,j,k}^{(n-A_{2})},{\tilde{s}}_{i,j,k}^{(n-A_{2}+1)},\cdots,{\tilde{s}}_{i,j,k}^{(n)}\}.
9:      Obtain CNN output Q~(s~i,j,k(n),xi,j,k)\tilde{Q}({\tilde{s}}_{i,j,k}^{(n)},{x}_{i,j,k}), xi,j,k𝒱\forall{x}_{i,j,k}\in\mathcal{V}, as Q-values with input 𝜸i(n){\bm{\gamma}}_{i}^{(n)} and 𝜽~(n)\tilde{{\bm{\theta}}}^{(n)}.
10:      Select xi,j,k(n){x}_{i,j,k}^{(n)} via ϵ\epsilon-greedy policy in Eq. (V-C) and send it to UAV jj.
11:     end if
12:     Observe and evaluate the payment yi,j,k(n){y}_{i,j,k}^{(n)}.
13:     Compute payoff πi(xi,j,k(n),yi,j,k(n))\pi_{i}({x}_{i,j,k}^{(n)},{y}_{i,j,k}^{(n)}) via Eq. (V-A).
14:     Update Ξi{𝜸i(n),xi,j,k(n),πi(n),𝜸i(n+1)}Ξi\Xi_{i}\leftarrow\{{\bm{\gamma}}_{i}^{(n)},{x}_{i,j,k}^{(n)},\pi_{i}^{(n)},{\bm{\gamma}}_{i}^{(n+1)}\}\cup\Xi_{i}.
15:     for d=1,2,,D2d=1,2,\cdots,D_{2} do
16:      Select {𝜸i(d),xi,j,k(d),πi(d),𝜸i(d+1)}Ξi\{{\bm{\gamma}}_{i}^{(d)},{x}_{i,j,k}^{(d)},\pi_{i}^{(d)},{\bm{\gamma}}_{i}^{(d+1)}\}\in\Xi_{i} at random.
17:      Update 𝜽~(n)\tilde{{\bm{\theta}}}^{(n)} via mini-batch SGD.
18:     end for
19:    end for
20: end for

DQN-based AoCR strategy of ground vehicle. The AoCR decision of each ground vehicle can be formulated as an MDP with finite states. As it is unrealistic for vehicles to have full knowledge of the network model and the UAV’s private payoff model in time, each vehicle ii can utilize the DQN method to obtain its optimal AoCR strategy in the dynamic game. In the DQN-based AoCR decision-making process, the payment of UAV jj at the last time slot is used by vehicle ii as the current state to decide the current AoCR action. The system state of vehicle ii at time slot nn is s~i,j,k(n)=yi,j,k(n1){\tilde{s}}_{i,j,k}^{(n)}=y_{i,j,k}^{(n-1)}, and its action at time slot nn is xi,j,k(n){x}_{i,j,k}^{(n)}. For simplicity, the feasible AoCRs of vehicle ii are discretized into VV levels, i.e., xi,j,k(n)𝒱={vV1ximax}0vV1x_{i,j,k}^{(n)}\in\mathcal{V}=\{\frac{v}{V-1}\cdot x_{i}^{\max}\}_{0\leq v\leq V-1}. The Q-function of vehicle ii for every state-action pair (s~i,j,k(n),xi,j,k(n))({\tilde{s}}_{i,j,k}^{(n)},{x}_{i,j,k}^{(n)}) is denoted by Q~(s~i,j,k(n),xi,j,k(n))\tilde{Q}({\tilde{s}}_{i,j,k}^{(n)},{x}_{i,j,k}^{(n)}) and is estimated by the CNN, i.e.,

Q~(s~i,j,k(n),xi,j,k(n))=𝔼s~i,j,k(n+1)[πi(n)+ςimaxxi,j,kQ~(s~i,j,k(n+1),xi,j,k)],\tilde{Q}\left({\tilde{s}}_{i,j,k}^{(n)},{x}_{i,j,k}^{(n)}\right)\!=\!\mathbb{E}_{{\tilde{s}}_{i,j,k}^{(n+1)}}\left[\pi_{i}^{(n)}\!+\!\varsigma_{i}\underset{{x}_{i,j,k}^{\prime}}{\max}\,\tilde{Q}\left({\tilde{s}}_{i,j,k}^{(n+1)},{x}_{i,j,k}^{\prime}\right)\right]\!,\!

(46)

where ςi[0,1]\varsigma_{i}\in[0,1] is the discount factor implying the myopic view of vehicle ii about the future return. s~i,j,k(n+1){\tilde{s}}_{i,j,k}^{(n+1)} is the new state transited from s~i,j,k(n){\tilde{s}}_{i,j,k}^{(n)} with action xi,j,k(n){x}_{i,j,k}^{(n)}. The payoff πi(xi,j,k(n),yi,j,k(n))\pi_{i}({x}_{i,j,k}^{(n)},{y}_{i,j,k}^{(n)}) means the immediate reward of vehicle ii, and is rewritten as πi(n)\pi_{i}^{(n)} for simplicity. As shown in Table II, the parameters of CNN for each vehicle are similar to those of UAV jj except that this CNN has VV outputs.

Each vehicle ii builds its state sequence 𝜸i(n){\bm{\gamma}}_{i}^{(n)} in DQN and reshapes it into a 6×66\times 6 matrix as the input of the CNN. The state sequence includes the recent A2+1A_{2}+1 states, i.e.,

𝜸i(n)={s~i,j,k(nA2),s~i,j,k(nA2+1),,s~i,j,k(n)}.\displaystyle{\bm{\gamma}}_{i}^{(n)}=\left\{{\tilde{s}}_{i,j,k}^{(n-A_{2})},{\tilde{s}}_{i,j,k}^{(n-A_{2}+1)},\cdots,{\tilde{s}}_{i,j,k}^{(n)}\right\}. (47)

Then, each vehicle ii stores its previous interaction experiences into a replay memory Ξi={ϕi(nd+1),,ϕi(n)}\Xi_{i}=\left\{{\bm{\phi}}_{i}^{(n-d+1)},\cdots,{\bm{\phi}}_{i}^{(n)}\right\}, where ϕi(n){\bm{\phi}}_{i}^{(n)} is the experience that vehicle ii has acquired at time slot nn, i.e.,

ϕi(n)={𝜸i(n),xi,j,k(n),πi(n),𝜸i(n+1)}.\displaystyle{\bm{\phi}}_{i}^{(n)}=\left\{{\bm{\gamma}}_{i}^{(n)},{x}_{i,j,k}^{(n)},\,\pi_{i}^{(n)},{\bm{\gamma}}_{i}^{(n+1)}\right\}. (48)

The parameters of the Q-network of vehicle ii, denoted by 𝜽~(n)\tilde{{\bm{\theta}}}^{(n)}, is updated by SGD based on the loss function, which is similar to Eq. (43). The SGD training process is repeated D2D_{2} times to update 𝜽~(n)\tilde{{\bm{\theta}}}^{(n)} at each time slot based on the randomly chosen experience from the replay memory, i.e., ϕi(d)Ξi{\bm{\phi}}_{i}^{(d)}\in\Xi_{i}.

To avoid staying at the local optimum, the ϵ\epsilon-greedy policy is exploited by each vehicle ii to opt its action xi,j,k(n){x}_{i,j,k}^{(n)}, i.e.,

Pr[xi,j,k(n)=xi,j,k]={ϵi,xi,j,k=argmaxxi,j,kQ~(s~i,j,k(n),xi,j,k);ϵi,otherwise.\Pr\left[{x}_{i,j,k}^{(n)}\!=\!{x}_{i,j,k}^{*}\right]\!=\!\left\{\begin{array}[]{ll}\epsilon_{i},~~~~{x}_{i,j,k}^{*}\!=\!\arg{\max\limits_{{x}_{i,j,k}^{\prime}}}\,\tilde{Q}(\tilde{{s}}_{i,j,k}^{(n)},{x}_{i,j,k}^{\prime});\\ {1\!-\!\epsilon_{i}},~{\rm{otherwise}}.\end{array}\right.

The detailed procedure of DQN-based AoCR for vehicle iji\in\mathcal{I}_{j} is shown in Algorithm 2.

Remark: The proposed two-tier DQN algorithm satisfies both convergency and robustness in the VFC-based offloading process.

1) Convergency. According to [50], it is theoretically guaranteed that Q-learning can converge to the optimal action-value function when nn\rightarrow\infty. Besides, based on the experimental validations in [49], the large CNN can be efficiently trained in a stable manner in DQN via experience replay and SGD. The stability of CNN training and the convergency of the two-tier DQN algorithm are demonstrated in the simulation section.

2) Robustness. By incorporating CNNs for state space compression and experience replay methods for mitigation of oscillations or divergence in learning, our DQN-based offloading algorithm can be effective in a large-scale network involving a large number of vehicles [48]. Besides, according to the experimental results in [51], the training data obtained from the historical experiments performed in similar scenarios can be exploited to accelerate the learning speed of Q-learning. Hence, each UAV or vehicle can pre-train its DQN model in an off-line manner (e.g., when they are in recharging state) by exploiting previous experiences learned from similar scenarios to speed up the convergence rate.

VI PERFORMANCE EVALUATION

VI-A Simulation Setup

Refer to caption
Fig. 4: The simulation area.

As shown in Fig 4, we consider a simulation scenario with 1010 UAVs, 200200 ground vehicles, and 22 ECVs in a disaster site with width of 2020m and length of 20002000m. UAVs fly along the predefined straight-line trajectory [6] with a fixed altitude of 5050m over the two-lane road. UAVs are initially deployed alongside the road every 200200m, and vehicles are randomly distributed on each lane of the road with minimum safe distance of 1010m. The minimum and maximum velocities of ground vehicles on the road are set as 2424 km/h and 7272 km/h. The number of UAV’s offloading tasks follows the uniform distribution between 1010 and 2020. The data size of each task Dj,kD_{j,k} is randomly in [1,10][1,10] Mbits. The required CPU cycles θi,j,k\theta_{i,j,k} for one-bit data processing is randomly selected in [100,200][100,200] CPU cycles/bit [34]. Each vehicle’s cost parameter ψi{\psi_{i}} is uniformly distributed between 44 and 1616 cents. Our RescueChain system is established atop the Tendermint Core consensus engine [52] by utilizing the Docker container environment run on three computers with Intel Core i7-8700 CPU, 8GB RAM, and Ubuntu 16.04 OS. Other parameters are summarized in Table III.

TABLE III: Simulation Parameters
Parameter Value Parameter Value
vjmaxv_{j}^{\max} 2020 m/s [34] PiTXP_{i}^{\mathrm{TX}} 0.10.1 W [33]
PjTXP_{j}^{\mathrm{TX}} 11 W [33] PjRXP_{j}^{\mathrm{RX}} 0.10.1 W [33]
φ0{\varphi_{0}} 50-50 dBm [9] σ02\sigma_{0}^{2} 100-100 dBm [33]
BiULB_{i}^{\mathrm{UL}} 1010 MHz [33] BjDLB_{j}^{\mathrm{DL}} 0.50.5 MHz [33]
RjA2AR_{j}^{\mathrm{A2A}} 400400 m RjA2GR_{j}^{\mathrm{A2G}} 200200 m
μ\mu 22 [4] κi\kappa_{i} 102810^{-28} [34]
λ1,λ2\lambda_{1},\lambda_{2} 0.0037,5.02060.0037,5.0206[29] CjC_{j} 500500 kJ [29]
ρj,aj{\rho_{j}},a_{j} 162,2m/s2162,2\,\mathrm{m/s^{2}} ϖp{\varpi_{p}} 0.50.5
λp,λc{\lambda_{p}},\lambda_{c} 8,0.058,0.05 αj,k{\alpha_{j,k}} [0.1,0.9][0.1,0.9]
yj,kmaxy_{j,k}^{\max} 1111 cents ximaxx_{i}^{\max} 66 GHz
ςj,ςi\varsigma_{j},\varsigma_{i} 0.80.8 ϵj,ϵi\epsilon_{j},\epsilon_{i} 0.92,0.950.92,0.95
A1,A2A_{1},A_{2} 1111 D1,D2D_{1},D_{2} 44
|Ξj|,|Ξi|\left|\Xi_{j}\right|,\left|\Xi_{i}\right| 10001000 W,VW,V 22,1222,12
NN 90009000 θj,k\theta_{j,k} [0.3,0.7][0.3,0.7]
Z,ΨZ,\Psi 10,710,7 uini,Δvol\Im_{u}^{\mathrm{ini}},\Delta_{vol} 33
Δcp,Δcv\Delta_{cp},\Delta_{cv} 5,35,3 Δwbc,Δnbc\Delta_{wbc},\Delta_{nbc} 5,1.55,1.5
Δsbc,Δsbv\Delta_{sbc},\Delta_{sbv} 4,24,2 Δrep,Δacc\Delta_{rep},\Delta_{acc} 1.5,2.51.5,2.5
Fig. 5: Evolution of average AoCR of vehicles, compared with two existing schemes.
Fig. 6: Evolution of average payment of UAVs, compared with two existing schemes.
Fig. 7: Evolution of average payoff of UAVs and vehicles, compared with two-tier Q-learning scheme.
Fig. 8: Optimal AoCR of vehicle vs. cost parameter ψi\psi_{i}, compared with two existing schemes.
Fig. 9: Payoff of UAV vs. cost parameter ψi\psi_{i}, compared with two existing schemes.
Fig. 10: Throughput and block latency vs. block size in RescueChain.

In the simulation, we consider two kinds of insider attacks launched by malicious insider UAVs: (i) spoofing attack: they behave honestly at first to gain reputation and then begin to commit misbehaviors; and (ii) collusion attack: they collude to vote malicious delegates and witness incorrect blocks in the consensus process. The performance of our proposal is evaluated by comparing with the following conventional schemes:

  • Two-tier Q-learning scheme [1]: both ground vehicles and UAVs employ the Q-learning method to seek their optimal AoCR and pricing strategies in the dynamic Stackelberg game, respectively.

  • Greedy scheme: both ground vehicles and UAVs greedily opt their optimal AoCR and pricing strategies during repeated interactions in the dynamic Stackelberg game, respectively.

  • Accumulative reputation-based Tendermint (ART) scheme: the reputation of validators in Tendermint are evaluated using the accumulative behavior effects in Eq. (15) with η=0\eta=0 and Ψ=Z\Psi=Z, where the level-2 validators and the time decay effect are not taken into account.

  • Naive Tendermint scheme [40]: the naive Tendermint protocol is operated by all full nodes to achieve consensus without considering the reputation assessment of full nodes.

VI-B Simulation Results

Fig. 11: Average round to reach consensus vs. ratio of Byzantine nodes PbP_{b}, compared with two existing schemes.
Fig. 12: Average delay in offloading vs. data size Dj,k{D_{j,k}} of task with different vehicle density χ\chi, compared with the scheme without VFC.
Fig. 13: Average saved energy of UAV in offloading vs. data size Dj,k{D_{j,k}} of task with different vehicle density χ\chi.

Fig. 7 and Fig. 7 show the evolutions of average AoCR and payment of ground vehicles and UAVs in different schemes. As seen in the two figures, the proposed two-tier DQN scheme effectively motivates higher AoCR of vehicles with lower payment and attains a smaller deviation with the SE analyzed in Eq. (V-B), compared with the two-tier Q-learning scheme and the greedy scheme. It can be explained as follows. In the two-tier Q-learning scheme, owing to the curse of dimensionality in searching the large state space, it is difficult for the UAV to efficiently find its optimal payment policy via Q-learning to encourage vehicles’ high computing resource contributions, thereby resulting in a relatively lower AoCR and higher payment. In the greedy scheme, since both UAVs and ground vehicles behave greedily during the strategy-making process, they can only attain the local optimum of the payoff, causing the low AoCR and high payment. Besides, the AoCR in our two-tier DQN scheme in Fig. 7 first increases then decreases, and finally reaches a stable value. The reason is that the initial high payment of UAV can motivate vehicles’ high AoCR, while vehicles tend to decrease the AoCR after observing the subsequent decreasing payment.

Fig. 7 show the evolutions of average payoffs of UAVs and vehicles in three schemes. From Fig. 7, the proposed two-tier DQN scheme outperforms the two-tier Q-learning scheme in attaining a smaller deviation with the SE, as our proposal can promote vehicles’ higher AoCR contributions with lower payments as shown in the above two figures. Besides, as time grows, the average payoff of UAVs increases, while that of vehicles is in a decline. It is because after seeing the initial high AoCR of vehicles, the UAV prefers decreasing its payment gradually to improve its payoff. Meanwhile, after observing UAV’s prior payments, vehicles intend to increase their AoCR to seek optimized profit.

Fig. 10 and Fig. 10 compare the proposed scheme with other two schemes in terms of the average AoCR of vehicles and the average payoff of UAVs, where the cost parameter cic_{i} of vehicle varies from 22 to 88 cents. From these two figures, the proposed scheme can achieve a higher AoCR of vehicles and a better payoff of UAV in comparison with the two-tier Q-learning scheme and the greedy scheme. It can be explained as follows. In the two-tier Q-learning scheme, due to the presence of curse of dimensionality resulted from the large state space, UAVs cannot efficiently motivate the high AoCR sharing of vehicles with low payment for enhanced payoffs. In the greedy scheme, as participants greedily seek the optimal AoCR and payment strategies in the Q-learning process, both UAV’s payoff and vehicles’ AoCR can only stay at the local optimums.

Fig. 10 illustrates the throughput and the block latency in our RescueChain when the block size (i.e., the number of transactions included in a block) increases from 1×1031\times 10^{3} to 1×1041\times 10^{4}. In this simulation, we set proposeTimeout=6proposeTimeout\!=\!6 seconds and commitTime=1commitTime\!=\!1 millisecond. As seen in Fig. 10, our RescueChain system can efficiently cope with thousands of transactions per second with about one-second block latency. Moreover, with the increase of block size, both the blockchain throughput and the block latency increase. The reason is that the higher block size implies more transactions to be processed within a block while it incurs higher block propagation latency.

Fig. 13 compares our RescueChain with other two schemes in terms of the average round to reach consensus, given different ratios of Byzantine nodes. As seen in Fig. 13, our RescueChain outperforms the ART scheme and the naive Tendermint scheme in achieving the lowest average round to reach consensus. This is because in the ART scheme, with a reputation decrease (as a punishment) for misbehaving participants, the chance of honest nodes becoming validators can be increased, resulting in a lower average round to reach consensus than the naive Tendermint scheme. Besides, as the time decay in reputation is absent, previous behaviors occupy a large weight in determining the current reputation, causing a higher risk of suffering spoofing attacks. Moreover, as the level-2 validators are not considered in the ART scheme in reputation evaluation, the chance of misbehaving participants acting as the leader in blockchain can be increased. Accordingly, the average round to achieve consensus in the ART scheme is relatively larger than that in our RescueChain.

Fig. 13 depicts the average offloading delay in two schemes, where the data size of task Dj,kD_{j,k} changes from 22 to 88 Mbits and three different values of vehicle density χ\chi are exploited. In the scheme without VFC, UAVs’ computation tasks on sensory data are offloaded to the ECVs. We can see that, compared with the scheme without VFC, our proposed scheme with VFC can efficiently decrease the offloading delay given different vehicular densities and task sizes. This is because the ground vehicles are provisioned with sufficient computation capacities and are closer to UAVs. Moreover, as all UAVs offload their heavy tasks to the remote ECVs, the ECVs’ limited computing resource can be fully occupied, resulting in a longer waiting time for computation and increased latency for result delivery. Besides, as seen in Fig. 13, given the task size, the average task delay decreases with the increase of vehicle density. The reason is that the higher vehicle density implies more neighboring vehicles for task offloading and the corresponding higher AoCR sharing in computation, causing a decline in the task latency.

Fig. 13 demonstrates the average saved energy of UAV in task offloading given different data sizes of task Dj,kD_{j,k} and vehicle densities χ\chi. From Fig. 13, the UAV’s saved energy increases with the task size and the vehicle density. It is because with the increase of vehicle density, more computation resources can be contributed to UAVs, thereby saving more limited battery energy of UAVs. Besides, the heavy computing tasks can be effectively offloaded to vehicles under VFC when the task size grows, thereby improving the energy efficiency of UAVs.

TABLE IV: Energy consumption of our RescueChain in reaching consensus on a new block, compared with naive Tendermint scheme, Bitcoin, and Ethereum
# validators (ZZ)
Total energy consumption of validators for reaching
consensus of a block including 1000 transactions (J)
          Proposal Naive Tendermint
ZZ = 10 28.41 J 28.55 J
ZZ = 50 144.02 J 145.89 J
 ZZ = 100 293.04 J 299.67 J
  ZZ = 1000 3589.95 J 4413.62 J
Average energy consumption per transaction in Bitcoin: 6.48×1096.48\times 10^{9} J
Average energy consumption per transaction in Ethereum: 6.44×1086.44\times 10^{8} J

Table IV shows the energy efficiency of our RescueChain in the consensus phase in comparison with the naive Tendermint scheme, Bitcoin, and Ethereum. Here, the number of validators in the blockchain varies from 1010 to 10001000, and the block size (i.e., number of transactions included in a block) is set as 10001000. As shown in Table IV, our proposed RescueChain attains a much-improved energy efficiency in reaching consensus than traditional Bitcoin and Ethereum blockchains. According to [53, 54], the numbers of current reachable Bitcoin nodes and Ethereum Mainnet clients are 1616916169 and 56835683, respectively. In both Bitcoin and Ethereum, the compute-intensive PoW consensus protocol is leveraged to reach the unambiguous consensus among huge number of miners, thereby consuming dramatically considerable energy in reaching consensus, which is much larger than both naive Tendermint and our RescueChain. Besides, the proposed consensus scheme is more energy-efficient than the naive Tendermint scheme, especially when the number of validators is very large (i.e., 10001000). The reason is that thanks to the proposed reputation evaluation mechanism, our RescueChain can enjoy a lower average round to reach consensus on a new block than the naive Tendermint scheme, which is also validated in Fig. 13. Thereby, our RescueChain can save more energy for validator UAVs in block transmission and vote verification during the consensus process, resulting in improved energy efficiency.

VI-C Insights for Implementation

In this subsection, we discuss the following key components in the practical implementation of the proposed scheme.

TABLE V: Measurements of communication protocols for A2A, A2G/G2A, V2E, and V2V links in UDRNs
Link Description Protocol Data Range Throughput
A2A UAV\leftrightarrowUAV
Wi-Fi 6
(802.11ax)
Task data, transactions <800m <1.2 GB/s
A2G/ G2A UAV\leftrightarrowvehicle
Wi-Fi 6
(802.11ax)
Task data, transactions,
task processing results
<800m <1.2 GB/s
UAV\leftrightarrowECV LTE cellular
Task data, transactions,
scheduling command
1-3 km \approx37.5 MB/s
V2E vehicle\leftrightarrowECV LTE cellular
Transactions,
scheduling command
1-3 km \approx37.5 MB/s
V2V vehicle\leftrightarrowvehicle
WAVE
(802.11p)
Rescue data,
coordination data
<1 km 3-27 Mb/s

1) Selection of communication protocols. In the aerial subnetwork, the Wi-Fi 6 (IEEE 802.11ax) protocol that operates in the unlicensed spectrum can be utilized to transmit the sensory data, task data, and transactions among UAVs due to its high data rate (<1.2 GB/s) and ease of management. In the ground subnetwork, the well-known WAVE (Wireless Access in Vehicular Environments) protocol supported by IEEE 802.11p standard can be available for V2V links among ground rescue vehicles, which utilizes the 5.9 GHz band. Besides, due to the sparse network connections and potential network partitions, if the multi-hop V2V/A2A link is unavailable, the vehicle or UAV can deliver the information in a store-carry-and-forward fashion [27]. For the A2G/G2A links between UAVs and ground vehicles, the Wi-Fi module can be used for the transmission of task data, processing results, blockchain transactions, etc. For the ECVs, they can quickly enter part of disaster areas and establish cellular networks with large coverage (about 1-3 km in LTE) and moderate throughput for UAVs and rescue vehicles for rescue coordination (e.g., collision avoidance and position adjustment). Besides, the UAV can offload its computation task to the ECV in its coverage via the cellular module if there are no available ground vehicles or they do not have sufficient computing resources, as a complement to the VFC paradigm. Table V shows the performance measurements of different communication protocols for A2A, A2G/G2A, vehicle-to-ECV (V2E), and V2V links in UDRNs.

2) Pre-training DQN models in task offloading. In DQN-based task offloading, as seen in Figs. 77, it takes a long time for UAVs and vehicles to learn the optimal offloading strategies, especially in a large-scale network involving a large number of participants. In the real implementation, each UAV and vehicle can pre-train its DQN model by exploiting historical experiences (i.e., previous interaction information) learned from similar scenarios, thereby accelerating learning speed and alleviating energy consumption.

3) Selection of blockchain parameters. As shown in Figs. 1013 and Table IV, the blockchain parameters such as block size, ratio of Byzantine nodes, and number of validators can have a large effect on the performance of our RescueChain in terms of throughput, block latency, average consensus round, and energy consumption. For example, the higher block size results in higher throughput but also higher block latency; an assumption of low ratio of Byzantine nodes leads to low average consensus rounds but sacrifices consensus security; and more validator UAVs indicates better decentralization but yields higher energy consumption. Thereby, a trade-off needs to be achieved in the practical implementation, where appropriate blockchain parameters should be determined depending on specific scenarios. For example, for the scenario with highly sparse network connections, a low block size is preferred to attain a small block propagation latency; for the scenario with highly constrained UAV battery energy, a low number of UAV validators is preferred to achieve low energy consumption; and for the scenario with few malicious nodes, the small ratio of Byzantine nodes is preferred for faster consensus reaching and higher throughput.

4) Cold start issue. Initially, it is assumed that there exist no less than four connected ECVs in the disaster area to facilitate rescue missions. The genesis block of the blockchain can be created by these ECVs via the consensus process, which mainly includes the initial reputation setting for nodes, the role of the CA, block rewards, creation time, and hash of this block.

VII Conclusion and Future Work

Our paper focuses on the fundamental issue of provisioning guaranteed security for data dissemination in UDRNs. We have proposed the RescueChain scheme in disaster sites to address secure data sharing and efficient computation offloading for resource-limited UAVs. Within this scheme, a lightweight blockchain-based framework has been presented under the collaborative aerial-ground network to immutably trace misbehaving entities and safeguard information sharing. Then, we have devised a reputation-based consensus protocol and a VFC-based off-chain mechanism for improved consensus efficiency and robustness in weakly connected UDRNs. To stimulate vehicles’ collaborative behaviors in VFC-based computation offloading for UAVs, we have formulated the interactions between them as a Stackelberg game and derived the SE of the static Stackelberg game. Apart from this, in the dynamic Stackelberg game, a novel two-tier DQN-based algorithm has been devised to intelligently and distributively seek the optimal payment strategies for UAVs (tier 1) and computing resource sharing strategies for vehicles (tier 2), without fully knowing the opponent’s private parameters and time-varying network parameters. At last, simulation results have validated the effectiveness of our RescueChain in terms of enhanced consensus and offloading efficiency, reduced task latency and UAVs’ energy consumption, and improved payoffs of participants.

For future work, we plan to investigate: 1) the cooperation mechanism among a swarm of UAVs for collaborative sensing, trajectory scheduling, and resource sharing in UDRNs; 2) the coordination of satellites in various earth orbits, UAV swarms, and ground equipments to construct efficient space-air-ground disaster rescue networks; and 3) the vehicle-assisted mobile charging mechanism for efficient UAV battery recharging by equipping wireless energy transfer (WPT) pads on vehicular roofs.

Appendix A proof of theorem 44

Proof:

Note that when βi,j,k=0\beta_{i,j,k}=0, it means that vehicle ii will not participate in the offloading process of task kk for UAV jj. Consequently, the corresponding payment of UAV jj to vehicle ii is zero. As such, xi,j,k=0x_{i,j,k}=0 and yi,j,k=0y_{i,j,k}=0. Here, we only consider the case βi,j,k=1\beta_{i,j,k}=1. The first order differential of vehicle ii’s payoff function with respect to xi,j,kx_{i,j,k} is πi(xi,j,k,yi,j,k)xi,j,k=λpyi,j,k2λcψixi,j,k\frac{\partial\pi_{i}\left(x_{i,j,k},y_{i,j,k}\right)}{\partial x_{i,j,k}}=\lambda_{p}y_{i,j,k}-2\lambda_{c}\psi_{i}x_{i,j,k}. The second order differential for πi(xi,j,k,yi,j,k)\pi_{i}\left(x_{i,j,k},y_{i,j,k}\right) with respect to qi,c1{q_{i,c}^{1}} is 2πi(xi,j,k,yi,j,k)xi,j,k2=2λcψi<0\frac{\partial^{2}\pi_{i}\left(x_{i,j,k},y_{i,j,k}\right)}{\partial{x_{i,j,k}}^{2}}=-2\lambda_{c}\psi_{i}<0, which indicates that the function πi(xi,j,k,yi,j,k)xi,j,k\frac{\partial\pi_{i}\left(x_{i,j,k},y_{i,j,k}\right)}{\partial x_{i,j,k}} is monotonically decreasing. The following two cases are considered.

Case 11: High payment. If the payment of UAV jj for task kk is high, i.e., 2λcψiximaxλpyi,j,kyj,kmax\frac{2\lambda_{c}\psi_{i}x_{i}^{\max}}{\lambda_{p}}\leq y_{i,j,k}\leq y_{j,k}^{\max}, we have limxi,j,kximaxπi(xi,j,k,yi,j,k)xi,j,k0\underset{x_{i,j,k}\rightarrow x_{i}^{\max}}{\lim}\frac{\partial\pi_{i}\left(x_{i,j,k},y_{i,j,k}\right)}{\partial x_{i,j,k}}\geq 0. As such, πi(xi,j,k,yi,j,k)\pi_{i}\left(x_{i,j,k},y_{i,j,k}\right) is a monotonic increasing function. Then, the optimal AoCR strategy of vehicle ii is xi,j,k=ximaxx_{i,j,k}^{*}=x_{i}^{\max}.

Case 22: Low payment. If the payment of UAV jj for task kk is low, i.e., 0<yi,j,k<2λcψiximaxλp0<y_{i,j,k}<\frac{2\lambda_{c}\psi_{i}x_{i}^{\max}}{\lambda_{p}}, we have limxi,j,kximaxπi(xi,j,k,yi,j,k)xi,j,k<0\underset{x_{i,j,k}\rightarrow x_{i}^{\max}}{\lim}\frac{\partial\pi_{i}\left(x_{i,j,k},y_{i,j,k}\right)}{\partial x_{i,j,k}}<0. As such, the utility function of vehicle ii is strict convex, and vehicle ii’s optimal AoCR strategy can be obtained by solving πi(xi,j,k,yi,j,k)xi,j,k=0\frac{\partial\pi_{i}\left(x_{i,j,k},y_{i,j,k}\right)}{\partial x_{i,j,k}}=0, i.e., xi,j,k=λpyi,j,k2λcψix_{i,j,k}^{*}=\frac{\lambda_{p}y_{i,j,k}}{2\lambda_{c}\psi_{i}}. Theorem 4 is proved. ∎

Appendix B proof of theorem 55

Proof:

If 2λcψiximaxλpyi,j,kyj,kmax\frac{2\lambda_{c}\psi_{i}x_{i}^{\max}}{\lambda_{p}}\leq y_{i,j,k}\leq y_{j,k}^{\max}, by substituting xi,j,k=ximaxx_{i,j,k}^{*}=x_{i}^{\max} into πj(xi,j,k,yi,j,k)\pi_{j}\left(x_{i,j,k},y_{i,j,k}\right), the utility function of UAV jj can be rewritten as πj(xi,j,k,yi,j,k)=ϖpλpyi,j,kximax+ρjαj,klog(1+ximax)(ϖp)Φi,j,k\pi_{j}(x_{i,j,k},y_{i,j,k})=-\varpi_{p}\lambda_{p}y_{i,j,k}x_{i}^{\max}+\rho_{j}\alpha_{j,k}\log\left(1+x_{i}^{\max}\right)-\left(1\!-\!\varpi_{p}\right)\Phi_{i,j,k}. As the above function is monotonically decreasing with respect to yi,j,ky_{i,j,k}, UAV jj’s optimal payment strategy can be attained as yi,j,k=2λcψiximaxλpy_{i,j,k}^{*}=\frac{2\lambda_{c}\psi_{i}x_{i}^{\max}}{\lambda_{p}}.

If 0<pi,c1<2λcψiximaxλp0<{p_{i,c}^{1}}<\frac{2\lambda_{c}\psi_{i}x_{i}^{\max}}{\lambda_{p}}, by substituting xi,j,k=λpyi,j,k2λcψix_{i,j,k}^{*}=\frac{\lambda_{p}y_{i,j,k}}{2\lambda_{c}\psi_{i}} into πj(xi,j,k,yi,j,k)\pi_{j}\left(x_{i,j,k},y_{i,j,k}\right), UAV jj’s payoff function can be rewritten as:

πj(xi,j,k,yi,j,k)\displaystyle\pi_{j}(x_{i,j,k},y_{i,j,k}) =ρjαj,klog(1+λpyi,j,k2λcψi)ϖp(λp)22λcψi(yi,j,k)2\displaystyle=\rho_{j}\alpha_{j,k}\log(1+\frac{\lambda_{p}y_{i,j,k}}{2\lambda_{c}\psi_{i}})-\frac{\varpi_{p}(\lambda_{p})^{2}}{2\lambda_{c}\psi_{i}}(y_{i,j,k})^{2}
(ϖp)Φi,j,k\displaystyle-\left(1\!-\!\varpi_{p}\right)\Phi_{i,j,k}

The first order differential of UAV jj’s payoff with respect to yi,j,ky_{i,j,k} is πj(xi,j,k,yi,j,k)yi,j,k=ρjαj,kλp2λcψi+λpyi,j,kϖp(λp)2yi,j,kλcψi\frac{\partial\pi_{j}(x_{i,j,k},y_{i,j,k})}{\partial y_{i,j,k}}\!=\!\frac{\rho_{j}\alpha_{j,k}\lambda_{p}}{2\lambda_{c}\psi_{i}\!+\!\lambda_{p}y_{i,j,k}}\!-\!\frac{\varpi_{p}(\lambda_{p})^{2}y_{i,j,k}}{\lambda_{c}\psi_{i}}. The second order differential of UAV jj’s payoff satisfies 2πj(xi,j,k,yi,j,k)yi,j,k2<0\frac{\partial^{2}\pi_{j}\left(x_{i,j,k},y_{i,j,k}\right)}{\partial y_{i,j,k}^{2}}<0. It implies that UAV jj’s payoff function is strictly convex, and the maximum value can be derived by solving πj(xi,j,k,yi,j,k)yi,j,k=0\frac{\partial\pi_{j}(x_{i,j,k},y_{i,j,k})}{\partial y_{i,j,k}}=0 with KKT conditions. The optimal payment strategy can be obtained as shown in Eq. (V-B). Theorem 5 is proved. ∎

References

  • [1] Y. Wang, Z. Su, Q. Xu, R. Li, and T. H. Luan, “Lifesaving with RescueChain: Energy-efficient and partition-tolerant blockchain based secure information sharing for UAV-aided disaster rescue,” in IEEE INFOCOM 2021, Vancouver, Canada, May 2021.
  • [2] M. Stute, F. Kohnhäuser, L. Baumgärtner, L. Almon, M. Hollick, S. Katzenbeisser, and B. Freisleben, “Rescue: A resilient and secure device-to-device communication framework for emergencies,” IEEE Transactions on Dependable and Secure Computing, vol. 19, no. 3, pp. 1722–1734, 2022.
  • [3] Y. Wang, W. Chen, T. H. Luan, Z. Su, Q. Xu, R. Li, and N. Chen, “Task offloading for post-disaster rescue in unmanned aerial vehicles networks,” IEEE/ACM Transactions on Networking, vol. 30, no. 4, pp. 1525–1539, 2022.
  • [4] Z. Su, Y. Wang, Q. Xu, and N. Zhang, “LVBS: lightweight vehicular blockchain for secure data sharing in disaster rescue,” IEEE Transactions on Dependable and Secure Computing, vol. 19, no. 1, pp. 19–32, 2022.
  • [5] C. Luo, M. N. Satpute, D. Li, Y. Wang, W. Chen, and W. Wu, “Fine-grained trajectory optimization of multiple UAVs for efficient data gathering from WSNs,” IEEE/ACM Transactions on Networking, vol. 29, no. 1, pp. 162–175, 2021.
  • [6] X. Liu, Z. Li, N. Zhao, W. Meng, G. Gui, Y. Chen, and F. Adachi, “Transceiver design and multihop D2D for UAV IoT coverage in disasters,” IEEE Internet of Things Journal, vol. 6, no. 2, pp. 1803–1815, April 2019.
  • [7] X. Zhong, Y. Guo, N. Li, and Y. Chen, “Joint optimization of relay deployment, channel allocation, and relay assignment for UAVs-aided D2D networks,” IEEE/ACM Transactions on Networking, vol. 28, no. 2, pp. 804–817, 2020.
  • [8] Y. Wang, Z. Su, N. Zhang, and D. Fang, “Disaster relief wireless networks: Challenges and solutions,” IEEE Wireless Communications, vol. 28, no. 5, pp. 148–155, 2021.
  • [9] S. Zhang and J. Liu, “Analysis and optimization of multiple unmanned aerial vehicle-assisted communications in post-disaster areas,” IEEE Transactions on Vehicular Technology, vol. 67, no. 12, pp. 12 049–12 060, 2018.
  • [10] M. Erdelj, E. Natalizio, K. R. Chowdhury, and I. F. Akyildiz, “Help from the sky: Leveraging UAVs for disaster management,” IEEE Pervasive Computing, vol. 16, no. 1, pp. 24–32, 2017.
  • [11] W. Chen, Z. Su, Q. Xu, T. H. Luan, and R. Li, “VFC-based cooperative UAV computation task offloading for post-disaster rescue,” in IEEE INFOCOM 2020, 2020, pp. 228–236.
  • [12] M. Erdelj, M. Krl, and E. Natalizio, “Wireless sensor networks and multi-UAV systems for natural disaster management,” Computer Networks, vol. 124, no. C, p. 72–86, Sep. 2017.
  • [13] Y. Wang, Z. Su, J. Ni, N. Zhang, and X. Shen, “Blockchain-empowered space-air-ground integrated networks: Opportunities, challenges, and solutions,” IEEE Communications Surveys & Tutorials, vol. 24, no. 1, pp. 160–209, 2022.
  • [14] S. Wu, Y. Chen, M. Li, X. Luo, Z. Liu, and L. Liu, “Survive and thrive: A stochastic game for DDoS attacks in Bitcoin mining pools,” IEEE/ACM Transactions on Networking, vol. 28, no. 2, pp. 874–887, 2020.
  • [15] Y. Wang, H. Peng, Z. Su, T. H. Luan, A. Benslimane, and Y. Wu, “A platform-free proof of federated learning consensus mechanism for sustainable blockchains,” IEEE Journal on Selected Areas in Communications, 2022, doi:10.1109/JSAC.2022.3213347.
  • [16] A. Islam and S. Y. Shin, “BUAV: A blockchain based secure UAV-assisted data acquisition scheme in internet of things,” Journal of Communications and Networks, vol. 21, no. 5, pp. 491–502, 2019.
  • [17] X. Li, Y. Wang, P. Vijayakumar, D. He, N. Kumar, and J. Ma, “Blockchain-based mutual-healing group key distribution scheme in unmanned aerial vehicles ad-hoc network,” IEEE Transactions on Vehicular Technology, vol. 68, no. 11, pp. 11 309–11 322, 2019.
  • [18] T. Bai, J. Wang, Y. Ren, and L. Hanzo, “Energy-efficient computation offloading for secure UAV-edge-computing systems,” IEEE Transactions on Vehicular Technology, vol. 68, no. 6, pp. 6074–6087, 2019.
  • [19] M. Messous, S. Senouci, H. Sedjelmaci, and S. Cherkaoui, “A game theory based efficient computation offloading in an UAV network,” IEEE Transactions on Vehicular Technology, vol. 68, no. 5, pp. 4964–4974, 2019.
  • [20] B. Liu, W. Zhang, W. Chen, H. Huang, and S. Guo, “Online computation offloading and traffic routing for UAV swarms in edge-cloud computing,” IEEE Transactions on Vehicular Technology, vol. 69, pp. 8777–8791, 2020.
  • [21] W. Liang, M. Tang, J. Long, X. Peng, J. Xu, and K. Li, “A secure fabric blockchain-based data transmission technique for industrial internet-of-things,” IEEE Transactions on Industrial Informatics, vol. 15, no. 6, pp. 3582–3592, 2019.
  • [22] W. Chen, Y. Chen, X. Chen, and Z. Zheng, “Toward secure data sharing for the IoV: A quality-driven incentive mechanism with on-chain and off-chain guarantees,” IEEE Internet of Things Journal, vol. 7, pp. 1625–1640, 2020.
  • [23] X. Jiang, F. R. Yu, T. Song, Z. Ma, Y. Song, and D. Zhu, “Blockchain-enabled cross-domain object detection for autonomous driving: A model sharing approach,” IEEE Internet of Things Journal, vol. 7, no. 5, pp. 3681–3692, 2020.
  • [24] Q. Kong, L. Su, and M. Ma, “Achieving privacy-preserving and verifiable data sharing in vehicular fog with blockchain,” IEEE Transactions on Intelligent Transportation Systems, pp. 1–10, 2020.
  • [25] D. Callegaro and M. Levorato, “Optimal computation offloading in edge-assisted UAV systems,” in IEEE GLOBECOM 2018, 2018, pp. 1–6.
  • [26] W. Chen, B. Liu, H. Huang, S. Guo, and Z. Zheng, “When UAV swarm meets edge-cloud computing: The QoS perspective,” IEEE Network, vol. 33, no. 2, pp. 36–43, 2019.
  • [27] Y. Zhou, N. Cheng, N. Lu, and X. S. Shen, “Multi-UAV-aided networks: Aerial-ground cooperative vehicular networking architecture,” IEEE Vehicular Technology Magazine, vol. 10, no. 4, pp. 36–44, 2015.
  • [28] F. Zhou, Y. Wu, R. Q. Hu, and Y. Qian, “Computation rate maximization in UAV-enabled wireless-powered mobile-edge computing systems,” IEEE Journal on Selected Areas in Communications, vol. 36, no. 9, pp. 1927–1941, 2018.
  • [29] S. Jeong, O. Simeone, and J. Kang, “Mobile edge computing via a UAV-mounted cloudlet: Optimization of bit allocation and path planning,” IEEE Transactions on Vehicular Technology, vol. 67, no. 3, pp. 2049–2063, 2018.
  • [30] Y. Zeng and R. Zhang, “Energy-efficient UAV communication with trajectory optimization,” IEEE Transactions on Wireless Communications, vol. 16, no. 6, pp. 3747–3760, 2017.
  • [31] C. Zhan and Y. Zeng, “Energy-efficient data uploading for cellular-connected UAV systems,” IEEE Transactions on Wireless Communications, vol. 19, no. 11, pp. 7279–7292, 2020.
  • [32] H. Zhou, B. Liu, F. Hou, T. H. Luan, N. Zhang, L. Gui, Q. Yu, and X. S. Shen, “Spatial coordinated medium sharing: Optimal access control management in drive-thru Internet,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 5, pp. 2673–2686, 2015.
  • [33] Z. Yu, Y. Gong, S. Gong, and Y. Guo, “Joint task offloading and resource allocation in UAV-enabled mobile edge computing,” IEEE Internet of Things Journal, vol. 7, no. 4, pp. 3147–3159, 2020.
  • [34] H. Guo and J. Liu, “UAV-enhanced intelligent offloading for internet of things at the edge,” IEEE Transactions on Industrial Informatics, vol. 16, no. 4, pp. 2737–2746, 2020.
  • [35] R. Alkadi, N. Alnuaimi, C. Y. Yeun, and A. Shoufan, “Blockchain interoperability in unmanned aerial vehicles networks: State-of-the-art and open issues,” IEEE Access, vol. 10, pp. 14 463–14 479, 2022.
  • [36] D. Boneh, B. Lynn, and H. Shacham, “Short signatures from the weil pairing,” in Advances in Cryptology–ASIACRYPT 2001, 2001, pp. 514–532.
  • [37] J. Benet, “IPFS - content addressed, versioned, P2P file system,” CoRR, vol. abs/1407.3561, 2014. [Online]. Available: http://arxiv.org/abs/1407.3561
  • [38] J. Kwon, “Tendermint: Consensus without mining,” 2014. [Online]. Available: https://tendermint.com/static/docs/tendermint.pdf
  • [39] C. Dwork, N. Lynch, and L. Stockmeyer, “Consensus in the presence of partial synchrony,” Journal of the ACM, vol. 35, no. 2, pp. 288–323, 1988.
  • [40] E. Buchman, “Tendermint: Byzantine fault tolerance in the age of blockchains,” 2016. [Online]. Available: https://allquantor.at/blockchainbib/pdf/buchman2016tendermint.pdf
  • [41] K. Cai, H. Xie, and J. C. S. Lui, “Information spreading forensics via sequential dependent snapshots,” IEEE/ACM Transactions on Networking, vol. 26, no. 1, pp. 478–491, 2018.
  • [42] A. Castiglione, G. Cattaneo, G. De Maio, A. De Santis, and G. Roscigno, “A novel methodology to acquire live big data evidence from the cloud,” IEEE Transactions on Big Data, vol. 5, no. 4, pp. 425–438, 2019.
  • [43] X. Li, M. Wang, H. Wang, Y. Yu, and C. Qian, “Toward secure and efficient communication for the internet of things,” IEEE/ACM Transactions on Networking, vol. 27, no. 2, pp. 621–634, 2019.
  • [44] H. Cui, R. H. Deng, and G. Wang, “An attribute-based framework for secure communications in vehicular ad hoc networks,” IEEE/ACM Transactions on Networking, vol. 27, no. 2, pp. 721–733, 2019.
  • [45] H. Wu, L. Wang, G. Xue, J. Tang, and D. Yang, “Enabling data trustworthiness and user privacy in mobile crowdsensing,” IEEE/ACM Transactions on Networking, vol. 27, no. 6, pp. 2294–2307, 2019.
  • [46] K. Wang, F. C. M. Lau, L. Chen, and R. Schober, “Pricing mobile data offloading: A distributed market framework,” IEEE Transactions on Wireless Communications, vol. 15, no. 2, pp. 913–927, 2016.
  • [47] L. Wu, “A tighter piecewise linear approximation of quadratic cost curves for unit commitment problems,” IEEE Transactions on Power Systems, vol. 26, no. 4, pp. 2581–2583, Nov 2011.
  • [48] L. Xiao, Y. Li, G. Han, H. Dai, and H. V. Poor, “A secure mobile crowdsensing game with deep reinforcement learning,” IEEE Transactions on Information Forensics and Security, vol. 13, no. 1, pp. 35–47, 2018.
  • [49] V. Mnih, K. Kavukcuoglu, D. Silver, and et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529–533, 2015.
  • [50] C. J. C. H. Watkins and P. Dayan, “Technical note: Q-learning,” Machine Learning, vol. 8, pp. 279–292, 1992.
  • [51] L. Xiao, D. Xu, N. B. Mandayam, and H. V. Poor, “Attacker-centric view of a detection game against advanced persistent threats,” IEEE Transactions on Mobile Computing, vol. 17, no. 11, pp. 2512–2523, 2018.
  • [52] (2020) Tendermint core consensus engine. [Online]. Available: https://github.com/tendermint/tendermint
  • [53] Accessed: June. 11, 2022. [Online]. Available: https://bitnodes.io/
  • [54] Accessed: June. 11, 2022. [Online]. Available: https://ethernodes.org/
[Uncaptioned image] Yuntao Wang received the Ph.D degree in Cyberspace Security from Xi’an Jiaotong University, Xi’an, China, in 2022. His research interests include security and privacy in IoT, network games, edge intelligence, and blockchain.
[Uncaptioned image] Zhou Su has published technical papers, including top journals and top conferences, such as IEEE Journal On Selected Areas In Communications, IEEE Transactions On Information Forensics And Security, IEEE Transactions On Dependable And Secure Computing, IEEE Transactions On Mobile Computing, IEEE/ACM Transactions On Networking, and INFOCOM. His research interests include multimedia communication, wireless communication, and network traffic. Dr. Su received the Best Paper Award of International Conference IEEE ICC2020, IEEE BigdataSE2019, and IEEE CyberSciTech2017. He is an Associate Editor of IEEE Internet Of Things Journal, IEEE Open Journal Of Computer Society, and IET Communications.
[Uncaptioned image] Qichao Xu received the Ph.D. degree from the School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, China, in 2019. He is currently an Assistant Professor with Shanghai University. His research interests include wireless network architecture and vehicular networks.
[Uncaptioned image] Ruidong Li received the D.Eng. degree from the University of Tsukuba in 2008. He is currently an Associate Professor with the College of Science and Engineering, Kanazawa University, Japan. His current research interests include future networks, big data networking, blockchain, the Internet of Things, and network security. He is the Secretary of IEEE ComSoC Internet Technical Committee and the Founder and Chair of the IEEE SIG on big data intelligent networking and IEEE SIG on intelligent Internet edge. He is a guest editor of prestigious journals, such as IEEE Communications Magazine, IEEE Network Magazine, and IEEE Transactions on Network Science and Engineering.
[Uncaptioned image] Tom H. Luan received the Ph.D. degree from the University of Waterloo, Canada, in 2012. He is currently a Professor with the School of Cyber Science and Engineering, Xi’an Jiaotong University, China. He has authored/coauthored more than 90 journal articles and 30 technical articles in conference proceedings. His research mainly focuses on content distribution and media streaming in vehicular ad hoc networks and peer-to-peer networking and the protocol design and performance evaluation of wireless cloud computing and edge computing. He served as a TPC Member for IEEE Globecom, ICC, and PIMRC.
[Uncaptioned image] Pinghui Wang received the BS and PhD degrees in information engineering from Xi’an Jiaotong University, Xi’an, China, in 2006 and 2012, respectively. He is currently a professor with the MOE Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University, China. His research interests include Internet traffic measurement and modeling, traffic classification, abnormal detection, and online social network measurement.