Dynamic asset addition management of a set of assets

Conventional unmanned aerial vehicles (UAVs), or drones, are useful for performing a number of tasks. Such drones may perform surveillance, delivery of commercial packages or weaponry, mapping of distant or inhospitable regions and/or other missions. Although useful, such drones suffer from a number of drawbacks. For example, drones are typically remotely piloted, may lack reliability, may be slower than desired, may have limited range, and/or may have other issues that adversely affect performance. Further, coordination of multiple assets, including one or more drones, is complicated and generally requires a user in a control center to control and coordinate the assets in connection with performance of an operation.

The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.

According to various embodiments, a system for grouping assets is disclosed. The system may include a communication interface, and one or more processors coupled to the communication interface. The one or more processors may be configured to (i) receive, via the communication interface, data associated with one or more tasks to be performed by a set of assets, wherein the set of assets comprises a plurality of drones, and the drones are at least semi-autonomous, (ii) determine, from among a plurality of assets, the set of assets to perform the one or more tasks, and (iii) communicate, via the communication interface, an instruction to at least one drone in the set of assets, the instruction indicating that the one or more tasks are to be completed at least in part by the at least one drone. The determining the set of assets to perform the one or more tasks may include determining one or more capabilities associated with one or more characteristics of the one or more tasks, and determining the plurality of drones based at least in part on (i) the one or more capabilities associated with one or more characteristics of the one or more tasks, and (ii) one or more drone capabilities respectively associated with the plurality of drones.

According to various embodiments, a drone is disclosed. The drone may be a semi-autonomous drone. The drone may include a communication interface, and one or more processors coupled to the communication interface. The one or more processors may be configured to (i) receive, via the communication interface, an indication that the drone is part of a set of assets, wherein the set of assets are tasked with performing one or more elements of one or more tasks, and the set of assets comprises a plurality of drones, (ii) communicate, via the communication interface, information pertaining to the one or more elements, and (iii) communicate, via the communication interface, information pertaining to the plan to perform the one or more tasks, wherein the information pertaining to the plan to perform the one or more tasks is communicated with at least one other drone in the set of assets. The information pertaining to the one or more elements may be communicated with at least one other drone in the set of assets. The information pertaining to the one or more elements may be based at least in part on information obtained by one or more sensors of the set of assets. The information pertaining to the one or more elements may be used in connection with determining a plan to perform the one or more tasks.

According to various embodiments, a drone is disclosed. The drone may be a semi-autonomous drone. The drone may include a communication interface, and one or more processors coupled to the communication interface. The one or more processors may be configured to (i) receive, via the communication interface, an indication that the drone is part of a set of assets, wherein the set of assets are tasked with performing one or more elements of one or more tasks, and the set of assets comprises a plurality of drones, (ii) determine that at least one drone of the set of assets has experienced a failure, (iii) in response to a determination that the at least one drone has experienced the failure, update a plan to an updated plan to perform the one or more tasks, and (iv) communicate, via the communication interface, information pertaining to the updated plan, wherein the information pertaining to the updated plan is communicated with at least one remaining drone of the set of assets. The one or more processors may be further configured to communicate, via the communication interface, information obtained during implementation of at least an element of the updated plan, wherein the information pertaining to the updated plan is communicated with at least one remaining asset of the set of drones.

According to various embodiments a system for dynamically grouping assets is disclosed. The system may include a communication interface, and one or more processors coupled to the communication interface. The one or more processors may be configured to (i) obtain data associated with one or more tasks being performed by a set of assets, wherein the set of assets comprises a plurality of drones and the plurality of drones is at least semi-autonomous, (ii) determine to modify the set of assets based at least in part on the data associated with the one or more tasks, and (iii) communicate, via the communication interface, an instruction to at least one drone in the set of assets, the instruction indicating the modification to the set of assets. To determine to modify the set of assets based at least in part on the data associated with the one or more tasks may include determining one or more capabilities associated with one or more remaining tasks, and determining a modification to the set of assets based at least in part on (i) the one or more capabilities associated with the one or more remaining tasks, and (ii) one or more drone capabilities respectively associated with the set of assets.

According to various embodiments a system is disclosed. The system may include a communication interface, and one or more processors coupled to the communication interface. The one or more processors may be configured to (i) cause a first user interface to be displayed, the first user interface comprising one or more selectable elements associated with a characteristic of one or more tasks to be performed, (ii) receive, via the first user interface, one or more user selections pertaining to the characteristic of the one or more tasks to be performed, (iii) in response to receiving the one or more user selections input to the first interface, cause a second user interface to be displayed, (iv) receive, via the second user interface, one or more user selections pertaining to the set of one or more assets to be deployed to perform the operation, (v) determine the operation to be performed, the operation being determined based at least in part on (a) the one or more user selections pertaining to the characteristic of the one or more tasks to be performed, and (b) the one or more user selections pertaining to the set of one or more assets to be deployed to perform the operation, and (vi) communicate, via the communication interface, information pertaining to the operation. The second user interface may be configured based at least in part on at least one of the one or more user selections input to the user interface. The second user interface may comprise one or more selectable elements associated with a set of one or more assets to perform an operation, wherein the set of one or more assets comprises one or more drones, and the one or more drones are semi-autonomous. The information pertaining to the operation is communicated to at least one drone of the set of assets. The information pertaining to the operation causes the set of assets to be deployed to implement at least part of the operation.

According to various embodiments, a system is disclosed. The system may include a communication interface, and one or more processors coupled to the communication interface. The one or more processors may be configured to (i) obtain data associated with one or more tasks to be performed by a set of assets, (ii) determine a discretized representation of the geographic area, wherein the discretized representation comprises a plurality of discrete elements each corresponding to a volume in the geographic area, (iii) annotate the discretized representation to create an annotated representation with the one or more parameters pertaining to the geographic area with at least a subset of the plurality of discrete elements based at least in part on a determination that the one or more parameters pertain to the geographic area, (iv) determine a plan to perform the one or more tasks, wherein the plan is based at least in part on the annotated representation, and (v) cause the one or more tasks to be performed based at least in part on the plan. The set of assets may comprise a plurality of drones and the plurality of drones is at least semi-autonomous. The data associated with the one or more tasks may comprise one or more parameters pertaining to a geographic area in which at least one asset of the set of assets is to perform the one or more tasks.

According to various embodiments, a system is disclosed. The system may include a communication interface, and one or more processors coupled to the communication interface. The one or more processors may be configured to (i) obtain data associated with one or more tasks to be performed by a set of assets, (ii) determine a discretized representation of the geographic area, wherein the discretized representation comprises a plurality of discrete elements each corresponding to a volume in the geographic area, (iii) annotate the discretized representation with the one or more other drone flight plans to create an annotated representation, (iv) determine a first flight plan of at least one drone of the plurality of drones, wherein the first flight plan is determined based at least in part on the annotated representation, (v) communicate information pertaining to the first flight plan to at least one other asset in the set of assets. The set of assets comprises a plurality of drones and the plurality of drones may be at least semi-autonomous. The data associated with the one or more tasks may comprise one or more other drone flight plans.

According to various embodiments, a system receives a high-level instruction to execute an operation using a set of drones. The high-level instruction may comprise a high-level description or definition of an operation. In response to receiving the high-level instruction, the system may determine one or more low-level instructions in connection with the performance of the operation using a set of drones. For example, semi-autonomous drone such as a leader drone may deconstruct the high-level instruction into one or more tasks for the set of drones, and the semi-autonomous drone may communicate the one or more tasks (or plans for completing the one or more tasks or elements of the tasks) to one or more drones in the set of drones (e.g., to follower drones). In some embodiments, a user or controls system provides a high-level definition of an operation, and one or more semi-autonomous drones autonomously determine a task or element of a task associated with the operation. A set of drones (e.g., comprising the one or more semi-autonomous drones) then performs a task or element of a task associated with the operation.

As used herein, “semi-autonomous drone” means a drone that, without human intervention, determines a plan to perform at least part of a task or operation based on a high level instruction to perform the task or operation, obtains feedback information pertaining to a context of the drone and/or plan, and updates or determines a new plan to react to the feedback information. The updated plan or new plan may provide an instruction for controlling the drone in a manner consistent with the high level instruction or the operation. The high level instruction may be provided to the drone by a user interface or by a server via a network connection. In response to receiving the high level instruction, the drone determines the plan to perform the part of the task or operation and implements the plan or instructs another asset to provide a part of the task or operation and provide feedback to the drone. The term “semi-autonomous drone” may be used interchangeably herein with “drone.” For example, a task of “scan a polygon and track moving targets” may involve planning out how the one or more drones is to scan the polygon effectively, but halfway through the scan pattern a new target may appear and the semi-autonomous drone would re-plan and start following that new target.

A high-level instruction may include one or more of an indication of a type of operation, a location at which the operation is to be performed, a target of the operation, a time at which the operation is to be performed, etc. A low-level instruction may provide more specific information or definitions with respect to the operation. In some embodiments, the low-level instruction may indicate a specific asset to perform a task in the operation, a manner by which the task is to be performed (e.g., speed, height, type of sensor, etc.). The low-level instruction may be further determined based on environmental information or context information within the area at which the operation is to be performed. An example of a high-level instruction may be an instruction to perform an operation of surveilling a particular road on a specific date. In contrast, the corresponding low-level instructions may include an instruction to asset in the set of assets that indicate a particular location (or range of location) along the road at which the particular asset is to capture information. A leader drone may determine the particular follower drones to perform a low-level instruction. The low-level instruction may indicate a type of information to be captured, a speed at which the asset is to travel, a height at which the asset is to travel or be during surveillance. As another example, a high level instruction may include “observe this target from several good vantage points,” and the corresponding low level instruction may be “fly drone 1 to 30 degrees heading with 100 meter standoff relative to target, and drone 2 to 180 degrees heading with 150 meter standoff.”

As used herein, “asset” may correspond to device, terminal, vehicle, and/or system that may receive information from a server (e.g., either directly or via another asset), and implement a plan to perform task or an element of a task, such as using a sensor to obtain information, communicating collected information to a leader drone/asset or the server, delivering a payload, etc. Examples of assets include, without limitation, a drone, a satellite, a communication tower, an observation station, a vehicle such as an autonomous or semiautonomous vehicle (e.g., a plane, a boat, a car, a truck, a helicopter, etc.), a drone (e.g., a large drone, a small drone, etc.) a jet, a small autonomous sensor (e.g., a sensor, a ‘dust’ sensor, etc.), etc. An asset has an associated identifier and set of defined characteristics, capabilities, traits, etc. (also collectively referred to herein as “capabilities”). The set of defined capabilities may be indicative of a set of functions that an asset may perform. For example, the set of capabilities may indicate a set of peripherals comprised, or otherwise operatively connected to, the asset such as a camera, a sensor, a payload, a payload deployment mechanism, a pre-loaded software, etc. Another example the set of defined capabilities may indicate whether the asset comprises turret (e.g., a multi-axis pointing mechanism that other capabilities are mounted to, for example the pan tilt unit of a surveillance camera). Another example the set of defined capabilities may indicate whether the asset has the ability to send and receive data from other sources of communication, including disabling/enabling transmission.

As used herein, “plan information” may correspond to information pertaining to the plan to perform a task, an element, and/or an operation. As an example, plan information may include parameters for the tsk, the element, and/or the operation. As an example, plan information may include an instruction for operating the asset.

As used herein, “capability information” may correspond to information pertaining to a capability of an asset such as a drone. As an example, capability information may include an indication of a capability and/or a set of capabilities of a drone.

As used herein, “sensor information” may correspond to information obtained by a sensor such as a sensor of an asset. The sensor information may be processed locally by the asset or device such as to convert raw sensor input to a predetermined format, etc.

As used herein, “state information” may correspond to information pertaining to a context or state of a device such as an asset (e.g., a drone). As an example, the state information may include information indicating a state of a particular capability, or set of capabilities, of a drone. As an example, the state information may include information indicating a status of one or more tasks (e.g., an extent to which an asset has completed a task). As an example, the state information may include result information of performance of the task, an element, and/or an operation.

Various embodiments include a system for configuring an operation that is to be performed by a set of assets, including a semi-autonomous drone. The system may include one or more servers, a client terminal, and/or a set of one or more assets. The server(s), client terminal, and set of one or more assets may communicate over one or more assets. The client terminal may provide a user interface with which an operation may be configured (e.g., defined), a set of assets may be deployed to perform at least part of the operation, and feedback pertaining to a status of performance of the operation is provided. In connection with configuring the operation, a set of assets to perform at least part of an operation may be determined. The part of the operation may be determined by a semi-autonomous asset such as a leader drone. For example, the leader drone may analyze the operation (e.g., a high-level instruction to perform the operation) and determine one or more tasks respectively corresponding to at least a part of the operation. For example, a first task can correspond to a first part of the operation, and a second task can correspond to a second part of the operation. The set of assets may at least semi-autonomously determine a plan to perform at least part of the operation, and implement the plan.

In some embodiments, the server may determine a set of assets to perform one or more tasks associated with an operation. The set of assets may be determined based at least in part on one or more capabilities associated with one or more characteristics of the one or more tasks. For example, the one or more capabilities associated with one or more characteristics of the one or more tasks may correspond to a function to be performed (e.g., in the case of a surveillance tasks, a capability may correspond to video capture, image capture, etc., in the case of drone delivery, a capability may correspond to range, a mechanism to carry a payload, weight restrictions of the payload, etc.). The set of assets to perform the one or more tasks may be determined based at least in part on one or more drone capabilities respectively associated with the plurality of drones. For example, in response to determining a configuration of an operation, the server may determine one or more tasks corresponding to the operation and determine a capability associated with a task (e.g., based at least in part on a mapping of tasks to capabilities, etc.). The server may determine one or more assets matching the capability associated with the task (e.g., based at least in part on a mapping of assets to capabilities, etc.), and determine a set of assets to perform the one or more tasks, or determine to add an asset to a predefined set of assets to perform the one or more tasks. In response to deploying the set of assets to perform the operation or to determining the set of assets, at least one asset in the set of assets (e.g., a leader drone) may be provided with information pertaining to the operation (e.g., one or more characteristics or identifiers associated with the one or more tasks, an indication of the set of assets to perform the one or more tasks, etc.). In some implementations, a plurality of the set of assets, or each asset within the set of assets, is provided with an instruction indicating that the particular asset is to be used in connection with performing the one or more tasks. The set of assets may be determined at least in part on an availability of one or more assets. For example, an ownership (e.g., a lease) of an asset may be used in connection with determining whether to include the asset in the set of assets to perform the one or more tasks.

Various embodiments include the grouping of a set of assets to perform at least part of an operation (e.g., one or more tasks). The set of assets may include one or more semi-autonomous drones. In response to the operation being configured (e.g., at the server), a high-level instruction of the operation (or a part thereof, such as a task) is provided to at least one drone in the set of assets. The high-level instruction may include a high-level definition of the operation or task, including one or more characteristics of the operation or task (e.g., a task to be performed, a location at which the task is to be performed, a time at which the task is to be performed, a region in which the set of assets are permitted to operate, etc.). In response to receiving information pertaining to the operation (e.g., an indication that the drone is included in the set of assets), a semi-autonomous drone may determine one or more elements of one or more tasks, or one or more tasks, to be performed by the set of assets. The semi-autonomous drone may be a leader drone among a plurality of drones within the set of assets. In some embodiments, the leader drone may determine a plan for at least one drone in the set of assets to perform at least an element of the one or more tasks. For example, the leader drone may execute a planning service (e.g., locally by a processor on the drone) that determines a plan for performing the one or more tasks assigned to the set of assets. The determining the plan for performing the one or more tasks may be based at least in part on one or more of (i) a function to be performed, (ii) an environment in which the task is to be performed, (iii) a capability associated with the task to be performed, (iv) a capability of at least one drone in the set of assets, etc. The plan may be updated based on feedback information provided by one or assets within the set of assets (e.g., an indication of asset failure, a change in the environment, a location of a target, etc.).

According to various embodiments, the assets within the set of assets may communicate with one another. For example, the leader drone within the set of assets may provide information pertaining to at least an element of a task, such a plan for an asset to perform the element of the task, or to perform the task. As another example, another drone in the set of assets or another asset in the set of assets may send to the leader drone information pertaining to a status of the implementation of the plan, a context of the asset (e.g., an environment of the asset, such as weather, etc.), etc. In some embodiments, one or more assets (e.g., drones) may listen to information being communicated between a leader drone and a follower drone or other asset within the set of assets. For example, a dormant leader may obtain the information communicated among the set of assets to allow the dormant leader to relatively seamlessly take over the responsibilities/role of the leader in response to a determination that the leader drone has failed, or in response to a determination that the dormant leader is to become a leader of a partition of the set of assets. The leader drone may also communicate with a control center such as a server managing/coordinating performance of the operation. For example, the leader drone may provide feedback information pertaining to the operation (e.g., a real time status of the operation such as an indication of whether the one or more tasks are completed, etc.). As another example, the leader drone may send a request for additional assets or an indication that a subset of the set of assets are released from the operation such as in the case that the subset of assets have completed corresponding elements of the one or more tasks and the capabilities associated with the subset of assets is no longer required to perform the one or more tasks.

According to various embodiments, the set of assets may semi-autonomously react to a failure of one or more assets within the set of assets. For example, in response to a control center (e.g., a server) providing an indication that the set of assets is to perform the operation or one or more tasks associated with the operation, the set of assets may decompose the operation or tasks to determine a plan to be implemented in connection with the performance of the operation or tasks associated therewith. During implementation of the plan, an asset may fail such as by loss of power, loss communication with the set of assets (the leader drone), a crash, or interaction with a third party. In response to a failure of an asset in the set of assets, the remaining assets within the set of assets may continue performance of the operation or the tasks associated therewith without human intervention. The remaining assets may determine that an asset has failed and update the plan for performing the operation or the tasks, for example, to reallocate the elements or functions that were assigned to the failed asset. In response to a determination that the leader drone has failed, another drone in the set of assets may be elevated to role of leader drone among the remaining assets of the set of assets. The drone to be elevated to leader drone may be based at least in part on a pre-determined ranking/priority, etc. In some implementations, a dormant leader is elevated to role of leader drone in response to the current leader drone failing. The dormant leader may correspond to a highest ranking/priority drone among the set of assets (e.g., after the leader drone), such as indicated on the pre-determined ranking/priority, etc. The dormant leader may sync with the leader drone in real-time or at predetermined intervals (e.g., short intervals to ensure that dormant leader may seamlessly take on the role of leader drone, if required).

In various embodiments, the set of assets determined to perform an operation (or one or more tasks associated with the operation) may be updated during performance of the operation. In some implementations, the set of assets may be updated based a characteristic or capability of a task, or an element of a task, that is still to be performed, and/or a characteristics or capability of an asset in the set of assets. For example, if an asset in the set of assets does not have a capability/characteristic of the remaining tasks to be performed, the asset may be released. As another example, if no asset in the set of assets, or an insufficient number of assets in the set of assets, has a capability/characteristics matching the capability/characteristic of the remaining tasks to be performed, then the set of assets may be updated to include one or more assets with capabilities/characteristics matching the capability/characteristic of the remaining tasks to be performed. The update to the set of assets may be determined by a control center (e.g., a server) managing or coordinating the operation, and the update to the set of assets may be communicated at least to a leader drone in the set of assets.

According to various embodiments, one or more user interfaces may be provided in connection with one or more of the configuration of the operation, the communication of a status of the operation (e.g., a real-time update), and/or the update to the operation (e.g., based on user input, etc.). In conventional systems for defining an operation, a user inputs with great-level of specificity the parameters of the operation, the particular assets to be used, and a plan to be implemented. In contrast, various embodiments include the conditional display of various user interfaces to walk a user through the configuration of an operation, a high level plan or definition of the operation is provided to at least the leader drone, and the set of assets (e.g., the leader drone) determines a plan for implementing tasks or elements of tasks deconstructed from the high-level plan or definition of the operation. In this manner, various embodiments provide a user interface wizard to input one or more characteristics or requirements of the operation. The server may cause a client terminal to display the user interface(s) to a user. The terminal may be caused to display first user interface first user interface comprising one or more selectable elements associated with a characteristic of one or more tasks to be performed, and in response to an input to the user interface, the terminal may be caused to display a second interface associated with a set of asset(s) to perform the operation. The second user interface may be configured based on inputs to the first user interface. In response to an input to the second user interface, an operation may be determined. The server may cause the terminal to sequentially configure and provide user interfaces based at least in part on a user input to a previous user interface, and the series of user interfaces may be used to collectively determine an operation to be performed. After an operation is determined and the set of assets are instructed to perform the operation, the server may cause the terminal to provide a user interface comprising a current status of the operation (e.g., a real-time feed, or a real-time update to various parts of the operation), etc. In some embodiments, the user may input one or more inputs to a user interface during performance of the operation in connection with pausing the operation, updating the operation (e.g., requesting quicker completion of the operation such as via an increase in the number of assets in the set of assets, requesting more firepower, request to more quickly cover a geographical area), cancelling the operation, etc.

In some embodiments, the system determines a discrete representation of a geographic location. The geographic location may correspond to a location at which an operation is to be performed, or a location at which a task (or element of the task) of the operation is to be performed. The discrete representation of the geographic location may comprise a plurality of discrete elements each corresponding to a volume in the geographic location. The system may use the discrete representation of the geographic location in connection with determining and/or communicating parameters associated with an operation (e.g., a parameter associated with a task or element of a task), determining a plan to perform a task, determining a flight path (e.g., a trajectory) of an asset such as a drone of the set of assets determined to perform an operation (e.g., a set of one or more tasks, etc.). According to various embodiments, a plurality of assets in the set of assets is configured to determine the discrete representation. For example, an asset running a planning service may be configured to determine the discrete representation of the location corresponding to the operation, or a location of a task or element that has been assigned (e.g., allocated) to the asset. As an example, the discrete representation is determined locally at an asset in the set of assets that is to perform the one or more tasks. In some embodiments, a plurality of assets determine their own respective version of the discrete representation of the location based at least in part on information communicated pertaining to the one or more tasks. As an example, a leader drone and/or a follower drone respectively determines a discrete representation of the geographic location (e.g., a local version of the discrete representation) based at least in part on one or more of (i) information pertaining the one or more tasks such as one or more parameters or characteristics associated with the one or more tasks which is received from a server such as a server associated with the configuration of the associated operation (e.g., an indication of a keep-in/keep-out definition for the one or more tasks, a location of a target, maximum height, etc.), and (ii) information pertaining to the geographic location from a third party service such as a service for map information (e.g., topographical information), a service for weather, a service for information obtained by another asset or sensor in or in proximity to the geographic location (e.g., a camera, a satellite, etc.), and (iii) information pertaining to the geographic location or the one or more tasks from another asset within the set of assets assigned to the operation (e.g., a status of the performance of the one or more tasks, information associated with a communication line of sight, a flight plan of an asset, an indication that a particular location/volume/area is occupied, etc.).

According to various embodiments, determining a discrete representation of a geographic location comprises determining (e.g., creating, generating, populating, etc.) a 3-D representation for the geographic location, the 3-D representation comprising a plurality of discrete elements respectively corresponding to a voxel(s). In some embodiments, the 3-D representation for the geographic location may be determined based at least in part on transforming the geographic location from a real world representation to the plurality of discrete elements based at least in part a curvature of Earth, and a downward direction in a grid of the 3-D representation corresponds to a direction of a force of gravity. The dimensions of the 3-D representation may be configured to include a substantially larger number of discrete elements (e.g., boxes) in the x and y axis as compared to the number of discrete elements in the z axis. The z axis may be an axis that is parallel to the force of gravity. As an example, if an asset traversing down the 3-D representation in the z-direction, the asset travels down towards the ground, and conversely, if the asset traverses up the 3-D representation in the z-direction, the asset travel towards the sky. The length or dimension of the discrete representation in the z-axis may be configured based at least in part on a configuration of a maximum height that is set in connection with configuration of the operation. For example, the system may have a default maximum height associated with an operation (or associated with different types of operation). As another example, a user may input the maximum height during the configuration of the operation.

According to various embodiments, the system (e.g., a server, a leader drone, a follower drone, etc.) generates a model of the geographic location associated with the one or more tasks. The generating the model of the geographic location may include determining discretized representation of the geographic location, and annotating the discretized representation to create an annotated representation. For example, the annotated representation may correspond to the model of the geographic location. In some embodiments, the system updates the annotated representation based at least in part on received information pertaining to the one or more tasks and/or the geographic location. For example, the system may iteratively update the annotated representation at periodic intervals, as updated or new information pertaining to the one or more tasks and/or the geographic location is received, etc. The annotated representation may be updated until the one or more tasks are complete (e.g., until a determination that the operation is complete, abandoned, paused, etc.). In some embodiments, the annotating the discretized representation of the geographic location includes respectively associating metadata with one or more discrete elements of the discretized representation. The metadata may include a value or indication associated with the one or more tasks, the geographic location, etc.

Various embodiments for managing, coordinating, and/or performing an operation improve an efficiency with respect to a use of resources (e.g., assets) assigned to perform an operation, improve a fault tolerance of the performance of the operation by enabling dynamic grouping of assets and/or dynamic updating of a plan to perform a task/element associated with an operation. Various embodiments improve the efficiency and effectiveness of defining an operation, and matching assets for performing the operation, and in determining a manner according to which the operation is to be performed. In related art methods, a human operator selects assets to use to perform an operation and a centralized service (e.g., running on a server) defines an operation in great detail. The centralized service provides detailed operation plans in order to perform the operation. Various embodiments include a system of user interfaces that are configured to intuitively walk a user through the providing a high level description of a desired operation. Various embodiments provide an efficient method for grouping a set of assets to perform an operation. The grouping of assets may be based at least in part on an automated negotiation of ownership availability, and capabilities associated with the operation and capabilities of the assets. The grouping of assets is more efficient and provides better organized task execution. In related art, a system for defining an operation generally requires a human operator to select an asset to use to perform an operation, and the set of assets available for selection by the human operator may be further limited because the availability of ownership is not readily recognizable. In some embodiments, the system for performing the operation is more fault tolerant. For example, the use of a leader drone (e.g., a semi-autonomous drone) in connection with determining one or more tasks to be performed and for coordinating/managing the performance of the tasks by one or more assets in the set of assets (e.g., follower drones) provides a more efficient manner for dynamically updating plans to performs a task or element of a task associated with an operation. As another example, a dynamic grouping of assets for performing an operation is more efficient because assets having capabilities no longer required in performing a remaining set of tasks may be released from the set of assets (and rendered available for reallocation to another operation). The dynamic grouping of assets is also more fault tolerant because in response to a failure of an asset or a change in a context of an operation, an additional asset may be allocated to the set of assets. The dynamic management/coordination of the performance of the operation (e.g., by the leader drone) enables tasks to be quickly reassigned to other asset(s) in the set of assets in response to a change in a capability of an asset and/or a change in a context of the operation such as the environment, etc.

is a diagram illustrating a system to perform an operation according to various embodiments of the present application. In the example illustrated in, systemmay include serverand one or more of assets,, and. Systemmay further include network, network, and/or client terminal. Serverand assets,, and/may communicate with each other such as via one or more networks (e.g., network, network, and/or any other appropriate network), which may include a wired network, and/or a wireless network such as a cellular network, a wireless local area network (WLAN0), etc. According to various implementations, servermay correspond to a single server or a plurality of servers. Similarly, systemmay include various other assets.

In some embodiments, servermay configure an operation, including determining a set of assets to perform the operation. The set of assets may include one or more of assets,, and/or. Servermay communicate to at least one asset of the set of assets an indication that the asset is to perform the operation and/or an indication of the set of assets (e.g., a list of assets within the set of assets, such as a list of identifiers of the assets). In some embodiments, the indication that the asset is to perform the operation may include a high-level definition or description of the operation, such as a high-level task to be performed (e.g., surveillance of an area or a target, delivery of a payload, etc.). At least one asset within the set of assets (e.g., a leader drone) may deconstruct the high-level definition or description of the operation and autonomously determine plan for at least part of the set of assets to perform/complete the operation. The high-level definition or description of the operation may further include certain parameters of the operation such as a location, an indication of a restricted zone in which the assets are not to enter/operate, an indication of an area/zone in which the assets are to operate, etc. After receiving the high-level definition or description of the operation, at least one asset (e.g., the leader asset) may autonomously determine a plan for at least part of the set of assets to perform/complete the operation, the at least one asset may cause the set of assets to implement the plan, and communicate information pertaining to the operation (e.g., a current status of the operation, etc.) to servervia networkand/or network. For example, the leader drone may deconstruct the high-level definition or description of the operation and determine a plan for one or more tasks corresponding to the operation. In some embodiments, a discretized representation of the geographic location associated with the one or more tasks is generated. As an example, the discretized representation of the geographic location is generated based at least in part on information pertaining to the geographic location and/or the one or more tasks. The information is received from serverand/or another asset in a set of assets (e.g., a follower asset, a leader drone, etc.). In some embodiments, the plan for the one or more tasks is determined based at least in part on the discretized representation. For example, the discretized representation is annotated to create an annotated representation. As an example, the annotating the discretized representation includes setting or associating metadata with one or more discrete elements of the discretized representation. In some embodiments, the asset among the set of assets (e.g., asset, asset, and/or asset), and/or serverdetermines the plan using the annotated representation such as by using information comprised in the metadata associated with a discrete element (e.g., to determine a trajectory or flight path of an asset, etc.). In response to determining a plan for the one or more tasks, the leader drone may send the plan for at least one task to another asset in the set of assets (e.g., a follower asset). In response to receiving the plan for the at least one task, the follower asset may implement the plan and may autonomously determine any additional instructions to be performed locally in order to perform the at least one task. The set of assets to perform an operation may be dynamically updated (e.g., during performance of the operation) to add an asset (e.g., in the case of a failure of an asset having a particular capability that is to be replaced by a new asset), to remove an asset (e.g., in the case of completion of any task requiring a particular capability for which the asset was included in the set of assets to provide), etc.

In various embodiments, servercommunicates with client terminalvia a network such as network. Servermay cause client terminalto display one or more user interfaces in connection with configuring the operation, displaying a status of the operation, and/or updating the operation (e.g., pausing, cancelling, or modifying the operation). Client terminalmay receive one or more user inputs to the various user interfaces and communicate an indication of the one or more user inputs to server. Servermay use the one or more user inputs to determine the various user interfaces to be displayed at client terminal, such as in connection with configuring the various user interfaces to create a wizard with which a user defines or inputs characteristics of the operation. During performance of the operation, servermay use a user input to change a status or parameter of the operation, and in response to such a change, servermay communicate the change to at least the leader asset/drone (e.g., to cause the leader drone to implement the change). For example, in response to changing a status or parameter of the operation, servermay update a high-level definition or description of the operation, and serverprovide a corresponding high-level instruction or update to the leader drone in the set of assets performing an operation. In some embodiments, serverreceives information pertaining to the operation (e.g., feedback information) from the set of assets, such as from the leader drone. The information pertaining to the operation may be indicative of a status of the operation, a real-time image or video of the operation, etc. In response to receiving the information pertaining to the operation, servermay provide a status update, a real-time operation status to client terminal. In some embodiments, servermay determine one or more recommendations or options for updating the operation, and provide at least one of the recommendations or options to a user via a user interface displayed at client terminal. As an example, the one or more recommendations or options may include an indication an option to increase a number of allocated assets in the set of assets to increase the speed with which the performance of the operation (e.g., indicate that adding a particular number of assets may decrease the time to completion by a computed estimated time).

is a block diagram illustrating a device to configure or control an operation according to various embodiments of the present application. In the example shown in, devicemay comprise a communication interfaceand/or one or more processors. Devicemay correspond to serverof, control centerof, and/or control center of. Devicemay implement processof, processof, processof, and/or processof, processof. According to various embodiments, one or more processorsmay comprise or execute one or more of communication module, asset ownership module, mapping of capabilities to assets, operation definition module, grouping module, pre-planning module, feedback information module, and user interface module.

Devicemay implement one or more modules in connection with determining an operation, including one or more characteristics of the operation, and determining a set of assets to perform the operation, and communicating an indication of the operation (e.g., a high level description of the operation) to one or more assets in the set of assets (e.g., the leader drone). During the implementation of the operation, devicemay implement one or modules in connection with communicating information with at least the leader drone to receive information pertaining to a current status of the operation or to send an indication of a change to the operation, and/or communicating information a client terminal to provide an update on the status of the operation or to receive a change to the operation.

Devicemay use communication moduleto communicate with the set of assets, a client terminal, another server or terminal, etc. For example, the communication module may provide to communication interfacethat is to be communicated. As another example, communication interfacemay provide to communication moduleinformation received by device.

According to various embodiments, the determination of the set of assets to perform an operation is based at least in part on an ownership of one or more assets in the set of assets. Devicemay determine to include in the set of assets only those assets for which devicehas ownership (e.g., an organization to which devicebelongs owns or controls the asset), or for which devicemay obtain at least a temporary ownership of the asset. Ownership of the asset may include the ability to control the asset, such as providing an instruction to the asset to perform a certain function (e.g., a task, an element of a task, etc.). As an example, ownership of an asset may include a lease during which the owner has the ability to control the asset. A lease may be for a fixed/pre-determined term, or associated with completion of a particular operation(s). In some embodiments, the transfer of ownership (e.g., in connection with the negotiation of ownership for determining the set of assets) is a permanent transfer of ownership, or otherwise for an undefined term, or until deviceor its organization releases ownership of the asset. In some implementations, the set of assets is configured to cause the leader drone to be subject to a term of ownership (e.g., length of lease, permanent ownership) that is longer than the terms of ownership of other assets in the set of assets (e.g., the follower drones). In some implementations, if the operation is determined to include a relatively higher likelihood that a follower drone may lose communication during performance of the operation (e.g., based on a terrain of the area in which the operation is performed, etc.), devicemay configure the set of assets to cause follower drones to have longer terms of ownership. The longer term of ownership may ensure that ownership over the follower drones survives the period(s) when the follower drone(s) are out of communication with the leader drone and/or device, etc. In some embodiments, the ownership of an asset includes a failover entity to which ownership of the asset is transferred in the event that the current entity having ownership fails. Information pertaining to ownership of an asset may be stored at one or more of (i) locally at the asset, (ii) a centralized ownership service such as a service hosted by one or more servers providing service across a plurality of organizations, (iii) device, and (iv) an owner of the asset. The information pertaining to an ownership of an asset may include one or more of an indication of the current owner, an indication of a length of current ownership, an availability of an ownership transfer such as an indication of an ability for another organization to acquire ownership (e.g., a willingness of a current owner to transfer ownership), a current failover owner, an indication of a protocol for negotiating transfer of ownership, etc. In some embodiments, the ownership of an asset may comprise an automatic take over ownership in which a device or organization requesting ownership (e.g., having appropriate permissions) may automatically take over ownership of the asset such as upon request for ownership.

In some embodiments, devicemay use asset ownership modulein connection with determining an ownership of a particular asset and/or to negotiate obtaining ownership of an asset. Asset ownership modulemay obtain the information pertaining to ownership of an asset, and devicemay use such information in connection with determining the set of assets. For example, devicemay use the information pertaining to ownership of the asset to determine whether a particular asset is available for allocation/assignment to the set of assets to perform the operation. Asset ownership modulemay negotiate with an entity for transfer of ownership of the asset for use in connection with the operation. For example, asset ownership modulemay request a lease for a predetermined term. The predetermined term may be an expected length of the operation, an expected length of time that the asset is expected to be required to perform a particular task or element of a task (e.g., a period of time for which the capability of the asset is needed in performance of the operation), etc. Asset ownership modulemay request the information pertaining to ownership from the particular asset, or a particular asset may advertise information pertaining to ownership of such asset. In some embodiments, devicesends a request/instruction to obtain ownership of an asset in response to a determination of a set of assets to be used in connection with performing the operation. For example, devicemay send the request to obtain ownership of an asset in conjunction with (e.g., as part of, or along with, etc.) providing to an asset an indication of the set of assets, such as an instruction indication indicating that the asset is included in the set of assets. As a further example, devicemay send the request to obtain ownership of one or more assets in connection with providing the leader with an indication of the set of assets, and the leader can thereafter communicate with one or more assets in the set of assets in connection with obtaining the applicable ownership of the one or more assets (e.g., for performing at least a task in the operation).

According to various embodiments, the set of assets to be used in connection with performing the operation is determined and/or updated based at least in part on a capability of the corresponding assets included in the set of assets. In connection with configuring the operation, devicemay determine a set of capabilities associated with performing the operation (e.g., capabilities required to perform the various functions comprised in performing the operation). Devicemay use the set of capabilities associated with performing the operation in connection with determining the set of assets. For example, devicemay match assets within a superset of assets that have capabilities matching set of capabilities associated with performing the operation. Devicemay query a mapping of capabilities to assets (or to asset identifiers) in connection with determining the superset of assets having capabilities matching at least one capability in the set of capabilities associated with performing the operation. In response to a determination that a capability of the operation does not have a matching asset (e.g., an asset with the corresponding capability) within a superset of available assets, devicemay determine whether two or more assets may be used to collectively attain the capability or whether to reconfigure the operation. For example, devicemay provide to the client terminal/user an indication that a capability of the operation has no matching asset and/or prompt the user for an indication of whether to reconfigure the operation (e.g., based on further user input), or to cancel the operation.

In some embodiments, devicemay use mapping of capabilities to assetsin connection with determining a capability of an asset. The mapping of capabilities to assetsmay store a mapping of such capabilities locally, or may communicate with a remote service storing the mapping. Mapping of capabilities to assetsmay query the mapping with a particular identifier for an asset to lookup the capabilities for such asset. Similarly, mapping of capabilities to assetsmay query the mapping of a particular capability to lookup the assets having such a capability. In some implementations, in response to receiving one or more parameters of an operation, devicemay determine one or more capabilities associated with the operation, and use mapping of capabilities to assetsto determine assets that have matching capabilities. For example, devicemay determine the set of assets to perform the operation based at least in part on a determination that the set of assets collectively have capabilities matching the capabilities associated with performing the operation. Devicemay determine the set of assets such that at least one asset in the set of assets has at least one capability matching a capability associated with performing the operation. Mapping of capabilities to assetsmay be stored locally at device, or a module for obtaining a particular mapping of capabilities may be used to query a remote service providing the mapping of capabilities to assets. The mapping of capabilities to assets may be updated in response to a determination that a capability of an asset has changed. For example, an asset may advertise a new capability, a loss of a capability (e.g., in response to detecting a failure of a peripheral), and/or an update to a capability. In some implementations, devicemay update the mapping of capabilities to assets. In some implementations, a remote service (e.g., a service hosted by a server) updates the capabilities mapped to a particular asset, and devicemay periodically sync updates to the mapping of capabilities to assets, or devicemay query the mapping of capabilities to assets, as necessary.

In some embodiments, an asset stores an indication of its capabilities. For example, an asset can locally store a capability definition and the asset may update the capability definition based on a current capability of the asset such as in response to failure of a module that provided a capability (e.g., a peripheral such as a sensor, a camera, or an added gimbal, etc.), addition of a new module (e.g., addition of a new peripheral, new software being loaded on the asset, an update to existing software, etc.). The capability definition may be a list of capabilities of an asset, or an indication of capabilities according to a predetermined format/protocol. In some implementations, an asset may advertise its capabilities such as via broadcast of the capability definition across one or more networks or to a leader drone during implementation of a plan/operation, etc., or the asset may provide an indication of its capabilities in response to a query from a server (e.g., a server configuring an operation), a centralized capability server, a leader drone, etc. In some implementations, device(e.g., mapping of capabilities to assets), or a remote service that manages a mapping of capabilities to assets, may ping one or more assets for an update with respect to a current capability of the one or more assets, or otherwise provide a ping for a health update of an asset, etc., and the corresponding mapping of capabilities to assets may be updated (e.g., to reflect a current set of capabilities for the one or more assets and/or a state of the capabilities).

According to various embodiments, devicemay define one or more operations to be performed by one or more assets. For example, definition of an operation may include at least one of setting one or more objectives associated with the operation, setting a location at which the operation is to be performed, setting a target of the operation, setting one or more restrictions with respect to performance of the operation (e.g., a rule of engagement for engaging the target, a restricted or permitted airspace, etc.), setting a date of an operation, etc. The one or more operations may be defined based at least in part on one or more user inputs, such as a user selection input to a user interface provided by user interface module. In some embodiments, deviceuses operation definition moduleto define the one or more operations. For example, operation definition modulemay receive the one or more user inputs by the user to the user input from communication moduleand/or user interface module, and operation definition modulemay determine one or more characteristics/parameters associated with the operation in connection with defining the operation. An example of a definition of an operation may include an operation to find or track a particular target (e.g., human XYZ, truck ABC, etc.) in Los Angeles, from Jan. 1, 2021 to Jan. 3, 2021, the assets are not to travel 100 miles outside Los Angeles, or enter a permanent prohibited area or within 2 miles of an airport. In some embodiments, the one or more user inputs associated with the operation may include an indication of one or more types of assets that are to be used in connection with performance of the operation. Continuing with the foregoing example, the definition of the operation may further include an indication that the operation is to be performed using a drone and/or a fixed-wing aircraft.

In some embodiments, the operation may be deconstructed into a set of one or more associated capabilities or characteristics. For example, the definition of the operation (e.g., the operation defined using the operation definition module) may be analyzed and one or more capabilities associated with the operation may be determined based on the analysis. The set of one or more capabilities associated with the operation may be determined based at least in part on querying a mapping of operation parameters to capabilities. The parameters associated with an operation may be determined based on the definition of the operation. Examples of the operation parameters include a type operation (e.g., scanning road network, find, fix, track, loiter, scan along path, etc.), a operation classification (e.g., offensive, defense, reconnaissance, etc.), a location, a path along which a task is to be performed (e.g., a path for tracking/scanning), a type of asset (e.g., drone, fixed wing, helicopter, boat, etc.), a target (e.g., a person, an air vehicle, a ground vehicle, an unknown type of target, etc.), a range for performing the operation, a location of the operation, a rule set for performance of the operation, an indication of what can/cannot be performed (e.g., permitted actions), survivability during the operation (e.g., how likely is it that the asset be shot down or otherwise terminated), communication requirements (e.g., an indication of when the asset can engage in a communications silent mode or lose link, an indication of the type of communications to be used, or other parameters or restrictions on communications during the operation, etc.), target behaviors (e.g., an indication of expected the target movement such as whether targets are static, moving on roads, movement patterns, etc.), potential threats (e.g., locations of sites that would be able to disrupt or disable the drone such as a SAM (surface to air missile launcher), an RF Jamming site, etc.), unlikely target zones (e.g., areas that are not expected to include any targets of interest as specified by the user, etc.), a maximum altitude (e.g., height) at which an asset is to travel, a keep-in area (e.g., an area in which the asset is to remain), a keep-out area (e.g., an area from which the asset is to be prevented/prohibited from entering), etc. In some implementations, operation definition moduledetermines the one or more capabilities associated with the operation. In other implementations, grouping moduledetermines the one or more capabilities associated with the operation.

According to various embodiments, devicedetermines a set of assets to perform the operation (e.g., a team of assets). Devicemay determine the set of assets to perform the operation automatically in response to an operation being defined (e.g., based on one or more user inputs. The set of assets may be determined based at least in part on one or more (i) operation parameters, (ii) one or more capabilities associated with the operation, (iii) a capability of one or more assets, and (iv) an ownership of one or more assets. Various other factors/variables may be used in connection with determining the set of assets to perform the operation. In some embodiments, devicedetermines the set of assets based on a best fit based at least in part on one or more factors/variables associated an asset and/or the operation. For example, devicemay use a cost function to determine the set of assets. For example, the cost function may comprise weightings for various factors used in determining the set of assets. Devicemay determine a set of assets based on a minimization of the overall cost value determined by the cost function for the combination of assets in the set of assets. In some implementations, the set of assets may be determined such that the corresponding aggregated cost value using the cost function is below a threshold cost value. The threshold cost value may be configurable (e.g., by an administrator, a user, etc.), such as in association with definition of the operation. In some implementations, the threshold cost value is determined to correspond to a predefined percentile with respect to all possible sets of assets (e.g., so that the set of assets to perform the operation is in top 10 percent of all possible sets of assets, or other configurable percentile). One example of a cost function in a context including pairing assets to road assignments may comprise a weighting of a) distance from the road, b) probability that a target exists on the road. Using the aforementioned cost function, if there are assets that are far from the roads then the assets will not be selected because the distance component of the cost function will remove the assets from being desirable, etc. Similarly, if there are low probability roads the low probability roads will be scanned in the future, and higher priority roads will take precedence.

In some embodiments, the determining the set of assets may include determining a number of assets to include in the set of assets. The number of assets to include in the set of assets may be determined based at least in part on the operation (e.g., a capability associated with the operation) and a capability of an asset. For example, the determining the number of assets may be based on the cost value of a grouping of assets, and/or a set of capabilities provided by the grouping of assets in relation to the capabilities associated with the operation. As discussed further below, the number of assets to be included in the set of assets may be based at least in part on a determination to include in the set of assets a redundancy with respect to one or more capabilities among the assets in the set of assets and/or an extent to which redundancy is to be included among assets in the set of assets.

Devicemay use grouping moduleto determine the set of assets. Grouping modulemay determine the set of assets based at least in part on the definition of the operation. For example, the set of assets may be determined based at least in part on the set of one or more capabilities associated with the operation, and/or one or more operation parameters. Grouping modulemay determine a set of one or more assets having capabilities matching the set of one or more capabilities associated with the operation. As an example, grouping modulemay determine the set of assets such that for each capability in the set of one or more capabilities associated with the operation the set of assets comprises at least one asset with a matching capability.

In some embodiments, devicedetermines a pre-plan of the operation in connection with determining the set of assets to perform the operation. Devicemay use pre-planning moduleto determine the pre-plan of the operation. The pre-plan for an operation may be a high-level description or decomposition of the operation (e.g., into a set of tasks or elements of tasks to be performed) to identify what is to be done during implementation of the operation. The generation of a pre-plan in connection with determining the set of assets to be used in the performance of the operation may allow grouping moduleto assess the capabilities associated with the operation (e.g., the various capabilities that are required to implement the operation) and/or properties or characteristics of an asset that are required or desired for performance of the operation (e.g., the set of tasks or elements of tasks to be performed in the operation). In some implementations, in response to determining the pre-plan of the operation, devicedetermines assets having one or more capabilities of matching the capabilities or requirements of the operation.

Devicemay determine to include redundancy in the set of assets for at least one of the one or more capabilities associated with the operation. For example, devicemay use grouping moduleto determine to include redundancy in the set of assets based at least in part on the type of operation, operation classification, location of operation, etc. (e.g., if the type of operation/operation classification/location is associated with a corresponding likelihood that an asset will fail that exceeds a threshold likelihood, or otherwise associated with an elevated risk of failure). If the operation is an offensive operation, or in a hostile environment, the likelihood that an asset fails may be elevated. The extent of redundancy that devicebuilds into the set of assets may be based on an extent to which an asset is likely to fail. For example, a plurality of thresholds may be defined, each threshold with a corresponding likelihood of asset failure, and the plurality of thresholds may be used in connection with determining an extent of redundancy to be built into the set of assets (e.g., an extent of redundancy may be mapped to a particular threshold corresponding likelihood of asset failure, and the mapping may be queried to determine the extent of redundancy to be applied to the set of assets). The extent of redundancy may indicate a number of assets to be included in the set of assets that each having a particular capability, such as a capability mapped to elevated risk of asset failure or a capability on an asset that is identified as having an elevated risk of asset failure.

Devicemay communicate an indication of the set of assets in response to determining the set of assets to be used in connection with performing an operation. For example, in response to grouping moduledetermining the set of assets, deviceuses communication moduleto communicate an indication of the set of assets. In some embodiments, devicesends to at least the leader (e.g., the leader drone) in the set of assets. The leader drone may in turn provide to the other assets in the set of assets an indication that the other assets are in the set of assets. For example, the leader drone may inform each asset that they are on the team of assets (e.g., in the set of assets) for an operation. As another example, the leader drone provide to each asset in the set of assets an indication of the assets in the assets, so that each member knows, for example, the entire set of assets in the team to perform the operation. The indication provided to the leader drone or to one or more other assets in the set of assets may be communicated in conjunction with, or separate from, a task or plan associated with performing the operation.

According to various embodiments, a pre-plan for performing the operation may be determined via a remote service and thereafter provided to the set of assets, such as at least the leader drone. The remote service to determine the pre-plan may be performed by device, such as via use of pre-planning module, or via one or more third party pre-planning modules with which pre-planning module, or device, may communicate. For example, the pre-planning modulemay comprise an application programming interface (API) with which the one or more third party pre-planning modules interface with deviceto provide a pre-plan associated with the operation. The pre-planning moduleand/or third-party pre-planning modules may obtain the definition of an operation and/or one or more parameters associated with the operation. In some embodiments, the determining the pre-plan for an operation is an iterative process of determining a pre-plan, providing information pertaining to the pre-plan to a user such as via user interface module(e.g., causing the pre-plan to be displayed on a client terminal), receiving feedback information pertaining to the pre-plan (e.g., one or more inputs with respect to a parameter of the pre-plan or the associated operation, and updating the pre-plan based on the feedback information. The updated pre-plan may thereafter be provided to a user, and feedback information may be received with respect to the pre-plan for further refinement. The providing of the pre-plan and receiving of feedback information may be iteratively performed, such as until a user provides an indication that the operation is to be initiated. As an example, devicemay determine a pre-plan indicating that the set of assets will track a target along a path, or will otherwise travel along a particular path, and devicemay cause the path to be displayed to the user at a client terminal. The user may provide one or more inputs modifying the path of the set of assets, and devicemay update the pre-plan accordingly. As another example, devicemay determine a pre-plan indicating the manner in which the asset will fly when scanning some region. For example, in the case of performing or analyzing a lawn service, if may be useful for the operation to see the back and forth lawn mowing patterns on a polygon and potentially manner of flying to scan the region if the original polygon shape does not result in very optimal scanning. For example, if a user performed lawn mowing vertically but built a long horizontal polygon to scan/analyze the lawn mowing, performing the scanning according to the original long horizontal polygon may result in a significant overshoot and back and forth of the asset, so the user or the drone might adjust to a horizontal scanning pattern in this case.

In some embodiments, devicesends a pre-plan according to which the set of assets is to perform the operation, and the leader asset may determine the plan which the set of assets is to perform the operation based on the pre-plan. As an example, device may determine and send the pre-plan if the operation is defined to ensure that the set of assets performing the operation are to adhere to a framework of a plan. The pre-plan may be a high level definition or description of what is to be accomplished by the set of assets or a subset thereof, and the leader drone may determine a lower-level plan that further specifies the manner by which the set of assets are to perform the operation (or a task thereof). The leader drone may in turn provide the plan for performing the operation to another asset in the set of assets (e.g., a follower drone(s)), which may in turn determine an even more specific plan for performing the plan provided by the leader drone. As an example in the context of a surveillance operation, device(e.g., a server) may determine a pre-plan indicating the path or perimeter that the set of assets are to travel to obtain surveillance information for a certain location, and devicemay provide the pre-plan to a leader asset in the set of assets (e.g., a leader drone). In response to receiving the pre-plan, the first asset may decompose the perimeter or the path in the pre-plan into smaller segments, assign such segments to various follower assets (e.g., follower drones), and provide the various follower assets with an instruction to travel to the segments correspondingly assigned to the follower assets to capture surveillance information with respect to the segment. Continuing with the example, the various follower assets may receive an instruction to capture surveillance information with respect to a particular segment, and the various follower assets may determine a specific manner by which to travel the segment and capture the surveillance information (e.g., the follower drone may plan a route to avoid obstacles along the route, such as trees, etc.).