Synchronization Protocol in a Cloud Computing Environment

With computer and Internet use forming an ever greater part of day to day life, security exploits and cyberattacks directed to stealing and destroying computer resources, data, and private information are becoming an increasing problem. Some attacks are carried out using “malware”, or malicious software. “Malware” refers to a variety of forms of hostile or intrusive computer programs that, e.g., disrupt computer operations or access sensitive information stored on a computer (e.g., viruses, worms, Trojan horses, ransomware, rootkits, keyloggers, spyware, adware, or rogue security software). Malware is increasingly obfuscated or otherwise disguised in an effort to avoid detection by security software. Determining whether a program is malware or is exhibiting malicious behavior can thus be very time-consuming and resource-intensive.

A computer may recognize malware in data by receiving events over a network from a client device and analyzing the events for malicious activity by a threat actor. However, events can be lost, out of order, or otherwise fail to be received by the computer for a variety of reasons thereby causing the computer to fail to accurately analyze all the events for potential malicious activity. In the time it takes for the computer to eventually receive all the events using typical approaches, the malicious activity that had gone undetected remains a security threat to the client device.

This application describes a protocol for exchanging data between a host device and a computing device in a cloud computing environment. The protocol can include a schema for identifying and/or tracking data packets associated with one or more events. The protocol can improve reliability of data exchanges by synchronizing data between the host device and the computing device. The host device can transmit portions of a data file (e.g., data packets, events, etc.) to the computing device using the protocol to enable the computing device to determine a sequence for the portions or to initiate recovery of a portion, for example. By using the protocol as described herein, a data file can be synchronized from a first device to a computing device in a cloud computing environment in less time and with more accuracy (e.g., relative to not using the protocol).

The techniques described herein can include synchronizing data based on descriptions for respective portions of a data file. For example, data from a host device may be sent to a computing device (e.g. a cloud service, a cloud computing device, etc.) configured to receive of determine data indicating a replicated state of activity occurring on the host device over a time period. The computing device can, for example, determine information such as a sequence number, hash value, payload, etc. associated with various portions (e.g., data packet, event, data string, byte slice, byte array, event stream, or the like) of the data. In some examples, the computing device can combine, or “stitch together”, the individual portions of data based on the respective information. In some examples, the host device can represent an endpoint and the techniques can be used to transfer, migrate, or otherwise replicate data from the endpoint to a storage device associated with a cloud computing environment.

In various examples, the computing device can detect a difference between the data sent from the host device and the data received by the computing device. The computing device can compare respective information associated with different portions of the data to determine whether a portion is out of order, missing, extraneous, etc. In some examples, the computing device can initiate recovery of a portion to complete synchronization. Additionally or alternatively, the computing device can initiate a request for a particular portion of data to mitigate a dropped data packet, missing data packet, etc.

In some examples, the computing device can validate that the combined data portions are the same as the data stored in a storage device associated with the host device. For instance, the computing device can determine that each of the portions of data received from the host device are in sequence and are otherwise identical to the portions sent from the host device. The computing device can store the sequenced, combined data in a storage device associated with the computing device (e.g., in a cloud computing environment) and/or output the stored data (or portions thereof) for access by a model or component for analysis such as to detect presence of malicious activity associated with the data received from the host device.

By way of example and not limitation, a host device can prepare a JavaScript object notation (JSON) file for sending to a cloud service (e.g., a data synchronization service, a security service, etc.) The host device can, for example, determine portions, data packets, events or the like, for representing the JSON file. The host device can also or instead determine a hash value for the data file, a sequence number, an index number, a payload, etc. for associating with each data packet sent to the cloud service. The techniques described herein can include the cloud service managing the transfer of data associated with the JSON file from the host device to a storage device associated with the cloud service.

In various examples, the techniques described herein can include the host device serializing the JSON file into a payload, transmitting the payload as separate data packets, combining the data packets to replicate the payload, and deserialize the payload for use in a cloud computing environment (e.g., generate deserialized data). In various examples, the information, identifiers, hash values, etc. enable analysis of the data packets to determine whether the JSON file that is deserialized is functionally equal to the JSON file that was serialized for sending by the host device.

The schema can represent an architecture for exchanging data between two devices or components. For example, the schema can include defining portions of data, determining information describing the portions of data, and performing various actions or operations based on the determined information. The schema can represent a data structure for storing the data in a database and/or for executing the data by a processor. The schema can implement a synchronization protocol to format the information associated with various portions of data for transmission. The synchronization protocol can include a set of rules, policies, or the like, for communicating data over one or more networks, for example. In some examples, the protocol can provide a communication channel for sending or receiving data defined by the schema.

The information (or schema) described herein can enable a variety of operations including, for example, determining information for a data packet including a unique identifier for each data packet, a sequence of a data packet relative to another data packet, a binary payload, an identifier of the host device, among others. In some examples, the schema can be used to cause the host device to send identifiable portions of data for reconstruction by a computing device in another environment. The schema may also or instead be used to detect an order for the data packets, a number of data packets associated with a particular JSON file, and/or a sequence for the data packets. In some examples, the schema may be used to request a data packet that was not received by the computing device such as due to a network failure. Details for preparing a JSON file (or other input data) for sending to another device are discussed throughout this disclosure.

In various instances, a computing device may install, and subsequently execute, a security agent as part of a security service system to monitor and record events and/or patterns on a plurality of computing devices (e.g., a host device, a client device, etc.) in an effort to detect, prevent, and mitigate damage from malware or malicious attack. In various examples, the security agent may detect, record, and/or analyze events on the computing device, and the security agent can send those recorded events (or data associated with the events) to a security system implemented in the “Cloud” (the “security system” also being referred to herein as a “security service system,” a “remote security service,” or a “security service cloud”). At the security system, the received events data can be further analyzed for purposes of detecting, preventing, and/or defeating malware and attacks. The security agent can, for instance, reside on the host device, observe and analyze events that occur on the host device, and interact with a security system to enable a detection loop that is aimed at defeating all aspects of a possible attack.

Some examples herein may relate to detecting activity from a threat actor accessing, or attempting to access, data associated with a host device, such as using a protocol for transmitting the data from the host device to a security service for analysis. For brevity and ease of understanding, as used herein, “suspicious” refers to events or behavior determined using techniques described herein as being possibly indicative of attacks or malicious activity. The term “suspicious” does not imply or require that any moral, ethical, or legal judgment be brought to bear in determining suspicious events.

In various examples, a system comprising a synchronization component can determine information to associate with data packets, events, or the like. For example, the system can determine information for portions or data packets representing a data file, a data string, byte slice, byte array, or data stream, just to name a few. By analyzing the information output by the system as described herein, a same or different system can determine presence of potential security threats (e.g., an unauthorized process, thread, executable, or other activity) in the input data and/or in subsequent data received at a later time.

In some examples, the system can automatically and proactively determine and/or identify information for various data packets independent of user input and/or a need to maintain a particular communication channel. Data packets can be sent over a network using various techniques, and be combined into a sequence that matches the initial sequence of data packets sent from the host device. In some examples, the system can provide functionality to cause a host device to improve analysis of data strings by the host device (e.g., to detect malicious activity by synchronizing data more efficiently).

In various examples, first data for sending from the host device and/or second data received by the computing device may be stored in a storage device for a period of time to enable access to missing data packets, for example. The stored information can be updated, deleted, added, or otherwise managed over time to maintain a list of data packets and associated information that can be provided to the various devices periodically and/or upon request. Additional description for determining metadata, or similar information, can be found throughout this disclosure including in the figures below.

Outputting the information by the system enables improved detection, remediation, and analysis of portions that represent a data file (versus not implementing the system). In various examples, the system can be implemented as a cloud-based service configured to synchronize data for storage and further analysis such as to improve subsequent detection of malicious events (e.g., by improving how portions are identified, combined, etc.).

In various examples, the system can receive, as input data, a portion of the data stream from a storage device (or receive the portion in real-time independent of the database), such as a data stream database that receives (and in some instances replicates) all data associated with the data stream. By using the techniques described herein, data usable for protecting the host device and/or the data stream can be identified in less time and with more accuracy (e.g., versus relying on a human to analyze and convey the analyzed data to a user of the host device).

The system can employ a variety of different models to perform the techniques described herein. As described herein, models may be representative of machine learned models, statistical models, heuristic models, or a combination thereof. That is, a model may refer to a machine learning model that learns from a training data set to improve accuracy of an output (e.g., a prediction). Additionally or alternatively, a model may refer to a statistical model that is representative of logic and/or mathematical functions that generate approximations which are usable to make predictions.

The techniques can be used to reduce computational resources required to determine that data received from a device has been replicated or otherwise received in a cloud computing environment. For example, the techniques can perform synchronization using fewer memory and/or processor resources by using a schema that enables comparison of data packets more efficiently. The schema can be used to speed up recovery of dropped packets or otherwise improve an amount of time required to arrange received data packets as replicated data.

The techniques described herein can improve the quality of data transmitted to synchronize data by reducing an amount of data transmitted over a network in association with a data synchronization. For instance, the techniques can improve network efficiency (e.g., save network bandwidth, free up memory and/or processor resources, etc.) by tracking data packets more effectively thereby requiring fewer requests for missing data or to rearrange data in a correct sequence.

The techniques described herein can also or instead improve functioning of a computing device by providing a scalable and efficient method for synchronizing data.

Although in some examples the system comprises a computing device and a host device, in other examples, the system may enable the techniques described herein to be performed by the host device independent of the computing device and/or independent of a network connection. That is, either the host device and/or the computing device may implement one or more components and/or models to generate data packet information usable to replicate data, and to analyze the replicated data to prevent potential malicious activity at the host device. In various examples, the data packet information can be generated independent of requiring a network connection, and can be optionally stored in a storage device for a threshold period of time (e.g., to retrieve missing data).

As used herein, the term “adversaries” or “threat actor” includes, e.g., malware developers, exploit developers, builders and operators of an attack infrastructure, those conducting target reconnaissance, those executing the operation, those performing data exfiltration, and/or those maintaining persistence in the network, etc. Thus the “adversaries” can include numerous people that are all part of an “adversary” group.

Some examples relate to receiving or processing a data string, byte slice, byte array, event stream, data sequence, or the like, indicating activities of system components such as processes or threads. Many system components, including malicious system components, perform a particular group of operations repeatedly. For example, a file-copy program repeatedly reads data from a source and writes data to a destination. In another example, a ransomware program repeatedly encrypts a file and deletes the un-encrypted original. Some examples relate to detecting such repetitions. Some examples locate repeated groups of operations based on the determined schema, permitting malware detection without requiring disassembly or other inspection of the code for that malware. Of course, the techniques can also be used to detect single, non-repetitive, instances that may occur in input data.

The techniques described herein may be implemented in a number of ways. Example implementations are provided below with reference to the following figures. Although discussed in the context of a security system, the methods, apparatuses, techniques, and systems, described herein can be applied to a variety of systems (e.g., data storage systems, service hosting systems, cloud systems, and the like), and are not limited to security systems.

1 FIG. 100 100 102 illustrates an example block diagramof an example computer architecture for synchronizing data using an example protocol, as described herein. The diagramincludes one or more computing device(s)associated with a service system such a security provider. In various examples, the service system may be part of, or associated with, a cloud-based service network that is configured to implement aspects of the functionality described herein.

1 FIG. 102 104 106 108 110 102 112 114 As shown in, the computing device(s)comprising an aggregation component, a synchronization component, one or more models, and a databaseto perform the functionality described herein. For instance, the computing device(s)can implement one or more components and/or one or more models to receive input datasuch as a portion of a data file and determine output dataindicating a reconstruction of the data file.

102 116 118 120 116 118 102 110 116 118 116 102 114 In various examples, the computing device(s)can exchange datawith one or more host device(s)over one or more network(s). The datacan represent data for synchronizing from the host device(s), a request from the computing device(s), confirmation that the data file has been stored in the database, etc., Depending on examples, the datamay be received from a variety of data sources. For instance, the host device(s)can transmit data packets representing the data file as the dataat a first time to the computing device(s)to cause the generation of the output data.

104 112 102 104 112 118 In various examples, the aggregation componentmay aggregate, identify, retrieve, access, or otherwise determine input datawhich may include an event(s), a data packet(s), a data string(s), a data sequence(s), or the like, for a model or component of the computing device(s)to process. In various examples, the aggregation componentmay receive the input datafrom a data stream, a host device, a memory, and/or a storage device (e.g., associated with the service system and/or associated with the host device(s)).

106 112 110 114 106 112 106 112 The synchronization componentrepresents functionality to synchronize the input datafor storage in the databaseand/or for generating the output data. For example, the synchronization componentcan replicate, reproduce, reconstruct or otherwise duplicate the input dataas a sequential set of portions based on specific information associated with the portions. The synchronization componentto determine an order for the portions regardless of whether the portions are received at different times. The portion of the input datacan represent a data packet, an event, a data string, a byte slice, a byte array, an event stream, or the like.

106 106 106 118 106 2 3 FIGS.and Additionally, or alternatively, the synchronization componentmay detect that data required for replicating the input data is missing, not been received, corrupted, or otherwise not available. In some examples, the synchronization componentmay retrieve the missing data required to replicate a data file in less time and with fewer data exchanges (relative to not implementing the synchronization component). For instance, a portion can represent a data packet, and the host device(s)can generate information to associate with each data packet for sending which may describe a position of the data packet relative to another data packet, an index number, a payload, a hash of a file, among others. In various examples, the information can include human-readable data and/or computer-readable data. Functionality associated with the synchronization componentis discussed throughout his disclosure including inbelow.

108 102 108 112 118 As described herein, the model(s)may be representative of machine learned models, statistical models, heuristic models, or a combination thereof. For instance, the computing device(s)can implement the model(s)to determine a sequence of data packets received as the input data(e.g., at particular time or over a time period), detect data missing from a sequence of data packets, reconstruct a data file associated with the host device(s), just to name a few.

110 112 114 110 112 The databasecan represent a storage device for storing some or all of the input dataand/or the output data, and may be accessible by a device and/or service to perform the techniques described herein. In some examples, the databasecan store data values representing information associated with the portions of data received as the input data, synchronized data, etc.

112 118 118 112 The input datacan vary depending on examples and can represent a portion of a data file (e.g., a data packet of a JSON file), event data associated with one or more events occurring at the host device(s), or a state of the host device(s)(e.g., a current hash value, index value, sequence number, etc.). In some examples, the input datacan include a request to synchronize an application, workload, data file, etc. and can include data of varying size, data formatting type, and so on.

102 112 118 118 102 106 In examples after initial data is received by the computing device(s)for processing (e.g., after a first portion is sent for synchronizing), the input datacan include a state of the host device(s)at a particular time, such as a recently transmitted sequence number, hash value, etc. The state of the host device(s)can be compared to information associated with a recently received data packet by the computing device(s). The synchronization componentcan determine whether any portions are missing based on whether the hash value or sequence number of a current state matches a corresponding hash value or sequence number of the recently received data packet.

114 118 114 114 122 118 116 110 114 122 The output datacan represent a sequential order of data portions, a determination of a missing portion of data, and/or validation that the combined portions represent the stored data at the host device(s)and can be analyzed for potential malicious activity, just to name a few. In some examples, the output datacan represent a classification (e.g., Are all the indexes received? “yes” or “no”). The output datacan be used to determine one or more actionssuch as transmitting synchronization results to the host device(s)as part of the dataat a second time, storing the output data in the database, detecting (by a security service) potential malicious activity in the output data(e.g., in the synchronized data). The actions(s)may also or instead include modifying a parameter of a network, device, or component to reduce or eliminate an impact of the potential malicious activity.

118 124 118 118 118 124 118 102 The host device(s)may implement one or more data componentswhich is stored in memory of the host device(s)and executable by one or more processors of the host device(s). The host device(s)may be or include any suitable type of device, including, without limitation, a mainframe, a work station, a personal computer (PC), a laptop computer, a tablet computer, a personal digital assistant (PDA), a cellular phone, a media center, an embedded system, a robotic device, a wearable device (e.g., sunglasses, clothing, etc.), a vehicle, a Machine to Machine device (M2M), an unmanned aerial vehicle (UAV), an Internet of Things (IoT), or any other type of device or devices capable of communicating via an instance of the data component(s). An entity may be associated with the host device(s), and the entity (user, computing device, organization, or the like) may have registered for security services provided by a service provider of the computing device(s).

120 120 118 102 120 In some embodiments, the network(s)may include any one or more networks, such as wired networks, wireless networks, and combinations of wired and wireless networks. Further, the network(s)may include any one or combination of multiple different types of public or private networks (e.g., cable networks, the Internet, wireless networks, etc.). In some instances, the host device(s)and the computing device(s)communicate over the network(s)using a secure protocol (e.g., https) and/or any other protocol or set of protocols, such as the transmission control protocol/Internet protocol (TCP/IP).

124 102 124 102 124 118 102 The data component(s)can represent software, firmware, hardware, or a combination thereof, that is configured to exchange data with the computing device(s), and the components thereof. In some examples, the data component(s)can be configured to send or receive data associated with a security concept to and/or from the computing device(s). The data component(s)may provide functionality for the host deviceto interface with the computing device(s)to manage a security concept, request security recommendations, and/or receive field description data as described herein.

124 102 102 124 124 118 102 118 The data component(s)may, in some examples, be kernel-level security agents, or similar security application or interface to implement at least some of the techniques described herein. Such kernel-level security agents may each include activity pattern consumers that receive notifications of events in a query that meet query criteria. The kernel-level security agents may each be installed by and configurable by computing device(s), receiving, and applying while live, reconfigurations of agent module(s) and/or an agent situational model. Further, the kernel-level security agents may each output query results to the computing device(s)that include the security-relevant information determined by the data component(s). The data component(s)may continue to execute on the host device(s)by observing and sending detected activity to the computing device(s)while the host device(s)is powered on and running.

124 102 102 124 102 In some embodiments, the data component(s)may be connected to the computing device(s)via a secure channel, such as a virtual private network (VPN) tunnel or other sort of secure channel and may provide query results security-relevant information to the computing device(s)through the secure channel. The data component(s)may also receive configuration updates, instructions, remediation, etc. from the computing device(s)via the secure channel.

1 FIG. 102 104 106 108 118 Though depicted inas separate components of the computing device(s), functionality associated with the aggregation component, the synchronization component, and/or the model(s)can be included in a different component of the service system, a single component, or be included in the host device(s). In some instances, the components described herein may comprise a pluggable component, such as a virtual machine, a container, a serverless function, etc., that is capable of being implemented in a service provider and/or in conjunction with any Application Program Interface (API) gateway.

2 FIG. 1 FIG. 200 202 204 106 204 202 illustrates a diagramof an example computing device implementing an example synchronization component to perform the techniques described herein. For example, a source devicecan exchange data with a target devicethat includes the synchronization componentof. The target devicecan receive portions of data from the source deviceand determine an order for the portions over time based on information associated with the received portions.

202 106 118 202 202 202 206 124 208 206 204 106 206 202 1 FIG. The source devicecan represent a computing device that sends data for synchronization to the synchronization componentand can represent the host device(s)of. In some examples, the source devicecan represent a mainframe computer in a mainframe computing environment while in other examples, the source devicemay represent a cloud computing device located in a cloud computing environment. The source devicecan include data component(s)which may be configured similar to the data component(s), and a database. The data component(s)can represent a security agent, sensor device, etc. for configuring data for sending to the target device. In some examples, at least some functionality associated with the synchronization componentcan be included in the data componentto cause the source deviceto generate information to append to or associate with portions of data for sending.

204 202 102 104 108 204 210 1 FIG. The target devicecan represent a computing device that receives the data for synchronization from the source deviceand can represent the computing device(s)of(and may further include the aggregation componentand/or the model(s)). The target devicecan further include a databasefor storing at least some of the data to be synchronized and/or synchronized data, depending on examples.

202 212 212 212 212 204 212 212 212 212 212 202 214 218 202 216 220 220 218 220 216 204 210 2 FIG. 2 FIG. 1 2 2 In some examples, the source devicecan send a first portionA, a second portionB, . . . , up to an Nth portionN where N is an integer greater than 1 (also referred to as “the portions”) to the target device. The portionscan be sent at approximately a same time or be sent at different times over a time period.denotes time T0 for sending the portions, though as mentioned the transmitting of each of the portionscan be sent at a periodic interval and/or a random interval. The portionscan, for instance, represent data packets associated with a data file, such as a JavaScript Object Notation (JSON) file. In various examples, the portionscan represent a set of data packets, events, or the like sent as initial data for synchronization. At time T, the source devicecan transmit a portionrepresenting incremental data(e.g., data received after the initial data). At time T, the source devicecan transmit a source staterepresenting status dataindicating information about a most recently transmitted portion. The status datacan indicate, for example, one or more of: a sequence number, a hash value (e.g., of a data file), a checkpoint hash value (e.g., included in the incremental dataand/or in status data), and/or payload information (e.g., “payload value” or “no payload”). Though shown at time Tin, the source statecan be sent at anytime and can vary in frequency to provide the target devicewith information usable for synchronizing data to the database.

202 204 202 212 214 212 202 By way of example and not limitation, the source devicecan identify a JSON file for sending to the target deviceand determine a hash value for the JSON file. For example, the source device can apply a hash algorithm to the JSON file to generate a hash value to represent the JSON file. The source devicecan also or instead determine portions of the JSON file (e.g., the portions, the portion, etc.) based on a size criteria or other criteria. To determine the portions, the source devicecan serialize JSON data of the JSON file using a schema to pre-process the data file for transmission.

202 212 212 204 212 214 216 The source devicecan determine information to assign or associate with a determined portion, such as a sequence number, a hash value, an index number, a total number of indexes associated with a sequence, a source device identifier, and a payload, though other information may also or instead be determined. For example, the first portionA can include information comprising a sequence number and/or hash value for identifying an origin of a respective portion. The index number and total number of indexes can be used to arrange or combine the portionsrelative to one another, for example. The target devicecan receive portions from a variety of different source devices and the source device identifier can be used to identify which portions belong to the different source devices. In some examples, the source device identifier can be included in a particular portion (e.g., the portions, the portion, the source state, etc.).

212 214 214 214 212 In various examples, one or more of the portionsand/or the portioncan include a session identifier to identify a session of portions. For example, the first portionA, the second portionB up to the Nth portionN include a session identifier “L” to indicate that the respective portions are part of a same session of data for processing from a host device. The session identifier can be used, for instance, to identify and/or to retrieve a dropped or missing portion.

204 106 212 214 216 218 214 204 The target devicecan implement the synchronization componentto analyze the information associated with the portions, the portion, the source state, and/or other data received as input. For example, the incremental datacan include another portion(s) in addition to the portion, a missing portion, etc. In some examples, any number of additional incremental data and/or source states can be received over time until all the portions associated with the JSON data file are received by the target device.

204 106 3 The target devicecan compare information of respective portions to identify missing and/or additional portions for a given time or over a time period. For instance, the synchronization componentcan determine that an incremental data at time Tis associated with a missing portion, duplicate portion, or additional portion based on a difference between the index numbers received and the total number of indexes.

204 204 204 In examples when a missing portion, corrupted portion, or the like is detected, the target devicecan generate a request identifier to associate with a message for sending to the source device requesting the missing portion. For example, the target devicecan identify a sequence number, a hash value, an index number, a session identifier, and/or a source identifier associated with the missing portion, and request the particular portions required to generate the JSON file. By way of example and not limitation, the target devicecan generate a request for specific portions usable to replay and/or or reconstruct a session of portions by including the session identifier and index values in the request.

2 FIG. 212 212 214 214 212 214 212 As shown in, the portionscan include a same sequence number or hash value for combining the portionswith subsequently received portion such as the portion. For example, the portiondenotes a sequence number two to indicate a sequence of the portionsrelative to the portion. Each of the portionsinclude a payload representing formatted data of a particular size (e.g., 64 KB) and format (e.g., serialized for sending as a payload).

216 202 202 204 216 202 204 214 216 216 204 204 218 214 204 The source statecan represent a current state of the source devicesuch as a last hash value, an indicator of “no” payload, etc. sent by the source device. In examples when a network issue causes a portion of data to be lost before receipt by the target device, the source statecan be used to determine that the source devicesent a sequence number 2 of hash B. For example, the target devicecan determine that the portionreceived prior to the source statealso includes the sequence number 2 and hash B and therefore no action is needed. However, if the sequence number or hash value of the source statediffers from the sequence number or hash value of most recently received data by the target device, then the target devicecan initiate recovery of at least a portion. In some examples, the incremental dataassociated with the portioncan include a hash value that is different from the hash value of another portion received at a different time, and the target devicecan initiate recovery of some or all of the portions of a session.

216 202 220 216 220 204 The source statecan also serve as an indicator for whether or not the source device, or associated with network(s) for exchanging the data, are functioning as expected or alternatively are experiencing an interruption. In various examples, the status dataof the source statecan indicate a sequence number, a hash value of a data file, and/or a checkpoint hash value. The status datacan represent a state of a data exchange and does not require or include a payload of the data file. The checkpoint hash value can, for instance, represent a data file that is being processed by the target deviceat a particular time.

218 202 204 202 218 210 208 218 218 218 In various examples, the incremental datacan represent a change to a data file over time, and can be transmitted from the source deviceto the target deviceat various times. In examples when the data file is a JSON file, the source devicecan generate the incremental datato represent differences is data stored in the databaseand the databaseat a particular time. For example, the incremental datacan represent portions of the JSON data file that change over time (e.g., a state of the stored information changes from a first state to a second state) rather than portions of the JSON data file that do not change over time (e.g., a state of the stored information in unchanged). In some examples, the incremental datacan represent changes to the JSON data file over time, such as an update to a JSON data object or value. By using the incremental dataat one or more times, changes to relatively large JSON data files can be captured incrementally without requiring computational resources to send or receive data that remains unchanged for a particular time period.

218 102 In some examples, an increment update to data (e.g., the incremental data) can represent an update while synchronizing a JSON document from a source device (e.g., an endpoint device) to a cloud computing environment. The incremental update can, for example, represent a data packet that includes a sequence number (e.g., for processing multiple data packets in sequence, etc.), a checkpoint hash value (e.g., a first hash value) and a current hash value (e.g., a second hash value), though other information may also be included. In various examples, the computing device(s)can, prior to performing an update the checkpoint hash value, compare, validate, or verify in view of a checkpoint hash value at a target device (e.g., a current checkpoint at a cloud computing device).

204 106 212 214 202 106 210 The target devicecan reconstruct or otherwise generate the JSON file from the portions received at one or more times. For example, the synchronization componentcan arrange the portions, the portion, and optionally additional portions in an order corresponding to an order of the portions transmitted from the source device. In this way, payloads corresponding to different portions can be arranged the synchronization componentto reproduce the JSON file for storage in the databaseand/or for output to another model or component.

204 202 122 204 202 In some examples, a security service can analyze the JSON file to detect potential security threats in the JSON file. In various examples, a target devicecan transmit indications of potential malicious activity to the source deviceand initiate one or more actions (e.g., the action(s)) to mitigate the potential security threat (e.g., run a test to verify the potential security threat, quarantine data, remove data from a database, or the like). In some examples, the security service can analyze the JSON file and determine that the JSON file does not include a potential security threat, and cause the target deviceto configure a message indicating that the portions or data packets associated with the data file are valid for execution by a processor of the source device.

3 FIG. 1 FIG. 2 FIG. 3 FIG. 300 300 102 204 104 106 108 114 302 302 110 210 300 204 302 210 is a pictorial diagram illustrating an example processfor synchronizing data between a source device and a target device, as described herein. The example processmay be implemented by a computing device such as the computing device(s)ofor the target deviceof. The computing device can implement the aggregation component, the synchronization component, and/or the model(s)to generate the output data.further depicts a storage device. The storage devicecan represent, for example, a registry, a memory, a database, or the like, such as the databaseor the database. In some examples, the processcan be performed by the target deviceand the storage devicecan be the database.

304 202 106 106 212 210 120 2 FIG. An operationcan include the source devicesending initial data to the synchronization component. For example, the synchronization componentcan receive input data representing data packets, portions, events, etc. associated with a data file. In some examples, the initial data can include the portionsofassociated with a JSON file for synchronizing to the database. In various examples, the initial data can represent serialized JSON data for transmitting over a network (e.g., the network(s)).

106 106 202 202 106 In various examples, the synchronization componentcan cause the source device to determine information describing different portions of the initial data. For instance, the synchronization componentcan send computer-readable instructions to a data component of the source deviceto cause the source deviceto assign a sequence number, hash value, index values, payload information, source identifier, and so on to various portions of data for sending to the synchronization component.

202 304 208 208 106 106 208 Data sent from the source deviceas part of the operationcan be stored in the databasefor a threshold amount of time for access at a later time. Data may be lost during transmission due to a network problem or other issue, and the databasecan store data for resending to the synchronization component. For example, portions that were not be received by the synchronization componentcan be accessed until expiration of the threshold amount of time when data stored in the databasemay be replaced with new data or otherwise deleted.

306 106 106 306 212 212 214 An operationcan include the synchronization componentdetermining an order for the portions received as the initial data based on the respective information associated with two or more of the portions. For example, the synchronization componentcan arrange the portions in order based on comparing the respective sequence numbers, index numbers, or the like. The operationcan include, for example, arranging the first portionA, the second portionB, and the portionbased on the sequence number and/or index number associated with each respective portion.

308 106 106 106 106 An operationcan include the synchronization componentdetermining that the expected number the index values for respective sequences have been received. For example, the synchronization componentcan compare the index values of portions received over time to a total number of indexes associated with a particular sequence and/or hash value. In examples when at least one index values is needed to meet the total number of indexes (e.g., index 1 of 3 and index 2 of 3 arrived and index 3 of 3 did not), the synchronization componentcan wait for a threshold amount of time for an incremental update to send one or more additional portions and detecting whether the additional portions includes an expected index (e.g., index 3 of 3). If after the threshold amount of time additional portions are not received, the synchronization componentcan initiate a request for a portion having a particular index value, for example.

106 In various examples, the synchronization componentcan receive portions as part of a synchronous transmission and/or an asynchronous transmission, and use the index values to determine an order of the portions and/or whether all the expected portions have been received.

310 106 302 106 302 An operationcan include the synchronization componentcombining portions and storing the combined portions in the storage device. For example, responsive to determining that the index values meet the total number of indexes (e.g., index 1 of 3, index 2 of 3, and index 3 of 3 arrived), the synchronization componentcan combine the portions (e.g., and payloads associated with therewith) and store that the combined portions in the storage device.

312 202 106 202 218 214 106 312 An operationcan include the source devicesending incremental data to the synchronization component. For example, the source devicecan transmit the incremental datacomprising the portionto the synchronization component. In various examples, the incremental data can represent updates to the data file after receiving the initial data and can represent additional portions of data received over a time period. In some examples, multiple iterations of incremental data can be received in addition to the incremental data sent as part of operation.

202 202 202 202 106 In various examples, the incremental data may be sent responsive to the source devicedetecting a change in activity associated with the data file. The data file can represent events stored in memory of the source deviceand/or events executed by a processor of the source device. In some examples, a security agent can detect changes in activity associated with the memory and/or the processor, and store the changes in a data file for security analysis. In some examples, the source devicecan implement a security agent or data component to send the data file to the synchronization componentas incremental data over a time period.

314 106 106 106 106 An operationcan include the synchronization componentcomparing the incremental data and the initial data. For example, the synchronization componentcan compare information of respective portions associated with the incremental data one to another to determine whether the incremental data includes missing or extraneous portions. The component the synchronization componentcan, also or instead compare information (e.g., hash values, sequence numbers, index values, or other information) associated with respective portion of the incremental data to corresponding information associated with the initial data. Based on the comparison(s), the synchronization componentcan determine whether to store the incremental data (e.g., all portions of data have been received based on comparing index values), arrange the portions in a sequence, identify a missing portion, just to name a few.

316 202 222 106 220 202 216 202 220 204 220 An operationcan include the source devicesending status datato the synchronization component. The status datacan provide an indication of a current state of the source devicesuch as by providing the source stateof the source deviceat a particular time. In some examples, the status datamay be sent periodically (e.g., a pre-determined interval in minutes, an hour, or other interval) and/or responsive to a request from the target device(e.g., a portion has not been received within an expected time frame). The status datamay indicate information associated with the most recently sent portion. For example, the incremental data can be associated with a first hash value and the status data can include a second hash value.

318 106 106 An operationcan include the synchronization componentcomparing the status data with other data. The synchronization componentcan, for example, compare a sequence number and/or a hash value associated with the status data to a corresponding sequence number and/or a corresponding hash value associated with one or more portions of the incremental data.

320 106 202 106 202 202 208 106 202 202 An operationcan include the synchronization componentinitiating a request from the source device. For example, the synchronization componentcan generate a request identifier (also referred to as a cloud identifier) usable for recognizing data returned from the source deviceas part of the request. By way of example and not limitation, the request for transmitting to the source devicecan indicate information (one or more of: an index number, a sequence number, a hash value, a source identifier, a session identifier, etc.) to identify a portion from the databaseto replace a missing portion. For example, by receiving status data at periodic intervals, the synchronization componentcan determine that the source deviceis operating as expected (e.g., the first and second hash values match and letting synchronization commence) or determine that the source devicefailed to send at least some portions (e.g., due to the first and second hash values not matching).

322 202 106 320 322 106 3 FIG. At operation, the source devicecan send the requested data to the synchronization component. The operationsandare shown inwith dashed lines to indicate the operations may be omitted in examples when all data associated with the data file is transmitted to the synchronization componentfree of errors (e.g., all index values received, all sequence numbers received, and so on).

324 106 302 314 318 302 An operationcan include the synchronization componentcombining portions storing the combined portions in the storage device. For example, responsive to one or more of: the comparing at operation, the comparing at operation, and/or receiving the requested data, portions associated with the incremental data and/or the requested data can be combined with the initial data in the storage device(e.g., as synchronized data).

326 302 208 202 302 102 An operationcan include the storage deviceproviding data for analysis to a model, component, service, device, or the like. For example, a security service can access the stored data (e.g., synchronized data) for determining presence of potential malicious activity in the combined payload representing the data file stored in the databaseof the source device. In some examples, combined portions representing synchronized data may be output to a model, component, or service independent of being stored in the storage device. In various examples, the computing device(s)can receive the first data associated with previous activity in a data stream of a host device (e.g., a potentially malicious process or thread, an instruction to write data to a memory, file, or the like).

4 FIG. 1 FIG. 400 400 400 102 is a flowchart depicting an example processfor combining portions of a data file, as described herein. Some or all of the processmay be performed by one or more components inas described herein. For example, some or all of processmay be performed by the computing device(s)(or service associated therewith).

402 402 102 112 118 At operation, the process can include receiving first data from a first device. In various examples, the first data can represent a first portion of a data file in memory of the first device and may include one or more of: a first identifier identifying the first data, a first payload, a hash value of the data file, a first index value, and a total number of indexes value, just to name a few. In some examples, the operationcan include the computing device(s)receiving the input datafrom the host device(s). By way of example and not limitation, the host device can transmit portions of a JSON data file using a schema as defined herein including defining information for the portions and each respective portion can represent a data packet, data string, byte array, or another data structure for analysis.

404 102 112 118 At operation, the process can include receiving second data from the first device. In some examples, the second data can represent a second portion of the data file in the memory of the first device and may include one or more of: a second identifier identifying the second data, a second payload, the hash value of the data file, a second index value, and the total number of indexes value. For example, the computing device(s)can receive the input datafrom the host device(s)at a same time as the first data or subsequent to receiving the first data. In various examples, the first identifier or the second identifier can represent a random value determined by a random number generator.

406 106 At operation, the process can include determining, based at least in part on the first index value, the second index value, the total number of indexes value, and the hash value, third data indicating that the first payload or the second payload is a last payload associated with the data file. For instance, the synchronization componentthe compare the first data with the second data and output an indication that all the expected index values and associated portions of been received.

408 102 106 At operation, the process can include determining an order for the first data relative to the second data based at least in part on the first index value and the second index value. In some examples, the order may be further determined based on the first data indicating that the first payload or the second payload is the last payload associated with the data file. For instance, the computing device(s)can implement the synchronization componentto determine a sequence of the first portion relative to the second portion using an order of the respective index values.

410 102 106 412 106 At operation, the process can include combining, as fourth data, the first payload of the first data and the second payload of the second data in the order. For instance, the computing device(s)can implement the synchronization componentto concatenate, position, arrange, or otherwise combine the first portion and the second portion as synchronized data (e.g., based on the determined order an indication that all portions for a particular data file have been received). In some examples, the combining of first payload of the first data and the second payload of the second data may be based at least in part on the first data and the second data including a same hash value (e.g., indicating the payloads are for a same data file). At operation, the process can include outputting, based on the third data indicating that the first payload or the second payload is the last payload, the fourth data to a second device configured to detect a potential malicious event in the fourth data. For instance, the synchronization componentcan transmit the combined, or synchronized data, to a cloud computing device configured to detect potential malicious event (e.g., a security threat by a threat actor) in the first payload and/or the second payload. In some examples, potential security threats can be transmitted to the first device for analysis, validation, or the like. By determining information specific for different portions of a data file as described herein, portions of a data file can be combined in less time and with more accuracy versus not implementing such techniques.

5 FIG. 1 FIG. 500 500 118 102 500 500 500 1 500 2 500 is a block diagram of an illustrative computing architecture of the computing device(s)to implement the techniques describe herein. In some embodiments, the computing device(s)can correspond to the host device(s)or the computing device(s)of. It is to be understood in the context of this disclosure that the computing device(s)can be implemented as a single device or as a plurality of devices with components and data distributed among them. By way of example, and without limitation, the computing device(s)can be implemented as various computing device(),(), . . . ,(N) where N is an integer greater than 1.

500 502 504 506 508 500 510 512 514 516 518 520 As illustrated, the computing device(s)comprises a memorystoring an aggregation component, a synchronization component, and model(s). Also, the computing device(s)includes processor(s), a removable storageand non-removable storage, input device(s), output device(s), and network interface.

502 504 506 508 502 504 506 508 In various embodiments, memoryis volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.) or some combination of the two. The aggregation component, the synchronization component, and the model(s)stored in the memorycan comprise methods, threads, processes, applications or any other sort of executable instructions. The aggregation component, the synchronization component, and the model(s)can also include files and databases.

502 502 502 In various embodiments, the memorygenerally includes both volatile memory and non-volatile memory (e.g., RAM, ROM, EEPROM, Flash Memory, miniature hard drive, memory card, optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium). The memorymay also be described as computer storage media or non-transitory computer-readable media, and may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. Computer-readable storage media (or non-transitory computer-readable media) include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, and the like, which can be used to store the identified information and which can be accessed by the security service system. Any such memorymay be part of the security service system.

504 124 118 504 504 506 The aggregation componentmay receive and store any client entity information and their associated security information including observed activity patterns received from the data component(s)on the respective host device(s). The aggregation componentmay gather data from other modules that may be stored in a data store. In some embodiments, the aggregation componentmay gather and store data associated with known information, such as domain information that is associated with known entities, for access as input data by the synchronization component(or other component).

504 104 1 FIG. In some examples, the aggregation componentcan correspond to, or otherwise include the functionality of, the aggregation componentof.

506 106 1 FIG. In some instances, the synchronization componentcan correspond to, or otherwise include the functionality of, the synchronization componentof.

508 108 1 FIG. In some instances, the model(s)can correspond to, or otherwise include the functionality of, the model(s)of.

500 500 500 5 FIG. In some instances, any or all of the devices and/or components of the computing device(s)may have features or functionality in addition to those thatillustrates. For example, some or all of the functionality described as residing within any or all of the computing device(s)may reside remotely from that/those computing device(s), in some implementations.

500 500 The computing device(s)may be configured to communicate over a telecommunications network using any common wireless and/or wired network access technology. Moreover, the computing device(s)may be configured to run any compatible device operating system (OS), including but not limited to, Microsoft Windows Mobile, Google Android, Apple iOS, Linux Mobile, as well as any other common mobile device OS.

500 516 518 The computing device(s)also can include input device(s), such as a keypad, a cursor control, a touch-sensitive display, voice input device, etc., and output device(s)such as a display, speakers, printers, etc. These devices are well known in the art and need not be discussed at length here.

5 FIG. 500 520 500 118 202 As illustrated in, the computing device(s)also includes the network interfacethat enables the computing device(s)of the security service system to communicate with other computing devices, such as any or all of the host device(s), or the source device.

3 4 FIGS.and 3 FIG. 4 FIG. 316 320 322 326 412 316 312 illustrate example processes in accordance with examples of the disclosure. These processes are illustrated as logical flow graphs, each operation of which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be omitted or combined in any order and/or in parallel to implement the processes. For instance, the example process ofmay omit operations,,, and/or. In some examples, the example process ofmay omit operation. In various examples, the operation(e.g., send status data) can occur before, during, or after another operation such as the operation(e.g., send incremental data).

The methods described herein represent sequences of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes. Moreover, the methods described herein can be combined in whole or in part with each other or with other methods.

The various techniques described herein may be implemented in the context of computer-executable instructions or software, such as program modules, that are stored in computer-readable storage and executed by the processor(s) of one or more computing devices such as those illustrated in the figures. Generally, program modules include routines, programs, objects, components, data structures, etc., and define operating logic for performing particular tasks or implement particular abstract data types.

Other architectures may be used to implement the described functionality and are intended to be within the scope of this disclosure. Furthermore, although specific distributions of responsibilities are defined above for purposes of discussion, the various functions and responsibilities might be distributed and divided in different ways, depending on circumstances.

Similarly, software may be stored and distributed in various ways and using different means, and the particular software storage and execution configurations described above may be varied in many different ways. Thus, software implementing the techniques described above may be distributed on various types of computer-readable media, not limited to the forms of memory that are specifically described.

While one or more examples of the techniques described herein have been described, various alterations, additions, permutations and equivalents thereof are included within the scope of the techniques described herein.

In the description of examples, reference is made to the accompanying drawings that form a part hereof, which show by way of illustration specific examples of the claimed subject matter. It is to be understood that other examples can be used and that changes or alterations, such as structural changes, can be made. Such examples, changes or alterations are not necessarily departures from the scope with respect to the intended claimed subject matter. While the steps herein can be presented in a certain order, in some cases the ordering can be changed so that certain inputs are provided at different times or in a different order without changing the function of the systems and methods described. The disclosed processes could also be executed in different orders. Additionally, various computations that are herein need not be performed in the order disclosed, and other examples using alternative orderings of the computations could be readily implemented. In addition to being reordered, the computations could also be decomposed into sub-computations with the same results.