A method, an electronic device, and a computer-readable storage medium for information processing are provided according to example embodiments of the present disclosure. The method comprises receiving, at a storage device, a data block and fingerprint information correlated with the data block, the fingerprint information being configured to identify the data block; and determining a storage position of the received data block based on predetermined correlations between fingerprint information and storage positions and the received fingerprint information, the predetermined correlations comprising at least correlations between historical fingerprint information correlated with stored data blocks and historical storage positions. Thus, a storage position of a data block can be determined based on received fingerprint information and predetermined correlations between fingerprint information and storage positions, thereby improving efficiency of redundant data deletion.
Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A method for information processing, comprising: receiving, at a storage device, a data block and fingerprint information correlated with the data block, the fingerprint information being configured to identify the data block; and determining a storage position of the received data block based on predetermined correlations between fingerprint information and storage positions as well as the received fingerprint information, the predetermined correlations comprising at least correlations between historical fingerprint information correlated with stored data blocks and historical storage positions; wherein receiving the data block and the fingerprint information correlated with the data block comprises: receiving, at a network switching device from a terminal device, the data block; and receiving, at the storage device from the network switching device, the data block and the fingerprint information correlated with the data block, wherein the received fingerprint information is generated by the network switching device based on the data block; wherein the network switching device comprises a programmable switch and is implemented in an edge device of a network separate from a storage system that includes the storage device; wherein the programmable switch of the network switching device is programmed to (i) perform hash operations and/or checksum operations on the data block to generate the fingerprint information, (ii) combine the fingerprint information with a packet that holds the data block, and (iii) forward the packet combined with the fingerprint information to the storage device; and wherein the terminal device comprises an Internet of Things device that is configured for communication with the edge device of the network.
Information processing and data storage. This invention addresses the efficient storage of data blocks by determining their storage location based on associated fingerprint information. The process involves receiving a data block and its corresponding fingerprint information at a storage device. The fingerprint information, which uniquely identifies the data block, is used to determine the storage position. This determination relies on pre-established correlations between fingerprint information and storage locations, including historical data. The reception of the data block and its fingerprint information occurs in stages. First, a terminal device, such as an Internet of Things (IoT) device, sends the data block to a network switching device. This network switching device, located at the network edge and separate from the main storage system, then generates the fingerprint information for the data block. This generation is achieved by performing hash or checksum operations on the data block using a programmable switch. The generated fingerprint information is then combined with the data block into a packet. Finally, this combined packet is forwarded from the network switching device to the storage device. The storage device then receives both the data block and its associated fingerprint information, enabling the determination of its storage position.
2. The method of claim 1 , wherein determining the storage position of the received data block comprises: searching the predetermined correlations for the received fingerprint information; and in response to a determination that the received fingerprint information is not found in the predetermined correlations, specifying a storage position for the received data block, and updating the predetermined correlations according to a correlation between the received fingerprint information and the specified storage position.
This invention relates to data storage systems that use fingerprint information to determine storage positions for data blocks. The problem addressed is efficiently managing data storage by reducing redundant storage and optimizing retrieval. The system generates fingerprint information for each data block, which is a unique identifier derived from the block's content. The system then searches a database of predetermined correlations between fingerprint information and storage positions. If the fingerprint is found, the data block is stored at the corresponding position. If not, a new storage position is assigned, and the correlation database is updated with the new fingerprint-storage position pair. This ensures that identical data blocks are stored only once, reducing storage redundancy. The system also handles dynamic updates to the correlation database as new data blocks are processed. The method improves storage efficiency by avoiding duplicate storage and speeds up data retrieval by using precomputed correlations. The invention is particularly useful in large-scale storage systems where minimizing redundancy and optimizing access times are critical.
3. The method of claim 2 , further comprising: storing the received data block in the specified storage position.
A method for managing data storage involves receiving a data block and determining a storage position for the data block based on a predefined storage policy. The storage policy defines rules for distributing data across multiple storage devices to optimize performance, reliability, or other factors. The method further includes storing the received data block in the specified storage position, ensuring the data is placed according to the policy's requirements. This approach helps balance workloads, improve access times, or enhance fault tolerance by distributing data intelligently. The method may also involve validating the storage position before writing the data to prevent errors or conflicts. By enforcing the storage policy, the system ensures consistent and efficient data placement across the storage infrastructure.
4. The method of claim 1 , wherein determining the storage position of the received data block comprises: searching the predetermined correlations for the received fingerprint information; and in response to a determination that the received fingerprint information is found in the predetermined correlations, acquiring a storage position correlated with the received fingerprint information as the storage position of the data block.
This invention relates to data storage systems that use fingerprint information to determine storage positions for data blocks. The problem addressed is efficiently locating and retrieving data blocks in large-scale storage systems by leveraging precomputed correlations between data fingerprints and storage positions. The system generates fingerprint information for each data block, such as a hash or checksum, and establishes predetermined correlations between these fingerprints and specific storage locations. When a data block is received, the system searches these correlations to find a matching fingerprint. If a match is found, the corresponding storage position is retrieved and used to store or retrieve the data block. This approach reduces search time and computational overhead by avoiding full scans of the storage system. The method ensures that data blocks with identical fingerprints are consistently stored in the same location, improving data consistency and retrieval efficiency. The system may also handle cases where fingerprints do not match existing correlations by assigning new storage positions or updating the correlation database. This technique is particularly useful in distributed storage systems, content-addressable storage, and deduplication systems where rapid data access and storage optimization are critical.
5. The method of claim 1 , wherein receiving the fingerprint information comprises: receiving at least one of hash information and checksum information that are correlated with the data block.
A method for verifying data integrity in a distributed storage system addresses the challenge of ensuring data consistency across multiple nodes. The system receives fingerprint information associated with a data block, which may include hash values or checksum values computed from the data block. These fingerprint values are used to verify that the data block has not been altered during storage or transmission. The method involves generating a request for the data block, transmitting the request to a storage node, and receiving the requested data block along with its associated fingerprint information. The fingerprint information is then compared to a locally computed fingerprint of the received data block to confirm its integrity. If the fingerprints match, the data block is deemed intact; if they do not match, the data block is flagged as corrupted. This method ensures reliable data verification in distributed systems where data integrity is critical, such as in cloud storage, blockchain, or peer-to-peer networks. The use of hash or checksum values provides a lightweight yet effective way to detect data corruption without requiring full data retransmission.
6. The method of claim 1 , wherein the predetermined correlations comprise a table of correlations between historical fingerprint information correlated with stored data blocks and historical storage positions.
A system and method for managing data storage and retrieval in a storage system involves tracking correlations between historical fingerprint information of data blocks and their corresponding storage positions. The method includes generating fingerprint information for data blocks, which are unique identifiers or signatures derived from the content of the data blocks. These fingerprints are then correlated with the storage positions where the data blocks are stored, creating a table of these correlations. This table is used to efficiently locate and retrieve data blocks by comparing current fingerprint information with the stored correlations. The system may also include mechanisms for updating the table as new data blocks are stored or existing ones are modified, ensuring the correlations remain accurate over time. The method improves data retrieval efficiency by reducing the need for exhaustive searches and leveraging historical storage patterns. The system may be applied in various storage environments, including distributed storage systems, databases, or file systems, where fast and accurate data access is critical. The use of fingerprint correlations helps optimize storage operations by predicting likely storage locations based on historical data, reducing latency and improving overall system performance.
7. An electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, wherein when executed by the at least one processing unit, the instructions cause the device to perform steps of: receiving a data block and fingerprint information correlated with the data block, the fingerprint information being configured to identify the data block; and determining a storage position of the received data block based on predetermined correlations between fingerprint information and storage positions as well as the received fingerprint information, the predetermined correlations comprising at least correlations between historical fingerprint information correlated with stored data blocks and historical storage positions; wherein receiving the data block and the fingerprint information correlated with the data block comprises: receiving, at a network switching device from a terminal device, the data block; and receiving, at the storage device from the network switching device, the data block and the fingerprint information correlated with the data block, wherein the received fingerprint information is generated by the network switching device based on the data block; wherein the network switching device comprises a programmable switch and is implemented in an edge device of a network separate from a storage system that includes the storage device; wherein the programmable switch of the network switching device is programmed to (i) perform hash operations and/or checksum operations on the data block to generate the fingerprint information, (ii) combine the fingerprint information with a packet that holds the data block, and (iii) forward the packet combined with the fingerprint information to the storage device; and wherein the terminal device comprises an Internet of Things device that is configured for communication with the edge device of the network.
This invention relates to a system for efficiently storing data in a networked environment, particularly in edge computing architectures involving Internet of Things (IoT) devices. The problem addressed is optimizing data storage by leveraging fingerprint information to determine storage positions, reducing computational overhead and improving retrieval efficiency. The system includes an electronic device, such as a storage device, with processing and memory components. The device receives a data block and associated fingerprint information from a network switching device. The fingerprint information is generated by the network switching device, which is implemented as a programmable switch in an edge device separate from the storage system. The switch performs hash or checksum operations on the data block to create the fingerprint, combines it with the data packet, and forwards it to the storage device. The storage device uses the received fingerprint to determine the storage position of the data block based on predetermined correlations between historical fingerprint information and storage positions. This approach leverages past storage patterns to optimize placement, enhancing data management efficiency. The terminal device, typically an IoT device, communicates with the edge device, which processes the data before storage. This distributed architecture offloads processing tasks from the storage system, improving scalability and performance in edge computing environments. The system ensures data integrity and efficient retrieval by using fingerprint-based storage positioning.
8. The device of claim 7 , wherein determining the storage position of the received data block comprises: searching the predetermined correlations for the received fingerprint information; and in response to a determination that the received fingerprint information is not found in the predetermined correlations, specifying a storage position for the received data block, and updating the predetermined correlations according to a correlation between the received fingerprint information and the specified storage position.
This invention relates to data storage systems that use fingerprint information to determine storage positions for data blocks. The problem addressed is efficiently managing data storage by leveraging fingerprint correlations to avoid redundant searches and optimize storage allocation. The system includes a storage device with a storage medium and a controller. The controller receives a data block and extracts fingerprint information from it. The fingerprint information is compared against predetermined correlations stored in a correlation table. If a match is found, the data block is stored at the corresponding position. If no match is found, the controller specifies a new storage position for the data block and updates the correlation table to include the new fingerprint-storage position relationship. The correlation table is dynamically updated as new data blocks are processed, ensuring that future storage operations can quickly reference the table to determine optimal storage positions. This reduces the need for repeated searches and improves storage efficiency. The system is particularly useful in environments where data blocks with similar fingerprints are frequently stored, such as in deduplication systems or databases with high data redundancy.
9. The device of claim 8 , wherein when executed by the at least one processing unit, the instructions further cause the device to perform a step of: storing the received data block in the specified storage position.
A system for data storage management processes data blocks for efficient storage. The system includes a processing unit executing instructions to receive a data block and determine a storage position for the block based on predefined criteria. The criteria may include factors such as data type, size, or access frequency. Once the storage position is identified, the system stores the data block in the specified location. The storage position may be within a memory device, a storage drive, or a distributed storage system. The system ensures that data is placed in an optimal location to improve retrieval speed, reduce fragmentation, or balance load across storage resources. The storage position may be dynamically adjusted based on real-time conditions, such as available capacity or performance metrics. The system may also track storage usage and update the storage position criteria to maintain efficiency over time. This approach enhances data management by automating storage decisions and adapting to changing storage demands.
10. The device of claim 7 , wherein determining the storage position of the received data block comprises: searching the predetermined correlations for the received fingerprint information; and in response to a determination that the received fingerprint information is found in the predetermined correlations, acquiring a storage position correlated with the received fingerprint information as the storage position of the data block.
A data storage system uses fingerprint information to efficiently determine storage positions for data blocks. The system addresses the challenge of quickly locating where data should be stored or retrieved in large-scale storage environments, improving search efficiency and reducing computational overhead. The system generates fingerprint information for each data block, which is a unique identifier derived from the block's content. These fingerprints are correlated with specific storage positions in a predefined mapping. When a data block is received, the system searches the predetermined correlations using the block's fingerprint. If a match is found, the corresponding storage position is retrieved and used to store or access the data block. This method ensures rapid data placement and retrieval by leveraging precomputed correlations, minimizing the need for real-time calculations. The system may also handle cases where fingerprints are not found by applying fallback mechanisms, such as hashing or other indexing techniques. The approach is particularly useful in distributed storage systems, databases, or file systems where fast and accurate data localization is critical.
11. The device of claim 7 , wherein receiving the fingerprint information comprises: receiving at least one of hash information and checksum information that are correlated with the data block.
A system for secure data verification involves a device that receives fingerprint information correlated with a data block to authenticate its integrity. The fingerprint information includes at least one of hash information or checksum information, which are cryptographic or error-detection values derived from the data block. The device compares the received fingerprint information with a locally generated fingerprint of the data block to verify whether the data has been altered. This process ensures data integrity by detecting unauthorized modifications or corruption during transmission or storage. The system may be part of a larger data processing or security framework, where the device also performs additional operations such as data retrieval, storage, or transmission based on the verification result. The fingerprint information may be generated using cryptographic hash functions (e.g., SHA-256) or checksum algorithms (e.g., CRC) to provide a reliable means of detecting changes in the data block. The device may further include mechanisms to handle multiple fingerprint types, allowing flexibility in verification methods. This approach is useful in applications requiring high data integrity, such as secure communications, blockchain systems, or file storage solutions.
12. The device of claim 7 , wherein receiving the fingerprint information comprises receiving checksum information that is correlated with the data block, the checksum information comprising an XOR checksum.
A fingerprinting system for data integrity verification involves capturing and analyzing fingerprint information to detect unauthorized modifications. The system generates a fingerprint for a data block by computing an XOR checksum, which is a compact representation of the data's content. This checksum is stored or transmitted alongside the data block to enable later verification. When the data block is accessed, the system recomputes the XOR checksum and compares it to the stored checksum. If they match, the data is deemed unaltered; if they differ, tampering is detected. The XOR checksum is efficient to compute and verify, making it suitable for real-time applications. The system may be used in secure data transmission, storage integrity checks, or digital rights management to ensure data authenticity. The fingerprinting process is reversible, allowing reconstruction of the original data block from the checksum under certain conditions, enhancing both security and usability. The system is particularly useful in environments where data integrity is critical, such as financial transactions, medical records, or software distribution.
13. The device of claim 7 , wherein the predetermined correlations comprise a table of correlations between historical fingerprint information correlated with stored data blocks and historical storage positions.
A data storage system uses fingerprint information to manage data placement and retrieval. The system addresses inefficiencies in traditional storage methods by leveraging historical correlations between fingerprint data and stored data blocks to optimize storage operations. Fingerprint information, such as hash values or other identifiers, is used to track relationships between data blocks and their storage positions over time. This historical data is stored in a correlation table, which maps fingerprint information to specific storage locations. The system retrieves this table to predict optimal storage positions for new data blocks based on past patterns, improving access speed and reducing fragmentation. The correlation table may also include additional metadata, such as access frequency or data block size, to further refine storage decisions. By analyzing these historical correlations, the system dynamically adjusts storage strategies to enhance performance and reliability. The invention is particularly useful in large-scale storage environments where efficient data placement is critical.
14. A non-transitory computer-readable storage medium with computer programs stored thereon, wherein when executed by a machine, the computer programs implement a method for information processing, the method comprising: receiving, at a storage device, a data block and fingerprint information correlated with the data block, the fingerprint information being configured to identify the data block; and determining a storage position of the received data block based on predetermined correlations between fingerprint information and storage positions as well as the received fingerprint information, the predetermined correlations comprising at least correlations between historical fingerprint information correlated with stored data blocks and historical storage positions; wherein receiving the data block and the fingerprint information correlated with the data block comprises: receiving, at a network switching device from a terminal device, the data block; and receiving, at the storage device from the network switching device, the data block and the fingerprint information correlated with the data block, wherein the received fingerprint information is generated by the network switching device based on the data block; wherein the network switching device comprises a programmable switch and is implemented in an edge device of a network separate from a storage system that includes the storage device; wherein the programmable switch of the network switching device is programmed to (i) perform hash operations and/or checksum operations on the data block to generate the fingerprint information, (ii) combine the fingerprint information with a packet that holds the data block, and (iii) forward the packet combined with the fingerprint information to the storage device; and wherein the terminal device comprises an Internet of Things device that is configured for communication with the edge device of the network.
This invention relates to a system for efficient data storage and retrieval in a networked environment, particularly for Internet of Things (IoT) devices. The problem addressed is optimizing storage positioning of data blocks by leveraging fingerprint information to reduce computational overhead and improve retrieval efficiency. The system includes a terminal device, such as an IoT device, that communicates with an edge network device separate from the storage system. The edge device, implemented as a programmable switch, processes incoming data blocks by performing hash or checksum operations to generate fingerprint information. This fingerprint is correlated with the data block and forwarded to a storage device within the storage system. The storage device uses the fingerprint to determine the optimal storage position for the data block based on predetermined correlations between historical fingerprint information and storage positions. These correlations are derived from previously stored data blocks and their associated fingerprints. The programmable switch in the edge device is configured to generate the fingerprint, combine it with the data block packet, and transmit the combined packet to the storage device. This approach offloads fingerprint generation from the storage system, reducing its computational load and improving overall system performance. The method ensures efficient data placement and retrieval by leveraging precomputed correlations between fingerprints and storage locations.
15. The non-transitory computer-readable storage medium of claim 14 , wherein determining the storage position of the received data block comprises: searching the predetermined correlations for the received fingerprint information; and in response to a determination that the received fingerprint information is not found in the predetermined correlations, specifying a storage position for the received data block, and updating the predetermined correlations according to a correlation between the received fingerprint information and the specified storage position.
This invention relates to data storage systems that use fingerprint information to determine storage positions for data blocks. The problem addressed is efficiently managing data storage by leveraging fingerprint correlations to optimize placement and retrieval. The system involves a non-transitory computer-readable storage medium that stores predetermined correlations between fingerprint information and storage positions. When a data block is received, the system searches these correlations for the block's fingerprint. If the fingerprint is not found, the system specifies a new storage position for the block and updates the correlations to include the new fingerprint-storage position relationship. This ensures that future blocks with the same fingerprint can be stored in the same location, improving storage efficiency and retrieval speed. The method dynamically adapts the storage system by learning and updating correlations over time, reducing redundant storage and enhancing data organization. The system may also include mechanisms for handling conflicts or collisions when multiple fingerprints map to the same storage position, ensuring reliable data management. The overall approach optimizes storage by minimizing redundant operations and improving access patterns through learned correlations.
16. The non-transitory computer-readable storage medium of claim 15 , further comprising: storing the received data block in the specified storage position.
A system and method for data storage management involves processing data blocks for storage in a distributed or networked storage environment. The invention addresses the challenge of efficiently and reliably storing data blocks in specified storage positions within a storage system, ensuring data integrity and accessibility. The system receives a data block from a client device or another system component, where the data block is associated with metadata that includes a storage position identifier. The storage position identifier specifies the exact location within the storage system where the data block should be stored. The system then processes the data block, which may include validating the data, compressing it, or encrypting it, to prepare it for storage. After processing, the system stores the data block in the specified storage position, ensuring that the data is placed correctly according to the storage position identifier. This method ensures that data is stored in the intended location, reducing errors and improving data retrieval efficiency. The system may also handle error conditions, such as failed storage operations, by retrying the operation or logging the error for further analysis. The invention is particularly useful in large-scale storage systems where precise data placement is critical for performance and reliability.
17. The non-transitory computer-readable storage medium of claim 14 , wherein determining the storage position of the received data block comprises: searching the predetermined correlations for the received fingerprint information; and in response to a determination that the received fingerprint information is found in the predetermined correlations, acquiring a storage position correlated with the received fingerprint information as the storage position of the data block.
This invention relates to data storage systems that use fingerprint information to determine storage positions for data blocks. The problem addressed is efficiently locating storage positions for data blocks in a storage system, particularly in scenarios where data deduplication or content-addressable storage is employed. The system generates fingerprint information for received data blocks and uses predetermined correlations between fingerprint information and storage positions to determine where each data block should be stored. When a data block is received, the system searches a set of predetermined correlations for the block's fingerprint information. If a match is found, the storage position correlated with that fingerprint is acquired and used as the storage position for the data block. This approach reduces the need for exhaustive searches or complex indexing mechanisms, improving storage efficiency and retrieval speed. The predetermined correlations may be established during an initial setup phase or dynamically updated as new data blocks are processed. The system ensures that data blocks with identical fingerprint information are stored at the same location, facilitating deduplication and reducing redundant storage. The method is particularly useful in large-scale storage systems where quick access to data is critical.
18. The non-transitory computer-readable storage medium of claim 14 , wherein receiving the fingerprint information comprises: receiving at least one of hash information and checksum information that are correlated with the data block.
A system and method for verifying data integrity using fingerprint information involves generating and comparing hash or checksum values to detect unauthorized modifications. The technology addresses the problem of ensuring data integrity in storage or transmission systems where data may be altered by malicious actors or errors. The system generates fingerprint information, such as hash or checksum values, for a data block and stores or transmits this fingerprint alongside the data. When verifying the data, the system receives the fingerprint information, which may include hash or checksum values, and compares it to a newly generated fingerprint of the data block. If the values match, the data is deemed intact; if they differ, tampering or corruption is detected. This method is particularly useful in secure data storage, file transfer protocols, and blockchain systems where integrity verification is critical. The fingerprint information may be generated using cryptographic hash functions or simpler checksum algorithms, depending on the security requirements. The system ensures that any unauthorized changes to the data block are detectable, enhancing trust in digital systems.
19. The non-transitory computer-readable storage medium of claim 14 , wherein receiving the fingerprint information comprises receiving checksum information that is correlated with the data block, the checksum information comprising an XOR checksum.
A system and method for secure data storage and retrieval involves verifying data integrity using checksums. The technology addresses the problem of ensuring data accuracy and detecting corruption during storage and retrieval operations in computer systems. The method includes storing data blocks in a storage system and receiving fingerprint information associated with the data blocks to verify their integrity. The fingerprint information includes checksum data, specifically an XOR checksum, which is correlated with the data block. The XOR checksum is computed by performing an exclusive OR (XOR) operation across the data block, providing a compact yet effective way to detect errors. The system may also include a storage controller that manages the storage and retrieval of data blocks, ensuring that the checksum information is properly generated and verified. The method further involves comparing the received checksum information with a recomputed checksum to confirm data integrity. If discrepancies are detected, corrective actions such as data recovery or re-transmission may be initiated. This approach enhances data reliability in storage systems by leveraging efficient checksum techniques to detect and mitigate data corruption.
20. The non-transitory computer-readable storage medium of claim 14 , wherein the predetermined correlations comprise a table of correlations between historical fingerprint information correlated with stored data blocks and historical storage positions.
A system and method for managing data storage and retrieval using fingerprint information and historical storage positions. The technology addresses the challenge of efficiently locating and accessing data blocks in storage systems, particularly in environments where data is frequently moved or reorganized. The system generates fingerprint information for data blocks, which serves as a unique identifier for each block. This fingerprint information is correlated with historical storage positions, allowing the system to track where data blocks have been stored over time. The correlations are stored in a table that maps fingerprint information to historical storage positions, enabling the system to quickly determine the current or past locations of data blocks. This approach improves data retrieval efficiency by leveraging historical storage patterns, reducing the need for exhaustive searches across the entire storage system. The system can also use the correlations to predict future storage positions based on past trends, further optimizing data access. The method is particularly useful in large-scale storage systems where data mobility is high, such as distributed storage networks or cloud-based storage solutions. By maintaining a record of historical storage positions, the system ensures that data blocks can be located even if they have been moved or reorganized, enhancing reliability and performance.
Cooperative Patent Classification codes for this invention. Click any code to explore related patents in that topic.
May 15, 2020
March 15, 2022
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.