Binary Quantization of Vector Embeddings in a Data Management System

This application claims the benefit of and priority to U.S. Provisional Application No. 63/733,179, filed Dec. 12, 2024, which is incorporated by reference herein in its entirety.

A vector index may be used to store vector representations of data, and the vector index is used to search against submitted queries. In conventional systems, vector indexes often store vectors as floating-point values (e.g., 32-bit or 16-bit precision). Storing and retrieving these high-precision vectors may use substantial memory bandwidth and storage capacity, and the cost of performing similarity computations scales with vector dimensionality and/or index size. As the vector index becomes larger in size, memory pressure increases, cache locality decreases, and/or query throughput may become constrained due to the amount of data to be scanned or transferred during search operations.

While quantization techniques reduce vector storage size, applying a uniform low-precision quantization to vectors may remove magnitude information that may be useful for distinguishing closely related embeddings, which can affect ranking accuracy during retrieval. In some conventional systems, the same quantization strategy is applied to both stored vectors and query vectors, so the system may lack sufficient resolution to reconstruct relative distances when scoring search results. As index size increases, these effects compound: the number of compressed vectors grows, the amount of information preserved per vector decreases, and/or the similarity comparison performed during search may have reduced fidelity due to limited representational precision.

This disclosure relates to efficiently indexing and searching high-dimensional data, such as product catalog embeddings, user profile vectors, or document summaries, by transforming them into highly compact, storage-optimized representations. A sequence of operations may involve ingesting a plurality of data vectors and applying a preconditioning matrix to perform an orthogonal transformation that is configured to make vector components approximately normally distributed. This step may help the data conform to assumptions made for the subsequent quantization method.

The vectors are adjusted around a representative point (e.g., a centroid) to create adjusted data vectors. The technique involves the generation of the bit vectors: for each (or some other subset of) individual adjusted data vector, a binary quantization engine determines a specific quantization interval by performing an optimization. This unique per-vector optimization may reduce (e.g., minimize or satisfy a threshold) the quantization error in a similarity computation by comparing the adjusted vector against a neighboring vector to calculate the best magnitude scaling for the compressed representation.

The resulting bit vectors and corresponding error correction metadata (which preserves lost magnitude information when converting to quantized form and the information lost when centering) are then stored in an index segment. During a search operation such as matching a user's query vector (e.g., an encoded search for “blue running shoes”) against millions of product embeddings, the process may utilize asymmetric quantization to help maintain high ranking fidelity while the index remains maximally compressed. The query vector is adjusted around a second representative point and encoded using a higher-precision multi-bit quantization (e.g., int4) to create a quantized query vector. This multi-bit vector is then subjected to a bit-slicing process that generates translated query components, aligning the data for hardware-accelerated comparison via single instruction, multiple data (SIMD) instructions.

The similarity score between the high-precision quantized query vector and the low-precision stored bit vectors is estimated using a functional decomposition of the dot product. This decomposition may include a query correction term, a data correction term, and a quantized residual term, collectively helping the score accurately reflect the original floating-point similarity.

The overall design may promote efficiency and scalability through its underlying structure. In some examples, the vector index is structured into vector clusters managed by hierarchical centroids. This allows the search engine to employ a load-on-demand mechanism, where the cluster partitions (e.g., product categories determined to be relevant by the coarse search against centroids) are loaded from disk at execution time. This approach may reduce memory footprint and enable efficient similarity search across large datasets that may otherwise exceed the capacity of random access memory (RAM) based vector systems. This combination of per-vector optimized compression, multi-bit asymmetric search, and/or clustered structure provides high-recall results with low latency for applications like real-time product recommendations and massive-scale semantic search.

In some aspects, the techniques described herein relate to a method including: computing a representative point for at least a plurality of data vectors representing data to be added to a vector index; generating an adjusted data vector by adjusting a data vector of the plurality of data vectors around the representative point; and generating a bit vector using the adjusted data vector, including: determining a quantization interval that reduces (e.g., minimizes) a quantization error (or the quantization error satisfies a minimum condition) in a similarity computation; and mapping values of the adjusted data vector to components of the bit vector using the quantization interval.

In some aspects, the techniques described herein relate to an apparatus including: a database storing a vector index; and an index engine including executable instructions that cause at least one processor to: compute a representative point for at least a plurality of data vectors representing data to be added to the vector index; generate an adjusted data vector by adjusting a data vector of the plurality of data vectors around the representative point; and generate a bit vector using the adjusted data vector, including: determine a quantization interval that reduces a quantization error in a similarity computation; and map values of the adjusted data vector to components of the bit vector using the quantization interval.

In some aspects, the techniques described herein relate to a non-transitory computer readable medium storing executable instructions that cause at least one processor to execute operations, the operations including: computing a representative point for at least a plurality of data vectors representing data to be added to a vector index; generating an adjusted data vector by adjusting a data vector of the plurality of data vectors around the representative point; and generating a bit vector using the adjusted data vector, including: determining a quantization interval that reduces a quantization error in a similarity computation; and mapping values of the adjusted data vector to components of the bit vector using the quantization interval.

This disclosure relates to an effective approach to processing and searching high-dimensional vector data, which differentiates it from previous techniques relying on generic transformations. The technical problem addressed by this approach stems from the challenges of scaling vector search systems. For example, storing and retrieving high-precision floating-point vectors may require substantial memory bandwidth and storage capacity, and the cost of performing similarity computations scales unfavorably with vector dimensionality and index size.

Some conventional quantization methods may reduce storage but often introduce significant error, which may affect ranking accuracy. The technical solution involves a multi-stage indexing and search process optimized for fidelity and/or speed. To prepare the data, an orthogonal transformation is applied using a preconditioning matrix configured to decorrelate components of the data vectors. This may help make vector components approximately normally distributed required for the subsequent quantization method. In some examples, a binary quantization engine generates the bit vectors by determining a specific quantization interval for each vector (or a subset of vectors). This interval may be calculated by comparing the adjusted data vector against a neighboring vector to reduce (e.g., minimize) the quantization error in a similarity computation. For search, the examples discussed here may use asymmetric quantization. While stored vectors are low-precision, the incoming query vector is adjusted around a second representative point and encoded with a higher-precision multi-bit quantization to create a quantized query vector. The similarity score is then estimated using a decomposition of the dot product that incorporates one or more scalar components such as a query correction term, a document correction term, and the quantized residual term.

In some examples, the optimization process is performed to determine the quantization interval such that the resulting quantization error satisfies a threshold (e.g., a predetermined fidelity threshold). The threshold may correspond to a minimum acceptable reconstruction accuracy for the similarity score. For instance, the binary quantization engine may be configured to select the quantization boundary parameters produce a quantization error below a specific maximum tolerance level. This approach ensures that the compressed bit vector preserves sufficient magnitude information to maintain high ranking accuracy during the subsequent search operation.

The dot-product specific quantization may preserve the similarity (e.g., required similarity) information despite encoding the index vectors using minimal bits, resulting in high fidelity with a high level of compression. The asymmetric quantization and scoring decomposition may reduce the computational bottleneck during search. In some examples, the technology may also be designed to enable hardware-accelerated comparison using SIMD instructions. In some examples, the vector index is structured into vector clusters managed by hierarchical centroids, allowing the search engine to employ a load-on-demand mechanism. This approach may reduce memory footprint and enable efficient similarity search across large, disk-resident datasets

1 1 FIGS.A toC 100 106 116 illustrate a systemfor binary vector quantization for converting multi-dimensional data vectorsinto bit vectorsaccording to an aspect.

100 100 158 100 152 160 152 118 158 The systemmay be a data management system. The systemmay ingest, store, and search data imported from one or more data sources. The systemincludes a data management platformexecutable by one or more server computers. The data management platformmay receive source datafrom one or more data sources.

158 158 158 118 150 152 118 A data sourceis a component in a computing system. A data sourcemay be an end point (e.g., a user device such as workstation, smartphone, laptop, or wearable device, etc.), a network device (e.g., router, switch, or firewall), a server (e.g., an application server, a database server, or a web server), or an application (e.g., email server, a cloud service, a server application, or a client application). The data sourcemay receive or generate source datathat is transmitted, over a network, to the data management platform. In some examples, the source dataincludes textual data, audio data, and/or video data.

118 118 100 118 118 118 100 118 118 158 In some examples, the source datacan be referred to as documents. The source datamay include private documents, e.g., documents that are accessible by authorized users of the system. The source datamay include public documents, e.g., documents that are accessible by the general public. The source datamay include private and public documents. In some examples, the source dataare organizational documents of an organization associated with the system. In some examples, the source dataincludes security-related data about system activity, network traffic, and/or security events. The source datamay include event data about security events about the data source(s).

152 104 106 118 104 118 106 104 104 The data management platformincludes an embedding modelconfigured to generate data vectorsfrom source data. The embedding modelmay be a software component, machine-learning model, or feature-extraction model that converts input data (e.g., the source data) into data vectors(e.g., numeric values). In some examples, the embedding modelis a neural-network-based encoder. In some examples, the embedding modelis a statistical or rule-based transformation model.

104 106 106 106 106 106 106 106 118 106 118 The output of the embedding modelincludes data vectors. The data vectormay represent an entire document, web page, paragraph, or user review. The data vectormay encode the document's content, context, and semantic meaning, enabling semantic search and question-answering systems. The data vectorsmay be multi-dimensional vectors that may include floating point values such as 32-bit values (e.g., float32) or 16-bit values (e.g., float16, bfloat16). In some examples, floating point values include smaller or larger precision formats depending on the representation requirements of the embedding model. A data vector may be a structured numeric representation of input data. A data vectormay also be referred to as a feature vector, an embedding vector, or a multi-dimensional vector. In some examples, a data vectoris referred to as a document embedding or a document vector. In some examples, a data vectorincludes a set of numeric components that each represent characteristics or learned features of the source data. In some examples, a data vectorincludes values that encode semantic, visual, or contextual relationships present in the source data.

152 108 112 106 108 106 106 108 108 160 108 106 116 114 112 The data management platformincludes an index engineconfigured to generate or update a vector indexusing the data vectors. The index enginerefers to a software component, indexing module, or vector-processing subsystem that receives data vectorsand transforms the data vectorsinto index-storable representations. In some examples, the index engineis a pipeline of software instructions that perform vector re-centering, vector quantization, and/or storage operations. In some examples, the index engineis implemented as one or more executable processes hosted on the server computer(s). The index enginetransforms the data vectorsinto index-storable representations, generating bit vectors, which are stored in an index segmentof a vector index.

112 110 114 110 153 110 116 The vector indexis storable in a vector database, which may function as a persistent storage structure, an index repository, or a database configured to store multiple index segments. The vector databasemay be an example of a memory device, a storage volume, or a structured data store that retains vectors and their derived representations. In some examples, the vector databasestores compressed or quantized representations such as bit vectorsin a storage format.

112 114 114 112 114 114 112 114 The vector indexincludes index segments. An index segmentrefers to a subset, grouping, or partition of vectors within the vector index. An index segmentmay store vectors that were ingested within a similar time window, vectors assigned to a common storage partition, or vectors grouped for update operations. In some examples, index segmentsenable incremental write, update, and/or merge operations. In some examples, the vector indexincludes multiple index segmentsto allow efficient search, incremental indexing, or partial rebuilding of index content.

108 106 116 116 114 116 116 106 108 116 116 108 114 112 The index engineis configured to receive at least one data vector, generate one or more bit vectors, and store the bit vectorsin an index segment. A bit vectorrefers to a vector representation encoded using one or more bits per dimension and may also be referred to as a binary vector, compressed vector, or quantized embedding representation. The bit vectorsmay represent component sign values, threshold-based encodings, or binary quantization outputs derived from the data vectors. In some examples, the index enginestores bit vectorsas one-bit-per-dimension encodings. In some examples, the bit vectorsinclude multi-bit encodings (e.g., two-bit or four-bit values). The index enginemay add the index segmentto the vector indexwhen the segment is generated, updated, or merged.

112 112 112 In some examples, the vector indexis implemented as a hierarchical navigable small world (HNSW) graph structure. In some examples, the vector indexis a multi-layer graph arrangement in which vectors are stored as nodes connected by proximity-based edges. In some examples, the HNSW graph includes multiple layers of connectivity, where upper layers represent coarse similarity connections and lower layers provide denser local connections for fine-grained retrieval. An HNSW implementation of the vector indexmay allow a search operation to traverse the graph by progressively navigating through vector neighborhoods until candidate vectors are identified.

112 108 112 In some examples, the vector indexincludes hierarchical cluster partitions. A hierarchical cluster partition refers to a multi-level vector grouping, cluster tree, or centroid-tree structure in which vectors are arranged in layers of clusters according to similarity. In some examples, a cluster is associated with a cluster-representative point, feature anchor, or cluster centroid, and child clusters provide increased resolution or finer similarity boundaries. In some examples, hierarchical cluster partitions allow the index engineto evaluate selected (e.g., only selected) partitions during search, rather than accessing vectors (e.g., all vectors) stored in the vector index.

1 FIG.C 108 106 106 114 108 106 106 114 106 As shown in, the index enginemay receive one or more data vectorsuntil the data vectorssatisfy an index segment. For example, the index enginemay receive one or more data vectorsover time until a collection of received data vectorsfills or satisfies a threshold for an index segment(e.g., when enough data vectorshave been buffered or when a flush condition triggers).

108 135 135 106 106 106 a. The index engineincludes a preconditioner. The preconditioneris configured to apply an orthogonal transformation to the data vectorsas a preconditioning step. The preconditioning transformation may be used to make the vector component distributions of the data vectormore normally distributed, which may improve the robustness of the quantization process for pathological vector distributions. This preconditioning step may be performed prior to the generation of adjusted data vectors

135 106 106 145 145 145 106 The preconditionerapplies an orthogonal transformation to a data vector. An orthogonal transformation corresponds to a change of basis that is desirable because it does not change a similarity metric (e.g., dot product, cosine similarity, or Euclidean distance). Applying an orthogonal transformation may include multiplying the data vectorby a preconditioning matrix. The preconditioning matrixis an orthogonal matrix. An orthogonal matrix is a square matrix whose columns and rows are orthonormal unit vectors (e.g., the transpose of the matrix is equal to its inverse). Applying the preconditioning matrixto a data vectorgenerates a transformed data vector. The resulting component distribution of the transformed vector becomes more normally distributed, which may be a consequence of the central limit theorem. This normality may be beneficial because the initialization procedure of the quantization assumes that the components are normally distributed.

135 106 145 145 106 106 139 135 The preconditioneris configured to apply an orthogonal transformation to the plurality of data vectorsusing a preconditioning matrix, where the preconditioning matrixis configured to transform the components of the plurality of vectorsto be individually normally and distributed (e.g., identically distributed). The components that are approximately normally and distributed (e.g., identically distributed) are the individual dimensional values (e.g., the floating point values) that form a data vector. The orthogonal transformation applies a unified rotation and reflection to the vector space, transforming the input data into a new coordinate system where the resulting components are sums of original components so the central limit theorem applies to their distribution. This step is performed because the subsequent binary quantization engine(which uses the component-wise threshold rule) operates accurately (e.g., most accurately) under the assumption that the input component values are normally and distributed (e.g., identically distributed). By making the components approximately normally distributed and distributed (e.g., identically distributed), the preconditionermay ensure that the data conforms better to the distribution assumptions for (e.g., required for) binary quantization.

Individually normally distributed may mean that the component values across the dataset follow a normal distribution, which is a distribution assumption needed for the subsequent quantization process. Distributed (e.g., identically distributed) may mean that the statistical properties (e.g., mean and variance) of components (e.g., all components) are substantially the same, allowing the same quantization threshold to be optimally applied across dimensions. The transformation is structural, applying a unified rotation to the vector space, transforming the input data into a new coordinate system where the resulting components exhibit minimal statistical correlation and suitable marginal distributions.

145 106 106 106 112 2 2 The preconditioning matrixincludes floating point values (e.g., d, where d is the vector dimension). Applying the transformation to the data vectorsmay require a plurality of floating point multiplications (e.g., ndfloating point multiplications where n is the number of documents (e.g., data vectors)). A dense preconditioner for vectors with 1024 dimensions may request 4 MB storage per matrix and may perform more than one million multiplications per document (e.g., per data vector) added to the vector index.

135 145 The preconditionermay use a sparse block diagonal matrix as the preconditioning matrix, which may overcome the computational cost of a dense orthogonal matrix. A sparse orthogonal matrix is an orthogonal matrix where the majority of elements are zero, which may result in a significantly lower memory and computational cost. The sparse orthogonal matrix may be block-diagonal, e.g., non-zero values are confined to sub-matrices (blocks) along the main diagonal. This architecture may ensure that a few components are mixed, which may be sufficient for high-dimensional vectors.

135 106 135 In some examples, the preconditionerapplies the sparse block diagonal matrix to a data vectorby assigning vector components to blocks (e.g., the sub-matrices that form the sparse block diagonal matrix). A block (e.g., each block) dictates which subset of the original vector components will be mixed together during the orthogonal transformation. In some examples, the preconditionerapplies the sparse block diagonal matrix to the blocks in a way that balances variance, thereby ensuring that the vector components mixed within a single block have homogeneous statistical properties. In some other examples, the preconditioner may choose components to mix in a single block at random.

135 135 135 135 135 135 135 135 135 j k k k j j th The preconditionermay execute an algorithm (e.g., a greedy algorithm) to apply the sparse block diagonal matrix. For example, the preconditionermay initialize a variance sum (v) to zero for each block (j) and set the block assignment set (b) to empty. The preconditionermay iterate through the vector component indices (i) in descending order of their component variance. For the component currently under consideration, the preconditionerselects the block (j) that currently holds the minimum accumulated variance (v):j←arg minv. Once the block (j) is selected, the preconditionerassigns the current component (i) to that selected block (j) (e.g., add i to b). The preconditionerthen checks if the size of the block (|b|) is less than the maximum dimension size for that block (d). If the size limit is not yet reached, the preconditioneradds the variance of the icomponent to the block's accumulated variance (vi). However, if the block has reached its dimension size, the preconditionersets the accumulated variance (vi) to infinity to minimize (e.g., prevent) subsequent components from being assigned to that block in the future. By executing this algorithm, the preconditionereffectively balances the variance across the blocks, which may result in an optimized block diagonal preconditioning that achieves robust quantization quality similar to a computationally expensive dense transformation, but with lower (e.g., significantly lower) memory and indexing overhead.

108 134 136 106 114 134 135 106 136 106 114 136 106 134 136 106 134 136 106 The index engineincludes a cluster processing engineconfigured to compute a representative pointusing the group of data vectorsthat form an index segment. In some examples, the cluster processing enginereceives the output of the preconditioner, e.g., the transformed data vectors. The representative pointmay be a single point in vector space that represents the set of data vectorsto be stored in the index segment. In some examples, the representative pointis a centroid, e.g., the center of a group of data vectors. In some examples, the cluster processing enginecomputes the representative pointby averaging corresponding component values across the data vectors. In some examples, the cluster processing enginecomputes the representative pointusing a weighted average of the component values of the data vectors.

134 106 106 136 106 136 106 106 136 106 106 106 136 116 a a a The cluster processing enginegenerates adjusted data vectorsby adjusting the data vectorsrelative to the cluster representative point. Adjusting the data vectorsrelative to the representative pointmay include centering the data vectorsaround the computed centroid. In some examples, the adjusted data vectorsare computed by subtracting a component (e.g., each component) of the representative pointfrom a component (e.g., each component) of the corresponding data vector. In some examples, the adjusted data vectorsmaintain the same dimensionality as the data vectorsbut are positioned around the representative pointto enable subsequent quantization into bit vectors.

108 139 139 116 106 139 106 116 106 139 139 116 106 116 106 139 a a a a The index engineincludes a binary quantization engine. The binary quantization engineis configured to generate bit vectorsusing a quantization process on the adjusted data vectors. In some examples, the binary quantization engineencodes each dimension of the adjusted data vectorsas one bit, and the resulting bit vectorsinclude a sequence of binary values that correspond to the dimensional components of the adjusted data vectors. In some examples, the binary quantization enginepacks the binary values into a byte-aligned format to reduce storage space, improve memory alignment, and/or increase SIMD-based comparison throughput during search operations. In some examples, the binary quantization engineproduces a fixed-length bit vectorfor adjusted data vector, where a bit vectoris substantially smaller in memory footprint than the corresponding original data vector. In some examples, the quantization process of the binary quantization engineis referred to as a first quantization process, which is further described as follows.

139 175 106 116 175 116 175 175 175 a i i The binary quantization enginemay compute (e.g., directly compute) a quantization intervalto use when mapping the values (e.g., the floating-point component values) of the adjusted data vectorsto the bit vectors. The component floating-point values are mapped to quantized values based on q=a +k×(b−a), where [a, b] is the quantization intervaland kare the components of the resulting bit vector. The quantization intervalmay represent a range of values (e.g., a continuous range of values) that is mapped to a discrete, quantized representation by snapping to the closest endpoint a or b. The term snapping may refer to an operation of assigning a continuous floating-point value to the nearest discrete boundary or level within the defined quantization interval. In some examples, the quantization intervalis defined by interval parameters such as a first boundary parameter a and a second boundary parameter b. In some examples, the interval parameters can be characterized as magnitude scale factors. In some examples, the interval parameters can be characterized as scaling and offset parameters to compress the vector's original magnitude distribution into the small discrete set of values.

139 106 106 a a. The binary quantization enginedetermines the first and second quantization boundary parameters a and b for each vector (or a subset of vectors) by performing an optimization to reduce (e.g., minimize) the error in the dot product between the vector being quantized and a vector near to it. In some examples, the optimization process is performed to determine the quantization interval such that the resulting quantization error satisfies a predetermined fidelity threshold. This fidelity threshold may correspond to a minimum acceptable reconstruction accuracy for the similarity score. The first quantization boundary parameter (a) may define the lower bound of the mapping interval, and the second quantization boundary parameter (b) defines the upper bound of the mapping interval. The quantization error is defined as the difference between the true dot product of the original float vector and the approximate dot product computed using the quantized vector after corrections are applied. The original float vector may be the adjusted data vector. The quantized vector may be the resulting bit vector

106 139 139 a This optimization process involves comparing the adjusted data vectorbeing quantized against a vector near to it to (e.g., minimize) the error in the dot product. The binary quantization enginedetermines the first and second quantization boundary parameters a and by minimizing the quantization error for the dot product metric between a vector x being quantized and a neighboring vector y. When the binary quantization enginesolves for the quantization boundary parameters to reduce (e.g., minimize) the dot product error, it is effectively calculating the optimal scaling and offset to compress (e.g., needed to compress) the vector's original magnitude distribution into the small discrete set of values.

106 139 a T T 2 T T For any given pairs of vectors (x,y), where x is the adjusted data vectorbeing indexed and y is a vector near it, the quantization error E may be defined as the squared difference between the true dot product and the approximated dot product: E=xy−Q(x)y, where xy is the true dot product (e.g., the target value) and Q(x)y is the approximated dot product (e.g., the result of the quantized vector). The binary quantization enginemay determine the values for the first and second quantization boundary parameters that reduce (e.g., minimize) the sum of the quantization error over training pairs.

If the distribution of the components of the vector are known to be normally distributed then the expected mean square error (MSE) between the quantized vector and the floating point vector for a vector picked at random from the dataset can be computed exactly. In some examples, the mean and variance of the components of the vector are computed, m(x) and σ(x). In some examples, for a number (e.g., each number) of bits being used to represent the components of the quantized vector, a constant is computed that reduces (e.g., minimizes) the expected MSE. In some examples, this constant c is used to compute initial values for the quantization boundary parameters a and b. For example, this constant c is 0.798, 1.493 and 2.051 for 1 bit, 2 bits and 4 bits, respectively. The initial values for a and b are set to m(x)+c σ(x). In some examples, the intervals are further refined by minimizing the following error function

139 106 a In some examples, λ is set to 0.1. This function is chosen to better preserve the error in the dot product which is more sensitive to errors parallel to the floating point vector x than errors perpendicular to it. The binary quantization enginedoes this for an adjusted data vector(x).

139 139 175 To find the optimal first and second quantization boundary parameters that reduce (e.g., minimize) the object function, the binary quantization enginemay perform a coordinate-descent type of iterative scheme that alternates between updating the components of the quantized vector by snapping to the new nearest quantization boundary parameter (a or b) and solving a closed-form solution derived from setting the derivative of the error function to zero to compute new quantization boundary parameters (a and b). This optimization process allows the binary quantization engineto calculate the most effective quantization intervalto maximize the fidelity of the dot product approximation using the ultra-compressed bit representation.

113 139 106 106 3 FIG. a a This optimization results in the application of a component-wise threshold rule to generate binary quantized bits(as shown in). The binary quantization engineapplies the component-wise threshold rule to a floating point component of the adjusted data vectors. If a component value of the adjusted data vectoris greater than

139 the binary quantization engineassigns a binary value of one. If the value is less than or equal to

139 the binary quantization engineassigns a binary value of zero. The application of this rule may produce one-bit-per-dimension encodings.

139 113 139 113 116 116 106 106 139 113 116 3 FIG. The binary quantization enginethen aggregates the binary quantized bitsin component order. In some examples, the binary quantization enginepacks eight binary quantized bitsinto a single eight-bit byte. This packing operation generates the compact bit vector. In some examples, the resulting bit vectoris substantially smaller in memory footprint than the corresponding original data vector. As such, for the first quantization process for data vectors, the binary quantization enginegenerates one-bit-per-dimension encodings, thereby reducing component values (e.g., float32 component value) into binary quantized bits(see), which are then packed into a byte-aligned format to generate a bit vector(e.g., a compact bit vector).

108 138 132 116 114 132 106 116 132 a The index engineincludes a correction error engineconfigured to compute error correction metadatafor a bit vectorof an index segment. The error correction metadataincludes one or more numerical values (e.g., error correction values) used to preserve information about the adjusted data vectorsthat is contained in the boundary parameters a and b and is lost during generation of the bit vectors. It is also used to undo the centering of the original vectors. In some examples, the error correction metadataincludes one or more numerical values for centroid adjustment and one or more numerical values for quantization.

138 106 106 116 132 a a In some examples, the correction error enginecomputes an error correction value for an adjusted data vectorby computing a residual difference between the adjusted data vectorand the corresponding bit vectorprojected back into the same dimensional space. In some examples, the error correction metadataincludes a scaling factor that represents the difference between the quantization boundary parameters divided by the largest value which can be represented by the quantized vector component

106 a where n is the number of bits used to quantize the adjusted data vectorbefore quantization, and the scaling factor is used to reconstruct or approximate the original distance or similarity between vectors during a search operation.

132 132 132 t In some examples, the error correction metadataincludes an additive correction factor per data vector proportional to the sum of components of the quantized data vector. In some examples the metadataincludes a per-vector dot product between the centroid m and the residual of the data vector x and the centroid to correct for centering: m(x−m). In some examples, the error correction metadataincludes multiple values, such as a scalar multiplier and a variance descriptor, that together increase reconstruction fidelity during similarity scoring.

108 116 132 114 116 114 132 116 116 116 132 108 108 114 110 116 132 The index enginestores the bit vectorsand the error correction metadatatogether in an index segment. In some examples, the bit vectorsare stored in a contiguous block within the index segment, and the error correction metadatais stored in an adjacent metadata block associated with the bit vectors. In some examples, each bit vector(or a subset of bit vectors) is stored in association with a corresponding error correction metadata, which allows the index engineto retrieve both binary and magnitude information for a vector during a search operation. In some examples, the index enginewrites the index segmentto the vector databaseafter the bit vectorsand error correction metadatahave been generated.

108 110 116 139 132 138 108 114 114 116 132 106 The index engineis configured to write the compressed data to the vector database. Following the computation of the bit vectorsby the binary quantization engineand the error correction metadataby the correction error engine, the index enginegenerates the index segment. The index segmentstores the plurality of bit vectorsand the associated error correction metadatain a contiguous, interleaved arrangement. This arrangement ensures that the compressed binary data and the magnitude correction information for a processed data vectorare stored efficiently together, allowing for high-throughput search operations where the data is read block-by-block

2 FIG. 2 FIG. 114 116 132 114 116 116 132 138 114 116 1 132 1 116 1 116 2 132 2 116 2 116 132 116 114 illustrates an example of an index segmentthat stores multiple bit vectorsand associated error correction metadata. The index segmentincludes multiple stored bit vectors, and a stored bit vectorhas a corresponding set of error correction metadatagenerated by the correction error engine. As shown in, the index segmentincludes a bit vector-, error correction metadata-associated with the bit vector-, a bit vector-, error correction metadata-associated with the bit vector-and continues through a bit vector-Y and error correction metadata-Y associated with the bit vector-Y, where Y represents the total number of vectors stored in the index segment.

114 116 132 116 132 The structure of the index segmentmay be stored in a file format (e.g., .veb) that sequentially stores the bit vectorand its associated error correction metadata. For each vector, the bit vectoris stored as (dim/8) number of bytes. The associated error correction metadatais stored as n×float number of bytes. The n×float size indicates that the metadata is stored as n floating point values. In some examples, n=3 floating point values are stored for Euclidean distance, or n=3 floating point values are stored for dot product similarity. In some examples, the floating point values are stored at reduced precision such as bfloat16.

116 106 116 114 116 132 a In some examples, a bit vectoris stored as a fixed-length binary encoding, where a bit corresponds to a dimension of the adjusted data vectorfrom which it was generated. In some examples, the bit vectorsare stored in a packed binary layout in the index segmentto reduce segment size and increase sequential read efficiency during retrieval. In some examples, a bit vectoris stored in association with a corresponding error correction metadatavalue or values that maintain magnitude information lost during binary quantization.

132 116 114 114 116 116 132 114 116 132 In some examples, the error correction metadatais stored adjacent to or interleaved with the bit vectorswithin the index segment. In some examples, the index segmentstores a metadata header that identifies the number of bit vectorsin the segment, the dimensionality of the bit vectors, and the size or type of the associated error correction metadata. In some examples, the index segmentis stored in a format that allows block-level reading of consecutive bit vectorsand their associated error correction metadatafor high-throughput search operations.

3 FIG. 3 FIG. 300 116 139 108 300 139 106 112 a illustrates a quantization processof generating bit vectorsusing the binary quantization engineof the index engine. The quantization processmay graphically depict the operation of the binary quantization engine.depicts how floating-point component values of the adjusted data vectorsare converted from multi-byte numeric values into one-bit binary values and how those binary values are packed into a compressed byte that is ultimately stored in the vector index. In the example shown, eight floating-point component values are reduced to eight binary values, which are then packed into a single eight-bit byte, thereby reducing memory footprint by orders of magnitude relative to storing the original floating-point values.

3 FIG. 3 FIG. 3 FIG. 108 106 106 136 106 111 106 111 111 111 111 111 a a a a b c d As shown in, the index enginegenerates adjusted data vectorsby offsetting the data vectorsrelative to the representative point. An adjusted data vectorincludes multiple floating-point component values, andillustrates an example set of float values (e.g., −0.09, 0.39, 0.15, −0.10, −0.13, −0.18, −0.05, −0.03).further depicts memory allocationscorresponding to each component of the adjusted data vectors. A memory allocationincludes four one-byte containers,,,(or another number depending on floating-point precision), which represent the underlying encoded bytes of a single floating-point value. In this example, eight float values require thirty-two total bytes prior to quantization.

139 106 113 106 a a The binary quantization engineconverts a floating-point component of the adjusted data vectorsinto a binary quantized bitusing a component-wise threshold rule. This threshold rule is the result of the optimization process, where the optimal threshold is calculated to reduce (e.g., minimize) the error in the dot product. If a component value of the adjusted data vectoris greater than the average of the quantization boundary parameters

139 the binary quantization engineengine assigns a binary value of 1. If the value is less than or equal to the average of the quantization boundary parameters

139 113 3 FIG. the binary quantization engineassigns a binary value of 0. Applying this rule to the eight float values shown inresults in eight binary quantized bits. In this example, the transformation reduces eight floating-point component values into eight one-bit binary values, shrinking the representation from thirty-two bytes to a single byte.

139 116 113 139 113 139 113 116 The binary quantization engineis configured to generate a bit vectorby aggregating the binary quantized bitsin component order. In some examples, the binary quantization enginepacks eight binary quantized bitsinto a single eight-bit byte. In some examples, the binary quantization enginepacks binary quantized bitsinto a multi-byte representation, depending on vector dimensionality. In some examples, bit vectorsmaintain consistent bit ordering across dimensions to ensure deterministic decoding, SIMD compatibility, and/or predictable memory alignment.

108 116 117 117 113 106 117 116 a The index enginestores the generated bit vectorinto a byte container, where a byte containercorresponds to a packed representation of binary quantized bitsfor an adjusted data vector. In the example shown, a packed bit pattern of 01100000 corresponds to a decimal value of six. In some examples, storing a decimal equivalent of the binary byte enables direct index lookup, faster comparison operations, and/or hardware-optimized retrieval. In some examples, multiple byte containersare stored contiguously to support block-aligned reads of many bit vectors.

3 FIG. 132 116 132 106 132 106 132 132 116 114 116 a a also illustrates error correction metadataassociated with a bit vector. The error correction metadataincludes one or more numerical values used to preserve magnitude information from the adjusted data vectors. In some examples, the error correction metadatais a scalar value computed from the adjusted data vector. In some examples, the error correction metadataincludes a variance-based adjustment or multiple correction values that enable approximate reconstruction of similarity scores during search. In some examples, the error correction metadatais stored alongside the bit vectorwithin the same index segmentso that retrieval of the bit vectorautomatically provides access to the associated magnitude information.

4 4 FIGS.A andB 2 FIG. 140 108 114 114 1 114 2 114 1 132 114 2 132 114 1 114 2 114 108 a illustrate a merging engineof the index engineconfigured to generate a merged index segmentby merging a first index segment-and a second index segment-. The first index segment-includes first bit vectors with corresponding interleaved error correction metadata. The second index segment-includes second bit vectors and corresponding interleaved error correction metadata. The first index segment-and the second index segment-have the persistent storage structure of the index segmentdepicted in. The merging process may be used by the index enginefor maintaining index performance, particularly in systems utilizing an inverted vector file (IVF) index or a hierarchical navigable small world (HNSW) graph structure.

140 114 112 106 a The merging enginecombines these two independently-generated segments into a single merged index segmentso that the vector indexmay remain compact, searchable, and/or incrementally updated without retaining original float-precision data vectors.

4 FIG.A 140 114 1 114 2 114 108 As shown in, the merging enginereceives the first index segment-and the second index segment-. In some examples, merging is triggered when multiple index segmentsaccumulate, when a segment reaches a threshold size, or when the index engineperforms a maintenance or compaction cycle.

170 140 116 132 114 1 114 2 136 144 116 106 a In operation, the merging enginecomputes a combined representative point based on the bit vectorsand associated error correction metadataof both index segments-and-. The combined representative point represents the distribution of vectors across the two original segments as though they were a single unified dataset. In some examples, the combined representative point is computed using a weighted average of the representative points(centroids) previously calculated for a segment. In some examples, the combined representative point is computed using the vector sum dataof each bit vector, which reflects the aggregate magnitude of the original adjusted data vectorsprior to quantization.

172 140 116 116 140 132 144 In operation, after determining the combined representative point, the merging enginere-centers or re-adjusts both sets of bit vectorsrelative to the combined representative point. Because bit vectorsstore only binary sign information, and because the original centered float values are not retained, the merging enginemay use the error correction metadataand vector sum datato reconstruct approximate magnitude information so that vectors can be re-quantized consistently.

174 140 114 1 114 2 155 174 140 155 116 132 114 4 FIG.B a. In operation, the merging enginethen re-quantizes all re-adjusted vectors from the first index segment-and the index segment-using a temporary file. In some examples, during operation, the merging enginemay generate a temporary file(e.g., .qvec.tmp) that stores a higher-precision reconstruction of the vectors, as shown in. The use of int4 data ensures that scoring steps, such as diversity and reverse-link scoring during HNSW graph update, are performed with the necessary fidelity to maintain graph quality. In some examples, this re-quantization process produces new bit vectorsand new error correction metadatafor the merged index segment

4 FIG.B 155 155 155 116 132 144 144 illustrates an example of the structure of the temporary file. The temporary filestores a vector using asymmetric quantization where the vector is quantized as an int4 query vector. The structure of the temporary fileincludes a repeating pattern for each vector Y: a bit vector-Y (e.g., sized (dim/2)), associated error correction metadata-Y, and vector sum data-Y. The (dim/2) size corresponds to the use of a 4-bit (int4) representation. The vector sum datastores magnitude information used to re-quantize vectors during merging and may be retained so that future merges can be performed without reverting to float precision data.

155 116 1 132 1 116 1 144 1 116 1 116 2 132 2 116 2 144 2 116 2 116 132 116 144 116 176 140 112 114 1 114 2 114 a The temporary fileincludes a bit vector-, error correction metadata-associated with the bit vector-, vector sum data-associated with the bit vector-, and similarly includes a bit vector-, error correction metadata-associated with the bit vector-, and vector sum data-associated with the bit vector-, continuing through a bit vector-Y, error correction metadata-Y associated with the bit vector-Y, and vector sum data-Y associated with the bit vector-Y. In operation, after re-quantization completes, the merging engineupdates the vector index, replacing the two original segments-and-with the merged segment. In some examples, this update process may free memory and storage previously occupied by the two original segments, reducing fragmentation and improving search efficiency.

178 140 114 116 132 114 a a In operation, the merging enginegenerates the merged index segment, which contains newly-generated bit vectors, and updated error correction metadata. The merged segmenthas consistent quantization across vectors that originally belonged to separate segments.

114 155 116 132 112 a After the merged index segmentis finalized, the temporary fileis deleted, and only the compact bit vectorsand reduced error correction metadataremain stored in the vector index.

5 FIG.A 5 FIG.A 126 128 124 104 126 120 126 108 124 116 139 106 116 126 124 112 a illustrates a binary quantization engineconfigured to generate a quantized query vectorfrom a query vectorgenerated by the embedding model. The binary quantization enginemay be included in a search engineand may operate during a search process, rather than during indexing. The binary quantization engineoperates asymmetrically relative to the index engine, which generates a higher-precision encoding of the query vectorcompared to the one-bit encoding of the stored bit vectors. In other words, in contrast to the binary quantization enginethat converts adjusted data vectorsinto one-bit per-dimension bit vectors, the binary quantization enginemay generate a higher-precision encoding of the query vectorso that search accuracy is maintained even though the stored vectors in the vector indexare one-bit representations.therefore may illustrate an asymmetric processing flow in which index-time quantization and search-time quantization are not the same.

120 122 165 115 102 122 104 122 124 124 124 124 106 The search enginemay receive a search querythrough an input fielddisplayed by an applicationexecuting on a computing device. The search querymay be text, audio, or visual content supplied by a user. The embedding modelmay convert the search queryinto a query vector, and, in some examples, the query vectoris a multi-dimensional query vector that includes floating-point values representing semantic, contextual, or visual meaning extracted from the input. In some examples, the query vectoris referred to as a query embedding. The query vectormay therefore be similar in structure to the data vectorsdescribed earlier but is processed separately for search.

126 137 124 137 124 136 114 137 124 The binary quantization enginemay compute a representative pointfor the query vector. The representative pointis a single reference point in vector space that represents the component distribution of the query vector, similar in form to the representative pointcomputed for an index segment. In some examples, the representative pointis a centroid computed from the component values of the query vector.

126 124 124 137 124 137 124 124 124 a a a The binary quantization enginemay generate an adjusted query vectorby adjusting the query vectorrelative to the representative point. In some examples, the adjusted query vectoris created by subtracting the representative pointfrom each component of the query vectorso that the representative point becomes the origin of the coordinate space. The adjusted query vectormaintains the same dimensionality as the original query vectorbut is re-centered for quantization.

126 146 124 128 124 a The binary quantization engineincludes a quantizerthat converts the adjusted query vectorinto a quantized query vectorusing a second quantization process. This second quantization process is asymmetric and may map component values into multiple discrete bins based on magnitude. For example, the second quantization process may encode the query vectorwith multiple bits per dimension (e.g., int4) providing more granularity than the one-bit encoding used at index time. In some examples, the second quantization process maps component values into multiple discrete bins based on magnitude, where the number of bins corresponds to the number of bits assigned to each dimension. For instance, a four-bit per-dimension encoding yields sixteen possible bin assignments, which provide more granularity than the one-bit encoding used at index time.

128 120 116 112 124 128 116 The resulting quantized query vectortherefore maintains magnitude differentiation, allowing the search engineto compute similarity against one-bit bit vectorsstored in the vector indexwith greater fidelity. In some examples, the asymmetry means the stored vectors are encoded using one bit per dimension, while the query vectormay be encoded with more than one bit per dimension. The additional precision of the quantized query vectormay preserve magnitude detail that is lost in the one-bit segment representation, allowing the query to remain expressive even when compared to heavily compressed bit vectors.

146 124 128 124 106 a The quantizerexecutes the second quantization process, which may involve a form of asymmetric optimized scalar quantization, converting the adjusted query vectorinto the quantized query vector. This process relaxes the constraint that the query embedding (e.g., the query vector) and document embeddings (e.g., data vectors) use the same centroid (or representative point) for centering.

106 124 146 y x t For a document embedding y (e.g., a data vector) centered around a representative point mand a query vector(x) centered around a query representative point m, the quantizermay estimate a dot product xy decomposing the calculation:

146 The quantizeris configured to compute an estimate of this dot product using three component terms, two of which are scalar corrections.

The first term is the query correction term

is a scalar value computed once per query (x) and used to process the whole posting listing of document vectors. The second term is the document correction term

116 x y t which is a scalar value that is computed once per document (y) and cached with the quantized document embeddings (e.g., with the bit vector), representing the interaction between the query's center mand the adjusted document residual y−m. This replaces the scalar m(x−m) that is stored when the vectors use the same centre so represents no additional storage.

x y x y t The third term is the quantized residual term (x−m)(y−m), which may be core component estimated using the binary quantization methodology, where the query residual (x−m) and document residual (y−m) vectors are quantized, and the dot product value is estimated from these quantized vectors.

146 146 124 q d y x q This asymmetric centering allows the quantizerto efficiently manage the search process in IVF structures. In some examples, document centroids are clustered into a smaller number of query centroids (k<k). This means there is a many-to-one mapping from the document representative points (m) to the query representative points (m). The quantizermay only need to quantize the query vector(x) against the document representative points at most ktimes, which may reduce (e.g., significantly reduce) the computational bottleneck during search. The final similarity score is computed by summing the contributions of the three decomposed terms.

5 FIG.B 5 FIG.B 500 128 146 181 181 124 181 183 183 124 a a illustrates a quantization processfor generating a quantized query vectorby the quantizer.depicts the intermediate result of the second quantization process as a set of quantized values(e.g., eight values quantized to int4). These quantized valuesare the integer results corresponding to the dimensions of the adjusted query vector. Below the quantized valuesis the binary representation. The binary representationexpresses a quantized value as a 4-bit binary code (e.g., if four bits are assigned to each component of the adjusted query vector).

146 185 183 124 185 185 The quantizergenerates translated query componentsby bit-shifting and aligning the binary representationof a quantized query component. The process reorganizes the bits so that bits (e.g., all bits) at the same positional value (e.g., the first bit of every component) are grouped together into a separate, compressed byte. This produces one byte per bit-position rather than one byte per dimension. If four bits are assigned to each component of the query vector, then four separate bytes are produced, collectively forming the translated query components. This bit-slicing process produces compact, byte-aligned translated query components.

5 FIG.B 5 FIG.B 128 187 185 185 120 128 116 114 shows example byte-aligned values such as 11001010 or 00001110, which reflect the bit-sliced encoding of the quantized query vector.further illustrates decimal byte values, which represent the decimal interpretation of each translated query component. Converting to decimal values may allow for faster lookup, cache locality improvements, and/or hardware acceleration during similarity evaluation. The translated query componentsare the representation that the search engineuses when comparing the quantized query vectoragainst the bit vectorsstored in an index segment.

128 120 185 116 112 185 The quantized query vectormay allow the search engineto perform a fast dot-product approximation by looping over the bytes of the translated query componentsand performing bitwise operations against corresponding bytes of the bit vectorsstored in the vector index. Because each byte represents the collective first, second, third, or fourth bit of the dimensions, bitwise processing can be applied to multiple dimensions simultaneously. In some examples, the translated query componentsenable a vectorized similarity computation using CPU SIMD instructions or GPU warps, which substantially increases throughput relative to scalar or floating-point comparison.

116 124 185 120 This representation also supports the asymmetric quantization process described above. The stored vectors remain compressed as one-bit per-dimension bit vectors, while the query vectormaintains multiple bits per dimension through the translated query components. The search enginealigns these representations by performing bitwise compare operations against each bit position independently, allowing magnitude information to influence the scoring process without decompressing the index.

120 128 116 132 120 128 116 5 FIG.B The search enginecomputes similarity between the quantized query vectorand the stored bit vectorsusing the error correction metadatato refine scoring accuracy. During scoring, the search engineperforms bitwise comparison operations between components of the quantized query vectorand corresponding components of a stored bit vector. In some examples, similarity is computed by counting matching bits (e.g., Hamming-style similarity), accumulating weighted scores for a bit position, or computing bit-dot-product approximations using the translated query components described in.

132 132 116 106 124 In some examples, the error correction metadatais applied during scoring to restore magnitude information lost during binary quantization. The error correction metadatamay store a compressed representation of the original float magnitudes for a bit vector, which allows the scoring engine to adjust similarity scores according to the magnitude distribution of the original data vectors. This asymmetric evaluation—where stored vectors are represented as one-bit encodings and the query vectoris represented as multi-bit quantized values—permits high-fidelity ranking while maintaining extremely compact index storage.

120 128 116 116 120 185 116 120 In some examples, the search enginecomputes similarity between the quantized query vectorand stored bit vectorsusing hardware-accelerated instruction execution. In some examples, the bit vectorsare stored as byte-aligned binary sequences, enabling evaluation of multiple bit positions in parallel using processor vector-register operations. In some examples, the search engineexecutes bitwise logical operations, including population-count, bitwise-AND, bitwise-XOR, and bit-shift instructions, over byte slices of the translated query componentsand corresponding byte-aligned slices of stored bit vectors. This allows the search engineto evaluate similarity across multiple dimensions per instruction rather than on a component-by-component basis.

185 128 120 132 In some examples, the translated query componentsare formatted so that each byte contains one bit from each of multiple dimensions, allowing one register to evaluate bit states for many dimensions of the quantized query vectorsimultaneously. In some examples, the search engineutilizes processor instruction sets such as AVX, SSE, or NEON to perform parallel bit evaluation. In some examples, the hardware execution path produces an intermediate similarity score derived from matching bit positions, which is subsequently adjusted using the error correction metadatato refine similarity.

120 116 120 185 116 116 120 116 In some examples, the search engineselectively evaluates stored bit vectorsby computing partial similarity scores and terminating evaluation early if a stored vector cannot exceed a relevance threshold. In some examples, the search enginecompares a subset of the translated query componentsto corresponding portions of a stored bit vectorand computes an intermediate similarity score. If the intermediate score is below a threshold that would allow the bit vectorto remain in contention, the search enginestops the comparison for that vector. In some examples, thresholding is applied sequentially over byte slices so that evaluation may terminate before all bits of the bit vectorare processed.

128 136 120 120 114 In some examples, branch-and-bound pruning is applied at the cluster selection stage as well. If the similarity between the quantized query vectorand a cluster representative pointdoes not meet a selection bound, the search enginebypasses stored vectors in that segment without loading or scoring them. This allows the search engineto evaluate multiple index segmentsin parallel while selectively skipping irrelevant segments based on centroid proximity or partial similarity.

120 116 120 120 132 116 114 In some examples, the search engineidentifies an oversampled subset of stored bit vectorsprior to producing final results. In some examples, the search engineselects a candidate set larger than the requested result count based on bitwise similarity alone. After selecting the candidate set, the search engineretrieves and applies associated error correction metadatato compute a corrected similarity score for a stored bit vectorin the candidate set. In some examples, this refinement is computed only for the oversampled subset rather than the full index segment.

120 132 In some examples, the search enginegenerates a re-ranked result order using the corrected similarity scores. In some examples, bitwise similarity contributes the base ranking while error correction metadatacontributes magnitude restoration, producing a refined order for the oversampled candidate set without reconstructing original floating-point vectors.

120 128 116 132 120 132 132 106 a. In some examples, the search enginecomputes an estimated dot-product between the quantized query vectorand a stored bit vectorusing the combination of bit-similarity and the corresponding error correction metadata. In some examples, the search enginecomputes an estimated inner-product score using the number of matching bit positions from the bitwise similarity computation and a magnitude factor encoded in the error correction metadata. In some examples, error correction metadataincludes a per-vector magnitude value, a constant, or a compressed measure of the vector norm from the original adjusted data vectors

120 In some examples, the search enginecomputes a similarity metric as a weighted combination of bitwise similarity counts and correction factors. In some examples, the estimated dot-product is computed without reconstructing original floating-point values. In some examples, this estimation is applied as the final ranking stage following oversample candidate selection.

6 FIG. 112 116 116 illustrates a technique in which the vector indexis arranged into vector clusters rather than stored as a single flat sequence of bit vectors. A vector cluster refers to a grouping of stored bit vectorsthat are located within the same region of the vector space.

116 132 148 148 148 1 2 120 6 FIG. A cluster includes a subset of the bit vectors, corresponding error correction metadata, and one or more hierarchical centroids. A hierarchical centroidis a representative reference point used to characterize the location of a cluster in the vector space. The hierarchical centroidsdefine multiple levels of spatial granularity, shown inas a cluster tree (H→H). Higher-level centroids represent broader partitions where many vectors may fall, while lower-level centroids represent more specific and densely separated groupings. Using a hierarchy of centroids enables a tiered navigation approach in which the search enginefirst performs a coarse-resolution comparison against high-level centroids, and then progressively narrows comparisons to lower-level centroids only where appropriate.

6 FIG. 116 112 As depicted in, separate regions of the vector space are represented by Cluster A and Cluster B, each shown with a centroid (for example, Centroid A and Centroid B) and a corresponding set of compressed bit vectorsthat may be stored on disk. Rather than requiring the entire vector indexto reside in memory simultaneously, the clusters (e.g., only the clusters) determined to be relevant to a specific query are loaded at execution time. This design reduces memory footprint, improves scalability for large-scale datasets, and enables vector search to extend far beyond the amount of memory typically required for graph-based approaches.

108 108 106 136 116 116 106 148 1 5 FIGS.to In some examples, the index engineconstructs these clusters during indexing. The index enginereceives incoming data vectors, adjusts them using representative points, and generates corresponding bit vectorsas previously described with respect to. Placement of a bit vectorinto a specific cluster is determined according to similarity between a bit vectorand one or more hierarchical centroids. In some examples, cluster boundaries are generated using iterative or recursive clustering processes (e.g., hierarchical k-means or other recursive partitioning methods).

6 FIG. 148 1 148 2 148 3 148 1 148 3 116 132 112 illustrates centroid hierarchy nodes-,-, and-, where-may correspond to a top-level centroid representing a large portion of the dataset and-may represent a more finely-divided subregion. Because each cluster maintains an independent association with its bit vectorsand error correction metadata, a single cluster can be retrieved and loaded without requiring the entire vector indexto be present in memory at once.

112 6 FIG. This behavior may enable the vector indexto scale to arbitrarily large corpora while maintaining efficient retrieval properties. Flat indexes and full-RAM graph structures degrade once memory limits are reached, whereas the clustered configuration shown incontinues to function even when most of the index is disk-resident.

120 124 122 124 128 120 124 148 124 5 5 FIGS.A-B During retrieval, the search enginegenerates a query vectorfrom the search queryand converts the query vectorinto a quantized query vectorusing the asymmetric multi-bit quantization process described in. The search enginecompares the query vectoragainst the hierarchical centroidsto determine which clusters most closely resemble the query vector. The comparison may begin at the certain (e.g., highest) centroid layer to eliminate large unrelated regions efficiently, after which lower-level centroids associated with promising partitions are evaluated. This cluster-based selection reduces the amount of data to be loaded and scored, improving performance without requiring full-index scanning.

120 116 132 190 190 120 116 128 6 FIG. Once a subset of clusters has been selected based on centroid proximity, the search engineloads the associated bit vectorsand error correction metadatafrom disk via the load-on-demand mechanism.illustrates this retrieval sequence beginning with operation(Compute Distance). In operation, the search engineperforms a bulk, hardware-accelerated comparison between the loaded bit vectorsand the quantized query vector. This scoring involves computing the preliminary bitwise similarity, which includes the accumulation of weighted bit-counts, and applying the scalar correction terms derived from the asymmetric quantization decomposition.

192 120 116 132 Next, in operation(Rank Candidates), the search engineuses the preliminary similarity scores to identify a set of candidate vectors that are most relevant to the query. This step may involve selecting an oversampled subset of the best-scoring bit vectorsand retrieving their associated error correction metadatato compute a refined, corrected similarity score for final ranking.

194 120 120 In operation(Output Top-K Nearest Neighbors), the search enginegenerates a re-ranked result order based on the corrected similarity scores and returns the top-k nearest neighbor results. This complete sequence allows the search engineto provide high-recall results with low latency despite the index segments residing on disk.

112 This load-on-demand design provides high-recall search capability with significantly lower memory cost than full-graph designs, while maintaining high throughput due to compressed representation and centroid-based routing. Because the clusters most relevant to the query are fetched, the vector indexremains largely disk-resident, reducing memory pressure and enabling efficient similarity search across extremely large datasets. This load-on-demand design provides high-recall search capability with significantly lower memory cost than full-graph designs, while maintaining high throughput due to compressed representation and centroid-based routing.

1 FIG.A 102 101 103 105 115 115 152 115 105 115 115 102 102 102 102 102 Referring back to, the computing devicemay be any type of computing device that includes one or more processors, one or more memory devices, and an operating systemconfigured to execute (or assist with executing) an application. The applicationmay be a program configured to communicate with the data management platform. In some examples, the applicationis a native application installable on the operating system. In some examples, the applicationis a web application executable by a browser application. In some examples, the applicationis a web page executable by a browser application. In some examples, the computing deviceis a laptop computer. In some examples, the computing deviceis a desktop computer. In some examples, the computing deviceis a tablet computer. In some examples, the computing deviceis a smartphone. In some examples, the computing deviceis a wearable device (e.g., a head-mounted display device such as an augmented reality (AR) or a virtual reality (VR) device).

101 The processor(s)may be formed in a substrate configured to execute one or more machine executable instructions or pieces of software, firmware, or a combination thereof.

101 103 101 103 105 115 101 103 152 101 152 103 101 The processor(s)can be semiconductor-based—that is, the processors can include semiconductor material that can perform digital logic. The memory device(s)may include a main memory that stores information in a format that can be read and/or executed by the processor(s). The memory device(s)may store the operating system, including the applicationthat, when executed by the processors, performs certain operations discussed herein. In some examples, the memory device(s)store one or more portions of the data management platformthat, when executed by the processors, performs certain operations discussed with reference to the data management platform. In some examples, the memory device(s)includes a non-transitory computer-readable medium that includes executable instructions that cause at least one processor (e.g., the processors) to execute the operations discussed herein.

160 160 160 160 160 150 150 150 The server computermay be computing devices that take the form of a number of different devices, for example a standard server, a group of such servers, or a rack server system. The server computermay represent a single server computer or multiple server computer. In some examples, the server computermay represent multiple server computers that are in communication with each other. In some examples, the server computermay be a single system sharing components such as processors and memories. In some examples, the server computermay be multiple systems that do not share processors and memories. The networkmay include the Internet and/or other types of data networks, such as a local area network (LAN), a wide area network (WAN), a cellular network, satellite network, or other types of data networks. The networkmay also include any number of computing devices (e.g., computers, servers, routers, network switches, etc.) that are configured to receive and/or transmit data within the network. The networkmay further include any number of hardwired and/or wireless connections.

160 151 153 153 153 160 151 151 153 151 153 152 151 153 151 The server computer(s)may include one or more processorsformed in a substrate, an operating system (not shown) and one or more memory devices. The memory device(s)may represent any kind of (or multiple kinds of) memory (e.g., RAM, flash, cache, disk, tape, etc.). In some examples, the memory devicesmay include external storage, e.g., memory physically remote from but accessible by the server computer(s). The processor(s)may be formed in a substrate configured to execute one or more machine executable instructions or pieces of software, firmware, or a combination thereof. The processor(s)can be semiconductor-based—that is, the processors can include semiconductor material that can perform digital logic. The memory device(s)may store information in a format that can be read and/or executed by the processor(s). The memory device(s)may store one or more portions of the data management platform, that, when executed by the processor(s), perform certain operations discussed herein. In some examples, the memory device(s)includes a non-transitory computer-readable medium that includes executable instructions that cause at least one processor (e.g., the processor(s)) to execute operations.

7 FIG. 7 FIG. 7 FIG. 700 700 700 is a flowchartdepicting example operations of binary vector quantization for converting multi-dimensional data vectors into binary vectors according to an aspect. The flowchartmay depict operations of a computer-implemented method. Although the flowchartofillustrates the operations in sequential order, it will be appreciated that this is merely an example, and that additional or alternative operations may be included. Further, operations ofand related operations may be executed in a different order than that shown, or in a parallel or overlapping fashion.

702 704 706 Operationincludes computing a representative point for at least a plurality of data vectors representing data to be added to a vector index. Operationincludes generating an adjusted data vector by adjusting a data vector of the plurality of data vectors around the representative point. Operationincludes generating a bit vector using the adjusted data vector, including determining a quantization interval that reduces a quantization error in a similarity computation and mapping values of the adjusted data vector to components of the bit vector using the quantization interval.

702 In some examples, prior to operation, the data vectors may undergo a transformation via a sparse preconditioning matrix to make the components distribution approximately normal. This is a technical advantage over prior art that either may not account for component distribution or use a more expensive dense preconditioning matrix, which subsequently improves the robustness of the quantization process. The components that the transform makes normally distributed are the individual dimensional values (e.g., the floating-point values) that make up each data vector.

704 106 a The adjustment in operationmay include centering the data vectors around the computed representative point (e.g., a cluster centroid) by subtracting the representative point from each component. This centering operation may create the adjusted data vector(or residual vector), maintaining the same dimensionality as the original data vector. This residual vector may be beneficial because it isolates the local deviations of each vector, which are the components used for accurate local similarity searches.

706 Clause 1. A method comprising: computing a representative point for at least a plurality of data vectors representing data to be added to a vector index; generating an adjusted data vector by adjusting a data vector of the plurality of data vectors around the representative point; and generating a bit vector using the adjusted data vector, including: determining a quantization interval that reduces a quantization error in a similarity computation; and mapping values of the adjusted data vector to components of the bit vector using the quantization interval. Clause 2. The method of clause 1, wherein determining the quantization interval includes: determining a first boundary parameter and a second boundary parameter; executing a comparison of the adjusted data vector with a neighboring vector; and adjusting the first boundary parameter and the second boundary parameter based on the comparison to reduce the quantization error in the similarity computation. Clause 3. The method of clause 1, wherein the representative point is a first representative point, wherein the bit vector is generated using a first quantization process, the method further comprising: generating an adjusted query vector by adjusting a query vector around a second representative point; and generating a quantized query vector by applying a second quantization process to the adjusted query vector, the second quantization process being different from the first quantization process. Clause 4. The method of clause 3, further comprising: estimating a similarity metric between the quantized query vector and the bit vector using a decomposition of a dot product. Clause 5. The method of clause 4, wherein the decomposition includes at least one of: a query correction term derived from the second representative point and the query vector; a data correction term derived from the first representative point and the bit vector; and a quantized residual term derived from a comparison of the quantized query vector and the bit vector. Clause 6. The method of clause 1, further comprising: applying an orthogonal transformation to the plurality of data vectors using a preconditioning matrix, wherein the preconditioning matrix is configured to cause the components of the plurality of data vectors to be approximately normally distributed. Clause 7. The method of clause 1, further comprising: generating error correction metadata corresponding to a plurality of bit vectors; and storing the plurality of bit vectors and the error correction metadata in an index segment of the vector index, the error correction metadata being interleaved with the plurality of bit vectors. Clause 8. The method of clause 7, wherein storing the plurality of bit vectors into the index segment includes packing a plurality of quantized bit values into a byte-aligned bit vector format. Clause 9. The method of clause 7, wherein the index segment is a first index segment, the plurality of bit vectors includes first bit vectors, the method further comprising: merging the first index segment with a second index segment having second bit vectors, including: computing a combined representative point for the first index segment and the second index segment; generating adjusted bit vectors for the first bit vectors and the second bit vectors based on the combined representative point; and generating quantized bit vectors using the adjusted bit vectors. Clause 10. An apparatus comprising: a database storing a vector index; and an index engine including executable instructions that cause at least one processor to: compute a representative point for at least a plurality of data vectors representing data to be added to the vector index; generate an adjusted data vector by adjusting a data vector of the plurality of data vectors around the representative point; and generate a bit vector using the adjusted data vector, including: determine a quantization interval that reduces (e.g., minimizes) a quantization error in a similarity computation; and map values of the adjusted data vector to components of the bit vector using the quantization interval. Clause 11. The apparatus of clause 10, wherein the executable instructions include instructions that cause the at least one processor to: determine a first boundary parameter and a second boundary parameter; execute a comparison of the adjusted data vector with a neighboring vector; and adjust the first boundary parameter and the second boundary parameter based on the comparison to reduce the quantization error in the similarity computation. Clause 12. The apparatus of clause 10, wherein the representative point is a first representative point, wherein the bit vector is generated using a first quantization process, wherein the executable instructions include instructions that cause the at least one processor to: generate an adjusted query vector by adjusting a query vector around a second representative point; and generate a quantized query vector by applying a second quantization process to the adjusted query vector, the second quantization process being different from the first quantization process. Clause 13. The apparatus of clause 10, wherein the executable instructions include instructions that cause the at least one processor to: estimate a similarity metric between the quantized query vector and the bit vector using a decomposition of a dot product. Clause 14. The apparatus of clause 13, wherein the decomposition includes at least one of: a query correction term derived from the second representative point and the query vector; a data correction term derived from the first representative point and the bit vector; and a quantized residual term derived from a comparison of the quantized query vector and the bit vector. Clause 15. The apparatus of clause 10, wherein the executable instructions include instructions that cause the at least one processor to: apply an orthogonal transformation to the plurality of data vectors using a preconditioning matrix, wherein the preconditioning matrix is configured to cause the components of the plurality of data vectors to be approximately normally distributed. Clause 16. The apparatus of clause 10, wherein the executable instructions include instructions that cause the at least one processor to: generate error correction metadata corresponding to a plurality of bit vectors; and store the plurality of bit vectors and the error correction metadata in an index segment of the vector index, the error correction metadata being interleaved with the plurality of bit vectors. Clause 17. A non-transitory computer readable medium storing executable instructions that cause at least one processor to execute operations, the operations comprising: computing a representative point for at least a plurality of data vectors representing data to be added to a vector index; generating an adjusted data vector by adjusting a data vector of the plurality of data vectors around the representative point; and generating a bit vector using the adjusted data vector, including: determining a quantization interval that reduces a quantization error in a similarity computation; and mapping values of the adjusted data vector to components of the bit vector using the quantization interval. Clause 18. The non-transitory computer readable medium of clause 17, wherein the operations further comprise: determining a first boundary parameter and a second boundary parameter; executing a comparison of the adjusted data vector with a neighboring vector; and adjusting the first boundary parameter and the second boundary parameter based on the comparison to reduce the quantization error in the similarity computation. Clause 19. The non-transitory computer readable medium of clause 17, wherein the representative point is a first representative point, wherein the bit vector is generated using a first quantization process, the operations further comprising: generating an adjusted query vector by adjusting a query vector around a second representative point; and generating a quantized query vector by applying a second quantization process to the adjusted query vector, the second quantization process being different from the first quantization process. Clause 20. The non-transitory computer readable medium of clause 19, wherein the operations further comprise: estimating a similarity metric between the quantized query vector and the bit vector using a decomposition of a dot product, wherein the decomposition includes a query correction term derived from the second representative point and the query vector, a data correction term derived from the first representative point and the bit vector, and a quantized residual term derived from a comparison of the quantized query vector and the bit vector. The quantization process described in operationmay determine a quantization interval on a per-vector basis through an optimization process. The quantization interval may be defined by quantization boundary parameters. This optimization may involve first minimizing the expected square error of the quantized vector compared to the original vector followed by direct optimization of the square dot product error between the vector and one close to parallel with it. By computing (e.g., directly computing) the scaling factors for the quantization boundary parameters, the resulting bit vector may maintain high fidelity for dot product approximation. This may be a technical advantage over methods that use static or global quantization thresholds. The resulting bit vector may be a compact one-bit-per-dimension encoding.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms “machine-readable medium” “computer-readable medium” refers to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., an OLED (Organic light emitting diode) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Alternatively, this can be implemented with a 3D user interaction system making use of trackers that are tracked in orientation and 3D position. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (“LAN”), a wide area network (“WAN”), and the Internet.

The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship with each other.

In this specification and the appended claims, the singular forms “a,” “an” and “the” do not exclude the plural reference unless the context clearly dictates otherwise. Further, conjunctions such as “and,” “or,” and “and/or” are inclusive unless the context clearly dictates otherwise. For example, “A and/or B” includes A alone, B alone, and A with B. Further, connecting lines or connectors shown in the various figures presented are intended to represent example functional relationships and/or physical or logical couplings between the various elements. Many alternative or additional functional relationships, physical connections or logical connections may be present in a practical device. Moreover, no item or component is essential to the practice of the implementations disclosed herein unless the element is specifically described as “essential” or “critical”.

Terms such as, but not limited to, approximately, substantially, generally, etc. are used herein to indicate that a precise value or range thereof is not required and need not be specified. As used herein, the terms discussed above will have ready and instant meaning to one of ordinary skill in the art. Moreover, use of terms such as up, down, top, bottom, side, end, front, back, etc. herein are used with reference to a currently considered or illustrated orientation. If they are considered with respect to another orientation, it should be understood that such terms must be correspondingly modified.

Although certain example methods, apparatuses and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. It is to be understood that terminology employed herein is for the purpose of describing particular aspects and is not intended to be limiting. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.