Secure Large Language Model (llm) Deployment in an Enterprise

This application is a continuation of U.S. Non-Provisional application Ser. No. 19/169,074, filed Apr. 3, 2025, and claims the benefit of U.S. Provisional Application No. 63/575099, filed Apr. 5, 2024, which is herein incorporated by reference.

The present disclosure relates to the field of machine learning (ML), and more specifically to secure large language model (LLM) deployment for an enterprise, including cloud-based enterprises.

An enterprise may wish to harness the power of machine learning (ML) and develop and train a large language model (LLM), allowing its employees or customers to intelligently search and query data files and documents stored in the enterprise's databases. However, use of various hosted LLM services may compromise security of private enterprise data, allow unauthorized access to confidential information, and may result in catastrophic data breaches. Alternatively, local LLM solutions are often too complicated and costly for many enterprises to develop and maintain on their own. The use of encrypted LLMs, whether hosted or local, provides the necessary level of security to the enterprise data. However, current LLM hosting encryption schemes require significant computational resources and, hence, are prohibitively costly for Fully Homomorphic Encryption (FHE) approaches. It is also very complex to implement granular access control because of the monolithic nature of LLM data, where common and confidential data are often mixed. Encryption schemes that manage to reduce the processing costs, do so at the expense of reduced accuracy. Therefore, there is a need for a secure, layered, efficient and accurate LLM deployment for enterprises.

Aspects of the disclosure relate to systems, methods, and computer program products for providing secure LLM deployment for an enterprise.

In some aspects, the techniques described herein relate to a method for providing a secure large language model (LLM) deployment for an enterprise, the method including: identifying one or more documents in an enterprise database including confidential data; setting up an encryption scheme for an LLM to be executed on a server, wherein the encryption scheme enables operations and data of the LLM to remain private and undecodable on the server; generating, using the encryption scheme, at least one encrypted value of at least one portion of the confidential data, for inclusion in a training dataset; assigning, to an authorized user of the confidential data, a decryption key for decrypting the at least one encrypted value; training, using the training dataset, the LLM to generate responses for input queries pertaining to information in the one or more documents, wherein the LLM is configured to include the at least one encrypted value in place of the at least one portion of the confidential data in any responses that are to include information from the at least one portion; generating, by the LLM, a response to an input query requesting the information from the at least one portion of the confidential data, wherein the response includes the at least one encrypted value and keeps the information from the at least one portion hidden; in response to determining that the input query is received from the authorized user: decrypting the at least one encrypted value using the decryption key assigned to the authorized user; and outputting, on a computing device, the response with the information from the at least one portion visible to the authorized user.

In some aspects, the techniques described herein relate to a method, wherein the one or more documents further include nonconfidential data, and wherein the training dataset includes unencrypted values corresponding to the nonconfidential data such that the LLM is trained to output both the unencrypted values and encrypted values based on information requested in a given input query.

In some aspects, the techniques described herein relate to a method, wherein assigning the decryption key to the authorized user includes: identifying an access control list (ACL) of the enterprise that indicates access levels of a plurality of users; determining an access level required to access the at least one portion of the confidential data; and assigning the decryption key to the authorized user in response to determining that an access level of the authorized user is equal to or greater than the access level required to access the at least one portion.

In some aspects, the techniques described herein relate to a method, further including: in response to determining that the input query is not received from the authorized user, outputting the response without decrypting the at least one encrypted value.

In some aspects, the techniques described herein relate to a method, wherein the encryption scheme is a partially homomorphic encryption (PHE) algorithm.

In some aspects, the techniques described herein relate to a method, wherein the encryption scheme is a fully homomorphic encryption (FHE) algorithm.

In some aspects, the techniques described herein relate to a method, further including: identifying a plurality of matrix operations that are performed during the training of the LLM and that are associated with the confidential data; and encrypting the plurality of identified matrix operations using the encryption scheme, wherein encrypting further includes: encrypting the at least one portion of the confidential data and related intermediate data or metadata stored in a matrix of the LLM; and encrypting logical operations performed on the matrix.

In some aspects, the techniques described herein relate to a method, wherein the LLM is initially trained using one or more other documents from the enterprise database that do not include any confidential data, and wherein the LLM is re-trained using the training dataset.

In some aspects, the techniques described herein relate to a method, wherein the LLM is a 1-bit LLM where an operation of multiplication of matrix to vector is replaced by changes of sign and addition.

In some aspects, the techniques described herein relate to a method, wherein LLM is a 1-bit LLM where an operation of multiplication of matrix to vector is replaced by an omission of one or more vector members, wherein the omission is an equivalent of multiplication by 0.

In some aspects, the techniques described herein relate to a method, wherein the LLM is deployed on a local enterprise server.

In some aspects, the techniques described herein relate to a method, wherein the LLM is deployed on a remote host server and/or in the cloud.

In some aspects, the techniques described herein relate to a system for providing a secure large language model (LLM) deployment in an enterprise, the system including: at least one memory; and at least one hardware processor coupled with the at least one memory and configured, individually or in combination, to: identify one or more documents in an enterprise database including confidential data; set up an encryption scheme for an LLM to be executed on a server, wherein the encryption scheme enables operations and data of the LLM to remain private and undecodable on the server; generate, using the encryption scheme, at least one encrypted value of at least one portion of the confidential data, for inclusion in a training dataset; assign, to an authorized user of the confidential data, a decryption key for decrypting the at least one encrypted value; train, using the training dataset, the LLM to generate responses for input queries pertaining to information in the one or more documents, wherein the LLM is configured to include the at least one encrypted value in place of the at least one portion of the confidential data in any responses that are to include information from the at least one portion; generate, by the LLM, a response to an input query requesting the information from the at least one portion of the confidential data, wherein the response includes the at least one encrypted value and keeps the information from the at least one portion hidden; in response to determining that the input query is received from the authorized user: decrypt the at least one encrypted value using the decryption key assigned to the authorized user; and output, on a computing device, the response with the information from the at least one portion visible to the authorized user.

In some aspects, the techniques described herein relate to a non-transitory computer readable medium storing thereon computer executable instructions for providing a secure large language model (LLM) deployment in an enterprise, including instructions for: identifying one or more documents in an enterprise database including confidential data; setting up an encryption scheme for an LLM to be executed on a server, wherein the encryption scheme enables operations and data of the LLM to remain private and undecodable on the server; generating, using the encryption scheme, at least one encrypted value of at least one portion of the confidential data, for inclusion in a training dataset; assigning, to an authorized user of the confidential data, a decryption key for decrypting the at least one encrypted value; training, using the training dataset, the LLM to generate responses for input queries pertaining to information in the one or more documents, wherein the LLM is configured to include the at least one encrypted value in place of the at least one portion of the confidential data in any responses that are to include information from the at least one portion; generating, by the LLM, a response to an input query requesting the information from the at least one portion of the confidential data, wherein the response includes the at least one encrypted value and keeps the information from the at least one portion hidden; in response to determining that the input query is received from the authorized user: decrypting the at least one encrypted value using the decryption key assigned to the authorized user; and outputting, on a computing device, the response with the information from the at least one portion visible to the authorized user.

The method and system of the present disclosure are designed to provide LLM functionality while preserving data security.

Exemplary aspects are described herein in the context of a system, method, and a computer program for providing a secure large language model (LLM) deployment in an enterprise IT environment. Those of ordinary skill in the art will realize that the following description is illustrative only and is not intended to be in any way limiting. Other aspects will readily suggest themselves to those skilled in the art having the benefit of the disclosure. Reference will now be made in detail to implementations of the example aspects as illustrated in the accompanying drawings. The same reference indicators will be used to the extent possible throughout the drawings and the following description to refer to the same or like items.

1 FIG.A 6 FIG. 100 100 illustrates a block diagram of an exemplary systemfor providing a secure local LLM deployment in an enterprise network. In one aspect, the components of systemmay be implemented on computer systems, such as that shown in.

100 101 121 123 100 101 101 101 122 101 121 123 1 FIG.A In one aspect, the systemincludes an enterprise networkwhich includes at least servers-. It is noted that the systemincludes any number of other network components andonly shows the components relevant for the illustrative example of the present disclosure. Users of the enterprise network(e.g., employees or customers) communicate with devices in the enterprise networkvia one of the servers, e.g., user A communicates with components of the enterprise networkvia server, and user B communicates with components of the enterprise networkvia server. Notably, the LLM of the present embodiment is implemented on the local enterprise server.

101 111 112 113 113 111 113 1 112 1 1 111 113 In addition, the enterprise networkincludes any number of database servers, such as the database serversand. In one aspect, data of the enterprise network may also be stored on a cloud storage device, such as the storage device(also referred to as database server). Thus, files of the enterprise network may be stored in any of the database servers-. For example, files-M, are shown as being stored on the database server. In one aspect, the files-M may contain any number of portions of data, with some portions being confidential data. Thus, at least some of the portions of the files-M may also be encrypted and stored on any of the database servers-.

1 FIG.B 130 140 130 illustrates a block diagram of an exemplary systemfor providing a secure hosted LLM deployment on a remote serverfor an enterprise. Thus, the systemis for the scenario in which the enterprise network accesses LLM functionality from a service provider (e.g., cloud service provider) rather than deploying the functionality on a server of the enterprise.

130 101 121 123 101 102 101 101 101 101 122 101 121 140 102 In one aspect, the systemincludes an enterprise networkwhich includes at least servers-. The enterprise networkis communicatively coupled to an LLM service provider networkfor accessing LLM functionalities. That is, rather than deploying the LLM functionality on the enterprise network, the enterprise subscribes to the LLM functionality from a service provider. Users of the enterprise networkcommunicate with devices in the enterprise networkvia one of the servers, e.g., user A communicates with components of the enterprise networkvia server, and user B communicates with components of the enterprise networkvia server. The LLM of the present disclosure is implemented on the serverlocated in the LLM service provider's network.

101 111 112 113 111 113 1 112 1 1 111 113 In addition, the enterprise networkalso includes any number of database servers, such as the database serversand. In one aspect, data of the enterprise network may also be stored on a cloud storage device, such as the storage device. Thus, files of the enterprise network may be stored in any of the database servers-. For example, files-M, are shown as being stored on the database server. In one aspect, the files-M may contain any number of portions of data, with some portions containing confidential data, such as salary information, medical records, customer lists, etc., which has a restricted access to the employees of the enterprise, much less to the outside vendors, such as hosted LLM service providers. Thus, at least some of the portions of the files-M may also be encrypted and stored on any of the database servers-.

140 101 140 140 140 111 113 111 113 To enable enterprise employees to use LLM services to intelligently search and query data files and documents stored in the enterprise database, in one exemplary aspect, the LLM servermay be configured to operate on the encrypted confidential data of the enterprise network. Particularly, in one aspect, the LLM servermay be configured to perform LLM training, LLM fine-tuning, and LLM inference (and any other required operations) using the encrypted data without being able to decrypt it, which provides a high-degree of security to the enterprise data. Thus, the 1-bit LLM functionality installed on LLM serverhas no access to encrypted versions of the confidential data. Moreover, in another example aspect, the user prompts may also be encrypted to allow an even greater degree of confidentiality. In another aspect, the LLM serveraccesses data stored in the database servers-, and performs all LLM operations including the encrypting of the content stored on the database servers-. Thus, the LLM service provider is a trusted service provider and can have access to unencrypted data stored in various databases of the enterprise. In this scenario, the training, retraining, and fine-tuning of the LLM may be performed by the trusted service provider.

For an illustrative non-limiting example, suppose the enterprise network comprises a hospital network with users having access to different portions of data stored in various databases of the hospital. In one aspect, the hospital may obtain LLM services from a trusted service provider. The trusted service provider may then access the data, encrypt the data as needed, set up access lists (if applicable) for various groups of users (e.g., doctors, nurses, administrators, IT personal, etc.), provide decryption keys to users allowed to access certain portions of data, etc. For example, portions of the medical records containing patients' names may be encrypted, but the information about patient's medical condition, treatment protocols and the results of the treatment may remain unencrypted. The LLM may be trained on these partially encrypted filed. When a query is received from a user for an LLM service (e.g., search for information about successful treatment of a particular medical condition), after authenticating the user and checking his access level, the inference module of the LLM server may generate a response to the user prompt. For example, the LLM, which was trained on the patient records, may identify successful treatment cases and summarize conditions of patients and their treatment protocols without revealing patients' names if users access level prohibits access to this information.

2 FIG. 200 101 140 200 210 220 230 240 250 is an example of a block diagram of functional modules of the systemfor secure LLM deployment for an enterprise according to one exemplary aspect. Some of these functional modules may be deployed locally on the servers of the enterprise networkor hosted on a remote server such as server. In one example aspect, the systemincludes the following functional modules: a user interface, an encryption/decryption module, an authentication module, an LLM server, and enterprise databases.

210 210 210 250 240 250 240 240 250 240 250 In one aspect, the user interfaceis designed to enable user endpoint devices to access enterprise's LLM functionality in a secure and confidential manner. User interfacemay be implemented as web-based interface or a desktop application. The user interfaceallows users to use text prompts to perform text-based searches for documents in enterprise database, to query the LLM serverfor answers to specific questions related to the documents and files stored in the enterprise database, or, depending on the natural language processing capabilities of the LLM server, to simulate a conversation with the LLM serveron topics related to the documents contained in the databaseor other topics on which the LLM serverhas been trained to answer. In one aspect, the access to the LLM services and/or to confidential documents in the enterprise databaseis allowed to authenticated users only and/or users who have an appropriate level of access (e.g., doctors, administrators, IT staff, etc.).

230 210 231 232 233 In one aspect, the authentication moduleis provided to enable authentication of users that access LLM services of the enterprise via the interface. In one example, the authentication may be performed using an Access Control List (ACL), identifying individual users and their respective access level to documents in the enterprise database. In another example, the authentication can be performed using cryptographic techniques, such as digital certificatesassociate with individual users. Yet in another example, various authentication rulesmay be used to specify the access level of individual users or groups/categories of users, what confidential data is accessible to the users, whether user's LLM prompts should be encrypted, etc. Alternatively, a combination of these and other known authentication techniques may be used.

231 For example, if a user query does not include the key(s) associated with an authorized user (as indicated in ACL), basic unencrypted LLM data and matrices are used. If the keys are provided, depending on the level of access, whole matrices and LLM data with both encrypted and encrypted data may be used. In some aspects, different LLMs are trained, each with a different amount of access to data. For example, a limited LLM may be able to provide simple answers without confidential data. A full LLM may provide more advanced answers for users having access keys.

240 220 101 222 223 240 In order to access LLM services external to the enterprise while maintaining the security of user prompts and confidential enterprise data, the enterprise may encrypt its confidential data using homomorphic encryption that allows LLM serverto perform operations on the encrypted data without decryption thereof. In one example, the encryption/decryption moduleis deployed on a server in the enterprise networkand configured to perform encryption/decryption of confidential data using a Partially Homomorphic Encryptions (PHEs) algorithm. In another example, a Fully Homomorphic Encryptions (FHEs) algorithmmay be used. An advantage of using PHE is that it is more efficient than FHE, particularly for 1-Bit LLM implementations. Specially, when data is encrypted using FHE, LLM serverperforms both matrix-vector addition and multiplication operations during LLM training and inference. In PHE, either matrix-vector addition or multiplication operations (but not both) are performed on the encrypted data, which is much more efficient.

220 220 221 220 221 Furthermore, since homomorphic encryption used by the moduleis a form of asymmetric encryption algorithm that uses private/public key pairs for encryption and decryption of data files, modulemay store all generated cryptographic key pairs in a datastore. Furthermore, since modulemay be also configured to encrypt user prompts, which provides an extra level of security and confidentiality to the enterprise, the cryptographic keys generated for each user to encrypt his/her prompts are also stored in the datastore.

PHE is a cryptographic technique that enables specific types of computations on encrypted data while maintaining its confidentiality. Unlike FHE, which allows arbitrary computations on encrypted data, PHE supports only certain operations (e.g., addition, multiplication-but not both simultaneously). Accordingly, when matrix operations involving addition or multiplication are performed by an LLM to generate outputs, the operations remain successful and generate proper results despite the encryption. In another example, suppose that the LLM is trained on a document that states “Mary was born on Jan. 1, 1990.” If the birthdate is encrypted (suppose that the encrypted value generated using an encryption key is 123432), the modified document may state “Mary was born on 123432.” The LLM may be trained using this modified document, which prevents the actual birthdate from being leaked/stolen. The trained LLM may generate an output stating “Mary's birthdate is 123432” to a user query “what is Mary's birthdate?”. Here, the output includes the encrypted value of the birthdate. A user with a decryption key may be able to generate the statement “Mary's birthdate is Jan. 1, 1990” using this key.

In some aspects, the PHE used in the present disclosure may be the Paillier cryptosystem, which supports addition operations on encrypted values. This means that one can perform additions on ciphertexts without decrypting them first. PHE is valuable in scenarios where specific computations need to be performed on sensitive data while it remains encrypted, such as in privacy-preserving computations in the cloud or secure multi-party computations. By allowing limited operations on encrypted data, PHE strikes a balance between data utility and confidentiality, enabling practical applications of secure computation in various domains, including finance, healthcare, and decentralized systems. In some aspects, PHE schemes can be performed with a pair of keys based on, for example, RSA (a public-key cryptosystem). In other aspects, PHE schemes can be performed with a single key based on, for example, the Paillier cryptosystem.

200 240 240 240 242 242 243 241 243 1 FIG. 2 FIG. In one example aspect, the systemfurther comprises an LLM serverthat executes an LLM program. The LLM servermay be deployed on a local enterprise server, as shown in, or on a remote host server, as shown in. The LLM serverincludes a LLM training module, LLM inference module, and LLM fine-tuning module. The training moduleis configured to train LLM on files stored in enterprise database. In one aspect, an LLM may be trained both on the unencrypted files that do not contain any confidential data and encrypted files that contain confidential data. In another aspect, LLM may be pretrained using unencrypted files, and then finetuned by moduleusing encrypted files. Notably, PHE encryption allows LLM training, finetuning, and inference to be performed on the encrypted files. Particularly, matrix-vector mathematical operations can be performed on the encrypted data. This allows enterprise to use LLM services while maintaining the secrecy of the confidential data.

243 243 In one aspect, fine-tuning modulemay implement Low-Rank Adaptation (LoRA) algorithm, which provides high-efficiency LLM optimization. For example, prompts and corresponding responses (e.g., samples from historical data) may be used for fine-tuning the LLM for a specific task. The fine-tuning using the LoRA technique involves differentiating new elements that are not well represented in previous training sets of data and modified elements that are recognized, but not adequately represented in previous training sets of data, and then modifying a small portion of weights of the model for performing the fine-tuning. Thus, the weights of the model affected by the new elements and modified elements are changed to improve the accuracy of the LLM training. In one aspect, the LoRA fine-tuning moduleof the present disclosure is used to further optimize the performance on the PHE encrypted data. LoRA-related data may be stored separately and be encrypted, e.g., by the PHE algorithm, in the same way as described above.

In terms of training, the LLM may be trained through a process called unsupervised learning on a large dataset comprised of text from across various sources (e.g., webpages, documents, articles, etc.). The training begins by initializing the model with random parameters. The LLM then processes sequences of text, ranging from a few words to entire paragraphs, predicting the next word in each sequence. These predictions are compared to the actual next words in the dataset, and the model adjusts its parameters to minimize the difference between its predictions and the actual text. This process, known as backpropagation, is repeated iteratively over several (millions or possibly billions) text examples, allowing the model to learn intricate patterns, grammar rules, contextual understanding, and semantic relationships. The model's objective during training is to maximize the likelihood of generating the correct next word given a sequence of previous words. Additionally, fine-tuning techniques may be applied to adapt the model to specific tasks or domains, further enhancing its performance and applicability. Through this iterative process, the LLM gradually develops a nuanced understanding of language and can generate coherent and contextually appropriate responses to a wide range of queries.

3 FIG. 300 310 300 illustrates a methodfor providing a secure LLM deployment in an enterprise in accordance with aspects of the present disclosure. In step, methodidentifies one or more files in an enterprise database containing confidential data. The enterprise database is configured to limit access to the confidential data based on an encryption of the confidential data.

In one aspect, the limit to the access to the confidential data is further based on a user's access level. For example, user A may have a different access level from user B. Moreover, based on their respective roles in the enterprise, users A and B may have different needs for accessing different portions of the confidential data. For instance, if the enterprise is a hospital, doctors, nurses, patients, hospital administrators, IT personal etc., would have differing needs for accessing confidential data. Thus, an access control list (ACL) may be used to facilitate compliance to established policies and regulations. The ACL may be implemented on any of the servers of the enterprise. Gateway devices communicating with users may then access the ACL to determine whether access to confidential data is to be granted to a particular user. As mentioned above, a user may be granted access to specific portions of confidential data.

Thus, in one aspect, the determination of whether the user from whom the request is received is one of the one or more authorized users is further based on an ACL of the enterprise.

320 300 In step, by a server, methodencrypts at least one portion of the confidential data in the identified files using a partial homomorphic encryption (PHE) algorithm, and provides decryption keys to one or more authorized users of the confidential data.

In one aspect, the encrypting of the at least one portion of the confidential data further includes: identifying a plurality of matrix-vector operations, performed during the training of the LLM, that are associated with the confidential data; and encrypting the plurality of identified matrix-vector operations using the PHE algorithm, wherein encrypting further includes: encrypting the confidential data stored in the matrix, and encrypting logical operations performed on vector-matrix.

330 300 In step, by the server, methodtrains the LLM using at least the files containing the encrypted confidential data. Once the training of the LLM is completed, the LLM server is ready to respond to prompts by performing an inference operation.

In one aspect, the LLM is a 1-bit LLM where an operation of multiplication of matrix to vector is efficiently replaced by changes of sign and addition.

In one aspect, the training of the LLM comprises: taking a LLM partially trained at least on files from enterprise database that do not contain any confidential data; and completing the training using the files containing the encrypted confidential data.

340 300 In step, by the server, methodreceives a query from a user, wherein the query comprises a request (i) for searching for the one or more files containing the confidential data or (ii) for obtaining information associated with said one or more files.

350 300 360 395 In step, by the server, methoddetermines whether the user from whom the request is received is one of the one or more authorized users of (i) the one or more files containing the confidential data or (ii) the information associated with said one or more files containing the confidential data. When the user from whom the request is received is one of the one or more authorized users, the method proceeds to step. When the user from whom the request is received is not one of the authorized users, the method proceeds to step.

In one aspect, the determination of whether the user from whom the request is received is one of the one or more authorized users, includes: identifying one or more files associated with the query received from the user; for each identified file associated with the query received from the user which is among the one or more files containing the confidential data, applying the ACL of the enterprise; and generating the response by executing the inference operation only on the one or more files for which the user's access level is determined as being sufficient.

360 300 In step, by the server, methodgenerates a response to the query by executing an inference operation using the LLM. For example, the server may prompt an LLM server for a response to the query.

In one aspect, the LLM operation may be implemented on the same server as the server interacting with the user. In another aspect, the server interacting with the user is distinct from the server performing the LLM operations.

In one aspect, the LLM is deployed on a server located in the network of the enterprise. In another aspect, the LLM is deployed on a remote server, which may be a cloud server or a server of a service provider providing LLM functionality to the enterprise.

370 300 In step, by the server, methodprovides a response to the query generated by the LLM, wherein, when the response includes the at least one portion of the confidential data that is encrypted, the encrypted portion of the confidential data is decryptable using the decryption key provided to the user of the one or more authorized users.

210 222 240 240 242 220 210 In one aspect, the generating of the response to the query by executing the inference operation using the LLM comprises: prompting the LLM using encrypted prompts, thereby an LLM hosting platform that performs the inference operation replies to the prompt without decrypting the encrypted at least one portion of confidential data. For example, the prompt from the user is processed by the user interfaceto generate a vector of features of the prompt. Then, the PHEis used to encrypt the vector and send the resulting encrypted prompt to the LLM server. The LLM serveroperates on the encrypted prompt to generate a response via the LLM inference module, and sends the generated response. Then, the response is decrypted by encryption/decryption moduleand sent to the user interface.

In one aspect, the response to the query from the user includes at least encrypted portions of (i) confidential data or (ii) information associated with said one or more files containing the confidential data.

In one aspect, once the computing device of the user receives the response from the server, the computing device of the user decrypts the encrypted portions of the (i) confidential data or (ii) the information associated with said one or more files containing the confidential data, to obtain decrypted data. Then, the computing device of the user presents the decrypted data to the user on a display device associated with the computing device of the user.

380 300 320 340 Thus, in optional step, by the computing device of the user, methoddecrypts the encrypted portions of the (i) confidential data or (ii) the information associated with said one or more files containing the confidential data, to obtain decrypted data; and presents the decrypted data to the user on a display device associated with the computing device of the user. The method then proceeds to stepand/orto continue encrypting newly received confidential data and/or receive queries from users.

395 300 320 340 In step, by the server, methodprovides a response to the query denying the request. The method then proceeds to stepand/orto continue encrypting newly received confidential data and/or receive queries from users.

In one aspect, operations of the enterprise other than the operations provided using the secure LLM are performed on unencrypted data.

In one aspect, operations of the enterprise other than the operations provided using the secure LLM are performed on data encrypted using a Fully Homomorphic Encryption (FHE) algorithm.

In one aspect, the method further comprises: executing steps without decrypting the at least one portion of the confidential data that is encrypted, at least for one of: inference operations, training of algorithms, retraining of algorithms, data preparation and specialization of the algorithm for a specific application.

300 300 4 FIG. As described above, during execution of the steps of method, the enterprise database is configured to limit access to the confidential data based on an encryption of the confidential data. However, the ACL was an optional feature. The usage of the ACL when it is not optional is further described below in conjunction with. Methodmainly uses encryption techniques for data security by providing the decrypting keys only to authorized users. Thus, users of the enterprise network may be provided different decryption keys for accessing different portions of confidential data. Alternatively, a method for providing the secure LLM may use both the encryption and the ACL in an integrated manner.

4 FIG. 400 illustrates an example of a methodfor providing a secure LLM deployment in an enterprise using encryption and Access Control List (ACL) in accordance with aspects of the present disclosure.

410 400 In optional step, methodreceives a partially trained LLM algorithm and stores the partially trained LLM on a server, e.g., a server of the enterprise.

415 400 In step, methodidentifies one or more files in an enterprise database containing confidential data. The enterprise database is configured to limit access to the confidential data based on an encryption of the confidential data and usage of ACL.

420 400 In step, by a server, methodencrypts at least one portion of the confidential data in the identified files using a PHE algorithm, and provides decryption keys to one or more authorized users of the confidential data.

425 400 In step, by a server, methodfine-tunes the trained LLM using files containing the encrypted confidential data.

440 400 In step, by the server, methodreceives a query from a user, wherein the query comprises a request (i) for searching for the one or more files containing the confidential data or (ii) for obtaining information associated with said one or more files.

445 400 In step, by the server, methodauthenticates the user.

450 400 400 455 490 In step, by the server, methoddetermines whether the user is authenticated successfully. When the user is authenticated successfully, methodproceeds to step. Otherwise, the method proceeds to step.

455 400 In step, by the server, methoddetermines the access level of the user from whom the query is received.

460 400 400 465 400 490 In step, by the server, methoddetermines whether the access level of the user permits access to the one or more files containing the confidential data or (ii) the information associated with said one or more files containing the confidential data. When the access level of the user permits access to the confidential data or (ii) information associated with said one or more files, methodproceeds to step. When the access level of the user does not permit access to the confidential data or (ii) for obtaining information associated with said one or more files, methodproceeds to step.

465 400 In step, by the server, methodgenerates a response to the query by executing an inference operation using the LLM.

470 400 In step, by the server, methodprovides a response to the query generated by the LLM, wherein, when the response includes the at least one portion of the confidential data that is encrypted, the encrypted portion of the confidential data is decryptable using the decryption key provided to the user of the one or more authorized users.

480 400 In optional step, by the computing device of the user, methoddecrypts the encrypted portions of the (i) confidential data or (ii) the information associated with said one or more files containing the confidential data, to obtain decrypted data; and presents the decrypted data to the user on a display device associated with the computing device of the user.

490 400 440 420 In step, methoddenies the query. The method may then proceed to stepto receive more queries, or to stepto receive more data for encryption.

In one aspect, the LLM is a 1-bit LLM where an operation of multiplication of matrix to vector is efficiently replaced by changes of sign and addition.

In one aspect, the LLM is deployed on a local enterprise server.

In one aspect, the LLM is deployed on a remote host server.

In one aspect, encrypting at least the confidential data further includes: identifying a plurality of matrix-vector operations, performed during the training of the LLM, that are associated with the confidential data; and encrypting the plurality of identified matrix-vector operations using the PHE algorithm, wherein encrypting further includes: encrypting the confidential data stored in the matrix, and encrypting logical operations performed on vector-matrix.

In one aspect, the response to the user's query includes at least encrypted portions of (i) confidential data or (ii) information associated with said one or more files containing the confidential data.

In one aspect, the determination of whether the user's access level permits access to (i) the one or more files containing the confidential data or (ii) the information associated with said one or more files containing the confidential data, includes: identifying one or more files associated with the user's query; for each identified file associated with the user's query which is among the one or more files containing the confidential data, applying the ACL of the enterprise; and generating the response to the user's query by executing the inference operation only on the one or more files for which the user's access level is determined as being sufficient.

In one aspect, operations of the enterprise other than the operations provided using the secure LLM are performed on unencrypted data.

In one aspect, operations of the enterprise other than the operations provided using the secure LLM are performed on data encrypted using a Fully Homomorphic Encryption (FHE) algorithm.

In one aspect, the method further comprises executing steps without decrypting the at least one portion of the confidential data that is encrypted, at least for one of: inference operations, training of algorithms, retraining of algorithms, data preparation and specialization of the algorithm for a specific application.

In one aspect, the generating of the response to the query by executing the inference operation using the LLM comprises: prompting the LLM using encrypted prompts, thereby an LLM hosting platform that performs the inference operation replies to the prompt without decrypting the encrypted at least one portion of confidential data.

Integrating PHE into training a LLM involves encrypting the sensitive data involved in the training process, such as the training data itself, gradients, or model parameters.

In one aspects, training data is encrypted using PHE before being sent to the training server. This ensures that the data remains confidential throughout the training process. Techniques like additive or multiplicative homomorphic encryption can be used based on the specific operations required during training.

5 FIG. 500 illustrates methodfor providing a secure LLM deployment in an enterprise.

510 220 111 At step, moduleidentifies one or more documents in an enterprise database comprising confidential data (e.g., database). For example, the one or more documents may include medical records indicating the medical history of multiple patients. Certain fields of these documents may include confidential data such as patient date of births, addresses, etc.

515 220 240 At step, modulesets up an encryption scheme for an LLM to be executed on a server (e.g., LLM). This encryption scheme enables operations and data of the LLM to remain private and undecodable on the server. In some aspects, the encryption scheme is a partially homomorphic encryption (PHE) algorithm. In other aspects, the encryption scheme is a fully homomorphic encryption (FHE) algorithm.

520 220 At step, modulegenerates, using the encryption scheme, at least one encrypted value of at least one portion of the confidential data, for inclusion in a training dataset. For example, the confidential data may state “the birthdate of Mary is Jan. 1, 1990.” The birthdate may be the portion of the confidential data that gets encrypted. Accordingly, the confidential data states “the birthdate of Mary is 123423,” where 123432 is the encrypted value.

525 220 220 220 At step, moduleassigns, to an authorized user of the confidential data, a decryption key for decrypting the at least one encrypted value. In some aspects, this may involve identifying an access control list (ACL) of the enterprise that indicates access levels of a plurality of users. For example, the access levels may be “low,” “medium,” and “high.” Modulemay determine an access level required to access the at least one portion of the confidential data. For example, to access birthdates, the user may need at least “medium” access as listed in the ACL. Modulemay then assign the decryption key to the authorized user in response to determining that an access level of the authorized user (e.g., “high”) is equal to or greater than the access level required to access the at least one portion.

530 241 At step, moduletrains, using the training dataset, the LLM to generate responses for input queries pertaining to information in the one or more documents. The LLM is specifically configured to include the at least one encrypted value in place of the at least one portion of the confidential data in any responses that are to include information from the at least one portion. In other words, if the output is supposed to include the date Jan. 1, 1990, the LLM outputs 123432 instead.

At the start of training, the LLM parameters are initialized. In some aspects, these parameters are in an encrypted form. These parameters are updated during the training process. More specifically, during each training iteration, the encrypted data is used to perform forward and backward passes through the LLM. The model computes gradients with respect to the encrypted data. The gradients are aggregated across multiple training samples or batches, preserving their encrypted form. In some aspects, techniques such as secure aggregation may be employed to ensure that the gradients are combined without revealing any sensitive information. The aggregated gradients are then used to update the model parameters. Since the gradients are encrypted, this update process is performed in the encrypted domain. Periodically, or at the end of training, the model parameters may be decrypted to evaluate the model's performance on a validation dataset. This step allows assessing the effectiveness of the training process without compromising data privacy. Once training is complete, the trained model may be deployed for inference on new, unencrypted data.

In some aspects, the LLM is initially trained using one or more other documents from the enterprise database that do not comprise any confidential data, and wherein the LLM is re-trained using the training dataset. For example, the LLM may be a pretrained transformer such as ChatGPT, Gemini, Claude, Llama, Mistral, etc. These models may be re-trained on the training dataset to finetune the outputs. Some of the layers in these models may be trained using confidential data and updates may be stored in PHE or FHE encrypted form. There may be different layers with different added data and training procedures using different encryption keys.

In some aspects, the LLM is a 1-bit LLM where an operation of multiplication of matrix to vector is replaced by changes of sign and addition. In some aspects, the operation of multiplication of matrix to vector is replaced by an omission of one or more vector members, wherein the omission is an equivalent of multiplication by 0.

In some aspects, the LLM is deployed on a local enterprise server. In some aspects, the LLM is deployed on a remote host server and/or in the cloud.

535 242 At, modulegenerates, by the LLM, a response to an input query requesting the information from the at least one portion of the confidential data. The response includes the at least one encrypted value and keeps the information from the at least one portion hidden. A Large Language Model (LLM) generates a response to a query like “What is Mary's birthdate?” using its understanding of language patterns learned during training on large datasets. Here's a simplified overview of the process. The LLM first processes the input query, “What is Mary's birthdate?” by tokenizing the query into individual words and represents them in a numerical format that the model can understand. These numerical values include the encrypted value. The numerical values may be inputs in a matrix operation (e.g., addition and/or multiplication), which results in the output response. As mentioned before, PHE and FHE enable the encrypted values to be part of the matrix operations without causing the output to be improper. In this scenario where the birthdates are encrypted for privacy, the LLM does not have direct access to the birthdates themselves. Based on the information retrieved about Mary's birthdate, the LLM generates a response with the encrypted value corresponding to the birthdate (e.g., Mary's birthdate is 123432).

540 230 500 545 220 220 221 At, moduledetermines whether the input query is received from the authorized user. In response to determining that the input query is received from the authorized user (e.g., based on metadata accompanying the query), methodadvances to, where moduledecrypts the at least one encrypted value using the decryption key assigned to the authorized user. For example, modulemay determine that the encrypted value 123432 is encrypted by an encryption key that pairs with the decryption key (stored in store). The decrypted value may be Jan. 1, 1990.

545 210 At, user interfaceoutputs, on a computing device, the response with the information from the at least one portion visible to the authorized user. For example, the final response may be “Mary's birthdate is Jan. 1, 1990.

540 500 555 210 In response to determining that the input query is not received from the authorized user at step, methodadvances to step, where user interfaceoutputs the response without decrypting the at least one encrypted value (e.g., “Mary's birthdate is 123432”).

It should be noted that the one or more documents further comprise nonconfidential data (e.g., Mary's name). Accordingly, the training dataset also includes unencrypted values corresponding to the nonconfidential data. As a result, the LLM is trained to output both the unencrypted values and encrypted values based on information requested in a given input query. In the output example given, the unencrypted values are “Mary's,” “birthdate,” “is.”

220 220 In some aspects, when generating the encryption scheme and the training dataset, modulemay identifying a plurality of matrix operations that are performed during the training of the LLM and that are associated with the confidential data. Based on the identified operations, the encryption scheme may involve encrypting the plurality of identified matrix operations. More specifically, modulemay encrypt the at least one portion of the confidential data and related intermediate data or metadata stored in a matrix of the LLM, and may further encrypt logical operations performed on the matrix.

6 FIG. 20 20 is a block diagram illustrating a computer systemon which aspects of systems and methods for providing a secure LLM deployment in an enterprise may be implemented. The computer systemcan be in the form of multiple computing devices, or in the form of a single computing device, for example, a desktop computer, a notebook computer, a laptop computer, a mobile computing device, a smart phone, a tablet computer, a server, a mainframe, an embedded device, and other forms of computing devices.

20 21 22 23 21 23 21 21 22 21 22 25 24 26 20 24 2 As shown, the computer systemincludes a central processing unit (CPU), a system memory, and a system busconnecting the various system components, including the memory associated with the central processing unit. The system busmay comprise a bus memory or bus memory controller, a peripheral bus, and a local bus that is able to interact with any other bus architecture. Examples of the buses may include PCI, ISA, PCI-Express, HyperTransport™, InfiniBand™, Serial ATA, IC, and other suitable interconnects. The central processing unit(also referred to as a processor) can include a single or multiple sets of processors having single or multiple cores. The processormay execute one or more computer-executable code implementing the techniques of the present disclosure. The system memorymay be any memory for storing data used herein and/or computer programs that are executable by the processor. The system memorymay include volatile memory such as a random access memory (RAM)and non-volatile memory such as a read only memory (ROM), flash memory, etc., or any combination thereof. The basic input/output system (BIOS)may store the basic procedures for transfer of information between elements of the computer system, such as those at the time of loading the operating system with the use of the ROM.

20 27 28 27 28 23 32 20 22 27 28 20 The computer systemmay include one or more storage devices such as one or more removable storage devices, one or more non-removable storage devices, or a combination thereof. The one or more removable storage devicesand non-removable storage devicesare connected to the system busvia a storage interface. In an aspect, the storage devices and the corresponding computer-readable storage media are power-independent modules for the storage of computer instructions, data structures, program modules, and other data of the computer system. The system memory, removable storage devices, and non-removable storage devicesmay use a variety of computer-readable storage media. Examples of computer-readable storage media include machine memory such as cache, SRAM, DRAM, zero capacitor RAM, twin transistor RAM, eDRAM, EDO RAM, DDR RAM, EEPROM, NRAM, RRAM, SONOS, PRAM; flash memory or other memory technology such as in solid state drives (SSDs) or flash drives; magnetic cassettes, magnetic tape, and magnetic disk storage such as in hard disk drives or floppy disks; optical storage such as in compact disks (CD-ROM) or digital versatile disks (DVDs); and any other medium which may be used to store the desired data and which can be accessed by the computer system.

22 27 28 20 35 37 38 39 20 46 40 47 23 48 47 20 The system memory, removable storage devices, and non-removable storage devicesof the computer systemmay be used to store an operating system, additional program applications, other program modules, and program data. The computer systemmay include a peripheral interfacefor communicating data from input devices, such as a keyboard, mouse, stylus, game controller, voice input device, touch input device, or other peripheral devices, such as a printer or scanner via one or more I/O ports, such as a serial port, a parallel port, a universal serial bus (USB), or other peripheral interface. A display devicesuch as one or more monitors, projectors, or integrated display, may also be connected to the system busacross an output interface, such as a video adapter. In addition to the display devices, the computer systemmay be equipped with other peripheral output devices (not shown), such as loudspeakers and other audiovisual devices.

20 49 49 20 20 51 49 50 51 The computer systemmay operate in a network environment, using a network connection to one or more remote computers. The remote computer (or computers)may be local computer workstations or servers comprising most or all of the aforementioned elements in describing the nature of a computer system. Other devices may also be present in the computer network, such as, but not limited to, routers, network stations, peer devices or other network nodes. The computer systemmay include one or more network interfacesor network adapters for communicating with the remote computersvia one or more networks such as a local-area computer network (LAN), a wide-area computer network (WAN), an intranet, and the Internet. Examples of the network interfacemay include an Ethernet interface, a Frame Relay interface, SONET interface, and wireless interfaces.

Aspects of the present disclosure may be a system, a method, and/or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

20 The computer readable storage medium can be a tangible device that can retain and store program code in the form of instructions or data structures that can be accessed by a processor of a computing device, such as the computing system. The computer readable storage medium may be an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. By way of example, such computer-readable storage medium can comprise a random access memory (RAM), a read-only memory (ROM), EEPROM, a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), flash memory, a hard disk, a portable computer diskette, a memory stick, a floppy disk, or even a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon. As used herein, a computer readable storage medium is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or transmission media, or electrical signals transmitted through a wire.

Computer readable program instructions described herein can be downloaded to respective computing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network interface in each computing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing device.

Computer readable program instructions for carrying out operations of the present disclosure may be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language, and conventional procedural programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a LAN or WAN, or the connection may be made to an external computer (for example, through the Internet). In some aspects, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

6 FIG. In various aspects, the systems and methods described in the present disclosure can be addressed in terms of modules. The term “module” as used herein refers to a real-world device, component, or arrangement of components implemented using hardware, such as by an application specific integrated circuit (ASIC) or FPGA, for example, or as a combination of hardware and software, such as by a microprocessor system and a set of instructions to implement the module's functionality, which (while being executed) transform the microprocessor system into a special-purpose device. A module may also be implemented as a combination of the two, with certain functions facilitated by hardware alone, and other functions facilitated by a combination of hardware and software. In certain implementations, at least a portion, and in some cases, all, of a module may be executed on the processor of a computer system (such as the one described in greater detail inabove). Accordingly, each module may be realized in a variety of suitable configurations, and should not be limited to any particular implementation exemplified herein.

In the interest of clarity, not all of the routine features of the aspects are disclosed herein. It would be appreciated that in the development of any actual implementation of the present disclosure, numerous implementation-specific decisions must be made in order to achieve the developer's specific goals, and these specific goals will vary for different implementations and different developers. It is understood that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking of engineering for those of ordinary skill in the art, having the benefit of this disclosure.

Furthermore, it is to be understood that the phraseology or terminology used herein is for the purpose of description and not of restriction, such that the terminology or phraseology of the present specification is to be interpreted by the skilled in the art in light of the teachings and guidance presented herein, in combination with the knowledge of those skilled in the relevant art(s). Moreover, it is not intended for any term in the specification or claims to be ascribed an uncommon or special meaning unless explicitly set forth as such.

The various aspects disclosed herein encompass present and future known equivalents to the known modules referred to herein by way of illustration. Moreover, while aspects and applications have been shown and described, it would be apparent to those skilled in the art having the benefit of this disclosure that many more modifications than mentioned above are possible without departing from the inventive concepts disclosed herein.