Selective Data Compression for In-Memory Databases

1. A computer-implemented method comprising: maintaining, by a database server, a database on persistent storage; wherein the database includes: a first data portion, that has been designated as eligible for mirroring within volatile memory; and a second data portion that has been also designated as eligible for mirroring within the volatile memory; storing first eligibility data that corresponds to the first data portion, wherein the first eligibility data reflects an estimated benefit of having the first data portion mirrored in the volatile memory; storing second eligibility data that corresponds to the second data portion, wherein the second eligibility data reflects an estimated benefit of having the second data portion mirrored in the volatile memory; wherein the first eligibility data and the second eligibility data are determined by the database server based on one or more factors; based on the first eligibility data, automatically determining that the first data portion should not be loaded into the volatile memory; based on the second eligibility data, automatically determining that the second data portion should be loaded into the volatile memory; and in response to a load event that causes data portions to be mirrored within the volatile memory, loading, into the volatile memory, data from the second data portion without loading into volatile memory any data from the first data portion.

2. The method of claim 1 , wherein the first data portion is a subset of the second data portion.

3. The method of claim 1 , wherein loading, into the volatile memory, the data from the second data portion comprises constructing an in-memory compression unit using the data from the second data portion.

4. The method of claim 3 , wherein, in the persistent storage, the data from the second data portion is organized in a row-major format, and, within the in-memory compression unit, the data from the second data portion is organized in a column-major format.

5. The method of claim 1 , wherein the one or more factors include at least one of: access statistics associated with a particular data portion for which particular eligibility data is being determined, a datatype of the particular data portion, operation statistics associated with the particular data portion, or data statistics associated with the particular data portion.

6. The method of claim 5 , wherein the access statistics associated with the particular data portion comprise heat map statistics of the particular data portion; and the method further comprising: evaluating the heat map statistics of the particular data portion, if the heat map statistics indicate frequent read accesses to the particular data portion, modifying the particular eligibility data to increase likelihood of the particular data portion to be loaded into the volatile memory; if the heat map statistics indicate infrequent read accesses to the particular data portion, modifying the particular eligibility data to decrease the likelihood of the particular data portion to be loaded into the volatile memory.

7. The method of claim 5 : wherein the access statistics associated with the particular data portion comprise heat map statistics of the particular data portion; and the method further comprising: evaluating the heat map statistics of the particular data portion, if the heat map statistics indicate frequent write accesses to the particular data portion, modifying the particular eligibility data to decrease likelihood of the particular data portion to be loaded into the volatile memory.

8. The method of claim 5 , further comprising: evaluating the datatype of the particular data portion, if the datatype is a large object type, modifying the particular eligibility data to decrease likelihood of the particular data portion to be loaded into the volatile memory.

9. The method of claim 1 , further comprising: evaluating for a certain data portion of the second data portion at least one of the following: access statistics of the certain data portion a data type of the certain data portion, performance statistics for one or more compressed data portions that are similar to the certain data portion; based on the evaluating, modifying a compression level for the certain data portion; compressing data in the certain data portion at the modified compression level; and loading the compressed data into the volatile memory.

10. The method of claim 9 , wherein the performance statistics, for the one or more compressed data portions that are similar to the certain data portion, are evaluated to indicate that a different compression level consumes less resources of the database server; and modifying the compression level for the certain data portion to approach the different compression level.

11. The method of claim 1 , wherein: wherein the database includes a third data portion, that has been designated as eligible for mirroring within the volatile memory; and the method further comprises: storing third eligibility data that corresponds to the third data portion, wherein the third eligibility data reflects an estimated benefit of having the third data portion mirrored in the volatile memory; while data from the third data portion is mirrored in the volatile memory and the data from the first data portion is not mirrored in the volatile memory, the database server performing the steps of: recalculating the first eligibility data; recalculating the third eligibility data; based on the first eligibility data, automatically determining that the data from the first data portion should be loaded into the volatile memory; performing a comparison, wherein the comparison is between the third eligibility data and either: the first eligibility data; or a threshold value; based on the comparison, the database server determining that the data from the third data portion is to be removed from the volatile memory; in response to a purge event that causes copies of mirrored data portions to be removed from the volatile memory, removing from the volatile memory, the data from the third data portion; in response to a load event that causes copies of data portions to be mirrored within the volatile memory, loading into the volatile memory the data from the first data portion.

12. One or more non-transitory storage media storing instructions, which when executed by one or more hardware processors, cause: maintaining, by a database server, a database on persistent storage; wherein the database includes: a first data portion, that has been designated as eligible for mirroring within volatile memory; and a second data portion that has been also designated as eligible for mirroring within the volatile memory; storing first eligibility data that corresponds to the first data portion, wherein the first eligibility data reflects an estimated benefit of having the first data portion mirrored in the volatile memory; storing second eligibility data that corresponds to the second data portion, wherein the second eligibility data reflects an estimated benefit of having the second data portion mirrored in the volatile memory; wherein the first eligibility data and the second eligibility data are determined by the database server based on one or more factors; based on the first eligibility data, automatically determining that the first data portion should not be loaded into the volatile memory; based on the second eligibility data, automatically determining that the second data portion should be loaded into the volatile memory; and in response to a load event that causes data portions to be mirrored within the volatile memory, loading, into the volatile memory, data from the second data portion without loading into volatile memory any data from the first data portion.

13. The one or more non-transitory storage media of claim 12 , wherein the first data portion is a subset of the second data portion.

14. The one or more non-transitory storage media of claim 12 , wherein instructions for loading, into the volatile memory, the data from the second data portion include instructions, which when executed by said one or more hardware processors further cause constructing an in-memory compression unit using the data from the second data portion.

15. The one or more non-transitory storage media of claim 14 , wherein, in the persistent storage, the data from the second data portion is organized in a row-major format, and, within the in-memory compression unit, the data from the second data portion is organized in a column-major format.

16. The one or more non-transitory storage media of claim 12 , wherein the one or more factors include at least one of: access statistics associated with a particular data portion for which particular eligibility data is being determined, a datatype of the particular data portion, operation statistics associated with the particular data portion, or data statistics associated with the particular data portion.

17. The one or more non-transitory storage media of claim 16 , wherein the access statistics associated with the particular data portion comprise heat map statistics of the particular data portion; and wherein the instructions further include instructions, which when executed by said one or more hardware processors, further cause: evaluating the heat map statistics of the particular data portion, if the heat map statistics indicate frequent read accesses to the particular data portion, modifying the particular eligibility data to increase likelihood of the particular data portion to be loaded into the volatile memory; if the heat map statistics indicate infrequent read accesses to the particular data portion, modifying the particular eligibility data to decrease the likelihood of the particular data portion to be loaded into the volatile memory.

18. The one or more non-transitory storage media of claim 16 : wherein the access statistics associated with the particular data portion comprise heat map statistics of the particular data portion; and wherein the instructions further include instructions, which when executed by said one or more hardware processors, further cause: evaluating the heat map statistics of the particular data portion, if the heat map statistics indicate frequent write accesses to the particular data portion, modifying the particular eligibility data to decrease likelihood of the particular data portion to be loaded into the volatile memory.

19. The one or more non-transitory storage media of claim 16 , wherein the instructions further include instructions for causing: evaluating the datatype of the particular data portion, if the datatype is a large object type, modifying the particular eligibility data to decrease likelihood of the particular data portion to be loaded into the volatile memory.

20. The one or more non-transitory storage media of claim 12 , wherein the instructions further include instructions, which when executed by said one or more hardware processors, further cause: evaluating for a certain data portion of the second data portion at least one of the following: access statistics of the certain data portion a data type of the certain data portion, performance statistics for one or more compressed data portions that are similar to the certain data portion; based on the evaluating, modifying a compression level for the certain data portion; compressing data in the certain data portion at the modified compression level; and loading the compressed data into the volatile memory.

21. The one or more non-transitory storage media of claim 20 , wherein the performance statistics, for the one or more compressed data portions that are similar to the certain data portion, are evaluated to indicate that a different compression level consumes less resources of the database server; and modifying the compression level for the certain data portion to approach the different compression level.

22. The one or more non-transitory storage media of claim 12 , wherein: wherein the database includes a third data portion, that has been designated as eligible for mirroring within the volatile memory; and wherein the instructions further include instructions, which when executed by said one or more hardware processors, further cause: storing third eligibility data that corresponds to the third data portion, wherein the third eligibility data reflects an estimated benefit of having the third data portion mirrored in the volatile memory; while data from the third data portion is mirrored in the volatile memory and the data from the first data portion is not mirrored in the volatile memory, the database server performing the steps of: recalculating the first eligibility data; recalculating the third eligibility data; based on the first eligibility data, automatically determining that the data from the first data portion should be loaded into the volatile memory; performing a comparison, wherein the comparison is between the third eligibility data and either: the first eligibility data; or a threshold value; based on the comparison, the database server determining that the data from the third data portion is to be removed from the volatile memory; in response to a purge event that causes copies of mirrored data portions to be removed from the volatile memory, removing from the volatile memory, the data from the third data portion; in response to a load event that causes copies of data portions to be mirrored within the volatile memory, loading into the volatile memory the data from the first data portion.