Package org.rocksdb

Class BlockBasedTableConfig

java.lang.Object
org.rocksdb.TableFormatConfig
org.rocksdb.BlockBasedTableConfig
public class BlockBasedTableConfig extends TableFormatConfig
The config for block based table sst format.

BlockBasedTable is a RocksDB's default SST file format.

  • Constructor Details

    • BlockBasedTableConfig

      public BlockBasedTableConfig()

  • Method Details

    • cacheIndexAndFilterBlocks

      public boolean cacheIndexAndFilterBlocks()
      Indicating if we'd put index/filter blocks to the block cache. If not specified, each "table reader" object will pre-load index/filter block during table initialization.
      Returns:
      if index and filter blocks should be put in block cache.

    • setCacheIndexAndFilterBlocks

      public BlockBasedTableConfig setCacheIndexAndFilterBlocks(boolean cacheIndexAndFilterBlocks)
      Indicating if we'd put index/filter blocks to the block cache. If not specified, each "table reader" object will pre-load index/filter block during table initialization.
      Parameters:
      cacheIndexAndFilterBlocks - and filter blocks should be put in block cache.
      Returns:
      the reference to the current config.

    • cacheIndexAndFilterBlocksWithHighPriority

      public boolean cacheIndexAndFilterBlocksWithHighPriority()
      Indicates if index and filter blocks will be treated as high-priority in the block cache. See note below about applicability. If not specified, defaults to true.
      Returns:
      if index and filter blocks will be treated as high-priority.

    • setCacheIndexAndFilterBlocksWithHighPriority

      public BlockBasedTableConfig setCacheIndexAndFilterBlocksWithHighPriority(boolean cacheIndexAndFilterBlocksWithHighPriority)
      If true, cache index and filter blocks with high priority. If set to true, depending on implementation of block cache, index and filter blocks may be less likely to be evicted than data blocks.
      Parameters:
      cacheIndexAndFilterBlocksWithHighPriority - if index and filter blocks will be treated as high-priority.
      Returns:
      the reference to the current config.

    • pinL0FilterAndIndexBlocksInCache

      public boolean pinL0FilterAndIndexBlocksInCache()
      Indicating if we'd like to pin L0 index/filter blocks to the block cache. If not specified, defaults to false.
      Returns:
      if L0 index and filter blocks should be pinned to the block cache.

    • setPinL0FilterAndIndexBlocksInCache

      public BlockBasedTableConfig setPinL0FilterAndIndexBlocksInCache(boolean pinL0FilterAndIndexBlocksInCache)
      Indicating if we'd like to pin L0 index/filter blocks to the block cache. If not specified, defaults to false.
      Parameters:
      pinL0FilterAndIndexBlocksInCache - pin blocks in block cache
      Returns:
      the reference to the current config.

    • pinTopLevelIndexAndFilter

      public boolean pinTopLevelIndexAndFilter()
      Indicates if top-level index and filter blocks should be pinned.
      Returns:
      if top-level index and filter blocks should be pinned.

    • setPinTopLevelIndexAndFilter

      public BlockBasedTableConfig setPinTopLevelIndexAndFilter(boolean pinTopLevelIndexAndFilter)
      If cacheIndexAndFilterBlocks is true and the below is true, then the top-level index of partitioned filter and index blocks are stored in the cache, but a reference is held in the "table reader" object so the blocks are pinned and only evicted from cache when the table reader is freed. This is not limited to l0 in LSM tree.
      Parameters:
      pinTopLevelIndexAndFilter - if top-level index and filter blocks should be pinned.
      Returns:
      the reference to the current config.

    • indexType

      public IndexType indexType()
      Get the index type.
      Returns:
      the currently set index type

    • setIndexType

      public BlockBasedTableConfig setIndexType(IndexType indexType)
      Sets the index type to used with this table.
      Parameters:
      indexType - IndexType value
      Returns:
      the reference to the current option.

    • dataBlockIndexType

      public DataBlockIndexType dataBlockIndexType()
      Get the data block index type.
      Returns:
      the currently set data block index type

    • setDataBlockIndexType

      public BlockBasedTableConfig setDataBlockIndexType(DataBlockIndexType dataBlockIndexType)
      Sets the data block index type to used with this table.
      Parameters:
      dataBlockIndexType - DataBlockIndexType value
      Returns:
      the reference to the current option.

    • dataBlockHashTableUtilRatio

      public double dataBlockHashTableUtilRatio()
      Get the #entries/#buckets. It is valid only when dataBlockIndexType() is DataBlockIndexType.kDataBlockBinaryAndHash.
      Returns:
      the #entries/#buckets.

    • setDataBlockHashTableUtilRatio

      public BlockBasedTableConfig setDataBlockHashTableUtilRatio(double dataBlockHashTableUtilRatio)
      Set the #entries/#buckets. It is valid only when dataBlockIndexType() is DataBlockIndexType.kDataBlockBinaryAndHash.
      Parameters:
      dataBlockHashTableUtilRatio - #entries/#buckets
      Returns:
      the reference to the current option.

    • checksumType

      public ChecksumType checksumType()
      Get the checksum type to be used with this table.
      Returns:
      the currently set checksum type

    • setChecksumType

      public BlockBasedTableConfig setChecksumType(ChecksumType checksumType)
      Sets
      Parameters:
      checksumType - ChecksumType value.
      Returns:
      the reference to the current option.

    • noBlockCache

      public boolean noBlockCache()
      Determine if the block cache is disabled.
      Returns:
      if block cache is disabled

    • setNoBlockCache

      public BlockBasedTableConfig setNoBlockCache(boolean noBlockCache)
      Disable block cache. If this is set to true, then no block cache should be used, and the setBlockCache(Cache) should point to a null object.

      Default: false

      Parameters:
      noBlockCache - if use block cache
      Returns:
      the reference to the current config.

    • setBlockCache

      public BlockBasedTableConfig setBlockCache(Cache blockCache)
      Use the specified cache for blocks. When not null this take precedence even if the user sets a block cache size.

      Cache should not be disposed before options instances using this cache is disposed.

      Cache instance can be re-used in multiple options instances.

      Parameters:
      blockCache - Cache Cache java instance (e.g. LRUCache).
      Returns:
      the reference to the current config.

    • setPersistentCache

      public BlockBasedTableConfig setPersistentCache(PersistentCache persistentCache)
      Use the specified persistent cache.

      If !null use the specified cache for pages read from device, otherwise no page cache is used.

      Parameters:
      persistentCache - the persistent cache
      Returns:
      the reference to the current config.

    • blockSize

      public long blockSize()
      Get the approximate size of user data packed per block.
      Returns:
      block size in bytes

    • setBlockSize

      public BlockBasedTableConfig setBlockSize(long blockSize)
      Approximate size of user data packed per block. Note that the block size specified here corresponds to uncompressed data. The actual size of the unit read from disk may be smaller if compression is enabled. This parameter can be changed dynamically. Default: 4K
      Parameters:
      blockSize - block size in bytes
      Returns:
      the reference to the current config.

    • blockSizeDeviation

      public int blockSizeDeviation()
      Returns:
      the hash table ratio.

    • setBlockSizeDeviation

      public BlockBasedTableConfig setBlockSizeDeviation(int blockSizeDeviation)
      This is used to close a block before it reaches the configured blockSize(). If the percentage of free space in the current block is less than this specified number and adding a new record to the block will exceed the configured block size, then this block will be closed and the new record will be written to the next block.

      Default is 10.

      Parameters:
      blockSizeDeviation - the deviation to block size allowed
      Returns:
      the reference to the current config.

    • blockRestartInterval

      public int blockRestartInterval()
      Get the block restart interval.
      Returns:
      block restart interval

    • setBlockRestartInterval

      public BlockBasedTableConfig setBlockRestartInterval(int restartInterval)
      Set the block restart interval.
      Parameters:
      restartInterval - block restart interval.
      Returns:
      the reference to the current config.

    • indexBlockRestartInterval

      public int indexBlockRestartInterval()
      Get the index block restart interval.
      Returns:
      index block restart interval

    • setIndexBlockRestartInterval

      public BlockBasedTableConfig setIndexBlockRestartInterval(int restartInterval)
      Set the index block restart interval
      Parameters:
      restartInterval - index block restart interval.
      Returns:
      the reference to the current config.

    • metadataBlockSize

      public long metadataBlockSize()
      Get the block size for partitioned metadata.
      Returns:
      block size for partitioned metadata.

    • setMetadataBlockSize

      public BlockBasedTableConfig setMetadataBlockSize(long metadataBlockSize)
      Set block size for partitioned metadata.
      Parameters:
      metadataBlockSize - Partitioned metadata block size.
      Returns:
      the reference to the current config.

    • partitionFilters

      public boolean partitionFilters()
      Indicates if we're using partitioned filters.
      Returns:
      if we're using partition filters.

    • setPartitionFilters

      public BlockBasedTableConfig setPartitionFilters(boolean partitionFilters)
      Use partitioned full filters for each SST file. This option is incompatible with block-based filters.

      Defaults to false.

      Parameters:
      partitionFilters - use partition filters.
      Returns:
      the reference to the current config.

    • optimizeFiltersForMemory

      @Experimental("Option to generate Bloom filters that minimize memory internal fragmentation") public boolean optimizeFiltersForMemory()
      Option to generate Bloom filters that minimize memory internal fragmentation.

      See setOptimizeFiltersForMemory(boolean).

      Returns:
      true if bloom filters are used to minimize memory internal fragmentation

    • setOptimizeFiltersForMemory

      @Experimental("Option to generate Bloom filters that minimize memory internal fragmentation") public BlockBasedTableConfig setOptimizeFiltersForMemory(boolean optimizeFiltersForMemory)
      Option to generate Bloom filters that minimize memory internal fragmentation.

      When false, malloc_usable_size is not available, or format_version < 5, filters are generated without regard to internal fragmentation when loaded into memory (historical behavior). When true (and malloc_usable_size is available and formatVersion() >= 5), then Bloom filters are generated to "round up" and "round down" their sizes to minimize internal fragmentation when loaded into memory, assuming the reading DB has the same memory allocation characteristics as the generating DB. This option does not break forward or backward compatibility.

      While individual filters will vary in bits/key and false positive rate when setting is true, the implementation attempts to maintain a weighted average FP rate for filters consistent with this option set to false.

      With Jemalloc for example, this setting is expected to save about 10% of the memory footprint and block cache charge of filters, while increasing disk usage of filters by about 1-2% due to encoding efficiency losses with variance in bits/key.

      NOTE: Because some memory counted by block cache might be unmapped pages within internal fragmentation, this option can increase observed RSS memory usage. With cacheIndexAndFilterBlocks() == true, this option makes the block cache better at using space it is allowed.

      NOTE: Do not set to true if you do not trust malloc_usable_size. With this option, RocksDB might access an allocated memory object beyond its original size if malloc_usable_size says it is safe to do so. While this can be considered bad practice, it should not produce undefined behavior unless malloc_usable_size is buggy or broken.

      Parameters:
      optimizeFiltersForMemory - true to enable Bloom filters that minimize memory internal fragmentation, or false to disable.
      Returns:
      the reference to the current config.

    • useDeltaEncoding

      public boolean useDeltaEncoding()
      Determine if delta encoding is being used to compress block keys.
      Returns:
      true if delta encoding is enabled, false otherwise.

    • setUseDeltaEncoding

      public BlockBasedTableConfig setUseDeltaEncoding(boolean useDeltaEncoding)
      Use delta encoding to compress keys in blocks.

      NOTE: ReadOptions.pinData() requires this option to be disabled.

      Default: true

      Parameters:
      useDeltaEncoding - true to enable delta encoding
      Returns:
      the reference to the current config.

    • filterPolicy

      public Filter filterPolicy()
      Get the filter policy.
      Returns:
      the current filter policy.

    • setFilterPolicy

      public BlockBasedTableConfig setFilterPolicy(Filter filterPolicy)
      Use the specified filter policy to reduce disk reads.

      Filter should not be closed before options instances using this filter are closed.

      Filter instance can be re-used in multiple options instances.

      Parameters:
      filterPolicy - Filter Filter Policy java instance.
      Returns:
      the reference to the current config.

    • setFilter

      @Deprecated public BlockBasedTableConfig setFilter(Filter filter)
      Deprecated.
      Use setFilterPolicy(Filter)
      Set the filter.
      Parameters:
      filter - the filter
      Returns:
      the reference to the current config.

    • wholeKeyFiltering

      public boolean wholeKeyFiltering()
      Determine if whole keys as opposed to prefixes are placed in the filter.
      Returns:
      if whole key filtering is enabled

    • setWholeKeyFiltering

      public BlockBasedTableConfig setWholeKeyFiltering(boolean wholeKeyFiltering)
      If true, place whole keys in the filter (not just prefixes). This must generally be true for gets to be efficient. Default: true
      Parameters:
      wholeKeyFiltering - if enable whole key filtering
      Returns:
      the reference to the current config.

    • verifyCompression

      public boolean verifyCompression()
      Returns true when compression verification is enabled.

      See setVerifyCompression(boolean).

      Returns:
      true if compression verification is enabled.

    • setVerifyCompression

      public BlockBasedTableConfig setVerifyCompression(boolean verifyCompression)
      Verify that decompressing the compressed block gives back the input. This is a verification mode that we use to detect bugs in compression algorithms.
      Parameters:
      verifyCompression - true to enable compression verification.
      Returns:
      the reference to the current config.

    • readAmpBytesPerBit

      public int readAmpBytesPerBit()
      Get the Read amplification bytes per-bit.

      See setReadAmpBytesPerBit(int).

      Returns:
      the bytes per-bit.

    • setReadAmpBytesPerBit

      public BlockBasedTableConfig setReadAmpBytesPerBit(int readAmpBytesPerBit)
      Set the Read amplification bytes per-bit.

      If used, For every data block we load into memory, we will create a bitmap of size ((block_size / `read_amp_bytes_per_bit`) / 8) bytes. This bitmap will be used to figure out the percentage we actually read of the blocks.

      When this feature is used Tickers::READ_AMP_ESTIMATE_USEFUL_BYTES and Tickers::READ_AMP_TOTAL_READ_BYTES can be used to calculate the read amplification using this formula (READ_AMP_TOTAL_READ_BYTES / READ_AMP_ESTIMATE_USEFUL_BYTES)

      value => memory usage (percentage of loaded blocks memory) 1 => 12.50 % 2 => 06.25 % 4 => 03.12 % 8 => 01.56 % 16 => 00.78 %

      Note: This number must be a power of 2, if not it will be sanitized to be the next lowest power of 2, for example a value of 7 will be treated as 4, a value of 19 will be treated as 16.

      Default: 0 (disabled)

      Parameters:
      readAmpBytesPerBit - the bytes per-bit
      Returns:
      the reference to the current config.

    • formatVersion

      public int formatVersion()
      Get the format version. See setFormatVersion(int).
      Returns:
      the currently configured format version.

    • setFormatVersion

      public BlockBasedTableConfig setFormatVersion(int formatVersion)

      We currently have five versions:

      • 0 - This version is currently written out by all RocksDB's versions by default. Can be read by really old RocksDB's. Doesn't support changing checksum (default is CRC32).
      • 1 - Can be read by RocksDB's versions since 3.0. Supports non-default checksum, like xxHash. It is written by RocksDB when BlockBasedTableOptions::checksum is something other than kCRC32c. (version 0 is silently upconverted)
      • 2 - Can be read by RocksDB's versions since 3.10. Changes the way we encode compressed blocks with LZ4, BZip2 and Zlib compression. If you don't plan to run RocksDB before version 3.10, you should probably use this.
      • 3 - Can be read by RocksDB's versions since 5.15. Changes the way we encode the keys in index blocks. If you don't plan to run RocksDB before version 5.15, you should probably use this. This option only affects newly written tables. When reading existing tables, the information about version is read from the footer.
      • 4 - Can be read by RocksDB's versions since 5.16. Changes the way we encode the values in index blocks. If you don't plan to run RocksDB before version 5.16 and you are using index_block_restart_interval > 1, you should probably use this as it would reduce the index size. This option only affects newly written tables. When reading existing tables, the information about version is read from the footer.
      • 5 - Can be read by RocksDB's versions since 6.6.0. Full and partitioned filters use a generally faster and more accurate Bloom filter implementation, with a different schema.
      Parameters:
      formatVersion - integer representing the version to be used.
      Returns:
      the reference to the current option.

    • enableIndexCompression

      public boolean enableIndexCompression()
      Determine if index compression is enabled.

      See setEnableIndexCompression(boolean).

      Returns:
      true if index compression is enabled, false otherwise

    • setEnableIndexCompression

      public BlockBasedTableConfig setEnableIndexCompression(boolean enableIndexCompression)
      Store index blocks on disk in compressed format.

      Changing this option to false will avoid the overhead of decompression if index blocks are evicted and read back.

      Parameters:
      enableIndexCompression - true to enable index compression, false to disable
      Returns:
      the reference to the current option.

    • blockAlign

      public boolean blockAlign()
      Determines whether data blocks are aligned on the lesser of page size and block size.
      Returns:
      true if data blocks are aligned on the lesser of page size and block size.

    • setBlockAlign

      public BlockBasedTableConfig setBlockAlign(boolean blockAlign)
      Set whether data blocks should be aligned on the lesser of page size and block size.
      Parameters:
      blockAlign - true to align data blocks on the lesser of page size and block size.
      Returns:
      the reference to the current option.

    • indexShortening

      public IndexShorteningMode indexShortening()
      Get the index shortening mode.
      Returns:
      the index shortening mode.

    • setIndexShortening

      public BlockBasedTableConfig setIndexShortening(IndexShorteningMode indexShortening)
      Set the index shortening mode. See IndexShorteningMode.
      Parameters:
      indexShortening - the index shortening mode.
      Returns:
      the reference to the current option.

    • blockCacheSize

      @Deprecated public long blockCacheSize()
      Deprecated.
      Get the size of the cache in bytes that will be used by RocksDB.
      Returns:
      block cache size in bytes

    • setBlockCacheSize

      @Deprecated public BlockBasedTableConfig setBlockCacheSize(long blockCacheSize)
      Deprecated.
      Use setBlockCache(Cache).
      Set the size of the cache in bytes that will be used by RocksDB. If cacheSize is negative, then cache will not be used. DEFAULT: 8M
      Parameters:
      blockCacheSize - block cache size in bytes
      Returns:
      the reference to the current config.

    • cacheNumShardBits

      @Deprecated public int cacheNumShardBits()
      Deprecated.
      Returns the number of shard bits used in the block cache. The resulting number of shards would be 2 ^ (returned value). Any negative number means use default settings.
      Returns:
      the number of shard bits used in the block cache.

    • setCacheNumShardBits

      @Deprecated public BlockBasedTableConfig setCacheNumShardBits(int blockCacheNumShardBits)
      Deprecated.
      Use setBlockCache(Cache).
      Controls the number of shards for the block cache. This is applied only if cacheSize is set to non-negative.
      Parameters:
      blockCacheNumShardBits - the number of shard bits. The resulting number of shards would be 2 ^ numShardBits. Any negative number means use default settings."
      Returns:
      the reference to the current option.

    • hashIndexAllowCollision

      @Deprecated public boolean hashIndexAllowCollision()
      Deprecated.
      This option is now deprecated. No matter what value it is set to, it will behave as if hashIndexAllowCollision() == true.
      Influence the behavior when kHashSearch is used. if false, stores a precise prefix to block range mapping if true, does not store prefix and allows prefix hash collision (less memory consumption)
      Returns:
      if hash collisions should be allowed.

    • setHashIndexAllowCollision

      @Deprecated public BlockBasedTableConfig setHashIndexAllowCollision(boolean hashIndexAllowCollision)
      Deprecated.
      This option is now deprecated. No matter what value it is set to, it will behave as if hashIndexAllowCollision() == true.
      Influence the behavior when kHashSearch is used. if false, stores a precise prefix to block range mapping if true, does not store prefix and allows prefix hash collision (less memory consumption)
      Parameters:
      hashIndexAllowCollision - points out if hash collisions should be allowed.
      Returns:
      the reference to the current config.

    • newTableFactoryHandle

      protected long newTableFactoryHandle()
      Description copied from class: TableFormatConfig

      This function should only be called by Options.setTableFormatConfig(), which will create a c++ shared-pointer to the c++ TableFactory that associated with the Java TableFormatConfig.

      Specified by:
      newTableFactoryHandle in class TableFormatConfig
      Returns:
      native handle address to native table instance.