Enum Normalization

pub enum Normalization {
    Level0,
    Level1,
    Level2,
    Level3,
}

The level for the normalized chunking used by FastCDC and StreamCDC.

Normalized chunking “generates chunks whose sizes are normalized to a specified region centered at the expected chunk size,” as described in section 4.4 of the FastCDC 2016 paper.

Note that lower levels of normalization will result in a larger range of generated chunk sizes. It may be beneficial to widen the minimum/maximum chunk size values given to the FastCDC constructor in that case.

Note that higher levels of normalization may result in the final chunk of data being smaller than the minimum chunk size, which results in a hash value of zero (0) since no calculations are performed for sub-minimum chunks.

Variants

Level0

No chunk size normalization, produces a wide range of chunk sizes.

Level1

Level 1 normalization, in which fewer chunks are outside of the desired range.

Level2

Level 2 normalization, where most chunks are of the desired size.

Level3

Level 3 normalization, nearly all chunks are the desired size.