Struct Mamba1NetworkConfig 

pub struct Mamba1NetworkConfig {
    pub n_layer: usize,
    pub vocab_size: usize,
    pub pad_vocab_size_multiple: usize,
    pub mamba_block: Mamba1Config,
    pub missing_lm_head: bool,
}

Fields

n_layer: usize

vocab_size: usize

If vocab_size is divisible by pad_vocab_size_multiple, this should be considered the unpadded vocab size. Otherwise, this is padded into ((vocab_size / self.pad_vocab_size_multiple) + 1) * pad_vocab_size_multiple.

pad_vocab_size_multiple: usize

If no pad is required, vocab_size must be divisible by pad_vocab_size_multiple. If pad is required, vocab_size increases until it’s divisible by pad_vocab_size_multiple.

To disable vocab padding, you can set this to 1.

mamba_block: Mamba1Config

missing_lm_head: bool

If set to true, lm_head is set to None and it re-utilizes the transposed embedding weights.

Implementations

impl Mamba1NetworkConfig

pub fn new( n_layer: usize, vocab_size: usize, pad_vocab_size_multiple: usize, mamba_block: Mamba1Config, missing_lm_head: bool, ) -> Self

Create a new instance of the config.

Arguments

Required Arguments

n_layer

vocab_size

If vocab_size is divisible by pad_vocab_size_multiple, this should be considered the unpadded vocab size. Otherwise, this is padded into ((vocab_size / self.pad_vocab_size_multiple) + 1) * pad_vocab_size_multiple.

pad_vocab_size_multiple

If no pad is required, vocab_size must be divisible by pad_vocab_size_multiple. If pad is required, vocab_size increases until it’s divisible by pad_vocab_size_multiple.

To disable vocab padding, you can set this to 1.

mamba_block

missing_lm_head

If set to true, lm_head is set to None and it re-utilizes the transposed embedding weights.

impl Mamba1NetworkConfig

pub fn init<B: Backend>(&self, device: &B::Device) -> Mamba1Network<B>

Returns the initialized model.

Trait Implementations

impl Clone for Mamba1NetworkConfig

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Config for Mamba1NetworkConfig

fn load_binary(data: &[u8]) -> Result<Self, ConfigError>

Loads the configuration from a binary buffer.

impl Debug for Mamba1NetworkConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for Mamba1NetworkConfig

fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>

where D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for Mamba1NetworkConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Serialize for Mamba1NetworkConfig

fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>

where S: Serializer,

Serialize this value into the given Serde serializer.