Struct MultiGateResidualConfig 

pub struct MultiGateResidualConfig {
    pub d_model: usize,
    pub n_stream: usize,
    pub init_bias: f64,
}

Configuration for a single MultiGateResidual.

Fields

d_model: usize

Model width d.

n_stream: usize

Number of parallel residual streams n.

init_bias: f64

Initial value for every entry of the gate bias b⁽ᵝ⁾ (see module header).

Implementations

impl MultiGateResidualConfig

pub fn new(d_model: usize, n_stream: usize) -> Self

Create a new instance of the config.

Arguments

Required Arguments

d_model

Model width d.

n_stream

Number of parallel residual streams n.

Default Arguments

init_bias

Initial value for every entry of the gate bias b⁽ᵝ⁾ (see module header).

• Defaults to 0.0

impl MultiGateResidualConfig

pub fn with_init_bias(self, init_bias: f64) -> Self

Sets the value for the field init_bias.

Initial value for every entry of the gate bias b⁽ᵝ⁾ (see module header).

• Defaults to 0.0

impl MultiGateResidualConfig

pub fn init(&self, device: &Device) -> MultiGateResidual

Allocate one layer’s MGR parameters (w⁽ᵝ⁾, w⁽ᵅ⁾ zero; b⁽ᵝ⁾ constant).

Trait Implementations

impl Clone for MultiGateResidualConfig

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Config for MultiGateResidualConfig

fn load_binary(data: &[u8]) -> Result<Self, ConfigError>

Loads the configuration from a binary buffer.

impl Debug for MultiGateResidualConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for MultiGateResidualConfig

fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>

where D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for MultiGateResidualConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Serialize for MultiGateResidualConfig

fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>

where S: Serializer,

Serialize this value into the given Serde serializer.