theonlyengine
/

flash-attention

Model card Files Files and versions Community

flash-attention / adamw-apex-distributed.yaml

theonlyengine's picture

Upload 421 files

3f9c425 verified 6 months ago

history blame contribute delete

123 Bytes

	# @package train.optimizer
	_target_: apex.contrib.optimizers.distributed_fused_adam.DistributedFusedAdam
	adam_w_mode: True