support lora
#1
by
jupyterjazz
- opened
jupyterjazz
changed pull request status to
open
Looks good! Some questions:
- Do we need another class to define in the automap of the config?
- What is this add_pooling_layer in the initialization?
- Does it make sense to change num_loras=1 by default or is 0 better?
- Maybe there are a few other things which might make sense to set in the config, e.g. the rank
jupyterjazz
changed pull request status to
merged