Unironically I think this is fixing the model even without layerskipping the more it trains.