This is LoRA checkpoint fine-tuned with the following CLI. The fine-tuning process is logged in W&B dashboard. I have used DGX workstation with 8 x A100(40G).
python finetune.py \
--base_model='elinas/llama-13b-hf-transformers-4.29' \
--data_path='alpaca_data.json' \
--num_epochs=10 \
--cutoff_len=1024 \
--group_by_length \
--output_dir='./lora-alpaca-13b-elinas' \
--lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
--lora_r=16 \
--lora_alpha=32 \
--batch_size=1024 \
--micro_batch_size=28
This LoRA checkpoint is recommended to be used with transformers >= 4.29 which should be installed with the following command currently(4/30/2023).
pip install git+https://github.com/huggingface/transformers.git
Please find out the example usage and results in the accompanied notebook. For the example outputs are listed below as well.
1. Instruction: Compose an engaging travel blog post about a recent trip to Hawaii, highlighting cultural experiences and must-see attractions.
The beauty of the islands in Hawaii is truly captivating. From stunning volcanoes like Kilauea, waterfalls tumbling into clear pools like Rainbow Falls, breathtaking coastal vistas like Hanauma Bay Beach Park, and lush tropical rainforests brimming with wildlife such as monk seals and green sea turtles - there's so much more than sandy beaches and palm trees!
I recently had the opportunity to explore several different sides of this unique destination during my visit last summer. I went snorkeling off the Big Island’s eastern shoreline near Punaluʻu Black Sand beach where it was easy to spot dozens of colorful fish amongst black lava rock formations. On Maui, we explored Haleakala National park for hiking opportunities amidst bamboo groves, bright blue skies and flourishing mountain scenery. Finally on Oahu, my group enjoyed surfing at Waimea Bay - one of the most popular surf spots along North Shore.
2. Instruction: Can you help me write a formal email to a potential business partner proposing a joint venture?
Dear [Name],
I am writing today in hopes of discussing the possibility for collaboration with your company. I believe our services can compliment each other and offer mutual benefits that would enable us both to grow as successful entities within this marketplace. We have some unique ideas which we feel could create synergy between companies, allowing us to expand our presence in existing markets while providing new opportunities into untapped territories. As such, we are interested in exploring the opportunity to work together on an equal partnership basis. If it is convenient for you, please let us know if there may be time to meet or talk further about these prospects over the phone. Looking forward to hearing from you soon!
3. Instruction: What factors would you consider when designing an inclusive and accessible public transportation system?
When designing a comprehensive, inclusive, and accessible transit network, there are several key aspects to take into consideration. Firstly, the size of a city is critical in determining what type of transit will be most suitable for its inhabitants; larger cities may benefit from subway or tram systems while smaller ones can use buses, shuttles, light rail, streetcars, ferries, bicycles, scooters, etc. Secondly, it’s important to look at accessibility – whether people with disabilities have equal access to the services provided by such networks must also be considered. Thirdly, the costs involved need to be assessed so that users feel they are receiving value for their money. Additionally, safety standards should be high enough to ensure that passengers do not face any threats onboard trains and other modes of mass transit. Finally, environmental sustainability has become increasingly crucial as concerns about climate change grow louder, thus eco-friendliness needs to play a major role in the development process of a robust public transportation system.
4. Instruction: What are some potential implications of using a single-use plastic bottle versus a reusable bottle on both the environment and human health?
Using single use plastics has many negative impacts. Plastic is not easily biodegradable, so it stays in landfills for decades and can cause environmental contamination when exposed to sunlight or other elements. It also takes hundreds of years before plastic decomposes into organic matter, which means that any resources used to produce these plastic items have been wasted over time due to their lack of recyclability. In addition, single use plastics create pollution through leaching chemicals such as bisphenol A (BPA) into waterways and oceans where animals ingest them with disastrous effects on their own ecosystem. On top of this, burning fossil fuels produces greenhouse gases responsible for global warming, making plastic production an even more unsustainable practice.
By contrast, the use of reusable materials such as glass and metal reduces waste from disposables while promoting sustainability by extending product life cycles. Reusables typically require fewer energy inputs during manufacturing and production than do most forms of plastic because they often last longer. These products can be collected and transported safely without causing significant harm to our ecological system either