google/gemma-7b · Training Gemma on Domain-Specific Data by Unsupervised Learning t for Specific Instructional Tasks

I aim to adapt Gemma for a domain-specific task through unsupervised learning, initially training it with my text corpus as both input and output. This step is intended to prepare Gemma for subsequent, more focused training on tasks relevant to my specialty. My primary apprehension is that this deviates from the conventional prompt-based training paradigm, which necessitates an 'instruction, input, and output' format. How should I proceed with unsupervised pre-training？Can such an approach effectively prime Gemma for improved performance on domain-specific questions？