Model output in human-understandable form
Hello bloopAI Team,
I have tested your model on a dataset containing a range of Cobol code snippets, specifically focusing on the model's ability to comprehend Cobol syntax and turn it into a human-interpretable form. I am asking simple questions such as how a function works, or where in a large piece of code a particular capability is executed. Using CodeLlama-34B I get the expected output but mAInframer seems to be able to respond in COBOL language only.
Is there a way of getting a human understandable output perhaps by changing the prompt?
Best Regards,
Sebastian
Hey Sebastian,
These models have only been trained to do COBOL code completion, so are unlikely to do a good job at responding to arbitrary questions (e.g. "What does this paragraph do?"). We are working on instruction-tuning, which should improve performance on the tasks you're interested in.
Cheers,
Gabriel
Many thanks for the quick response. Looking forward to the instruction fine tuned model!
Cheers,
Sebastian
Hi Gabriel,
great project, I'm in Sebastian's shoes with a project to review legacy Cobol code rather than generate more code. Looking forward to your next fine tuned model. From your work to date, are other models like Code-LLama or GPT4 able to do a reasonable job at explaining Cobol even though they can't generate it so well?
Cheers
David
GPT-4 does a pretty good job of explaining COBOL - in my experience it's quite a lot better than CodeLlama - so I'd give that a go!
Gabriel
Cool thanks, still keen for your teams next updates as well : )