Marked the logger as a daemon thread so it doesn't prevent the exit of the python interpreter

Fixed bug where the Logger was logging its own name and not that of the architecture.

Added environment variable to explicitly flag to the tokenizers that we are doing multi-threading and to prevent a bunch of warnings arising

Added ability to set the number of testing threads dynamically from the UI

Implemented single threaded worker on writing the logs to the json file for controlled access to the resource on the file system now we are multi-threading the tests.

Modified test runner to dispatch requests in parallel to make use of the fact that there is a lot of wait time for the LLM. Defaulting to 16 threads.

Saved trace records offline

Saved trace records offline

Removed a debug print line

Added runner for pricing fact checks to assess the level of fact embedding in the latest model

Correct the numbering on the latest architecture

Added a setup for V7 of the fine-tuned model to test that

Saved file records to DB. Fixed a print to show the correct test-group name.

Tried a tweak to the prompt to lean in to facts

Added a config to test V6 fine-tuned model

Saved test records and refactored reporter UI code into smaller functions

Updated the testing page to show the request/response pairs

Updated the offline save to save the actual request and response text

Added the safety components to the fine-tuning model to make the test fair

Fixed display of architecture name

Logged start of architecture invocation so if one stalls you can see what it is in the logs

Tidying up old setups - leaving with the best working fine-tuned one for now

Update to the new URL for model v5

Update to the new URL for model v3

Trying fine-tuning yet another way - all run, now testing v3 of the model

Added raw prompt format to just do passthrough so I can test a number of different examples just typed in

Added another test prompt format

Added a push button to generate a random question to the UI, so users don't have to phrase something themselves.

Tweaked the test generator and updated the tests

Added the option to pause a failed endpoint in order to be able to kick it with a restart

Added the test question generator and increased the size of the question bank to 500

Added a missing comment

Fixed a bug where if the architecture had entirely failed and not generated a response the whole load of TestGroups would crash. Need to fix the root cause of the failure to generate a response, but also should be caught gracefully here in any event.

Refactored loading the TestGroups to make the structure of the json load and the DB load the same and clearer

Added comments throughout

Tweaked message

Switched endpoint control to use the writeable token as it was inconsistent with the normal token.

Added front page message about the endpoints being scheduled.

Fixed bug with default prompt style not being valid

Configured more architectures to try and debug the fine-tuning issue each with different prompt styles

Trying a different prompting style

Added possibility to change the prompt style via parameter as I try and debug the fine-tuning issue.

Password protected the system controls page

Tweaked the training data format to try and fix the issue of the model repeating the question over and over

Added ability to select which models to compare side by side, allowing for more flexibility in testing my fine-tuned llamas

Moved trace records to DB

Added a time/response length plot to the charts

Added loading of test groups from both the DB and the local file and merging these two

Added message to system status

Adding the sqlite db where I will archive the test results and added the archiving code