Updated the status message on the endpoint start

Fixed bug where display value was preventing end point from starting

Added logging to endpoint start

Migrated from using print statements in the application code to using logger module (left prints in files intended to be run as scripts)

Fixed bug in status bar

Added system status to the default on each page with option for users to request start of the LLM inference endpoints. Set the endpoints to 15 minute timeout in line with this.

Updated one type hint

Reviewed comments and type hints

Reviewed comments and type hints

Reviewed comments and type hints

Added on type hint

Reviewed for comments and type hints

Checked type hints

Tidied up generate_data.py

Made updates to support automatic reload of the TestGroups after a test run

Updated from using random.choices to random.sample throughout where I need a random distinct set as choices does replacement so you can get the same item twice. Discovered in pricing testing.

Marked the logger as a daemon thread so it doesn't prevent the exit of the python interpreter

Fixed bug where the Logger was logging its own name and not that of the architecture.

Implemented single threaded worker on writing the logs to the json file for controlled access to the resource on the file system now we are multi-threading the tests.

Modified test runner to dispatch requests in parallel to make use of the fact that there is a lot of wait time for the LLM. Defaulting to 16 threads.

Added runner for pricing fact checks to assess the level of fact embedding in the latest model

Saved file records to DB. Fixed a print to show the correct test-group name.

Updated the testing page to show the request/response pairs

Updated the offline save to save the actual request and response text

Fixed display of architecture name

Logged start of architecture invocation so if one stalls you can see what it is in the logs

Update to the new URL for model v5

Trying fine-tuning yet another way - all run, now testing v3 of the model

Added raw prompt format to just do passthrough so I can test a number of different examples just typed in

Added another test prompt format

Tweaked the test generator and updated the tests

Added the test question generator and increased the size of the question bank to 500

Added a missing comment

Fixed a bug where if the architecture had entirely failed and not generated a response the whole load of TestGroups would crash. Need to fix the root cause of the failure to generate a response, but also should be caught gracefully here in any event.

Refactored loading the TestGroups to make the structure of the json load and the DB load the same and clearer

Added comments throughout

Switched endpoint control to use the writeable token as it was inconsistent with the normal token.

Fixed bug with default prompt style not being valid

Configured more architectures to try and debug the fine-tuning issue each with different prompt styles

Trying a different prompting style

Tweaked the training data format to try and fix the issue of the model repeating the question over and over

Added loading of test groups from both the DB and the local file and merging these two

Adding the sqlite db where I will archive the test results and added the archiving code

Added option to directly pass the HF hub token when wiping the trace file, so I can use it locally outside of streamlit. Defaulted to None avoid changing existing behaviour.

Fixed bugs from the refactor of repo access. Now it should save trace again.

Added the test reporting structure

Refactored to bring common variables together. Also added a utility to get all the trace records as a list of records

Fixed the time.time bug here. Also a call to reset the Chroma DB

Tweaked the way the prompt is formatted going into the LLM query, to avoid the fine-tuned model giving nonsense answers

Flipped the default dataset to be the baseline not the "All products"