Here’s the video of Dave helping me get it installed for anyone else who is interested:
Thanks
Dave
Awesome - happy you were able to get this installed!
Good idea on the python script @DJJones - that makes things easy
1 Like
Easy works for me, only big change I would make to the script beyond at this point would be to clear the model from ollama before adding the new one. This would allow for updating and such.
1 Like
Hi @mhowlett started playing with the model again and it’s definitely much more accurate this time. However I am having some weirdness with the tagging especially on the question below where I ask about BrainDrive studio. Do you think that may be something with my setup or something we need to look at with the model itself? Screenshots below thanks.
Try adding /no_think to the end of the prompt, see if it handles the tags differently. Example: “What is BrainDrive? /no_think”
This is just a test and if it does work, odds are it would work in the system prompt as well so can hide it from the user.
Almost seems abnormally aggressive with the tags as well, may want to grab the initial model that was used to fine-tune from and ask it the same questions. I know it won’t answer correctly but I wonder if the fine-tune perhaps created a tag happy LLM.
The initial model does not add as many tags - this issue is on my to-do list.
I haven’t been able to play with it yet, wrapping up 0.2.0 first. an idea might be to zero shot the system prompt, providing an example question and how to respond. I would intentionally add something odd like a link to BrainDrive community (community.braindrive.ai) at the bottom of each response as a test to determine how closely it is following instructions, this model should do that very well.
Hi Matt hope all is well. Thoughts on next steps here?
Thanks
Dave
Hi Dave,
Apologies for the delay — I’ve been tied up with another project. I should have some time to work on this tomorrow and early next week, and I’m aiming to share an update with you then.
Thank you,
Matt
ok thanks for the update
Any chance you could make the dataset public on hugging face as well, this has me curious on a few things.
Thank you,
David
Find it here: mhowl/braindrive-concierge-QA · Datasets at Hugging Face
Here’s a demo video showing the progress on removing unwanted tags from model responses. The issue stemmed from the backend API’s chat message formatting. I added a formatting function to clean the responses and implemented keyword-based link injection.
I’m facing challenges making the injected links clickable on the frontend. I tried using react-markdown, as well as manually wrapping links in HTML <a> tags, but neither worked. So, I did not include any frontend changes in the recent GitHub pull request.
Additionally, I added the .gguf model and .jsonl dataset to the .gitignore file due to their large size. You can download them from the Hugging Face links in this thread.
Let me know if you have any questions!
Matt moved this to a separate branch to make things cleaner for @DJJones . Putting the link here so we have it for the future and for anyone who is following the thread:
Feature/BrainDrive Concierge Development
Thanks
Dave