MinimalChat Is a Full-Featured and Self-Contained LLM Chat Application (github.com)
from Fingerthief@infosec.pub to selfhosted@lemmy.world on 14 Jun 21:53
https://infosec.pub/post/13676291

I’ve been building MinimalChat for a while now, and based on the feedback I’ve received, it’s in a pretty decent place for general use. I figured I’d share it here for anyone who might be interested!

Quick Features Overview:

The idea is to make it essentially foolproof to deploy or set up while being generally full-featured and aesthetically pleasing. No additional databases or servers are needed, everything is contained and managed inside the web app itself locally.

It’s another chat client in a sea of clients but it is unique in its own ways in my opinion. Enjoy! Feedback is always appreciated!

Self Hosting Wiki Section github.com/…/Self-Hosting-With-Docker

#selfhosted

threaded - newest

stonerboner@lemmynsfw.com on 14 Jun 22:40 next collapse

Sounds interesting! May I ask where the data used to train came from?

Fingerthief@infosec.pub on 14 Jun 22:58 collapse

This app is more of an interface to use while connecting to any number of LLM Models that have an API available. The application itself has no model.

For example you can choose to use GPT-4 Omni by providing an API key from OpenAI.

But you can also connect to services like OpenRouter with an API key and select between 20+ different models that they provide access to as seen below

<img alt="" src="https://infosec.pub/pictrs/image/307bf9cd-35eb-47b7-b39f-f54728e037c2.png">

It also supports connecting to fully local models via programs like LM Studio which downloads models from Hugging Face to your machine and will spin up a local API to connect and chat with the model.

stonerboner@lemmynsfw.com on 15 Jun 04:56 collapse

Thanks for clarifying. Cool project! I’ve been looking for a guilt free LLM that sourced its training data in an ethical way. Tell me if I’m way off base, but I take it your app is to the LLMs similar to how the Ice Cubes is an interface for the fediverse. Nice!

I wish you well with your project. If any of the models you work with fit what I’m looking for, or you know of any such models please let me know!

Fingerthief@infosec.pub on 15 Jun 15:39 collapse

Yep that’s a pretty good comparison!

I’m curious on what you mean by sourcing training data in an ethical way? I know OpenAI has come under well deserved scrutiny for apparently using content that is hidden behind paywalls without purchasing it themselves in their training data. Which is quite unethical, but aside from that instance I’m interested in hearing some other concerns for my own education.

In general there are definitely loads of models on places like Hugging Face that are fully open source and provide training data sources for many.

I believe for Microsoft’s new Phi 3 models they actually generated synthetic data themselves for training as well which is an interesting approach that seems to yield good results.

In the open source LLM world the new Meta Llama 3 models are the latest and greatest, I haven’t seen any cause for concerns with it yet. Might be worth looking into those!

JackGreenEarth@lemm.ee on 14 Jun 22:59 next collapse

If you want to make it more unique than ‘just another ChatGPT client’, you could try adding local model support, not sure how difficult that would be.

macgyver@federation.red on 14 Jun 23:04 next collapse

github.com/oobabooga/…/12-‐-OpenAI-API like this? ;)

Pretty sure this is what you want and why it’s not duplicated effort

JackGreenEarth@lemm.ee on 14 Jun 23:09 collapse

I use Jan already, and I like that it’s a native app rather than a webui, I don’t really like webuis. I wasn’t saying there weren’t any local model apps, but that there are far less than glorified ChatGPT clients.

And if they were going to make theirs cross platform, it would in fact be the first FOSS local model app for Android. (Layla Lite exists but is not FOSS).

Fingerthief@infosec.pub on 14 Jun 23:06 collapse

Local models are indeed already supported! In fact any API (local or otherwise) that uses the OpenAI response format (which is the standard) will work.

So you can use something like LM Studio to host a model locally and connect to it via the local API it spins up.

If you want to get crazy…fully local browser models are also supported in Chrome and Edge currently. It will download the selected model fully and load it into the WebGPU of your browser and let you chat. It’s more experimental and takes actual hardware power since you’re fully hosting a model in your browser itself. As seen below.

<img alt="" src="https://infosec.pub/pictrs/image/c55a4dd0-8f77-43d4-9486-52a038abc0e6.png">

JackGreenEarth@lemm.ee on 14 Jun 23:10 collapse

Cool, thanks!

possiblylinux127@lemmy.zip on 14 Jun 23:33 next collapse

Will it work with Ollama?

Fingerthief@infosec.pub on 14 Jun 23:47 collapse

I haven’t personally tried it yet with Ollama but it should work since it looks like Ollama has the ability to use OpenAI Response Formatted API github.com/ollama/ollama/blob/main/…/openai.md

I might give it go here in a bit to test and confirm.

muntedcrocodile@lemm.ee on 15 Jun 06:30 collapse

Does it have tool use? What language is it? Does it use langchain?

Fingerthief@infosec.pub on 15 Jun 16:16 collapse

This project is entirely web based using Vue 3, it doesn’t use langchain and I haven’t looked into it before honestly but I do see they offer a JS library I could utilize. I’ll definitely be looking into that!

As a result there is no LLM function calling currently and apps like LM Studio don’t support function calling when hosting models locally from what I remember. It’s definitely on my list to add the ability to retrieve outside data like searching the web and generating a response with the results etc…

muntedcrocodile@lemm.ee on 15 Jun 16:44 collapse

I wrote this github.com/muntedcrocodile/Sydney

Been having decent success with it but gotta implement proper document embedding.