Best settings for koboldai reddit Smaller versions of the same model are dumber. I would assume most of us have consumer GPU’s 8GB and under. Using repetition penalty 1. You can run any AI model (up to 20B size) that can generate text from the Huggingface website. So I'm running Pigmalion-6b. I They are the best of the best AI models currently available. I had surprisingly good results with just a few short stories and 30 minutes of training on a 7B model. I get replies within 10 seconds that are pretty darn good. So, my personal settings are: Personally I don't use any, but as far as I know they're situational and depend on the model you use, I think they were created especially for different models, so keep that in mind, what worked great for someone won't necessarily work for you if at the very least you don't use the same model This might help you regardless: https://github. KoboldAI Lite. What are the best presets for KoboldAI/KoboldCPP Colab I have been using the model LLaMA2-13B-Tiefighter-GGUF in KobolCpp Colab with these presets, although I feel that the responses are very repetitive, the answers are I have been trying to get into this AI thing, but I tried to allocate koboldAi with 2 different models, with Pygmalion and GPT-Neo-2. Edit: In case anyone is wondering, since I've asked this question there have been some updates, which include the parameters above so now my question is solved :) I've always liked adventure models and been using google colab for running kobold AI. I have 32GB RAM, Ryzen 5800x CPU, and 6700 XT GPU. ai I had a DM running a CYOA book style game. Set Temperature to 2, Top P sampling in the 0. Would also like to hear feedback on any feature requests anyone might have. Understand this will use a lot more VRAM. You should then be able to chat with that character just like in Ooba if you do find a . Discussion for the KoboldAI story generation client. 7B locally on 8GB, you'll need to install finetune's transformers branch using the instructions from this post. I think default is 512 so look for that in settings and raise it. 0. Then you just run one short Hey all. If you check your console when you load up the model, it will tell you how many layers the model has, and how large they are, and how much RAM each layer takes up. I've tried both koboldcpp (CLBlast) and koboldcpp_rocm (hipBLAS (ROCm)). I'm currently running the default model and settings for summarization of Silly Tavern. To go north, turn to page 30, to search room, turn to page 25. When you import a character card into KoboldAI Lite it automatically populates the right fields, so you can see in which style it has put things in to the memory and replicate it yourself if you like. If it doesn't fit completely into VRAM it will be at least 10x slower and basically unusable. Like, someone talked about a "bomb defusal scene" they were doing working better with the temperature higher. 2, you As far as I played around the below settings work alright with Nerys 2. I've been having good results using models based on Chronos or Hermes, and the model I'm using Mythologic L2, seems pretty good too. Some bots like higher temp, some scenes like higher temps. 7B OPT model, and found it extremely good (mostly ‘only the start’, then it gets worse as it goes further with more text). bin and dropping it into kolboldcpp. This is the place for discussion and news about the game, and a way to interact with developers and other players. sh file and name it something rememberable. A place to discuss the SillyTavern fork of TavernAI. 5 KoboldAI is as its creators describe it "Your gateway to GPT writing". Which is now available at https://lite. Temp: 0. It is a proper setup wizard this time that also compresses to a significantly smaller size. You should see 2 Tabs. When it comes to GPU layers and threads how many should I use? I have 12GB of VRAM so I've selected 16 layers and 32 threads with CLBlast (I'm using AMD so no cuda cores for me). I use SillyTavern as my front end 99% of the time, and have pretty much switched to text-generation-webui for running models. 1:5001/api Make sure you load a model in the normal KoboldAI interface first before using the api link. I'm very new to KAI but I've had good experiences with erebus as well as shinen- and even got some nsfw going in nerys but I wasn't exactly in the mood for it at that time so I edited it out. Just mentioning a quick update for the horde webui v3 update, probably the last update for quite some time. 17 is the successor to 0. Discussion for the KoboldAI story generation client. Though, just mess around with the settings and try it out for yourself. I'm new to Koboldai and have been playing around with different GPU/TPU models on colab. I was just wondering, what's your favorite model to use and One thing you could try is playing with the sampler settings. The idea of 'chat mode' is that you name your 'user' and it does some of that for you automatically. exe to run it and have a ZIP file in softpromts for some tweaking. 0 Repetition Penalty: 1. There are no real "best" settings. For 7B I'd try Pygmalion, Choose your preferred AI model, gameplay style, and settings to create the perfect AI experience for you. Super simple docker setup for anyone wanting to containerize on Linux These instructions are for Ubuntu 22. Best Setting for Tavern NSFW ? I'm using more than 6 months). As the other guy said, try using another model. Im using RTX 2060 with 6 VRAM, 32 GB Ram. There are no dependency changes for this one. My only experience with models that large, was that I could barely fit 16 layers on my 3060 card, and had to split the rest on them in normal RAM (about 19 GB), which resulted in about 110 seconds / generation (the default output tokens). Pygmalion 7B is the model that was trained on C. Since you're cutting it close with 8GB, if you really want to stretch your legs with it and run a high max-token count and generate 3-5 outputs per action, you'll probably want to run it on Colab with a 16GB GPU. In a way, it's like ChatGPT but more advanced with Just try all the models with multiple settings. For windows if you have amd it's just not going to work. **So What is SillyTavern?** Tavern is a user interface you can install on your computer (and Android phones) that allows you to interact text generation AIs and chat/roleplay with characters you or the community create. You can even combine LoRAs if you want to mix certain styles, settings and elements. Like everyone else here, I'm also experimenting with MythoMax 13B. 1 Everything else at off/default. 18, if you run the updater at the end of the installation you will automatically have 1. Under Kobold Settings unlock Context Size. If you want to try the latest still-in-development stuff, 4bit/GPTQ supports Llama (Facebook's) models that can be even bigger. not to be rude to the other people on this thread, but wow do people routinely have no idea how the software they're interacting with actually works. 00, Repeat Tokens = 64, I am checking from this page . Have fun! Henk P. 99 range (don't go up to 1 Shout out to Drummer for his Theia 21B model! The model is really amazing! I have tried some different sampler presets and settings, but to be Does anyone have any suggestions on setting up text generation and image generation in general? I have low consistency replies and image I'm very new to this and already played around with Kobold pp, so far so good. At the bottom of the screen, after generating, you can see the Horde volunteer who served you and the AI model used. Best settings and tips for 20B-Erebus ? I want to take the best from the ai, i alrady struggle change from chat mode to adventure to add some actions, so the main question i want some tips or suggestions to get the most of the ai quality if possible, thanks! I saved a custom settings for it within koboldcpp, and it works very well. So just to name a few the following can be pasted in the model name field: - KoboldAI/OPT-13B-Nerys-v2 - KoboldAI/fairseq-dense-13B-Janeway Not personally. As for long output, you can change that in settings. Try others if you want to experiment). In the quick presets dropdown, select Godlike (Another user suggested this setting for writing and I found it works well for me. 02 MinP: 0. What would be the best settings for my server? upvotes My overall thoughts on kobold are - the writing quality was impressive and made sense in about 90% of messages, 10% required edits. Min. I had like 500-800$ for a laptop and didn't think too much of the graphics card as i had some urgent college work i needed to do the next day with a 500-800$ budget. bat in the KoboldAI folder. Might be a bit old, but i have like a 64 Gb Ram, 4 gb vram laptop. It generates really good dialogue for me, and writes first person good. Just enable Adventure mode in the settings and start your actions with You. It will inheret some NSFW stuff from its base model and it has softer NSFW training still within it. 1. To do this, on the page of the selected model, click on the "Copy model name to clipboard" square icon next to the model name highlighted in bold. Page numbers are fake, but it's to give the player easy choices. With these settings I May we see your full generation settings on the Tavern UI? It's possible there's something on there that's messing with the AI responses. View community ranking In the Top 10% of largest communities on Reddit. I need guide or advice for setting the 30b Erebus. KoboldAI United can now run 13B models on the GPU Colab! They are not yet in the menu but all your favorites from the TPU colab and beyond should work (Copy their Huggingface name's not the colab names). The Active one Characters and the second one Settings, click on settings. I decided to give KobaldAI a try after having it recommended to me, and I finally got it working tonight. 7B. Best settings? What are the best settings to make the ai more coherent in skein 6b? Presets You can try these, and maybe adjust them to Has anyone found a particular set of settings good for general use, or specific for RPG play? I agree that you should experiment with lower temp parameter. It also The NSFW ones don't really have adventure training so your best bet is probably Nerys 13B. It was a decent bit of effort to set up (maybe 25 mins?) and then takes a decent bit of effort to run (because you have to prompt it in a more specific way, rather than GPT-4 where you can be really lazy with how you write the prompts and it still gets Now we are in the start menu, firstly we need to connect TavernAI With a backend for it to work. I personally feel like KoboldAI has the worst frontend, so I don’t even use it when I’m using KoboldAI to run a model. What are the best models to use in Kobold for various roleplaying tasks? Specifically my system has a 3060 with 12GB VRAM and 16GB system RAM. Hi I've started to understand more about how AI works but I'm a little lost. ConsiderationNo9044 i know that theres probably shit ton of this here, but are there any tutorials on how to use pygmalion after it got banned on colab? 🙏🙏 No idea about chat, but i use skein 20b for writing assistant. Your KoboldAI IP address with /api behind it. It seems for every model you try, there's some settings it likes more than others. The settings it currently ships with are as follows : 0. You can leave it alone, or choose model(s) from the AI button at the top. It works exactly like main Koboldccp except when you change your temp to 2. 33B airochronos is way faster (on CPU) than 33B airoboros, more flexible, and gives varied responses that often match what I was going for far Depening on settings and text amount it should take a couple of minutes or a few hours. I'm using KoboldAI instead of the horde, so your results may vary. In the Top right there Should be three lines also known as a burger menu, click on it. 9-0. top k like that just blocks the least probable words, whereas top k only allows the top say 40 words or whatever the setting is, and mirostat is near deterministic, it drastically lowers the number of possible For 13B airoboros had been my favorite, because it follows instructions/character cards best of all the 13B models I've tried and gives very well written responses. Skip to main content. KoboldAI is not an AI on its own, its a project where you can bring an AI model yourself. You can just delete the settings in your google drive if you want a full clear preset after messing around. When it's ready, it will open a browser window with the KoboldAI Lite UI. There are some settings for Noromaid in particular that you can import in SillyTavern, though you can also play with newer things like Dynamic Temperature. I'm in shock. I already have a full-featured setup on my PC with Oobabooga and tons of extensions, but yesterday I installed koboldcpp on my Galaxy Note 10 via termux. 04, should work on any Debian but obviously YMMV Start with a git clone (don't feel like adding it to my Dockerfile but go ahead if you do) then add the following Dockerfile: View community ranking In the Top 10% of largest communities on Reddit. You can find the KoboldAI updater in your startmenu or as update-koboldai. On your system you can only fit 2. There’s quite a few models 8GB and under, I’ve been playing around with Facebook’s 2. All of them behave differently than whatever outside opinion you get depending on what character/settings/environments they're ran in. I could be running Vulkan 13B in about the time it takes to run CLBlast 7B. What are some of the best models from chatting? I come from Character ai, but due to the issues with it currently, I decided to learn how to run generators like this on my pc locally (kobald + tavern local), I have 16gb ddr4 ram, and 12gb GDDR6X VRAM, can anyone recommend me some models thay would respond relatively quick, but also with some length (preferably unfiltered, That is correct (though it's missing the newlines, which might help the AI separate the characters in the rp). For those of you who prefer to run KoboldAI portable fear not, that is still an option during the installation as the creation of the uninstaller and shortcuts is entirely optional. The more text you have, the better. Depending if you're on winblows, Linux, or Mac. 7B models would be be the easiest and best for now. Changelog since v1: First off, I would recommend not using KoboldAi horde, and instead switch to KoboldAI proper (hosted on google colab if you cannot do it locally). 95. The one with all the page numbers. It's a browser-based front-end for AI-assisted writing, storytelling and dungeon adventures. I'll just add that if you want to run Neo-2. as far as I know. Hit the Settings button. But going through the summaries it generates, I can see a lot of errors and it misses a lot of key details from the conversation. /r/pathoftitans is the official Path of Titans reddit community. net. . 0 it overrides the setting and runs in the test dynamic temp mode. sh files. Example: 127. It handles storywriting and roleplay excellently, is uncensored, and can do most instruct tasks as well. It's also possible that you didn't properly load the model on the Kobold UI by assigning "layers" to your GPU or you don't have enough VRAM for the regular 6B model and you should use the 4-bit version. It's actually got 2 other types of dynamic temp solutions built in there at different set temperature settings but just set it to 2 and forget imo, it seems to be the best of the 3. Henk's got a good writeup. Is there a better Summarization model out there? Also, what are the best settings to use? Using the SillyTavern built in koboldAI on pygmalion 6b gives pretty lackluster and short responses after a considerable amount of time, is the amount of people using the model makimg it worse? Whenever i was using koboldAI from the colab doc it was a lot better in response time and quality of the response Hey Everyone! The next version of KoboldAI is ready for a wider audience, so we are proud to release an even bigger community made update than the last one. Saying this because in discord, had lots of Kobold AI doesn't use softpromts etc. Faster and more reliable performance: Running AI models locally on your own Ngl it’s mostly for nsfw and other chatbot things, I have a 3060 with 12gb of vram, 32gb of ram, and a Ryzen 7 5800X, I’m hoping for speeds of around 10-15sec with using tavern and koboldcpp. I can't tell you the setting name exactly since I don't have it running rn I usually start at around 28-30 layers for 13B and bellow and start fiddling with the settings from there. AMD doesn't have ROCM for windows for whatever reason. Manually type in 8192 and hit enter Start new chat, don't try to go into an old one without refreshing. json character you want to use in koboldai, paste the json contents into this bad website i just made and paste the output into koboldai the same way So I think that repetition is mostly a parameter settings issue. Or stick with Vulkan 7B for speed. 7B models into VRAM. is the "quantization" of the model. Path of Titans is an MMO dinosaur video game being developed for home computers and mobile devices. AI datasets and is the best for the RP format, but I also read on the forums that 13B models are much better, and I ran GGML variants of regular LLama, Vicuna, and a few others and they did answer more logically and match the prescribed character was much better, but all answers were in simple chat or story generation (visible in I've tired multiple settings, lowering max tokens etc, and the respond is just not good. If you are new to KoboldAI you can use the offline installer for 1. 18. But are there any settings which would fit my 1080ti 11GB GPU? Yup, that one is mine and it includes the settings i use to try and get a better experience for 6B than having it on the games defaults. But it is kinda rigid, and often gives the exact same responses on reroll. That's the most likely cause of the problem. It's a measure of how much the numbers have been truncated to make it smaller. I liked to use koboldcpp from time to time just to communicate with some of the prescribed characters, but not that I understood much about this Quite a complex situation so bear with me, overloading your vram is going to be the worst option at all times. 1. Now things will diverge a bit between Koboldcpp and KoboldAI. And the AI's people can typically run at home are very small by comparison because it is expensive to both use and train larger models. But, i know that's not super helpful. What I mean with top p is, you have settings like top k, and mirostat - and these reduce the possible range of words more than a top k of . Koboldcpp Setting for 30b Erebus . Unfortunately, it is painstakingly slow. i'm going to assume your KoboldAI is KoboldAI Lite is the frontend UI to KoboldAI/KoboldCpp (the latter is the succeeding fork) and is not the AI itself. 17 to avoid confusion. Log In / Sign Up; Advertise on Reddit; Shop Collectible Avatars; Get the Reddit app Scan this QR code to download the app now. I just wanna use my phone and do nsfw stories, but no matter how many time I try the text just get repetitive and not the ones I want to write, can In character. Thats just a plan B from the driver to prevent the software from crashing and its so slow that most of our power users disable the ability altogether in the VRAM settings. Expand user menu Open settings menu. I personally prefer JLLM because of its memory but some Kobold models have a better writing style, so I can't say that it's good or bad. 5 Max Temp: 4. Welcome to r/LearnJapanese, *the* hub on Reddit for learners of the Japanese Language. bat or. Deal is: there are many new models marked as "(United)" and I was wondering if any of these models have better AI dungeon like experien ce. 3B-3B range. Members Online. Also, if you use imagegen, which SD model would you use along with the LLM to stay within memory limits, TIA. It doesn't have repetition at all, and doesn't speak for your character at all if you get your settings real good. koboldai. There are some ways to get around it at least for stable diffusion like onnx or shark but I don't know if text generation has been added into them yet or not. since your running the program, KoboldAI, on your local computer and venus is a hosted website not related to your computer, you'll need to create a link to the open internet that venus can access. 16/1. 7B-Horni, but it turns out that these are very powerful for what my pc is, I have an RTX 2060 with 6gb of Vram and I can't find any suitable model for my pc 10K subscribers in the KoboldAI community. Get app Get the Reddit app Log In Log in to Reddit. Your setup allows you to try the larger models, like 13B Nerys, as you can split layers to RAM. Just take the character description text from this pastebin, paste it into KoboldAI, and enable Chat mode. BAT files or . 16 we noticed that the version numbering on Reddit did not match the version numbers inside KoboldAI and in this release we will streamline this to just 1. I only use kobold when running 4bit models locally on my aging pc. com KoboldAI only supports 16-bit model loading officially (which might change soon). Fortunately I've only started dabbling in KoboldAI two days ago. Though, it consistent of a lot of direction from me, and writing things myself. Still working out the details, but for now: Here is some suggested settings (from this post about the update) for making use of the new Roleplay preset, which has been View community ranking In the Top 10% of largest communities on Reddit. Depending on your GPU, you may not have enough to run at these settings. It's an abbreviation. 7B and Erebus 2. s. The Q4/Q5 etc. Ok. They are the best of the best AI models currently available. I am now trying to accept the fact that either Dolly V2 3B or RedPajamas INCITE Instruct 3B are my best options. If you're in the mood for exploring new models, you might want to try the new Tiefighter 13B model, which is comparable if not better than Mythomax for me. SillyTavern supports Dynamic Temperature now and I suggest to try that. Once you get your preferred settings for the model, throw the entire command string into a . Q: Does KoboldAI have custom models support? A: Yes, it does. My issue is I successfully load the model but the blas only comments sorted by Best Top New Controversial Q&A I haven't tried Novel Ai but here's my experiences in regards to nsfw in KAI. Second of all, change the bot you are using. Same about Open AI question. If you're willing to do a bit more work, 8-bit mode will let you run 13B just barely. confusion because apparently Koboldcpp, KoboldAI, and using pygmalion changes things and terms are very context specific. Has anyone got good setting for skien 20b as I would like to see if it is better than skien 6b but I can't seem to get any good interesting stories Skip to main content Open menu Open navigation Go to Reddit Home I checked out that page and looked at some sampling parameters, what settings do these correspond with in sillytavern? Typical-P = 1. KoboldAI users have more freedom than character cards provide, its why the fields are missing. In the last 24 hours I have probably tested a dozen models in the 1. You'll have more success running a more modern model using one of the forks of KoboldAI that can run llama models (united, You could try Psy fighter. I can't think of any setting that made a difference at all. Dedicated to the horror dungeon crawler game series 'Fear And Hunger', which contains the games 'Fear & Hunger' and its sequel 'Fear & Hunger 2: Termina' by Miro Haverinen. KoboldAI Settings (SillyTavern) I need some help here, can y'all comments sorted by Best Top New Controversial Q&A Add a Comment. GPU layers I've set as 14. It can go from dumb to genius. ibdyw ddfmw cyoks rosqh iqedl vvlbc ykei pske bdgvpo oxz