---
license: apache-2.0
---
|                                               **Coffee & AI**                                               |
| :---------------------------------------------------------------------------------------------------------: |
| [![Discord](https://img.shields.io/discord/232596713892872193?logo=discord)](https://discord.gg/2JhHVh7CGu) |

Disclaimer: The specific method of tuning has likely removed the entirety of the models safeguards, including its tendency to soft-refuse and redirect. This also means that safeguards that you'd typically *want or expect* in a model are probably not present, this is more of a hammer method than a scalpel.


This model is an experimental finetune of gemma-26b-4A ~ it follows the exact same instruct prompting methods as the original model did.


Two parts: A new kind of qualtiy-preserving abliteration (loosely based on heretic) followed by retraining the abliterated model using an evolutionary strategy loosely based off of <https://arxiv.org/abs/2511.16652>

I've talked about the abliteration in my prior model setup, so I'll discuss only the finetuning method here:

### Data, briefly

We start with a baseline sample of a variety of (primarily books), chunk them, and then have the abliterated model caption the stories as a prompt. -> "Generate a story with a protagnoist named Alice..."

### Tuning Method, also briefly

The model was abliterated and trained in full (BF16) precision. Low Rank was empoyed on a per-sample basis (Rank 1 LORA per sample into a full-precision buffer -- with stochastic rounding) this means each update is highly-approximated, but the noisy landscape is represented in full-precision buffer, so eventually we get noise cancellation in the buffer and it becomes a full-rank tuning method in a gaussian landscape.

We use a non-differentiable objective combined with teacher forcing. The specific non-differentiable objective is overly complex to describe but the primary part exploits the zipf structure of language as a proxy for long-term dependencies in stories.

### Hardware and Software notes

This training was all done purely on some very powerful CPUs I have in my garage running at about 350W and took approximately 280 hours. The cost of power to me comes out to about ~20-25$ USD, the hardware cost is about ~20K USD at the time of this model upload. The main downside of this method is time, it's quite slow. All of this was done in a custom inference engine I'm writing in mojo, including the abliteration and ES training.