quantizes
#6
by LeroyDyer - opened
will there be quantizes please?
Also moe is always great especially as you have trained the full 8 experts ! (very good , but locally it take ages to respond _ )
could you make a 30 non moe model ? as it seems with good RAM these models can be loaded locally with partial gpu and cpu settings ! so even with a low gpu it can be loaded locally ! ( quantized ) (Q4)