Model,Size,BPW,Perplexity
BF16-baseline,1.1G,16.08,406.0250
Q8_0,610M,8.58,409.5689
Q6_K,472M,6.65,417.3712
Q5_1,442M,6.23,426.9407
Q5_K_M,424M,5.96,442.9431
Q5_0,416M,5.86,413.1916
Q5_K_S,416M,5.86,414.9329
Q4_1-imat,390M,5.49,403.0646
Q4_K_M-imat,378M,5.32,406.6788
Q4_K_S-imat,365M,5.14,406.9947
Q4_0-imat,364M,5.13,419.8843
IQ4_NL-imat,364M,5.12,435.0203
Q3_K_L-imat,351M,4.94,412.0217
IQ4_XS-imat,351M,4.94,451.4025
Q3_K_M-imat,331M,4.66,406.6408
IQ3_M-imat,320M,4.51,460.9405
IQ3_S-imat,308M,4.34,475.4797
Q3_K_S-imat,308M,4.34,340.2907
IQ3_XS-imat,298M,4.20,520.3907
Q2_K-imat,282M,3.97,797.8549
Q2_K_S-imat,267M,3.76,1561.2449
IQ3_XXS-imat,266M,3.74,613.9329
IQ2_M-imat,252M,3.55,1283.4407
IQ2_S-imat,242M,3.41,1857.4142
TQ2_0-imat,236M,3.32,16937766.5110
IQ2_XS-imat,231M,3.25,3632.9250
IQ2_XXS-imat,219M,3.08,5641.8950
TQ1_0-imat,216M,3.04,16937766.5110
IQ1_M-imat,206M,2.90,7495.4178
IQ1_S-imat,198M,2.79,23414.9432