AI Model Testing on a Single Graphics Card: Ornith-1.5 and Qwen3.8
PROSTO24 specialists have conducted an intensive series of tests to evaluate the performance of various quantized artificial intelligence models on a single graphics card. The primary objective of this research was to identify optimal configurations for local deployment of language models and to assess the practicality of acquiring future high-performance cards, such as the RTX 5090, for home users.
Methodology and Results from Testing 18 Configurations
The initial phase of the study involved testing 18 distinct configurations. This process included evaluating three different AI models, with five to six quantization variants (Q5/Q6) applied to each. All configurations were subjected to the same set of 22 real-world practical tasks. Preliminary results indicate that models with 9B and Q5/Q6 quantization offer sufficient performance for most everyday tasks, raising questions about the immediate necessity for flagship solutions like the RTX 5090 for average users.
Comparing DeepSeek V4 Flash with Ornith1.5-35B and Qwen3.8-27B
Further investigation into the performance of locally hosted models continued with a comparison of DeepSeek V4 Flash against Ornith1.5-35B and Qwen3.8-27B. Previous tests had compared Qwen3.8-27B with DeepSeek V4 Flash via API. The current phase of the study focused on evaluating DeepSeek running on the same card, but quantized down to 2 bits.
- DeepSeek V4 Flash (2-bit): 0.697 points
- DeepSeek V4 Flash (full precision): 0.731 points
The 0.034-point difference between the 2-bit version and the full-precision version raises questions about the true ‘cost’ of quantization and the potential role of noise in the obtained data. These results underscore the importance of carefully evaluating the trade-offs between performance and efficiency when selecting quantized models for local use.
I’ve been playing around with Qwen3.8-27B quantized models on my old 3090, and the performance hit with Q5/Q6 really isn’t as bad as I expected for most creative writing tasks. The speedup is noticeable, but sometimes the output can get a bit ‘fuzzy’ or less coherent on complex prompts, especially at Q4. My tip would be to always test your specific use-case thoroughly across different quantization levels. For me, Q6_K_M seems to be the sweet spot, balancing quality and VRAM use effectively without needing to jump to a 5090 just yet.