The Evolution of an Under-Desk AI Platform: A Solo Developer’s Journey
What began as a personal quest to build a ChatGPT alternative has evolved into a sophisticated AI platform, entirely self-hosted on a developer’s under-desk hardware. Over two years, a simple Telegram bot project blossomed into a complex system featuring 66 containers across two nodes, all powered by home-based infrastructure.
The initial bot, launched on August 26, 2024, was rudimentary: it utilized a single model, lacked context, and struggled with lengthy messages. Despite its basic nature, it quickly found users among family and friends, prompting the developer to embark on what was initially conceived as “minor improvements.” These minor adjustments ultimately led to the creation of a multi-functional platform.
Platform Architecture and Capabilities
The current iteration of the platform boasts an impressive array of features and architectural solutions:
- Multi-Channel Delivery: Support for six communication channels, including Telegram, VK, MAX, Discord, a web interface, and email for notifications and long-running task results.
- Advanced Features: Integrated billing, a voice agent, and a Retrieval-Augmented Generation (RAG) system for document processing.
- Robust Infrastructure: Comprising 66 containers across two nodes, BGP routing, a custom Continuous Integration (CI) system, and a dedicated helpdesk.
- GPU Node: Inclusion of a GPU node with an RTX 3090 graphics card for local models, which significantly increased power consumption compared to the initial setup that consumed less than a gaming console.
The developer notes that a substantial portion of the platform’s code was generated by AI. This approach was not without its challenges; shortly after the first deployment, the platform faced a ransomware attack that wiped out its database. This incident highlighted the critical importance of understanding security implications when deploying AI applications online.
In its current state, the platform is capable of handling up to 1500 concurrent requests, leveraging queues, pgvector for memory, deterministic scenarios, an isolated sandbox, and task recovery mechanisms after failures. This project stands as a testament to how enthusiasm and a willingness to experiment can lead to the creation of powerful and complex systems, even without a team or budget, illustrating the transformative potential of continuous development in a home infrastructure setting.
This case study provides fascinating insight into the practical scalability and operational complexities of self-hosted AI infrastructure. The evolution from a single model Telegram bot to a 66-container system with BGP routing and RAG capabilities, all on under-desk hardware, underscores the immense potential of distributed, resource-optimized deployments. The mention of an RTX 3090 for local models is particularly telling regarding the computational demands for achieving acceptable inference speeds, even for a non-enterprise setup. The ransomware incident also highlights critical security vectors often overlooked in rapid AI development.