Viewing context:Principal Architect

Private LLM R&D Lab

MDSi

Private LLM research and evaluation lab comparing 10+ open-source models across coding, reasoning, and general enterprise use cases. Established cost-effective AI infrastructure at ~$4K/year versus $15K+ for commercial alternatives while maintaining data privacy and eliminating API dependencies.

10+

Models Evaluated

3B-70B

Parameter Range

~$4K/yr

vs $15K+ Commercial

AI Infrastructure Design

Contribution as Principal Architect

Designed scalable Ollama deployment with model registry
Created IDE integration architecture with Continue.dev
Established security patterns for private AI access

Core Capabilities

Model Benchmarking

Comparative analysis of coding models across accuracy, speed, and cost

IDE Code Assistance

Private code completion and chat via Continue.dev

Data Privacy

Zero code exposure to external APIs - all inference runs locally

Cost Optimization

Fixed ~$324/month vs per-token API pricing

Modules & Services

Model Evaluation Framework

Systematic evaluation of open-source LLMs for enterprise coding assistance - Llama 3.1 (8B-70B), Mistral 7B, CodeLlama 7B, Gemma2 9B, Deepseek Coder 6.7B, Qwen2.5, Phi-3, Mistral-Nemo 12B

Azure Infrastructure

Enterprise-grade VM deployment with Bicep IaC, automated provisioning, and cost-optimized configuration

IDE Integration Research

Continue.dev integration for VS Code and JetBrains, enabling real-time code suggestions from private models

Security Hardening

Zero-trust security with SSH key authentication, fail2ban protection, and locked-down network security groups

Tech Stack

Languages

BashPython

Frameworks

OllamaContinue.dev

Infrastructure

Azure VMAzure BicepUbuntu 22.04

Data

None

Loading project...