PKC Benchmark Tool MARK (Public Edition) Analysis Report
๐ Overview
This project is an AI model performance measurement tool. Simply put, it's a "program that tests how fast and accurately AI models operate."
Korean Performance Test Video: [Link]
๐ฏ Key Features
1. One-Click Installation
- Windows: Double-click OneClick_RUN.bat
- macOS: Double-click oneclick.command
- Linux: Run oneclick.sh
- → Ready to use without complex setup
2. Automatic AI Model Management
- Search/download models from HuggingFace
- Auto-detection of GGUF and Transformers formats
- → No need for users to manually find and install models
3. Performance Measurement
- Response speed (TPS - Tokens Per Second)
- Memory usage (VRAM)
- Time to First Token (TTFT)
- GPU temperature/power consumption
- → Verify actual AI model performance with numbers
4. Real-time Chat Testing
- Direct conversation with models after benchmark completion
- → Check not only performance metrics but also actual usability
๐๏ธ Technical Architecture
Backend (Server)
- Language: Python
- Framework: FastAPI
- Main Role: Model loading, performance measurement, API provision
Frontend (User Interface)
- Language: HTML + JavaScript
- Styling: Tailwind CSS
- Main Role: User interaction, result visualization
Installation/Execution Scripts
- Windows: .bat batch files
- Linux/macOS: .sh shell scripts
- Main Role: Automatic environment setup, dependency installation
๐ Supported Platforms
Operating System Support Level Notes
| Windows 10/11 | โ Full Support | Recommended environment |
| macOS (Intel/M1/M2/M3) | โ Full Support | Metal acceleration support |
| Ubuntu/Debian Linux | โ Full Support | CUDA support |
| Other Linux | โ Mostly Supported |
๐ฎ GPU Support
GPU Type Support Level Description
| NVIDIA (CUDA) | โ Fully Optimized | RTX, GTX series |
| Apple Silicon | โ Fully Optimized | M1/M2/M3 Metal acceleration |
| CPU Only | โ Full Support | Works on all environments |
| AMD GPU | โ ๏ธ Experimental | Linux ROCm partial support |
๐ง Core File Roles
Execution Files
- OneClick_RUN.bat (Windows) / oneclick.sh (Linux/Mac)
- Role: Automate all installation and execution with one click
- Process: Create Python virtual environment → Install libraries → Start server → Open browser
Core Programs
- benchmark_server.py
- Role: Main server program
- Features: Model loading, performance measurement, web API provision, HuggingFace integration
- benchmark_canvas.html
- Role: User interface
- Features: Model selection, settings adjustment, result verification, chart display
Configuration Files
- requirements.txt: List of required Python libraries
- config_json_public.json: Program configuration values
- models_json_public.json: Model information management
Utilities
- install_wizard.py: GPU optimization automatic installation tool
๐ Project Strengths
1. User-Friendliness
- Minimized entry barriers with one-click installation
- Intuitive web interface
- Multi-language support (Korean/English)
2. Technical Completeness
- Cross-platform support (Windows/Mac/Linux)
- Various GPU optimizations (NVIDIA/Apple/AMD)
- Real-time performance monitoring
3. Scalability
- Complete HuggingFace ecosystem integration
- Modular architecture
- API-based design
4. Practicality
- Provides actual performance data
- Model comparison functionality
- Real usability verification through chat testing
๐ฏ Target Users
- AI Developers: Model performance comparison and optimization
- Researchers: Benchmark data collection
- General Users: AI model experience and testing
- Enterprises: Production environment performance verification (license verification required)
๐ Usage Scenarios
Development Stage
- Performance measurement of new models
- Hardware-specific optimization verification
- Memory usage analysis
Research Stage
- Performance comparison between models
- Benchmark data collection
- Experimental data generation for papers
Operation Stage
- Production environment performance verification
- Server resource planning
- Cost-effectiveness analysis
Conclusion: This project is a highly complete tool that makes the specialized task of AI model performance measurement easily accessible to everyone. The biggest advantage is its design that allows immediate use without installation complexity.
The PKC Benchmark Tool will be distributed through this blog after revision work. (Though we don't know when that will be...)