๐ข Imagehoster Go Rewrite Announcement
๐ Imagehoster Go Rewrite Announcement
We are thrilled to announce that the Go language rewrite of Imagehoster, the image hosting service for the Steem blockchain, is now complete and ready for testing and deployment!
๐ Project Overview
Imagehoster is a decentralized image hosting service designed specifically for the Steem blockchain, allowing Steem users to upload images through signature verification, with features for image proxying, transformation, and avatar hosting. This rewrite completely refactors the original Node.js/TypeScript codebase into Go, delivering significant performance improvements and enhanced maintainability while maintaining 100% API compatibility.
โจ Core Features
Complete Feature Implementation
- โ Image Upload - Secure upload with Steem signature verification
- โ Image Serving - Efficient image storage and retrieval
- โ Image Proxy & Transformation - Resize and format conversion support
- โ User Avatar Service - Avatar hosting for Steem users
- โ Blacklist Management - Redis-driven blacklist checking (coordinated with Admin Panel)
- โ Rate Limiting - Redis-based sliding window rate limiting algorithm
- โ Health Checks - Multiple health check endpoints
- โ Observability - Prometheus metrics, pprof profiling, structured logging
Technical Architecture Upgrades
Core Components:
- HTTP Server - Gin high-performance web framework
- Steem Integration - Steem RPC client for account verification and signature validation
- Storage Abstraction - Unified BlobStore interface with local disk and S3 implementations
- Image Processing - High-performance libvips-based image processing (bimg)
- Cache Layer - Redis for blacklist checking and rate limiting
- Monitoring - Prometheus metrics, pprof profiling, Zap structured logging
๐ Performance Improvements
Compared to the original Node.js version, the Go rewrite delivers significant performance gains:
| Metric | Node.js Version | Go Version | Improvement |
|---|---|---|---|
| CPU Usage | 53.5% | 25-35% | โ 35-53% |
| Memory Usage | 662 MB | 200-400 MB | โ 40-70% |
| Startup Time | Several seconds | < 1 second | โ >80% |
| Concurrent Requests | Baseline | 2-3x improvement | โ 100-200% |
๐ง Tech Stack
- Language: Go 1.23
- Web Framework: Gin
- Configuration: Viper (supports environment variables, YAML, TOML)
- Logging: Zap (structured JSON logging)
- Redis:
go-redis/v9 - Storage: AWS SDK Go v2 (S3), local filesystem
- Image Processing: bimg (libvips bindings)
- Blockchain: steemgosdk, steemutil
- Monitoring: Prometheus, pprof
- Testing: standard library testing, testify
๐ฆ Deployment Features
Flexible Storage Configuration
- Dual Bucket Support - Separate S3 buckets for uploaded and proxied images
- Local Storage - Supports local filesystem storage (dev/test environments)
- S3 Storage - Supports AWS S3 and compatible services (production)
- IAM Roles - Supports EC2 IAM roles for automatic credential retrieval
Containerized Deployment
- Docker Support - Optimized Dockerfile and Docker Compose configuration
- Multi-stage Builds - Reduced image size, improved deployment efficiency
- Health Checks - Container health checks and graceful shutdown
Configuration Management
- Environment Variable Priority - Environment variables override config files
- Multi-environment Configs - Separate configs for default, production, and test environments
- Backward Compatibility - Legacy configuration options preserved for smooth migration
๐งช Test Coverage
The project has established a comprehensive testing framework:
- Unit Tests - Core business logic coverage
- Integration Tests - HTTP handler integration testing
- E2E Tests - End-to-end functionality validation
- Docker Tests - Containerized test environment (no local libvips required)
๐ API Compatibility
The Go version maintains 100% API compatibility with the Node.js version:
- โ All API paths identical
- โ Request/response formats unchanged
- โ HTTP status codes and error codes match exactly
- โ Storage key naming rules fully compatible
- โ Can read images stored by Node.js version
API Endpoints
| Method | Path | Description |
|---|---|---|
| POST | /:username/:signature | Upload image |
| GET | /:hash | Get image |
| GET | /p/:base58url | Proxy and transform image |
| GET | /:width}x:height/:url | Legacy proxy endpoint |
| GET | /avatar/:username/:size | Get user avatar |
| GET | / | Health check |
| GET | /.well-known/healthcheck.json | Health check (JSON) |
๐ Project Status
- โ Core Features Complete - All major functionality implemented
- โ Comprehensive Testing - Unit tests, integration tests, E2E tests
- โ CI/CD Ready - GitHub Actions workflows
- โ Production Ready - Ready for production deployment
๐ฏ Next Steps
While core features are complete, we continue to optimize:
- Performance Optimization - Further reduce resource usage
- Task Scheduling - Periodic cleanup of unaccessed images
- Open Source - When the project is stable, we will open source
๐ Contact
Any issues found, feel free to leave a message here.
That's great to hear!
Are there any plans to save images that are pasted via the clipboard as .jpg or .webp? This would certainly also be a significant improvement in terms of file size and loading time.
0.00 SBD,
2.74 STEEM,
2.74 SP
WebP would be a solution to be considered.
0.00 SBD,
2.05 STEEM,
2.05 SP
What do you think, is that a risk?
Upvoted! Thank you for supporting witness @jswit.