๐Ÿ“ข Imagehoster Go Rewrite AnnouncementsteemCreated with Sketch.

in #steem โ€ข 2 days ago

๐ŸŽ‰ Imagehoster Go Rewrite Announcement

We are thrilled to announce that the Go language rewrite of Imagehoster, the image hosting service for the Steem blockchain, is now complete and ready for testing and deployment!

๐Ÿ“‹ Project Overview

Imagehoster is a decentralized image hosting service designed specifically for the Steem blockchain, allowing Steem users to upload images through signature verification, with features for image proxying, transformation, and avatar hosting. This rewrite completely refactors the original Node.js/TypeScript codebase into Go, delivering significant performance improvements and enhanced maintainability while maintaining 100% API compatibility.

โœจ Core Features

Complete Feature Implementation

  • โœ… Image Upload - Secure upload with Steem signature verification
  • โœ… Image Serving - Efficient image storage and retrieval
  • โœ… Image Proxy & Transformation - Resize and format conversion support
  • โœ… User Avatar Service - Avatar hosting for Steem users
  • โœ… Blacklist Management - Redis-driven blacklist checking (coordinated with Admin Panel)
  • โœ… Rate Limiting - Redis-based sliding window rate limiting algorithm
  • โœ… Health Checks - Multiple health check endpoints
  • โœ… Observability - Prometheus metrics, pprof profiling, structured logging

Technical Architecture Upgrades

Core Components:

  • HTTP Server - Gin high-performance web framework
  • Steem Integration - Steem RPC client for account verification and signature validation
  • Storage Abstraction - Unified BlobStore interface with local disk and S3 implementations
  • Image Processing - High-performance libvips-based image processing (bimg)
  • Cache Layer - Redis for blacklist checking and rate limiting
  • Monitoring - Prometheus metrics, pprof profiling, Zap structured logging

๐Ÿš€ Performance Improvements

Compared to the original Node.js version, the Go rewrite delivers significant performance gains:

MetricNode.js VersionGo VersionImprovement
CPU Usage53.5%25-35%โ†“ 35-53%
Memory Usage662 MB200-400 MBโ†“ 40-70%
Startup TimeSeveral seconds< 1 secondโ†“ >80%
Concurrent RequestsBaseline2-3x improvementโ†‘ 100-200%

๐Ÿ”ง Tech Stack

  • Language: Go 1.23
  • Web Framework: Gin
  • Configuration: Viper (supports environment variables, YAML, TOML)
  • Logging: Zap (structured JSON logging)
  • Redis: go-redis/v9
  • Storage: AWS SDK Go v2 (S3), local filesystem
  • Image Processing: bimg (libvips bindings)
  • Blockchain: steemgosdk, steemutil
  • Monitoring: Prometheus, pprof
  • Testing: standard library testing, testify

๐Ÿ“ฆ Deployment Features

Flexible Storage Configuration

  • Dual Bucket Support - Separate S3 buckets for uploaded and proxied images
  • Local Storage - Supports local filesystem storage (dev/test environments)
  • S3 Storage - Supports AWS S3 and compatible services (production)
  • IAM Roles - Supports EC2 IAM roles for automatic credential retrieval

Containerized Deployment

  • Docker Support - Optimized Dockerfile and Docker Compose configuration
  • Multi-stage Builds - Reduced image size, improved deployment efficiency
  • Health Checks - Container health checks and graceful shutdown

Configuration Management

  • Environment Variable Priority - Environment variables override config files
  • Multi-environment Configs - Separate configs for default, production, and test environments
  • Backward Compatibility - Legacy configuration options preserved for smooth migration

๐Ÿงช Test Coverage

The project has established a comprehensive testing framework:

  • Unit Tests - Core business logic coverage
  • Integration Tests - HTTP handler integration testing
  • E2E Tests - End-to-end functionality validation
  • Docker Tests - Containerized test environment (no local libvips required)

๐Ÿ”Œ API Compatibility

The Go version maintains 100% API compatibility with the Node.js version:

  • โœ… All API paths identical
  • โœ… Request/response formats unchanged
  • โœ… HTTP status codes and error codes match exactly
  • โœ… Storage key naming rules fully compatible
  • โœ… Can read images stored by Node.js version

API Endpoints

MethodPathDescription
POST/:username/:signatureUpload image
GET/:hashGet image
GET/p/:base58urlProxy and transform image
GET/:width}x:height/:urlLegacy proxy endpoint
GET/avatar/:username/:sizeGet user avatar
GET/Health check
GET/.well-known/healthcheck.jsonHealth check (JSON)

๐Ÿ“š Project Status

  • โœ… Core Features Complete - All major functionality implemented
  • โœ… Comprehensive Testing - Unit tests, integration tests, E2E tests
  • โœ… CI/CD Ready - GitHub Actions workflows
  • โœ… Production Ready - Ready for production deployment

๐ŸŽฏ Next Steps

While core features are complete, we continue to optimize:

  1. Performance Optimization - Further reduce resource usage
  2. Task Scheduling - Periodic cleanup of unaccessed images
  3. Open Source - When the project is stable, we will open source

๐Ÿ“ž Contact

Any issues found, feel free to leave a message here.

Sort: ย 

Ready for production

That's great to hear!

Are there any plans to save images that are pasted via the clipboard as .jpg or .webp? This would certainly also be a significant improvement in terms of file size and loading time.

WebP would be a solution to be considered.

What do you think, is that a risk?

Upvoted! Thank you for supporting witness @jswit.