ryan/turbovault-app
1
0
Fork 0
mirror of https://github.com/ryankazokas/turbovault-app.git synced 2026-04-16 23:22:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

Condenses Documentation

Browse Source
This commit is contained in:
Ryan Kazokas
2026-03-29 02:55:42 -04:00
parent 323484a33a
commit 55fe839882
14 changed files with 363 additions and 2971 deletions
Download Patch File Download Diff File Expand all files Collapse all files

445
docs/IGDB_INTEGRATION.md
View File

@@ -1,391 +1,114 @@
# IGDB Integration Guide
# IGDB Integration
## Overview
TurboVault now integrates with IGDB (Internet Game Database) to automatically match your games with their database entries. This provides access to cover art, metadata, and consistent game identification across users.
TurboVault integrates with the [Internet Game Database (IGDB)](https://www.igdb.com/) to automatically enrich your games with metadata, cover art, and genre information.
## How It Works
### 1. **User Opt-In**
- Users must enable IGDB sync in Settings
- Default: OFF (privacy by design)
- Can be toggled on/off anytime
1. **Background sync** runs every 30 minutes (configurable)
2. **Searches IGDB** for games without metadata
3. **Creates suggestions** with confidence scores (0-100%)
4. **Users review** and approve/reject matches
5. **Metadata applied** when approved (cover art, genres, release dates)
### 2. **Automatic Sync Job**
- Runs every 30 minutes
- Only processes users with sync enabled
- Matches games that don't have IGDB IDs yet
## Setup
### 3. **Smart Matching**
- Searches IGDB using game title + platform
- Returns top 3 matches with confidence scores
- Uses fuzzy matching and platform filtering
### Get IGDB Credentials
### 4. **User Review**
- Users review suggested matches
- See cover art, release year, platform
- Approve or reject each match
- Even high-confidence matches require approval (prevents errors)
1. Create account at https://dev.twitch.tv
2. Register an application
3. Copy Client ID and Client Secret
### 5. **Public Cache**
- Matched games stored in `igdb_games` table
- Shared across all users (public data only)
- Reduces API calls for common games
### Configure
## Features
Add to `.env`:
### ✅ What's Implemented
1. **User Settings**
- Enable/disable IGDB sync
- Last sync timestamp
2. **Background Job**
- Automatic recurring sync (every 30 minutes)
- Rate limiting (4 req/sec)
- Progress tracking
- Error handling
3. **Smart Matching**
- Title similarity scoring
- Platform matching
- Confidence calculation (0-100%)
- Top 3 results per game
4. **Review Interface**
- See all pending matches
- View cover images
- Approve/reject matches
- Bulk reject option
5. **Public Game Cache**
- Shared IGDB data
- Cover URLs
- Release dates
- Match popularity tracking
6. **Platform Mappings**
- 26 platforms mapped to IGDB IDs
- Easy to extend
## Database Schema
### New Tables
**igdb_games** - Public cache of IGDB game data
- igdb_id (unique)
- name, slug, cover_url
- summary, first_release_date
- match_count (popularity)
**igdb_platform_mappings** - Maps our platforms to IGDB
- platform_id → igdb_platform_id
- Pre-seeded with 26 common platforms
**igdb_match_suggestions** - Pending matches for review
- game_id + igdb_id
- confidence_score
- status (pending/approved/rejected)
- Cover and metadata for review
### New Columns
**users**
- igdb_sync_enabled (boolean)
- igdb_last_synced_at (timestamp)
**games**
- igdb_matched_at (timestamp)
- igdb_match_status (string)
- igdb_match_confidence (decimal)
## API Credentials
**Backend Only - Never Exposed to Frontend**
Set in `.env` (gitignored):
```bash
IGDB_CLIENT_ID=your_client_id
IGDB_ACCESS_TOKEN=your_token
IGDB_CLIENT_ID=your_client_id_here
IGDB_CLIENT_SECRET=your_client_secret_here
```
Get credentials from: https://api-docs.igdb.com/
### Enable for Users
Current credentials are already set in the `.env` file.
New users have IGDB sync enabled by default. Users can opt-out in Settings.
## Usage
### For Users
### Review Matches
1. **Enable Sync**
- Go to Settings
- Check "Enable IGDB game matching"
- Click "Update Profile"
Visit `/igdb_matches` to see pending suggestions.
2. **Trigger Sync**
- Go to IGDB Matches page
- Click "Sync Now"
- Wait a few minutes
Each suggestion shows:
- Game title
- Cover art
- Platform
- Confidence score
- Genres from IGDB
3. **Review Matches**
- View pending matches
- See confidence scores
- See cover art and release year
- Approve correct matches
- Reject incorrect ones
**Actions:**
- ✅ **Approve** - Apply metadata to your game
- ❌ **Reject** - Dismiss suggestion
- 🔄 **Sync Now** - Manually trigger sync for your games
4. **Check Progress**
- View stats: Matched, Unmatched, Pending Review
- See last sync time
- Badge in navigation shows pending count
### Manual Commands
### For Developers
**Manually Trigger Sync:**
```ruby
IgdbSyncJob.perform_later
```
**Search IGDB Directly:**
```ruby
service = IgdbService.new
results = service.search_game("Ocarina of Time", platform)
```
**Check Platform Mappings:**
```ruby
IgdbPlatformMapping.igdb_id_for_platform(Platform.find_by(name: "Nintendo 64"))
# => 4
```
**Seed More Platform Mappings:**
```ruby
IgdbPlatformMapping.seed_common_mappings!
```
## Match Confidence Scoring
Confidence score is 0-100%:
**Title Matching (0-70 points)**
- Exact match: 70 points
- Contains match: 50 points
- Word overlap: 0-40 points
**Platform Matching (0-30 points)**
- Exact platform: 30 points
- Similar platform: 20 points
- No match: 0 points
**Result Categories:**
- 95-100%: Very High (auto-suggest first)
- 70-94%: High Confidence
- 50-69%: Medium Confidence
- 0-49%: Low Confidence
## Rate Limiting
**IGDB Limits:** 4 requests/second
**Our Implementation:**
- 0.3s sleep between requests
- 1s sleep every 10 games
- Handles 429 errors gracefully
- Job can be safely interrupted
## Job Scheduling
**Recurring Schedule:**
- Every 30 minutes
- Configured in `config/queue.yml`
- Uses Solid Queue
**Single Instance:**
- Uses Rails cache lock
- Prevents multiple instances running
- 2-hour timeout (auto-releases)
**Manual Trigger:**
- "Sync Now" button in UI
- Or: `IgdbSyncJob.perform_later`
## Troubleshooting
### No Matches Found
**Possible Reasons:**
1. Game title too different from IGDB
2. Platform not mapped
3. Game not in IGDB database
**Solutions:**
- Check game title spelling
- Try alternate titles
- Check if platform is mapped
- Some games may not be in IGDB
### Rate Limit Errors
If you see 429 errors:
- Job will automatically pause
- Will resume on next run
- Check IGDB API status
### Job Not Running
**Check:**
```ruby
# Is job scheduled?
SolidQueue::RecurringTask.all
# Is job running?
IgdbSyncJob.running?
# Clear lock if stuck
Rails.cache.delete("igdb_sync_job:running")
```
### Matches Not Appearing
**Check:**
1. Is sync enabled for user?
2. Are there unmatched games?
3. Check logs: `tail -f log/development.log`
4. Run manually: `IgdbSyncJob.perform_now`
## Adding New Platform Mappings
```ruby
# Find IGDB platform ID from: https://api-docs.igdb.com/#platform
platform = Platform.find_by(name: "Your Platform")
IgdbPlatformMapping.create!(
platform: platform,
igdb_platform_id: 123, # IGDB ID
igdb_platform_name: "Platform Name"
)
```
## Cover Images
**IGDB CDN URLs:**
```ruby
# In model:
igdb_game.cover_image_url("cover_big")
# => https://images.igdb.com/igdb/image/upload/t_cover_big/abc123.jpg
# Available sizes:
# - cover_small (90x128)
# - cover_big (264x374)
# - screenshot_med (569x320)
# - screenshot_big (1280x720)
# - screenshot_huge (1920x1080)
# - 720p, 1080p
```
## Privacy & Security
**What's Shared:**
- Only IGDB game data (names, covers, dates)
- NO user-specific data (prices, locations, notes)
- `igdb_games` table is public metadata only
**What's Private:**
- User's game ownership
- Collections
- Personal notes, prices, locations
- Which user matched which game
**API Credentials:**
- Stored in ENV variables
- Never sent to frontend
- Backend-only service class
## Performance
**Typical Sync:**
- ~100 games: 2-3 minutes
- ~500 games: 10-15 minutes
- Rate limited for API safety
**Database:**
- Indexes on all foreign keys
- Cache lookup before API calls
- Efficient batch processing
## Future Enhancements
**Phase 2 Ideas:**
- Auto-approve very high confidence (>95%)
- Bulk approve/reject
- Search IGDB directly from Add Game form
- Download and store cover images locally
- More metadata (genres, ratings, descriptions)
- Manual IGDB search for failed matches
- Game recommendations based on IGDB data
## API Documentation
**IGDB API Docs:** https://api-docs.igdb.com/
**Authentication:**
- Requires Twitch Client ID + Access Token
- Token must be refreshed periodically (check expiry)
**Endpoints Used:**
- `POST /v4/games` - Search and fetch game data
**Query Example:**
```
search "Zelda Ocarina";
fields id, name, slug, cover.url, summary, first_release_date, platforms.name;
where platforms = (4);
limit 3;
```
## Testing
**Test the Flow:**
1. Enable sync in settings
2. Add a game without IGDB match
3. Click "Sync Now"
4. Wait 1-2 minutes
5. Refresh page - should see matches
6. Approve or reject matches
**Test Games:**
- "The Legend of Zelda: Ocarina of Time" (N64) - Should find match
- "Super Mario 64" (N64) - Should find match
- "Made Up Game Name" - Should find no results
## Support
**Check Logs:**
```bash
tail -f log/development.log | grep IGDB
```
# Trigger sync manually
task igdb:sync
**Rails Console:**
```ruby
# Check sync status
User.find_by(email: "demo@turbovault.com").igdb_sync_enabled
task igdb:status
# View pending matches
User.first.igdb_match_suggestions.status_pending
# Test IGDB service
service = IgdbService.new
results = service.search_game("Mario 64", Platform.find_by(abbreviation: "N64"))
# Clear stuck jobs
task igdb:clear
```
## Summary
## How Matching Works
The IGDB integration is now fully functional:
- ✅ User opt-in settings
- ✅ Automatic sync every 30 minutes
- ✅ Smart matching with confidence scores
- ✅ Review UI with cover art
- ✅ Public game cache
- ✅ Rate limiting and error handling
- ✅ Privacy-preserving design
**Confidence scoring:**
- Title match: 0-70 points
- Platform match: 0-30 points
- 100% = exact title + platform match
Users can now match their games with IGDB for better organization and future features like cover art display!
**Fallback search:**
- First tries with platform filter
- Falls back to no platform if no results
**Genre mapping:**
IGDB genres are automatically mapped to your local genres (e.g., "Role-playing (RPG)" → "RPG").
## Platform Support
26 platforms supported including:
- Nintendo (NES, SNES, N64, GameCube, Wii, Switch)
- PlayStation (PS1-PS5, PSP, Vita)
- Xbox (Xbox, 360, One, Series X/S)
- Sega (Genesis, Dreamcast, Saturn)
- PC platforms (Steam, Epic, GOG)
## Background Job
**Schedule:** Every 30 minutes
**Configuration:** `config/queue.yml`
**Job:** `IgdbSyncJob`
The job:
- Only processes users with `igdb_sync_enabled: true`
- Skips games with `igdb_id` already set
- Creates up to 3 match suggestions per game
- Respects rate limits (4 requests/second)
## API Rate Limits
- **Limit:** 4 requests per second
- **Handled:** Automatic 0.3s delay between requests
- **Caching:** Games cached in `igdb_games` table
## Disabling IGDB
Users can disable in Settings → "Enable IGDB metadata sync"
Admins can disable globally by removing env vars or stopping the recurring job.
## Attribution
IGDB attribution link in footer (required by IGDB terms).