mirror of
https://github.com/ryankazokas/turbovault-app.git
synced 2026-04-16 23:22:53 +00:00
Moving to github
This commit is contained in:
391
docs/IGDB_INTEGRATION.md
Normal file
391
docs/IGDB_INTEGRATION.md
Normal file
@@ -0,0 +1,391 @@
|
||||
# IGDB Integration Guide
|
||||
|
||||
## Overview
|
||||
|
||||
TurboVault now integrates with IGDB (Internet Game Database) to automatically match your games with their database entries. This provides access to cover art, metadata, and consistent game identification across users.
|
||||
|
||||
## How It Works
|
||||
|
||||
### 1. **User Opt-In**
|
||||
- Users must enable IGDB sync in Settings
|
||||
- Default: OFF (privacy by design)
|
||||
- Can be toggled on/off anytime
|
||||
|
||||
### 2. **Automatic Sync Job**
|
||||
- Runs every 30 minutes
|
||||
- Only processes users with sync enabled
|
||||
- Matches games that don't have IGDB IDs yet
|
||||
|
||||
### 3. **Smart Matching**
|
||||
- Searches IGDB using game title + platform
|
||||
- Returns top 3 matches with confidence scores
|
||||
- Uses fuzzy matching and platform filtering
|
||||
|
||||
### 4. **User Review**
|
||||
- Users review suggested matches
|
||||
- See cover art, release year, platform
|
||||
- Approve or reject each match
|
||||
- Even high-confidence matches require approval (prevents errors)
|
||||
|
||||
### 5. **Public Cache**
|
||||
- Matched games stored in `igdb_games` table
|
||||
- Shared across all users (public data only)
|
||||
- Reduces API calls for common games
|
||||
|
||||
## Features
|
||||
|
||||
### ✅ What's Implemented
|
||||
|
||||
1. **User Settings**
|
||||
- Enable/disable IGDB sync
|
||||
- Last sync timestamp
|
||||
|
||||
2. **Background Job**
|
||||
- Automatic recurring sync (every 30 minutes)
|
||||
- Rate limiting (4 req/sec)
|
||||
- Progress tracking
|
||||
- Error handling
|
||||
|
||||
3. **Smart Matching**
|
||||
- Title similarity scoring
|
||||
- Platform matching
|
||||
- Confidence calculation (0-100%)
|
||||
- Top 3 results per game
|
||||
|
||||
4. **Review Interface**
|
||||
- See all pending matches
|
||||
- View cover images
|
||||
- Approve/reject matches
|
||||
- Bulk reject option
|
||||
|
||||
5. **Public Game Cache**
|
||||
- Shared IGDB data
|
||||
- Cover URLs
|
||||
- Release dates
|
||||
- Match popularity tracking
|
||||
|
||||
6. **Platform Mappings**
|
||||
- 26 platforms mapped to IGDB IDs
|
||||
- Easy to extend
|
||||
|
||||
## Database Schema
|
||||
|
||||
### New Tables
|
||||
|
||||
**igdb_games** - Public cache of IGDB game data
|
||||
- igdb_id (unique)
|
||||
- name, slug, cover_url
|
||||
- summary, first_release_date
|
||||
- match_count (popularity)
|
||||
|
||||
**igdb_platform_mappings** - Maps our platforms to IGDB
|
||||
- platform_id → igdb_platform_id
|
||||
- Pre-seeded with 26 common platforms
|
||||
|
||||
**igdb_match_suggestions** - Pending matches for review
|
||||
- game_id + igdb_id
|
||||
- confidence_score
|
||||
- status (pending/approved/rejected)
|
||||
- Cover and metadata for review
|
||||
|
||||
### New Columns
|
||||
|
||||
**users**
|
||||
- igdb_sync_enabled (boolean)
|
||||
- igdb_last_synced_at (timestamp)
|
||||
|
||||
**games**
|
||||
- igdb_matched_at (timestamp)
|
||||
- igdb_match_status (string)
|
||||
- igdb_match_confidence (decimal)
|
||||
|
||||
## API Credentials
|
||||
|
||||
**Backend Only - Never Exposed to Frontend**
|
||||
|
||||
Set in `.env` (gitignored):
|
||||
```bash
|
||||
IGDB_CLIENT_ID=your_client_id
|
||||
IGDB_ACCESS_TOKEN=your_token
|
||||
```
|
||||
|
||||
Get credentials from: https://api-docs.igdb.com/
|
||||
|
||||
Current credentials are already set in the `.env` file.
|
||||
|
||||
## Usage
|
||||
|
||||
### For Users
|
||||
|
||||
1. **Enable Sync**
|
||||
- Go to Settings
|
||||
- Check "Enable IGDB game matching"
|
||||
- Click "Update Profile"
|
||||
|
||||
2. **Trigger Sync**
|
||||
- Go to IGDB Matches page
|
||||
- Click "Sync Now"
|
||||
- Wait a few minutes
|
||||
|
||||
3. **Review Matches**
|
||||
- View pending matches
|
||||
- See confidence scores
|
||||
- See cover art and release year
|
||||
- Approve correct matches
|
||||
- Reject incorrect ones
|
||||
|
||||
4. **Check Progress**
|
||||
- View stats: Matched, Unmatched, Pending Review
|
||||
- See last sync time
|
||||
- Badge in navigation shows pending count
|
||||
|
||||
### For Developers
|
||||
|
||||
**Manually Trigger Sync:**
|
||||
```ruby
|
||||
IgdbSyncJob.perform_later
|
||||
```
|
||||
|
||||
**Search IGDB Directly:**
|
||||
```ruby
|
||||
service = IgdbService.new
|
||||
results = service.search_game("Ocarina of Time", platform)
|
||||
```
|
||||
|
||||
**Check Platform Mappings:**
|
||||
```ruby
|
||||
IgdbPlatformMapping.igdb_id_for_platform(Platform.find_by(name: "Nintendo 64"))
|
||||
# => 4
|
||||
```
|
||||
|
||||
**Seed More Platform Mappings:**
|
||||
```ruby
|
||||
IgdbPlatformMapping.seed_common_mappings!
|
||||
```
|
||||
|
||||
## Match Confidence Scoring
|
||||
|
||||
Confidence score is 0-100%:
|
||||
|
||||
**Title Matching (0-70 points)**
|
||||
- Exact match: 70 points
|
||||
- Contains match: 50 points
|
||||
- Word overlap: 0-40 points
|
||||
|
||||
**Platform Matching (0-30 points)**
|
||||
- Exact platform: 30 points
|
||||
- Similar platform: 20 points
|
||||
- No match: 0 points
|
||||
|
||||
**Result Categories:**
|
||||
- 95-100%: Very High (auto-suggest first)
|
||||
- 70-94%: High Confidence
|
||||
- 50-69%: Medium Confidence
|
||||
- 0-49%: Low Confidence
|
||||
|
||||
## Rate Limiting
|
||||
|
||||
**IGDB Limits:** 4 requests/second
|
||||
|
||||
**Our Implementation:**
|
||||
- 0.3s sleep between requests
|
||||
- 1s sleep every 10 games
|
||||
- Handles 429 errors gracefully
|
||||
- Job can be safely interrupted
|
||||
|
||||
## Job Scheduling
|
||||
|
||||
**Recurring Schedule:**
|
||||
- Every 30 minutes
|
||||
- Configured in `config/queue.yml`
|
||||
- Uses Solid Queue
|
||||
|
||||
**Single Instance:**
|
||||
- Uses Rails cache lock
|
||||
- Prevents multiple instances running
|
||||
- 2-hour timeout (auto-releases)
|
||||
|
||||
**Manual Trigger:**
|
||||
- "Sync Now" button in UI
|
||||
- Or: `IgdbSyncJob.perform_later`
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### No Matches Found
|
||||
|
||||
**Possible Reasons:**
|
||||
1. Game title too different from IGDB
|
||||
2. Platform not mapped
|
||||
3. Game not in IGDB database
|
||||
|
||||
**Solutions:**
|
||||
- Check game title spelling
|
||||
- Try alternate titles
|
||||
- Check if platform is mapped
|
||||
- Some games may not be in IGDB
|
||||
|
||||
### Rate Limit Errors
|
||||
|
||||
If you see 429 errors:
|
||||
- Job will automatically pause
|
||||
- Will resume on next run
|
||||
- Check IGDB API status
|
||||
|
||||
### Job Not Running
|
||||
|
||||
**Check:**
|
||||
```ruby
|
||||
# Is job scheduled?
|
||||
SolidQueue::RecurringTask.all
|
||||
|
||||
# Is job running?
|
||||
IgdbSyncJob.running?
|
||||
|
||||
# Clear lock if stuck
|
||||
Rails.cache.delete("igdb_sync_job:running")
|
||||
```
|
||||
|
||||
### Matches Not Appearing
|
||||
|
||||
**Check:**
|
||||
1. Is sync enabled for user?
|
||||
2. Are there unmatched games?
|
||||
3. Check logs: `tail -f log/development.log`
|
||||
4. Run manually: `IgdbSyncJob.perform_now`
|
||||
|
||||
## Adding New Platform Mappings
|
||||
|
||||
```ruby
|
||||
# Find IGDB platform ID from: https://api-docs.igdb.com/#platform
|
||||
platform = Platform.find_by(name: "Your Platform")
|
||||
IgdbPlatformMapping.create!(
|
||||
platform: platform,
|
||||
igdb_platform_id: 123, # IGDB ID
|
||||
igdb_platform_name: "Platform Name"
|
||||
)
|
||||
```
|
||||
|
||||
## Cover Images
|
||||
|
||||
**IGDB CDN URLs:**
|
||||
```ruby
|
||||
# In model:
|
||||
igdb_game.cover_image_url("cover_big")
|
||||
# => https://images.igdb.com/igdb/image/upload/t_cover_big/abc123.jpg
|
||||
|
||||
# Available sizes:
|
||||
# - cover_small (90x128)
|
||||
# - cover_big (264x374)
|
||||
# - screenshot_med (569x320)
|
||||
# - screenshot_big (1280x720)
|
||||
# - screenshot_huge (1920x1080)
|
||||
# - 720p, 1080p
|
||||
```
|
||||
|
||||
## Privacy & Security
|
||||
|
||||
**What's Shared:**
|
||||
- Only IGDB game data (names, covers, dates)
|
||||
- NO user-specific data (prices, locations, notes)
|
||||
- `igdb_games` table is public metadata only
|
||||
|
||||
**What's Private:**
|
||||
- User's game ownership
|
||||
- Collections
|
||||
- Personal notes, prices, locations
|
||||
- Which user matched which game
|
||||
|
||||
**API Credentials:**
|
||||
- Stored in ENV variables
|
||||
- Never sent to frontend
|
||||
- Backend-only service class
|
||||
|
||||
## Performance
|
||||
|
||||
**Typical Sync:**
|
||||
- ~100 games: 2-3 minutes
|
||||
- ~500 games: 10-15 minutes
|
||||
- Rate limited for API safety
|
||||
|
||||
**Database:**
|
||||
- Indexes on all foreign keys
|
||||
- Cache lookup before API calls
|
||||
- Efficient batch processing
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
**Phase 2 Ideas:**
|
||||
- Auto-approve very high confidence (>95%)
|
||||
- Bulk approve/reject
|
||||
- Search IGDB directly from Add Game form
|
||||
- Download and store cover images locally
|
||||
- More metadata (genres, ratings, descriptions)
|
||||
- Manual IGDB search for failed matches
|
||||
- Game recommendations based on IGDB data
|
||||
|
||||
## API Documentation
|
||||
|
||||
**IGDB API Docs:** https://api-docs.igdb.com/
|
||||
|
||||
**Authentication:**
|
||||
- Requires Twitch Client ID + Access Token
|
||||
- Token must be refreshed periodically (check expiry)
|
||||
|
||||
**Endpoints Used:**
|
||||
- `POST /v4/games` - Search and fetch game data
|
||||
|
||||
**Query Example:**
|
||||
```
|
||||
search "Zelda Ocarina";
|
||||
fields id, name, slug, cover.url, summary, first_release_date, platforms.name;
|
||||
where platforms = (4);
|
||||
limit 3;
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
**Test the Flow:**
|
||||
1. Enable sync in settings
|
||||
2. Add a game without IGDB match
|
||||
3. Click "Sync Now"
|
||||
4. Wait 1-2 minutes
|
||||
5. Refresh page - should see matches
|
||||
6. Approve or reject matches
|
||||
|
||||
**Test Games:**
|
||||
- "The Legend of Zelda: Ocarina of Time" (N64) - Should find match
|
||||
- "Super Mario 64" (N64) - Should find match
|
||||
- "Made Up Game Name" - Should find no results
|
||||
|
||||
## Support
|
||||
|
||||
**Check Logs:**
|
||||
```bash
|
||||
tail -f log/development.log | grep IGDB
|
||||
```
|
||||
|
||||
**Rails Console:**
|
||||
```ruby
|
||||
# Check sync status
|
||||
User.find_by(email: "demo@turbovault.com").igdb_sync_enabled
|
||||
|
||||
# View pending matches
|
||||
User.first.igdb_match_suggestions.status_pending
|
||||
|
||||
# Test IGDB service
|
||||
service = IgdbService.new
|
||||
results = service.search_game("Mario 64", Platform.find_by(abbreviation: "N64"))
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
The IGDB integration is now fully functional:
|
||||
- ✅ User opt-in settings
|
||||
- ✅ Automatic sync every 30 minutes
|
||||
- ✅ Smart matching with confidence scores
|
||||
- ✅ Review UI with cover art
|
||||
- ✅ Public game cache
|
||||
- ✅ Rate limiting and error handling
|
||||
- ✅ Privacy-preserving design
|
||||
|
||||
Users can now match their games with IGDB for better organization and future features like cover art display!
|
||||
Reference in New Issue
Block a user