Automation Toolkit

Auto-Fetch Scheduler

Streamline catalog growth by connecting partner feeds to our fetcher pipeline. This blueprint explains the ingestion flow, cron recommendations, and monitoring best practices.

Pipeline overview

Source discovery

Pull from JSON APIs, RSS feeds, or custom partner endpoints. Each source maps to a fetch profile with rate limits and dedupe rules.

Content normalisation

We harmonise titles, categories, and tags against our internal taxonomy before writing to database tables.

Asset handling

Thumbnails are downloaded to /thumbs, while iframe URLs are verified for HTTPS delivery and CORS compatibility.

Automated cron jobs monitoring dashboard

Cron scheduling examples

`/30 * * *`

/usr/bin/php /www/wwwroot/sheworld.best/includes/fetcher.php --source=gameflare --dry-run

Preview new inventory every 30 minutes without writing to the database.

`0 /4 * *`

/usr/bin/php /www/wwwroot/sheworld.best/includes/fetcher.php --source=partners --commit

Ingest approved partner feeds every four hours with full logging.

`15 1 * * *`

/usr/bin/php /www/wwwroot/sheworld.best/admin/analytics_rollup.php --period=daily

Optional companion job to sync engagement metrics with the leaderboard service.

Monitoring & safeguards

Logging & alerts

Fetcher runs write structured JSON logs to storage/logs/fetcher/. Pipe them into your monitoring stack or configure email alerts for failures.

Partner throttling

Respect third-party rate limits by staggering jobs. Use the --limit flag within includes/fetcher.php to control batch sizes.

Quality assurance

Combine auto-fetch with our JSON importer for manual review before publishing to production.

Next steps

Pair auto-fetch with the JSON importer for sandbox verification, then publish remotely hosted titles via the Remote Add portal.

Schedule integration consult