⚽ FootyDraft

FootyDraft is a Telegram‑first bot that uses AI to help you run a high quality, on‑trend football Twitter (X) account. It continuously monitors news sources, fetches live match statistics, generates context‑aware tweet drafts (news, stats, and insights) using a large language model, and delivers them to you via a Telegram bot for manual review, copying, and optional Xquik posting. Over time, it learns from your engagement metrics to refine its writing style – all while keeping you in full control.

✨ Features

Real‑time news ingestion (RSS, Reddit, Google News, API‑Football) - 100% free
- Fast transfer news lane – dedicated high‑frequency polling (every 15 minutes) of six fast‑updating RSS feeds for near‑real‑time transfer gossip
- Post‑match & half‑time stat packs – possession, xG, shots, passes automatically generated for every finished match
- Live in‑game stat snapshots – triggered by the first goal of each half, showing key match stats in real time
- Weekly leaderboards – top scorers & assists for the World Cup and any future league (via football‑data.org, free tier)
- Nerdy stats of the week – automatic detection of unusual match patterns (xG overperformance, possession without result, shot barrages, passing extremes, xG shutouts)
- Strong freshness filter – all news items older than 12 hours are discarded, preventing yesterday’s stories from reappearing
Interactive Streamlit Dashboard – view drafts, performance analytics, source health, rules, and backups in one place
Hybrid tweet drafting with three personas (pundit, fan, analyst)
Three variants per normal event, one per live match event
Telegram bot for reviewing, copying, and tracking drafts
Manual feedback loop - enter tweet engagement metrics without the X API
Optional Xquik posting - publish approved drafts only when XQUIK_POSTING_ENABLED=1
Automatic database backups – compressed backup on every boot and nightly at 3 AM UTC (Dropbox primary, Telegram secondary)
Real‑time error tracking – Sentry integration to monitor crashes and bugs
Built‑in uptime monitor – /uptime command shows how long the bot has been running (WAT)
Gated database restore – /restore command with one‑time code, backup file picker, and automatic staging
Safe remote restart – /restart with two‑step confirmation, no computer needed
Weekly analytics that suggest style improvements based on your best‑performing tweets
Deduplication & age filters – 12‑hour age cutoff for RSS, Google News, and Reddit; content‑hash dedup prevents repeated drafts, with special date‑stamping for weekly recurring drafts
Deployable 24/7 on Fly.io (or any Docker‑compatible platform)

🧠 How It Works

News Fetchers pull headlines from BBC, Sky Sports, ESPN, The Guardian, Daily Mail, Reddit, Google News, and live match data from API‑Football. A dedicated fast lane polls six transfer‑focused RSS feeds every 15 minutes. Weekly leaderboard data is sourced from football‑data.org.
Event Classifier tags each story (goal, transfer, stats, debate, meme, etc.) using keyword rules.
Deduplication Engine ensures the same story doesn’t generate repeated drafts within 24 hours.
Prompt Builder combines your account’s persona, the selected mode (pundit/fan/analyst), active style rules, and your top‑performing tweets as few‑shot examples.
LLM Client (Groq with Mistral fallback) generates tweet drafts.
Telegram Bot shows pending drafts in a queue with inline “Copy” buttons, optional /postx publishing, and instant live‑event drafts.
Analytics Engine runs weekly, identifies patterns in your engagement data, and suggests natural‑language rules you can approve or reject.
Stats Engine – after every full‑time whistle, match statistics are stored in the database. A weekly job analyses the accumulated data and creates a ‘Nerdy Stats of the Week’ draft highlighting unusual patterns.
Leaderboard Engine – twice a week, top scorers and assist leaders are fetched from football‑data.org and turned into a combined draft. Inactive leagues are automatically skipped.
Backup Engine – automatically creates a gzipped database backup on every machine restart and daily at 3 AM UTC, uploading to Dropbox (primary) and Telegram (secondary).
Monitoring – the /uptime command shows the bot’s start time and elapsed uptime; Sentry integration tracks errors in real time.
Recovery – the /restore command uses a gated one‑time code to list the last 5 backups from Dropbox, then stages the selected backup; a /restart reboot applies it automatically.

🏗 Architecture

footydraft/
├── config/
│   ├── settings.py          # Environment variables & constants
│   └── personas.yaml        # Persona definitions (pundit, fan, analyst)
├── core/                    # Engine (UI‑agnostic)
│   ├── ingestion/           # News fetchers
│   │   ├── base.py          # Abstract BaseFetcher + NewsItem
│   │   ├── rss_fetcher.py   # BBC, Sky, ESPN, Guardian, etc.
│   │   ├── reddit_fetcher.py
│   │   ├── google_news_fetcher.py
│   │   ├── api_football_fetcher.py
│   │   ├── espn_fetcher.py  # ESPN hidden API (CL, Europa, cups)
│   │   └── monitor.py       # Source health monitoring
│   ├── classification/
│   │   ├── event_tagger.py  # Keyword‑based tagger
│   │   └── dedup.py         # Content‑hash deduplication
│   ├── generation/
│   │   ├── prompt_builder.py
│   │   ├── llm_client.py    # Groq (primary), Mistral (fallback)
│   │   └── queue_manager.py # Draft creation & daily caps
│   ├── publishing/          # Optional Xquik posting
│   │   ├── __init__.py
│   │   └── xquik.py
│   ├── analytics/
│   │   └── engine.py        # Weekly rule suggestions
│   ├── backup.py            # Dropbox & Telegram backup (gzipped)
│   ├── database.py          # SQLAlchemy init & session
│   ├── models.py            # ORM models (Draft, Tweet, Rule, etc.)
│   └── scheduler.py         # Main loop
├── bot/
│   ├── handlers.py          # All command handlers
│   ├── keyboard.py          # Inline keyboards (Copy, pagination)
│   └── main.py              # Bot entry point + live‑draft push
├── dashboard/               # Streamlit dashboard
│   ├── app.py               # Main dashboard entry point
│   ├── ui_components.py     # Shared UI (CSS, sidebar, navigation)
│   ├── utils.py             # Shared utilities (Dropbox, database helpers)
│   ├── requirements.txt     # Dashboard dependencies
│   ├── Dockerfile.dashboard # Dockerfile for dashboard service
│   ├── static/              # Static assets (favicon, etc.)
│   └── pages/               # Dashboard pages
│       ├── Backup_Browser.py
│       ├── Database_Stats.py
│       ├── Drafts.py
│       ├── Live_Check.py
│       ├── Performance.py
│       ├── Rule_Manager.py
│       └── Source_Health.py
├── data/                    # SQLite database (auto‑created)
├── scripts/                 # Diagnostic & utility scripts
│   ├── diagnostic_tags.py   # Check event classifier tag distribution
│   ├── diagnostic.py        # Check API‑Football live data & classification
│   └── direct_api_check.py  # Compare live=all vs date endpoints
├── tests/                   # Unit & integration tests
├── .env.example             # Environment template
├── Dockerfile               # Containerised deployment
├── fly.toml                 # Fly.io configuration
├── requirements.txt
├── start.sh                 # Startup script (scheduler + bot)
└── README.md

🚀 Quick Start

Prerequisites

Python 3.11+
Groq API key
Telegram bot token (from @BotFather)
API‑Football key for live match data

1. Clone & set up environment

git clone https://github.com/MikkyPrestige/footydraft.git
cd footydraft
python -m venv venv
source venv/bin/activate   # or venv\Scripts\activate on Windows
pip install -r requirements.txt
cp .env.example .env
# Now edit .env with your real API keys

2. Initialise the database

python -c "from core.database import init_db; init_db()"

3. Run the scheduler & bot locally

Terminal 1 – Core scheduler (fetches news, generates drafts):

python -m core.scheduler

Terminal 2 – Telegram bot:

python -m bot.main

4. Open Telegram & try commands

/start – welcome message and command list
/queue – view pending drafts (all, normal, or live) with Copy buttons & pagination
/queue normal – only normal pending drafts
/queue live – only live-match drafts
/drafts – browse all drafts (all, pending, held, posted) with pagination and copy buttons
/drafts pending – only pending normal drafts
/drafts held – only held drafts
drafts posted – only posted drafts
/leaderboard – fetch latest top scorers & assists leaderboard
/nerdystats – generate nerdy stats of the week draft
/hold <draft_id> – quarantine a pending draft
/release <draft_id> – return a held draft to the queue
/posted <draft_id> – mark a draft as manually posted & link a tweet
/postx <draft_id> [variant] – post an approved draft via Xquik (requires XQUIK_POSTING_ENABLED=1)
/metrics <tweet_ref> <likes> <retweets> <replies> <impressions> – enter engagement numbers
/stats – top & bottom tweets by likes (default)
/impressions – top & bottom tweets by impressions
/tweets – list all posted tweets with refs
/rules – manage style rules (accept/reject auto‑suggestions)
/addrule <text> – add a manual style rule
/source_status – check health of news sources
/livecheck – force check for live matches
/clearqueue – delete all pending drafts (or only normal / live)
/clearqueue normal – delete only normal pending drafts
/clearqueue live – delete only live pending drafts
/uptime – bot uptime and start time (WAT)
/restore – gated database restore from Dropbox backups
/restart – Restart the bot (two-step confirmation)

5. (Optional) Run the Streamlit Dashboard

# Install dashboard dependencies
pip install -r dashboard/requirements.txt

# Run the dashboard
streamlit run dashboard/app.py

The dashboard will be available at http://localhost:8501

6. (Optional) Run both bot and dashboard with Docker Compose

Start both services

docker compose up -d

Start only the dashboard

docker compose up -d dashboard

Start only the bot

docker compose up -d bot

The dashboard will be available at http://localhost:8501

☁️ Deploying to Fly.io (24/7)

The project includes a Dockerfile and fly.toml for one‑click deployment to Fly.io. It runs both the scheduler and the Telegram bot in a single small VM (~$1.94/month).

Install the Fly CLI: curl -L https://fly.io/install.sh | sh
Log in: flyctl auth login

Create the app & volume (run inside the project folder):

flyctl apps create footydraft --yes
flyctl volumes create agent_data --region iad --size 1 -a footydraft

Set secrets (replace with your values): flyctl secrets set OPENAI_API_KEY="sk-..." TELEGRAM_BOT_TOKEN="123:abc" ... -a footydraft
Deploy: flyctl deploy -a footydraft
Check logs: flyctl logs -a footydraft

Your bot will now run 24/7 without needing your computer.

📊 Deploying the Dashboard to Streamlit Cloud (24/7)

The dashboard can be deployed to Streamlit Cloud for 24/7 access.

Push your code to GitHub.
Go to Streamlit Cloud and sign in with GitHub.
Click "New app" and select your repository.
Set the main file path to: dashboard/app.py
In Settings → Secrets, add the following environment variables:

DROPBOX_APP_KEY = "your_key"
DROPBOX_APP_SECRET = "your_secret"
DROPBOX_REFRESH_TOKEN = "your_token"
TELEGRAM_BOT_TOKEN = "your_token"
ADMIN_CHAT_ID = "your_chat_id"

# Required for Xquik toggle (via Fly.io CLI only)
XQUIK_POSTING_ENABLED = "1"

# Required if using Xquik toggle (optional)
FLY_API_TOKEN = "your_token"
FLY_APP_NAME = "footydraft"
FLY_MACHINE_ID = "your_machine_id"

Click Deploy.

The dashboard is password‑protected. On first visit you’ll see a login screen. After entering the correct password (set via PASSWORD secret in Streamlit Cloud), the session persists across page refreshes.

Use this command to add the dashboard password to your Fly.io secrets:

flyctl secrets set PASSWORD=your‑chosen‑password -a footydraft

The dashboard will be available at https://your-app-name.streamlit.app

⚙️ Configuration

All important settings are in config/settings.py and can be overridden with environment variables:

Variable	Default	Description
`OPENAI_API_KEY`	–	API key (Groq or any OpenAI‑compatible provider)
`TELEGRAM_BOT_TOKEN`	–	Telegram bot token
`GROQ_API_KEY`	–	Groq API key (primary)
`MISTRAL_API_KEY`	–	Mistral API key (fallback when Groq is rate‑limited)
`API_FOOTBALL_KEY`	–	API‑Football key
`FOOTBALL_DATA_KEY`	–	API key for football‑data.org – required for weekly leaderboards
`DATABASE_URL`	`sqlite:///data/agent.db`	DB connection string
`XQUIK_POSTING_ENABLED`	`0`	Set to `1` to enable `/postx` publishing
`XQUIK_API_KEY`	–	Xquik API key used by `/postx`
`XQUIK_API_BASE_URL`	`https://xquik.com`	Override for custom Xquik deployments
`NORMAL_DAILY_CAP`	5	Max normal tweets posted per day
`LIVE_MATCH_HOURLY_CAP`	10	Max live tweets per match hour
`MAX_ITEM_AGE_HOURS`	12	Discard news older than this
`DROPBOX_APP_KEY`	–	Dropbox app key for automatic backups
`DROPBOX_APP_SECRET`	–	Dropbox app secret
`DROPBOX_REFRESH_TOKEN`	–	Dropbox refresh token (never expires)
`SENTRY_DSN`	–	Sentry DSN for error tracking
`FLY_API_TOKEN`	–	Fly.io org token (required for `/restart`)
`STREAMLIT_SERVER_PORT`	`8501`	Port for Streamlit dashboard
`STREAMLIT_SERVER_ADDRESS`	`0.0.0.0`	Address for Streamlit dashboard

🔐 Xquik Posting Control

Xquik posting allows the bot to automatically post tweets to X (Twitter). This is controlled via Fly.io environment variables.

Enable Xquik

flyctl secrets set XQUIK_POSTING_ENABLED=1 -a footydraft
flyctl machines restart <machine_id> -a footydraft

Disable Xquik

flyctl secrets set XQUIK_POSTING_ENABLED=0 -a footydraft
flyctl machines restart <machine_id> -a footydraft

Get machine ID

flyctl machines list -a footydraft

The /postx command in the Telegram bot will only work when XQUIK_POSTING_ENABLED=1

[!NOTE] The Xquik toggle has been removed from the Streamlit dashboard due to persistence issues on Streamlit Cloud. The backend functionality remains fully available via the Fly.io CLI.

🧪 Testing

# Run individual test scripts
python -m tests.test_rss_fetcher
python -m tests.test_llm_client
python -m tests.test_queue_manager
python -m tests.test_personas
python -m tests.test_prompt_builder
## more in the tests/ folder

🗺 Roadmap / Future

Optional scheduled auto‑posting with kill‑switch
Multi‑language support

👐 Contributing

Contributions are welcome! This is a personal project, but if you have ideas for improvements or new fetchers, feel free to open a PR or issue. Please keep the core UI‑agnostic principle in mind.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

⚽ FootyDraft

✨ Features

🧠 How It Works

🏗 Architecture

🚀 Quick Start

Prerequisites

1. Clone & set up environment

2. Initialise the database

3. Run the scheduler & bot locally

4. Open Telegram & try commands

5. (Optional) Run the Streamlit Dashboard

6. (Optional) Run both bot and dashboard with Docker Compose

☁️ Deploying to Fly.io (24/7)

📊 Deploying the Dashboard to Streamlit Cloud (24/7)

⚙️ Configuration

🔐 Xquik Posting Control

🧪 Testing

🗺 Roadmap / Future

👐 Contributing

About

Uh oh!

Releases

Packages

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 170 Commits
.devcontainer		.devcontainer
.streamlit		.streamlit
.vscode		.vscode
bot		bot
config		config
core		core
dashboard		dashboard
scripts		scripts
tests		tests
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
docker-compose.yml		docker-compose.yml
fly.toml		fly.toml
requirements.txt		requirements.txt
start.sh		start.sh

Folders and files

Latest commit

History

Repository files navigation

⚽ FootyDraft

✨ Features

🧠 How It Works

🏗 Architecture

🚀 Quick Start

Prerequisites

1. Clone & set up environment

2. Initialise the database

3. Run the scheduler & bot locally

4. Open Telegram & try commands

5. (Optional) Run the Streamlit Dashboard

6. (Optional) Run both bot and dashboard with Docker Compose

☁️ Deploying to Fly.io (24/7)

📊 Deploying the Dashboard to Streamlit Cloud (24/7)

⚙️ Configuration

🔐 Xquik Posting Control

🧪 Testing

🗺 Roadmap / Future

👐 Contributing

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages