Skip to content

Commit 3adad24

Browse files
GJFRGJFR
committed
feat: flesh out core architecture and make worker images subject-specific
Core: - Auto-detect image tag from running container via Docker API, removing the need to manually set BUGHOG_VERSION; falls back to env var if detection fails - Group evaluations by subject in Main.run() so each subject gets its own WorkerManager with the correct image, instead of all sharing the first subject's - Fix wait_until_all_evaluations_are_done() to use pool queue size instead of container count, fixing race conditions - Move debugpy import inside the DEVELOPMENT guard; only debug a single worker when max containers is 1 - Forward subject/project/experiment name in plot_data WebSocket update Workers: - Rename image resolution from subject_type to subject_name based lookup - Add MANAGE_CERTS and MANAGE_XVFB env vars for opt-in boot steps; remove Xvfb from core boot (core never needed a display) Infrastructure: - Add docker-bake.hcl as single source of truth for all image builds - Replace scattered CI workflows with unified build-and-push.yml pipeline - Add deploy.sh, build-deploy.sh, remove-images.sh scripts - Add CONTRIBUTING_DOCKER.md documenting the build/release pipeline
1 parent c7fdcb8 commit 3adad24

35 files changed

Lines changed: 1252 additions & 801 deletions

‎.github/workflows/build-and-push.yml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: Build and push Docker images
2+
3+
on:
4+
push:
5+
branches:
6+
- dev
7+
- beta
8+
tags:
9+
- 'v[0-9]+.[0-9]+.[0-9]+'
10+
11+
jobs:
12+
build:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- name: Checkout
16+
uses: actions/checkout@v4
17+
18+
- name: Resolve build metadata
19+
id: meta
20+
run: |
21+
if [[ "$GITHUB_REF" == refs/tags/* ]]; then
22+
echo "version=${GITHUB_REF_NAME#v}" >> $GITHUB_OUTPUT
23+
echo "additional_tag=latest" >> $GITHUB_OUTPUT
24+
elif [[ "$GITHUB_REF_NAME" == "beta" ]]; then
25+
echo "version=beta" >> $GITHUB_OUTPUT
26+
echo "additional_tag=" >> $GITHUB_OUTPUT
27+
else
28+
echo "version=dev" >> $GITHUB_OUTPUT
29+
echo "additional_tag=" >> $GITHUB_OUTPUT
30+
fi
31+
32+
- name: Set up Docker Buildx
33+
uses: docker/setup-buildx-action@v3
34+
35+
- name: Log in to Docker Hub
36+
uses: docker/login-action@v3
37+
with:
38+
username: ${{ secrets.DOCKER_HUB_USERNAME }}
39+
password: ${{ secrets.DOCKER_HUB_TOKEN }}
40+
41+
- name: Build and push
42+
run: docker buildx bake all --push
43+
env:
44+
BUGHOG_VERSION: ${{ steps.meta.outputs.version }}
45+
ADDITIONAL_TAG: ${{ steps.meta.outputs.additional_tag }}

‎.github/workflows/create-release.yml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
name: Create GitHub release
2+
3+
on:
4+
create:
5+
6+
jobs:
7+
package_release:
8+
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
9+
runs-on: ubuntu-latest
10+
steps:
11+
- name: Checkout
12+
uses: actions/checkout@v4
13+
14+
- name: Create Release
15+
id: create_release
16+
uses: actions/create-release@v1
17+
env:
18+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
19+
with:
20+
tag_name: ${{ github.ref }}
21+
release_name: ${{ github.ref }}
22+
draft: true
23+
prerelease: false

‎.github/workflows/trigger-dev-build-job.yml‎

Lines changed: 0 additions & 19 deletions
This file was deleted.

‎.github/workflows/trigger-release-build-job.yml‎

Lines changed: 0 additions & 38 deletions
This file was deleted.

‎.vscode/launch.json‎

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -4,21 +4,6 @@
44
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
55
"version": "0.2.0",
66
"configurations": [
7-
{
8-
"name": "BugHog: Attach Worker",
9-
"type": "debugpy",
10-
"request": "attach",
11-
"connect": {
12-
"host": "localhost",
13-
"port": 5678
14-
},
15-
"pathMappings": [
16-
{
17-
"localRoot": "${workspaceFolder}/bughog",
18-
"remoteRoot": "/app/bughog"
19-
}
20-
]
21-
},
227
{
238
"name": "BugHog",
249
"type": "debugpy",
@@ -35,6 +20,21 @@
3520
"MOZ_DISABLE_CONTENT_SANDBOX": "1",
3621
"PATH":"${PATH}:$HOME/.local/bin"
3722
}
23+
},
24+
{
25+
"name": "BugHog: Attach Worker (Dev Container)",
26+
"type": "debugpy",
27+
"request": "attach",
28+
"connect": {
29+
"host": "bh_worker_0",
30+
"port": 5678
31+
},
32+
"pathMappings": [
33+
{
34+
"localRoot": "${workspaceFolder}/bughog",
35+
"remoteRoot": "/app/bughog"
36+
}
37+
]
3838
}
3939
]
4040
}

‎Dockerfile‎

Lines changed: 19 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -22,60 +22,36 @@ CMD ["start.sh"]
2222

2323
FROM python:3.13-slim-bullseye AS base
2424
COPY --from=ghcr.io/astral-sh/uv:0.9.7 /uv /uvx /bin/
25+
RUN apt-get update
2526
WORKDIR /app
27+
ENV PATH="/app/.venv/bin:$PATH"
2628

27-
RUN apt-get update
28-
RUN apt install -y curl gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libnspr4 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils libgbm-dev xvfb dbus-x11 libnss3-tools python3-tk python3-xlib gnome-screenshot vim git procps &&\
29-
rm -rf /var/lib/apt/lists/*
3029

30+
FROM base AS core
3131
# Install Docker
32-
RUN curl -sSL https://get.docker.com/ | sh
32+
RUN apt install -y \
33+
curl \
34+
docker.io \
35+
git \
36+
procps
3337

34-
# We manually install deprecated libraries for older browser versions.
35-
# Most of them are Debian 11 (buster) packages.
36-
# Stuff needed for chrome versions < 40
37-
RUN curl -sSLo multiarch-support.deb https://snapshot.debian.org/archive/debian-security/20240630T105336Z/pool/updates/main/g/glibc/multiarch-support_2.28-10%2Bdeb10u4_amd64.deb &&\
38-
curl -sSLo libgcrypt11.deb https://snapshot.debian.org/archive/debian/20130820T215153Z/pool/main/libg/libgcrypt11/libgcrypt11_1.5.3-2_amd64.deb &&\
39-
curl -sSLo libudev0.deb https://snapshot.debian.org/archive/debian/20111118T151858Z/pool/main/u/udev/libudev0_175-2_amd64.deb &&\
40-
curl -sSLo libpng12.deb https://snapshot.debian.org/archive/debian/20151118T214328Z/pool/main/libp/libpng/libpng12-0_1.2.54-1_amd64.deb &&\
41-
curl -sSLo pango1.0.deb https://snapshot.debian.org/archive/debian/20200504T084128Z/pool/main/p/pango1.0/libpango-1.0-0_1.42.4-8~deb10u1_amd64.deb &&\
42-
curl -sSLo libpangocairo-1.deb https://snapshot.debian.org/archive/debian/20210326T204420Z/pool/main/p/pango1.0/libpangocairo-1.0-0_1.42.4-8~deb10u1_amd64.deb &&\
43-
curl -sSLo libpangoft2.deb https://snapshot.debian.org/archive/debian/20210326T204420Z/pool/main/p/pango1.0/libpangoft2-1.0-0_1.42.4-8~deb10u1_amd64.deb &&\
44-
curl -sSLo libgtk-3.deb https://snapshot.debian.org/archive/debian/20210326T204420Z/pool/main/g/gtk+3.0/libgtk-3-0_3.24.5-1_amd64.deb &&\
45-
dpkg -i multiarch-support.deb &&\
46-
dpkg -i libgtk-3.deb libpangoft2.deb libpangocairo-1.deb pango1.0.deb libgcrypt11.deb libudev0.deb libpng12.deb &&\
47-
rm multiarch-support.deb libgtk-3.deb libpangoft2.deb libpangocairo-1.deb pango1.0.deb libgcrypt11.deb libudev0.deb libpng12.deb &&\
48-
# Stuff needed for chrome versions < 17
49-
ln -s /usr/lib/x86_64-linux-gnu/libnss3.so /usr/lib/x86_64-linux-gnu/libnss3.so.1d &&\
50-
ln -s /usr/lib/x86_64-linux-gnu/libnssutil3.so /usr/lib/x86_64-linux-gnu/libnssutil3.so.1d &&\
51-
ln -s /usr/lib/x86_64-linux-gnu/libsmime3.so /usr/lib/x86_64-linux-gnu/libsmime3.so.1d &&\
52-
ln -s /usr/lib/x86_64-linux-gnu/libssl3.so /usr/lib/x86_64-linux-gnu/libssl3.so.1d &&\
53-
ln -s /usr/lib/x86_64-linux-gnu/libplds4.so /usr/lib/x86_64-linux-gnu/libplds4.so.0d &&\
54-
ln -s /usr/lib/x86_64-linux-gnu/libplc4.so /usr/lib/x86_64-linux-gnu/libplc4.so.0d &&\
55-
ln -s /usr/lib/x86_64-linux-gnu/libnspr4.so /usr/lib/x86_64-linux-gnu/libnspr4.so.0d
38+
RUN curl -sSLo builx-plugin.deb https://download.docker.com/linux/debian/dists/bullseye/pool/stable/amd64/docker-buildx-plugin_0.31.1-1~debian.11~bullseye_amd64.deb &&\
39+
dpkg -i builx-plugin.deb &&\
40+
rm builx-plugin.deb
5641

57-
COPY subject/web_browser/profiles /app/subject/web_browser/profiles
58-
COPY --chmod=0755 scripts/ /app/scripts/
59-
RUN cp /app/scripts/daemon/xvfb /etc/init.d/xvfb &&\
60-
mkdir -p /app/logs
61-
62-
# Install python packages
63-
COPY pyproject.toml uv.lock /app/
42+
COPY bughog pyproject.toml uv.lock /app/
6443
RUN uv sync --no-dev --locked
65-
ENV PATH="/app/.venv/bin:$PATH"
66-
67-
# Initiate PyAutoGUI
68-
RUN touch /root/.Xauthority && \
69-
xauth add ${HOST}:0 . $(xxd -l 16 -p /dev/urandom)
7044

71-
72-
FROM base AS core
73-
# Copy rest of source code
45+
COPY --chmod=0755 scripts/ /app/scripts/
7446
COPY bughog /app/bughog
7547
ENTRYPOINT [ "/app/scripts/boot/core.sh" ]
7648

7749

7850
FROM base AS worker
79-
# Copy rest of source code
80-
COPY bughog /app/bughog
51+
52+
COPY pyproject.toml uv.lock /app/
53+
RUN uv sync --no-dev --locked
54+
55+
COPY bughog /app/bughog/
56+
COPY --chmod=0755 scripts/ /app/scripts/
8157
ENTRYPOINT [ "/app/scripts/boot/worker.sh" ]

‎README.md‎

Lines changed: 6 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -53,11 +53,8 @@ Since then, it has continued to evolve and has been exhibited at major cybersecu
5353
git clone https://github.com/DistriNet/BugHog
5454
cd BugHog
5555

56-
# Pull the pre-built Docker images from Docker Hub
57-
./scripts/pull.sh
58-
59-
# Start BugHog
60-
./scripts/start.sh
56+
# Pull the latest pre-built images from Docker Hub and start BugHog
57+
./scripts/deploy.sh
6158
```
6259

6360
Open your web browser and navigate to [http://localhost:80](http://localhost:80) to access the graphical user interface.
@@ -72,7 +69,7 @@ If BugHog is started on a remote server, substitute 'localhost' with the appropr
7269
> [!TIP]
7370
> Our [30-minute tutorial](https://github.com/DistriNet/BugHog/wiki/Tutorial) will guide you on how to use BugHog to trace a real bug's lifecycle!
7471
75-
To stop BugHog, simply run this in the project root:
72+
To stop BugHog (including any running worker containers), run this in the project root:
7673

7774
```bash
7875
./scripts/stop.sh
@@ -81,28 +78,17 @@ To stop BugHog, simply run this in the project root:
8178

8279
## Development :hammer_and_wrench:
8380

84-
Use the following commands to build the Docker images yourself, for instance after you made changes to the source code:
81+
Use the following command to build all Docker images from the current codebase and start BugHog, for instance after you made changes to the source code:
8582

8683
```bash
87-
# Build BugHog images
88-
./scripts/build.sh
89-
90-
# Run the freshly built images
91-
./scripts/start.sh
84+
./scripts/build-deploy.sh
9285
```
9386

94-
> [!NOTE]
95-
> For reference, building takes about 4 minutes on a machine with 8 CPU cores and 8 GB of RAM.
87+
This builds all images (core, nginx, and all subject workers) tagged as `dev`.
9688

9789

9890
### Debugging
9991

100-
The most convenient debugging approach is to launch an interactive Node environment.
101-
The UI can be visited at [http://localhost:5173](http://localhost:5173).
102-
103-
```bash
104-
./scripts/node_dev.sh
105-
```
10692

10793
For debugging the core application, consider using the VS Code dev container.
10894
You can utilize the configuration in [.devcontainer](.devcontainer) for this.

0 commit comments

Comments
 (0)