postgres-local-sync

Helper to download a remote Postgres database into a local development environment, using S3 as intermediate storage, and Heroku for scheduling/orchestration.

Setup

First make sure you have provisioned a suitable S3 bucket without public access, and create an IAM user that can write to the bucket.

Then provision a new Heroku app, run the following, and then deploy this repository to it:

heroku buildpacks:set https://github.com/lfittl/heroku-buildpack-awscli.git
heroku config:set AWS_DEFAULT_REGION=us-east-1
heroku config:set AWS_ACCESS_KEY_ID=...
heroku config:set AWS_SECRET_ACCESS_KEY=...
heroku config:set S3_PATH=s3://my-bucket/staging.dump

# Either set DATABASE_URL directly, or attach an add-on if using Heroku Postgres
heroku config:set DATABASE_URL=postgres://...

Usage

Either use the Heroku scheduler to run the sync task once a day, or run manually before calling sync_url to get the download URL.

Use this as follows to refresh your local development database:

heroku run sync # Optional
curl -o tmp/latest.dump `heroku run --no-notify --no-tty sync_url 2> /dev/null`
pg_restore -l tmp/latest.dump > /dev/null

LICENSE

Copyright (c) 2018, Lukas Fittl lukas@fittl.com postgres-local-sync is licensed under the 3-clause BSD license, see LICENSE file for details.