bagit-gython

bagit-gython is an experimental library that wraps bagit-python using an embedded modern Python interpreter. Its goal is to make Python's battle-tested implementation available to the Go ecosystem, avoiding the need to reimplement it from scratch. By leveraging Python's well-tested version, bagit-gython benefits from the extensive community of users and the robustness of the existing implementation.

It depends on go-embed-python, a CGO-free library that provides an embedded distribution of Python compatible with a number of architecture and operative systems.

Installing

Using bagit-python is easy. First, use go get to install the latest version of the library.

go get -u github.com/artefactual-labs/bagit-gython

Next, include bagit-gython in your application:

import "github.com/artefactual-labs/bagit-gython"

Usage

For long-running applications, create one Validator at process startup and reuse it:

validator, err := bagit.NewValidator(bagit.WithPoolSize(4))
if err != nil {
    return err
}
defer func() {
    if err := validator.Close(); err != nil {
        // Handle cleanup error.
    }
}()

if err := validator.Validate("/tmp/valid-bag"); err != nil {
    return err
}

Validator owns one shared embedded Python extraction and a bounded pool of BagIt runners. At most poolSize validations run at once; additional calls wait for a runner instead of creating new Python extractions. By default, the extracted runtime is cached under the user's cache directory in bagit-gython. With WithPoolSize(4), a process uses one runtime cache root and up to four runner processes for that validator lifecycle.

Runtime cache configuration is explicit:

Configuration	Result
Omit WithCacheDir	Use the default persistent cache, os.UserCacheDir()/bagit-gython.
WithCacheDir("")	Use the default persistent cache, os.UserCacheDir()/bagit-gython.
WithCacheDir("/path/to/cache")	Use that persistent cache directory.
WithTempCacheDir()	Disable the persistent cache and use a temporary runtime root that Close removes.
WithDeferredRuntime()	Delay runtime extraction and runner pool creation until the first validation request.

If cache configuration comes from a config file, treat the field as optional: when it is unset or empty, omit WithCacheDir or pass WithCacheDir("") to use the default cache; when it is set to a non-empty path, pass that path. Use WithTempCacheDir() only when the operator explicitly wants a temporary runtime root. Cached runtime files are content-hash scoped, so changes to the embedded Python, bagit-python, or runner files extract into new cache directories while warm starts can reuse existing files.

By default, NewValidator extracts the embedded runtime and creates the runner pool before returning. Use WithDeferredRuntime() when startup should avoid that work until the validator is actually used. The first validation request performs setup synchronously; concurrent first requests wait behind that setup and then share the initialized pool. TryValidate can also pay this setup cost after it acquires a runner slot.

Use ValidateContext when waiting for an available runner should respect caller cancellation or deadlines. Use TryValidate when the caller should get ErrBusy immediately instead of waiting for a runner.

This is the preferred API for worker processes and Temporal activities. For example, a Temporal worker can create the validator during startup, pass it to an activity that accepts a Validate(path string) error interface, and close it during worker shutdown:

validator, err := bagit.NewValidator(bagit.WithPoolSize(4))
if err != nil {
    return err
}
defer func() {
    if err := validator.Close(); err != nil {
        // Handle cleanup error.
    }
}()

tw.RegisterActivityWithOptions(
    bagvalidate.New(validator).Execute,
    activity.RegisterOptions{Name: bagvalidate.Name},
)

BagIt is still available as a lower-level single-runner API, but it is not safe for concurrent operations. Prefer Validator unless you are deliberately managing one BagIt instance per caller.

The example directory contains a small command-line program that validates a bag with either API:

$ cd example
$ go run . -api validator -validate ../internal/testdata/valid-bag -pool-size 2
Valid!

$ go run . -api validator -validate ../internal/testdata/valid-bag -cache-dir /tmp/bagit-cache
Valid!

$ go run . -api bagit -validate ../internal/testdata/valid-bag
Valid!

Supported architectures

• darwin-amd64
• darwin-arm64
• linux-amd64
• linux-arm64
• windows-amd64

These are the platform-architecture combinations for which go-embed-python provides compatibility.

Version of bagit-python

The specific version of bagit-python used by this project is specified in the internal/dist/requirements.txt file. Instead of using the latest official release, we are using a commit from the main branch that includes compatibility fixes for recent Python releases. This commit has not yet been included in an official release.

Acknowledgement

• bagit-python project: https://github.com/LibraryOfCongress/bagit-python
• go-embed-python project: https://github.com/kluctl/go-embed-python

License

Apache 2.0. See LICENSE.