Skip to content

Module github.com/bide-ai/bide/anchor/awss3 ​

Go module github.com/bide-ai/bide/anchor/awss3; all modules.

anchor/awss3 ​

github.com/bide-ai/bide/anchor/awss3 · pkg.go.dev · Stability: experimental

Package awss3 is an audit.Anchor that writes every signed tree head as an object in an S3 bucket under Object Lock (a WORM store), and a Reader that reads the heads back and verifies them.

Constants

  • const MinRetention = 24 * time.Hour: MinRetention is the shortest retention New accepts.
  • const ObjectFormat = "bide.audit.s3-anchor.v1": ObjectFormat is the format of an anchored object's body.

Variables

  • var WriterPolicy string: WriterPolicy is the policy template for the application that publishes heads.
  • var ReaderPolicy string: ReaderPolicy is the policy template for an auditor or monitor that only reads heads.
  • var BucketPolicy string: BucketPolicy is the bucket policy template that denies deletes, governance bypass and weaker writes.
  • var ErrUnavailable = errors.New("awss3: anchor unavailable"): ErrUnavailable wraps an error that means S3 could not be reached or did not answer in time (a transport error, a context that ended, a throttle or a server error).
  • var ErrWeak: ErrWeak wraps the error of a stored object whose lock, retention or encryption is weaker than the configuration asked for.
  • var ErrConflict: ErrConflict wraps the error of a key that already holds different bytes than the head being published.
  • var ErrNotFound = errors.New("awss3: no anchored head"): ErrNotFound is returned by the Reader when a partition holds no head.
  • var ErrBadSignature: ErrBadSignature wraps (with ErrObject) the error of an object that is a well-formed head of its partition whose signature does not verify under the log key: a forgery, a head signed by another key (a rotated one, say), or a re-signed one.
  • var ErrEquivocation: ErrEquivocation wraps the error of two validly signed heads of one tree and size that commit to different roots: the log key signed two histories.
  • var ErrObject: ErrObject wraps the error of an object that is not a well-formed anchored head: a key and a body that disagree, a body that is not an anchored object, or a head that does not verify.

Functions

  • func Layout() anchorobj.Layout: Layout is this backend's naming and encoding of anchored heads: its object format and partition tag.
  • func LoadDir(dir string, v audit.Verifier) ([]audit.SignedTreeHead, error): LoadDir reads a downloaded set of anchored objects (for example the output of "aws s3 sync" of the anchor prefix) and verifies each against v.
  • func PolicyFor(template string, p PolicyParams) string: PolicyFor fills a policy template in.

type Anchor ​

go
type Anchor struct {
	// contains filtered or unexported fields
}

Anchor writes signed tree heads to S3 under Object Lock.

Constructors and functions

  • func New(ctx context.Context, cfg Config) (*Anchor, error): New validates the bucket and returns an Anchor.

Methods

  • func (a *Anchor) Publish(ctx context.Context, runID string, sth audit.SignedTreeHead) error: Publish writes sth as an object under Object Lock and reads it back; it returns nil only when the object is stored, locked as configured and equal to sth.

type Client ​

go
type Client interface {
	HeadBucket(ctx context.Context, in *s3.HeadBucketInput, opts ...func(*s3.Options)) (*s3.HeadBucketOutput, error)
	GetBucketVersioning(ctx context.Context, in *s3.GetBucketVersioningInput, opts ...func(*s3.Options)) (*s3.GetBucketVersioningOutput, error)
	GetObjectLockConfiguration(ctx context.Context, in *s3.GetObjectLockConfigurationInput, opts ...func(*s3.Options)) (*s3.GetObjectLockConfigurationOutput, error)
	GetBucketEncryption(ctx context.Context, in *s3.GetBucketEncryptionInput, opts ...func(*s3.Options)) (*s3.GetBucketEncryptionOutput, error)
	PutObject(ctx context.Context, in *s3.PutObjectInput, opts ...func(*s3.Options)) (*s3.PutObjectOutput, error)
	GetObject(ctx context.Context, in *s3.GetObjectInput, opts ...func(*s3.Options)) (*s3.GetObjectOutput, error)
	ListObjectVersions(ctx context.Context, in *s3.ListObjectVersionsInput, opts ...func(*s3.Options)) (*s3.ListObjectVersionsOutput, error)
}

Client is the part of the S3 API this package calls.

type Config ​

go
type Config struct {
	Client Client

	Bucket string

	ExpectedBucketOwner string

	Prefix string

	Mode types.ObjectLockMode

	AllowGovernance bool

	Retention time.Duration

	LegalHold bool

	KMSKeyID string

	EncryptionNone bool

	OnWarn func(error)

	RefuseSuspectIDs bool

	Now func() time.Time
}

Config configures an Anchor.

type Finding ​

go
type Finding struct {
	Key     string
	Version string
	Problem string
}

Finding is something Audit found wrong with the bucket's lock state.

Methods

  • func (f Finding) String() string: String is the finding as one line.

type PolicyParams ​

go
type PolicyParams struct {
	Bucket    string
	Prefix    string
	KMSKeyARN string

	LockMode string

	MinRetentionDays int

	AdminRoleARN string
}

PolicyParams fills in a policy template.

type Reader ​

go
type Reader struct {
	// contains filtered or unexported fields
}

Reader reads anchored heads back from S3 and verifies them.

Constructors and functions

  • func NewReader(cfg ReaderConfig) (*Reader, error): NewReader returns a Reader.

Methods

  • func (r *Reader) Audit(ctx context.Context, kind, runID string, now time.Time) (Report, error): Audit reads every version of every object of a tree and reports their lock state: Heads are the verified heads, and Findings lists every delete marker, every version that is not locked, every version whose retention has ended (it can now be deleted), every governance-mode lock (a principal with s3:BypassGovernanceRetention can delete it), every key with more than one version (the key is never rewritten by this package), every object that is not a verified head (and, when its key claims a size larger than the largest verified head, that too), and two verified heads of one size with different roots.
  • func (r *Reader) Head(ctx context.Context, kind, runID string, size int) ([]audit.SignedTreeHead, error): Head returns the heads anchored for the tree at exactly the given size (more than one only when two processes anchored one run at the same size, with the same root), or ErrNotFound; two with different roots are an error wrapping ErrEquivocation.
  • func (r *Reader) Heads(ctx context.Context, kind, runID string) ([]audit.SignedTreeHead, error): Heads returns every head anchored for the tree, in increasing size (heads of one size in key order), each verified, with the skipping of Latest.
  • func (r *Reader) Latest(ctx context.Context, kind, runID string) (audit.SignedTreeHead, error): Latest returns the head of the largest size anchored for the tree of the given kind (audit.TreeJournal or audit.TreeEvents) and run or trail ID.

type ReaderConfig ​

go
type ReaderConfig struct {
	Client Client

	Bucket string

	ExpectedBucketOwner string

	Prefix string

	KMSKeyID string

	Retention time.Duration

	Mode types.ObjectLockMode

	OnSkip func(Finding)

	Verifier audit.Verifier
}

ReaderConfig configures a Reader.

type Report ​

go
type Report struct {
	Heads    []audit.SignedTreeHead
	Findings []Finding
}

Report is the result of Audit.

anchor/awss3/awss3test ​

github.com/bide-ai/bide/anchor/awss3/awss3test · pkg.go.dev · Stability: experimental

Package awss3test is an in-memory S3 bucket with versioning and Object Lock, for testing code that uses the awss3 package without a network.

Functions

  • func APIError(code string, status int) error: APIError returns an S3-style error with the given code and HTTP status.

type Fake ​

go
type Fake struct {
	Bucket      string
	Owner       string
	Versioning  types.BucketVersioningStatus
	LockEnabled bool
	DefaultMode types.ObjectLockRetentionMode
	DefaultDays int32

	DefaultYears int32
	DefaultKMS   bool

	DefaultKMSKeyID string
	NoEncryptRead   bool

	Clock func() time.Time

	Hook func(op string, n int) error

	AfterCommit func(n int) error

	Tamper func(key string, v *Version)

	Seen map[string][]string
	// contains filtered or unexported fields
}

Fake is a bucket.

Constructors and functions

  • func New(bucket string) *Fake: New returns a bucket that is a correct anchor bucket: owned by account "111122223333", versioned, with Object Lock and a 365-day compliance default retention, and SSE-KMS default encryption.

Methods

  • func (f *Fake) Calls(op string) int: Calls is the number of requests of an operation so far.
  • func (f *Fake) Delete(key, version string, bypassGovernance bool) error: Delete deletes a version as S3 does: a version still under compliance retention, or under governance retention without bypass, or under a legal hold, is refused (AccessDenied).
  • func (f *Fake) GetBucketEncryption(_ context.Context, in *s3.GetBucketEncryptionInput, _ ...func(*s3.Options)) (*s3.GetBucketEncryptionOutput, error): GetBucketEncryption returns the bucket's default encryption, or AccessDenied when NoEncryptRead is set.
  • func (f *Fake) GetBucketVersioning(_ context.Context, in *s3.GetBucketVersioningInput, _ ...func(*s3.Options)) (*s3.GetBucketVersioningOutput, error): GetBucketVersioning returns the bucket's versioning status.
  • func (f *Fake) GetObject(_ context.Context, in *s3.GetObjectInput, _ ...func(*s3.Options)) (*s3.GetObjectOutput, error): GetObject returns a version (the current one without a VersionId) with its lock and encryption metadata.
  • func (f *Fake) GetObjectLockConfiguration(_ context.Context, in *s3.GetObjectLockConfigurationInput, _ ...func(*s3.Options)) (*s3.GetObjectLockConfigurationOutput, error): GetObjectLockConfiguration returns the bucket's Object Lock configuration, or the not-found error S3 returns when Object Lock is not enabled.
  • func (f *Fake) HeadBucket(_ context.Context, in *s3.HeadBucketInput, _ ...func(*s3.Options)) (*s3.HeadBucketOutput, error): HeadBucket checks the bucket and the expected owner.
  • func (f *Fake) Keys() []string: Keys returns every key that has a current (not deleted) version, in order.
  • func (f *Fake) ListObjectVersions(_ context.Context, in *s3.ListObjectVersionsInput, _ ...func(*s3.Options)) (*s3.ListObjectVersionsOutput, error): ListObjectVersions lists versions and delete markers, each key's newest first, in pages of two.
  • func (f *Fake) Put(key string, body []byte): Put stores a body under key as another writer would, with no lock (or the bucket default).
  • func (f *Fake) PutObject(_ context.Context, in *s3.PutObjectInput, _ ...func(*s3.Options)) (*s3.PutObjectOutput, error): PutObject stores a new version, honouring If-None-Match, the lock headers and SSE-KMS, as S3 does.
  • func (f *Fake) Versions(key string) []Version: Versions returns the stored versions of key, oldest first.

type Version ​

go
type Version struct {
	ID            string
	Body          []byte
	Marker        bool
	LockMode      types.ObjectLockMode
	RetainUntil   time.Time
	LegalHold     types.ObjectLockLegalHoldStatus
	SSE           types.ServerSideEncryption
	KMSKey        string
	ChecksumSHA25 string
	Created       time.Time
}

Version is one stored version of a key.

Apache-2.0 licensed.