Module github.com/bide-ai/bide/anchor/awss3
Go module github.com/bide-ai/bide/anchor/awss3; all modules.
anchor/awss3
github.com/bide-ai/bide/anchor/awss3 · pkg.go.dev · Stability: experimental
Package awss3 is an audit.Anchor that writes every signed tree head as an object in an S3 bucket under Object Lock (a WORM store), and a Reader that reads the heads back and verifies them.
Constants
const MinRetention = 24 * time.Hour: MinRetention is the shortest retention New accepts.const ObjectFormat = "bide.audit.s3-anchor.v1": ObjectFormat is the format of an anchored object's body.
Variables
var WriterPolicy string: WriterPolicy is the policy template for the application that publishes heads.var ReaderPolicy string: ReaderPolicy is the policy template for an auditor or monitor that only reads heads.var BucketPolicy string: BucketPolicy is the bucket policy template that denies deletes, governance bypass and weaker writes.var ErrUnavailable = errors.New("awss3: anchor unavailable"): ErrUnavailable wraps an error that means S3 could not be reached or did not answer in time (a transport error, a context that ended, a throttle or a server error).var ErrWeak: ErrWeak wraps the error of a stored object whose lock, retention or encryption is weaker than the configuration asked for.var ErrConflict: ErrConflict wraps the error of a key that already holds different bytes than the head being published.var ErrNotFound = errors.New("awss3: no anchored head"): ErrNotFound is returned by the Reader when a partition holds no head.var ErrBadSignature: ErrBadSignature wraps (with ErrObject) the error of an object that is a well-formed head of its partition whose signature does not verify under the log key: a forgery, a head signed by another key (a rotated one, say), or a re-signed one.var ErrEquivocation: ErrEquivocation wraps the error of two validly signed heads of one tree and size that commit to different roots: the log key signed two histories.var ErrObject: ErrObject wraps the error of an object that is not a well-formed anchored head: a key and a body that disagree, a body that is not an anchored object, or a head that does not verify.
Functions
func Layout() anchorobj.Layout: Layout is this backend's naming and encoding of anchored heads: its object format and partition tag.func LoadDir(dir string, v audit.Verifier) ([]audit.SignedTreeHead, error): LoadDir reads a downloaded set of anchored objects (for example the output of "aws s3 sync" of the anchor prefix) and verifies each against v.func PolicyFor(template string, p PolicyParams) string: PolicyFor fills a policy template in.
type Anchor
type Anchor struct {
// contains filtered or unexported fields
}Anchor writes signed tree heads to S3 under Object Lock.
Constructors and functions
func New(ctx context.Context, cfg Config) (*Anchor, error): New validates the bucket and returns an Anchor.
Methods
func (a *Anchor) Publish(ctx context.Context, runID string, sth audit.SignedTreeHead) error: Publish writes sth as an object under Object Lock and reads it back; it returns nil only when the object is stored, locked as configured and equal to sth.
type Client
type Client interface {
HeadBucket(ctx context.Context, in *s3.HeadBucketInput, opts ...func(*s3.Options)) (*s3.HeadBucketOutput, error)
GetBucketVersioning(ctx context.Context, in *s3.GetBucketVersioningInput, opts ...func(*s3.Options)) (*s3.GetBucketVersioningOutput, error)
GetObjectLockConfiguration(ctx context.Context, in *s3.GetObjectLockConfigurationInput, opts ...func(*s3.Options)) (*s3.GetObjectLockConfigurationOutput, error)
GetBucketEncryption(ctx context.Context, in *s3.GetBucketEncryptionInput, opts ...func(*s3.Options)) (*s3.GetBucketEncryptionOutput, error)
PutObject(ctx context.Context, in *s3.PutObjectInput, opts ...func(*s3.Options)) (*s3.PutObjectOutput, error)
GetObject(ctx context.Context, in *s3.GetObjectInput, opts ...func(*s3.Options)) (*s3.GetObjectOutput, error)
ListObjectVersions(ctx context.Context, in *s3.ListObjectVersionsInput, opts ...func(*s3.Options)) (*s3.ListObjectVersionsOutput, error)
}Client is the part of the S3 API this package calls.
type Config
type Config struct {
Client Client
Bucket string
ExpectedBucketOwner string
Prefix string
Mode types.ObjectLockMode
AllowGovernance bool
Retention time.Duration
LegalHold bool
KMSKeyID string
EncryptionNone bool
OnWarn func(error)
RefuseSuspectIDs bool
Now func() time.Time
}Config configures an Anchor.
type Finding
type Finding struct {
Key string
Version string
Problem string
}Finding is something Audit found wrong with the bucket's lock state.
Methods
func (f Finding) String() string: String is the finding as one line.
type PolicyParams
type PolicyParams struct {
Bucket string
Prefix string
KMSKeyARN string
LockMode string
MinRetentionDays int
AdminRoleARN string
}PolicyParams fills in a policy template.
type Reader
type Reader struct {
// contains filtered or unexported fields
}Reader reads anchored heads back from S3 and verifies them.
Constructors and functions
func NewReader(cfg ReaderConfig) (*Reader, error): NewReader returns a Reader.
Methods
func (r *Reader) Audit(ctx context.Context, kind, runID string, now time.Time) (Report, error): Audit reads every version of every object of a tree and reports their lock state: Heads are the verified heads, and Findings lists every delete marker, every version that is not locked, every version whose retention has ended (it can now be deleted), every governance-mode lock (a principal with s3:BypassGovernanceRetention can delete it), every key with more than one version (the key is never rewritten by this package), every object that is not a verified head (and, when its key claims a size larger than the largest verified head, that too), and two verified heads of one size with different roots.func (r *Reader) Head(ctx context.Context, kind, runID string, size int) ([]audit.SignedTreeHead, error): Head returns the heads anchored for the tree at exactly the given size (more than one only when two processes anchored one run at the same size, with the same root), or ErrNotFound; two with different roots are an error wrapping ErrEquivocation.func (r *Reader) Heads(ctx context.Context, kind, runID string) ([]audit.SignedTreeHead, error): Heads returns every head anchored for the tree, in increasing size (heads of one size in key order), each verified, with the skipping of Latest.func (r *Reader) Latest(ctx context.Context, kind, runID string) (audit.SignedTreeHead, error): Latest returns the head of the largest size anchored for the tree of the given kind (audit.TreeJournal or audit.TreeEvents) and run or trail ID.
type ReaderConfig
type ReaderConfig struct {
Client Client
Bucket string
ExpectedBucketOwner string
Prefix string
KMSKeyID string
Retention time.Duration
Mode types.ObjectLockMode
OnSkip func(Finding)
Verifier audit.Verifier
}ReaderConfig configures a Reader.
type Report
type Report struct {
Heads []audit.SignedTreeHead
Findings []Finding
}Report is the result of Audit.
anchor/awss3/awss3test
github.com/bide-ai/bide/anchor/awss3/awss3test · pkg.go.dev · Stability: experimental
Package awss3test is an in-memory S3 bucket with versioning and Object Lock, for testing code that uses the awss3 package without a network.
Functions
func APIError(code string, status int) error: APIError returns an S3-style error with the given code and HTTP status.
type Fake
type Fake struct {
Bucket string
Owner string
Versioning types.BucketVersioningStatus
LockEnabled bool
DefaultMode types.ObjectLockRetentionMode
DefaultDays int32
DefaultYears int32
DefaultKMS bool
DefaultKMSKeyID string
NoEncryptRead bool
Clock func() time.Time
Hook func(op string, n int) error
AfterCommit func(n int) error
Tamper func(key string, v *Version)
Seen map[string][]string
// contains filtered or unexported fields
}Fake is a bucket.
Constructors and functions
func New(bucket string) *Fake: New returns a bucket that is a correct anchor bucket: owned by account "111122223333", versioned, with Object Lock and a 365-day compliance default retention, and SSE-KMS default encryption.
Methods
func (f *Fake) Calls(op string) int: Calls is the number of requests of an operation so far.func (f *Fake) Delete(key, version string, bypassGovernance bool) error: Delete deletes a version as S3 does: a version still under compliance retention, or under governance retention without bypass, or under a legal hold, is refused (AccessDenied).func (f *Fake) GetBucketEncryption(_ context.Context, in *s3.GetBucketEncryptionInput, _ ...func(*s3.Options)) (*s3.GetBucketEncryptionOutput, error): GetBucketEncryption returns the bucket's default encryption, or AccessDenied when NoEncryptRead is set.func (f *Fake) GetBucketVersioning(_ context.Context, in *s3.GetBucketVersioningInput, _ ...func(*s3.Options)) (*s3.GetBucketVersioningOutput, error): GetBucketVersioning returns the bucket's versioning status.func (f *Fake) GetObject(_ context.Context, in *s3.GetObjectInput, _ ...func(*s3.Options)) (*s3.GetObjectOutput, error): GetObject returns a version (the current one without a VersionId) with its lock and encryption metadata.func (f *Fake) GetObjectLockConfiguration(_ context.Context, in *s3.GetObjectLockConfigurationInput, _ ...func(*s3.Options)) (*s3.GetObjectLockConfigurationOutput, error): GetObjectLockConfiguration returns the bucket's Object Lock configuration, or the not-found error S3 returns when Object Lock is not enabled.func (f *Fake) HeadBucket(_ context.Context, in *s3.HeadBucketInput, _ ...func(*s3.Options)) (*s3.HeadBucketOutput, error): HeadBucket checks the bucket and the expected owner.func (f *Fake) Keys() []string: Keys returns every key that has a current (not deleted) version, in order.func (f *Fake) ListObjectVersions(_ context.Context, in *s3.ListObjectVersionsInput, _ ...func(*s3.Options)) (*s3.ListObjectVersionsOutput, error): ListObjectVersions lists versions and delete markers, each key's newest first, in pages of two.func (f *Fake) Put(key string, body []byte): Put stores a body under key as another writer would, with no lock (or the bucket default).func (f *Fake) PutObject(_ context.Context, in *s3.PutObjectInput, _ ...func(*s3.Options)) (*s3.PutObjectOutput, error): PutObject stores a new version, honouring If-None-Match, the lock headers and SSE-KMS, as S3 does.func (f *Fake) Versions(key string) []Version: Versions returns the stored versions of key, oldest first.
type Version
type Version struct {
ID string
Body []byte
Marker bool
LockMode types.ObjectLockMode
RetainUntil time.Time
LegalHold types.ObjectLockLegalHoldStatus
SSE types.ServerSideEncryption
KMSKey string
ChecksumSHA25 string
Created time.Time
}Version is one stored version of a key.