Module github.com/bide-ai/bide/anchor/gcs
Go module github.com/bide-ai/bide/anchor/gcs; all modules.
anchor/gcs
github.com/bide-ai/bide/anchor/gcs · pkg.go.dev · Stability: experimental
Package gcs is an audit.Anchor that writes every signed tree head as an object in a Google Cloud Storage bucket whose retention policy is locked (a WORM store), and a Reader that reads the heads back and verifies them.
Constants
const DefaultPrefix = "bide-anchor/": DefaultPrefix is the object name prefix used when Config.Prefix is empty.const Locked = "Locked": Locked is the per-object retention mode that cannot be shortened or removed.const MinRetention = 24 * time.Hour: MinRetention is the shortest retention New accepts.const ObjectFormat = "bide.audit.gcs-anchor.v1": ObjectFormat is the format of an anchored object's body.
Variables
var WriterRole string: WriterRole is the role for the application that publishes heads.var WriterObjectRetentionRole string: WriterObjectRetentionRole is WriterRole plus storage.objects.setRetention, which a create that sets a per-object retention needs (Config.ObjectRetention); verified against a real bucket, where the plain writer role was refused with a 403 naming that permission.var ReaderRole string: ReaderRole is the role for an auditor or monitor that only reads heads.var HoldReleaserRole string: HoldReleaserRole is the role for the identity that releases temporary holds.var ErrUnavailable = errors.New("gcs: anchor unavailable"): ErrUnavailable wraps an error that means Cloud Storage could not be reached or did not answer in time (a transport error, a context that ended, a throttle or a server error).var ErrWeak: ErrWeak wraps the error of a stored object whose lock, retention or encryption is weaker than the configuration asked for.var ErrConflict: ErrConflict wraps the error of a name that already holds different bytes than the head being published.var ErrNotFound = errors.New("gcs: no anchored head"): ErrNotFound is returned by the Reader when a partition holds no head.var ErrVanished: ErrVanished is returned by the Reader when a generation that was listed cannot be read: it was deleted between the listing and the read (at the end of its retention) or is hidden.var ErrWrongProject: ErrWrongProject wraps (with agent.ErrConfig) the error of a bucket that another project owns than the one expected: a bucket name that was deleted and created again by someone else, say.var ErrBadSignature: ErrBadSignature wraps (with ErrObject) the error of an object that is a well-formed head of its partition whose signature does not verify under the log key: a forgery, a head signed by another key (a rotated one, say), or a re-signed one.var ErrEquivocation: ErrEquivocation wraps the error of two validly signed heads of one tree and size that commit to different roots: the log key signed two histories.var ErrObject: ErrObject wraps the error of an object that is not a well-formed anchored head: a key and a body that disagree, a body that is not an anchored object, or a head that does not verify.var ErrExists = errors.New("gcs: object exists"): ErrExists is Create's answer to a name that holds a live object (HTTP 412).var ErrNoObject = errors.New("gcs: no such object"): ErrNoObject is the answer to a name or generation that does not exist (HTTP 404).var ErrDenied = errors.New("gcs: permission denied"): ErrDenied is the answer to a call the caller's identity may not make (HTTP 401 or 403).var MethodPermissions: MethodPermissions maps each Store method to the IAM permissions it needs.var ObjectRetentionPermissions = []string{"storage.objects.setRetention"}: ObjectRetentionPermissions are the permissions a Create that sets a per-object retention needs in addition to MethodPermissions["Create"].
Functions
func Layout() anchorobj.Layout: Layout returns this backend's layout: its object format, partition tag and error values.func LoadDir(dir string, v audit.Verifier) ([]audit.SignedTreeHead, error): LoadDir reads a downloaded set of anchored objects (for example the output of "gcloud storage cp -r" of the anchor prefix) and verifies each against v.func RolePermissions(role string) ([]string, error): RolePermissions returns the permissions a role document (WriterRole and the others) grants.
type Anchor
type Anchor struct {
// contains filtered or unexported fields
}Anchor writes signed tree heads to Cloud Storage under a locked retention policy.
Constructors and functions
func New(ctx context.Context, cfg Config) (*Anchor, error): New validates the bucket and returns an Anchor.
Methods
func (a *Anchor) Publish(ctx context.Context, runID string, sth audit.SignedTreeHead) error: Publish writes sth as an object and reads it back; it returns nil only when the object is stored, retained as configured, encrypted with the configured key and equal to sth.
type BucketInfo
type BucketInfo struct {
ProjectNumber uint64
RetentionPeriod time.Duration
RetentionLocked bool
Versioning bool
DefaultKMSKey string
UniformAccess bool
PublicAccessPrevented bool
ObjectRetention bool
SoftDeleteDuration time.Duration
}BucketInfo is a bucket's configuration, in the terms the anchor checks.
type Config
type Config struct {
Store Store
ExpectedProjectNumber uint64
Prefix string
Retention time.Duration
ObjectRetention bool
LegalHold bool
KMSKey string
EncryptionNone bool
OnWarn func(error)
RefuseSuspectIDs bool
Now func() time.Time
}Config configures an Anchor.
type CreateOptions
type CreateOptions struct {
KMSKey string
TemporaryHold bool
Retention *ObjectRetention
}CreateOptions are the attributes Create writes with an object.
type Finding
type Finding struct {
Key string
Version string
Problem string
}Finding is something Audit found wrong with the bucket's lock state.
Methods
func (f Finding) String() string: String is the finding as one line.
type ObjectInfo
type ObjectInfo struct {
Name string
Generation int64
Size int64
Created time.Time
Noncurrent time.Time
CRC32C uint32
KMSKey string
RetentionExpiry time.Time
Retention *ObjectRetention
TemporaryHold bool
EventHold bool
}ObjectInfo is what Cloud Storage reports about one generation of an object.
type ObjectRetention
type ObjectRetention struct {
Mode string
RetainUntil time.Time
}ObjectRetention is a per-object retention configuration.
type Reader
type Reader struct {
// contains filtered or unexported fields
}Reader reads anchored heads back from Cloud Storage and verifies them.
Constructors and functions
func NewReader(ctx context.Context, cfg ReaderConfig) (*Reader, error): NewReader returns a Reader.
Methods
func (r *Reader) Audit(ctx context.Context, kind, runID string, now time.Time) (Report, error): Audit reads every generation of every object of a tree and reports its retention state: Heads are the verified heads, and Findings lists every noncurrent generation (something overwrote or deleted the live object), every generation no retention covers (it can be deleted), every generation whose retention has ended (it can now be deleted), every per-object retention that is not Locked, every object not encrypted with the configured key (when ReaderConfig.KMSKey is set), every name with more than one generation (the name is never rewritten by this package), every object that is not a verified head (and, when its name claims a size larger than the largest verified head, that too), and two verified heads of one size with different roots, and each bucket-level protection that is gone (the retention policy absent, unlocked or under the minimum; versioning, uniform bucket-level access or public access prevention off; with ReaderConfig.KMSKey, another default key).func (r *Reader) Head(ctx context.Context, kind, runID string, size int) ([]audit.SignedTreeHead, error): Head returns the heads anchored for the tree at exactly the given size (more than one only when two processes anchored one run at the same size, with the same root), or ErrNotFound; two with different roots are an error wrapping ErrEquivocation.func (r *Reader) Heads(ctx context.Context, kind, runID string) ([]audit.SignedTreeHead, error): Heads returns every head anchored for the tree, in increasing size (heads of one size in name order), each verified, with the skipping of Latest and its ErrVanished.func (r *Reader) Latest(ctx context.Context, kind, runID string) (audit.SignedTreeHead, error): Latest returns the head of the largest size anchored for the tree of the given kind (audit.TreeJournal or audit.TreeEvents) and run or trail ID.
type ReaderConfig
type ReaderConfig struct {
Store Store
ExpectedProjectNumber uint64
Prefix string
KMSKey string
Retention time.Duration
OnSkip func(Finding)
Verifier audit.Verifier
}ReaderConfig configures a Reader.
type Report
type Report struct {
Heads []audit.SignedTreeHead
Findings []Finding
}Report is the result of Audit.
type Store
type Store interface {
Bucket(ctx context.Context) (BucketInfo, error)
Create(ctx context.Context, name string, body []byte, o CreateOptions) (ObjectInfo, error)
Stat(ctx context.Context, name string, generation int64) (ObjectInfo, error)
Read(ctx context.Context, name string, generation int64, limit int64) ([]byte, ObjectInfo, error)
List(ctx context.Context, prefix string, versions bool, fn func(ObjectInfo) bool) error
}Store is the part of Cloud Storage this package uses, for one bucket.
Constructors and functions
func NewStore(c *storage.Client, bucket string) Store: NewStore returns the Store for a bucket of a Cloud Storage client.
anchor/gcs/gcstest
github.com/bide-ai/bide/anchor/gcs/gcstest · pkg.go.dev · Stability: experimental
Package gcstest is an in-memory gcs.Store that models the parts of Cloud Storage the anchor relies on: object generations, create-only writes (ifGenerationMatch=0), a bucket retention policy that can be locked, noncurrent generations under versioning, temporary holds, per-object retention and Cloud KMS keys.
Constants
const DefaultKMSKey: DefaultKMSKey is the default Cloud KMS key of a new Fake.const ProjectNumber = 123456789012: ProjectNumber is the project number of a new Fake's bucket.
Variables
var ErrRetained: ErrRetained is what a delete of a retained or held generation returns.
type Fake
type Fake struct {
// contains filtered or unexported fields
}Fake is the in-memory bucket.
Constructors and functions
func New() *Fake: New returns a bucket that is what the anchor needs: a locked 30-day retention policy, versioning, uniform access, enforced public access prevention and DefaultKMSKey as its default key.
Methods
func (f *Fake) Advance(d time.Duration): Advance moves the clock forward.func (f *Fake) AlterNextCreate(fn func(*gcs.ObjectInfo)): AlterNextCreate changes the attributes of the generation the next Create writes, as a bucket that stores an object under other terms than were asked for would.func (f *Fake) Bucket(context.Context) (gcs.BucketInfo, error): Bucket implements gcs.Store.func (f *Fake) Calls() []string: Calls returns the Store methods called so far, in order.func (f *Fake) Configure(fn func(*gcs.BucketInfo)): Configure changes the bucket's configuration (to make a bucket the anchor must refuse).func (f *Fake) Create(_ context.Context, name string, body []byte, o gcs.CreateOptions) (gcs.ObjectInfo, error): Create implements gcs.Store.func (f *Fake) DeleteGeneration(name string, generation int64) error: DeleteGeneration permanently deletes one generation.func (f *Fake) DeleteLive(name string) error: DeleteLive deletes the live generation of name.func (f *Fake) FailNext(method string, err error, times int): FailNext makes the next times calls of method (Bucket, Create, Stat, Read or List) return err without running.func (f *Fake) Generations(name string) []gcs.ObjectInfo: Generations returns the infos of every generation of name, oldest first.func (f *Fake) Inject(info gcs.ObjectInfo, body []byte) gcs.ObjectInfo: Inject adds a generation exactly as given (a foreign writer's object: an unlocked one, a stray one, one under another key) and returns it.func (f *Fake) List(_ context.Context, prefix string, versions bool, fn func(gcs.ObjectInfo) bool) error: List implements gcs.Store.func (f *Fake) LoseResponseNext(method string, err error): LoseResponseNext makes the next call of method run and then return err: the effect happened and the caller does not know.func (f *Fake) Names() []string: Names returns every object name, sorted.func (f *Fake) Now() time.Time: Now is the fake's clock.func (f *Fake) Overwrite(name string, body []byte): Overwrite replaces the live object with other bytes (a new generation; the old one becomes noncurrent), as a principal with storage.objects.create and storage.objects.delete can.func (f *Fake) Read(_ context.Context, name string, generation, limit int64) ([]byte, gcs.ObjectInfo, error): Read implements gcs.Store.func (f *Fake) ReleaseHold(name string, generation int64): ReleaseHold releases the temporary hold on a generation (what the separate hold-release identity does).func (f *Fake) Report(name string, generation int64, fn func(*gcs.ObjectInfo)): Report changes what Cloud Storage reports for a generation (the live one when generation is 0), as a server that answers inconsistently would.func (f *Fake) Stat(_ context.Context, name string, generation int64) (gcs.ObjectInfo, error): Stat implements gcs.Store.func (f *Fake) Tamper(name string, generation int64, body []byte): Tamper rewrites the bytes of a generation in place, which Cloud Storage does not allow; it models damage or a forged listing that the Reader must catch by checking the bytes.func (f *Fake) VanishOnRead(name string): VanishOnRead makes every read of name fail with ErrNoObject while its listing still shows it, as an object deleted at the end of its retention between a listing and a read.