Upgrade versions of vendored libs

vendor/github.com/cockroachdb/pebble/internal/base/lazy_value.go

@@ -0,0 +1,256 @@
+// Copyright 2022 The LevelDB-Go and Pebble Authors. All rights reserved. Use
+// of this source code is governed by a BSD-style license that can be found in
+// the LICENSE file.
+
+package base
+
+import "github.com/cockroachdb/pebble/internal/invariants"
+
+// A value can have user-defined attributes that are a function of the value
+// byte slice. For now, we only support "short attributes", which can be
+// encoded in 3 bits. We will likely extend this to "long attributes" later
+// for values that are even more expensive to access than those in value
+// blocks in the same sstable.
+//
+// When a sstable writer chooses not to store a value together with the key,
+// it can call the ShortAttributeExtractor to extract the attribute and store
+// it together with the key. This allows for cheap retrieval of
+// AttributeAndLen on the read-path, without doing a more expensive retrieval
+// of the value.
+//
+// In general, the extraction code may want to also look at the key to decide
+// how to treat the value. Our current needs can be satisfied by configuring a
+// keyspan for which the ShortAttributeExtractor should be used, so we do not
+// expose the key to the extractor.
+//
+// Write path performance: The ShortAttributeExtractor func cannot be inlined,
+// so we will pay the cost of this function call. However, we will only pay
+// this when (a) the value is not being stored together with the key, and (b)
+// the key-value pair is being initially written to the DB, or a compaction is
+// transitioning the key-value pair from being stored together to being stored
+// separately.
+
+// ShortAttribute encodes a user-specified attribute of the value.
+type ShortAttribute uint8
+
+// MaxShortAttribute is the maximum value of the short attribute (3 bits).
+const MaxShortAttribute = 7
+
+// ShortAttributeExtractor is an extractor that given the value, will return
+// the ShortAttribute.
+type ShortAttributeExtractor func(value []byte) (ShortAttribute, error)
+
+// AttributeAndLen represents the pair of value length and the short
+// attribute.
+type AttributeAndLen struct {
+	ValueLen       int32
+	ShortAttribute ShortAttribute
+}
+
+// LazyValue represents a value that may not already have been extracted.
+// Currently, it can represent either an in-place value (stored with the key)
+// or a value stored in the value section. However, the interface is general
+// enough to support values that are stored in separate files.
+//
+// LazyValue is used in the InternalIterator interface, such that all
+// positioning calls return (*InternalKey, LazyValue). It is also exposed via
+// the public Iterator for callers that need to remember a recent but not
+// necessarily latest LazyValue, in case they need the actual value in the
+// future. An example is a caller that is iterating in reverse and looking for
+// the latest MVCC version for a key -- it cannot identify the latest MVCC
+// version without stepping to the previous key-value pair e.g.
+// storage.pebbleMVCCScanner in CockroachDB.
+//
+// Performance note: It is important for this struct to not exceed a sizeof 32
+// bytes, for optimizing the common case of the in-place value. Prior to
+// introducing LazyValue, we were passing around a []byte which is 24 bytes.
+// Passing a 40 byte or larger struct causes performance to drop by 75% on
+// some benchmarks that do tight iteration loops.
+//
+// Memory management:
+// This is subtle, but important for performance.
+//
+// - A LazyValue returned by an InternalIterator or Iterator is unstable in
+//   that repositioning the iterator will invalidate the memory inside it. A
+//   caller wishing to maintain that LazyValue needs to call
+//   LazyValue.Clone(). Note that this does not fetch the value if it is not
+//   in-place. Additionally, Clone() *must* only be called if
+//   LazyValue.Value() has *not* been called, since (a) it only makes a
+//   promise about stability of LazyValue and not the underlying value, (b) if
+//   LazyValue.Value() has been called already, the caller has the value
+//   []byte slice so it is unnecessary to fiddle around with LazyValue.
+//
+// - A user of an iterator that calls LazyValue.Value() wants as much as
+//   possible for the returned value []byte to point to iterator owned memory.
+//   - [P1] The underlying iterator that owns that memory also needs a promise
+//     from that user that at any time there is at most one value []byte slice
+//     that the caller is expecting it to maintain. Otherwise, the underlying
+//     iterator has to maintain multiple such []byte slices which results in
+//     more complicated and inefficient code.
+//   - [P2] The underlying iterator, in order to make the promise that it is
+//     maintaining the one value []byte slice, also needs a way to know when
+//     it is relieved of that promise. One way it is relieved of that promise
+//     is by being told that it is being repositioned. Typically, the owner of
+//     the value []byte slice is a sstable iterator, and it will know that it
+//     is relieved of the promise when it is repositioned. However, consider
+//     the case where the caller has used LazyValue.Clone() and repositioned
+//     the iterator (which is actually a tree of iterators). In this case the
+//     underlying sstable iterator may not even be open. LazyValue.Value()
+//     will still work (at a higher cost), but since the sstable iterator is
+//     not open, it does not have a mechanism to know when the retrieved value
+//     is no longer in use. We refer to this situation as "not satisfying P2".
+//     To handle this situation, the LazyValue.Value() method accepts a caller
+//     owned buffer, that the callee will use if needed. The callee explicitly
+//     tells the caller whether the []byte slice for the value is now owned by
+//     the caller. This will be true if the callee attempted to use buf and
+//     either successfully used it or allocated a new []byte slice.
+//
+//  To ground the above in reality, we consider three examples of callers of
+//  LazyValue.Value():
+//  - Iterator: it calls LazyValue.Value for its own use when merging values.
+//    When merging during reverse iteration, it may have cloned the LazyValue.
+//    In this case it calls LazyValue.Value() on the cloned value, merges it,
+//    and then calls LazyValue.Value() on the current iterator position and
+//    merges it. So it is honoring P1.
+//  - Iterator on behalf of Iterator clients: The Iterator.Value() method
+//    needs to call LazyValue.Value(). The client of Iterator is satisfying P1
+//    because of the inherent Iterator interface constraint, i.e., it is calling
+//    Iterator.Value() on the current Iterator position. It is possible that
+//    the Iterator has cloned this LazyValue (for the reverse iteration case),
+//    which the client is unaware of, so the underlying sstable iterator may
+//    not be able to satisfy P2. This is ok because Iterator will call
+//    LazyValue.Value with its (reusable) owned buffer.
+//  - CockroachDB's pebbleMVCCScanner: This will use LazyValues from Iterator
+//    since during reverse iteration in order to find the highest version that
+//    satisfies a read it needs to clone the LazyValue, step back the iterator
+//    and then decide whether it needs the value from the previously cloned
+//    LazyValue. The pebbleMVCCScanner will satisfy P1. The P2 story is
+//    similar to the previous case in that it will call LazyValue.Value with
+//    its (reusable) owned buffer.
+//
+// Corollary: callers that directly use InternalIterator can know that they
+// have done nothing to interfere with promise P2 can pass in a nil buf and be
+// sure that it will not trigger an allocation.
+//
+// Repeated calling of LazyValue.Value:
+// This is ok as long as the caller continues to satisfy P1. The previously
+// fetched value will be remembered inside LazyValue to avoid fetching again.
+// So if the caller's buffer is used the first time the value was fetched, it
+// is still in use.
+//
+// LazyValue fields are visible outside the package for use in
+// InternalIterator implementations and in Iterator, but not meant for direct
+// use by users of Pebble.
+type LazyValue struct {
+	// ValueOrHandle represents a value, or a handle to be passed to ValueFetcher.
+	// - Fetcher == nil: ValueOrHandle is a value.
+	// - Fetcher != nil: ValueOrHandle is a handle and Fetcher.Attribute is
+	//   initialized.
+	// The ValueOrHandle exposed by InternalIterator or Iterator may not be stable
+	// if the iterator is stepped. To make it stable, make a copy using Clone.
+	ValueOrHandle []byte
+	// Fetcher provides support for fetching an actually lazy value.
+	Fetcher *LazyFetcher
+}
+
+// LazyFetcher supports fetching a lazy value.
+type LazyFetcher struct {
+	// ValueFetcher, given a handle, returns the value. It is acceptable to call
+	// the ValueFetcher as long as the DB is open. However, one should assume
+	// there is a fast-path when the iterator tree has not moved off the sstable
+	// iterator that initially provided this LazyValue. Hence, to utilize this
+	// fast-path the caller should try to decide whether it needs the value or
+	// not as soon as possible, with minimal possible stepping of the iterator.
+	//
+	// buf will be used if the fetcher cannot satisfy P2 (see earlier comment).
+	// If the fetcher attempted to use buf *and* len(buf) was insufficient, it
+	// will allocate a new slice for the value. In either case it will set
+	// callerOwned to true.
+	ValueFetcher func(handle []byte, buf []byte) (val []byte, callerOwned bool, err error)
+	// Attribute includes the short attribute and value length.
+	Attribute   AttributeAndLen
+	fetched     bool
+	value       []byte
+	callerOwned bool
+	err         error
+}
+
+// Value returns the underlying value.
+func (lv *LazyValue) Value(buf []byte) (val []byte, callerOwned bool, err error) {
+	if lv.Fetcher == nil {
+		return lv.ValueOrHandle, false, nil
+	}
+	// Do the rest of the work in a separate method to attempt mid-stack
+	// inlining of Value(). Unfortunately, this still does not inline since the
+	// cost of 85 exceeds the budget of 80.
+	//
+	// TODO(sumeer): Packing the return values into a struct{[]byte error bool}
+	// causes it to be below the budget. Consider this if we need to recover
+	// more performance. I suspect that inlining this only matters in
+	// micro-benchmarks, and in actual use cases in CockroachDB it will not
+	// matter because there is substantial work done with a fetched value.
+	return lv.fetchValue(buf)
+}
+
+// INVARIANT: lv.Fetcher != nil
+func (lv *LazyValue) fetchValue(buf []byte) (val []byte, callerOwned bool, err error) {
+	f := lv.Fetcher
+	if !f.fetched {
+		f.fetched = true
+		f.value, f.callerOwned, f.err = lv.Fetcher.ValueFetcher(lv.ValueOrHandle, buf)
+	}
+	return f.value, f.callerOwned, f.err
+}
+
+// InPlaceValue returns the value under the assumption that it is in-place.
+// This is for Pebble-internal code.
+func (lv *LazyValue) InPlaceValue() []byte {
+	if invariants.Enabled && lv.Fetcher != nil {
+		panic("value must be in-place")
+	}
+	return lv.ValueOrHandle
+}
+
+// Len returns the length of the value.
+func (lv *LazyValue) Len() int {
+	if lv.Fetcher == nil {
+		return len(lv.ValueOrHandle)
+	}
+	return int(lv.Fetcher.Attribute.ValueLen)
+}
+
+// Clone creates a stable copy of the LazyValue, by appending bytes to buf.
+// REQUIRES: LazyValue.Value() has not been called.
+// Ensure that this is only called before LazyValue.Value is called. We don't
+// actually care if it was in-place but we don't want confusing things about whether
+// we need to clone the value inside the fetcher.
+func (lv *LazyValue) Clone(buf []byte) (LazyValue, []byte) {
+	if invariants.Enabled && lv.Fetcher != nil && lv.Fetcher.fetched {
+		panic("value has already been fetched")
+	}
+	// INVARIANT: LazyFetcher.value is nil, so there is nothing to copy there.
+	vLen := len(lv.ValueOrHandle)
+	lvCopy := LazyValue{Fetcher: lv.Fetcher}
+	if vLen == 0 {
+		return lvCopy, buf
+	}
+	bufLen := len(buf)
+	buf = append(buf, lv.ValueOrHandle...)
+	lvCopy.ValueOrHandle = buf[bufLen : bufLen+vLen]
+	return lvCopy, buf
+}
+
+// MakeInPlaceValue constructs an in-place value.
+func MakeInPlaceValue(val []byte) LazyValue {
+	return LazyValue{ValueOrHandle: val}
+}
+
+// TODO(sumeer): test that callers are adhering to promise P1. In the sstable
+// iterators, when invariants are enabled, create a testingFetcher struct that
+// contains the sstable name and the []byte backing for the last fetched
+// value. The sstable iterator, instead of returning an in-place value, will
+// return a copy of the key in ValueOrHandle, and a ValueFetcher implemented
+// by testingFetcher.valueFetcher. Each time this method is called, it will
+// mangle the last fetched value []byte slice, allocate a new one, open the
+// sstable, seek to the key, and copy the value into this new byte slice and
+// return.