Identity, serialization and solver hand-off#

Two questions face any consumer storing results: which space produced this number, and which point in it. This page answers both, then walks through the three shapes a solver hand-off takes. The running example is a NAS-shaped hyperparameter space.

import designspace as ds

space = ds.space(
    ds.param("lr").real(1e-5, 1.0).log_scale(),
    ds.param("weight_decay").real(1e-6, 1e-2).log_scale(),
    ds.param("n_layers").integer(1, 5),
    ds.param("width").integer(8, 256).log_scale().repeat(ds.param("n_layers")),
    ds.param("optimizer").categorical("adam", "sgd"),
).forbid(ds.param("lr") > 0.5)
print(space)

Space: 5 params, 0 conditional, 1 constraints
  lr            real         [1e-05, 1.0]  log
  weight_decay  real         [1e-06, 0.01]  log
  n_layers      integer      [1, 5]
  width         list         count = n_layers, of integer [8, 256]
  optimizer     categorical  {'adam', 'sgd'}

  forbid  lr > 0.5

Serialization#

to_json produces a plain dict, and Space.from_json reconstructs it.

doc = space.to_json()
sorted(doc)

['conditions', 'constraints', 'params', 'version']

doc["version"], len(doc["params"])

(1, 5)

version is the shared format integer, frozen since the space format was fixed. The parameter entries carry the resolved declaration:

doc["params"][0]

{'path': 'lr',
 'kind': 'real',
 'domain': {'kind': 'real', 'lo': 1e-05, 'hi': 1.0},
 'periodic': False,
 'prior': {'kind': 'log'}}

restored = ds.Space.from_json(doc)
restored.fingerprint() == space.fingerprint()

True

A custom parameter needs a type_key to factory registry to reconstruct, since core cannot know how to rebuild a user-defined type: ds.Space.from_json(doc, custom_types={...}).

Fingerprints#

A fingerprint identifies a space. Equal fingerprints guarantee identical valid-configuration sets; unequal ones guarantee nothing, because identity is structural after desugaring rather than semantic.

space.fingerprint()

'1:full:203983b4d5e42ce9621e079ec29bd6bf07a3ef4e55b0f7088a489b3df9b7b5ba'

Two scopes are available. "full", the default, is document identity. "sampling" covers only what fixes the feasible set, the measure and the chart geometry, so a change to identity-level bookkeeping moves one and not the other.

tagged = space.meta(experiment="baseline")
(
    tagged.fingerprint(scope="sampling") == space.fingerprint(scope="sampling"),
    tagged.fingerprint(scope="full") == space.fingerprint(scope="full"),
)

(True, False)

Observation identity#

ds.config_hash is a configuration’s own stable key. The pair (space.fingerprint(), config_hash(config, space)) identifies one observation globally, and is what results should be keyed on.

config = space.sample_one(seed=0)
key = (space.fingerprint(), ds.config_hash(config, space))
key[1]

'b7e2ee4bb478c7893e5baa0002b5f465df7e07de5cf9935eb0180908183a6fc4'

The hash follows the configuration across a round-trip of the space:

assert ds.config_hash(config, space) == ds.config_hash(config, restored)
assert restored.validate(config).valid

Three shapes of hand-off#

designspace declares spaces. It does not search them, and it ships no operators, no distances and no adapters. Pointing a solver at a Space is therefore work for the consumer or the solver author, and it takes one of three shapes.

Which shape applies depends on a fact about the solver rather than about the space: every solver defines the space it can work with. Base CMA-ES is ℝⁿ. Variants add integers and categoricals. SMAC and irace add conditionals. Others work on graphs.

Shape 1: interpret the Space directly#

A solver that understands the IR walks it. Topological order first, then activity from conditions, then the charts.

space.topological_order

['lr', 'weight_decay', 'n_layers', 'width', 'optimizer']

space.dependency_graph

{'lr': frozenset(),
 'weight_decay': frozenset(),
 'n_layers': frozenset(),
 'width': frozenset({'n_layers'}),
 'optimizer': frozenset()}

Every generative scalar parameter carries a chart, which is what gives a solver type-appropriate perturbation with no per-type code: mutate in [0, 1], then decode.

lr = space.params["lr"].chart
[round(lr.from_unit(u), 6) for u in (0.0, 0.5, 1.0)]

[1e-05, 0.003162, 1.0]

round(lr.to_unit(0.001), 4)

0.4

Capability negotiation is ordinary introspection. There is no protocol to implement: the solver checks what it needs and fails with its own message, since only it knows what it supports.

(
    space.is_conditional,
    space.has_variable_length,
    space.is_finite,
    space.has_nongenerative_params,
)

(False, True, False, False)

Shape 2: convert to a foreign representation#

Core ships no adapter for ConfigSpace or its kin, and takes no dependency on one. The public, bidirectional IR is the socket: walk params, emit the foreign declaration, and map back.

[(p, pd.type_kind) for p, pd in space.params.items()]

[('lr', 'real'),
 ('weight_decay', 'real'),
 ('n_layers', 'integer'),
 ('width', 'list'),
 ('optimizer', 'categorical')]

Shape 3: bridge with a Representation#

Where the solver’s genotype differs from the declared phenotype, space.represent() builds the induced chart representation, derived mechanically from the charts already on the declaration.

rep = space.represent()
rep.encoded

('lr', 'weight_decay', 'width')

n_layers is excluded and reported separately, because transport rewrites conditions and constraints but never a count:

rep.excluded_by_prop

('n_layers',)

The target is an ordinary Space of unit-interval coordinates, so shape 1 applies to it unchanged. A bridge introduces no new vocabulary; it only moves where shape 1 gets applied.

rep.target.params["lr"].domain

[0.0, 1.0]
genotype = rep.target.sample_one(seed=0)
genotype

{'lr': 0.6369616873214543,
 'weight_decay': 0.2697867137638703,
 'n_layers': 1,
 'width': [0.016527635528529094],
 'optimizer': 'sgd'}

rep.decode(genotype)

{'lr': 0.015304122616866646,
 'weight_decay': 1.1999049779393521e-05,
 'n_layers': 1,
 'width': [8],
 'optimizer': 'sgd'}

Decoding is guaranteed total: every genotype the target calls valid decodes to a phenotype the source calls valid.

phenotype = rep.decode(genotype)
assert space.validate(phenotype).param_errors == ()
assert rep.target.is_feasible(genotype) == space.is_feasible(phenotype)

The reverse round-trip is not a law, because an integer chart is many-to-one:

rep.encode(rep.decode(genotype)) == genotype

False

rep.check(n, seed) runs the conformance laws as a tool. It covers decode totality, feasibility agreement, and, where invertible, the one-directional round-trip. It returns a report and never raises.

result = rep.check(n=200, seed=1)
result.ok, result.n, result.failures

(True, 200, ())

Custom types negotiate per parameter#

A .custom() parameter is an open world, and it offers two independent channels.

The generation ladder, where the richest available rung wins:

  1. a native adapter that recognizes the type’s type_key;

  2. a Representation whose target this solver can handle, with the geometry authored by the type author and the loss declared rather than silent;

  3. opaque sample(rng), sufficient for random search and resampling moves.

The modeling channel, orthogonal to generation: properties() featurizes values for surrogates and reporting whichever rung produced them, and to_json/config_hash give observation identity. A type opaque to generation can still be rich to modeling, because the two channels are independent.

Adapter conventions#

Strategy-entangled operations are the only things forced into adapters: crossover schemes, mutation policies, trust regions. When writing one:

  • key it by the same type_key used in serialization;

  • give it the live ParamType instance and derive domain facts from it through describe, validate and extract, instead of re-declaring them;

  • pass it a Representation instead of embedding one;

  • scope it per (capability, type). Scoping per (solver, type) multiplies adapters for no gain.

Where to go next#

The design notes cover the decisions behind these mechanisms, and the API reference documents every exported name.