Sampling and validation

A declared space answers two questions: what configurations does it produce, and is a given configuration one of them. This page continues with the simulated annealing space from declaring a space.

import designspace as ds

space = ds.space(
    ds.param("initial_temp").real(1e-2, 1e3).log_scale(),
    ds.param("min_temp").real(1e-4, 1.0).log_scale(),
    ds.param("cooling_rate").real(0.80, 0.999).quantized(step=0.005),
    ds.param("steps_per_temp").integer(1, 500),
    ds.param("neighborhood").categorical("swap", "insert", "reverse"),
    ds.param("acceptance").ordinal("greedy", "boltzmann", "metropolis"),
).forbid(ds.param("min_temp") >= ds.param("initial_temp"))
print(space)

Space: 6 params, 0 conditional, 1 constraints
  initial_temp    real         [0.01, 1000.0]  log
  min_temp        real         [0.0001, 1.0]  log
  cooling_rate    real         [0.8, 0.999]  step=0.005
  steps_per_temp  integer      [1, 500]
  neighborhood    categorical  {'swap', 'insert', 'reverse'}
  acceptance      ordinal      ('greedy' < 'boltzmann' < 'metropolis')

  forbid  min_temp >= initial_temp

One draw

sample_one returns a plain nested dict. The seed makes it reproducible.

config = space.sample_one(seed=0)
config

{'initial_temp': 15.304122616866628,
 'min_temp': 0.0011999049779393507,
 'cooling_rate': 0.805,
 'steps_per_temp': 9,
 'neighborhood': 'reverse',
 'acceptance': 'metropolis'}

A config can be pretty-printed against its space using pretty, showing each value beside the domain it satisfies.

print(ds.pretty(config, space))

Config: 6 params, 6 set, 0 inactive, valid
  initial_temp    = 15.304122616866628  in [0.01, 1000.0]
  min_temp        = 0.0011999049779393507  in [0.0001, 1.0]
  cooling_rate    = 0.805  in [0.8, 0.999]
  steps_per_temp  = 9  in [1, 500]
  neighborhood    = 'reverse'  in {'swap', 'insert', 'reverse'}
  acceptance      = 'metropolis'  in ('greedy' < 'boltzmann' < 'metropolis')

  forbid  min_temp >= initial_temp  ok  margin -15.303

cooling_rate lands on the declared grid, and the two temperatures respect the forbid, because the sampler rejects any draw that trips a hard constraint.

config["min_temp"] < config["initial_temp"]

True

Many draws

sample_dicts returns a list of the same shape.

batch = space.sample_dicts(5, seed=1)
[c["neighborhood"] for c in batch]

['swap', 'reverse', 'swap', 'insert', 'swap']

Since the forbid holds by construction, it holds across the whole batch:

assert all(c["min_temp"] < c["initial_temp"] for c in space.sample_dicts(200, seed=2))

Validation

validate checks a configuration against the space and returns a ValidationResult rather than raising.

result = space.validate(config)
result.valid

True

A configuration outside a domain reports which parameter and why. Nothing is clamped: an out-of-range value is an error, never a rounded input.

bad = dict(config, steps_per_temp=9999)
space.validate(bad).param_errors

(steps_per_temp: out_of_bounds (value=9999),)

validate_param checks a single value without needing a whole configuration:

space.validate_param("acceptance", "simulated").param_errors

(acceptance: out_of_bounds (value='simulated'),)

Feasibility

validate covers two separate things: whether each value is legal for its parameter, and whether the constraints hold. valid is the conjunction, so it is false if either fails, and the two halves of the result are what tell them apart.

# Both values sit inside their own domains; it is the pair that is illegal.
infeasible = dict(config, initial_temp=0.05, min_temp=0.5)
result = space.validate(infeasible)
result.valid, result.param_errors

(False, ())

Every value is in range, so param_errors is empty; the forbid is what makes valid false. is_feasible asks only the constraint half of that question:

space.is_feasible(infeasible)

False

# A malformed config and a well-formed but infeasible one both report
# valid=False, and `param_errors` is what distinguishes them.
assert space.validate(infeasible).param_errors == ()
assert space.validate(bad).param_errors != ()

infeasibility_reasons names what failed:

space.infeasibility_reasons(infeasible)

['forbid violated (margin=0.45): min_temp >= initial_temp']

The flattened view

A configuration is nested, matching the declaration. ds.flatten re-keys it by instance path, which is the grammar DataFrame columns, expression references and error messages all use.

ds.flatten(config, space)

{'initial_temp': 15.304122616866628,
 'min_temp': 0.0011999049779393507,
 'cooling_rate': 0.805,
 'steps_per_temp': 9,
 'neighborhood': 'reverse',
 'acceptance': 'metropolis'}

ds.unflatten reverses it exactly:

ds.unflatten(ds.flatten(config, space), space) == config

True

Where to go next

Conditionality and structure makes the set of parameters depend on the values drawn.