Reward models #
Finite models whose transitions are lists of edges, each paying a reward that is added to the eventual output.
Edge rewards translate successful outputs. Rejection and divergence discard them.
- size : ℕ
The number of states.
The state in which evaluation starts.
- kind : Fin self.size → FiniteModel.StateKind
Whether each state is transient, returns a value, or rejects.
The outgoing edges of each state.
Instances For
The probability of moving from state i to state j in one step: the total probability of
the edges from i to j.
Equations
- Determinize.Spec.RewardModel.controlWeight model i j = (List.map (fun (e : Determinize.Spec.RewardModel.Edge model.size) => if e.target = j then e.probability else 0) (model.edges i)).sum
Instances For
The law of x + reward when x has law law.
Equations
- Determinize.Spec.RewardModel.shift reward law = MeasureTheory.Measure.map (fun (x : ℝ) => x + ↑reward) law
Instances For
Unnormalized output accumulated within steps transitions from a state. Taking an edge adds
its reward to the eventual output.
Equations
- One or more equations did not get rendered due to their size.
- model.outputWithin 0 x✝ = match model.kind x✝ with | Determinize.Spec.FiniteModel.StateKind.returned value => MeasureTheory.Measure.dirac ↑value | x => 0
Instances For
The output law from a state: the limit of outputWithin over the number of steps.
Equations
- model.outputAt state = ⨆ (steps : ℕ), model.outputWithin steps state
Instances For
The output law of the model: the output law from its initial state.
Equations
- model.outputMeasure = model.outputAt model.initial
Instances For
Exact source correspondence includes failure freedom and the entire output law.
Equations
- model.Matches program = (Determinize.Spec.Paper.DomainSafe program ∧ model.outputMeasure = Determinize.Spec.Paper.bigStepMeasure program)