Package akka.javasdk.testkit.eval

Record Class ReportDocument

Object
Record
akka.javasdk.testkit.eval.ReportDocument
Record Components:
format - always "akka-eval-report"
formatVersion - 2; increments when a property changes meaning or is removed
name - the name given with Experiment.name(java.lang.String), or the test method that ran the experiment
startedAt - when the run started
finishedAt - when the last attempt was evaluated
runs - how many times every case ran
gate - the gate verdict
summary - the case and attempt counts and the pass rate
evaluators - one entry per evaluator label, in the order the labels first appear in the attempts, counting every attempt
spend - model calls, tokens and latency summed over the attempts whose evidence carries model calls
cases - one entry per case, in the order the cases were given, with its outcome over all runs and its requirements
attempts - one entry per attempt: every case in the order the cases were given, its runs ascending
public record ReportDocument(String format, int formatVersion, String name, Instant startedAt, Instant finishedAt, int runs, ReportDocument.Gate gate, ReportDocument.Summary summary, List<ReportDocument.EvaluatorCounts> evaluators, ReportDocument.Spend spend, List<ReportDocument.Case> cases, List<ReportDocument.Attempt> attempts) extends Record
The report file as data: what Experiment.run() writes to target/eval-reports/<name>-<start time>.json, and what read(java.nio.file.Path) gives back. The file is this record as JSON, in the akka-eval-report format described by akka/javasdk/testkit/eval/eval-report.schema.json on the testkit classpath. A reader ignores properties it does not know.

An attempt is one case in one run. Without Experiment.repeat(int) every case has one attempt.

  • Field Details

  • Constructor Details

  • Method Details

    • read

      public static ReportDocument read(Path file)
      Reads a report file written by Experiment.run().

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. Reference components are compared with Objects::equals(Object,Object); primitive components are compared with '=='.
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • format

      public String format()
      Returns the value of the format record component.
      Returns:
      the value of the format record component

    • formatVersion

      public int formatVersion()
      Returns the value of the formatVersion record component.
      Returns:
      the value of the formatVersion record component

    • name

      public String name()
      Returns the value of the name record component.
      Returns:
      the value of the name record component

    • startedAt

      public Instant startedAt()
      Returns the value of the startedAt record component.
      Returns:
      the value of the startedAt record component

    • finishedAt

      public Instant finishedAt()
      Returns the value of the finishedAt record component.
      Returns:
      the value of the finishedAt record component

    • runs

      public int runs()
      Returns the value of the runs record component.
      Returns:
      the value of the runs record component

    • gate

      public ReportDocument.Gate gate()
      Returns the value of the gate record component.
      Returns:
      the value of the gate record component

    • summary

      public ReportDocument.Summary summary()
      Returns the value of the summary record component.
      Returns:
      the value of the summary record component

    • evaluators

      public List<ReportDocument.EvaluatorCounts> evaluators()
      Returns the value of the evaluators record component.
      Returns:
      the value of the evaluators record component

    • spend

      public ReportDocument.Spend spend()
      Returns the value of the spend record component.
      Returns:
      the value of the spend record component

    • cases

      public List<ReportDocument.Case> cases()
      Returns the value of the cases record component.
      Returns:
      the value of the cases record component

    • attempts

      public List<ReportDocument.Attempt> attempts()
      Returns the value of the attempts record component.
      Returns:
      the value of the attempts record component