Cogment Fundamentals Guide

Introduction

Welcome to the Cogment Fundamentals guide. It contains information that is pertinent to both the high-level SDK and the low-level API.

Core Concepts

Trials

A deployed Cogment project is tasked with running trials. Each trial is populated with its own actors that observe and interact with the trial's environment.

As a trial runs, a few things happen:

  • Actors choose and take actions based on their observation of the environment.
  • The environment updates itself based on the actions taken by the actors and generates updated observations.
  • Multiple Feedbacks for the actors are generated either by the environment or other actors.
  • Actors asynchronously receive reward information based on received feedbacks.
  • A datalog of the observation/action/reward is produced and stored.

A trial begins at the request of a frontend application and finishes when either the frontend application ends it, when a predefined amount of time has elapsed (either in real time, or number of updates), or if the trial does not see any activity for an extended period of time.

Actors and actor classes

Each actor within a trial is defined primarily by what information it receives from the environment (its observation space), and what actions it can perform (its action space).

Two cogment actors that share the same observation space and action space are said to belong to the same actor class.

An actor is controlled either by an agent, or by a human. Whichever the case, the same process of generating actions based on observations remains the same, and they are treated as the same by the environment.

The cogment.yaml

At the heart of every Cogment project is a yaml file typically called cogment.yaml. Its primary role is to define the actor classes present within the project, including their nature and number participating in each trial.

Services

A deployed Cogment project consists of a cluster of service applications. These are composed of service applications provided by the cogment framework itself (depicted below in blue) and those implemented by the project (depicted below in orange) either by employing the cogment SDK or by directly implementing the underlying protocol.

Screenshot

Environment service

The environment service is responsible for creating and updating environments for each trial. It must provide three functions:

Start: Create the initial observation set for a trial.

Update: Create an updated observation set given the actions of all actors within the trial.

End: Cleanup any internal state related to a trial

Agent Service(s)

Start: Announce that an agent is requested for a trial.

Decide: Given an observation, choose an action.

Reward: Be notified about reward information associated with a previous decision.

N.B. Currently, live rewards are only sent for the previous time step. Retroactively updated rewards will still find their way into the datalog.

End: Cleanup any internal state related to a trial

Orchestrator

The Orchestrator is the glue that binds everything together. From the perspective of a framework user, it can be considered as the live interpreter of the cogment.yaml configuration file. It is the service that client applications will connect to in order to start and run trials.

Envoy

The Orchestrator is not capable of handling traffic from web browsers by itself. So client applications using the Javascript API in the browser must connect to a translation proxy. That's where envoy comes in. With a simple boilerplate configuration, it automatically translates the web-based protocol into the native version.

If a cogment project does not support web-based client applications, then the envoy proxy can be ommited from any deployment.