Step 4 : Create a friendly UI

The previous sections implemented the client using the python programming language. This section outlines implementing a web based client using javascript and webpack.

Implementation

Create a directory called frontend in the main project directory which will be used as the workspace and cd into it. In the frontend directory, create a js directory. Copy the following three files into the js directory.

package.json

{
  "name": "rps",
  "version": "1.0.0",
  "description": "",
  "private": true,
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "build": "npx webpack main.js"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "cogment": "^0.1.3",
    "webpack": "^4.39.0"
  },
  "devDependencies": {
    "webpack-cli": "^3.3.6"
  }
}

The above file declares the required dependencies (cogment-sdk and webpack are installed).

index.html

<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8">
  <link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.9.0/css/all.css">
  <title>RPS</title>
  <script src="./dist/main.js"></script>
  <style>
    body {
      background-color: linen;
      text-align: center;
    }

    #human_actions>i {
      cursor: pointer;
      margin: 0 1em;
    }

    #machine_actions>span {
      color: #ccc;
    }

    #machine_actions>span>i {
      margin: 0 1em;

    }

    #machine_actions>span.selected {
      color: #000;
    }

    #scores {
      margin: 3em;
      font-size: 3em;
    }

    #human_score, #agent_score {
      border: 2px solid #000;
      padding:0.5em;
      margin: 0 2em;
    }

  </style>
</head>

<body>
  <div id="game">
    <h1>What do you play ?</h1>
    <div id="human_actions" class="actions">
      <i data-value="ROCK" class="fas fa-hand-rock fa-9x"></i>
      <i data-value="PAPER" class="fas fa-hand-paper fa-9x"></i>
      <i data-value="SCISSOR" class="fas fa-hand-scissors fa-9x"></i>
    </div>
    <div id="scores">
      <span id="agent_move"></span>
      <i class="fas fa-user-alt"></i> <span id="human_score">0</span> <span id="agent_score">0</span> <i class="fas fa-robot"></i>
    </div>
    <h1>The machine plays :</h1>
    <div id="machine_actions" class="actions">
      <span><i data-value="ROCK" class="fas fa-hand-rock fa-9x"></i></span>
      <span><i data-value="PAPER" class="fas fa-hand-paper fa-9x"></i></span>
      <span><i data-value="SCISSOR" class="fas fa-hand-scissors fa-9x"></i></span>
    </div>
  </div>
</body>

</html>

Above, is the simple web page that will be used as the frontend interface.

main.js

import { Connection } from 'cogment';
import cog_settings from './cog_settings';

const rps_protos = require("./data_pb")

const API_URL = "http://127.0.0.1:8080"

function display(observation) {
  // console.log(observation.toObject())

  document.getElementById("agent_score").innerHTML = observation.getP1Score();
  document.getElementById("human_score").innerHTML = observation.getP2Score();


  const prev_action = observation.getPreviousP1Action();
  if (prev_action) {
    const agent_choice = prev_action.getDecision();
    if (agent_choice > 0) {
      document.querySelector("#machine_actions > span:nth-child(" + (agent_choice) + ")").classList.add("selected");
    }
  }
}

function cleanup() {
  const buttons = document.querySelectorAll("#machine_actions > span");
  buttons.forEach(b => {
    b.classList.remove("selected");
  })
}

async function launch() {
  const connect = new Connection(cog_settings, API_URL)
  const trial = await connect.start_trial(cog_settings.actor_classes.player);

  let ready = true;

  display(trial.observation);

  const buttons = document.querySelectorAll("#human_actions > i")
  buttons.forEach(b => {
    b.onclick = async function () {
      if (!ready) return;

      cleanup();
      ready = false

      const action = new rps_protos.PlayerAction();
      action.setDecision(rps_protos.Decision[this.getAttribute('data-value')]);

      const observation = await trial.do_action(action);

      display(observation);
      ready = true;
    }
  });
}

window.addEventListener("DOMContentLoaded", (event) => {
  launch()
  console.log("Game started");
});

The above js file demonstrates how to implement communication between the frontend and the orchestrator.

Install our dependencies and package our app by editing and adding this new service to your docker-compose.yaml

docker-compose.yaml

  ...
  webui:
    image: node:12.4-alpine
    working_dir: /app
    volumes:
      - ./frontend/js:/app
    command: /bin/sh -c "npm install && npm run-script build"

This service starts a node image, defines the workdir as /app, mounts the current frontend/js in the workdir, installs the dependencies and compiles the javascript code. Note, that this command is for quick testing of the framework only and not for production.

Whenever you make a change to node.js or that you rebuild the protocol buffers, make sure to run docker-compose run webui. Run it now.

Generating config files

In the root directory of the project, run the following to generate the config in js format:

cogment generate --js_dir=frontend/js

This produces the cog_settings.js and data_pb.js files.

What is envoy and why is it required?

A web browser is HTTP based and does not directly support GRPC calls. Envoy is a proxy which can convert HTTP calls into GRPC through a filter and is provided below. The configuration of grpc-web is outside of the scope of this tutorial. See grpc-web for more information.

To start a docker container running envoy, in the root of your project directory, create an envoy directory which contains the following two files.

Dockerfile

FROM envoyproxy/envoy:latest
COPY ./envoy.yaml /etc/envoy/envoy.yaml
CMD /usr/local/bin/envoy -c /etc/envoy/envoy.yaml

envoy.yaml

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 0.0.0.0, port_value: 9901 }

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 8080 }
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match: { prefix: "/" }
                route:
                  cluster: orchestrator_svc
                  max_grpc_timeout: 0s
              cors:
                allow_origin:
                - "*"
                allow_methods: GET, PUT, DELETE, POST, OPTIONS
                allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
                max_age: "1728000"
                expose_headers: custom-header-1,grpc-status,grpc-message
                filter_enabled:
                  default_value:
                    numerator: 100
                    denominator: HUNDRED
          http_filters:
          - name: envoy.grpc_web
          - name: envoy.cors
          - name: envoy.router
  clusters:
  - name: orchestrator_svc
    connect_timeout: 0.25s
    type: logical_dns
    http2_protocol_options: {}
    lb_policy: round_robin
    hosts: [{ socket_address: { address: orchestrator, port_value: 9000 }}]

The key sections in the above envoy code are the hosts entry (specifying connecting envoy to the orchestrator on port 9000) and the http_filters entry (converts the HTTP request into GRPC).

To start the proxy, edit and add the following service to your docker-compose.yaml

docker-compose.yaml

  ...
  envoy:
    build:
      context: ./envoy
    ports:
      - "8080:8080"
      - "9901:9901"
    depends_on:
      - orchestrator

The proxy to the orchestrator will expose port 8080. Port 9901 is the admin port for envoy.

Thus the workflow becomes : frontend --> envoy:8080 --> orchestrator:9000

Shut down all the service and then restart them as follows -

docker-compose down
docker-compose up envoy

By starting only the envoy service, you won't see the logs of all its dependencies (env, player and orchestator). There are two alternatives in order to bypass this -

  1. Explicitely start the service you want to monitor - docker-compose up envoy orchestator ...

  2. Ask for logs only when needed - docker-compose logs --follow player This will output (--follow) the logs of the player service in real-time.

Open the index.html file in your browser and you should be able to play RPS.