Experiment monitoring

While participants are answering queries, you can track the state of the experiment by visiting http://[url]:8421/dashboard. Initially, this will be fairly barren because users haven’t responded.

After users respond, the dashboard will show the following information:

  • Basic information (start time, number of participants, etc)

  • User response times (e.g., when did the users respond?)

  • Server side timing (e.g., how long did it take to process each API endpoint?)

  • Client side timing (e.g., how long did they spend waiting or thinking about their response?)

  • Algorithm timing (e.g., how long do model updates take?)

  • An embedding for each algorithm.

  • The list of targets.

A static dashboard is visible at dashboard.html. This example doesn't render the images, but they're shown in the static images below. Here are some screenshots from that example:

Basic information

_images/basic.png

Bolded links are particularly useful. I download all bolded links before stopping or restarting the machine running Salmon (except the query page).

Server side timing

_images/server-side.png

Client side timing

_images/client-side.png

Embeddings

_images/embedding.png

Algorithm timing

_images/alg-timing.png

Database

_images/database.png

Targets

I only show the first 4 targets below; the other targets can be seen in the embedding above.

_images/targets.png