API

Configuration

salmon.triplets.manager.Config

Customization of Salmon sampling and HTML.

This configuration has two optional components:

salmon.triplets.manager.HTML

Stylistic settings to configure the HTML page.

salmon.triplets.manager.Sampling

Settings to configure how more than two samplers are used.

Offline embeddings

This class can be used to create embeddings from a set of downloaded responses from Salmon:

salmon.triplets.offline.OfflineEmbedding([...])

Generate an embedding offline (after responses are downloaded from Salmon).

Samplers

These classes are used to collect responses with Salmon. They can be configured using init.yaml as mentioned in Sampler configuration. They all conform to the following API:

salmon.backend.sampler.Sampler([ident])

Run a sampling algorithm.

This class enables running a triplet embedding algorithm on Salmon: it provides convenient hooks to the database like get_queries and post_answers if you want to customize the running of the algorithm. By default, the algorithm uses Sampler.run to run the algorithm.

Every class below inherits from Sampler.

Passive Algorithms

salmon.triplets.samplers.Random(n[, d, ...])

Choose the triplet queries randomly, only ensuring objects are not repeated.

salmon.triplets.samplers.RoundRobin(*args, ...)

Let the head of the triplet query rotate through the available items while choosing the bottom two items randomly.

salmon.triplets.samplers.Validation(n[, d, ...])

Ask about the same queries repeatedly

Active Algorithms

Every active algorithm inherits from one class:

salmon.triplets.samplers.Adaptive(*, n[, d, ...])

The sampler that runs adaptive algorithms.

The class Adaptive runs the adaptive algorithm and depends on Embedding for optimization:

salmon.triplets.samplers.adaptive.Embedding(...)

An optimization algorithm that produces an embedding from human responses of the form [head, winner, loser].

To customize the optimization, all extra keyword arguments are passed to the optimizer.

Then, all of these classes inherit from Adaptive:

salmon.triplets.samplers.ARR([R, n_top, ...])

An adaptive "round robin" sampler.

salmon.triplets.samplers.TSTE([alpha])

The t-Distributed STE (t-STE) embedding algorithm [R4f12f95c5f32-1].

salmon.triplets.samplers.SOE(**kwargs)

The soft ordinal embedding detailed by Terada et al. [R4edb93585d19-1].

salmon.triplets.samplers.STE(**kwargs)

The Stochastic Triplet Embedding [R1e0364a4a1f5-1].

salmon.triplets.samplers.CKL([mu])

The crowd kernel embedding.

salmon.triplets.samplers.GNMDS(**kwargs)

The Generalized Non-metric Multidimensional Scaling embedding [Rc897032fff92-1].

We have tested out the top three algorithms—ARR, TSTE and SOE—in our experiments. We use ARR for our adaptive sampling (which defaults to the noise model in TSTE) and use SOE for the offline embeddings.

These adaptive algorithms are all the same except for the underlying noise model, with the exception of ARR. ARR introduces some randomness by fixing the head and adding the top 1 * n triplets to the database. This is useful because the information gain measure used by all of these algorithms (by default) is a rule-of-thumb.

Note

Use of ARR is recommended as it performs well in the experiments we have run.