Welcome to Salmon’s documentation!
Salmon is a tool to easily allow collection of “triplet queries.” These queries are relative similarity judgments of the form “is object \(a\) or \(b\) closer to object \(h\)?” An example is shown below with facial similarities:
These queries provide a relative similarity measure: a response indicates that object \(a\) is closer to object \(b\) than object \(c\) as determined by humans. For example, these triplet queries have been used by psychologists to determine what facial emotions human find similar:
Only distance is relevant in this embedding, not the vertical/horizontal axes. However, if you look closely, you can see two axes: positivity and intensity.
Salmon provides efficient methods for collecting these triplet queries. Typically, generating the embeddings above requires far too many human responses. Salmon provides the ability to generate the same embeddings with fewer human responses – in our experiments, about 1,000 queries are required to reach a particular quality level instead of about 3,000 queries. If you’re paying for each human response (say on Mechanical Turk), this means that collecting responses will be reduced by a factor of 3 when compared with naive methods of collecting triplet queries.
If you’d like to report bugs/issues, or improve Salmon please see Salmon’s contribution guide. The list of dependicies and their licenses is available at Dependencies. Salmon is licensed under the BSD License. Details are at LICENSE.txt.
Users
Salmon is currently being actively used by pyschologists from the University of Wisconsin–Madison, and has seen some user from pyschologists at the Louisiana State University and Canada’s Western University.
If you use Salmon, please cite “Efficiently Learning Relative Similarity Embeddings with Crowdsourcing”, which mentions this specific BibTex:
@article{Sievert2023,
author = {Scott Sievert and Robert Nowak and Timothy Rogers},
title = {Efficiently Learning Relative Similarity Embeddings with Crowdsourcing},
journal = {Journal of Open Source Software}
doi = {10.21105/joss.04517},
url = {https://doi.org/10.21105/joss.04517},
year = {2023},
publisher = {The Open Journal},
volume = {8},
number = {84},
pages = {4517},
}
Salmon also has a citation file file at salmon/CITATION.cff.
Usage
- Installation
- Getting started
- Experiment initialization
- Sampler configuration
- Frontend customization
- Deploying
- Experiment monitoring
- Generating embeddings offline
- FAQ
- How do I cite Salmon?
- When should I use random/active sampling?
- How many responses will be needed?
- What active samplers are recommended?
- Can I choose a different machine?
- How do I ask specific questions?
- How do I specify when samplers are used?
- How do I ask every crowdsourcing user exactly the same questions?
- How do I see the Dask dashboard?
- How do I customize the participant unique identifier aka “puid”?
- How do I use HTTPS with Salmon?
- API
Algorithm Developers
Other sources of this documentation
This documentation is available at these locations:
On GitHub as a raw PDF (alternate link).
On GitHub as a zipped HTML directory, which requires unzipping the directory then opening up
index.html.
Please file an issue if you can not access the documentation above.