salmon.triplets.manager.Config
- pydantic settings salmon.triplets.manager.Config
Customization of Salmon sampling and HTML.
This class is often specified through uploading an
init.yamlfile as described in Experiment initialization with YAML file.The default configuration is specified completely below, which is rendered into a YAML file in an example.
Some details on specific fields:
The only required field is targets.
targetsis often specified with the upload of a ZIP file. See YAML file with ZIP file for more detail. If not, specify a list of strings (which will be rendered as HTML).
samplingis not relevant untilsamplersis customized.
Here’s how to customize the config a bit with a YAML file:
samplers: testing: {class: Random} ARR: {} Validation: n_queries: 20 sampling: probs: {"ARR": 40, "Validation": 20, "testing": 20} common: d: 3 # dimensions to embed to; passed to every sampler random_state: 42 # argument to Adaptive initial_batch_size: 128 # argument to Embedding html: instructions: Click buttons, <b><i>or else.</i></b>
Full documentation is below.
Note
sampler_per_useranddetailonly affect the HTML page shown to the user. They do not affect the backend code executed. They are not grouped withHTMLbecause they do affect the responses (at least when using Salmon’s default config and not writing a custom query page).Show JSON schema
{ "title": "Config", "description": "Customization of Salmon sampling and HTML.\n\nThis class is often specified through uploading an ``init.yaml`` file as\ndescribed in :ref:`yamlinitialization`.\n\nThe default configuration is specified completely below, which is rendered into a YAML file in `an example`_.\n\n.. _an example: https://github.com/stsievert/salmon/blob/master/examples/default.yaml\n\nSome details on specific fields:\n\n* The only required field is targets.\n\n * ``targets`` is often specified with the upload of a ZIP file. See\n :ref:`yaml_plus_zip` for more detail. If not, specify a list of strings (which\n will be rendered as HTML).\n\n* ``sampling`` is not relevant until ``samplers`` is customized.\n\nHere's how to customize the config a bit with\na YAML file:\n\n.. code-block:: yaml\n\n samplers:\n testing: {class: Random}\n ARR: {}\n Validation:\n n_queries: 20\n sampling:\n probs: {\"ARR\": 40, \"Validation\": 20, \"testing\": 20}\n common:\n d: 3 # dimensions to embed to; passed to every sampler\n random_state: 42 # argument to Adaptive\n initial_batch_size: 128 # argument to Embedding\n html:\n instructions: Click buttons, <b><i>or else.</i></b>\n\nFull documentation is below.\n\n.. note::\n\n ``sampler_per_user`` and ``detail`` only affect the HTML page\n shown to the user. They do not affect the backend code executed.\n They are not grouped with :class:`~salmon.triplets.manager.HTML`\n because they do affect the responses (at least when using\n Salmon's default config and not writing a custom query page).", "type": "object", "properties": { "targets": { "title": "Targets", "description": "A list of targets that will be rendered as HTML. If a ZIP file is\n uploaded, this field is populated automatically.\n See :ref:`yaml_plus_zip` for more detail.", "env_names": "{'targets'}", "anyOf": [ { "type": "integer" }, { "type": "array", "items": { "type": "string" } } ] }, "html": { "title": "Html", "description": "Stylistic settings to configure the HTML page.\n See :class:`~salmon.triplets.manager.HTML` for more detail.", "default": { "title": "Similarity judgements", "instructions": "Please select the <i>comparison</i> item that is most similar to the <i>target</i> item.", "debrief": "<b>Thanks!</b> Please use the participant ID below.", "skip_button": false, "css": "", "max_queries": 50, "arrow_keys": true, "query_params": [ "puid" ], "js": "", "element_top": "", "element_middle": "", "element_bottom": "", "element_standalone": "" }, "env_names": "{'html'}", "allOf": [ { "$ref": "#/definitions/HTML" } ] }, "samplers": { "title": "Samplers", "description": "Samplers to use, and their initialization parameters. See\n above or \":ref:`adaptive-config`\" for more detail on\n customization, and :ref:`experiments` for experiments/benchmarks.", "default": { "random": { "class": "Random" } }, "env_names": "{'samplers'}", "type": "object" }, "sampling": { "title": "Sampling", "description": "Settings to configure how more than two samplers are\n used. See :class:`~salmon.triplets.manager.Sampling` for more detail.", "default": { "common": { "d": 2 }, "probs": null, "samplers_per_user": 0, "details": {} }, "env_names": "{'sampling'}", "allOf": [ { "$ref": "#/definitions/Sampling" } ] } }, "additionalProperties": false, "definitions": { "HTML": { "title": "HTML", "description": "Stylistic settings to configure the HTML page.\n\nThe default configuration is specified completely below, which is rendered into a YAML file in `an example`_.\n\n.. _an example: https://github.com/stsievert/salmon/blob/master/examples/default.yaml\n\nArbitrary keys are allowed in this class\n(which might allow for customization on the HTML page).", "type": "object", "properties": { "title": { "title": "Title", "description": "The title of the HTML page (the text shown in the tab bar)", "default": "Similarity judgements", "env_names": "{'title'}", "type": "string" }, "instructions": { "title": "Instructions", "description": "The instructions the user sees above each query.", "default": "Please select the <i>comparison</i> item that is most similar to the <i>target</i> item.", "env_names": "{'instructions'}", "type": "string" }, "debrief": { "title": "Debrief", "description": "The message that the user sees after ``max_queries`` responses\n have been completed. The participant ID (``puid``) is shown below\n this message.", "default": "<b>Thanks!</b> Please use the participant ID below.", "env_names": "{'debrief'}", "type": "string" }, "skip_button": { "title": "Skip Button", "description": "Wheter to show a button to skip queries. Most\n useful when participants are instructed to skip queries only when the\n know nothing about any object.", "default": false, "env_names": "{'skip_button'}", "type": "boolean" }, "css": { "title": "Css", "description": "CSS to be included the in the query page. This CSS is\n inserted just before the ``</style>`` tag in the HTML head.", "default": "", "env_names": "{'css'}", "type": "string" }, "max_queries": { "title": "Max Queries", "description": "The number of queries that the user will answer before\n seeing the ``debrief`` message). Set ``max_queries=0`` or\n ``max_queries=-1`` to ask unlimited queries.", "default": 50, "env_names": "{'max_queries'}", "type": "integer" }, "arrow_keys": { "title": "Arrow Keys", "description": "Wheter to allow using the arrow keys as input. Specifying\n ``arrow_keys=True`` might allow bad input (though there is a delay of\n 200ms between queries).", "default": true, "env_names": "{'arrow_keys'}", "type": "boolean" }, "query_params": { "title": "Query Params", "description": "A list of flags to include in the params to /query.\n The default is ``query_params=['puid']``, which hits the endpoint\n ``/query?puid=foo`` if the participant's UID is ``foo``. Developers\n writing custom query pages might be interested in customizing this\n value.\n ", "default": [ "puid" ], "env_names": "{'query_params'}", "type": "array", "items": { "type": "string" } }, "js": { "title": "Js", "description": "JavaScript to include in standalone ``<script>`` tag", "default": "", "env_names": "{'js'}", "type": "string" }, "element_top": { "title": "Element Top", "description": "Custom HTML to include at top of page", "default": "", "env_names": "{'element_top'}", "type": "string" }, "element_middle": { "title": "Element Middle", "description": "Custom HTML to include in middle of page (between target and comparisons)", "default": "", "env_names": "{'element_middle'}", "type": "string" }, "element_bottom": { "title": "Element Bottom", "description": "Custom HTML to include at top of page (above footer)", "default": "", "env_names": "{'element_bottom'}", "type": "string" }, "element_standalone": { "title": "Element Standalone", "description": "Custom HTML to include as a standalone element, only\n encased in the ``head`` tag.\n ", "default": "", "env_names": "{'element_standalone'}", "type": "string" } } }, "Sampling": { "title": "Sampling", "description": "Settings to configure how more than two samplers are used.\n\nThese settings are used in the HTML but are not stylistic. The\nexception is ``common``, which is passed to every ``sampler`` during\ninitialization and will likely be optional arguments for\n:class:`~salmon.triplets.samplers.Adaptive`.\n\nThe default configuration is specified completely below, which is rendered into a YAML file in `an example`_.\n\n.. _an example: https://github.com/stsievert/salmon/blob/master/examples/default.yaml", "type": "object", "properties": { "common": { "title": "Common", "description": "Arguments to pass to every sampler for initialization (likely\n values for :class:`~salmon.triplets.samplers.Adaptive`; note that\n values for ``n`` and ``ident`` are already specified). Any\n values specified in this field will be overwritten by\n sampler-specific arguments.", "default": { "d": 2 }, "env_names": "{'common'}", "type": "object" }, "probs": { "title": "Probs", "description": "The percentage to sample each ``sampler`` when given the\n opportunity (which depends on ``samplers_per_user``). The percentages\n in this sampler must add up to 100.\n If not specified (default), choose each sampler with equal\n probability.", "env_names": "{'probs'}", "type": "object", "additionalProperties": { "type": "integer" } }, "samplers_per_user": { "title": "Samplers Per User", "description": "The number of samplers to assign to each user. Setting\n ``samplers_per_user=1`` means any user only sees queries generated\n from one sampler, and ``sampler_per_user=0`` means the user sees a\n new sampler every query", "default": 0, "env_names": "{'samplers_per_user'}", "type": "integer" }, "details": { "title": "Details", "description": "Different options for a deterministic choice of samplers.\n\n This dictionary is of the form ``{query_shown_to_user: options}``. The\n ``options`` is a dictionary with up to two keys:\n\n - ``sampler`` (required), which reflects which sampler receives the\n responses)\n\n - ``query`` (optional), which is a list of length 3 indicating the target\n indices appear in the query.\n\n For example, this YAML will ensure the 1st and 10th query the\n crowdsourcing user sees will be from:\n\n .. code-block:: yaml\n\n targets: [zero, one, two, three, four, five, six]\n # ^ list of (textual) targets; target \"zero\" has index 0 and\n # is targets[0] in Python\n\n samplers:\n ARR: {}\n Validation: {}\n valid2:\n class: Validation\n n_queries: 3\n\n sampling:\n probs: {ARR: 100, Validation: 0, valid2: 0}\n details:\n # Each key \"n\" is the n-th query the user sees\n # So here the 1st and 10th queries the user sees is customized\n 1: {sampler: \"Validation\", query: [0, 2, 3]}\n 10: {sampler: \"valid2\"}\n\n html:\n # ask 10 queries according to \"sampling.probs\".\n # The probabilistic sampling will be overriden by sampling.details\n max_queries: 10\n\n In this case, the crowdsourcing user will see the following:\n\n * 1st query shown will have head \"zero\", and feet \"two\" and \"three\".\n * Queries 2 and 9 will be generated by the :class:`~salmon.triplets.samplers.ARR` sampler.\n * The 10th query they see (also the last query): one of three\n (random) fixed/static queries.\n\n The sampler ``valid2`` will receive answers to 3 fixed/static queries,\n and the ``Validation`` sampler will receive answers to the query\n ``[0, 2, 3]``.\n ", "default": {}, "env_names": "{'details'}", "type": "object" } }, "additionalProperties": false } } }
- Fields
- field html: HTML = HTML(title='Similarity judgements', instructions='Please select the <i>comparison</i> item that is most similar to the <i>target</i> item.', debrief='<b>Thanks!</b> Please use the participant ID below.', skip_button=False, css='', max_queries=50, arrow_keys=True, query_params=['puid'], js='', element_top='', element_middle='', element_bottom='', element_standalone='')
Stylistic settings to configure the HTML page. See
HTMLfor more detail.
- field samplers: dict = {'random': {'class': 'Random'}}
Samplers to use, and their initialization parameters. See above or “Sampler configuration” for more detail on customization, and Active sampling for experiments/benchmarks.
- field sampling: Sampling = Sampling(common={'d': 2}, probs=None, samplers_per_user=0, details={})
Settings to configure how more than two samplers are used. See
Samplingfor more detail.
- field targets: Optional[Union[int, List[str]]] = None
A list of targets that will be rendered as HTML. If a ZIP file is uploaded, this field is populated automatically. See YAML file with ZIP file for more detail.
- parse(user_config)