Playground
Arkane Cloud playground
Arkane Cloud provides a model playground where users can test various AI models available in Arkane Cloud Playground through a web interface without coding.
How to use the playground
On the Models page, choose a model and click Deploy on its card.
In System prompt, enter instructions that the model should follow when generating output. You can include examples of good in-context learning outputs (few-shot prompting). For example:
Add a fun fact about trains to the end of each response. The fun fact may not be related to what I'm asking you about. Here is an example:
- User: What is the capital of Croatia? - You: The capital of Croatia is Zagreb. By the way, did you know that the first railroad in New England was powered by horses?
Start chatting to the model.
Model parameters
The playground allows you to set basic sampling parameters for the model you have chosen.
For more parameters supported by the inference API, refer to the vLLM documentation.
Temperature
Affects: Output randomness
Models: All
Type: Number
Range of values: From 0 to 1 for meta-llama/Meta-Llama-3, from 0 to 2 for models in other families
Default value: From 0.3 to 0.7, depending on the model
Temperature determines how "hot-headed" the model predictions are. The higher the temperature, the more random and less deterministic and conservative the output.
For example, a 0.8 temperature makes outputs more creative and random than a 0.5.
Max tokens
Affects: Computational cost
Models: All
Type: Number
Range of values: Depends on the model (Usually between 1 to 4096 tokens)
Default value: Depends on the model
The maximum number of tokens that the model generates.
Presence penalty
Affects: Output randomness
Models: All
Type: Number
Range of values: From −2 to 2
Default value: 0
The presence penalty is applied to new tokens that have previously appeared in the output. Positive values penalize such tokens and negative values favor them.
Top-p threshold
Affects: Output randomness
Models: All
Type: Number
Range of values: From 0 to 1
Default value: From 0.9 to 1, depending on the model
In top-p sampling, also known as nucleus sampling, the model considers only the most probable tokens whose combined probability mass is equal to the specified threshold.
For example, with a threshold of 0.1, only the tokens that comprise the top 10% of the probability mass are considered.
To consider all tokens, set the threshold to 1.
Last updated