PotionUI / Documentation
DocumentationArchitecture

Start here

Architecture

People and permissions in. A generation out. One place to keep the work.

PotionUI connects your team’s tools to the compute that runs them. Follow a generation from user and group access through native or ComfyUI execution, then into the same History gallery.

One studio, end to end

PotionUI How a generation travelsSystem map / 01
  1. Access

    People get the tools they need.

    UserDirect assignments
    Group membershipsInherited assignments
    Available to this accountPresets Models Assistant configurations

    Administrators manage accounts, groups, and resource assignments.

  2. Compose

    Turn an idea into a generation request.

    Preset + modePrompts + input mediaForm settings
    GeneratePotionUI checks access and prepares the request.
    Server

    The preset defines the pipeline and declares its engine.

  3. Route, queue & execute

    Match the work to compatible compute.

    Generation routerMatching engine → model availability → preset requirementsChoose an eligible default backend, otherwise highest priority.
    Highlight an example route
    Native engine

    Local GPU

    Native pipeline inside PotionUI.

    On this machine
    Native engine

    Remote worker

    Native pipeline on a connected worker.

    Your server or cloud GPU
    ComfyUI engine

    ComfyUI server

    Workflow engine via the ComfyUI Backend plugin.

    Local or remote server
    Additional engines

    Plugin backend

    Connect another engine through a plugin.

    Extension point
    One PotionUI queue per backendExample · FIFO order

    PotionUI holds one active slot per backend. Pending jobs stay in PotionUI until that slot is released; separate backends can run in parallel.

    Local GPU

    1. ActiveJob 01User A
    2. NextJob 02User A
    3. WaitingJob 03User B

    Remote worker

    1. ActiveJob 04User B
    2. NextJob 05User C
    3. WaitingJob 06User A

    ComfyUI

    1. ActiveJob 07User C
    2. NextJob 08User A
    3. WaitingJob 09User B

    Plugin backend

    1. ActiveJob 10User A
    2. NextJob 11User B
    3. WaitingJob 12User C

    ComfyUI executes the active workflow and streams progress. PotionUI holds its slot while waiting for completion, then submits the next job. Plugin backends follow the same queue contract.

    PotionUI’s FIFO (default) or Fair scheduling policy determines ordering across users for every backend. How ordering works ↓

    Execute on the selected backend

    One request takes one compatible route. Progress and previews return to the workspace while it runs.

    Local native: execution stays in PotionUI’s process. Outputs pass directly to its output handlers.

    Remote native: the worker receives the pipeline and input assets. Returned media enters PotionUI’s normal output handlers.

    ComfyUI: the server executes one submitted workflow and streams progress. PotionUI waits for results with the backend slot occupied; the next job stays in PotionUI’s queue until completion.

    Plugin backend: a plugin driver executes locally or through another service. PotionUI owns its queue and holds one active slot until completion. Returned outputs enter the same gallery.

  4. Save

    Keep the media and its generation context.

    All routes converge on PotionUI’s output processing.

    Media storageOutput files + thumbnailsLocal disk or configured S3 storage
    Generation recordsOwner, prompt, seed, models, settingsStored in PotionUI’s database
  5. Keep & reuse

    One gallery, every engine.

    ImagesVideoAudio3D

    History Tags & collections Inspect & reuse

    Generation access stays with the owner or an administrator.

Follow the arrows from assigned tools to saved results. Route highlighting illustrates execution options; the actual backend is selected by PotionUI’s routing rules.

Access follows the person

Direct user assignments and assignments inherited through group membership determine available resources. Administrators manage these grants. Assistant access supports prompt composition; a generation uses its selected preset and models.

Group membership grants access to tools. Generation records retain their own owner, with owner-or-administrator access enforced when reading or changing results.

The preset chooses the engine; the router chooses the backend

The router starts with enabled backends matching the preset’s engine. It uses model availability and cached requirement checks to exclude known-ineligible candidates, then chooses the eligible default or highest-priority backend. Unchecked requirements remain unknown and trigger a background refresh.

Native local and remote workers share the native engine. A ComfyUI preset requires a ComfyUI backend. If no backend qualifies, PotionUI reports the reasons instead of switching engines. RunPod supplies hosting for remote compute; it is not a separate generation engine. See Models & backends for setup.

The dashed Plugin backend branch marks an extension point: plugins can register more engine types, their backend configuration, and compatible pipes and presets. These integrations join the same generation flow and return results to the same gallery. See the backend extension reference ↗.

PotionUI owns every backend’s queue

Routing assigns a request to a backend; PotionUI’s scheduler decides when to dispatch it. PotionUI keeps one active generation per backend, so a busy local GPU does not block a separate remote worker. All four lanes show example jobs in FIFO order: the default policy dispatches them in arrival order.

Administrators can choose Fair scheduling under Admin → Backends → Scheduling, for any engine. It rotates between users while preserving each user’s own job order, and prefers jobs compatible with the loaded model. By default, after three consecutive jobs for that model, the longest-waiting eligible job for a different model goes next. If no other model is waiting, same-model work continues. See the scheduling policy reference ↗.

ComfyUI owns execution of the active workflow and its progress stream. PotionUI keeps the backend slot occupied while waiting for results, then releases it before submitting the next job. Each configured backend therefore has at most one PotionUI job submitted at a time. Plugin engines registered through backend.register use the same PotionUI queue and single-slot contract, whether their driver executes locally or calls another service.

Different compute, the same History

Local outputs, imported remote-worker outputs, and ComfyUI results enter the common output-handling system. Media handlers save files and thumbnails, while generation records keep the available parameters and model information. Progress and previews also return to the workspace.

The History gallery brings those saved results together for filtering, tagging, collections, inspection, and reuse. Backend location changes where generation happens; ownership remains attached to the generation.

Users & groups ↗Routing rules ↗Remote execution ↗Output processing ↗Generation ownership ↗