Polling for the Result

Because third-party APIs can take time to respond, forwardRequest() uses a queue-and-poll model. Every call is queued on the server and you can poll for the result.

Automatic polling with poll

Pass a poll interval (in milliseconds) together with onResponse and onError callbacks. With poll greater than 0, Skapi auto-generates a queue and starts polling automatically; the final result is delivered to onResponse (and errors to onError). Note that when onResponse is provided, the awaited return value of forwardRequest() is the callback's return value, so don't rely on the returned promise for the status object in the auto-poll case. The immediate status object with a poll() method ({ id, status, ... }) is only returned when poll is omitted or 0 and onResponse is not supplied.

skapi.forwardRequest({ model: 'gpt-image-1.5', prompt: 'A cute baby sea otter', n: 1, size: '1024x1024' }, {
    secretName: 'openai',
    url: 'https://api.openai.com/v1/images/generations',
    method: 'POST',
    headers: {
        'Content-Type': 'application/json',
        Authorization: 'Bearer $CLIENT_SECRET'
    },
    poll: 2000,              // poll every 2 seconds
    onResponse(result) {
        console.log('Done:', result);
    },
    onError(err) {
        console.error('Failed:', err);
    }
});

Manual polling with poll()

When poll is omitted or 0, the promise resolves with the status object plus a poll() method. Call it whenever you are ready to start polling:

const res = await skapi.forwardRequest({ model: 'gpt-image-1.5', prompt: 'A cute baby sea otter', n: 1, size: '1024x1024' }, {
    secretName: 'openai',
    url: 'https://api.openai.com/v1/images/generations',
    method: 'POST',
    headers: {
        'Content-Type': 'application/json',
        Authorization: 'Bearer $CLIENT_SECRET'
    },
    queue: 'image-queue'
});

// res = { id, status: 'running', queue_name, in_queue, poll }
res.poll({
    latency: 2000, // start polling at 2-second intervals
    onResponse(result) {
        console.log('Done:', result);
    },
    onError(err) {
        console.error('Failed:', err);
    }
});

WARNING

Only requests with status running or pending can be polled. A request enters running or pending only when either params.queue (queue name) or params.poll (poll interval) is provided.

How long the call actually took

onResponse receives a second argument, meta, carrying facts about the request rather than about the response. A settled poll resolves with the destination's own answer, and that answer belongs to the destination: there is nowhere in it to put a value of skapi's without changing what you asked for. So request-level facts arrive beside it instead.

meta.executed is when the worker began running the request, in milliseconds. Subtract it from the request's updated to time the call on its own:

res.poll({
    latency: 2000,
    onResponse(result, meta) {
        if (meta?.executed) {
            console.log('the call began at', new Date(meta.executed));
        }
        console.log('Done:', result);
    }
});

Use executed rather than created when you are timing the destination. created is when the request was queued, and a queued request can sit there for a long time before a worker picks it up, so updated - created measures your own queue depth plus the call while updated - executed measures the call.

meta.executed is optional and can legitimately be absent, so branch on it rather than assuming it. The value rides on the status envelope that a running poll tick returns, so a request that began and finished between two ticks never showed one. When it is missing, show nothing: substituting created reports a queue backlog as though the destination had been slow. The value is on the request's own row either way, so a later forwardRequestHistory() will still report it as executed.

A poll started from a forwardRequestHistory() item is the exception: it takes the execution start from the listing that produced it, so it can hand you meta.executed even when the request settles on the poll's very first read.

A callback that declares one parameter is unaffected, and onResponse on a direct, non-queued forwardRequest() receives no meta, since nothing ever recorded an execution start for a request that was never queued.

Stopping Polling

Polling and the request itself are separate things. stopForwardRequestPolling() stops watching a request; the server keeps working on it, and you can pick the result up later.

This matters because polling is real network traffic. A long-running request polled every second keeps issuing calls for as long as it takes, whether or not anyone is looking at the result. Stop polling when the user navigates away or the tab is hidden, and start again when they come back.

// Stop watching one request. It keeps running on the server.
skapi.stopForwardRequestPolling({
    url: 'https://api.openai.com/v1/images/generations',
    method: 'POST',
    id: 'stamp:entropy'   // the id from the forwardRequest response
});

// Stop every poll on a queue.
skapi.stopForwardRequestPolling({ queue: 'image-queue' });

// Stop everything this client is polling.
skapi.stopForwardRequestPolling();

Each call returns how many polls it stopped.

Handling a stopped poll

A stopped poll resolves with { id, status: 'stopped' } rather than rejecting, so await sites do not need a catch. Its onResponse and onError callbacks are not called, because a stop is not a result. Use isPollStopped() to tell the two apart:

const res = await skapi.forwardRequest(null, { /* ... */ });

const result = await res.poll({ latency: 2000 });

if (skapi.isPollStopped(result)) {
    // We stopped watching. Nothing failed, and the request may still be running.
    return;
}

console.log('Done:', result);

Resuming

There is nothing to resume, as such: just poll again. Fetch the request from forwardRequestHistory() and call poll() on it. If it finished while you were not watching, the history entry already carries the result.

document.addEventListener('visibilitychange', () => {
    if (document.visibilityState === 'hidden') {
        skapi.stopForwardRequestPolling({ queue: 'image-queue' });
    }
});

stopForwardRequestPolling(params?): number

isPollStopped(res): boolean

TIP

Requests sharing a queue are polled one at a time. A request that never settles therefore holds up every poll queued behind it, so stopping it also unblocks the rest. Stopping a request that has not started yet removes it from the queue entirely, freeing the slot immediately.

Cancelling a Request

To cancel a pending request before it is processed, call cancelForwardRequest():

const result = await skapi.cancelForwardRequest({
    url: 'https://api.openai.com/v1/images/generations',
    method: 'POST',
    id: 'stamp:entropy',  // the id from the forwardRequest response
    queue: 'image-jobs'   // required if the request was submitted with a queue name
});

console.log(result.removed); // true if successfully removed

Provide queue when the original request was submitted with a queue name. This removes the pending job from the client-side queue in addition to cancelling it on the server.

INFO

cancelForwardRequest() cancels the request. stopForwardRequestPolling() only stops watching it: the request carries on, and its result stays available. Use cancel when the work is no longer wanted, and stop-polling when only the traffic is.

cancelForwardRequest(params): Promise<{ removed: boolean; message: string }>

Checking Queue Size

To check how many requests are currently waiting in a named queue, use forwardRequestQueueCount():

const info = await skapi.forwardRequestQueueCount({ queue: 'image-jobs' });
console.log(info.in_queue); // number of requests waiting

forwardRequestQueueCount(params): Promise<{ queue_name: string; in_queue: number }>