Flavia - voice AI data analyst that joins your meetings

Overview

Skylight instances are designed for programmatic control, meaning you don’t need a physical mouse or keyboard to operate the desktop. To enable this, the Interactions API provides a variety of endpoints for performing actions on the virtual machine. The API can be used for manual Robotic Process Automation (RPA) or integrated into computer-use agents (for more information, see Agents).

Available Operations

screenshot - Take a screenshot
click - Perform a mouse click
move - Move the mouse cursor
drag - Perform a drag operation
scroll - Scroll the screen
type - Type text
keypress - Press keys simultaneously
get_file - Download a file from the VM
install - Install applications

screenshot

Take a screenshot of the virtual desktop.

# Take a screenshot 
screenshot = skylight.interact.screenshot(instance_id="<id>").image
# Returns a base64 encoded image that can be saved or processed

// Take a screenshot
const screenshot = await skylight.interact.screenshot("<id>").image;
// Returns a base64 encoded image that can be saved or processed

screenshot returns a JSON with an ‘image’ key. The value is a Base64-encoded JPEG image of the instance screenshot (1366x768).

click

Perform a click operation at specified coordinates or current position.

Input	Type	Required	Default	Description
`x`	`integer \| null`	No	`null`	X coordinate to click at
`y`	`integer \| null`	No	`null`	Y coordinate to click at
`button`	`enum`	No	`"left"`	Mouse button to use (`"left"`, `"right"`, `"middle"`)
`clicks`	`integer`	No	`1`	Number of clicks (range: 1-10)
`interval`	`number \| null`	No	`0`	Time between clicks in seconds (range: 0-1)

# Simple left click at coordinates
skylight.interact.click(instance_id="<id>", x=300, y=400)

# Right click
skylight.interact.click(instance_id="<id>", x=300, y=400, button="right")

# Middle click
skylight.interact.click(instance_id="<id>", button="middle")

# Double click
skylight.interact.click(instance_id="<id>", x=300, y=400, clicks=2)

# Click at current position (no x,y specified)
skylight.interact.click(instance_id="<id>")

# Click with 0.5 second interval between clicks
skylight.interact.click(instance_id="<id>", x=300, y=400, clicks=2, interval=0.5)

// Simple left click at coordinates
await skylight.interact.click("<id>", { x: 300, y: 400 });

// Right click
await skylight.interact.click("<id>", { x: 300, y: 400, button: "right" });

// Middle click at current position
await skylight.interact.click("<id>", { button: "middle" });

// Double click
await skylight.interact.click("<id>", { x: 300, y: 400, clicks: 2 });

// Click at current position
await skylight.interact.click("<id>");

// Click with 0.5 second interval between clicks
await skylight.interact.click("<id>", { x: 300, y: 400, clicks: 2, interval: 0.5 });

click returns a JSON with two keys: ‘status’ (e.g. “Success”) and ‘message’.

move

Move the mouse to the specified coordinates.

Input	Type	Required	Description
`x`	`integer`	Yes	X coordinate to move to
`y`	`integer`	Yes	Y coordinate to move to

# Move mouse to coordinates
skylight.interact.move(instance_id="<id>", x=100, y=200)

// Move mouse to coordinates
await skylight.interact.move("<id>", { x: 100, y: 200 });

move returns a JSON with two keys: ‘status’ (e.g. “Success”) and ‘message’.

drag

Perform a drag operation along a path of coordinates.

Input	Type	Required	Default	Description
`path`	`object[]`	Yes	-	List of points to drag through, each with `x` and `y` coordinates
`button`	`enum`	No	`"left"`	Mouse button to use (`"left"`, `"right"`, `"middle"`)
`step`	`integer \| null`	No	`null`	Step size for drag movements (range: 1-100)

# Drag from one point to another
skylight.interact.drag(instance_id="<id>", path=[
    {"x": 100, "y": 200},
    {"x": 300, "y": 400}
])

# Specify step size for more granular movement
skylight.interact.drag(instance_id="<id>", path=[
    {"x": 100, "y": 200},
    {"x": 300, "y": 400}
], step=10)

# Use a different mouse button
skylight.interact.drag(instance_id="<id>", path=[
    {"x": 100, "y": 200},
    {"x": 300, "y": 400}
], button="right")

// Drag from one point to another
await skylight.interact.drag("<id>", {
  path: [
    {x: 100, y: 200},
    {x: 300, y: 400}
  ]
});

// Specify step size for more granular movement
await skylight.interact.drag("<id>", {
  path: [
    {x: 100, y: 200},
    {x: 300, y: 400}
  ],
  step: 10
});

// Use a different mouse button
await skylight.interact.drag("<id>", {
  path: [
    {x: 100, y: 200},
    {x: 300, y: 400}
  ],
  button: "right"
});

drag returns a JSON with two keys: ‘status’ (e.g. “Success”) and ‘message’.

scroll

Scroll at the specified coordinates.

Input	Type	Required	Default	Description
`x`	`integer`	Yes	-	X coordinate to scroll at
`y`	`integer`	Yes	-	Y coordinate to scroll at
`scroll_x`	`integer`	No	`0`	Horizontal scroll amount
`scroll_y`	`integer`	No	`0`	Vertical scroll amount
`step`	`integer \| null`	No	-	Step size for scroll movements (range: 1-100)

# Scroll down
skylight.interact.scroll(instance_id="<id>", x=400, y=300, scroll_y=100)

# Scroll up
skylight.interact.scroll(instance_id="<id>", x=400, y=300, scroll_y=-100)

# Scroll right
skylight.interact.scroll(instance_id="<id>", x=400, y=300, scroll_x=100)

# Scroll left
skylight.interact.scroll(instance_id="<id>", x=400, y=300, scroll_x=-100)

# Scroll with steps
skylight.interact.scroll(instance_id="<id>", x=400, y=300, scroll_y=100, step=5)

// Scroll down
await skylight.interact.scroll("<id>", { x: 400, y: 300, scroll_y: 100 });

// Scroll up
await skylight.interact.scroll("<id>", { x: 400, y: 300, scroll_y: -100 });

// Scroll right
await skylight.interact.scroll("<id>", { x: 400, y: 300, scroll_x: 100 });

// Scroll left
await skylight.interact.scroll("<id>", { x: 400, y: 300, scroll_x: -100 });

// Scroll with steps
await skylight.interact.scroll("<id>", { x: 400, y: 300, scroll_y: 100, step: 5 });

scroll returns a JSON with two keys: ‘status’ (e.g. “Success”) and ‘message’.

type

Type the specified text with optional delay between keystrokes.

Input	Type	Required	Default	Description
`text`	`string`	Yes	-	Text to type
`interval`	`number \| null`	No	`0.05`	Time between keystrokes in seconds (range: 0.01-0.5)

# Type text with default interval between keystrokes
skylight.interact.type(instance_id="<id>", text="Hello World")

# Type with custom interval between keystrokes (in seconds)
skylight.interact.type(instance_id="<id>", text="Hello World", interval=0.1)

// Type text with default interval between keystrokes
await skylight.interact.type("<id>", { text: "Hello World" });

// Type with custom interval between keystrokes (in seconds)
await skylight.interact.type("<id>", { text: "Hello World", interval: 0.1 });

type returns a JSON with two keys: ‘status’ (e.g. “Success”) and ‘message’.

keypress

Press the specified keys simultaneously.

Input	Type	Required	Description
`keys`	`string[]`	Yes	List of keys to press simultaneously

# Press Enter key
skylight.interact.keypress(instance_id="<id>", keys=["Enter"])

# Press Ctrl+C
skylight.interact.keypress(instance_id="<id>", keys=["ctrl", "c"])

# Press Ctrl+Alt+Delete
skylight.interact.keypress(instance_id="<id>", keys=["ctrl", "alt", "delete"])

// Press Enter key
await skylight.interact.keypress("<id>", { keys: ["Enter"] });

// Press Ctrl+C
await skylight.interact.keypress("<id>", { keys: ["ctrl", "c"] });

// Press Ctrl+Alt+Delete
await skylight.interact.keypress("<id>", { keys: ["ctrl", "alt", "delete"] });

keypress returns a JSON with two keys: ‘status’ (e.g. “Success”) and ‘message’.

get_file

Get a URL to download the file.

# Get a file from the VM
download_url = skylight.interact.get_file(instance_id="<id>", request_body={"key": "/path/to/file.txt"})
# Returns a presigned URL to download the file

// Get a file from the VM
const downloadUrl = await skylight.interact.getFile("<id>", { "key": "/path/to/file.txt" });
// Returns a presigned URL to download the file

get_file returns a JSON with two keys: ‘download_url’ and ‘expires_in’.

install

Install additional software packages using the Chocolatey package manager. install takes in a list of Chocolatey package names to install. A complete list of available packages can be found at https://community.chocolatey.org/packages.

# Install packages
result = skylight.interact.install(instance_id="<id>", packages=["googlechrome", "vscode"])

// Install packages
const result = await skylight.interact.install("<id>", {
  packages: ["googlechrome", "vscode"]
});

The install endpoint returns a JSON with these properties:

Property	Description	Example
`status`	Status of the operation	`"success"`
`message`	Human-readable status message	`"Operation completed successfully"`
`state`	State of the instance	`"running"`
`command_id`	Command ID for tracking installation progress	`"cmd-123456"` or `null`

Practical Examples

Open an application

# Move mouse to Start button and click
skylight.interact.click(instance_id="<id>", x=20, y=980)

# Type the application name
skylight.interact.type(instance_id="<id>", text="Word")

# Press Enter to launch
skylight.interact.keypress(instance_id="<id>", keys=["Enter"])

// Move mouse to Start button and click
await skylight.interact.click("<id>", { x: 20, y: 980 });

// Type the application name
await skylight.interact.type("<id>", { text: "Word" });

// Press Enter to launch
await skylight.interact.keypress("<id>", { keys: ["Enter"] });

Drag and drop

# Drag a file to a new location
skylight.interact.drag(instance_id="<id>", path=[
    {"x": 200, "y": 300},  # Starting position (file location)
    {"x": 500, "y": 400}   # Ending position (destination)
])

// Drag a file to a new location
await skylight.interact.drag("<id>", {
  path: [
    {x: 200, y: 300},  // Starting position (file location)
    {x: 500, y: 400}   // Ending position (destination)
  ]
});

Copy and paste

# Click to position cursor
skylight.interact.click(instance_id="<id>", x=300, y=400)

# Type some text
skylight.interact.type(instance_id="<id>", text="This is some sample text.")

# Select all text (Ctrl+A)
skylight.interact.keypress(instance_id="<id>", keys=["ctrl", "a"])

# Copy text (Ctrl+C)
skylight.interact.keypress(instance_id="<id>", keys=["ctrl", "c"])

# Click somewhere else
skylight.interact.click(instance_id="<id>", x=300, y=500)

# Paste text (Ctrl+V)
skylight.interact.keypress(instance_id="<id>", keys=["ctrl", "v"])

// Click to position cursor
await skylight.interact.click("<id>", { x: 300, y: 400 });

// Type some text
await skylight.interact.type("<id>", { text: "This is some sample text." });

// Select all text (Ctrl+A)
await skylight.interact.keypress("<id>", { keys: ["ctrl", "a"] });

// Copy text (Ctrl+C)
await skylight.interact.keypress("<id>", { keys: ["ctrl", "c"] });

// Click somewhere else
await skylight.interact.click("<id>", { x: 300, y: 500 });

// Paste text (Ctrl+V)
await skylight.interact.keypress("<id>", { keys: ["ctrl", "v"] });

Downlad files

# Get a file from the VM
download_url = skylight.interact.get_file(instance_id="<id>", request_body={"key": "C:/Users/Administrator/Documents/example.txt"})

# Download the file using the URL
import requests
response = requests.get(download_url)
with open("example.txt", "wb") as f:
    f.write(response.content)

// Get a file from the VM
const downloadUrl = await skylight.interact.getFile("<id>", { 
  "key": "C:/Users/Administrator/Documents/example.txt" 
});

// Download the file using the URL
const response = await fetch(downloadUrl);
const fileContent = await response.arrayBuffer();
fs.writeFileSync('example.txt', Buffer.from(fileContent));

Get Started

Data

Sessions

General

Interactions

Overview

Available Operations

screenshot

click

move

drag

scroll

type

keypress

get_file

install

Practical Examples

Open an application

Drag and drop

Copy and paste

Downlad files

​Overview

​Available Operations

​screenshot

​click

​move

​drag

​scroll

​type

​keypress

​get_file

​install

​Practical Examples

​Open an application

​Drag and drop

​Copy and paste

​Downlad files

Overview

Available Operations

screenshot

click

move

drag

scroll

type

keypress

get_file

install

Practical Examples

Open an application

Drag and drop

Copy and paste

Downlad files