> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tryflavia.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Interactions

> Control your Windows instance remotely

## Overview

Skylight instances are designed for programmatic control, meaning you don't need a physical mouse or keyboard to operate the desktop. To enable this, the Interactions API provides a variety of endpoints for performing actions on the virtual machine. The API can be used for manual Robotic Process Automation (RPA) or integrated into computer-use agents (for more information, see [Agents](/agents/built-in)).

## Available Operations

* [screenshot](#screenshot) - Take a screenshot
* [click](#click) - Perform a mouse click
* [move](#move) - Move the mouse cursor
* [drag](#drag) - Perform a drag operation
* [scroll](#scroll) - Scroll the screen
* [type](#type) - Type text
* [keypress](#keypress) - Press keys simultaneously
* [get\_file](#get_file) - Download a file from the VM
* [install](#install) - Install applications

### screenshot

Take a screenshot of the virtual desktop.

<CodeGroup>
  ```python Python theme={null}
  # Take a screenshot 
  screenshot = skylight.interact.screenshot(instance_id="<id>").image
  # Returns a base64 encoded image that can be saved or processed
  ```

  ```typescript TypeScript theme={null}
  // Take a screenshot
  const screenshot = await skylight.interact.screenshot("<id>").image;
  // Returns a base64 encoded image that can be saved or processed
  ```
</CodeGroup>

`screenshot` returns a JSON with an 'image' key. The value is a Base64-encoded JPEG image of the instance screenshot (1366x768).

### click

Perform a click operation at specified coordinates or current position.

| Input      | Type              | Required | Default  | Description                                           |
| ---------- | ----------------- | -------- | -------- | ----------------------------------------------------- |
| `x`        | `integer \| null` | No       | `null`   | X coordinate to click at                              |
| `y`        | `integer \| null` | No       | `null`   | Y coordinate to click at                              |
| `button`   | `enum`            | No       | `"left"` | Mouse button to use (`"left"`, `"right"`, `"middle"`) |
| `clicks`   | `integer`         | No       | `1`      | Number of clicks (range: 1-10)                        |
| `interval` | `number \| null`  | No       | `0`      | Time between clicks in seconds (range: 0-1)           |

<CodeGroup>
  ```python Python theme={null}
  # Simple left click at coordinates
  skylight.interact.click(instance_id="<id>", x=300, y=400)

  # Right click
  skylight.interact.click(instance_id="<id>", x=300, y=400, button="right")

  # Middle click
  skylight.interact.click(instance_id="<id>", button="middle")

  # Double click
  skylight.interact.click(instance_id="<id>", x=300, y=400, clicks=2)

  # Click at current position (no x,y specified)
  skylight.interact.click(instance_id="<id>")

  # Click with 0.5 second interval between clicks
  skylight.interact.click(instance_id="<id>", x=300, y=400, clicks=2, interval=0.5)
  ```

  ```typescript TypeScript theme={null}
  // Simple left click at coordinates
  await skylight.interact.click("<id>", { x: 300, y: 400 });

  // Right click
  await skylight.interact.click("<id>", { x: 300, y: 400, button: "right" });

  // Middle click at current position
  await skylight.interact.click("<id>", { button: "middle" });

  // Double click
  await skylight.interact.click("<id>", { x: 300, y: 400, clicks: 2 });

  // Click at current position
  await skylight.interact.click("<id>");

  // Click with 0.5 second interval between clicks
  await skylight.interact.click("<id>", { x: 300, y: 400, clicks: 2, interval: 0.5 });
  ```
</CodeGroup>

`click` returns a JSON with two keys: 'status' (e.g. "Success") and 'message'.

### move

Move the mouse to the specified coordinates.

| Input | Type      | Required | Description             |
| ----- | --------- | -------- | ----------------------- |
| `x`   | `integer` | Yes      | X coordinate to move to |
| `y`   | `integer` | Yes      | Y coordinate to move to |

<CodeGroup>
  ```python Python theme={null}
  # Move mouse to coordinates
  skylight.interact.move(instance_id="<id>", x=100, y=200)
  ```

  ```typescript TypeScript theme={null}
  // Move mouse to coordinates
  await skylight.interact.move("<id>", { x: 100, y: 200 });
  ```
</CodeGroup>

`move` returns a JSON with two keys: 'status' (e.g. "Success") and 'message'.

### drag

Perform a drag operation along a path of coordinates.

| Input    | Type              | Required | Default  | Description                                                       |
| -------- | ----------------- | -------- | -------- | ----------------------------------------------------------------- |
| `path`   | `object[]`        | Yes      | -        | List of points to drag through, each with `x` and `y` coordinates |
| `button` | `enum`            | No       | `"left"` | Mouse button to use (`"left"`, `"right"`, `"middle"`)             |
| `step`   | `integer \| null` | No       | `null`   | Step size for drag movements (range: 1-100)                       |

<CodeGroup>
  ```python Python theme={null}
  # Drag from one point to another
  skylight.interact.drag(instance_id="<id>", path=[
      {"x": 100, "y": 200},
      {"x": 300, "y": 400}
  ])

  # Specify step size for more granular movement
  skylight.interact.drag(instance_id="<id>", path=[
      {"x": 100, "y": 200},
      {"x": 300, "y": 400}
  ], step=10)

  # Use a different mouse button
  skylight.interact.drag(instance_id="<id>", path=[
      {"x": 100, "y": 200},
      {"x": 300, "y": 400}
  ], button="right")
  ```

  ```typescript TypeScript theme={null}
  // Drag from one point to another
  await skylight.interact.drag("<id>", {
    path: [
      {x: 100, y: 200},
      {x: 300, y: 400}
    ]
  });

  // Specify step size for more granular movement
  await skylight.interact.drag("<id>", {
    path: [
      {x: 100, y: 200},
      {x: 300, y: 400}
    ],
    step: 10
  });

  // Use a different mouse button
  await skylight.interact.drag("<id>", {
    path: [
      {x: 100, y: 200},
      {x: 300, y: 400}
    ],
    button: "right"
  });
  ```
</CodeGroup>

`drag` returns a JSON with two keys: 'status' (e.g. "Success") and 'message'.

### scroll

Scroll at the specified coordinates.

| Input      | Type              | Required | Default | Description                                   |
| ---------- | ----------------- | -------- | ------- | --------------------------------------------- |
| `x`        | `integer`         | Yes      | -       | X coordinate to scroll at                     |
| `y`        | `integer`         | Yes      | -       | Y coordinate to scroll at                     |
| `scroll_x` | `integer`         | No       | `0`     | Horizontal scroll amount                      |
| `scroll_y` | `integer`         | No       | `0`     | Vertical scroll amount                        |
| `step`     | `integer \| null` | No       | -       | Step size for scroll movements (range: 1-100) |

<CodeGroup>
  ```python Python theme={null}
  # Scroll down
  skylight.interact.scroll(instance_id="<id>", x=400, y=300, scroll_y=100)

  # Scroll up
  skylight.interact.scroll(instance_id="<id>", x=400, y=300, scroll_y=-100)

  # Scroll right
  skylight.interact.scroll(instance_id="<id>", x=400, y=300, scroll_x=100)

  # Scroll left
  skylight.interact.scroll(instance_id="<id>", x=400, y=300, scroll_x=-100)

  # Scroll with steps
  skylight.interact.scroll(instance_id="<id>", x=400, y=300, scroll_y=100, step=5)
  ```

  ```typescript TypeScript theme={null}
  // Scroll down
  await skylight.interact.scroll("<id>", { x: 400, y: 300, scroll_y: 100 });

  // Scroll up
  await skylight.interact.scroll("<id>", { x: 400, y: 300, scroll_y: -100 });

  // Scroll right
  await skylight.interact.scroll("<id>", { x: 400, y: 300, scroll_x: 100 });

  // Scroll left
  await skylight.interact.scroll("<id>", { x: 400, y: 300, scroll_x: -100 });

  // Scroll with steps
  await skylight.interact.scroll("<id>", { x: 400, y: 300, scroll_y: 100, step: 5 });
  ```
</CodeGroup>

`scroll` returns a JSON with two keys: 'status' (e.g. "Success") and 'message'.

### type

Type the specified text with optional delay between keystrokes.

| Input      | Type             | Required | Default | Description                                          |
| ---------- | ---------------- | -------- | ------- | ---------------------------------------------------- |
| `text`     | `string`         | Yes      | -       | Text to type                                         |
| `interval` | `number \| null` | No       | `0.05`  | Time between keystrokes in seconds (range: 0.01-0.5) |

<CodeGroup>
  ```python Python theme={null}
  # Type text with default interval between keystrokes
  skylight.interact.type(instance_id="<id>", text="Hello World")

  # Type with custom interval between keystrokes (in seconds)
  skylight.interact.type(instance_id="<id>", text="Hello World", interval=0.1)
  ```

  ```typescript TypeScript theme={null}
  // Type text with default interval between keystrokes
  await skylight.interact.type("<id>", { text: "Hello World" });

  // Type with custom interval between keystrokes (in seconds)
  await skylight.interact.type("<id>", { text: "Hello World", interval: 0.1 });
  ```
</CodeGroup>

`type` returns a JSON with two keys: 'status' (e.g. "Success") and 'message'.

### keypress

Press the specified keys simultaneously.

| Input  | Type       | Required | Description                          |
| ------ | ---------- | -------- | ------------------------------------ |
| `keys` | `string[]` | Yes      | List of keys to press simultaneously |

<CodeGroup>
  ```python Python theme={null}
  # Press Enter key
  skylight.interact.keypress(instance_id="<id>", keys=["Enter"])

  # Press Ctrl+C
  skylight.interact.keypress(instance_id="<id>", keys=["ctrl", "c"])

  # Press Ctrl+Alt+Delete
  skylight.interact.keypress(instance_id="<id>", keys=["ctrl", "alt", "delete"])
  ```

  ```typescript TypeScript theme={null}
  // Press Enter key
  await skylight.interact.keypress("<id>", { keys: ["Enter"] });

  // Press Ctrl+C
  await skylight.interact.keypress("<id>", { keys: ["ctrl", "c"] });

  // Press Ctrl+Alt+Delete
  await skylight.interact.keypress("<id>", { keys: ["ctrl", "alt", "delete"] });
  ```
</CodeGroup>

`keypress` returns a JSON with two keys: 'status' (e.g. "Success") and 'message'.

### get\_file

Get a URL to download the file.

<CodeGroup>
  ```python Python theme={null}
  # Get a file from the VM
  download_url = skylight.interact.get_file(instance_id="<id>", request_body={"key": "/path/to/file.txt"})
  # Returns a presigned URL to download the file
  ```

  ```typescript TypeScript theme={null}
  // Get a file from the VM
  const downloadUrl = await skylight.interact.getFile("<id>", { "key": "/path/to/file.txt" });
  // Returns a presigned URL to download the file
  ```
</CodeGroup>

`get_file` returns a JSON with two keys: 'download\_url' and 'expires\_in'.

### install

Install additional software packages using the Chocolatey package manager. `install` takes in a list of Chocolatey package names to install. A complete list of available packages can be found at [https://community.chocolatey.org/packages](https://community.chocolatey.org/packages).

<CodeGroup>
  ```python Python theme={null}
  # Install packages
  result = skylight.interact.install(instance_id="<id>", packages=["googlechrome", "vscode"])
  ```

  ```typescript TypeScript theme={null}
  // Install packages
  const result = await skylight.interact.install("<id>", {
    packages: ["googlechrome", "vscode"]
  });
  ```
</CodeGroup>

The `install` endpoint returns a JSON with these properties:

| Property     | Description                                   | Example                              |
| ------------ | --------------------------------------------- | ------------------------------------ |
| `status`     | Status of the operation                       | `"success"`                          |
| `message`    | Human-readable status message                 | `"Operation completed successfully"` |
| `state`      | State of the instance                         | `"running"`                          |
| `command_id` | Command ID for tracking installation progress | `"cmd-123456"` or `null`             |

## Practical Examples

### Open an application

<CodeGroup>
  ```python Python theme={null}
  # Move mouse to Start button and click
  skylight.interact.click(instance_id="<id>", x=20, y=980)

  # Type the application name
  skylight.interact.type(instance_id="<id>", text="Word")

  # Press Enter to launch
  skylight.interact.keypress(instance_id="<id>", keys=["Enter"])
  ```

  ```typescript TypeScript theme={null}
  // Move mouse to Start button and click
  await skylight.interact.click("<id>", { x: 20, y: 980 });

  // Type the application name
  await skylight.interact.type("<id>", { text: "Word" });

  // Press Enter to launch
  await skylight.interact.keypress("<id>", { keys: ["Enter"] });
  ```
</CodeGroup>

### Drag and drop

<CodeGroup>
  ```python Python theme={null}
  # Drag a file to a new location
  skylight.interact.drag(instance_id="<id>", path=[
      {"x": 200, "y": 300},  # Starting position (file location)
      {"x": 500, "y": 400}   # Ending position (destination)
  ])
  ```

  ```typescript TypeScript theme={null}
  // Drag a file to a new location
  await skylight.interact.drag("<id>", {
    path: [
      {x: 200, y: 300},  // Starting position (file location)
      {x: 500, y: 400}   // Ending position (destination)
    ]
  });
  ```
</CodeGroup>

### Copy and paste

<CodeGroup>
  ```python Python theme={null}
  # Click to position cursor
  skylight.interact.click(instance_id="<id>", x=300, y=400)

  # Type some text
  skylight.interact.type(instance_id="<id>", text="This is some sample text.")

  # Select all text (Ctrl+A)
  skylight.interact.keypress(instance_id="<id>", keys=["ctrl", "a"])

  # Copy text (Ctrl+C)
  skylight.interact.keypress(instance_id="<id>", keys=["ctrl", "c"])

  # Click somewhere else
  skylight.interact.click(instance_id="<id>", x=300, y=500)

  # Paste text (Ctrl+V)
  skylight.interact.keypress(instance_id="<id>", keys=["ctrl", "v"])
  ```

  ```typescript TypeScript theme={null}
  // Click to position cursor
  await skylight.interact.click("<id>", { x: 300, y: 400 });

  // Type some text
  await skylight.interact.type("<id>", { text: "This is some sample text." });

  // Select all text (Ctrl+A)
  await skylight.interact.keypress("<id>", { keys: ["ctrl", "a"] });

  // Copy text (Ctrl+C)
  await skylight.interact.keypress("<id>", { keys: ["ctrl", "c"] });

  // Click somewhere else
  await skylight.interact.click("<id>", { x: 300, y: 500 });

  // Paste text (Ctrl+V)
  await skylight.interact.keypress("<id>", { keys: ["ctrl", "v"] });
  ```
</CodeGroup>

### Downlad files

<CodeGroup>
  ```python Python theme={null}
  # Get a file from the VM
  download_url = skylight.interact.get_file(instance_id="<id>", request_body={"key": "C:/Users/Administrator/Documents/example.txt"})

  # Download the file using the URL
  import requests
  response = requests.get(download_url)
  with open("example.txt", "wb") as f:
      f.write(response.content)
  ```

  ```typescript TypeScript theme={null}
  // Get a file from the VM
  const downloadUrl = await skylight.interact.getFile("<id>", { 
    "key": "C:/Users/Administrator/Documents/example.txt" 
  });

  // Download the file using the URL
  const response = await fetch(downloadUrl);
  const fileContent = await response.arrayBuffer();
  fs.writeFileSync('example.txt', Buffer.from(fileContent));
  ```
</CodeGroup>