Running Models on VaikerAI Using Python

Learn how to integrate and run machine learning models on VaikerAI directly from your Python code, whether it's in an app, notebook, or script.

1. Install the Python Library

To interact with VaikerAI, you'll need to install our open-source Python client. Use pip to install it:

pip install vaikerai

2. Authenticate

Before running models, you'll need to authenticate with VaikerAI. Generate an API token by visiting your account's API tokens page. Copy the token and set it as an environment variable in your shell:

export REPLICATE_API_TOKEN=8UY56_....

3. Run a Model

You can run any public model on VaikerAI with just a few lines of Python. Here’s an example that uses the stability-ai/sdxl model to generate an image based on a text prompt:

import vaikerai

output = vaikerai.run(
  "stability-ai/sdxl:39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
  input={"prompt": "an iguana on the beach, pointillism"}
)
print(output)

The output will be a URL to the generated image:

['https://files.vaikerai.com/pbxt/VJyWBjIYgqqCCBEhpkCqdevTgAJbl4fg62aO4o9A0x85CgNSA/out-0.png']

4. Using Local Files as Inputs

Some models require files as input. You can use local files or provide a file's HTTPS URL.

Example: Using a Local File

Here’s an example using a local image file with the LLaVA vision model, which processes an image and a text prompt to generate a response:

import vaikerai

image = open("my_fridge.jpg", "rb")
output = vaikerai.run(
    "yorickvp/llava-13b:a0fdc44e4f2e1f20f2bb4e27846899953ac8e66c5886c5878fa1d6b73ce009e5",
    input={
        "image": image,
        "prompt": "Here's what's in my fridge. What can I make for dinner tonight?"
    }
)
print(output)

The model's response might be:

You have a well-stocked refrigerator filled with various fruits, vegetables, and ...

5. Using URLs as Inputs

If your file is already hosted online or is large, using its URL as input is more efficient.

Example: Using a URL

Here’s an example using a public HTTPS URL of an image:

image = "https://example.com/my_fridge.jpg"
output = vaikerai.run(
    "yorickvp/llava-13b:a0fdc44e4f2e1f20f2bb4e27846899953ac8e66c5886c5878fa1d6b73ce009e5",
    input={
        "image": image,
        "prompt": "Here's what's in my fridge. What can I make for dinner tonight?"
    }
)
print(output)

The model will return a text response similar to:

You have a well-stocked refrigerator filled with various fruits, vegetables, and ...

6. Handling Output

Some models stream their output as they process the input. These models return an iterator, allowing you to process each chunk of output as it becomes available.

Example: Streaming Output

Here’s how to handle streamed output from the mistralai/mixtral-8x7b-instruct-v0.1 model:

iterator = vaikerai.run(
  "mistralai/mixtral-8x7b-instruct-v0.1",
  input={"prompt": "Who was Dolly the sheep?"},
)
for text in iterator:
    print(text)
    

As the model runs, you might see output like this:

🐑
Dolly the sheep was the first mammal to be successfully cloned from an adult cell...

Next Steps

For more detailed information and advanced usage, refer to the full Python client documentation available on GitHub.

---