> ## Documentation Index
> Fetch the complete documentation index at: https://openlayer.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Trace multimodal data
> Learn how to attach images, audio, and files to traces sent to Openlayer
Not every AI system runs on text alone. A claims agent reads a photo of a receipt, a
support bot listens to a voice note, a document pipeline parses a PDF.
**Attachments** let your traces carry that unstructured data. The media itself is uploaded
to your workspace storage, and the trace keeps a reference to it — so the platform can
show you the actual image, play the actual audio, and page through the actual document
next to the rest of the trace.
Attachments work the same way in the Python and TypeScript SDKs, and both write the same
[attachment format](#the-attachment-format).
## Enable attachment uploads
Attachment uploads are **disabled by default**. Until you turn them on,
attachments are recorded in the trace but never uploaded — and anything that
was not uploaded cannot be displayed.
```python Python theme={null}
from openlayer.lib import init
init(attachment_upload_enabled=True)
```
```typescript TypeScript theme={null}
import { configure } from "openlayer/lib/tracing/tracer";
configure({ attachmentUploadEnabled: true });
```
With uploads enabled, Openlayer uploads each attachment when the trace completes and
stores the resulting reference in the trace data.
If you attach media that already lives at an **external URL**, also enable URL uploads so
Openlayer fetches it and keeps its own copy:
```python Python theme={null}
init(attachment_upload_enabled=True, url_upload_enabled=True)
```
```typescript TypeScript theme={null}
configure({ attachmentUploadEnabled: true, urlUploadEnabled: true });
```
Without it, an external URL is recorded as-is. That keeps your trace pointing at a
resource Openlayer cannot read — if the URL later expires or sits behind
authentication, the media is gone.
Attachments require `openlayer>=0.17.0` in Python (`url_upload_enabled`
requires `openlayer>=0.17.9`) and `openlayer>=0.32.0` in TypeScript. Uploading
media sends that data to Openlayer, so enable uploads only when your privacy
requirements allow it.
## Attach media to a step
Call `log_attachment()` in Python or `logAttachment()` in TypeScript inside any traced
function to attach media to the step currently being recorded:
```python Python theme={null}
from openlayer.lib import init, trace
from openlayer.lib.tracing import log_attachment
init(attachment_upload_enabled=True)
@trace()
def triage_expense_claim(claim_id: str) -> str:
# Attach the artifacts the claim arrived with
log_attachment("receipt.png", metadata={"source": "mobile upload"})
log_attachment("voice_note.wav", metadata={"channel": "voicemail"})
log_attachment("policy.pdf")
return review(claim_id)
```
```typescript TypeScript theme={null}
import trace, { configure, logAttachment } from "openlayer/lib/tracing/tracer";
configure({ attachmentUploadEnabled: true });
const triageExpenseClaim = trace(async function triageExpenseClaim(
claimId: string,
): Promise {
// Attach the artifacts the claim arrived with
logAttachment("receipt.png", { metadata: { source: "mobile upload" } });
logAttachment("voice_note.wav", { metadata: { channel: "voicemail" } });
logAttachment("policy.pdf");
return review(claimId);
});
```
The Python helpers live in `openlayer.lib.tracing`, not in `openlayer.lib`. In
TypeScript, `logAttachment` is exported from `openlayer/lib/tracing/tracer`.
The helper accepts a file path, raw bytes, or an `Attachment` you built yourself. Python
also accepts a file-like object, and TypeScript accepts a `Buffer`, `Uint8Array`, or
`ArrayBuffer`:
```python Python theme={null}
# A path — the media type is detected from the extension
log_attachment("invoices/june.pdf")
# Raw bytes — pass a name and media type so the media renders correctly
log_attachment(chart_png, name="chart.png", media_type="image/png")
```
```typescript TypeScript theme={null}
// A path — the media type is detected from the extension
logAttachment("invoices/june.pdf");
// Raw bytes — pass a name and media type so the media renders correctly
logAttachment(chartPng, { name: "chart.png", mediaType: "image/png" });
```
Every attachment can carry metadata. Use it for whatever you need to filter or debug on
later: the upload channel, a page count, an audio duration, a document revision.
### Build attachments explicitly
For more control, construct an `Attachment` and pass it to the helper:
```python Python theme={null}
from openlayer.lib.tracing import Attachment, log_attachment
log_attachment(Attachment.from_file("receipt.png", name="Receipt"))
log_attachment(Attachment.from_url("https://example.com/receipt.png"))
log_attachment(
Attachment.from_bytes(png, name="chart.png", media_type="image/png")
)
log_attachment(
Attachment.from_base64(b64, name="clip.wav", media_type="audio/wav")
)
```
```typescript TypeScript theme={null}
import { Attachment } from "openlayer/lib/tracing/attachments";
import { logAttachment } from "openlayer/lib/tracing/tracer";
logAttachment(Attachment.fromFile("receipt.png", { name: "Receipt" }));
logAttachment(Attachment.fromUrl("https://example.com/receipt.png"));
logAttachment(
Attachment.fromBytes(png, { name: "chart.png", mediaType: "image/png" }),
);
logAttachment(
Attachment.fromBase64(b64, { name: "clip.wav", mediaType: "audio/wav" }),
);
```
| Python | TypeScript | Use it for | Notes |
| - | - | - | - |
| `from_file()` | `fromFile()` | Media on local disk | Media type is guessed from the extension; size and checksum are computed; the path is recorded |
| `from_url()` | `fromUrl()` | Media hosted somewhere else | Only fetched into Openlayer when URL uploads are enabled |
| `from_bytes()` | `fromBytes()` | Media generated in memory | Name and media type are required; in TypeScript the bytes are copied, so you can reuse a buffer |
| `from_base64()` | `fromBase64()` | Media already base64-encoded | Useful for provider payloads that are encoded in transit |
`from_file()` / `fromFile()` records the file's absolute local path in the
trace (`filePath`). If you don't want local paths in your traces, read the
file and use `from_bytes()` / `fromBytes()` instead.
## Media in inputs and outputs
Attachments don't have to hang off a step. You can pass one to a traced function, or return
one, and it is uploaded and displayed like any other attachment:
```python Python theme={null}
from openlayer.lib import trace
from openlayer.lib.tracing import Attachment
@trace()
def transcribe(recording: Attachment) -> str:
return speech_to_text(recording.get_bytes())
transcribe(
Attachment.from_bytes(audio, name="caller.wav", media_type="audio/wav")
)
```
```typescript TypeScript theme={null}
import { Attachment } from "openlayer/lib/tracing/attachments";
import trace from "openlayer/lib/tracing/tracer";
const transcribe = trace(async function transcribe(
recording: Attachment,
): Promise {
return speechToText(recording.getBytes());
});
await transcribe(
Attachment.fromBytes(audio, { name: "caller.wav", mediaType: "audio/wav" }),
);
```
The platform renders an input or output as media when the value **is** an attachment, when
it is an object whose values are attachments (for example `{ "audio": attachment }`), or
when it is an array of [content items](#multimodal-messages). A content item nested inside
another object is not rendered as media — put the bare attachment there instead.
## Multimodal messages
To model a **message that is itself part text and part media** — the shape a vision or
audio model actually receives — use content items:
```python Python theme={null}
from openlayer.lib import init, trace
from openlayer.lib.tracing import Attachment, create_step
from openlayer.lib.tracing.content import (
AudioContent,
FileContent,
ImageContent,
TextContent,
)
from openlayer.lib.tracing.enums import StepType
init(attachment_upload_enabled=True)
@trace()
def answer_claim_question(question: str) -> str:
receipt = Attachment.from_file("receipt.png")
voice_note = Attachment.from_file("voice_note.wav")
policy = Attachment.from_file("policy.pdf")
with create_step(
name="Claim assistant", step_type=StepType.CHAT_COMPLETION
) as step:
answer = call_your_model(question, receipt, voice_note, policy)
step.log(
inputs={
"prompt": [
{
"role": "user",
"content": [
TextContent(text=question),
ImageContent(attachment=receipt),
AudioContent(attachment=voice_note),
FileContent(attachment=policy),
],
}
]
},
output=answer,
)
return answer
```
```typescript TypeScript theme={null}
import { Attachment } from "openlayer/lib/tracing/attachments";
import {
AudioContent,
FileContent,
ImageContent,
TextContent,
type ContentItem,
} from "openlayer/lib/tracing/content";
import trace, { configure } from "openlayer/lib/tracing/tracer";
configure({ attachmentUploadEnabled: true });
const answerClaimQuestion = trace(async function answerClaimQuestion(
message: ContentItem[],
): Promise {
return callYourModel(message);
});
await answerClaimQuestion([
new TextContent("Is this claim within policy?"),
new ImageContent(Attachment.fromFile("receipt.png")),
new AudioContent(Attachment.fromFile("voice_note.wav")),
new FileContent(Attachment.fromFile("policy.pdf")),
]);
```
There are four content items — `TextContent`, `ImageContent`, `AudioContent`, and
`FileContent` — and a single message can mix as many as you need. Openlayer renders the
message in order, so the text and the media it refers to stay together.
## Automatic capture
Some integrations attach media for you once uploads are enabled.
### OpenAI (Python)
If you send multimodal messages through a traced OpenAI client in Python, Openlayer reads
the content array and converts it to attachments, on both the Chat Completions and
Responses APIs:
```python Python theme={null}
import openai
from openlayer.lib import init
init(attachment_upload_enabled=True)
client = openai.OpenAI() # auto-traced
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "What is the total?"},
{"type": "image_url", "image_url": {"url": data_url}},
],
}
],
)
```
Images (`image_url`, `input_image`), audio (`input_audio`), and files (`file`) are all
recognized, whether they arrive as a URL, a base64 data URL, or an uploaded file ID.
Generated images in Responses API output are captured the same way.
### Azure AI Speech (Python and TypeScript)
The [Azure AI Speech integration](/docs/integrations/azure-speech) attaches synthesized audio to
each synthesis step, and the audio you pass for recognition to each recognition step, in
both SDKs.
## How attachments appear in Openlayer
Uploaded attachments are rendered wherever the trace is shown — in the row, in the row
detail view, and on the individual step:
| Media | What you get |
| - | - |
| Images | Rendered inline, click to open full size |
| Audio | A player you can scrub; only one clip plays at a time |
| PDFs | A page-by-page viewer, click to open the full document |
| Everything else | A card with the file name and size, and a download button |
Every attachment can be downloaded, whatever its type. Downloads keep the attachment's
name and extension as part of the filename.
## How uploads work
* **When:** attachments are uploaded when the trace completes, before the trace is
published, so the published trace already carries each attachment's `storageUri`.
* **Deduplication:** attachments are deduplicated by MD5 checksum, so attaching the same
image to three steps costs one upload.
* **Failures:** a failed upload is logged and the trace is still published — you get the
trace without that media rather than an exception.
* **Where:** everything found on a step's attachments and in its inputs and outputs is
uploaded, including nested steps.
## The attachment format
Attachments are plain JSON inside your trace, so any client that can publish a row can
publish an attachment. This is what both SDKs write:
```json theme={null}
{
"id": "1f1d3e0c-6a19-4d1e-9f3c-2b7c0d84a501",
"name": "receipt.png",
"mediaType": "image/png",
"storageUri": "s3://openlayer-assets/.../attachments/ebb63407.png",
"sizeBytes": 35016,
"checksumMd5": "ebb634079cb8555ceef04266b5a976ee",
"metadata": { "source": "mobile upload" }
}
```
`storageUri` is what makes an attachment displayable — it is the reference to the copy in
your workspace storage. To obtain one for media you upload yourself, request a presigned
URL, upload the bytes to it, and keep the `storageUri` that comes back:
```bash theme={null}
curl -X POST \
"https://api.openlayer.com/v1/storage/presigned-url?objectName=receipt.png" \
-H "Authorization: Bearer $OPENLAYER_API_KEY"
```
The response contains the `url` to upload to and the `storageUri` to record. How you upload
depends on your deployment's storage:
* **Openlayer Cloud (S3):** the response also contains form `fields`. Send a multipart
`POST` to `url` with every field first and the file last, in a part named `file`.
* **Self-hosted on GCS, Azure Blob Storage, or Oracle Object Storage:** there are no
`fields`. Send a `PUT` of the raw bytes to `url`, with a `Content-Type` header. Azure also
requires `x-ms-blob-type: BlockBlob`.
* **Self-hosted with local storage:** send a multipart `POST` to `url` with the file in a
part named `file`.
You can then put the attachment in a column when you
[stream the row](/docs/api-reference/rest/monitoring/stream-data):
```json theme={null}
{
"config": {
"inputVariableNames": ["question", "receipt"],
"outputColumnName": "output",
"inferenceIdColumnName": "inferenceId"
},
"rows": [
{
"inferenceId": "claim-4417",
"question": "Is this meal claim within policy?",
"receipt": {
"storageUri": "...",
"mediaType": "image/png",
"name": "receipt.png"
},
"output": "The claim is within policy."
}
]
}
```
The column holding the attachment must be listed in `inputVariableNames`. A
row containing an attachment in an undeclared column is still accepted — the
request returns success — but the platform has no column to attach it to, so
it is never displayed.
A column can also hold a full multimodal message, mixing media with text:
```json theme={null}
[
{ "type": "text", "text": "Is this meal claim within policy?" },
{
"type": "image",
"attachment": {
"storageUri": "...",
"mediaType": "image/png",
"name": "receipt.png"
}
}
]
```
## Complete example
An expense claim that arrives as a photo, a voice note, and a policy document — attached
to the trace, then answered by a model:
```python Python theme={null}
import base64
import openai
from openlayer.lib import init, trace
from openlayer.lib.tracing import log_attachment
# Attachment uploads are off by default
init(attachment_upload_enabled=True)
client = openai.OpenAI() # auto-traced
def as_data_url(path: str, media_type: str) -> str:
with open(path, "rb") as file:
encoded = base64.b64encode(file.read()).decode("utf-8")
return f"data:{media_type};base64,{encoded}"
@trace()
def read_receipt(receipt_path: str) -> str:
"""The image sent to OpenAI becomes an attachment automatically."""
image_url = as_data_url(receipt_path, "image/png")
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Read this receipt."},
{"type": "image_url", "image_url": {"url": image_url}},
],
}
],
)
return response.choices[0].message.content
@trace()
def triage_expense_claim(claim_id: str) -> str:
# Attach the artifacts the claim arrived with
log_attachment("receipt.png", metadata={"source": "mobile upload"})
log_attachment("voice_note.wav", metadata={"channel": "voicemail"})
log_attachment("policy.pdf", metadata={"revision": 4})
extracted = read_receipt("receipt.png")
return f"Claim {claim_id} triaged. {extracted}"
if __name__ == "__main__":
print(triage_expense_claim("CLM-4417"))
```
```typescript TypeScript theme={null}
import * as fs from "fs";
import { Attachment } from "openlayer/lib/tracing/attachments";
import { AudioContent, TextContent } from "openlayer/lib/tracing/content";
import trace, { configure, logAttachment } from "openlayer/lib/tracing/tracer";
// Attachment uploads are off by default
configure({ attachmentUploadEnabled: true });
const triageExpenseClaim = trace(async function triageExpenseClaim(
claimId: string,
receipt: Attachment,
): Promise> {
// Attach the artifacts the claim arrived with
logAttachment("voice_note.wav", { metadata: { channel: "voicemail" } });
logAttachment("policy.pdf", { metadata: { revision: 4 } });
const summary = await summarizeClaim(claimId, receipt);
const spokenSummary = await textToSpeech(summary); // audio bytes
// Return a multimodal message: the text plus its spoken version
return [
new TextContent(summary),
new AudioContent(
Attachment.fromBytes(spokenSummary, {
name: "summary.mp3",
mediaType: "audio/mpeg",
}),
),
];
});
await triageExpenseClaim(
"CLM-4417",
Attachment.fromBytes(fs.readFileSync("receipt.png"), {
name: "receipt.png",
mediaType: "image/png",
}),
);
```
Run it with your credentials set, and the trace arrives with the attachments you logged,
the media passed in as inputs, and the media in the output. In Python, the receipt image
OpenAI received is attached too.
## Troubleshooting
Check that uploads are enabled first (`attachment_upload_enabled=True` in
Python, `attachmentUploadEnabled: true` in TypeScript) — they are off by
default, and without them nothing is uploaded. If the attachment came from
`from_url()` / `fromUrl()`, you also need URL uploads enabled, otherwise
Openlayer never fetches a copy it can display.
Check where it sits. An input or output renders as media when it is an
attachment, an object whose values are attachments, or an array of content
items. A single content item nested inside another object is not rendered —
use the bare attachment there instead.
Attachments with no data and no reference are dropped rather than published
— most often because a file path does not exist. The SDK logs a warning when
this happens. In Python, enable debug logging while you debug:
```python theme={null}
import logging
logging.getLogger("openlayer").setLevel(logging.DEBUG)
```
Uploads happen when the trace completes, after your traced function returns,
so your code isn't held up while the media uploads. A failed upload is logged
and the trace is still published — you get the trace without the media
rather than an exception.
Looking to add non-media context to your traces instead? See [Add metadata to
traces](/docs/monitoring/metadata).
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.