/ 
Langertha-0.503
/ Langertha::Engine::SGLang

NAME

Langertha::Engine::SGLang - SGLang inference server

VERSION

version 0.503

SYNOPSIS

use Langertha::Engine::SGLang;

# 1. Simple chat
my $sglang = Langertha::Engine::SGLang->new(
    url   => 'http://localhost:30000/v1',
    model => 'Qwen/Qwen2.5-7B-Instruct',
);

print $sglang->simple_chat('Say something nice');

# 2. Streaming
$sglang->simple_chat_stream(sub {
    print shift->content;
}, 'Write a haiku about Perl');

# 3. MCP tool calling (requires a tool-call-parser-compatible model)
use Future::AsyncAwait;

my $sglang = Langertha::Engine::SGLang->new(
    url         => 'http://localhost:30000/v1',
    model       => 'Qwen/Qwen2.5-7B-Instruct',
    mcp_servers => [$mcp],
);

my $response = await $sglang->chat_with_tools_f('Add 7 and 15');

# 4. Multimodal input (vision-capable models served by SGLang)
use Langertha::Content::Image;

my $img  = Langertha::Content::Image->from_url('https://example.com/cat.jpg');
my $resp = await $sglang->simple_chat_f({
    role    => 'user',
    content => [ 'What is in this image?', $img ],
});

# 5. Prometheus /metrics scraping (Runtime::MetricsPoll)
my $records = await $sglang->poll_metrics_f('sglang:');

# 6. Embeddings (server launched with an embedding model)
my $embedder = Langertha::Engine::SGLang->new(
    url   => 'http://localhost:30000/v1',
    model => 'Alibaba-NLP/gte-Qwen2-1.5B-instruct',
);
my $vector  = $embedder->simple_embedding('Some text to embed');
my $vectors = $embedder->simple_embedding([ 'first', 'second' ]);

DESCRIPTION

Adapter for SGLang's OpenAI-compatible endpoint. SGLang is typically exposed as /v1/chat/completions with optional tool-calling support depending on model/backend setup.

Extends Langertha::Engine::OpenAIBase (which composes Langertha::Role::OpenAICompatible, Langertha::Role::OpenAPI, Langertha::Role::Models, Langertha::Role::Temperature, Langertha::Role::ResponseSize, Langertha::Role::SystemPrompt, Langertha::Role::ResponseFormat, Langertha::Role::Streaming, Langertha::Role::Chat, Langertha::Role::ReasoningEffort, and Langertha::Role::PromptCache); SGLang itself additionally composes Langertha::Role::Tools (MCP tool calling), Langertha::Role::Embedding (OpenAI-compatible /v1/embeddings) and Langertha::Role::Runtime::MetricsPoll (Prometheus /metrics scrape).

Supports chat, streaming, tool calling, embeddings, structured output, multimodal input, and Prometheus /metrics scraping. Transcription is not exposed on the OpenAI-compatible surface SGLang serves.

Only url is required. Use the full /v1 base URL. No API key is required for local setups.

See https://docs.sglang.ai/ for installation and configuration details.

EMBEDDINGS

Composes Langertha::Role::Embedding. SGLang serves /v1/embeddings when launched with an embedding model (decoder-style models also need --is-embedding). The request carries embedding_model if you set it, else model if you set it, else no model field at all: the server embeds with the model it serves. A string returns one vector, an ArrayRef of strings one vector per input, in input order.

CAPABILITIES

Advertised flags (derived from composed roles via Langertha::Role::Capabilities):

A forced tool_choice (required or a named tool) together with a response_format of json_schema, json_object or structural_tag croaks before the request is sent: the SGLang server rejects that combination with HTTP 400. tool_choice auto with a response_format is sent.

SEE ALSO

SUPPORT

Issues

Please report bugs and feature requests on GitHub at https://github.com/Getty/langertha/issues.

IRC

Join #langertha on irc.perl.org or message Getty directly.

CONTRIBUTING

Contributions are welcome! Please fork the repository and submit a pull request.

AUTHOR

Torsten Raudssus <getty@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2026 by Torsten Raudssus https://raudssus.de/.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.