[PR-1309] add sglang cancellation

[phatvo9]

What

Add sglang openai class + cancellation

Tests

• Manual tests via local runner
• Mock tests

[christineyu123]

Looks good! What I would suggest is if you can find a model that you are currently hardcoding the cancellation, and try replace model.py using this PR's model class and see if def generate()/openai_stream_support(), etc. are still working. Especially the token count and pricing are still good. To ensure nothing wrong if future models start to inherit that .

[phatvo9]

Looks good! What I would suggest is if you can find a model that you are currently hardcoding the cancellation, and try replace model.py using this PR's model class and see if def generate()/openai_stream_support(), etc. are still working. Especially the token count and pricing are still good. To ensure nothing wrong if future models start to inherit that .

@christineyu123 it worked on prod api

[github-actions]

Package	Line Rate	Health
clarifai	45%	❌
clarifai.cli	62%	➖
clarifai.cli.templates	67%	➖
clarifai.cli.templates.toolkits	100%	✔
clarifai.client	64%	➖
clarifai.client.auth	67%	➖
clarifai.constants	100%	✔
clarifai.datasets	100%	✔
clarifai.datasets.export	69%	➖
clarifai.datasets.upload	75%	➖
clarifai.datasets.upload.loaders	37%	❌
clarifai.models	100%	✔
clarifai.rag	0%	❌
clarifai.runners	51%	➖
clarifai.runners.models	62%	➖
clarifai.runners.pipeline_steps	45%	❌
clarifai.runners.pipelines	77%	➖
clarifai.runners.utils	61%	➖
clarifai.runners.utils.data_types	72%	➖
clarifai.schema	100%	✔
clarifai.urls	56%	➖
clarifai.utils	62%	➖
clarifai.utils.evaluation	16%	❌
clarifai.workflows	95%	✔
examples	64%	➖
Summary	61% (12658 / 20741)	➖

Minimum allowed line rate is 50%