Add prompt converter

[sainivedh]

Support OpenAI to respective llm prompt converter tool

• alpaca
• falcon
• mosaic
• llama2_chat
• claude
• mistral
• ollama
• Huggingface chat template
• Gemini text Image

Eg:

from clarifai_datautils.text.prompt_factory import hf_chat_template


messages = [
        {"role": "system", "content": "You are a helpful assistant that answers in JSON."},
        {"role": "user", "content": "Create a user named Alice, who lives in 42, Wonderland Avenue."}
    ]

print(hf_chat_template("Qwen/QwQ-32B", messages, hf_token="xxx"))

<|im_start|>system
You are a helpful assistant that answers in JSON.<|im_end|>
<|im_start|>user
Create a user named Alice, who lives in 42, Wonderland Avenue.<|im_end|>

[Copilot]

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (1)

tests/test_prompt_factory.py:6

• [nitpick] Consider renaming 'test_codellama_prompt_format' to 'test_llama_2_chat_pt_format' to clearly reflect the underlying function being tested.

def test_codellama_prompt_format():

[luv-bansal]

I don't think we need most of these old model prompt formats, since we're not gonna use the older models. Keeping them just adds unnecessary clutter to the file

[luv-bansal]

I see it just returns single list of prompts and images, do we need this?

I see there SDK only supports Dict of list of message, so do you think should be support util function of converting openai messages to gemini messages?

[luv-bansal]

Same with this. Anthropic latest SDK only supports messages for inference, so should be have util function of converting openai message to anthropic message?

[luv-bansal]

This is good, but I think we could also add another function to convert openai message to hf transformer message

[ackizilkale]

@sainivedh @luv-bansal what are we going to do with this PR?

[sainivedh]

@luv-bansal and I discussed that this should in SDK runner/utils and keep this in hold for now