Cloudflare Llama 3.1 Free Preview：Chinese Performance Beyond Your Expectations

Cloudflare recently launched a service on its Worker AI platform that supports the Llama 3.1 model. Now, you can use this powerful model for free via the Workers AI Playground or API, with the model ID @cf/meta/llama-3.1-8b-instruct.

Meta's Llama model series is renowned for its exceptional performance in areas such as common sense, controllability, mathematical computation, tool usage, and multilingual translation. The latest Llama 3.1 model supports high precision (bfloat16) release, integrates function calls, and adds support for eight languages. The built-in multilingual support allows you to write prompts and receive responses in English, French, German, Hindi, Italian, Portuguese, Spanish, and Thai. This means your applications can easily reach a global audience with just one model. Meta claims that the Llama 3.1 model can already be compared to GPT-4:

Additionally, Llama 3.1 introduces native function calling, enabling large language models to generate structured JSON output and input it into different APIs. This out-of-the-box functionality requires no specialized fine-tuning, significantly simplifying the tool usage process.

Let's test the performance of the Llama 3.1 model using Cloudflare's interface and Playground. The results will definitely surprise you.

Interface Trial

After applying for an API Key from Cloudflare, you can access the Worker AI interface through code. Here is an example of making a request using the curl command.

curl -X POST \
  https://api.cloudflare.com/client/v4/accounts/b8670bc8c2657831fe6c8bb12ba17ccf/ai/run/@cf/meta/llama-3-8b-instruct \
  -H "Authorization: Bearer {{API_KEY}}" \
  -d '{"messages":[{"role":"system","content":"You are a friendly assistant that helps write stories"},{"role":"user","content":"Write a short story about a llama that goes on a journey to find an orange cloud "}]}'

Example of making a request using Python.

import requests


API_BASE_URL = "https://api.cloudflare.com/client/v4/accounts/b8670bc8c2657831fe6c8bb12ba17ccf/ai/run/"
headers = {"Authorization": "Bearer {{API_KEY}}"}


def run(model, inputs):
    input = { "messages": inputs }
    response = requests.post(f"{API_BASE_URL}{model}", headers=headers, json=input)
    return response.json()


inputs = [
    { "role": "system", "content": "You are a friendly assistant that helps write stories" },
    { "role": "user", "content": "Write a short story about a llama that goes on a journey to find an orange cloud "}
]
output = run("@cf/meta/llama-3-8b-instruct", inputs)
print(output)

After execution, you will get the corresponding response:

{'result': {'response': "Once upon a time, in a lush green valley surrounded by towering mountains, there lived a curious llama named Lola. Lola was known for her love of exploring new places and discovering hidden wonders. One day, while lounging in her favorite spot under a ancient oak tree, she gazed up at the sky and spotted something peculiar – an orange cloud!\n\nAccording to the wise old and blind llama, Rufus, such a cloud was said to possess magical properties, healing the sick and granting wisdom to those who reached it. Lola's curiosity was piqued, and she decided that she had to find this mystical cloud, no matter what the journey entailed.\n\nWith a determined look on her face, Lola set off early in the morning, packing a light backpack with some snacks, a water bottle, and a map. She traversed through the winding trails, crossing meadows and streams, meeting various creatures along the way. There were chatty squirrels, giggling butterflies, and even a wise old badger who offered her sage advice.\n\nAs the sun began to set, casting a warm orange glow on the landscape, Lola found herself at the foot of a steep mountain. The path ahead was treacherous, but she knew she had to push on. With a deep breath, she began the ascent, her llama legs strong and steady.\n\nThe night grew darker, and Lola used her keen senses to navigate the rocky terrain. She spotted fireflies lighting up the path, leading her to a serene lake nestled in the mountains. The water's surface reflected the starry sky, and Lola felt an overwhelming sense of peace wash over her.\n\nThe next morning, she continued her journey, and after a few hours of climbing, she reached the top of the mountain. And there, before her, lay the most breathtaking sight she had ever seen – the orange cloud! It glowed with an ethereal light, radiating warmth and tranquility.\n\nLola approached the cloud cautiously, sensing its power. As she reached out to touch it, the cloud enveloped her in a soothing mist. Suddenly, visions of the natural world flooded her mind – the interconnectedness of all living beings, the beauty of the earth, and the importance of preserving it for future generations.\n\nOverwhelmed with gratitude, Lola vowed to share her newfound wisdom with others. She spent the next few days learning the secrets of the orange cloud, and when she returned to her valley, she shared her story with Rufus and the other llamas.\n\nFrom that day on, Lola became"}, 'success': True, 'errors': [], 'messages': []}

Chinese Experience

Next, let's ask the model a few questions in Chinese through the Workers AI LLM Playground.

Why is TCP layered differently from OSI? Please answer in Chinese.

What data structure does Redis use for strings?

Does Java synchronized support up and down grading?

Please introduce Donald Trump.

From the answers to the above questions, the model's results are quite poor. It starts in Chinese but later mixes in other languages, which is quite confusing.

Conclusion

Through the above attempts, we found that although the Llama 3.1 model is quite outstanding, there is still much room for improvement in Chinese support. This also provides new opportunities for domestic large model manufacturers. We believe we will see a new wave of large model upgrades within a month or two, which is very exciting.